Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
NVFP4 pre-training with MS-EDEN, an unbiased gradient estimator that reduces quantization error, and GPU kernels for NVIDIA Blackwell.
Biography
I’m a PhD candidate at the Institute of Science and Technology Austria (ISTA) and a visiting ELLIS PhD at EPFL.
I work on making large language models more efficient to train and run. My research focuses on low-precision training, quantization, and scaling laws: how to use less computation and memory while preserving model quality. I connect these questions with practical GPU implementations.
My industry experience includes an AI PhD internship at NVIDIA, working on software–hardware co-design for LLM pre-training infrastructure.
Find my publications on Google Scholar, my professional background on LinkedIn, and code on GitHub.
Interests
- Low-Precision LLM Training
- Quantization and Model Compression
- Scaling Laws and Efficient Deep Learning
Education
PhD in Computer Science, 2024-now
Institute of Science and Technology Austria (ISTA)
Visiting ELLIS PhD in Computer Science, 2024-now
École Polytechnique Fédérale de Lausanne (EPFL)
Graduate Program in Machine Learning, 2021-2023
Yandex School of Data Analysis (YSDA)
BSc in Applied Mathematics and Physics, 2020-2024
Moscow Institute of Physics and Technology (MIPT)
Featured Research
Low-precision training for large language models. Browse selected publications or visit Google Scholar.
Experience
AI PhD Intern · NVIDIA
Software–hardware co-design for LLM pre-training infrastructure.
PhD Candidate · ISTA
2024–present · Institute of Science and Technology Austria
Research on efficient large language models, low-precision training, and quantization.
Visiting ELLIS PhD · EPFL
2024–present · École Polytechnique Fédérale de Lausanne
Honors
International Physics Olympiad
Gold Medal