Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
Abstract
NVFP4 pre-training with MS-EDEN, an unbiased gradient estimator that reduces quantization error, and GPU kernels for NVIDIA Blackwell.
Type
Publication
ICML 2026
Andrei Panferov
PhD Candidate in Computer Science
I work on efficient large language models, with a focus on low-precision training, quantization, and scaling laws.