Thinking Machines Lab has published a detailed recipe for training Inkling 2 Small with 4-bit arithmetic in parts of its model pipeline. The headline is an 8% pretraining speedup and a 20% reinforcement-learning speedup in the company's largest jobs, compared with its full-MXFP8 baseline. Those are the lab's measurements on NVIDIA GB300 systems, not a general guarantee that switching a model to four bits will produce the same gains.
The most important qualification is what the lab means by “end-to-end.” Its setup uses native NVFP4 for expert-layer forward passes from pretraining through the final reinforcement-learning rollout. The backward pass uses MXFP8, some later layers stay in MXFP8, and expert-to-expert communications use a separate 8-bit format. The achievement is continuity of a 4-bit forward path through the training lifecycle, not a claim that every value, gradient and operation is 4-bit.
Why use 4-bit values at all?
A floating-point format with fewer bits stores less information about each number. That can reduce memory traffic and allow specialized GPU matrix units to perform more operations per second. But training repeatedly multiplies, rounds and updates values; small errors can accumulate or bias an update. The challenge is finding where narrower numbers save time without changing learning so much that the model pays back the speedup in poorer results.
NVFP4 is NVIDIA's 4-bit floating-point format for Blackwell GPUs. Its values are scaled in small blocks: each group of 16 four-bit numbers shares an eight-bit scale factor. That makes its effective storage 4.5 bits per value before other overhead, rather than exactly four. Thinking Machines lists GB300 peak matrix throughput at 15 PFLOPS for NVFP4, 5 for MXFP8 and 2.5 for BF16. Those hardware peaks explain the attraction, but they are not an end-to-end training benchmark.
A precision choice for each stage
The team says it applied NVFP4 to forward computation in the mixture-of-experts layers, where tokens are routed to selected experts. It did not choose the same format for every stage. Backpropagation is sensitive to systematic rounding bias, so the backward multiplications use MXFP8. Saved weights and activations are converted again with stochastic rounding: instead of always choosing the nearest representable value, the process sometimes rounds up and sometimes down so the expected value is preserved.
That distinction matters because a forward tensor can be grouped differently when transposed for a backward calculation. Reusing its low-bit values under a new scale can introduce a consistent directional error. Thinking Machines says it de-quantizes saved values, transposes them at higher precision, then stochastically re-quantizes them to MXFP8. In its tests, that mix offered a better accuracy-speed balance than forcing NVFP4 through the backward pass.
The recipe also keeps the final 16.6% of layers in MXFP8. The team says the share matches its model's layer-count scaling, and notes NVIDIA's cited recipe keeps a similar portion in higher precision. For activations it uses a Four Over Six technique to choose between block scales, aiming to lower quantization error. These decisions are evidence that successful low-precision training depends on exceptions and calibration, not just the bit width printed on a chip specification.
Why communication became the next bottleneck
Once matrix multiplication became faster, sending data between GPUs took a larger share of the work. A mixture-of-experts layer dispatches tokens to GPUs holding the selected experts, then gathers their outputs. Thinking Machines says it sends dispatch values in NVFP4, but keeps the combined outputs at higher precision by default because reducing them to MXFP8 hurt performance in its tests.
The lab's alternative transforms groups of values with a random Hadamard rotation, which tends to spread large outliers more evenly, then sends them in block-scaled S2F6, an 8-bit fixed-point format. The receiver reconstructs and combines the values, then reverses the transform. Thinking Machines reports that this cut combine traffic from 16 bits to 8.25 bits per value without a measurable accuracy loss in its training runs. The claim is specific to its model, kernels and GB300 setup.
This is why the headline speedup is a systems result. The lab also built custom kernels to combine matrix operations, quantization, rounding and data movement. Faster arithmetic alone would have exposed the communication bottleneck; the rest of the stack had to change to keep the GPUs busy.
What did the speedup cost?
Thinking Machines reports that its NVFP4 pretraining recipe gives up about 2% in compute efficiency compared with MXFP8, meaning it reaches a given loss as if it had used around 2% fewer FLOPs, while still running faster in wall-clock time. It reports a larger net benefit in reinforcement learning and no discernible reward difference there. In its largest jobs, it measured 8% faster pretraining and 20% faster RL than full MXFP8.
The post also says Inkling 2 Small was pretrained in 68 hours on GB300 NVL72 racks, compared with several weeks for an earlier model of similar size. That comparison includes many advances across the training stack. The lab explicitly describes its precision recipe as one contribution among several, so the full elapsed-time difference should not be credited to 4-bit arithmetic alone.
Earlier papers have already explored fully quantized or NVFP4 language-model training. Thinking Machines frames its claim more narrowly: Inkling 2 Small is the largest model it is aware of trained with a native 4-bit forward pass throughout the full training lifecycle. Its post is a company engineering report, not an independently reproduced benchmark, and does not publish enough details to compare every result across labs.
For model builders, the practical takeaway is to benchmark the whole training run, including communication and backward passes, and track both quality and wall-clock time. Lower precision can make a GPU's fastest math units useful, but the gains depend on where it is safe to quantize, how error is controlled, and whether the rest of the system can keep up.
Thinking Machines Lab: training Inkling 2 Small with a native 4-bit forward pass
NVIDIA: Blackwell support for low-precision model arithmetic
2025 paper on fully quantized LLM training
Research on unbiased gradient estimation for NVFP4 pretraining
What is NVIDIA GR00T 2, and how does it transfer robot skills?
VIBPS is the social network for software, where makers can publish what they build and share their work with people who want to follow it.