What happened
> An arXiv CS.LG preprint (arXiv:2609.25624v1) proposes fixing the floating-point reduction order of linear layers as a pure function of problem shape, so every GPU runs the same operation sequence. > The authors say this reduces cross-architecture reproducibility to correct IEEE-754 arithmetic rather than keeping rounding differences below a tie-flip threshold. > The paper attributes nondeterministic greedy decoding to floating-point non-associativity plus hardware-dependent kernel selection, and says prior solutions have imperfect cross-architecture reproducibility and a significant performance penalty. [1]
What the authors claim
> The authors report bitwise-identical linear-layer outputs across NVIDIA Ampere, Ada, and Hopper GPUs. > They claim 1.17x to 3.1x faster end-to-end performance than the state-of-the-art solution, and half the weight-memory traffic. > Method: fixed-configuration fused-upcast GEMM kernels load 16-bit weights, upcast to FP32 in registers, and accumulate under IEEE-754 in a device-independent reduction order. > These are the paper's own claims; no independent benchmark or third-party reproduction is in the supplied evidence. [1]
Why it matters
> The paper targets reproducibility of the linear layers specifically, so it does not by itself establish end-to-end bitwise-identical full-model output. > The claim is cross-architecture determinism for three NVIDIA generations; nothing in the supplied evidence covers other vendors' GPUs. > Practical relevance is reproducibility testing and debugging, where identical runs across hardware are otherwise not guaranteed. [1]
Sources
- Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures
ArXiv CS.LG (Machine Learning) · Reporting ·