Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch — mentioned by ux266478

Mon, 12 Jan 2026 22:27:12 GMT

> Strange then that the vast majority of LLMs that people use produce non-deterministic output. > I ran the "non-shitty non-corpo completely determenistic model" through ollama... And immediately got two different answers for the same input. With deterministic hardware in the same configuration, using the same binaries, providing the same seed, the same input sequence to the same model weights will produce bit-identical outputs. Where you can get into trouble is if you aren't actually specifying your seed, or with non-deterministic hardware in varying configurations, or if your OS mixes…

Paper and discussion history. A mention is not an endorsement.

Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch · HackerPapers discussions

Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch — mentioned by ux266478