DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence · HackerPapers discussions

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence — mentioned by gpugreg

Wed, 09 Sep 2026 16:40:30 GMT

> those things are not deterministic Determinism was an explicit goal of DeepSeek-V4. From their paper: https://arxiv.org/html/2606.19348v1#S3.SS3 > we implement end-to-end, bitwise batch-invariant, and deterministic kernels with minimal performance overhead Of course, providers may not implement deterministic inference for various reasons, but it is possible.

Paper and discussion history. A mention is not an endorsement.

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence — mentioned by throwdbaaway

Tue, 07 Jul 2026 07:20:50 GMT

Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1.

Paper and discussion history. A mention is not an endorsement.

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence — mentioned by doctorpangloss

Mon, 22 Jun 2026 00:23:26 GMT

look... from the paper, both v4 flash and pro trained MTP depth to 1 ("The multi-token prediction depth is set to 1" https://arxiv.org/pdf/2606.19348v1#subsection.2.1 pg 25). it doesn't predict the next 2 tokens. the verifier is the whole model. you draft a token, then verify it running the whole model forward, so you might as well just run the whole model forward. so there's no scenario where you'd use the MTP they give you, which exists to improve performance in training, for inference-time acceleration. you can do something else. alternatively, by all means, see for yourself. you can…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence — mentioned by wolttam

Sun, 21 Jun 2026 19:07:59 GMT

> MTP in Inference. Our MTP strategy mainly aims to improve the performance of the main model, so during inference, we can directly discard the MTP modules and the main model can function independently and normally. *Additionally, we can also repurpose these MTP modules for speculative decoding to further improve the generation latency.*[1] (emphasis mine) > Instead of predicting just the next single token, DeepSeek-V3 predicts the next 2 tokens through the MTP technique. Combined with the framework of speculative decoding (Leviathan et al., 2023; Xia et al., 2023), it can significantly…

Paper and discussion history. A mention is not an endorsement.