HackerPapers

arXiv:1805.02867cs.PF

Online normalizer calculation for softmax

Maxim Milakov, Natalia Gimelshein

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

The Softmax function is ubiquitous in machine learning, multiple previous works suggested faster alternatives for it. In this paper we propose a way to compute classical Softmax with fewer memory accesses and hypothesize that this reduction in memory accesses should improve Softmax performance on actual hardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to 1.3x and Softmax+TopK combined and fused by up to 5x.

arXiv comments
1) Added link to the benchmark code, 2) Benchmarked Safe Softmax + Top-K fused and attributed part of 5x explicitly to fusion in sections 5.2 and 6, 3) Stylistic changes, 4) Minor clarifications
Journal reference
DOI
Subjects
cs.PFPerformance cs.AIArtificial Intelligence cs.CLComputation and Language

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Oct 2025 to Oct 2025.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. We bought the whole GPU, so we're damn well going to use the whole GPU 1 comment

    • > they set their hiring bar for engineers too high Not sure I agree, if you look at the head count growth of companies like OpenAI, Anthropic etc, it is super fast, its already pretty hard to keep everything working smoothly with that rate of employee growth, so going faster…

      sailingparrotView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.