https://hackerpaper.kaipfister.de/p/2105.13120/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=45025139urn:hackerpapers:submission:2105.13120:45025139Tue, 26 Aug 2025 11:38:20 GMT<p>Sequence Parallelism: Long Sequence Training from System Perspective (2021)</p><p></p><p>2 points; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2105.13120/">Paper and source history</a>. A submission is not an endorsement.</p>cs.LGhttps://news.ycombinator.com/item?id=40295452urn:hackerpapers:mention:2105.13120:40295452Wed, 08 May 2024 07:54:38 GMT<p>No, it means dividing the sequence into multiple chunks and processing them one by one, very similar to recurrence. See [1]. Sequence parallelism is needed when the sequence can&#x27;t fit in a single GPU. Sequence parallelism is the hardest parallelism, but it is required for longer sequence. Many models just trains for smaller sequence length for majority of the training and switch to sequence parallelism for last few percentage of training. [1]: https://arxiv.org/pdf/2105.13120</p><p><a href="https://hackerpaper.kaipfister.de/p/2105.13120/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG