Unlimiformer: Long-Range Transformers with Unlimited Length Input · HackerPapers discussions

Unlimiformer: Long-Range Transformers with Unlimited Length Input — HN submission

Thu, 01 Aug 2024 17:12:15 GMT

Unlimited Context Length possible? hear NeurIPS paper author Friday Paper Club

Is Unlimited Context Length really possible? At what cost? Amanda Bertsch, author of 2023 NeurIPS paper Unlimiformer, will describe the architecture and take questions at this Friday's Oxen.ai Paper Club. Greg Schoeninger u/FallMindless3563, Oxen CEO and Master of Plain Speak, will help interp the concept and relate it to other papers we have reviewed. Call: https://oxen.ai/community The trick asserted to make Unlimited Context Length possible: Offload the cross attention calc to a K-Nearest Neighbors (K-NN) index. I tweeted someone's clever animation of K-NN here…

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — mentioned by maskedpirate

Fri, 12 May 2023 07:07:12 GMT

Is there any paper or architecture about this model that explains how they achieved it? is it unlimiformer [https://arxiv.org/abs/2305.01625?utm_source=tldrai] or something different?

Paper and discussion history. A mention is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — mentioned by j2kun

Fri, 05 May 2023 18:37:15 GMT

Appendix C of the PDF has an example book summary of The Brothers Karamazov, with noted minor factual errors https://arxiv.org/pdf/2305.01625.pdf > Unlimiformer summary: > The first part of the novel focuses on the question of whether or not the Russian nobleman, Dmitri Fyodorovitch, has killed his father. In the town of Ivanovna, the lieutenant-colonel of the Mushenkhanovitch is accused of the murder of his brother Ivanovitch. The lieutenant-incommand, Vasilyevitch, takes the form of a dog, and the two men–the two men and the woman who are questioned by the court-martial–murphy. The two men…

Paper and discussion history. A mention is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — HN submission

Fri, 05 May 2023 17:53:50 GMT

Unlimiformer: Long-Range Transformers with Unlimited Length Input

335 points; 101 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — HN submission

Fri, 05 May 2023 07:37:11 GMT

Unlimiformer: Long-Range Transformers With Unlimited Length Input

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — HN submission

Thu, 04 May 2023 22:35:56 GMT

Unlimiformer: Long-Range Transformers with Unlimited Length Input

3 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — HN submission

Wed, 03 May 2023 15:21:46 GMT

Unlimiformer: Long-Range Transformers with Unlimited Length Input

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Unlimiformer: Long-Range Transformers with Unlimited Length Input — HN submission

Wed, 03 May 2023 07:08:13 GMT

Unlimiformer: Long-Range Transformers with Unlimited Length Input

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.