Conditional Positional Encodings for Vision Transformers — mentioned by kaiokendev

Thu, 22 Jun 2023 11:07:37 GMT

It seem the site does not allow the in-page anchor: https://kaiokendev.github.io/til#extending-context-to-8k Im not sure what is the rule in linking reddit but here is the related thread [0] Llama.cpp has also started implementing it [1] and any help will be appreciated Im sure. Also, here is what I wrote in the thread for some further clarification: > I suspected there was no way that I was the first to try something like this. After getting into contact with Ofir Press [2], he offered some kind words and pointed me in the direction of Vision Transformers. It turns out that conditional…

Paper and discussion history. A mention is not an endorsement.

Conditional Positional Encodings for Vision Transformers · HackerPapers discussions

Conditional Positional Encodings for Vision Transformers — mentioned by kaiokendev