Frozen Transformers in Language Models Are Effective Visual Encoder Layers — mentioned by vjsrinivas

Fri, 13 Mar 2026 20:29:25 GMT

Great work and love the detailed breakdown. This is kind of tangential, but it reminded me of this work: https://arxiv.org/pdf/2310.12973 (Frozen Transformers in Language Models are Effective Visual Encoder Layers). The paper puts out an interesting hypothesis that these LLM-derived transformer layers have the ability to "refine" any set of learned tokens, even in different modalities. I wonder if what you're seeing here is related?

Paper and discussion history. A mention is not an endorsement.

Frozen Transformers in Language Models Are Effective Visual Encoder Layers · HackerPapers discussions

Frozen Transformers in Language Models Are Effective Visual Encoder Layers — mentioned by vjsrinivas