Scaling and evaluating sparse autoencoders — mentioned by leogao

Sat, 22 Nov 2025 19:02:46 GMT

I'm also very excited about SAE/Transcoder based approaches! I think the big tradeoff is that our approach (circuit sparsity) is aiming for a full complete understanding at any cost, whereas Anthropic's Attribution Graph approach is more immediately applicable to frontier models, but gives handwavier circuits. It turns out "any cost" is really quite a lot of cost - we think this cost can be reduced a lot with further research, but it means our main results are on very small models, and the path to applying any of this to frontier models involves a lot more research risk. So if accepting a bit…

Paper and discussion history. A mention is not an endorsement.

Scaling and evaluating sparse autoencoders — HN submission

Thu, 21 Nov 2024 20:37:56 GMT

Show HN: Llama 3.2 Interpretability with Sparse Autoencoders

I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1] https://transformer-circuits.pub/2024/scaling-monosemanticit... [2]…

579 points; 99 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Scaling and evaluating sparse autoencoders · HackerPapers discussions

Scaling and evaluating sparse autoencoders — mentioned by leogao

Scaling and evaluating sparse autoencoders — HN submission