Tracr: Compiled Transformers as a Laboratory for Interpretability · HackerPapers discussions

Tracr: Compiled Transformers as a Laboratory for Interpretability — mentioned by syllogism

Tue, 03 Nov 2026 13:23:49 GMT

Here's the work that shows how rule-based logic can be compiled into transformer weights: * Thinking Like Transformers” by Weiss, Goldberg & Yahav (ICML 2021).: https://arxiv.org/abs/2106.06981 * Tracr: Compiled Transformers as a Laboratory for Interpretability https://arxiv.org/abs/2301.05062 (code at https://github.com/deepmind/tracr) So it's not really an "LLMs can modify weights" thing. Rather it's "LLMs can write programs that can write or modify weights". Indeed LLMs can write many programs, it's impressive.

Paper and discussion history. A mention is not an endorsement.

Tracr: Compiled Transformers as a Laboratory for Interpretability — mentioned by yurimo

Tue, 03 Nov 2026 11:56:36 GMT

I don't think I am. I have cited ROME as an early example of weight editing in large language models, the research has moved on since then. I think you are misinformed or don't know what has been done previously and extrapolating the results in this paper. In exp 1, you can recover a lot of the game geometry simply by visualizing the first-layer weights with numpy, which I actually did this morning, while reading the paper. In exp 2, the shapes are visible in the output-layer weights. The authors themselves acknowledge that these experiments exploit this. For 3, Astra uses Python scripts to…

Paper and discussion history. A mention is not an endorsement.

Tracr: Compiled Transformers as a Laboratory for Interpretability — mentioned by nojvek

Wed, 13 Dec 2023 14:44:16 GMT

There are papers written about this. https://openreview.net/pdf/0c9a0f445d430a4d2bf9a08078c435091... Without a loop - an LLM takes a fixed set of tokens and produces a fixed set of tokens. It has some internal state. So without a loop an LLM cannot be a general purpose computer (Turing machine). Turing machines are simple. They need 3 things - infinitely long tape, ability to have internal state, and conditionally write and move tape based on what is on tape. So if an LLM had access to infinite memory, all it needs is to implement a subleq instruction which is quite simple, and run itself in…

Paper and discussion history. A mention is not an endorsement.

Tracr: Compiled Transformers as a Laboratory for Interpretability — mentioned by teddykoker

Fri, 22 Sep 2023 17:43:10 GMT

A related line of work is "Thinking Like Transformers" [1]. They introduce a primitive programming language, RASP, which is composed of operations capable of being modeled with transformer components, and demonstrate how different programs can be written with it, e.g. histograms, sorting. Sasha Rush and Gail Weiss have an excellent blog post on it as well [2]. Follow on work actually demonstrated how RASP-like programs could actually be compiled into model weights without training [3]. [1] https://arxiv.org/abs/2106.06981 [2] https://srush.github.io/raspy/ [3] https://arxiv.org/abs/2301.05062

Paper and discussion history. A mention is not an endorsement.

Tracr: Compiled Transformers as a Laboratory for Interpretability — mentioned by rslice

Sat, 11 Feb 2023 03:36:09 GMT

Perhaps GPT-like models are already capable enough to do math, but they need to store what we call mathematical reasoning as one of many distinct processing pathway and tap into it whenever the context is appropriate. Easy to say obviously but there's some promising work in this direction[1] [1]Tracr: Compiled Transformers as a Laboratory for Interpretability, https://arxiv.org/pdf/2301.05062.pdf

Paper and discussion history. A mention is not an endorsement.