Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? — mentioned by mbowcut2

Mon, 28 Jul 2025 17:39:46 GMT

You can, and there has been some interesting work done with it. The technique is called LogitLens, and basically you pass intermediate embeddings through the LMHead to get logits corresponding to tokens. In this paper they use it to investigate whether LLMs have a language bias, i.e. does GPT "think" in English? https://arxiv.org/pdf/2408.10811 One problem with this technique is that the model wasn't trained with intermediate layers being mapped to logits in the first place, so it's not clear why the LMHead should be able to map them to anything sensible. But alas, like everything in DL…

Paper and discussion history. A mention is not an endorsement.

Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? · HackerPapers discussions

Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? — mentioned by mbowcut2