HackerPapers

arXiv:1904.02679cs.HC

Visualizing Attention in Transformer-Based Language Representation Models

Jesse Vig

Abstract

We present an open-source tool for visualizing multi-head self-attention in Transformer-based language representation models. The tool extends earlier work by visualizing attention at three levels of granularity: the attention-head level, the model level, and the neuron level. We describe how each of these views can help to interpret the model, and we demonstrate the tool on the BERT model and the OpenAI GPT-2 model. We also present three use cases for analyzing GPT-2: detecting model bias, identifying recurring patterns, and linking neurons to model behavior.

arXiv comments
Journal reference
DOI
Subjects
cs.HCHuman-Computer Interaction cs.LGMachine Learning stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Jul 2020 to May 2023.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Why the original transformer figure is wrong, and some other tidbits about LLMs 1 comment

    • The transformer doesn't have the nice pyramid shape of CNNs, but it still needs multiple layers. There have been papers showing non-trivial interactions between successive layers, forming more complex circuits. https://transformer-circuits.pub/2021/framework/index.html (warning…

      visargaView on HNParent context

  2. How GPT3 Works – Visualizations and Animations 1 comment

    • Yes this seems like an early work in progress, compared to Jay's previous Transformer articles. In addition to your link, I've found a really good Transformer explanation here (backed by a Github repo w/ lively Issues talk): http://www.peterbloem.nl/blog/transformers…

      ypcxView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.