AttentionViz: A Global View of Transformer Attention · HackerPapers discussions

AttentionViz: A Global View of Transformer Attention — mentioned by kordlessagain

Sat, 13 May 2023 17:47:02 GMT

I've been working on a project I've called DocGPT: https://github.com/featurebasedb/DocGPT. This is a work in progress and I'll be putting together a release next week, once I've got the instructions for running it done. It was my assumption that GPT-4 could be used for keyterm and question extraction from the fragments of a document and perhaps used to build a semantic graph that could be leveraged for rapid prompt assembly, augmenting the vector searches that are normally done. The idea here is to set "attention flags" on the document during discussion. In this example, I indexed the arxiv…

Paper and discussion history. A mention is not an endorsement.

AttentionViz: A Global View of Transformer Attention — mentioned by djoldman

Tue, 09 May 2023 14:14:37 GMT

About button: Attention Viz is an interactive tool that visualizes global attention patterns for transformer models. To create this tool, we visualize the joint embeddings of query and key vectors. paper: https://arxiv.org/abs/2305.03210 docs: https://catherinesyeh.github.io/attn-docs/

Paper and discussion history. A mention is not an endorsement.

AttentionViz: A Global View of Transformer Attention — HN submission

Mon, 08 May 2023 02:40:05 GMT

AttentionViz: A Global View of Transformer Attention

3 points; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.