BERT Rediscovers the Classical NLP Pipeline · HackerPapers discussions

BERT Rediscovers the Classical NLP Pipeline — mentioned by Xmd5a

Wed, 23 Apr 2025 06:05:13 GMT

https://arxiv.org/abs/1906.04341 https://arxiv.org/abs/1905.05950 https://en.wikiversity.org/wiki/Psycholinguistics/Models_of_...

Paper and discussion history. A mention is not an endorsement.

BERT Rediscovers the Classical NLP Pipeline — mentioned by jayalammar

Thu, 25 May 2023 20:27:55 GMT

This is a field I find fascinating. It's generally the research field of Machine Learning Interpretability. The BlackboxNLP workshop is one of the main places for investigating this and is a very popular academic workshop https://blackboxnlp.github.io/ One of the most interesting presentations in the last session of the workshop is this talk by David Bau titled "Direct Model Editing and Mechanistic Interpretability". David and his team locate exact information in the model, and edit it. So for example they edit the location of the Eiffel Tower to be in Rome. So whenever the model generates…

Paper and discussion history. A mention is not an endorsement.

BERT Rediscovers the Classical NLP Pipeline — mentioned by KhoomeiK

Fri, 17 Feb 2023 00:37:26 GMT

Several papers have explored emergent linguistic structure in LMs. Here's some early introductory work in this space. Despite having explicit syntax parses etc as input, models seem to learn something like syntax. https://arxiv.org/abs/1905.05950 https://aclanthology.org/N19-1419/ https://arxiv.org/abs/1906.04341

Paper and discussion history. A mention is not an endorsement.

BERT Rediscovers the Classical NLP Pipeline — mentioned by desku

Fri, 12 Mar 2021 11:48:14 GMT

Here's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950

Paper and discussion history. A mention is not an endorsement.