A Primer in BERTology: What we know about how BERT works · HackerPapers discussions

A Primer in BERTology: What we know about how BERT works — mentioned by sigtstp

Mon, 05 May 2025 08:17:15 GMT

I feel this makes some fundamental conceptual mistakes and is just riding the LLM wave. "Semantics" is literally behavior under execution. This is syntactical analysis by a stochastic language model. I know the NLP literature uses "semantics" to talk about representations but that is an assertion which is contested [1]. Coming back to testing, this implicitly relies on the strong assumption of the LLM correctly associating the code (syntax) with assertions of properties under execution (semantic properties). This is a very risky assumption considering, once again, these things are stochastic…

Paper and discussion history. A mention is not an endorsement.

A Primer in BERTology: What we know about how BERT works — mentioned by MacsHeadroom

Wed, 29 Mar 2023 03:52:18 GMT

I don't understand what you want me to ask it about. So I asked it something, as usual every citation and URL is real despite not having internet access or training data after 2021: <PROMPT>List some citations of recent Machine Learning papers discussing reasoning ability in Machine Learning Models such as HellaSwag. [REPLY]" As of my knowledge cutoff in September 2021, I can provide you with some citations for ML papers that discuss reasoning abilities in models and datasets like HellaSwag. However, you may want to search for more recent papers using a platform like arXiv or Google Scholar.…

Paper and discussion history. A mention is not an endorsement.

A Primer in BERTology: What we know about how BERT works — HN submission

Tue, 10 Nov 2020 06:23:01 GMT

A Primer in BERTology: What We Know About How Bert Works

81 points; 16 discussion replies on HN. Paper and source history. A submission is not an endorsement.

A Primer in BERTology: What we know about how BERT works — HN submission

Fri, 06 Mar 2020 00:51:13 GMT

A Primer in BERTology: What we know about how BERT works

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.