Probing Neural Network Comprehension of Natural Language Arguments · HackerPapers discussions

Probing Neural Network Comprehension of Natural Language Arguments — mentioned by nanolith

Tue, 28 May 2024 15:10:25 GMT

"On the Dangers of Stochastic Parrots" (doi:10.1145/3442188.3445922) is a great introduction to this, even with its flaws. I also recommend Mitchell 2023 (doi:10.1073/pnas.2215907120) and Niven 2019 (arXiv:1907.07355) as good starting points. These don't directly address your question, but within the context of these papers, it's possible to see how the weighting of prose (which is a relation to prompts) and code can become skewed easily.

Paper and discussion history. A mention is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — mentioned by nicklecompte

Tue, 26 Dec 2023 15:48:11 GMT

AI is not even close to having true logical reasoning, that's probably decades away. The issue is that cognitive scientists are clueless. Scientists have a good model for associative reasoning, which is the basis of modern neural networks, but we don't have a clue how abstract reasoning actually works. All birds and mammals have advanced abstract reasoning and are far more intelligent than GPT-4: - birds and mammals are inherently able to count in almost any context because they understand what numbers actually mean; GPT-4 can only be trained to count in certain contexts. GPT-4 would be like…

Paper and discussion history. A mention is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — mentioned by YeGoblynQueenne

Sun, 17 Nov 2019 16:17:29 GMT

Regarding the "failed paradigm" (i.e: of logic- and knowledge-based AI) and how neural nets scale, specifically on language modelling tasks. I study a class of machine learning algorithms that learn logic programs (as in Prolog, or ASP) from examples. The field is Inductive Logic Programming. ILP algorithms are characterised by their ability to incorporate background knowledge and impose strong inductive biases on the hypothesis space to learn accurately from a handful of examples. My favourite example of this is learning a grammar of the aⁿbⁿ language. My implementation of one of the…

Paper and discussion history. A mention is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — mentioned by YeGoblynQueenne

Thu, 15 Aug 2019 21:01:23 GMT

I think I read this paper because someone posted it on HN (or linked to it in a comment): https://arxiv.org/abs/1907.07355 We are surprised to find that BERT's peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs the construction of an adversarial dataset on which all models…

Paper and discussion history. A mention is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — mentioned by codingslave

Mon, 22 Jul 2019 20:18:16 GMT

Similar to this post is this: https://arxiv.org/abs/1907.07355 "Probing Neural Network Comprehension of Natural Language Arguments Timothy Niven, Hung-Yu Kao (Submitted on 17 Jul 2019) We are surprised to find that BERT's peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs…

Paper and discussion history. A mention is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — HN submission

Mon, 22 Jul 2019 00:41:14 GMT

Probing Neural Network Comprehension of Natural Language Arguments

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — HN submission

Sun, 21 Jul 2019 11:44:43 GMT

BERT's success due to 'spurious statistical cues'

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Probing Neural Network Comprehension of Natural Language Arguments — HN submission

Fri, 19 Jul 2019 00:30:17 GMT

Probing Neural Network Comprehension of Natural Language Arguments

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.