HackerPapers

arXiv:2107.06751cs.DL

Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals

Guillaume Cabanac, Cyril Labbé, Alexander Magazinov

Abstract

Probabilistic text generators have been used to produce fake scientific papers for more than a decade. Such nonsensical papers are easily detected by both human and machine. Now more complex AI-powered generation techniques produce texts indistinguishable from that of humans and the generation of scientific texts from a few keywords has been documented. Our study introduces the concept of tortured phrases: unexpected weird phrases in lieu of established ones, such as 'counterfeit consciousness' instead of 'artificial intelligence.' We combed the literature for tortured phrases and study one reputable journal where these concentrated en masse. Hypothesising the use of advanced language models we ran a detector on the abstracts of recent articles of this journal and on several control sets. The pairwise comparisons reveal a concentration of abstracts flagged as 'synthetic' in the journal. We also highlight irregularities in its operation, such as abrupt changes in editorial timelines. We substantiate our call for investigation by analysing several individual dubious articles, stressing questionable features: tortured writing style, citation of non-existent literature, and unacknowledged image reuse. Surprisingly, some websites offer to rewrite texts for free, generating gobbledegook full of tortured phrases. We believe some authors used rewritten texts to pad their manuscripts. We wish to raise the awareness on publications containing such questionable AI-generated or rewritten texts that passed (poor) peer review. Deception with synthetic texts threatens the integrity of the scientific literature.

arXiv comments
Journal reference
DOI
Subjects
cs.DLDigital Libraries cs.CLComputation and Language cs.CYComputers and Society cs.IRInformation Retrieval

On Hacker News

Linked by 4 identifiable commenters in 8 comments across 5 threads, from Aug 2021 to Aug 2026. Found in 1 HN submission from Aug 2021 to Aug 2021. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing 1 comment

    • I wonder if this watermarking system to lead to an increase in tortured phrases, believed to be caused by plagiarism evasion tools that change random words to applicable synonyms. https://arxiv.org/abs/2107.06751

      GaryBlutoView on HNParent context

  2. Research papers using "kidney disappointment" instead of "kidney failure" 4 comments

    • From 2021: https://arxiv.org/abs/2107.06751 And SCIgen was generating bullshit papers that actually managed to be published in 2005: https://en.wikipedia.org/wiki/SCIgen

      armchairhackerView on HNParent context

    • Some more examples of these tortured phrases: https://arxiv.org/abs/2107.06751 A few of my favorites are "counterfeit consciousness" (artificial intelligence), "discourse acknowledgement" (voice recognition), and "arbitrary get right of passage to memory" (random access memory).

      duskwuffView on HNParent context

    • The phenomenon of tortured phrases in published papers has been going on since at least 2020 - long before LLMs were a significant factor, and certainly before watermarking their output was a consideration. The general working theory is that these phrases are the result of a…

      duskwuffView on HNParent context

    • https://arxiv.org/abs/2107.06751 found that a lot of the outputs were characteristic of an online tool called SpinBot. (The tool no longer seems to work the same way that it did at the time.)

      duskwuffView on HNParent context

  3. Tortured conference series thriving in computer science 1 comment

    • > The reason seems to have been the use of machine translation from languages like Chinese No, it's more deliberate than that. Per the original paper [1], the tortured phrases observed in these papers can be reproduced by running text through an online tool called "Spinbot"…

      duskwuffView on HNParent context

  4. Junk websites filled w AI-generated text pulling in money from programmatic ads 1 comment

    • And, unfortunately, it's still an ongoing problem. Some of it even ends up getting published by IEEE, e.g. [1]. There was some research a few years ago [2] into just how widespread this issue was in scientific publishing. The situation has likely only gotten worse with the…

      duskwuffView on HNParent context

  5. Tortured phrases: A dubious writing style emerging in science 1 comment

    • The paper is at https://arxiv.org/abs/2107.06751. (We merged this thread and https://news.ycombinator.com/item?id=28108111)

      dangView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.