HackerPapers

arXiv:2405.00704cs.CL

A Survey on the Real Power of ChatGPT

Ming Liu, Ran Liu, Ye Zhu, Hua Wang, Youyang Qu, Rongsheng Li, Yongpan Sheng, Wray Buntine

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

ChatGPT has changed the AI community and an active research line is the performance evaluation of ChatGPT. A key challenge for the evaluation is that ChatGPT is still closed-source and traditional benchmark datasets may have been used by ChatGPT as the training data. In this paper, (i) we survey recent studies which uncover the real performance levels of ChatGPT in seven categories of NLP tasks, (ii) review the social implications and safety issues of ChatGPT, and (iii) emphasize key challenges and opportunities for its evaluation. We hope our survey can shed some light on its blackbox manner, so that researchers are not misleaded by its surface generation.

arXiv comments
18 pages, 2 tables
Journal reference
DOI
Subjects
cs.CLComputation and Language cs.AIArtificial Intelligence

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from May 2024 to May 2024. Found in 1 HN submission from May 2024 to May 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun 1 comment

    • Some of the SBERT models now are based on T5 and newer architectures so there's not. The FlagEmbedding model that the author uses https://huggingface.co/BAAI/bge-base-en-v1.5 is described as an "LLM" by the people who created it. It can be used in the SBERT framework. I tried…

      PaulHouleView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.