Large Language Models Encode Clinical Knowledge · HackerPapers discussions

Large Language Models Encode Clinical Knowledge — mentioned by FlyingLawnmower

Fri, 31 Mar 2023 02:04:47 GMT

Late to the thread here, but the paper announcing Med-PaLM (https://arxiv.org/abs/2212.13138) does not report many benchmark results on Med-PaLM and is instead mostly about Flan-PaLM 540B (which is compared against in this paper). I am curious if any other Med-PaLM benchmarks have been published but I don't believe it is currently possible to do any further comparison against Med-PaLM given that the model is not public and no other open benchmark results are reported in the original Med-PaLM paper.

Paper and discussion history. A mention is not an endorsement.

Large Language Models Encode Clinical Knowledge — mentioned by aix1

Mon, 27 Mar 2023 07:01:05 GMT

One thing that jumps out is that the paper fails to provide a proper direct comparison with Med-PaLM, only mentioning the latter in passing almost as an afterthought. Looking at arXiv timestamps, the Med-PaLM preprint has been out of three months (given the rate at which this field moves, an eternity). I understand Med-PaLM the current state of the art and would be a natural point of comparison. https://arxiv.org/abs/2212.13138

Paper and discussion history. A mention is not an endorsement.

Large Language Models Encode Clinical Knowledge — mentioned by nindalf

Thu, 26 Jan 2023 11:49:04 GMT

LLMs can already pass a medical exam. https://arxiv.org/abs/2212.13138 So maybe?

Paper and discussion history. A mention is not an endorsement.

Large Language Models Encode Clinical Knowledge — mentioned by gardenfelder

Wed, 11 Jan 2023 19:40:57 GMT

The paper Large Language Models Encode Clinical Knowledge https://arxiv.org/abs/2212.13138

Paper and discussion history. A mention is not an endorsement.

Large Language Models Encode Clinical Knowledge — HN submission

Wed, 11 Jan 2023 09:23:56 GMT

Large Language Models Encode Clinical Knowledge

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Large Language Models Encode Clinical Knowledge — HN submission

Tue, 10 Jan 2023 16:26:54 GMT

Large Language Models Encode Clinical Knowledge

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Large Language Models Encode Clinical Knowledge — HN submission

Wed, 04 Jan 2023 15:27:01 GMT

Large Language Models Encode Clinical Knowledge

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Large Language Models Encode Clinical Knowledge — HN submission

Tue, 03 Jan 2023 22:20:52 GMT

Large Language Models Encode Clinical Knowledge

9 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Large Language Models Encode Clinical Knowledge — mentioned by nopinsight

Tue, 03 Jan 2023 15:08:02 GMT

Related: "Large Language Models Encode Clinical Knowledge" https://arxiv.org/abs/2212.13138 "On the MedQA dataset consisting of USMLE style questions with 4 options, our Flan-PaLM 540B model achieved a multiple-choice question (MCQ) accuracy of 67.6%..." "The percentages of correctly answered items required to pass varies by Step and from form to form within each Step. However, examinees typically must answer approximately 60 percent of items correctly to achieve a passing score." -- https://www.usmle.org/bulletin-information/scoring-and-score... . It seems like the models in the paper could…

Paper and discussion history. A mention is not an endorsement.

Large Language Models Encode Clinical Knowledge — mentioned by rafaelero

Fri, 30 Dec 2022 19:00:10 GMT

Oh, I mistook this paper for the one that was released more recently from DeepMind + Google. [1] [1] Can large language models reason about medical questions? https://arxiv.org/abs/2212.13138

Paper and discussion history. A mention is not an endorsement.

Large Language Models Encode Clinical Knowledge — HN submission

Tue, 27 Dec 2022 14:50:44 GMT

Large Language Models Encode Clinical Knowledge

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.