Adapting Large Language Models to Domains via Reading Comprehension — mentioned by Jimmc414

Wed, 20 Sep 2023 18:44:25 GMT

https://huggingface.co/papers/2309.09530 Abstract We explore how continued pre-training on domain-specific corpora influences large language models, revealing that training on the raw corpora endows the model with domain knowledge, but drastically hurts its prompting ability for question answering. Taken inspiration from human learning via reading comprehension--practice after reading improves the ability to answer questions based on the learned knowledge--we propose a simple method for transforming raw corpora into reading comprehension texts. Each raw text is enriched with a series of tasks…

Paper and discussion history. A mention is not an endorsement.

Adapting Large Language Models to Domains via Reading Comprehension — HN submission

Wed, 20 Sep 2023 18:41:44 GMT

Adapting Large Language Models via Reading Comprehension

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Adapting Large Language Models to Domains via Reading Comprehension · HackerPapers discussions

Adapting Large Language Models to Domains via Reading Comprehension — mentioned by Jimmc414

Adapting Large Language Models to Domains via Reading Comprehension — HN submission