LIMA: Less Is More for Alignment · HackerPapers discussions

LIMA: Less Is More for Alignment — mentioned by tucnak

Tue, 15 Oct 2024 08:12:00 GMT

The LIMA paper, I think, would be a good place to start https://arxiv.org/abs/2305.11206 You can create inputs for DPO/ORPO synthetically which is a huge one as previously it would require gigantic investments https://arxiv.org/abs/2402.10379 There's also the gemma2 paper has advanced SOTA in distil; on a side-note, there's many reasons for it but vocab_size and good sizes 9b/27b, IMHO it's currently the best model for i.e. Ukrainian. in fact, I prefer it to anything else there's, including the much larger llama's—by a mile! The model is a triumph of synthetic datasets.…

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by dvt

Thu, 04 Apr 2024 19:42:35 GMT

I think you may be misunderstanding what fine tuning does. It does not teach the model new knowledge. In fact, Meta has a paper out that argues you only need a data set of 1000[1] to achieve pretty good alignment (fine-tuning) results. (100M is way overkill.) For knowledge retrieval, you need RAG (usually using the context window). [1] https://arxiv.org/pdf/2305.11206.pdf

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by isaacfung

Fri, 22 Sep 2023 11:37:17 GMT

LIMA: Less Is More for Alignment https://arxiv.org/abs/2305.11206 AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701 Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463 Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models https://arxiv.org/abs/2307.14430

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by isaacfung

Mon, 18 Sep 2023 05:25:16 GMT

Quality matters just as much as quantity. LIMA: Less Is More for Alignment https://arxiv.org/abs/2305.11206 AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701 Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by antupis

Fri, 25 Aug 2023 11:03:49 GMT

If they are very good examples then less is more https://arxiv.org/abs/2305.11206

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by thaw13579

Mon, 19 Jun 2023 18:01:14 GMT

I don't think it's so cut and dry. The article paints a somewhat simplistic picture of how the best performing LLMs work. The unsupervised pre-trained networks are indeed data hungry, but the secondary supervised learning stages actually can get by with a far smaller set of highly curated prompt-response data, e.g. LIMA (https://arxiv.org/abs/2305.11206). Another factor is that generative data distributed online may be quite high quality (because people find it interesting enough to share), so it's plausible this could actually improve model performance. Some LLMs have been trained with data…

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by anon373839

Thu, 01 Jun 2023 08:30:25 GMT

Sure, here are two: 1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks https://huggingface.co/papers/2305.14201 2. Gorilla: Large Language Model Connected with Massive APIs https://arxiv.org/abs/2305.15334 Consider also these 2 papers supporting the feasibility of fine-tuning: 3. LIMA: Less Is More for Alignment [showing that a very small number of high quality examples is sufficient to align a base model] https://arxiv.org/abs/2305.11206 4. QLoRA: Efficient Finetuning of Quantized LLMs [showing that LLMs can now be fine-tuned quickly on consumer-grade GPUs]…

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by cubefox

Thu, 25 May 2023 16:54:29 GMT

I wonder whether these approaches fit into the above categories: https://arxiv.org/abs/2305.13735 https://arxiv.org/abs/2305.11206

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — mentioned by srslack

Mon, 22 May 2023 14:20:33 GMT

People are literally doing this for free. You said that there is no individuals, and no non-profits, that can do this. Which is laughable. They're doing it right now: https://arxiv.org/abs/2305.11206 It may even be the case that all of that RLHF training that OpenAI does simply lessens the quality of generations, as suggested by one of their own papers and the paper above.

Paper and discussion history. A mention is not an endorsement.

LIMA: Less Is More for Alignment — HN submission

Mon, 22 May 2023 12:18:44 GMT

LIMA: Less Is More for Alignment

39 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

LIMA: Less Is More for Alignment — HN submission

Mon, 22 May 2023 03:30:34 GMT

LIMA: Less Is More for Alignment

27 points; 9 discussion replies on HN. Paper and source history. A submission is not an endorsement.