Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM · HackerPapers discussions

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM — mentioned by mindcrime

Wed, 12 Jun 2024 03:29:14 GMT

Possible? Sure. There's a huge body of research into that sort of thing going back to the 1980's and including things like Minsky's Society of Mind[1] approach, Blackboard architectures[2], and the entire field of "multi-agent systems"[3]. On a more contemporary note, there's all sorts of ensemble systems[4], and approaches using model blending[5], mixture of experts models[6], etc. Will it actually turn out that way? Who's to say? But if I had a gun to my head and had to guess, I'd guess that future AI's will incorporate some sort of "multiple models" approach where some may be very big…

Paper and discussion history. A mention is not an endorsement.

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM — mentioned by kgeist

Tue, 13 Feb 2024 15:09:25 GMT

Here they simply used different models for different turns and apparently it gave more "engaging" results: https://arxiv.org/abs/2401.02994

Paper and discussion history. A mention is not an endorsement.

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM — HN submission

Thu, 11 Jan 2024 13:00:51 GMT

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

147 points; 45 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM — HN submission

Wed, 10 Jan 2024 23:37:27 GMT

Blending Is All You Need – Cheaper and Better to Trillion-Param LLM

5 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.