Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems — mentioned by mateiz

Mon, 08 Apr 2024 01:31:54 GMT

This is a cool paper showing there is value in using an LLM multiple times, but in recent research we showed that with majority voting, quality can decrease past some point as you make more calls. Check out https://arxiv.org/pdf/2403.02419.pdf. It raises the natural question of how to design the best inference algorithm given an LLM you can call multiple times.

Paper and discussion history. A mention is not an endorsement.

Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems · HackerPapers discussions

Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems — mentioned by mateiz