Efficient Guided Generation for Large Language Models · HackerPapers discussions

Efficient Guided Generation for Large Language Models — mentioned by mcyc

Sat, 17 Jan 2026 00:33:27 GMT

This is a fantastic guide! I did a lot of work on structured generation for my PhD. Here are a few other pointers for people who might be interested: Some libraries: - Outlines, a nice library for structured generation - https://github.com/dottxt-ai/outlines - Guidance (already covered by FlyingLawnmower in this thread), another nice library - https://github.com/guidance-ai/guidance - XGrammar, a less-featureful but really well optimized constrained generation library - https://github.com/mlc-ai/xgrammar - This one has a lot of cool technical aspects that make it an interesting project Some…

Paper and discussion history. A mention is not an endorsement.

Efficient Guided Generation for Large Language Models — mentioned by tikhonj

Sat, 25 May 2024 10:48:07 GMT

Looks like it actually changes how you do token generation to conform to a given context-free grammar. It's a way to structure how you sample from the model rather than a tweak to the prompt, so it's more efficient and guarantees that the output matches the formal grammar. There's a reference to the paper that describes the method at the bottom of the README: https://arxiv.org/pdf/2307.09702

Paper and discussion history. A mention is not an endorsement.

Efficient Guided Generation for Large Language Models — mentioned by remilouf

Mon, 14 Aug 2023 20:28:38 GMT

Figure 2 in our paper (https://arxiv.org/abs/2307.09702) shows the difference between guidance and outlines to generate a sequence that is valid to a regex. Jsonformer uses the same technique as guidance. Extrapolate this to several fields. Note that we still need to manage the KV cache in outlines. It’s a small interface change that will be made this week hopefully, but we’ve been focusing on constrained generation so far.

Paper and discussion history. A mention is not an endorsement.

Efficient Guided Generation for Large Language Models — mentioned by remilouf

Mon, 14 Aug 2023 20:21:35 GMT

Figure 2 in our paper (https://arxiv.org/abs/2307.09702) shows the difference for a single regex.

Paper and discussion history. A mention is not an endorsement.

Efficient Guided Generation for Large Language Models — HN submission

Mon, 14 Aug 2023 18:52:54 GMT

Show HN: LLMs can generate valid JSON 100% of the time

Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We can transform this…

854 points; 303 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Efficient Guided Generation for Large Language Models — mentioned by farissbahi

Sat, 22 Jul 2023 00:16:52 GMT

There's a paper as well. :) https://arxiv.org/pdf/2307.09702.pdf

Paper and discussion history. A mention is not an endorsement.