Guiding Language Models of Code with Global Context using Monitors · HackerPapers discussions

Guiding Language Models of Code with Global Context using Monitors — mentioned by _jayhack_

Wed, 14 May 2025 01:57:49 GMT

Also worth checking out MultiLSPy, effectively a python wrapper around multiple LSPs: https://github.com/microsoft/multilspy Used in multiple similar publications, including "Guiding Language Models of Code with Global Context using Monitors" (https://arxiv.org/abs/2306.10763), which uses static analysis beyond the type system to filter out e.g. invalid variable names, invalid control flow etc.

Paper and discussion history. A mention is not an endorsement.

Guiding Language Models of Code with Global Context using Monitors — mentioned by emrek1

Tue, 12 Mar 2024 01:20:05 GMT

Each of those is great, and more sophisticated strategies are being developed all the time: incorporating static analysis at inference time to ensure code is more than just syntactically correct [0]; to multi-LLM call agent-based frameworks [1]; decomposing hard tasks into many "easier" LLM tasks [2]; analyzing internal activations to identify potential hallucinations [3]; activation steering to control generations [4]; and "Language Model Arithmetic" to compose biases for style, vocab, etc [5]. But each also either requires tight integration with an LLM inference/serving engine (to access…

Paper and discussion history. A mention is not an endorsement.

Guiding Language Models of Code with Global Context using Monitors — mentioned by disconcision

Thu, 30 Nov 2023 23:01:50 GMT

it's been done, in a limited way: https://arxiv.org/abs/2306.10763; we're working on extended this, but yeah, there are a lot of details to work out

Paper and discussion history. A mention is not an endorsement.

Guiding Language Models of Code with Global Context using Monitors — mentioned by namibj

Tue, 14 Nov 2023 08:57:22 GMT

Yeah. E.g. GPT-4-turbo's JSON-mode seems to forcibly block non-JSON-compliant outputs, at least in some way. They document that forgetting to instruct it to emit JSON may lead to producing whitespace until the output length limit is reached. In related info, there is "Guiding Language Models of Code with Global Context using Monitors" ( https://arxiv.org/abs/2306.10763 ), which essentially gives IDE-typical type-aware autocomplete to an LLM to primarily study the scenario of enforcing type-consistent method completion in a Java repository.

Paper and discussion history. A mention is not an endorsement.

Guiding Language Models of Code with Global Context using Monitors — mentioned by LakshyAAAgrawal

Tue, 15 Aug 2023 06:33:35 GMT

In our experience, at least for code generation, the experience has been that base models can be improved significantly by guiding token level generation. In our paper titled "Guiding Language Models of Code with Global Context using Monitors" (https://arxiv.org/abs/2306.10763), we propose Monitor Guided Decoding, which interfaces LLMs to static analysis, and guides the model to generate type-consistent code. Without any kind of fine-tuning, we show that using static analysis to guide token level generation at specific points leads to significantly improved quality of generated code, both in…

Paper and discussion history. A mention is not an endorsement.

Guiding Language Models of Code with Global Context using Monitors — mentioned by LakshyAAAgrawal

Mon, 14 Aug 2023 21:12:28 GMT

Hi, the paper at https://arxiv.org/abs/2306.10763 titled "Guiding Language Models of Code with Global Context using Monitors" shows how to have the language models generate code without hallucinated dereferences.

Paper and discussion history. A mention is not an endorsement.