Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?

Some of the noise comes from writing for two audiences(humans and agents) in the same file. There are established conventions for separating the two, and research suggests that the “repository overview” documents agents tend to favor aren’t even helpful to agents themselves.…

ghboo0927Oct 12, 2026View on HNParent context

Theres a study from earlier this year which suggests the opposite: https://arxiv.org/abs/2602.11988 Most of the agents.md and what people use it for / write into is does, in fact, not make a difference. Now, sure, this study is a bit old for LLM standards - as everything beyond…

Systemerror7A69Aug 20, 2026View on HNParent context

Studies show that LLM-generated context files have a negative impact on LLM performance: https://arxiv.org/abs/2602.11988

iLoveOncallJun 3, 2026View on HNParent context

Believe it or not, after writing this comment I was doing some more reading on the task. I'm planning to reorganize our context repo after finding this paper (it argues that AI generated context files can stunt the performance of models): https://arxiv.org/abs/2602.11988 For…

NichoPaolucciJun 4, 2026View on HNParent context

Are you referring to the one (1) study that showed that when cheaper LLM's auto-generated an AGENTS.md, it performed more poorly than human editted AGENTS.md? https://arxiv.org/abs/2602.11988 I'd love to see other sources that seek to academically understand how LLM's use…

criley2Apr 25, 2026View on HNParent context

Great, so how do you know this stuff works? Did you evaluate it against other approaches? How do you know it's actually reliable? The Vercel team had some interesting findings[1]: > In 56% of eval cases, the skill was never invoked. The agent had access to the documentation but…

imiricMar 27, 2026View on HNParent context

Another reference: https://arxiv.org/abs/2602.11988. Too many times they ignore what is defined depending on the prompt and/or context. It's just another task to sync a source of truth.

morganneyMar 22, 2026View on HNParent context

The research is: Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? https://arxiv.org/abs/2602.11988 --- Overall, it makes sense to me that excessive and/or unnecessary instructions dilute the importance of each instruction. Ideally it shouldn't…

OutOfHereMar 6, 2026View on HNParent context

This method of copying an LLM-generated summary of your preferences into Claude memory feels similar to their recommendation to use /init to generate a CLAUDE.md based on the project, which recent research[0] suggests may be counterproductive. I would assume both Claude memory…

sheeptMar 1, 2026View on HNParent context

ETH Zurich tested this: LLM-generated prose context = -3% performance, +20% cost. Even human-written = +4% at +19% cost. The problem is prose bloat. Structured formats avoid that by design. https://arxiv.org/abs/2602.11988

wolfejamFeb 27, 2026View on HNParent context

https://arxiv.org/abs/2602.11988

withinboredomFeb 27, 2026View on HNParent context

The persistent identity files are interesting but there's a cost problem. A recent paper (arxiv 2602.11988 https://arxiv.org/html/2602.11988v1) found context files increase inference cost by 20%+ with marginal performance gains; LLM-generated ones actually decreased success…

mercutio93Feb 26, 2026View on HNParent context

I'm actually on the fence with skills. Vercel shared a study where they claimed skills performed actually worse [0] - than just injecting into the context directly via agents.md. Similarly, there was a paper recently that suggested the same [1] Of course, the classic response to…

neyaFeb 25, 2026View on HNParent context

There is some papers [0] showing that the skill and agent files reduce the reasoning effectiveness in some use cases (e.g. autogenerated) [0] https://arxiv.org/abs/2602.11988 reference: https://news.ycombinator.com/item?id=47034087

mycallFeb 24, 2026View on HNParent context