Does whispering to agents in docs help?

27 points by theletterf 6 days ago|9 comments

•

heymaikol 22 hours ago

I’ve noticed something similar with coding agents. Specific rules seem to work better than simply saying “this is for AI.” I wonder if files like AGENTS.md work differently because the tools are designed to read from them first.

•

theletterf 22 hours ago

Indeed. Another thing worth exploring is how much the position of instructions matter; Claude claims to read docs as a whole unless they're very large, though it might be that it only grabs the first lines of a doc sometimes, in which case there is an "above the fold" effect.

•

psygn89 20 hours ago

Not sure, but if shared in the same docs as what we would read, it also allows us to skip over it with the assumption that it's a special set of instructions for AI.

•

theletterf 20 hours ago

I think it's easier to serve those bits as Markdown to agents only through content negotiation.

•

croemer 21 hours ago

The experiment in TFA is saturated so there's no signal there, unfortunately.

•

theletterf 20 hours ago

Agreed on the first experiment, that's why the post says it hit the ceiling. The others are not saturated. I'd love to find a way of making the first experiment more discriminating.

•

orsorna 22 hours ago

Documentation is only useful for agents if the source is unavailable, and if you are trying to save on tokens and trust that the docs are accurate.

If none of that is relevant to you, or rather accuracy is your concern, it's much better just to search the source code for your answers, from how to set up the application to configuration questions to troubleshooting. Bonus points if you strip out all the comments programmatically via linting before you even let the agent view the code, to avoid possible context poisoning.

•

theletterf 22 hours ago

Code doesn't explain best practices, architectural concerns, and so on. Best combo? Docs + code.

•

jaggederest 21 hours ago

It also redirects the agents from adding enormous sloppy comments in the code - I have a linter that only allows 1 line comments with a reference to a docs folder tag `See BOB-IS-UNCLE.md`

Then you can ruthlessly prune the docs folder on PR review and regular passes when e.g. claude really wants to write a novel about the glorious intricacies of bugs it's solved.