Latest posts

Terence Eden, in "I paid people to try and follow my README", argues that developers are poor at writing documentation, often leaving users with scattered notes, obsolete tips and dead forums.

He has previously recommended testing a README himself on a fresh virtual machine, but concedes that authors struggle to see past their own assumptions: that certain commands need sudo, that a typo like -foo obviously meant --foo, or that a reboot is implied¹. His solution for ActivityBot, an ActivityPub bot, was to recruit real users to proofread and execute. As part of an NLnet grant application he budgeted €25 per person for an hour of their time, found volunteers via Mastodon, and ran video sessions in which each participant shared their screen and narrated aloud what they were doing, what confused, frustrated or delighted them.

It's a brilliant process, honestly.

The sessions exposed a long list of problems. The link to the demo tool was wrong. Some people read READMEs in a terminal. Basic instructions like renaming a file, and especially a hidden file, were unclear. Participants questioned whether the demo tool should run on the web or locally, and why some text needed quoting while other text did not. Technical terminology needed explaining. The section order was confusing, the README never actually explained what the software did, and a section the author found technically interesting baffled everyone else. Not all web servers handle permissions the same way, and his jokes were not just unfunny but actively confusing².

Eden revised the README after each session and retested it with the next participant, iterating across roughly €150 of total payments.

He addresses the obvious possibility - simulating users with an LLM - by saying he simply wants to talk to real people³, who bring unique perspectives and whose audible frustration signals which mistakes matter most, and drew on his technical writing experience at GOV.UK, where human second-eye reviewers stripped out flowery prose and caught errors no spellchecker would. He took notes by hand rather than using AI tools.

Eden does not claim the README is now perfect, only demonstrably easier to follow, using actual data from actual users. His recommendation is that developers should watch real people attempt their instructions while thinking aloud. Payment is not required if unaffordable, but the practice reliably improves documentation.

What I do is different, and it isn't a replacement for what he did. It's just the bit that comes earlier.

The authorial pass

Nearly every piece of documentation I write gets an edit pass that demands Simplified Technical English⁴, written for three sets of readers.

The first set of readers is the junior developer: someone who will follow the instructions literally and has no reserve of experience to fill a gap with.

The second is the disinterested reader: someone competent who does not care about this software (or branch) and will give it exactly as much attention as it takes to get the task done. This is someone who'll copy-paste instructions they see, without reading the admonition that says "run this only if..."

The third is the adversarial reader: someone trying to misread the instructions, who might resent what's happening, who asks what happens if a step is skipped, done twice, or done with the wrong value.

STE's rules are simple.

• One instruction per sentence.
• One word for one thing, no synonyms.
• No humor⁵.
• No ambiguity about what is literal and what is a description. Write the command, not the act.

Run Eden's list against that set of requirements and most of the list goes away before anyone sees the README: the inconsistent quoting, the jokes, the unexplained terms, and the hidden-file rename all die in the STE pass, because the pass refuses to let you write "rename the file" when what you mean is a specific invocation of mv - and even that mv assumes you're on a POSIX system and not, say, Windows. STE would demand you show the right command for each supported operating system as that operating system would need it.

The structural findings go elsewhere. Personally, I try to use Diátaxis⁶, which splits documentation into tutorials, how-to guides, reference, and explanation, and refuses to let them mix. Eden's confusing section order, his technically-interesting section that confused everyone, and his never saying what the software does are all Diataxis failures: explanation leaking into a how-to, and the tutorial and explanation quadrants simply absent. You can't finish a README without the quadrants, because the first thing a junior or disinterested reader needs is the explanation of what this is, and the second is a tutorial that gets them to a working state.

Let's be clear about Diataxis: for any software with real complexity it is a massive pain to do well, and I don't think I've ever considered it "finished." I apply it iteratively: every pass over the docs moves something to the quadrant it belongs in, and the next pass does it again. The attempt is ongoing, which is the point. A structure you keep attempting doesn't rot the way a structure you declared "done" does.

There are still missing pieces, though

Most of Eden's findings are errors of writing, and discipline prevents errors of writing. One is an error of fact: the demo link was wrong, which is checkable, and I'll come back to that. The rest are errors of environment, and no discipline catches those. As mentioned, some people read the README in a terminal: that's about the reader's environment, which the author can't see from inside their own. Web servers differ on permissions: that's a fact about deployments the author hasn't done. Only a reader exposes those, and only a reader who isn't you.

That's the shape of it. The authorial pass shrinks the space a reader can get lost in. The reader finds what's left. Ideally, you'd have all of it: the authorial pass with as small an exposed surface as possible, and reader passes that poke at the holes.

Eden takes his notes by hand and won't feed the machine, and I appreciate the stance. I feed the machine for what it's good at doing: checking a document for STE conformance is a mechanical job with a specification⁷, and a model does it well and tirelessly; that is a different thing from having it simulate a reader, which he rightly declines, and a very different thing from having it write the README. The model finds the sentence with two instructions in it, and it finds the dead link, because checking a link is the kind of thing it does without being asked twice. It can't tell me the reader is in a terminal.

The adversarial reader is the one piece of my pass that Eden's method doesn't have, and I'd offer it to him. His volunteers were cooperative. They wanted him to succeed and were on a video call to help. An adversarial read finds the class of failure where the instructions are correct but not robust, where the docs are right and the reader still ends up somewhere bad. That reader costs nothing, because you can be them yourself, and the pass is worth it.