Kickoff
Three days after Christmas 2025, I started building a system whose entire job was to write down what the AI actually did.
Not what I asked it to do. Anyone can write down a request; my two hundred prompt files were basically a museum of requests. What it did — every tool call and whether it succeeded, every file touched, every outcome: success, partial success, failure, blocked, needs review. I called the project ACME, and its one-line pitch was simple: track LLM-assisted development work directly in your git repository.
The itch behind it: agents were now doing real work in my repos, and I had no record a future me could trust. If a change landed three weeks ago, what did the agent do? What did it decide? What did I approve, and did I mean to? Chat logs scroll away. Dashboards live somewhere else and die when the subscription does. But git was already the system of record for the work itself — so I stored the tracking data in git notes, riding alongside the commits they describe, syncing with an ordinary push and pull. No new infrastructure. The audit trail travels with the code, the way a passport travels with a person instead of staying filed at the embassy.
The Run
Naming the things
The part that shaped everything I built afterwards wasn’t the storage. It was being forced to name the things. What actually happens in LLM-assisted work? I ended up with a small vocabulary: an effort (a unit of LLM work), a thought (an idea surfaced along the way), a task (work discovered but not done), a review (a question queued for a human), a rule (something the LLM must or must not do), a standard, and eventually an observation. Once those existed as schema rather than vibes, whole new behaviours became possible. Vibes cannot be queried. A WHERE outcome = 'blocked' can.
You don’t ask for compliance. You build it.
Including the one I’m most glad I built. By January 9th, ACME had a server-side state machine enforcing the workflow — and one rule I’d single out for any team: the LLM cannot approve its own implementation plan. There’s a commit that says exactly that, which is a strange sentence to have needed a commit for, and an even stranger one to have needed five months earlier.
e08de830 feat(mcp): enforce mandatory ACME workflow with server-side state machine
fdfca061fix(mcp): prevent LLM auto-approval of implementation plans (#60)Rules could hard-stop an agent mid-flight and queue a review for a human. Not prompt-level pleading — “please check with me first” — but structural enforcement the model couldn’t talk its way around. Nobody has ever needed to write MANDATORY into a state machine. It just is one. After months of shouting in all-caps, I finally understood: you don’t ask for compliance. You build it.
The Debrief
This stage is about putting observability in before you need it, which is a sentence that is much easier to write than it was to live by:
On The Training Ground
One-Day Effort Capture. A working day of logging what your agents actually did — no tooling to install, just a standing instruction and a file. Tomorrow morning you’ll learn how much you’d already forgotten.
Info
You’ll need: any agentic coding tool, one normal working day, and a repo you’ll be working in. Optional but better: a tool with stop-hooks (Claude Code has them) so the logging is automatic instead of remembered.
At your repo root:
touch LOG.jsonl && echo "LOG.jsonl" >> .gitignoreThen put the standing instruction where your agent will see it every session — your CLAUDE.md or equivalent:
At the end of every session, append ONE line to LOG.jsonl:
{"when": "<ISO date>", "task": "...", "files": [...],
"outcome": "done|partial|abandoned", "surprised_me": "..."}The surprised_me field is the one that pays for the whole exercise — keep it.
If your tool has lifecycle hooks, wire the instruction to fire on session end instead of trusting anyone’s memory. In Claude Code that’s a Stop hook in .claude/settings.json; the agent gets reminded, every time, mechanically. (If not — skip this step and accept a few missed entries as data about why hooks exist.)
No curation, no performing for the log. Parallel sessions, abandoned experiments, dead ends — all of it logs one line each. The log’s value comes precisely from the sessions you wouldn’t have bothered writing up.
Next morning, before anything else, read the log. Then go one better — fresh session:
Read LOG.jsonl. Summarise yesterday: what was attempted, what finished,
what was abandoned. Flag anything that looks risky or unfinished, and
anything two sessions did that might conflict.Count how many entries describe things you had already forgotten happened. That number is your case.
If the morning read was worthless, delete the file — you spent one day learning that cheaply. If it caught a forgotten loose end or a conflict (it usually does), you now have the evidence to do capture properly: in-repo, structured, automatic — the acme argument, sized for your team.
Tip
What you just proved: you can’t manage what you can’t see, and by 9am you’d already lost sight of yesterday. The record belongs with the repo, not in a chat history nobody scrolls back through — and when compliance eventually asks what your agents did, you’ll be the team that already has an answer.
I could not tell you, under any pressure, what I had for lunch three Tuesdays ago. I have never once needed to. It is possible I am the last unlogged system in this entire pipeline.