Three layers, three refusals

I asked a coding agent to remember my name.

It wrote the fact into its own memory directory. Not the memory I had just installed for it. Its own.

That one line of behaviour cost me an architecture decision record. The lesson was not “the agent is daft”. It was that guidance has to arrive at the exact moment the tool’s built-in habit would otherwise win, or the habit wins every time.

Everything that matters about agentic development lives around the model, and most of it comes down to boundaries: who is allowed to decide what, and when.

I’ve been building three small tools to find those boundaries. ynh is a harness. ynm is a memory. ynf is a factory loop that drives the other two. The code is less interesting than the decision records, which have settled on one idea: each layer refuses the other two’s job. That is the whole series. The rest is receipts, and you don’t need my tools to use any of it.

Info

These are experiments, not products. They exist to prove the concepts to me. Most of what they do is available, in part, from vendors already: Anthropic’s memory tool, Letta, the Ralph loop, Factory, Codex’s harness conventions. This series is the exercise section: making sure I understand what is going on under the hood before I evaluate, or adopt, somebody else’s implementation. The point is to be able to tell, when a vendor says “memory” or “factory”, what they have left out.

The shape, in five lines

Version the harness in git, and make its checks discoverable by a machine, not just by a reader.
Keep memory append-only, outside the working tree, with a binary line between what may leave your machine and what may not.
Put the loop between runs in something that can die between any two steps and come back, re-probes the facts before it decides, and never lets the model touch control flow.
Give the agent a sandbox and no credentials. Let the operator push.
Let a human merge.

Nothing in that list is a new piece of infrastructure. The harness is a git package. The memory is git notes. The factory’s output is a draft pull request. The repository you already have is the store, its remote is the sync server, and the pull request you already review is the output. That is not an accident of the tools. It is the point of them.

                 ┌───────────────────────────┐
  ticket ──────▶ │  FACTORY (ynf)  decides   │ ── draft PR ──▶  HUMAN
                 │  never guides, never      │                  merges
                 │  reads memory to a prompt │
                 └──────┬─────────────┬──────┘
           runs it      │             │  writes every failure
                        ▼             ▼
       ┌───────────────────┐   ┌───────────────────┐
       │ HARNESS (ynh)     │   │ MEMORY (ynm)      │
       │ guides            │──▶│ remembers         │
       │ one bounded loop  │   │ advises, never    │
       │ no keys, no push  │   │ decides           │
       └─────────┬─────────┘   └───────────────────┘
                 │ sensors check
  ───────────────▼──────────────────────────────────────
   ACCEPTANCE is yours: what "done" means, as commands

Only one arrow ends at a person, and it carries a pull request.

The harness guides

Thoughtworks, OpenAI and Anthropic have landed on the same definition: a harness is everything in a coding agent except the model, and weak results are usually harness problems. I’ll take that as read. The part the definitions skip is that a harness is configuration, configuration drifts, and three vendors’ formats drift three ways.

So a harness here is a versioned bundle in a git repository, and the one non-obvious decision is that it declares its own feedback. Linters, tests and type checks are listed as sensors with a thin pass or fail contract. Not because prose couldn’t describe them. Because anything that wants to loop against the harness has no way to discover what it exposes unless it is declared.

The harness runs one bounded loop: plan, act, check the sensors, feed the failures back, stop when it converges or hits a budget. It refuses everything between runs. And it refuses to claim containment. Its answer to “is this safe” is “run me inside a container you configured”, never “I provide isolation”. A control that is declared, believed and absent is worse than none.

The memory remembers, and never steers

Memory is advisory only. Nothing reads a memory and takes a control-flow decision from it. That is the rule I’d defend hardest, and it is the one most memory products are built to break, by design and in good faith.

The rest follows from wanting memory to be boring. The repository is already cloned, backed up and access-controlled, so memory lives there, as git notes hung off the root commit where no rebase touches them. Personal memory stays on your machine. Shared memory travels with git push. No model on the write path. Two physical stores, not two flags, because tools that keep everything in one place and filter on the way out are one careless push away from publishing all of it.

The expensive part of memory is not storage. It is judgement: is this the same as that, does this keep happening. A generative model asked “are these the same?” answers fluently, and is confidently wrong often enough to matter. So judgement is a separate, calibrated thing, and it will not merge anything on a guess.

The factory decides, and never merges

Here’s the layer every “AI software factory” post skips. The inner loop lasts minutes. Nobody owns what happens between runs: noticing a ticket is ready, watching the pull request, reacting to red CI, deciding whether to retry, ask a human, or stop.

The factory is that operator. A loop around a loop, with the shape of a Kubernetes controller rather than a chatbot. None of that is new to anyone who has written a controller. What is new is pointing it at a thing that can be talked into things. The useful question is not whether the agent can be talked into something, but what it can do if it is. So:

Webhooks are hints. Facts are re-probed before every decision.
The decider is a pure function. Models make changes, not control flow.
Ticket text is untrusted input. It never reaches a shell.
The agent never holds write credentials. The factory commits, pushes and opens the pull request.
On your own machine, a repository’s policy cannot grant the agent permission to edit. Only the person at the terminal can.
The factory never merges. A human always merges.

How the three fit

Each layer refuses the other two’s job. That is what makes them composable, and it is the bit I’d keep even if you threw the tools away.

The factory writes every failure to memory, with what the run actually reported. It never reads memory back into an agent’s task, so a retried ticket gets the same prompt every time. Three failures with the same signature become a reflective memory on their own. But the factory keeps its own plain counters. Guards read counters; people and agents read memory. Memory carries the explanation; counters carry the decision.

And none of it requires the others. Without a harness, the runner is any command, even gofmt. Without memory, failures queue until it comes back. The outer loop does not care what the inner loop is.

Receipts, since this is built on them: writing the tutorial that goes with this series found real bugs in all three tools. The best one was in the inner loop: a run capped at one turn quietly finished a second turn before it exited. It is fixed now. That is what writing things down is for.

The Debrief

Lights-out plants arrived last in manufacturing, after the gates, the standard work and the stop-the-line cord. Any programme that reaches for the autonomous end first is repeating an experiment whose result is already documented, in another industry, at considerable expense.

  4 ┌──────────────────────────────────────────────────────────┐
    │ UNATTENDED LANES  in a container; a person only merges   │
    └──────────────────────────────────────────────────────────┘
         ▲  earned by: a sandbox that holds
  3 ┌──────────────────────────────────────────────────────────┐
    │ ATTENDED RUNS  a person starts it and grants edits       │
    └──────────────────────────────────────────────────────────┘
         ▲  earned by: measured yield
  2 ┌──────────────────────────────────────────────────────────┐
    │ SHADOW MODE  closed tickets only, proposes nothing       │
    └──────────────────────────────────────────────────────────┘
         ▲  start here
  1 ┌──────────────────────────────────────────────────────────┐
    │ BY HAND  a person runs the agent                         │
    └──────────────────────────────────────────────────────────┘

You don’t skip rungs. Each step up is paid for with evidence.

TL;DR

Agents get safer to stack when every layer has one job and refuses the other two. This series tests that with three small tools: ynh, a harness that guides an agent; ynm, a memory that remembers; and ynf, a factory loop that decides what happens between runs. A human merges.

None of it needs new infrastructure. The harness is a git package, memory is git notes, and the factory’s output is a draft pull request you already know how to review. The tools are experiments, not products. The point is to understand what is under the hood before you evaluate somebody else’s.

The ideas

The harness declares its checks as sensors a machine can find, runs one bounded loop, and never claims to provide isolation.
Memory only advises. It lives in git outside your working tree, with personal and shared memory in separate stores.
The factory treats webhooks as hints, re-checks the facts before it decides, never gives the agent a credential, and never merges.
Each layer works without the others. Guards read counters. People and agents read memory.

What to take away

Separate guiding, remembering and deciding before you buy anything. Most products blur at least two.
Ask a memory product whether anything takes a decision from what it remembers, and a factory product what the agent can do if it is talked into something.
Start in shadow mode, on tickets that are already closed.

Where to start

The rest of this series builds all of it, one notch at a time, and every part runs. Each part ends with a Tutorial tab: install the tools, reset anything left over, then copy and paste your way through it.

Part 1: the harness guides20 min

Declared sensors, a ratchet, a definition of done the agent cannot see, a budget that holds, and a request for safety that refuses rather than pretends.

Parts 2 and 3: memory, then the factory on its own45 min

Memory that never touches your code, then a loop that turns a ticket into a pull request with no agent in it at all.

Parts 4 to 6: together, then taken apart1 hour

The factory drives the harness, writes its failures to memory, and keeps working when you take either away.

The harness guides. The memory remembers. The factory decides. A human merges.