The Code-Frequency Spike Is the First Honest Metric for Agent Labor

Simon Willison went looking for a picture of what coding agents have done to his own output. He found it in the least glamorous place available: a GitHub code-frequency chart for his Datasette project. The chart tracks lines changed per week over the project's life, and near the end there is a spike he attributes directly to a specific cluster of model releases.

In his words, "The big spike in activity at the end aligns with Opus 4.8, GPT-5.5, Fable 5 and GPT-5.6 Sol" (datasette code-frequency chart on GitHub). No benchmark. No leaderboard. Just the shape of one maintainer's real work bending upward when the model tier changed.

This matters because it is one of the first pieces of coverage that treats agent productivity as a measured quantity rather than a promise. Every model launch arrives wrapped in eval scores and demo videos. Almost none of them show you the output. Willison's chart does, and the same week produced supporting artifacts: a working Doom clone built inside SQLite and a tidy GitHub Actions caching recipe, both shipped fast, both the kind of side-quest that agent labor makes cheap enough to actually finish.

The reason to care is not that models got better. Everyone knew that. The reason to care is where the improvement shows up. It shows up as raw code volume from a single, already-productive human. That is a specific claim about the shape of agent labor, and it is worth taking apart.

Start with a Wardley Mapping question: where on the value chain did the improvement land? A model is a component. A coding agent is a harness that wires that component into a repository, a test suite, and a maintainer's intent. The benchmark measures the component. The code-frequency chart measures the harness doing work.

This is the Harness Hypothesis in its cleanest form. The value in AI isn't in the model; it's in the harness that connects the model to the world. Willison's spike is not evidence that Opus 4.8 is smart in the abstract. It is evidence that the harness around it finally converts model capability into merged, tested, shipped changes at a higher rate. The model moved. The conversion rate moved more.

The distinction is not pedantic. It tells you which layer captures the gains. If the benchmark score were the whole story, the value would accrue to whoever trains the best model. But a code-frequency spike is a property of the maintainer plus the harness plus the model, and the maintainer is the one who owns the repository, the taste, and the merge button. The model is the commodity input. The harness is the thing doing the aggregating.

Hold that frame. It reframes the entire "which model won" conversation, because the honest metric for agent labor is not how the model scores. It is how much work a human can now push through the pipe they already controlled.

The reflex when a new frontier model ships is to ask which one is best. That question is now close to meaningless for the reader doing real work, and the code-frequency chart shows why.

Willison did not attribute his spike to one model. He named four in the same breath: Opus 4.8, GPT-5.5, Fable 5, and GPT-5.6 Sol (datasette code-frequency chart). That is not sloppiness. It is the actual experience of a power user in July 2026. The models are close enough, and swap cheaply enough inside the harness, that the productivity gain reads as a tier event rather than a vendor event. The maintainer felt a step change in what a good agent could do. He did not feel a Claude event or an OpenAI event.

This is Aggregation Theory playing out at the individual level. Platforms win by aggregating demand and commoditizing supply. Here the demand is the maintainer's intent, the queue of issues and features he wants shipped. The supply is model capability, and it is being commoditized in real time, four vendors deep, interchangeable inside a single workflow. The reader who owns the demand relationship, the repository and the judgment about what to merge, captures the value regardless of which supplier's weights ran that afternoon.

Now look at where the volume actually landed. It did not land as a startup shipping a new product. It landed as an existing maintainer of an existing project doing more of what he already did. The code-frequency chart is a chart of amplification, not substitution. Agent labor in this data point flows to people who already had leverage: taste, a codebase, a merge button, and the ability to tell good output from plausible output.

That has an uncomfortable corollary. The productivity gain is concentrating, not democratizing. It rewards the maintainer who can review a large diff fast and reject the wrong parts. The bottleneck moved from writing code to judging code, and judgment does not scale by buying more tokens. The same week's SQLite Doom clone is the tell. That project is pure exploration, the kind of thing a person does when the marginal cost of trying a weird idea drops toward zero. When execution gets cheap, the scarce input becomes knowing which weird ideas are worth executing.

The ecosystem numbers rhyme with this. Codex usage reportedly climbed more than 10x in six months to roughly 7 million users, with a million added in about a day (Latent Space's AINews recap). That is adoption of the harness, not affection for a specific model. People are pouring into the layer that converts capability into merged work, and the code-frequency spike is what that conversion looks like from inside one repository.

So the reader's takeaway is not "upgrade to Opus 4.8." It is: the model tier is now a commodity input, and your leverage lives in the harness you drive and the judgment you bring to its output. The chart that matters is not the leaderboard. It is your own output curve, and whether it bends when the tier moves.

The obvious objection: one maintainer's chart is an anecdote, not data. Correlation with a model release is not causation, and Willison himself hedged, calling it the best illustration he'd "found so far." A single spike could reflect a burst of unrelated work, a big refactor, a slow month before it. Fair.

Take it seriously. A code-frequency chart measures lines changed, which is a famously bad proxy for value. More code can mean more debt. An agent that generates volume is not obviously producing better software, and a spike could be the harness making it easy to churn.

But the argument here does not rest on the spike being good. It rests on the spike being real and being attributed, by the person who lived it, to a model tier rather than a model brand. That attribution is the load-bearing claim, and it is corroborated by the broader adoption pattern in the Codex numbers. Whether the extra output is net-positive is a separate, important question. The point stands: the honest metric for agent labor is output measured on a real project, and that metric is telling you the gains accrue to the harness and the human who drives it, not to the model on the label.