Dyna-2.1's 99.4% Success Rate Is the First Real Test of Agent Reliability Outside Software

Start with a distinction you will use for the rest of this piece. Capability asks: can the system do the task at least once? Reliability asks: can you stop watching it?

Think about hiring a new assistant. On day one you check whether they can file an expense report at all. That is capability. By month three the question is whether you can hand them the pile on Friday and not think about it until Monday. That is reliability, and it is the only version that actually saves you time.

Robotics demos have historically answered the first question. A robot folds a shirt, the clip gets edited, the internet applauds. What you never learn is how many takes it took. Ma's talk, as summarized by Latent Space, aims squarely at the second question. The model's title promises hour-long autonomy, which means the unit of measurement is time without human intervention, not a single successful motion.

Here is the comparison laid out plainly:

• Capability demo: one success, unknown attempts, often edited.
• Reliability claim: a success rate across many attempts, implying a defined failure condition.
• Autonomy claim: sustained operation over a long horizon without a human resetting the scene.

Dyna is claiming the last two. That is why the 99.4% figure matters more than the footage, even though the footage is what will circulate.

One small signal in Dyna's favor: the episode includes a live demo of the Taku robot rather than relying solely on a highlight reel. A live run is harder to fake than a cut. Don't overweight it, though. A live demo is still one run in front of a friendly audience. It tells you the system works; it does not tell you the failure rate. Only the reliability number does that, which is why the next section is about reading it carefully.

Why should you, as someone who runs software agents rather than robots, care? Because the reliability question is identical across both worlds. Your agent that triages inbox email and Dyna's robot doing a long physical task face the same core problem: errors compound over a long horizon, and every intervention you need erodes the value of automating the job at all. Physical systems just make that problem impossible to hide.

Here is the gotcha that trips almost everyone who reads a reliability number: you have to know the denominator. What counts as one trial? The published episode summary cuts off right as it begins describing what the 99.4% applies to, so this is precisely the detail you'll want to chase in the full talk before you repeat the figure.

Why does it matter so much? Run the arithmetic yourself.

• If 99.4% is the success rate per step, and a task has 100 steps, the chance of finishing the whole task cleanly is 0.994 multiplied by itself 100 times. That comes to roughly 55%. A coin flip with extra steps.
• If 99.4% is the success rate per complete long-horizon task, then out of 1,000 hour-long runs you'd expect about six failures. That is a deployable system for many jobs.

Same number. Wildly different products. Given that Ma's framing centers on long-horizon autonomy, the second reading is the more natural one, but you should treat that as a reasonable inference and confirm it against the source.

When you evaluate any reliability claim, robotic or software, ask three questions in this order:

• What is one trial? A single grasp, a full task, or an hour of operation?
• What counts as failure? This is where numbers quietly inflate. If a human nudges the object back into place and the run still counts as a success, the system is not autonomous; it is assisted. Look for whether interventions are counted.
• Where was it measured? A lab cell the team has run a thousand times is not your warehouse, your kitchen, or your customer's messy shared drive.

You'll notice these are exactly the questions you should be asking of software agent benchmarks too. A coding agent that "passes 95% of tasks" may be passing 95% of short, well-specified tasks with clean test suites. The compounding math is the same. The only difference is that software failures are cheap to retry, so vendors rarely feel pressure to define the denominator precisely.

Robots don't get that luxury, which is exactly what makes this claim useful. A physical agent that fails 0.6% of the time leaves evidence: a mess, a stop, a person walking over. If Dyna's number holds up under the three questions above, it is a harder-won 99.4% than most software figures you will see this year.

The 99.4% did not appear from nowhere. It sits on top of Dyna-2, which Latent Space describes as a world-action model pre-trained on one million hours of human video. Dyna also claimed several new scaling laws on human data across four orders of magnitude.

Let's apply the Feynman Technique and explain why that matters as if you'd never thought about robots before.

Language models got good because there was an enormous pile of text on the internet to learn from. Pre-train on the pile, then teach the model specific jobs afterward. Robotics never had an equivalent pile. Robot data has to be collected by running robots, which is slow, expensive, and limited to whatever hardware you own. That has been the field's bottleneck for years.

Human video is the closest thing to an internet-scale pile for physical work. People film themselves cooking, assembling, cleaning, repairing. If a model can learn how the physical world behaves from watching humans act in it, robot-specific data becomes the fine-tuning step rather than the whole foundation. That is the exact structure that worked for text.

A scaling law is the claim that performance improves predictably as you add data or compute. Four orders of magnitude is a ten-thousand-fold span. Showing a consistent trend across that range is what lets a lab say "more data will keep helping" with a straight face, rather than hoping.

Now for the warning you need before you get excited. These are Dyna's claims, presented by Dyna. The source pack contains no independent replication. Scaling laws in robotics are newer and less tested than in language, and a curve that holds across four orders of magnitude can still bend at the fifth. Treat the claim as a strong hypothesis with a credible track record behind its software cousin, not as settled physics.

What you can take away with more confidence is the shape of Dyna's strategy: release a broad pre-trained model, then follow it with an agent product (Dyna-2.1) that wraps that model in everything needed to finish real tasks reliably. That two-step pattern, foundation model first and operational agent second, closely resembles how the software agent stack matured. It suggests the same economics may apply: the model gets you capability, and the surrounding system gets you the reliability number that someone will actually pay for.

To see why a physical result matters so much, look at what software agents have done in the last few months, and at the critique that has followed them.

The acceleration is real. Simon Willison flagged Anthropic's account of redesigning source control, in which the company built its own alternative to Git because the rate of code changes had exploded. In May 2026, Anthropic averaged around 2,500 merged changes per day. By September, it was merging over 30,000 per day. That is a twelvefold jump in four months, and the tooling literally broke under it.

Meanwhile, The Sequence describes AI agents helping build AI, bringing recursive self-improvement into the research lab, with OpenAI's automated research intern milestone as the reference case. The Sequence's reading is measured: the milestone shows how much agents can execute, and also how much scientific judgment still sits with humans.

Here is the critique you'll hear from skeptics, and it is a fair one. Software is a forgiving environment. Code can be tested, reverted, and re-run for pennies. A software agent that fails a fifth of the time can still look productive if a harness catches the failures and retries. So when software agents post big reliability or throughput numbers, you can never fully separate genuine competence from the cheapness of undo.

Physical work removes that escape hatch. There is no git revert for a knocked-over container. Retries cost wall-clock time, wear on hardware, and sometimes a human walking across a room. That is why a physical reliability number functions as an out-of-sample test for the whole agent thesis. If the recipe of scale plus a reliability-focused harness only worked because software forgives mistakes, it should fail when applied to bodies.

Dyna's result suggests it doesn't fail. Be careful with how far you push that. This is one company, one model family, and one headline number whose exact scope you still need to confirm. It is not proof that every physical task is now automatable. What it does offer is the first serious evidence that the dynamics driving software agents are not an artifact of software's forgiving nature. That is a categorical shift in what the evidence can support, even if it is early.

If you take one framework from this piece, take the Harness Hypothesis: the value in AI isn't in the model; it's in the harness that connects the model to the world. Dyna's emphasis on reliability is, read this way, an emphasis on harness.

The clearest small-scale illustration in this week's sources comes from software. Geoffrey Huntley, quoted by Simon Willison, describes a problem every agent user has hit: his coding agent kept bypassing his diagnostics library. He could have updated the agent's instruction file, but as he puts it, "there's still a delta, and it'll drift." Instead he wrote a pre-commit hook that detects the bad pattern, fails the commit, and injects a prompt telling the agent to use the right library. His result: "All of a sudden it's ten out of ten."

Notice what he did not do. He did not switch models. He did not write longer instructions. He added a check that detects deviation and feeds back a correction. That is the move that turns an agent that works most of the time into one that works every time.

The physical analog is direct. A long-horizon robot task fails in two ways: the policy makes a mistake, or the world changes underneath it. Getting from a strong demo to 99.4% almost certainly requires the robot to notice when something went wrong and recover without a human. Dyna's specific recovery mechanisms aren't described in the source excerpt, so treat that as inference from the framing rather than a documented feature. But it is hard to imagine hour-long autonomy without detect-and-recover loops doing much of the work.

Here is how you apply this to your own agents today, whether you run OpenClaw, Hermes, or Claude Managed Agents:

• Stop adding instructions when the agent drifts. Instructions erode over long sessions, exactly as Huntley warns.
• Add a check at the point of output. Before the email sends, before the file saves, before the commit lands, verify the one thing that keeps going wrong.
• Make the check feed back, not just block. A failure that tells the agent what to fix is a recovery; a failure that just stops is an interruption you'll have to handle yourself.

The gotcha: checks only catch failures you've anticipated. The 0.6% that remains in any good system tends to be the failures nobody wrote a check for. That residue is what the final section is about.

The Scout's framing for this story was blunt: the question is no longer whether agents can be autonomous, but whether companies have the guts to deploy them. That is half right. Guts matter. Calibration matters more.

Reach for the Autonomy Spectrum: deployments range from copilot (human approves every action) to full autonomy (human reviews outcomes, if at all). Most failures come from deploying at the wrong point, not from deploying at all. A 99.4% system placed where a 0.6% failure is catastrophic is a bad deployment. The same system placed where failures are cheap and visible is a great one.

Even the frontier labs behave this way. The Sequence's account of OpenAI's research intern includes what happened when the lab pulled the safety brake, and frames the open question as what evidence would signal a transition from research assistant to researcher. That is an organization with every incentive to push autonomy, still deliberately holding judgment in human hands. You should treat that as a template, not a timidity.

If you're tempted by the zero human company pitch, here is a step-by-step way to move an agent, physical or software, along the spectrum without getting burned:

• Step 1: Pick the task where failure is cheapest. Not the most valuable task. The one where a miss costs a retry, not a customer.
• Step 2: Count interventions, not just successes. Every time a human touches the run, log it. That number is your real autonomy rate, and it's usually worse than the success rate.
• Step 3: Move one notch at a time. From approve-every-action to approve-batches, from batches to spot-checks. Each notch should earn its way with data.
• Step 4: Decide your brake before you need it. Write down, in advance, what failure rate or failure type sends the agent back a notch. Deciding during an incident is how teams talk themselves into staying exposed.

The mistake to avoid is reading Dyna's number as permission. It is evidence that the reliability ceiling is higher than skeptics assumed, in a domain where reliability is hard to fake. That should make you more willing to experiment with autonomy. It should not make you skip the steps that tell you where your particular task sits.

The agent category spent two years asking whether autonomous labor was possible. A robot with hour-long autonomy and a published failure rate changes the question. Now you have to answer the harder one: for this task, in this environment, with these consequences, how much autonomy has the system actually earned?