The autonomy slider

Debates about agent autonomy tend to collapse into two positions. One says supervision defeats the purpose: if a person has to approve everything, you have built an expensive form of autocomplete. The other says unsupervised agents are unacceptable in any consequential setting. Both are arguing about a single global setting for a decision that is not global.

Autonomy is not a property of a system. It is a property of a task within a system, and the same agent should sit at different points on the dial depending on what it is doing.

Nobody asks whether an employee is autonomous. They ask what that person can sign off on alone, and the answer is different for different things.

The three settings that matter

In practice most work sorts into three modes, and the design problem is deciding which applies where.

• Unattended. The action is reversible, low cost to get wrong, and high volume. Approval here is pure friction and its main effect is training the approver to click yes.
• Gated. The action is consequential or hard to reverse. The agent does the work and proposes it; a person commits it. The agent still absorbs the effort, which is where most of the value was anyway.
• Advisory. The judgment is genuinely contested or the stakes make delegation inappropriate. The agent assembles evidence, and the decision was never on the table.

The failure most teams make is picking one of these for the entire system. Set everything to gated and you get approval fatigue, which is worse than no approval because it manufactures the appearance of oversight. Set everything to unattended and the first serious incident sets the program back years.

The loop is not free, and the loss is measurable

There is a hidden premise in these debates: that a human in the loop is a dependable control. The best measurements we have say it is a real control but a lossy one, and how lossy it is depends on the task. The cleanest numbers come from clinical decision support, where a person reviews an automated recommendation and the outcome is recorded against a known answer. That is a good proxy for any setup where a person signs off on what a system proposes.

Oversight is a lossy control. Pooling four controlled studies, people reviewing automated advice reversed their own initially correct judgment to follow an erroneous recommendation between 6% and 11% of the time, and the miss rate climbed with task complexity and interruption (Goddard et al., 2012). A human in the loop lowers risk. It does not zero it, and how much it lowers risk is a property of the task, not of the word oversight.

Two things follow. Oversight has a failure rate, so keep a human in the loop is a setting with a cost rather than a guarantee. And that failure rate is not a constant: it moved with the task in the same experiments. Which is the whole argument. The right amount of human involvement is not a principle you assert once, it is a quantity that changes with the task and can be measured on it.

Reversibility is the real axis

The most useful question when placing a task is not how important it is. It is how expensive it is to undo. Sending an email is hard to reverse. Drafting one is free. Writing to a staging database is trivially recoverable. Writing to production is not. Posting publicly is permanent in a way that almost nothing internal is.

Sorting by reversibility rather than by importance produces a much better distribution, because it concentrates human attention on the small number of actions where attention actually changes the outcome. Most of what an agent does all day is reversible, and treating it as though it were not is how oversight gets spent in the wrong place.

The brake matters more than the gate

Preventive controls get most of the attention, and they are the less important half. What determines whether an incident is a footnote or a crisis is the response path: can you stop it, can you see what it did, can you undo it.

Concretely that means a way to halt a running agent immediately, a way to demote it to a mode where it can still be useful without being able to act, a complete record of what it touched, and a path to reverse it. A system with a strong brake can afford a looser gate. A system with a strong gate and no brake is more fragile than it looks, because the gate only helps with the failures you predicted.

Controls as a product surface

One more thing, which is easy to miss when you build this for engineers. If controls are only reachable through configuration, only engineers can set them, and the people who actually understand the risk of a given action are usually not engineers. The operations lead knows which approvals matter. The compliance officer knows what has to be reversible. If the dial is buried in a config file, their knowledge never reaches the system.

How we think about it

Controls on Recursiv are per-agent and per-tool rather than global: an agent can act freely in one area and require approval in another. There is a hard stop that halts execution immediately, and a chat-only mode that keeps an agent useful while removing its ability to act. Actions are logged in a record the agent does not control, and approvals are a first-class object rather than a convention.

The point of the dial is that it moves. Start conservative, measure, and open it up where the evidence says you can. That is a very ordinary way to extend trust to anything, and it works for the same reasons it always has.