Software Cost Does Not End When the Code Is Generated

My workplace does not have a weekly token budget yet. There is no allowance for each team to manage and no dashboard that turns every conversation with an agent into a financial decision.

We have, however, already held internal workshops on using simpler models, optimizing the skills we give agents, and declaring strict rules about what they should not do. The immediate goal is to reduce unnecessary operations. The deeper signal is more interesting: consumption is moving from a vendor detail to an architectural decision.

I can also see the next step. If agents are going to participate in more of the development process, asking them to respond more cheaply will not be enough. We will have to let them complete more work without human intervention. Otherwise, the cost of supervising each step will grow alongside the automation that was supposed to reduce it.

That creates a tension. Autonomy can save attention, but it can also let a wrong decision consume more tools, change more state, or travel farther before anyone detects it.

This is why I no longer think the cost of an agent can be summarized by its token price.

The real cost of an agent is the total cost of allowing it to act.

The visible bill is only one layer

When we compare models, we tend to begin with a convenient number: the price of one million tokens. That number helps us buy inference. It does not tell us what it costs to finish a job correctly.

An instruction such as “update this dependency” can trigger planning, repository search, documentation retrieval, tool calls, file changes, tests, corrections, and retries. Stripe explains the difference from a metering perspective: one user action can fan out into dozens or hundreds of model calls, tool invocations, and retrieval steps. The same instruction can produce different costs across runs.

That makes our usual unit incomplete. A token measures model consumption. It does not measure whether the result was accepted, whether a person had to reconstruct it, whether a test caught a regression, or whether an action had to be reversed.

In June 2026, Gartner published a forecast recommending that leaders model AI returns at four times current costs, partly because of more complex agent workflows and the potential end of vendor subsidies. I do not treat that multiplier as a truth that applies to every team. It is a forecast, not an observed result. The useful warning is that an initiative that works at the price of one response may stop working when we price the complete workflow.

I would call that broader unit the total cost of delegation:

I am not proposing this table as a standard financial formula. I am proposing it as a way to resist an illusion: reducing the first row does not automatically reduce the cost of the task.

The cheapest model can produce the most expensive outcome

Simpler models are a useful tool. A stable classification, a bounded transformation, or a search with explicit criteria does not always need the most capable model. Sending everything to the most expensive model is also a form of waste.

The bias appears when we confuse price per call with system efficiency.

A cheap model that needs five attempts, changes files outside its scope, and leaves an explanation someone must reconstruct can cost more than an expensive model that produces a verifiable result in one attempt. The relationship can reverse as well: using a frontier model for a deterministic task can consume much more without improving the decision.

There is no universal ranking. The question is which combination produces an accepted result at the lowest total cost and within a tolerable level of risk.

A recent Microsoft Research benchmark helps explain why the end of the workflow matters. DELEGATE-52 simulated long editing tasks across 52 domains and 19 models. By the end of those workflows, even frontier models had incorrectly changed an average of 25% of the evaluated content.

That is not a production failure rate, and it does not prove that one quarter of all delegated work will be wrong. It is a stress test designed around long workflows. Its value is in the mechanism it exposes: small errors can accumulate silently when a task spans many steps and files. If we measure only the final response or each call’s price, the cost of that degradation stays hidden until validation or repair.

A prompt is not a control plane

Strict rules inside a skill are useful. They can prevent irrelevant searches, choose deterministic tools first, and remind an agent not to modify files outside the requested scope. But a rule written for a model remains a probabilistic instruction.

It should not carry the same responsibility as a limit enforced by the system.

OWASP recommends against using system prompts to enforce critical controls. Privilege separation and authorization bounds should exist outside the model in a deterministic, auditable form. That distinction changes how I think about optimization:

• A skill or prompt guides behavior.
• A tool permission limits capability.
• A validator provides a correctness signal.
• A budget limits time, calls, money, and retries.
• A checkpoint or rollback creates a recovery path.
• An escalation rule returns ambiguity and accountability to a person.

Telling an agent “do not make more than ten calls” is not the same as stopping call eleven. Asking it not to access secrets is not the same as keeping those secrets outside its environment. An instruction can improve behavior; a boundary reduces what an error can reach.

Current products provide concrete examples of that architecture. The Codex sandbox keeps network access disabled by default and restricts writes to the current workspace. Anthropic describes a similar pattern: after encountering approval fatigue, it introduced an operating-system sandbox that reduced permission prompts by 84% while retaining filesystem and network boundaries.

These are vendor reports about their own products, not independent comparisons. They still illustrate a sound decision: automating inside a trusted boundary can be safer and less expensive than asking a person to confirm every step.

Autonomy should be earned

If every action needs approval, the human remains inside every iteration and the system’s capacity stays limited by their attention. If we remove every approval at once, we turn saved attention into expanded risk.

The alternative I find convincing is to treat autonomy as something a workflow must earn.

1. 01AssistanceThe agent reads, analyzes, and proposes. It changes no state.
2. 02SandboxIt executes inside an isolated, disposable environment.
3. 03ReversibleIt changes state only after recovery has been tested.
4. 04BoundedIt completes the workflow within explicit budgets and permissions.
5. 05By exceptionA person enters when a signal exceeds the contract.

This ladder does not measure how intelligent the model appears. It measures how prepared the system is to absorb its mistakes.

Before promoting a workflow, I would require repeatable acceptable results, observable failures, tested recovery, costs that remain within a predictable range, least privilege, stop conditions, and an accountable owner. Promotion belongs to the use case, not to the entire model. The same agent might work by exception when updating documentation and remain at the assistance level when changing authorization or moving money.

NIST notes that human-AI configurations can range from fully manual to fully autonomous and that the need for oversight depends on context. That is a useful correction to both extremes: not every task needs a person at every step, and not every technical capability deserves autonomy.

The governing question is no longer how much work the agent can do alone. It becomes:

How much error can the system contain and reverse before it needs a person?

An agent needs an execution contract

A prompt describes what to attempt. An execution contract describes the conditions under which continuing to attempt it is worthwhile.

Consider a fictional example: an agent responsible for updating dependencies and opening a pull request. Instead of granting broad access and hoping for a good result, I would define these terms first:

• Outcome: update only patch or minor versions within one package and explain lockfile changes.
• Budget: a maximum execution window, a bounded number of tool calls, and one retry after a known failure.
• Permissions: write only to an isolated branch; access only approved registries and documentation; never deploy or read credentials.
• Validation: inspect the diff, run the relevant tests, and record any installation-script change.
• Exit: stop on a major version, an unexpected file, a new network source, or a test that still fails after the allowed attempt.
• Recovery: discard the branch without affecting the primary workspace.
• Escalation and owner: present the evidence and uncertainty; a person decides whether the pull request deserves acceptance.

The specific numbers would depend on the repository and the value of the result. What matters is that they exist before execution. Stripe reaches a similar conclusion from the consumption side: limits, reservations, and circuit breakers must act while usage is being generated, not after the bill has recorded it.

The contract turns a broad intention into a system capable of stopping. It also exposes a decision that is usually hidden: how much we are willing to spend to prove that a result deserves trust.

Measure accepted outcomes, not activity

A token chart can help identify an anomaly. On its own, it cannot tell us whether an agent is creating value.

I would begin by connecting consumption to outcomes:

• total cost per accepted result;
• validation cost per accepted result;
• retries that did not change the conclusion;
• human time spent on exceptions and repair;
• runs stopped by the correct boundaries;
• work whose proof cost exceeded the value of completing it.

Not every signal needs to become money. Human time, blast radius, and reversibility can retain their own units. Forcing them into one number too early would create another form of false precision.

What matters is preserving the link between consumption and outcome. If a simpler model lowers the bill while keeping acceptance, validation, and repair within the same bounds, the optimization worked. If it lowers tokens but doubles retries or human attention, it merely moved the cost.

The same reasoning tells us where to automate more. A stable, reversible, and observable workflow can remove routine confirmations from a person’s day. An ambiguous, irreversible, or hard-to-evaluate workflow should spend more on evidence and escalate earlier.

The person leaves the loop after designing it

I believe allowing more work without human intervention will become necessary. Not because autonomy is always better, but because manually supervising every operation also carries a cost and can decay into approval theater.

Removing a person from the execution loop does not mean removing their decisions. It means moving those decisions earlier:

• decide which outcome is worth pursuing;
• bound what the agent may touch;
• establish how much it can spend and how long it can persist;
• define what counts as evidence;
• prepare the recovery path;
• and state when it must stop and ask for help.

The person then returns when the system encounters a real exception, not at every predictable step.

That is the difference between autonomy and abandonment. Autonomy embeds human decisions in the architecture. Abandonment expects a well-written instruction to replace budgets, permissions, validation, and recovery.

Software does not become cheap when it costs less to produce. It becomes sustainable when we know what it costs to trust it with a decision.