The plan cost 1.47x the whole job, and made the job 29% cheaper

Plan mode is recommended as the cheaper way to work: it “is more cost-efficient than pure auto-accept on open-ended prompts”, because a model that does not pause for approval “takes more actions per task, increasing both tool call count and session length”.

The first attempt to measure that had a defect. Edits were auto-accepted but bash was not, and plan mode gets read-only tools without asking, so the plan arm could run the test suite and the auto-accept arm could not. One arm was doing empirical diagnosis the other was structurally denied.

This is the same experiment with --allowedTools Bash on every call in both arms, eight runs per arm, and --output-format stream-json so every tool call is recorded and no turn count has to be taken on trust. Removing the confound widened the gap.

Medians, 8 runs per arm	Auto-accept	Plan then execute	Ratio
Tokens	114,268	230,152	2.01x
Dollars	$0.1493	$0.3250	2.18x
Turns	6	11.5	1.92x
Tool calls	5	9.5	1.90x
Succeeded	8 of 8	8 of 8	—

No overlap on any of the four. The worst auto-accept run beat the best plan-then-execute run every time: 114,371 tokens against 216,460, $0.1510 against $0.2991, 6 turns against 10, 5 tool calls against 8.

Both halves of the finding are now separated

The interesting question is which of the plan arm’s two calls the extra sits in, and the answer is not the one you would guess.

Per call, medians	Dollars	Tokens	Turns	Tool calls
Auto-accept, does the whole job	$0.1493	114,268	6	5
Plan call	$0.2192	138,072	6.5	5.5
Execute call, plan in hand	$0.1057	92,284	5	4

Planning made the execution cheaper, 8 of 8, with no overlap. The most expensive execute call, $0.1236, is below the cheapest bare auto-accept run, $0.1488. At the median that is 29% off, and it is the advice’s own mechanism doing exactly what the advice says it does: with the thinking already done, the model takes fewer actions, 4 tool calls against 5.

The plan cost more than the whole job, 8 of 8, with no overlap. The cheapest plan call, $0.1999, is above the most expensive complete auto-accept run, $0.1510. The saving planning buys is $0.0436. The plan that buys it costs $0.2192, five times as much.

So the mechanism is real and the arithmetic still loses. That is a more useful result than the mechanism simply failing, because it locates the break-even: planning first pays only when the execution it improves is big enough that 29% of it exceeds the cost of one planning call.

The claim is about tool calls, so here are the tool calls

The advice’s stated reason is that auto-accept “takes more actions per task, increasing tool call count”. Turn counts are a proxy for that. The action record is not.

Auto-accept ran the identical five-tool sequence in all eight runs: Bash, Read, Read, Edit, Bash. It lists the directory, reads both files, makes the one-line fix, then runs the suite once to verify. Eight runs, one sequence, six turns every time, and a total context spread of 126 tokens across all eight. There is no version of this run that takes more actions than the workflow it is being compared against: five against nine and a half, separated.

Within the plan workflow the picture splits the same way as the cost. The execute call is the leanest thing in the dataset at 4 tool calls, and the plan call is the fattest at 5.5, before the execute call has even started.

Why a plan call costs more than doing the job

Four of the eight plan calls used exactly five tool calls, the same number as every auto-accept run, on similar context, 125,008 against 114,268. They still cost $0.1999 to $0.2283 against $0.1488 to $0.1510. Roughly 1.4x for the same number of actions.

The difference is what comes out. A plan call emitted a median 1,949 output tokens against auto-accept’s 873, because prose is the deliverable rather than a one-line edit, and output is the expensive kind of token. Plan mode is not costly because it does more; it is costly because what it produces is text.

The obvious objection, and why it does not survive

Seven of the eight plan calls spent a tool call on ToolSearch, and two spent two. They were hunting for ExitPlanMode, plan mode’s own way of handing a plan over, and none of them found it. All eight replies report it unavailable or disabled, including the one call whose action record shows it invoked, which got back that the tool “is disabled in this session”. One reply adds that it was “not in my tool list, and not resolvable via ToolSearch”. All eight fell back to writing the plan to a file. So a share of the plan call’s cost is this setup rather than planning.

It is not enough to matter. In the plan phase each additional tool call costs about 30,000 tokens, from 125,008 at five to 153,396 at six to 185,543 at seven, and about $0.046. Subtract one entirely from the median plan call and it still costs $0.173 against the auto-accept run’s $0.1493. The stronger version needs no arithmetic at all: the four plan calls that used the same five tool calls as auto-accept still cost more than every auto-accept run in the dataset.

What this does not settle

The advice says “open-ended prompts”, and this task is not one. It is one bug in one small module with a test suite defining success exactly. The claimed benefit of planning is that it prevents expensive wrong turns, and a task with a definite right answer gives it very little room to do that. On well-specified work, planning first was more expensive with no success benefit. Whether it pays for itself on open-ended work has since been measured too: it cost 2.27x there, with no overlap, and the execution saving above shrank to about 3%.

Equal permissions did not produce equal behaviour. The fix removed the structural cause of the earlier asymmetry, not the asymmetry. With bash granted to both arms, the auto-accept arm still never ran the suite before editing: in 8 of 8 runs its first command is a directory listing and its only pytest run comes after the edit. Six of the eight plan calls ran the suite before proposing anything, and quote the failure output back. So the plan call is still buying a round of empirical diagnosis the auto-accept run never buys, and the execute call inherits it for free. That is the transfer the per-call split is measuring, which means the 1.47x and the 29% are two sides of one movement and both are upper bounds on their own effect. The workflow total, which is what you actually pay, does not care how the two calls divide it.

Two things changed against the earlier run, not one. Bash was granted in both arms, and the output format moved from json to stream-json to capture the action record. The second should not affect what the model does, but it was not held fixed, so read the comparison between the two corpora as indicative. It is decisive on one point: the earlier post said its permission asymmetry made 1.69x and 1.95x upper bounds on the true ratio. That claim is withdrawn there and here. Removing the asymmetry raised the ratio. Denial did not truncate the auto-accept arm, it padded it, with a median 9 turns and 185,160 tokens spent partly on retrying blocked commands against a uniform 6 turns and 114,268 once the commands were allowed. Within this corpus every arm was run identically.

No wall-clock claim is made. Other work ran on this machine during the second batch, and two auto-accept runs took 31.5s and 55.9s against 16.5 to 20.8 for the rest. Their token counts sit inside the same 126-token spread as every other auto-accept run, which is a useful reminder that wall clock measures the machine and tokens measure the work.

Controls

The bash grant was proven to have arrived, not merely passed. --allowedTools Bash is asserted onto the command line, and then every call has to record at least one Bash tool use in its own action record. 24 of 24 calls did, one per auto-accept run and two per plan-then-execute run. No reply in the corpus reports a denial, against ten of ten in the earlier run.

A prose heuristic scanning replies for denial wording fires on four runs, and it is wrong on all four: the only match is a plan reply saying “Approve and I’ll apply it”, which is plan mode asking for approval of the plan. It is published unchanged in the data and scores nothing, because the mechanical control is the one that counts.

The fixture discriminates, proven before any model call. Unfixed, the suite fails. With the correct non-mutating fix, it passes. With the obvious in-place sort, which makes the failing test pass, it still fails because it breaks a different test.

The test suite was checksummed before and after every run, and the pass was determined by the harness running pytest, not by the model reporting one. All 16 runs left test_work.py byte-identical and all 16 passed.

Two batches of four rounds, run separately, and they agree. Auto-accept came in at 114,245 to 114,256 tokens in the first and 114,280 to 114,371 in the second. Both batches are published with their batch label so they can be separated again.

Isolation on every call, --setting-sources project, asserted. Tokens from modelUsage, not .usage.

Every run, with its full tool-call record, is published in plan-mode-equal-permissions.json.