Planning made the execution cheaper, and cost more than the job it was planning

The advice is specific and it is repeated: use plan mode, because it “is more cost-efficient than pure auto-accept on open-ended prompts”, since a model that does not pause for approval “takes more actions per task, increasing both tool call count and session length”.

That is a claim about turns and tokens, so it can be measured. On a task with a definite right answer, the mechanism worked and the workflow still lost money.

	Auto-accept	Plan then execute	Ratio
Tokens	185,160	313,308	1.69x
Dollars	$0.2062	$0.4014	1.95x
Turns	9	15	1.67x
Wall clock	35.4s	69.3s	1.96x
Succeeded	4 of 4	4 of 4	—

Four runs per arm, medians shown.

No overlap on any of the four

The arms do not merely differ on average. On every measure, the worst auto-accept run beat the best plan-then-execute run:

• Tokens: 209,378 against 264,464
• Dollars: $0.2311 against $0.3514
• Turns: 10 against 13
• Wall clock: 38.1s against 55.1s

Complete separation on all four, with four runs each.

Splitting the workflow is where the finding is

The plan arm is two calls, so the interesting question is which of them the extra sits in. The obvious guess is the second one, because it carries the plan text as additional input. That guess is wrong.

Per call, medians	Dollars	Tokens	Turns
Auto-accept, does the whole job	$0.2062	185,160	9
Plan call	$0.2301	150,236	7
Execute call, plan in hand	$0.1685	162,954	8

Planning made the execution cheaper. The execute call came in under a bare auto-accept run in all four rounds, on dollars every time: $0.1507 against $0.1913, $0.1919 against $0.2311, $0.1402 against $0.2081, $0.1862 against $0.2043. That is the advice’s own mechanism, and it is doing what the advice says it does. Tokens agree in three rounds of four, and turns lean the same way, 8 against 9, though those ranges overlap and only the dollar direction is unanimous.

The plan cannot pay for it. The saving is $0.0377 per run at the median. The plan call that produced the saving costs $0.2301, which is 1.12x an entire auto-accept run on its own, and it beat that whole run in all four rounds: $0.2247 against $0.1913, $0.2356 against $0.2311, $0.2112 against $0.2081, $0.3177 against $0.2043. Two of those four margins are under 2%, so read the direction rather than the size. The size that matters is the other one: the plan costs about six times the saving it buys.

So the mechanism is real and the arithmetic still goes the wrong way. That is a more useful result than the mechanism simply failing, because it says where the break-even is. Planning first pays only when the execution it improves is large enough that a fifth of it exceeds the cost of the planning call.

Why the dollar gap is wider than the token gap

Tokens 1.69x, dollars 1.95x. The gap between those two ratios is not rounding. The plan call is a more expensive kind of call per token: it uses fewer context tokens than an auto-accept run, 150,236 against 185,160, and fewer turns, 7 against 9, and still costs more.

Across all fifteen calls published here, including the pilot, price per million context tokens separates perfectly. No plan call is below $1.49 and none is above $2.00. No execute-phase call, in either arm, is above $1.19. The plan calls also emit the most output, median 2,119 tokens against 1,557 for auto-accept, and output is priced far above cached input. This corpus publishes context as one summed figure rather than split into fresh, cache-write and cache-read, so that is an association with an obvious candidate cause, not a decomposition.

The practical form of it: the penalty scales with how many planning calls you make, not with how many tokens you move.

Why plan mode is two calls

Plan mode cannot finish a task on its own. Asked to fix the bug with --permission-mode plan, the model produces a plan and stops: four of the five plan replies say ExitPlanMode is not available in the session. Three of them wrote the plan out to a file of their own, outside the working tree. In every plan-then-execute round the fix arrives only with the second call, whose reply still describes replacing the original scores.sort(reverse=True), so the plan call left work.py alone. Only test_work.py was checksummed, so that last point rests on the execute replies rather than on a control.

So the fair comparison is the workflow the advice actually recommends: plan, then execute carrying that plan. The cost of that arm is the sum of both calls, because that is what a user pays.

What this does not settle

The two arms did not run under the same permissions, and that is the largest limit here. Edits were auto-accepted; bash was not. All ten calls made outside plan mode, meaning every auto-accept run and every execute phase, report their attempts to run the suite being denied by the permission layer. None of the five plan calls reports a denial, and two of them show the suite actually being run: one quotes pytest output, assert [9, 7, 3, 1] == [3, 9, 1, 7] and 1 failed, 2 passed, and the pilot’s plan says the diagnosis was “confirmed by running the suite”. So the plan phase got a round of empirical diagnosis that the auto-accept arm was structurally denied, which inflates the plan arm and truncates the other one.

That limit was first published with the reasoning that both effects push the ratio up, so 1.69x and 1.95x were upper bounds. That claim is withdrawn. The re-run with bash granted in both arms came in at 2.01x tokens and 2.18x dollars, both larger. Denial did not truncate the auto-accept arm, it padded it: blocked, the arm spent a median 9 turns and 185,160 tokens partly on retrying commands it could not run, and once the commands were allowed it settled at a uniform 6 turns and 114,268 tokens across eight runs. Read the figures on this page as this corpus’s own, not as a bound in either direction.

The advice says “open-ended prompts”, and this task is not one. It is one bug in one small module with a test suite defining success exactly. The claimed benefit of planning is that it stops expensive wrong turns before they happen, and a task with a definite right answer gives it very little room to do that. On well-specified work, planning first was more expensive with no success benefit. The open-ended case has since been measured and it did not rescue the result: 2.27x with no overlap.

Controls

Correction, 2026-08-12. This post was published as “Planning first cost 1.95x more, and the reason given for it was backwards”, and that second claim is withdrawn along with the title carrying it. It compared 15 turns across two calls against 9 turns in one, which is two sessions against one. Split by call, the execute phase used fewer turns than auto-accept, 8 against 9, and 18% fewer dollars, so pausing for approval did reduce the work in the phase where the work happens. The mechanism is not backwards; it is real and too small to cover the plan. Two further errors are corrected here: the quotation of the advice had dropped the words “on open-ended prompts” from inside the quotation marks, and the post then concluded that the advice was given unconditionally, which its own next section contradicted. Both the truncated quote and that conclusion are gone. The claim that the model “reports the working tree read-only” is also gone, because no reply says it. The token, dollar, turn and wall-clock figures are unchanged, and the permission limit disclosed above was not in the original.

Second correction, same day. The permission limit was published with a directional claim, that the asymmetry made these figures upper bounds. Re-running the experiment with both arms allowed the same tools refuted it: the ratio went up, to 2.01x tokens and 2.18x dollars over eight runs per arm. That claim is withdrawn in the section above. Everything else on this page stands, and the re-run reproduces both of its findings with complete separation instead of thin margins.

The fixture discriminates, proven before any model call. Unfixed, the suite fails. With the correct non-mutating fix, it passes. With the obvious in-place sort, which makes the failing test pass, it still fails because it breaks a different test. Without that spread a pass would be indistinguishable from a rig that cannot fail anything.

The test suite was checksummed before and after every run. A suite the model can edit is not a pass/fail signal. All 8 measured runs left test_work.py byte-identical, and the pass was determined by the harness running pytest itself, not by the model reporting one.

A pilot round in a separate session agrees on the headline and disagrees on one detail. Run before these four with the same design, it gave 2.33x tokens and 3.04x dollars, in the same direction and larger. Its execute call, however, came in 1.3% above its auto-accept run, $0.1893 against $0.1869, where all four measured rounds went the other way. It is published separately rather than pooled, because it was a different session.

Isolation on every call, --setting-sources project, asserted onto the command line. Tokens from modelUsage, not .usage. Model replies in the corpus are stored truncated at 400 characters, so the longer ones stop mid-sentence.

Every run is published in plan-mode-cost.json.