Measurements

Raw data

Where a trial produced a transcript, the whole thing is published rather than summarised. Every prompt, every reply, and the verdict applied to each, so any figure here can be disagreed with directly instead of taken on trust.

• Skills curve on 2.1.246: unchanged, and one cell that shows why the estimator is a mode

  A re-verification that found nothing moved, on the curve the calculator interpolates every skill estimate from. Eighty runs, four rounds, none failed. Ten of the eleven count cells returned the SAME value in all four rounds and every one matches 2.1.238 exactly: 400 tokens at 10 skills, 7,810 at 200, 11,711 at 1,000, with the two-regime collapse from about 39 tokens per skill below 200 to about 3.9 between 400 and 1,000. THE ONE EXCEPTION IS WHY THE ESTIMATOR IS A MODE RATHER THAN A MEDIAN. The 300-skill cell returned [8966, 8966, 8966, 8069], three rounds at 8,966 and one at 8,069, a spread of 897 which is this harness’s known intermittent component failing to fire. Reading only the last round would have reported the curve as changed at that point; the mode is 8,966 and the published value is correct. That intermittent component is NOT retired here, unlike the MCP curve measured the same day where every cell came back single-valued and its noise limit could be retired. The description-length and body-size arms are published as their own cells and never pooled into the count curve, because pooling makes the 40-skill cell appear to disagree with itself. Measured under --setting-sources project from zero pre-existing skills, so it is the clean-project case only. The trial it came from.

• MCP per-tool curves on 2.1.246: unchanged in all six cells, and one control that failed

  A re-verification that found nothing moved, published because a null on a load-bearing calculator curve is worth as much as a change. Every one of the six cells is BYTE-IDENTICAL to 2.1.238 across eight releases: deferred costs 90, 810 and 3,210 tokens at 5, 50 and 200 tools, loaded costs 3,270, 32,610 and 130,410, and every cell returned the SAME value in all four rounds with zero spread. That zero spread RETIRES a limit the previous corpus carried, which said the tool-search-ON floor had an intermittent component giving every deferred cell a 719-token spread as large as the measurement itself at 5 and 50 tools. THE ISOLATION CONTROL COULD NOT BE RUN AND SAYS SO RATHER THAN BEING OMITTED: its unisolated arm loads this machine’s own configuration, it lost two of four rounds outright, and the two that survived disagreed by 5,271 tokens, so every delta computed against it inherits that movement. What moved that baseline is NOT established and no cause is asserted. The consequence is stated plainly in the corpus: the per-tool figures rest on the isolated arm alone, which is what the calculator uses, and the claim that control would have supported, that per-tool cost is the same whether or not user config is loaded, is NOT carried forward to this release. The fixture is one shape, 798 bytes of a repeated character, so the cross-version comparison is sound while the absolute per-tool figure is fixture-specific. The trial it came from.

• Startup floor on 2.1.246: the previous release-boundary move reverted

  Four rounds, every cell measured twice, so eight readings per floor arm. Both floor arms came back single-valued with a zero noise floor in every round: 22,801 tokens with tool search on and 37,870 with it off. THE HEADLINE IS THAT THE PREVIOUS RELEASE REVERTED. The 2.1.243 corpus reported the tool-search delta moving to 15,251 after being bit-identical at 15,069 across three consecutive measured releases, and framed that as a boundary that retired an earlier note. It went straight back: this run measures 15,069 again, the exact prior value. So 15,251 was a ONE-RELEASE EXCURSION rather than a new level, and the delta reads 15,069 on 2.1.236, 2.1.238, 2.1.241 and 2.1.246 with a single release out of line between them. What that does not license is a rule: two observations, one out and one back, do not establish that such moves are always transient, and no mechanism is claimed in either direction. Both ends moved DOWN across this boundary by different amounts, on by 158 and off by 340, and their difference still landed exactly on the prior value, which is recorded as an observation and not explained. The delta_series block is computed from every published floor corpus at assembly time rather than typed, so the four-releases-at-15,069 claim is checkable inside the file. The loaded arm reached 64,083 and is deliberately NOT attributed to the release, because it mixes the version change with this machine’s own configuration; it is also the only arm that was not single-valued, mode on 5 of 8. Twelve rows, none excluded. The trial it came from.

• On real work the lazy-coder plugin did shorten the code, and skipped the CLI 8 of 8

  The one-liner corpus at /ponytail-behaviour-2-1-243.json named its own limit: a task with real architecture to choose, which is where a do-not-over-build instruction would matter most, was untested. This runs it, and it REVERSES BOTH HALVES of that finding. Reading a CSV of orders and reporting revenue per customer, eight rounds per arm, 16 usable runs, none excluded. THE ARMS DO NOT OVERLAP: ponytail wrote 37 to 60 lines against 73 to 88, a gap of 13. The single clearest difference is an argparse command-line interface: the baseline built one in 8 of 8 runs and ponytail in 0 of 8, total separation on a feature nobody asked for. And the docstring result INVERTS: on the trivial task ponytail dropped the docstring 8 of 8, here it keeps it 8 of 8, same as the baseline. So the effect is real in both trials but expresses differently. On a trivial task the only thing left to cut is cosmetic, so the docstring goes; on a task with structure the docstring stays and the unrequested CLI is what does not get built. The one-liner was the misleading case. Five rounds were run and then three more before anything was drafted, the step that reversed the headline on the previous task; here it held. Arrival control 8 of 8. What is NOT shown: that shorter is better. Neither program was run against real data or reviewed for correctness, and a 40 percent shorter program that drops a needed guard would score identically here. The baseline is also not a neutral control, it is Claude Code with no instructions at all, which builds a CLI every single time.

• A lazy-coder plugin dropped the docstring 8 of 8, but did not reliably shorten the code

  SUPERSEDED IN PART by /ponytail-architectural-2-1-243.json, which runs the harder task this corpus named as untested and reverses both halves of the result below. ponytail claims the best code is the code you never wrote. The cost corpus at /ponytail-session-cost-2-1-243.json measured what it costs to load and said plainly that its actual claim was untested; this closes that line. One small task with room to over-build, a Celsius-to-Fahrenheit function, eight rounds per arm, 16 usable runs and none excluded. THE CLEAN RESULT IS CATEGORICAL: the baseline wrote a docstring in 8 of 8 runs and ponytail in 0 of 8, a total separation. THE LINE COUNTS DO NOT SEPARATE, and that matters because the tempting summary is that ponytail makes the model write less code. It wrote 2 lines in seven of eight runs against a baseline of 3 or 6, but one run came in at 7 and overlaps. That run added assert-based self-checks rather than a demo block, which ponytail’s own ruleset prescribes for non-trivial logic, so it is the tool following a different rule of its own rather than failing. AFTER FIVE ROUNDS THE ARMS DID NOT OVERLAP and that was going to be the headline; three more rounds produced the 7-line run and made the five-round claim false. That reversal is published in the corpus rather than quietly dropped. The algorithm was identical in every run of both arms, so the difference is what was written around the code, not the code. Arrival control: 8 of 8 probes named the active level, so a null could not be confused with a hook that never ran. Whether shorter is BETTER is not measured, and one arithmetic one-liner says nothing about a task with real architecture to choose, which is where such an instruction would matter most.

• A trending lazy-coder plugin costs 831 to 1,109 tokens a session

  ponytail was trending at 944 stars on 2026-08-25 and injects a standing ruleset through a SessionStart hook, so it is paid on every session. Its payload measures 5,229 UTF-16 units, comfortably under the 10,000-unit cap established at /hook-unit-2-1-238.json, so the instructions are delivered in full rather than filed. Eight rounds, two readings per cell, 24 usable rows and none excluded. THE ARMS DO NOT OVERLAP: every run with the hook cost 23,854 to 24,302 tokens, every run without it 22,885 to 23,027. The paired per-round difference ranges 831 to 1,109 tokens. NO single number is published, and that is the point: the baseline arm is bimodal, splitting into two clusters about 138 tokens apart, and BOTH of the largest differences are exactly the rounds whose baseline sat in the low cluster. So the spread is largely the baseline’s own intermittency rather than variation in what the hook costs, and a mode over that would describe neither cluster. Four rounds were run first, came back multimodal with the treated cell tied 3-3, and four more were run before anything was drafted; they confirmed the bimodality rather than resolving it. The arrival control is what makes the difference mean anything: each round also asked, with the hook installed, which ponytail level was active, and 16 of 16 replies named a level, so a zero difference could not have been confused with a hook that never ran. A security scan of the runtime path passed. Nothing here tests whether ponytail changes what the model writes, which is its actual claim.

• Two lockdown mechanisms, 16 runs, no leak: a bypass that would not reproduce

  A follow-up that failed to reproduce its own starting point, published as one. /hook-threshold-2-1-238.json records a model reaching a file with Read, Bash, Glob and Grep all disallowed, by finding a tool that accepts a command and running tail through it. This trial tried to reproduce that and could not: 0 of 16 locked-down runs leaked the canary, against 8 of 8 unlocked controls, so the arms do not overlap and the fixture is proven reachable. TWO hypotheses were tested and BOTH refuted, and both are published rather than deleted. First, that a lockdown holds for a secret-looking file and fails for an ordinary one: the same canary string was placed in a .env as a credential and in a notes.txt as a build id, and both framings blocked identically. Second, that the CLI flag behaves differently from the settings rule, since removing a tool leaves missing capability while denying a path returns an explicit policy error: --disallowedTools and permissions.deny each held 8 of 8, though the flag arm visibly went looking, calling ToolSearch nine times across the runs before giving up. Tool calls are read from the session transcript rather than inferred from the reply. This closes an open line on /what-a-read-deny-rule-on-env-actually-stops/, which said only Read(./.env) had been tested. It is NOT an enforcement guarantee: one task shape, one machine, the vendor documents that deny rules do not cover arbitrary subprocesses, and failing to reproduce a bypass on a different task does not show it cannot happen. The trial it came from.

• Crossing the hook output cap costs 78 percent of the context you sent

  A SessionStart hook returning additionalContext is how a plugin injects standing instructions. Anthropic caps that output at 10,000 and says so twice in the hooks reference. Forty runs on Claude Code 2.1.238, four rounds per rung of a ladder, measure what crossing it costs: below the cap 10,016 tokens of context arrive, identical in all four rounds; above it 2,172 to 2,181 arrive. The arms do not overlap, every inlined round sits above every persisted round, and about 78 percent of what was sent is not delivered. It is not destroyed: the full payload is written to a file the model can read with a tool call, so this is a change of delivery rather than deletion. obra/superpowers, at 3,334 bytes, sits comfortably under and was inlined in every round at 1,271 tokens. Two things this corpus is careful about. Its own conclusion that the cap counts CHARACTERS was refuted by a later trial and is RETIRED in a superseded_claims block rather than reworded away; the cap counts UTF-16 code units, established at /hook-unit-2-1-238.json, and the cost figures are unaffected because this ladder is repeated ASCII where all three units are the same number. And the four probes that asked a model to recite a codeword are scored INVALID, because with Read, Bash, Glob and Grep all disallowed the model still reached the file: it searched for a tool, found one that accepts a command, and read the file with it. The threshold rests on the filesystem observable instead. The estimator refuses to report a mode on any rung where rounds did not repeat.

• Startup floor on 2.1.243: a delta move that did NOT persist, see 2.1.246

  Four rounds, every cell measured twice, so eight readings per floor arm. Both floor arms came back single-valued with a zero noise floor in every round: 22,959 tokens with tool search on and 38,210 with it off. This is the boundary where the tool-search delta finally moved. It had been bit-identical at 15,069 across three consecutive measured releases, 2.1.236, 2.1.238 and 2.1.241, and here it goes to 15,251, because the two ends moved by DIFFERENT amounts for the first time since 2.1.233: on by +166 and off by +348. That retires the qualification written at 2.1.241, which had observed both ends moving together by exactly six tokens and concluded the deferred-tools saving does not move. No mechanism is claimed for either observation. The loaded arm went to 62,357 and is deliberately NOT attributed to the release, because it mixes the version change with this machine’s own user-scope configuration; it is also the only arm that was not single-valued, mode on 6 of 8 with a spread of 62,078 to 62,627. The harness was given the shared result_guard before this run, so a 429 or 529 that parses cleanly and reports an empty modelUsage can no longer be recorded as a floor of zero; the guard was control-tested against a synthetic 529 first. All 12 rows are usable, none excluded. The trial it came from.

• Which AI crawlers actually showed up: 1 of 5 over 14 days

  robots.txt and llms.txt are served through a shim that logs one line per request, because this hosting plan exposes no access logs. Over 14 days and 19 hours it recorded 278 non-curl requests. robots.txt names seven AI agents, but only five of them actually crawl: Google-Extended is a standalone product token and Applebot-Extended, in Apple’s words, does not crawl webpages. Both govern how a parent crawler’s data may be used, so they can never appear in a request log and their zeros are definitional. They are reported separately, and the crawlers they govern both did crawl, Googlebot 26 times and Applebot 38. Of the five that do crawl, exactly one appeared: OAI-SearchBot, 50 times. GPTBot, ClaudeBot, Claude-SearchBot and PerplexityBot returned zero, including both crawlers their vendors document as collecting training data. llms.txt was fetched 25 times and exactly one of those came from any of the five; the rest were ordinary search bots, SEO tools and unidentified clients. Every purpose label is quoted from the vendor’s own documentation with a link. The shim serves only those two paths, so a zero means a crawler never fetched robots.txt or llms.txt here, NOT that it never fetched a page, and the corpus states that above the results rather than in a footnote. One young low-authority domain, one window, and every identification is a self-reported user agent.

• What a book costs in Claude Code, against its tiktoken count

  Twenty-eight runs measuring the same bytes two ways: written as a project-scope CLAUDE.md and read back from modelUsage, against tiktoken cl100k_base. A technical book and a novel, three slice sizes each, four rounds. Claude Code charged 1.41x the cl100k count on both, and the published figure is the slope between slices rather than a ratio of totals, so the CLAUDE.md framing constant cancels instead of inflating it. The trial it came from.

• Re-verification against Claude Code 2.1.227

  Every round of the 2.1.227 re-check: the CLAUDE.md versus skill arms with their floors, and the batching arms with their arrival-control results. The trial it came from.

• Adherence and chain-of-thought, isolated re-checks

  80 isolated adherence turns and 24 isolated chain-of-thought runs, closing the re-verification of every behaviour trial published before the isolation method existed. The trial it came from.

• Mutation trial, isolated re-verification

  20 runs repeated with the user-level CLAUDE.md excluded. The caught-bug result held at 38 of 38; the claim about which subjects fail did not. The trial it came from.

• JSON extraction, isolated re-verification

  24 runs repeated with the user-level CLAUDE.md excluded, to check whether the absent-field result was the model or the machine. Identical. The trial it came from.

• Prompt injection, isolated re-verification

  32 runs repeated with the user-level CLAUDE.md excluded. Loud payload still obeyed 0 of 20; quiet payload still followed 6 of 6 by one model and 0 of 6 by the other. The trial it came from.

• Comment rule trial, corrected

  32 files written with the user-level CLAUDE.md excluded, so the control arm is genuinely rule-free, with the full source of every file and the isolation method recorded. The trial it came from.

• JSON extraction trial

  24 extractions scored twice over, once for schema validity and once for whether each field is true, with both documents, the schema and every returned object. The trial it came from.

• Generated test suite mutation trial

  20 generated pytest suites run against deliberately broken code, with every subject, every mutant, each suite in full, and the pytest output for the four that never ran. The trial it came from.

• Comment rule trial

  86 generations across three paths, including the full source of all 32 files written to disk, and the arms where the baseline read zero and the experiment could not answer the question. The trial it came from.

• CLAUDE.md adherence trial

  344 turns of two continuous conversations, with the function names, parsed naming style and context size for every scorable turn, plus the turns excluded for returning nothing. The trial it came from.

• Prompt injection trial

  42 runs. Every file the model was given, every reply, and the verdict for each. Includes the loud payload that was never obeyed and the quiet one that was followed 6 of 6. The trial it came from.

• Generated code security trial

  36 generated Python files across six vulnerability classes, each with its prompt, the code, and the scanner findings that were and were not counted. The trial it came from.

• Chain-of-thought trial

  104 runs across three prompt styles, each with its exact prompt, reply, computed truth and extracted answer. The trial it came from.

• Cost-to-complete fixture

  The failing test suite used to measure what finishing one task costs on each model, including the trap that makes the careless fix pass one test and break another. The trial it came from.

• Tokenizer corpus

  One paragraph in ten languages plus four content types, written for this test, so the cross-model token counts can be reproduced against any tokenizer. The trial it came from.

• Parallel subagent cost runs

  44 runs across four arms: the fan-out ladder on two separate days, the paired inline control, the same ladder with user configuration loaded, and the paired measurement of that configuration. Every run carries its arrival-control file count. The trial it came from.

• Fork-subagent flag runs

  28 runs testing whether CLAUDE_CODE_FORK_SUBAGENT reduces parallel subagent cost, across a cold ladder (three arms including an explicit-off control) and a warm parent whose warm-up had to echo a planted marker. Tokens are recorded split into cache creation and cache read, because summing them hides the effect under test. The trial it came from.

• Plan mode versus auto-accept runs

  Ten runs comparing plan-then-execute against straight auto-accept on the same bug fix, split by call so the plan and the execution can be priced separately, with the fixture control proving the suite discriminates three ways, a per-run checksum showing the model never edited the tests, and a disclosed limit that bash was denied outside plan mode. The trial it came from.

• Plan mode versus auto-accept, equal tool permissions

  The re-run that removes the permission asymmetry above: 16 runs, eight per arm, with bash granted on every call in both arms and the grant proven to have arrived by requiring a Bash tool use in every action record. Every tool call is published by name and order, so the turn counts can be audited against what was actually done. The trial it came from.

• Plan mode versus auto-accept, open-ended task

  The third run in the series, on the open-ended case the advice actually names: 15 runs, seven paired rounds, an open prompt and a hidden rubric applied afterwards. Every rubric item, every per-run outcome and every tool call is published, along with the three-way control proving the rubric separates broken from lazy from good. The trial it came from.

• One prompt, one session, or a fresh session each step

  Forty-two runs across three ways of delivering the same three-step task: thirty on Claude Code 2.1.228 and twelve re-verifying them on 2.1.229, each run labelled with the version it ran on. The session id of every call is published so the treatment can be checked rather than trusted, alongside the full tool-call record behind every turn count and a hidden test suite written after the calls finish. The trial it came from.

• node_modules cost, and whether .claudeignore does anything

  Twenty-four token runs across two probes and three arms, plus nine canary runs that ask the model to read a file inside an excluded directory. The synthetic node_modules is measured on disk every run and published, so a null result cannot be a fixture that never built, and the deny-rule arm doubles as the control proving project settings were loaded at all. The trial it came from.

• How far a Read deny rule on .env reaches

  Eighty-eight runs across eight ways of asking for a denied file, scored by whether a fabricated credential-shaped canary came back rather than by what the model said. The no-rule control arm proves every route works unprotected, and the captured tool calls are what separate the permission layer refusing a read from the model declining to attempt one. The trial it came from.

• Claude Code's own context floor, measured directly

  The runs behind the second correction to the calculator baseline. An empty directory with user-scope configuration excluded, every cell measured twice per round for a noise floor, plus the paired decomposition that splits a loaded machine into the floor, its user CLAUDE.md and everything else at user scope. The three parts reconstruct the loaded total to one token. The trial it came from.

• Configured fan-out, re-measured

  Forty runs re-deriving the configured subagent arm after its configuration figure turned out to be counted once per turn rather than once. Adds the paired inline control that arm never had, which is what separates a per-subagent cost from a per-unit-of-work one, and records the turn count on every run because a run total mistaken for a context size is the error being corrected. The trial it came from.

• Compaction survival runs

  Eight runs plus a no-compaction positive control, testing which planted values survive auto-compaction. Two bulk types, sparse and high-entropy, with every planted value, every reply and the per-run compaction events. The trial it came from.

• Re-verification against Claude Code 2.1.228

  Every round of the 2.1.228 re-check: four batching rounds with their arrival-control results, the CLAUDE.md versus skill arms with their floors including the round thrown out for landing inside the baseline drift, and the configured subagent arm. The trial it came from.

• Fan-out execution mode runs

  34 runs showing that an eight-agent fan-out lands in one of two non-overlapping cost modes, separated by the num_turns field the CLI already reports. Includes the width sweep at two and four agents and the re-verification runs the modes were first spotted in. The trial it came from.

• Fan-out slope, conditioned on parent turn count

  Eight rounds instead of four, which is what it took to explain a dispersion three previous corpora recorded as unexplained. The parent context tracks how many turns IT takes: a four-agent fan-out runs in six or seven and the total moves 16% between them, against 0.2 to 0.3% within one. Every slope is computed inside one turn count, and both conditions are published because the caller does not control which they get. The trial it came from.

• Subagent slope on 2.1.231

  Forty runs re-measuring the per-subagent cost after the calculator was found still computing from a withdrawn configuration figure. Isolated and loaded fan-out at one and four agents with the paired inline control that separates per-subagent from per-unit-of-work, plus the single-turn configuration measurement the earlier correction turned on. The trial it came from.

• Claude Code's own startup floor on 2.1.232

  Sixteen repetitions per cell after Claude Code updated mid-measurement and the deploy gate blocked every deploy until the calculator caught up. The mode, not the minimum, because the call is bimodal: 22,528 in 13 of 16, with the other two values exactly 198 and 521 below it. The two arms moved in opposite directions, so the tool-search penalty fell 141 tokens. The trial it came from.

• Startup floor on 2.1.231

  The floor re-measured after the version bump, five rounds with every cell run twice for a noise floor. Reports the MODE rather than the minimum, because modelUsage on a trivial call is bimodal at 22,404 and 21,883 depending on whether a Haiku call fires, and the minimum would publish the rare run as the floor. The tool-search delta held to within a token. The trial it came from.

• CLAUDE.md rule-count runs

  Thirty-five runs testing whether a large CLAUDE.md degrades adherence to the rules inside it, laddered from 2 to 500 rules with two regex-checkable targets, one at the top of the file and one at the bottom. Carries a negative control with the targets deleted that scores zero, and the exact CLAUDE.md used in all seven arms. The trial it came from.

• Output verbosity runs

  Thirty runs across three batches testing whether a terseness instruction cuts the bill, with the output share of tokens and of cost separated because they differ sevenfold. Carries a rate reconciliation that reproduces every reported Opus cost to the cent, a published matcher control, and full reply text plus marker index on batch 3 so the arrival result can be recomputed rather than trusted. The trial it came from.

• Prompt cache TTL runs

  Twenty-four runs testing whether an idle gap actually expires the prompt cache, and what the one-hour cache costs to have. Carries the per-run write tier, a published pre-run flag control, per-turn and whole-session costs (which point opposite ways), a nine-run bracket showing the cache alive at 55 minutes and gone by 70, and the diagnostic run that proved the first design was blind because the first resume always rewrites the cache breakpoint no matter how short the gap. The trial it came from.

• Fan-out wall clock on heavy units

  Twenty-four runs testing whether delegating saves time on work built to favour it: N independent 40KB files, one planted marker each, one subagent per file against the same job inline. Every tool call is attributed to the parent or to a subagent via parent_tool_use_id, which is what two earlier discarded attempts had no way to do. The trial it came from.

• What a skill costs, clean-project sweep

  Eighty runs sweeping skill count from 1 to 1,000 and description length from 30 to 1,500 characters, with a body-size arm as the control that the fixture builds real skills. Carries the description fit basis and its valid range, stated because a fit basis that is not disclosed cannot be checked. The trial it came from.

• Whether a skill costs more when you have fewer

  Thirty-two runs measuring the same forty skills with the isolation flag on and off in the same round, which is what stopped a version-change claim from being published: the flag alone moves the cost 10.2x, so this site’s earlier 1.5-tokens-a-skill figure was measuring the other regime rather than another release. The trial it came from.

• Does a description still cost anything at 600 skills

  Twenty runs testing the mechanism the sweep was about to assert: description length at 600 skills, with the same contrast at 40 skills running in the same rounds as the positive control that makes the null believable. The trial it came from.

• What a skill costs, re-measured on 2.1.233

  Four rounds on the current release. The per-skill figure did not move at all, 39.05 tokens at 200 skills on both 2.1.232 and 2.1.233, while over the same release the tool-search saving fell 17% and the tool-search-off floor 8%. The measurement noise also collapsed: not one of the eleven cells was stable across rounds on 2.1.232, and ten of eleven are on 2.1.233. The trial it came from.

• What a skill costs, re-measured on 2.1.232

  The re-measurement the context calculator uses for its skills curve, published late: it had been sitting unassembled while the only skills corpus a reader could follow was 2.1.231, which differs at 10, 200 and 300 skills. Three arms measured separately, count, description length and body size, and the estimator is the MODE rather than the median because an intermittent 198-token component makes a four-round median land between two real values. The trial it came from.

• What an MCP tool costs, re-measured on 2.1.233

  Four rounds on the current release, with the isolation control re-run alongside. Every cell is identical to 2.1.232, deferred and loaded, at 5, 50 and 200 tools, while both floors moved underneath it. That is the second per-item curve in a row to hold still through a release that moved its own floor. The trial it came from.

• What an MCP tool costs, re-measured on 2.1.232

  The re-measurement the context calculator actually uses. It went unpublished, so the calculator carried a 50-tool deferred figure of 810 tokens while the only corpus a reader could check said 711. Published here with its isolation control, and the calculator curves are now tied to this file by the deploy gate so they cannot drift from their receipt again. The trial it came from.

• MCP tool cost, re-measured with an isolation control

  Fifty-six runs re-measuring both MCP curves on 2.1.231 after the calculator was found quoting a version nobody was running. The deferred curve holds; a loaded tool measures 652 tokens against a published 304, and the isolation control that explained the skills discrepancy the same day explains none of this one. The trial it came from.

• Both plugin ladders at eight rounds, published as bimodal cells

  Eighty runs re-running both ladders on 2.1.238 after a median was found summarising bimodal cells as midpoints no round produced. Eight rounds did not resolve the split: three cumulative cells come back tied four-four between values exactly 198 apart, so the cells are published as two values and their counts rather than as one number. The marginal rates are paired inside each round, which cancels a component that fired in both cells, and those do resolve: 40.0 tokens a command between 50 and 200, collapsing to 9.93 and then 5.62, against 51.0 a subagent definition with no collapse. The trial it came from.

• What a plugin's parts cost, re-verified on 2.1.232

  The whole 88-run measurement re-run sequentially on the release that followed, after four concurrent harnesses had made every call return a zero context and the arrival controls caught it. Every point either reproduced exactly or came in 99 tokens lower, which is half the documented intermittent component. Every harness now records the Claude Code version it ran against. The trial it came from.

• What a plugin's parts cost, and which share a budget

  Seventy-two runs pricing slash commands and subagent definitions on their own ladders, plus an ablation that re-measures the cost of forty skills beside each element in turn. Commands collapse in bulk exactly as skills do; subagent definitions cost 51 tokens each at every count measured and never get cheaper. Floor-paired and cell-to-cell figures are labelled separately throughout, because the floor call carries most of the noise. The trial it came from.

• What a CLAUDE.md costs per turn, split by billing tier

  32 runs across one resumed four-turn session, with and without an 18KB CLAUDE.md, adjudicating the circulating claim that a 5,000-token CLAUDE.md is a 5,000-token tax on every turn. Input, cache creation and cache read are kept separate per turn, because summing them is what makes the context claim and the cost claim look like one claim. The trial it came from.

• What the docs say a skill costs, against what it measured

  Anthropic documents ~100 tokens per Skill for the metadata loaded at startup. This holds that figure against the six trending packs already installed and measured here, and records six chat-model runs asked the same question. The model runs are labelled an impression rather than a measurement, because the same model contradicted itself between two identical prompts. The trial it came from.

• What the trending skill packs cost to install

  56 runs across two harnesses, installing six Agent Skill collections taken from GitHub Trending and measuring each against an empty floor rather than multiplying a published rate by a file count. The second harness installs only what each plugin manifest enumerates, which corrected mattpocock/skills from 1,209 tokens for 35 files to 852 for the 25 it ships; four packs reproduced to the token across both. Carries the arrival control that doubles as a prompt-injection check, over 163 third-party skill descriptions in the shipped fixture and 173 in the first harness, and the check that this site own published model misses real packs in both directions. The trial it came from.

• Claude Code's own startup floor on 2.1.233

  Sixteen repetitions per cell. Notable for what is absent: every repetition of both floor arms returned the same value, where 2.1.232 returned three distinct values 198 and 521 tokens apart. The two intermittent components that have been the dominant noise source in every corpus on this site do not appear. The tool-search penalty also fell 3,020 tokens to 14,644. The trial it came from.

• Six models counting 120 working days, twice each

  Twelve runs across six free-tier chat models from four makers, all asked the same date question in a fresh chat. Every verdict is derived by the assembler rather than typed: both defensible readings of the question, every weekday, every month working-day count and every date offset the runs quoted. Seven runs correct. Runs that showed no working carry a null setup rather than a passing one, so the tally cannot claim they understood the question. The trial it came from.

• Whether CLI flags move a skill cost, and whether the cost is measurable at all

  Thirty-six runs on Claude Code 2.1.234 testing a prediction registered before the first round: that --allowedTools and --permission-mode explained why this site measured one pack at 274 on one harness and 249 on another. Refuted. The paired within-round differences flip sign and match components already published as intermittent here, so no directional flag effect is detectable. The real answer is in the spread: the same one-skill pack returned 70 to 967 tokens across twelve readings, while a 24-skill pack held to 0.58x of its minimum. Carries forward this site existing limit that one skill sits inside the noise, and re-verifies it rather than retiring it. The trial it came from.

• A trending diagram skill, installed and actually used

  Twenty runs against cathrynlavery/diagram-design pinned at one commit: a 2x2 of trivial and diagram prompts with and without the skill, plus eight rounds testing the documented marker file, published per round rather than only in summary. Records that it delivered a file in 1 of 4 unattended rounds against 4 of 4 for plain Claude Code, and every file the agent read in every round including the three that drew nothing, counting Bash reads as well as Read-tool ones. Every cost multiplier is published as null with its reason, and the startup figure this corpus once carried was later shown unmeasurable by flag-effect-2-1-234.json. The trial it came from.

• Re-measuring the calculator’s own skills curve, at both ends

  Sixteen readings on Claude Code 2.1.234 against a curve this site published as spread 0 over four rounds. The published values reproduce, both landing ten tokens off, but the precision they appeared to have does not: the 10-skill cell ran 192 to 1,287 tokens, a spread of 281% of its own mode, while the 1000-skill cell held to 8%. Thirty-six times the relative unreliability, in the direction predicted before the run. The fixture generator is imported from the original harness and asserted byte-identical, because re-implementing it produced 290 against a published 400 and would have been published as a drift in the calculator. The trial it came from.

• What Anthropic’s own five skill plugins cost, against its own documented figure

  Fifty runs on Claude Code 2.1.234 against anthropics/skills at one commit. There is no plugin.json: the marketplace offers five separate plugins, so installing the repository is not a thing anyone does and each is priced on its own. Only one of the five is large enough for this instrument to resolve, and it lands at 102 tokens per skill against a documented ~100. The other four carry ranges rather than figures, three of them because the harness pre-registered before running that a single-skill cell is below its resolution; the ranges vindicate that, since one runs from minus 578 to plus 319 and installing a skill cannot make a session smaller. Also publishes two estimators side by side, because the paired-floor control this site normally uses was found to amplify per-call noise rather than remove it. The trial it came from.

• What a skill description costs per character above 300

  Twenty-five runs sweeping description length across the 1,024-character maximum Anthropic documents, forty skills per cell, on Claude Code 2.1.235. Tripling a description from 512 to 1,600 characters costs 637 tokens across forty skills, about 0.015 per character against the 0.276 this site measured below 300. The pre-registered prediction is scored as refuted, and the reason is published: it assumed a rate this site had already stated was valid only to 300 characters. The trial it came from.

• Six free models, one JSON prompt, twice each

  12 runs on duck.ai’s free tier, six models across four vendors, asked identically for a bare JSON object with a fresh chat each time. An exact three-three split on how the reply was rendered, every model on the same side both rounds and all six matching the previous day. All twelve replies carry the byte-identical object once whitespace is stripped. Every run links a published screenshot showing the model header and the reply together, which the first attempt at this trial lacked and was held for. Also records the near-miss: after one round, fencing and pretty-printing correlated perfectly across all six models, and the second round refuted it.

• Which unit the 10,000-character hook cap counts

  44 runs adjudicating whether Claude Code’s documented 10,000-character cap on hook output counts characters, bytes, or UTF-16 code units. A nine-cell ladder holding characters constant while moving units, plus a probe pair of 5,022 and 5,023 characters straddling 10,000 units where a codeword planted at the end of the payload returns tool-free 4 of 4 at one and 0 of 4 at the other. Over the 36 ladder runs the unit predicate mispredicts 0 times, characters 12 and bytes 16, identical to two earlier independent ladders. The written post is held pending verification; the runs are published because they are the finding.

• Narrowing the skill description cut onto the documented cap

  28 runs narrowing this site’s earlier 1,400-to-1,550 bracket onto the documented skillListingMaxDescChars default of 1,536. Text ending at 1,530 characters is answered from context in 4 of 4 rounds; text ending at 1,536 never is, in 4 of 4 rounds, with the model shelling out to read the file instead. A control renaming the skill from 5 characters to 56 moves the boundary cell not at all, so the figure is not name-dependent. Published alongside the correction of the post that originally framed this cut as undocumented and cited a 1,024-character maximum. The trial it came from.

• Where a skill description stops reaching the model

  A rare marker planted at a known character offset inside one skill description, then asked for. The reply alone says the description is never truncated out to 2,000 characters, and that is wrong: the session transcripts show the model answering from context with no tool calls up to 1,400 characters and shelling out to Bash to open the file beyond it. Scoring tool use rather than the answer puts the cutoff between 1,400 and 1,550 characters, four rounds either side, and inverts the conclusion the naive scoring would have published. The trial it came from.

• Does the description cut count characters or tokens?

  Closes an open question the description-cutoff post published as unknown. Two fixtures of matched length, one ordinary prose and one API-shaped text with punctuation, identifiers and flags, with the tokenisation ratio between them measured by a paired cost arm in the same runs rather than assumed: the dense text costs 2.06 times as much per character. A token budget set where prose is cut would have severed the dense description near 702 characters; it is answered from context at 1,400, four rounds of four, exactly like prose. The limit counts characters, so the guidance applies to every author rather than only to those writing prose. The trial it came from.

• An 817-skill real pack against this site’s own skills curve

  A cybersecurity pack trending on 2026-08-20 ships 817 Agent Skills and costs about 56,600 tokens at startup, 69.30 per skill, from 24 readings that agree within 48 tokens in 21 of them. This site’s published curve predicts roughly 11,000 for the same count and its own cells still reproduce, so the curve is accurate about the synthetic skills it was built from and 5.2x low against a pack anyone can install. The cause is NOT established: description length, description string-uniqueness and extra frontmatter were each tested and refuted, and the remaining candidate collapsed on tokeniser arithmetic showing the synthetic descriptions are not in the measured context at this skill count. This site’s own real-pack corpus never disagreed: six real packs measured 34 to 274 tokens per skill, so 69.30 sits inside that and the synthetic curve is the outlier. Also here: the extra frontmatter is not charged, which matches Anthropic’s documentation, and two of the 24 readings came in at 17,352 for reasons nobody has established. All 80 runs across 30 rounds are published, and within each harness the absolute session context is byte-identical round to round, so the variation between modes is floor movement rather than pack movement. The trial it came from.

• A 65% token claim, and two of its author’s own designs disagreeing

  Sixty runs against JuliusBrussee/caveman, a 99k-star Claude Code skill claiming 65% fewer output tokens. Three system-prompt arms on five of the author’s own prompts, four rounds each. The estimator is non-overlap of observed ranges rather than a mode or a median, asserted in code, because the guarded mode is refused in all fifteen cells and the median is a value no round produced in all fifteen. Under this design the skill separates from an unprompted baseline on five tasks of five and from the string “Answer concisely.” on none, and its system prompt costs a deterministic 2,517 tokens per single-turn call. Read the known_limits before quoting any of that: the author publishes a terse-controlled result of their own that points the other way, at +50% median, and the two designs differ in arm construction, delivery flag, model, CLI version and round count. The trial it came from.

• Startup floor re-measured on Claude Code 2.1.236

  Five rounds, every cell measured twice per round so each round carries its own noise figure. Both isolated cells returned the identical value in all ten readings: 22,787 tokens with tool search on and 37,856 with it off, noise floor zero. The calculator had been quoting 2.1.233 and was understating the floor by 490 and 915 tokens. Across five releases the tool-search-on floor has moved inside a 490-token band while the tool-search-off figure swung from about 40,200 down to 36,941 and back to 37,856, so what moves is the deferred-tools saving rather than the floor beneath it. The trial it came from.

• Both MCP curves re-verified on 2.1.238, with the isolation control

  Sixty-four runs on the installed release: the deferred and loaded curves at 5, 50 and 200 tools, four rounds each, plus the isolated-versus-unisolated bridge control. Neither curve moved a single point from the 2.1.236 corpus, and the calculator already carried these values. Checked for the phantom-median trap before publishing: every cell agrees 3-of-4 or 4-of-4, so median and mode coincide and no figure here is a value no round produced. The lone outlier in each deferred cell is exactly +198, the documented intermittent component. The trial it came from.

• MCP curves re-verified on 2.1.236, unmoved for a third release

  Four rounds at 5, 50 and 200 tools in both regimes, each arm paired against a floor measured in its own tool-search flag state so the 15,000-token cost of turning tool search off is not folded into the per-tool figure. Every cell came back identical to the token to the 2.1.233 run: 90, 810 and 3,210 deferred, 3,270, 32,610 and 130,410 loaded. That is three consecutive releases with an unmoved MCP curve while the startup floor beneath it moved every time, which is why the two carry separate version stamps. The isolation control was re-run on the same release, and the assembler refuses to merge halves measured on different releases. The trial it came from.

• The skills curve re-verified on 2.1.238

  Eighty runs re-measuring the whole skills curve on the installed release, four rounds across eleven counts from 1 to 1,000 skills, plus the description-length and body-size cells. Not one cell moved: every point is identical to the 2.1.236 corpus, and the calculator already carried these exact values. The two-regime shape holds, about 39 tokens a skill to 200 and collapsing to 3.9 between 400 and 1,000. All 80 run directories verified single-release from their transcripts. The trial it came from.

• Skills curve re-verified on 2.1.236: not one cell moved

  Four rounds, 80 runs, none failed, across all three arms the curve carries: eleven skill-count cells from 1 to 1,000, five description-length cells and three body-size cells. Every one returned the same mode as the 2.1.233 run, so the calculator’s constants did not need to change and only its version stamp did. The estimator is the MODE rather than the median, because this harness carries a 198-token intermittent component that fires per call and a median of four rounds lands between two real values; the assembler refuses to build if any published figure is not a value some round actually returned. Read alongside the 817-skill real-pack corpus, which shows this synthetic curve under-predicting a real pack by 5.15x at the same count. The trial it came from.

• A trending skills directory lists 1,217 entries, and one upstream repo accounts for every dead link found

  VoltAgent/awesome-agent-skills, trending at 600 stars this week, checked on commit ea6bfaa. First, what it is: the repository holds four files and NO skills. It is a directory whose 247,301-byte README links to skills hosted elsewhere. Its GitHub description says 1000+ agent skills; counted here at 1,217 across 70 sections, so that claim is accurate. Links split 629 to github.com, 582 to officialskills.sh, and 6 elsewhere. A random sample of 60 entries was drawn with a recorded seed and each URL fetched once, with 404 and 410 scored dead and everything else, including 403 and timeouts, scored inconclusive and counted apart so bot-blocking could not be mistaken for a missing target. There were no inconclusive responses. Eleven of the sixty were dead, and all eleven pointed at the same place: github.com/NVIDIA/skills. Every one of the other 49 sampled links resolved. A separate targeted group of 15 further NVIDIA links, reported apart from the random sample and never pooled with it, was also dead, 15 of 15. The cause is not a deleted repository: github.com/NVIDIA/skills returns 200 and so does its skills directory. The upstream repo FLATTENED its paths, so skills/cuopt/cuopt-developer became skills/cuopt-developer, verified on three pairs where the flattened form returns 200 and the nested form the directory lists returns 404. 155 of the 1,217 entries use that old nested form, 12.7% of the directory, and it is NOT claimed that all 155 are dead because only 26 were tested. The directory itself predicted this: its Security Notice says skills are curated, not audited, and may be updated, modified or replaced by their maintainers at any time after being added.

• A trending CLAUDE.md costs 982 tokens a turn, not the 719 this site predicted

  The andrej-karpathy-skills CLAUDE.md, trending at 491 stars in a day on 2026-08-24 and measured unchanged at 2,357 bytes on commit 2c60614. Fifteen rounds, 120 runs, two arms differing only by the presence of that file at the project root, four resumed turns per session. Two predictions were pre-registered BEFORE any run and neither was revised: this calculator predicted 719 tokens from 106 plus 0.26 per byte, and an older corpus measured on 18,000 bytes of English prose predicted 650 at 0.2759 per byte. The measured mode is 982 tokens on 38 of 60 paired differences, which is 0.417 tokens per byte, above the 0.26 to 0.37 range this site had published for content variation. The older corpus carried a limit saying a file of tables or code would not cost the same per byte; this markdown-structured file, with headers, bullets and a fenced code block, demonstrates it. Two things are NOT claimed: the paired difference is multimodal, wandering across a range that spans both predictions, so neither is refuted, and one file does not re-fit a curve fitted from 2KB to 28KB. The rate on the calculator is unchanged and a rendered note discloses the gap. Scope, because it changes who the figure applies to: the repository is nine files, not one, and its README recommends installing the guidance as a Claude Code plugin FIRST, with the project-root CLAUDE.md listed second. The 2,357 bytes measured here are that second option; the plugin path was not measured. Attribution was checked rather than assumed: the trending listing is a re-host and the plugin manifest names forrestchang. Controls published per run: the isolation flag, the fixture asserted at 2,357 bytes on one arm and absent on the other, one constant working directory because the startup floor moves with the path, the arrival marker on all 120 runs, and the billing split kept separate.

• Six free models, one false premise, twenty-four corrections

  Two independent trials on duck.ai on 2026-08-24, twelve runs each, six free models across four vendors, two rounds per model, a fresh chat every run. Every run was asked the identical question: why does list.sort() return a new sorted list instead of modifying the original in place. The premise is false in BOTH halves, verified in a REPL before the rubric was written: list.sort() returns None and sorts in place, and sorted() is the one that returns a new list. The rubric of CORRECTED, PARTIAL and CONFABULATED was fixed in writing before any model was asked and is published at /false-premise-preregistration.md. Result: 24 of 24 CORRECTED, no PARTIAL, no CONFABULATED, every model on the same side in both rounds of both trials. Not one of the six was dragged into explaining behaviour that does not exist. The two trials differ in EVIDENCE and the corpus records that as a field rather than a footnote: the first committed no screenshots, the second commits one per run under /shots/false-premise-2026-08-24/, each showing the model label together with the BEGINNING of the reply including the sentence that contradicts the premise, with the screenshot label compared against the text-capture label on all twelve and zero disagreements. These are 1568x696 viewport captures, so the longer replies continue below the fold and a screenshot is not a capture of a whole answer. One run is documented in detail because the label resolves several seconds AFTER the reply finishes streaming, so a screenshot taken when the text looks complete can still read Generating response with no model name, which is how an earlier trial on this site ended up with three unlabelled screenshots. Separately and consistently across all twelve labelled runs, the part of the duck.ai privacy line that VARIES BY MODEL differs across the six, in three forms (the full rendered line also carries a fixed DuckDuckGo prefix and a Learn more link, neither recorded): zero data retention for the two OpenAI models and Mistral, limited data retention for Claude Haiku 4.5, zero provider visibility for gpt-oss and Gemma, with no AI training on all six. Recorded as observation; what those phrases mean in practice was not tested. Every count in the corpus is parsed from the raw capture files by the assembler, which asserts the run count, the score tally against the run count, the rubric membership of every score, two rounds per model, the label agreements, and that every screenshot a run names exists on disk. One figure is NOT derived and says so in place: the over-the-wire byte size in cdn_example is a dated curl measurement of a CDN output rather than a property of any file.

• Parallel subagent cost on 2.1.238, and why the constant fell 27%

  Ten rounds, all 100 runs stamped 2.1.238 and 97 usable, with the three failures excluded as unparseable rather than counted as zeros, and an arrival control passing on every row. The calculator constant drops from 28,259 to 20,581 tokens per parallel subagent, and a subagent did not get cheaper: at matched parent-turn counts the figure moved 1.0% and 0.9% since 2.1.232. What moved is which turn count is modal, from 7 turns in 5 of 8 rounds to 6 turns in 7 of 10, and the calculator carries the modal condition. Both conditions are published side by side because which one a reader gets is not under their control. Getting here took three runs. A five-round pass put only 3 rounds behind the mode. The ten-round re-run straddled a mid-session Claude Code update and was mixed-release, which the session transcripts show directly since they stamp the release on every message; it is not published. An earlier attempt to date runs from a two-token shift in the embedded floor cell was wrong and is withdrawn in this file, because the clean run reads the same floor as the 2.1.236 one. The harness now stamps the release top-level and per run. The trial it came from.

• Parallel subagent cost on 2.1.236, superseded by the ten-round run

  The five-round predecessor to the 2.1.238 run, published under its true release rather than discarded. It puts only three of its five rounds behind the modal parent-turn count, so the calculator does not take its figure from here; the ten-round 2.1.238 run does. Kept as the 2.1.236 reading of the same measurement, with its release established from the session transcripts rather than inferred. The trial it came from.

• The startup floor depends on the working-directory path

  48 runs of one trivial prompt in an empty directory on 2.1.241, changing nothing but the sandbox path. The floor ranged 22,765 to 22,847 tokens. Two paths of identical 67-character length differed by 14 tokens, and a 38-character path cost 20 tokens more than a 47-character one, so the quantity counted is the tokens of the path rather than its length. Deliberately publishes no tokens-per-character rate, because the ladder uses a repeated character that tokenises unlike any real path. Two cells were measured in separate harness invocations and reproduced across the gap.

• Startup floor re-verified on 2.1.241

  Four rounds, every cell measured twice, so eight readings per floor arm. Both came back single-valued with a zero noise floor: 22,793 tokens with tool search on and 37,862 with it off. Across three releases from 2.1.238 each arm moved by exactly six tokens, so the tool-search delta is bit-identical at 15,069 and the deferred-tools saving did not move at all. The 198- and 521-token intermittent components that dominated noise on this site for weeks did not appear, and that sentence is generated from this run’s own histogram rather than carried over from the previous corpus, which is how a stale description of a distribution used to survive a re-measurement. The loaded arm moved 643 tokens and is deliberately not attributed to the release, because it mixes the version change with two days of this machine’s own configuration.

• Startup floor on 2.1.238: unchanged, and the release checked rather than assumed

  Five rounds, every cell measured twice per round. Both isolated cells returned the same values as the 2.1.236 run, 22,787 tokens with tool search on and 37,856 with it off, so only the version stamp moved. The release is established from the Claude Code session transcripts, which stamp it on every message: 30 of 30 transcript directories read 2.1.238. That check exists because a corpus published earlier the same day turned out to span two releases after Claude Code updated mid-run, and had been dated from a two-token shift in an embedded floor cell instead. This run shows that floor does not move across that boundary at all, so the proxy was measuring something else. The trial it came from.

• Context measurement script

  The shell script that produces the startup-context figures, published in full. Measures its own noise floor first and refuses to report a number smaller than its own uncertainty. The trial it came from.

• What language a skill description is written in costs you

  Nine skills installed one at a time on 2.1.247, four rounds each, floor re-measured every round. Per description CHARACTER, descriptions with no CJK cost 0.363 to 0.482 tokens and majority-Chinese ones cost 0.887 to 1.048: the arms do not overlap and the two character-length ranges do overlap, so length is not the explanation. Per BYTE of UTF-8 the arms OVERLAP (0.363 to 0.482 against 0.459 to 0.513), so most of the per-character gap is the unit rather than the tokeniser, and the corpus says so; it matters anyway because the documented description cap counts characters. Includes a release bridge: addyosmani/agent-skills returns the identical 2,302 tokens measured on 2.1.233 fourteen releases earlier. The trial it came from.

• What a CLAUDE.md costs in Chinese against English

  One translation pair, the same instructions in both languages, byte sizes matching within 1.4 percent while character counts differ by 2.8x. Sixteen self-paired rounds on 2.1.247, one constant working-directory path because the startup floor moves with the path. English 604 tokens, Chinese 678, a modal difference of 1.12x. Three predictions were pre-registered before the run and all three are scored, including the one this site most needed to refute: that Chinese is cheaper because it uses fewer characters, which predicted 183 to 248 tokens against 678 measured. The arms overlap round to round and the corpus says so. The trial it came from.

• The command and subagent-definition prices re-verified fifteen releases on

  The oldest stamp on the calculator was 2.1.232, and once the staleness banner began rendering the lag per row it was the most conspicuous at fifteen patch releases behind. Twelve pooled rounds on 2.1.247, floor re-measured every round, 96 rows and none refused. Both published figures hold: 40.05 tokens per slash command at 200 (published 40.0) and 51.2 per subagent definition at 50 (published 51). Forty skills on a clean project came back at 1,568, identical to the token. The published constants were NOT changed, because the 6-token movements sit inside a 198-token intermittent component that this corpus publishes as a full distribution per cell. Four rounds would have got it wrong: pooling twelve moved two modes, one of them the per-command figure. The calculator stamp deliberately stays at 2.1.232, because that is the version the full ladders were measured on and this re-verified two points rather than the curve. The trial it came from.

• What a vendor skill costs when it actually fires

  JetBrains ships go-modern-guidelines as a Claude Code plugin with one skill, whose body tells the model to shell out to a pinned Go CLI before editing Go code. Four paired rounds on 2.1.247: the same task cost 45,582 to 45,785 tokens without the skill and 146,286 to 146,872 with it, about 3.2x, and the arms do not overlap. Every run in both arms wrote the file and the CLI genuinely installed, both recorded per run. The code produced was identical except for one word in a doc comment. The task deliberately has no modern-Go idiom to apply, so this measures the cost being paid rather than the benefit being absent, and the corpus says so. An earlier pass that ran without a permission mode measured permission-denial retries instead and is reported as invalid rather than dropped. The trial it came from.

• Does the vendor skill actually change the code

  The missing arm of the companion corpus, which could show the cost being paid but not the benefit delivered because its task had no modern idiom to apply. The task here is taken from the CLI own published ruleset: use slices.SortFunc with cmp.Compare instead of sort.Slice. Six paired rounds on 2.1.247. Without the skill the model wrote sort.Slice in 6 of 6 runs; with it, slices.SortFunc with cmp.Compare in 6 of 6. No run went the other way. Cost was 45,733 to 45,960 tokens without and 146,181 to 146,691 with, arms not overlapping. The full source of one run per arm is published so the machine-checked classification can be verified by eye. The trial it came from.

• The startup floor fell 24.5 percent on 2.1.251

  The largest move this site has recorded in the figure every other estimate is built on. Eight rounds, every cell measured twice, so sixteen readings per floor arm, and both arms came back SINGLE-VALUED: 17,207 tokens with tool search on and 32,400 with it off, against 22,801 and 37,870 on 2.1.246. The paired tool-search delta barely moved, 15,069 to 15,193, so whatever shrank is common to both arms rather than specific to tool search. An independent 4-round invocation minutes earlier returned 17,205 and 32,398 with an identical 15,193 delta; it is reported rather than pooled because its working directory was one character shorter and this site has measured that the floor moves with the path. No cause is asserted: nothing here establishes what was removed. The calculator constant moved with it. The trial it came from.

• A 229 KB skill body still costs nothing at startup

  This site published that a skill body is free at any size UP TO 28KB. mvanhorn/last30days-skill, 60,605 stars, ships one skill whose SKILL.md is 229,438 bytes, roughly 8.2x beyond that range, so the claim was being applied outside where it was measured. Twelve pooled rounds on 2.1.251, floor re-measured every round: installed and never invoked, the whole skill costs 101 tokens, which is the listing for one skill plus a 246-character description. The body contributes zero. The published model predicts 106.9 against 101 measured. Three predictions were pre-registered and the site own position was the one that held, extending its validated range 8.2x on a real pack rather than a synthetic fixture. The tool was deliberately NOT run: it auto-installs CLIs at runtime via npx and can extract browser cookies behind an opt-in, so nothing here prices what it costs when it fires. The trial it came from.

• The skills curve moved, but only past 200 skills

  Eleven pooled rounds and 220 runs on 2.1.251, none refused, re-measuring all three arms the calculator interpolates from. Six of nineteen cells moved and every move is DOWNWARD. The count curve is identical to the token at 1, 10, 40, 100, 150 and 200 skills, and lower at 300 (-197), 400 (-174), 600 (-180), 800 (-174) and 1000 (-174). The description curve moved at exactly one point, 800 characters, and the body-size curve did not move at all. So the change sits entirely in the collapsed regime past roughly 200 skills and is close to constant there rather than scaling with count. No cause is asserted. Worth reading beside the floor corpus from the same day: the absolute startup floor fell 24.5 percent over the same interval and thirteen of these nineteen paired cells did not move at all, which is what a paired design is for. The trial it came from.

• What an always-on output-style hook costs, and whether it measurably shortens replies

  ayghri/i-have-adhd, 25,853 stars and MIT, pitches itself as stopping your coding agent burying the answer. Its optional always-on mode is a SessionStart hook injecting the full ruleset every session. First the cap check, because this site measured that hook output above 10,000 UTF-16 units silently drops 78 percent: the payload is 6,598 units, 66 percent of the cap, so it arrives intact. Then twelve paired rounds on 2.1.251 running the real hook. COST: 1,475 tokens every session, single-valued in both arms, and that figure independently confirms the payload was not truncated because 6,598 units of prose predicts about 1,478. BEHAVIOUR: replies are shorter, median 855 characters against 712, about 0.83x, but THE ARMS OVERLAP, so it is a median difference across twelve rounds and not a resolved per-run effect. At four rounds they did not overlap and the headline would have been wrong. The trial it came from.

• The MCP curves did not move, in the week the floor fell 24.5 percent

  Ten pooled rounds on 2.1.251, none refused, re-measuring both MCP curves the calculator interpolates from. All six cells are IDENTICAL to 2.1.246 across five releases: a deferred tool holds at about 16 tokens and a loaded one at about 652. The contrast is the point. In the same week the startup floor fell 24.5 percent and the skills curve moved at every count past 200, and these did not move at all. No cause is offered for why one curve moved and another did not. Every single deviation from every mode across sixty cell readings is exactly plus or minus 897 tokens, this machine known intermittent component, and nothing else appeared, which is why a nine-of-ten cell is not a weak cell here. A null on a load-bearing calculator curve is worth as much as a change, and holding from 2.1.232 to 2.1.251 is a reason to trust these today and none at all to stop re-checking them. The trial it came from.

• The floor fell 24.5 percent and the path term did not move at all

  Your working-directory path sits in the system prompt and changes the startup floor. The sharpest version of that finding is a reciprocal: a 38-character path made of many one-character segments costs MORE than a 47-character path made of one repeated character, which rules out length directly rather than by inference. Re-measured on 2.1.251 across twelve pooled rounds: 17,102 against 17,082, a gap of exactly 20 tokens, identical to the gap measured on 2.1.241 ten releases earlier. Both paths fell by exactly the same 5,727, so the release change shifted this term rather than rescaling it. Every excursion in 24 readings was exactly 897, the known component. The ladder and the ten-path range were NOT re-run, so the calculator stamp deliberately stays at 2.1.241 and the corpus says so. The trial it came from.

• A 286-skill pack the synthetic curve got right, next to an 817-skill pack it got 5x wrong

  affaan-m/ECC, 244,921 stars and MIT. Two findings. FIRST, counting SKILL.md files on disk gives 898 and would have made this the largest pack measured here; 519 of those are documentation copies and the manifest declares only ./skills/, so what actually installs is 286. That is a 3.1x over-count and a concrete instance of the a_repo_is_not_always_what_the_plugin_ships limit this site already publishes. SECOND, the real 286 skills cost 9,210 tokens at startup, 32.2 per skill, single-valued in 12 of 12 rounds. This site synthetic curve interpolates to 8,635 at that count, so it is 6.7 percent out. The same curve was 5.15x LOW on a real 817-skill pack. No cause is asserted for the contrast: the packs differ in description length, median 258 against 396, and the 817 corpus own ablation showed identical descriptions costing 9,952 where unique ones cost 56,622, but nothing here measures which of those drives the gap. The trial it came from.

• What preserving PDF structure costs in tokens

  Feeding a PDF to a model means extracting text first, and the extractor changes both what survives and what you pay. firecrawl/pdf-inspector (17,147 stars, MIT) against pdftotext on the same 10-page PDF. WHAT SURVIVES: pdf-inspector keeps 60 markdown headings and 40 table rows; pdftotext keeps none of either. The decisive case is a colour table, reconstructed as pipe-delimited rows with each hex under its colour name, where pdftotext emits names and hex codes as disconnected lines and the association is lost. WHAT IT COSTS: 2,162 tokens against 1,986, so structure costs 176 extra tokens or 8.9 percent, measured across six paired rounds. Per-byte rates are nearly identical, 0.4459 against 0.4509, so the extra cost is the extra characters rather than markdown tokenising worse. SPEED: both vendor claims hold with margin and classification beats its own stated band, 0.6ms against a claimed 10 to 50. The OCR path was deliberately not run and the corpus says why. The trial it came from.

• What book-to-skill costs to have installed, and which of its own published figures reproduce

  virgiliojr94/book-to-skill (27,490 stars, MIT) turns a book into an agent skill and publishes unusually specific first-party numbers. WHAT IT COSTS: 159 tokens a session at a 398-character description, and three arms cost that same figure to the token, the whole 104-file cloned tree, SKILL.md alone, and SKILL.md truncated to its YAML frontmatter, so the 38,205-byte body is free while the skill is not invoked. WHICH OF THIS SITE’S OWN TWO ANSWERS IS RIGHT: the /cost/ model of 0.276 tokens per character is printed with a 30-to-300 validity bound and extending it here predicts 134, which is 25 low; the band measured on five real English descriptions installed one at a time, 0.363 to 0.482 per character, predicts 144 to 192 and brackets the measurement, whose own rate is 0.400. The page has been corrected to carry the bound. THE DOC’S EXTRACTION FIGURES REPRODUCE: raw pdftotext gives 119,237 against a claimed 119K and 229,371 against 229K; the current CLI reports 0.6 to 1.0 percent lower because its own clean_pdftotext strips headers and dehyphenates, a change the changelog dates after the table. THE CHAPTER COUNT DOES NOT: 28 against a documented 19 on Think Python 2, and 5 on Pro Git of which none is a chapter, four being Subversion revisions in git svn blame output and the fifth a stranded superscript from 2^80 in ordinary prose. No warning fires, because the guard needs a different detection method. On Pro Git the project’s own benchmark then treats that superscript as the target chapter and reaches a baseline of 73 percent of the whole book. On Think Python 2 the same benchmark misses its own published discovery figure by 60 percent, and that gap is NOT the miscount: suppressing all nine spurious numbers moves it ten tokens. The trial it came from.

• The floor fell 23 percent and a parallel subagent did not get cheaper

  A re-measurement of the calculator constant that had gone thirteen releases without one, and the answer is a null worth having. Forty runs, eight rounds, none failed, every arrival control held, and every run on ONE identical working directory so no difference carries a path term. THE SAME ROUNDS MEASURE BOTH THINGS, which is what makes the comparison worth anything: the trivial isolated floor came back 17,048 in all eight rounds with zero spread, against 21,890 to 22,787 from the same cell of the same harness on 2.1.238, a fall of about 23 percent. The per-subagent slope did not follow it. At the modal six parent turns it is 20,751 to 21,326, and the 2.1.238 figure the calculator still quotes, 20,581, sits just below that, 170 tokens or 0.8 percent from the nearest edge. So the release that made a session a quarter cheaper did not make an extra parallel agent cheaper. NO POINT ESTIMATE IS PUBLISHED and that is the data rather than the round count: no context value repeats across rounds in either fan-out cell, so guarded mode refuses, and more rounds would tighten the range without producing a repeating value. The inline control, which separates per-subagent from per-unit-of-work, is 166.3 to 169.7 tokens a file. THE LOADED ARM WAS DELIBERATELY NOT RUN, so configShare is marked not measured rather than carried forward, and the corpus declares in its own limits that it does not license a stamp bump: the calculator consumes a point per turn count and needs two conditions, this publishes one, and the estimator refuses a point in it. The trial it came from.

• Which deny-rule path shapes actually match a .env in the project root

  An earlier corpus here held the rule fixed at Read(./.env) and varied the ROUTE, closing with an open line about other glob patterns and absolute paths. This varies the RULE, which is the half a reader controls. Forty runs, ten arms, four rounds each, none excluded. ALL SIX RELATIVE SHAPES BLOCK 4 of 4: ./.env, .env, **/.env, *.env, ./*.env and ./.env*, and so does the documented absolute form //path. TWO DO NOT. A rule with ONE leading slash, which looks absolute, never blocks: the Read tool succeeds and returns the file in all four rounds. That is documented behaviour and the documentation warns about it in a Warning box, so it is not reported as a defect; what is published is that the read comes back with no error. The same file named through /tmp, which on macOS is a symlink to the real system temporary directory, in the documented absolute form also did not block, reported as an observation and not as a doc contradiction, since the page describes Claude ACCESSING a symlink rather than a rule WRITTEN through one. THE INSTRUMENT IS THE SESSION TRANSCRIPT, NOT A CANARY, and that is the real contribution: on 2 of 40 runs the model read the file and then chose not to print the secret, so canary scoring would have called those protected. The model opened the deny rule itself in 23 of 40 runs, which is the largest confound in the design and is published as one. permission_denials was empty on all 40 runs including all 28 blocked ones, so it observes nothing and nothing is concluded from it. The trial it came from.