A 19,700-star diagram skill drew once in four unattended runs. One file fixes it

cathrynlavery/diagram-design is a Claude Code skill that draws editorial diagrams as self-contained HTML. It was top of GitHub Trending’s weekly board when I scraped it, with about 15,600 stars gained that week, and stood at 19,728 total when I measured it on 2026-08-17.

I picked it because its README makes a specific, checkable claim about how it behaves at runtime, and because a diagram skill is a good test of something this site has already published: a skill body is free until you invoke it. Nobody here had measured what happens when you invoke it.

To be clear about what that claim is and is not: the pack makes no claim about token cost anywhere. Every use of the word “token” in its README and SKILL.md is a style token, a colour or a typeface. What it does claim is a routing behaviour, that the agent reads only the one type reference it needs, and that is what gets scored here.

So I installed it and ran it, four rounds each way, against the same request with and without the skill.

What it costs to have and not use, which is a question I cannot answer

I had a number here. It was wrong to publish it, and the way it turned out wrong is worth more than the number was.

Four rounds put the cost of installing this pack at 249 tokens. That looked solid: three rounds identical, the fourth explained by a component this site has documented. This site had also already published the same pack at 274, from a different harness, and I set out to explain the gap.

It is not explainable, because there is nothing stable to explain. Twelve further readings of the same pack, same commit, same isolation, returned 70 to 967 tokens — a 12.8x spread. 249, 274 and 466 are all draws from that.

So: installing this pack is cheap, and that is the entire honest content of this section. One skill sits inside the noise floor, which this site had already said before I ignored it. A 24-skill pack measured the same day held to 0.58x of its minimum, so the problem is the size of the signal, not the instrument.

For scale, and these are counted off disk rather than estimated: 2,014,140 bytes across 148 files, including a 37,571-byte SKILL.md and 40 reference documents. Almost all of that stays on disk until you ask for a diagram. What loads at every session start, diagram or not, is the frontmatter of that one SKILL.md: its name and its 579-character description. That listing is the thing the readings above tried and failed to price. The body and the 40 references are charged only when something triggers the skill.

Then I asked for a diagram

The request, identical in every round: a three-stage deploy pipeline as a self-contained HTML file. Nothing exotic, and nothing that needs the skill.

Round	Plain Claude Code	With the skill installed
1	50,784 tokens, 2 turns, 6,805-byte file	456,492 tokens, 11 turns, 12,075-byte file
2	51,515 tokens, 2 turns, 8,731-byte file	330,555 tokens, 11 turns, no file
3	338,892 tokens, 10 turns, 13,547-byte file	207,173 tokens, 8 turns, no file
4	144,887 tokens, 5 turns, 11,751-byte file	196,096 tokens, 9 turns, no file

Plain Claude Code delivered a diagram four times out of four. The skill delivered once.

The three that produced nothing spent 733,824 tokens between them.

Be careful what that table establishes, because there are two different comparisons here and only one of them is solid.

Skill against plain Claude Code is 1 of 4 against 4 of 4, a one-tailed hypergeometric p of 0.071. This site has refused to call a result at that bar before and refuses here. Read it as suggestive and no more.

The comparison that is established is a different one: the skill without the marker file against the skill with it, 1 of 4 against 8 of 8, p = 0.018. That is the finding this post’s advice rests on, and it says nothing about how the skill compares to plain Claude Code. Those eight rounds are all skill rounds; no plain-Claude round is in that test.

They all stopped, but not all at the same gate

All three transcripts stop before drawing anything, and the cause is written into the skill — but not at one gate. SKILL.md has two separate pause points, and the three misses do not all stop at the same one. Round 2 ends on the branding question and nothing else:

Before I draw — this is the first diagram in this project, and the skill’s style guide is still at its shipped default (white-smoke paper, jet-black ink, atomic-tangerine accent). Want me to brand it first?

Round 3 opens identically but blocks on both pause points, the branding gate and sign-off on its own plan: “say ‘go’ with your choice and I’ll build it”. Round 4 never blocks on the branding question at all. It answers that gate itself, recommends “(e) proceed with the default”, lays out its plan, and closes with “Proceeding on default styling unless you redirect” — then ends the turn without drawing anything. It cleared the first gate and stopped at the second.

And the second pause point tells it not to do that. Section 3, “Confirm before drawing”, says to state the plan first, then: “If the user is reachable, let them redirect before you draw; if not, proceed and note the assumptions beside the deliverable.” There was no user to reach. The instruction written for exactly this case is to proceed, and it is the one instruction in the pair that anticipates an unattended run. It was not followed.

SKILL.md section 0 is a first-run gate. It tells the agent to check the project for a .diagram-design marker, and if there is none and the style tokens are still the shipped defaults, to “pause and ask the user” which brand to use. In an interactive session you answer and carry on. In an unattended one there is nobody to answer. Two of the three misses ended right there, having already read the skill, listed the references and loaded between three and five of them.

It does not fire every time, and it does not fail the same way twice. All four rounds read style-guide.md, the file section 0 sends them to. One drew anyway. One asked and stopped. One asked two questions and stopped. One cleared the gate itself, said it was proceeding, and still stopped. Section 0 is a gate the agent meets reliably and resolves inconsistently, which is far worse to plan around than one that always fires.

That is a design decision, not a bug, and for a person at a terminal it is arguably the right one. It is a serious problem for anyone putting this in a script, a hook or a CI job, which is exactly where “19,700 stars” tends to lead people.

The fix, tested rather than repeated

SKILL.md says a marker skips the first-run gate, and references/profiles.md agrees: its instruction to “ensure default.md exists” is defined in that same file as a check-then-create procedure, not a precondition. The two documents do not contradict each other, and an earlier draft of this post said they did, which was my misreading rather than their ambiguity.

What was still worth checking is whether the bare marker is enough in practice on a fresh install that has no profile library at all. Telling you to drop a marker on the strength of the docs would be exactly the untested advice this site exists not to publish. So I ran both. Four rounds each.

Create a file called .diagram-design in the project root containing one line:

profile: default

That alone took it from 1 of 4 to 4 of 4. Seeding the profile library as well also delivered 4 of 4, so the extra step is not required for this case and SKILL.md’s wording is the one that holds.

One thing I cannot explain, and am not going to pretend to. The marker is documented as skipping the first gate, section 0. It says nothing about section 3, the confirm-before-drawing step that stopped round 4. Yet all eight marker rounds delivered. So the marker fixes delivery in practice and the mechanism I can point at only covers one of the two pause points. The advice stands on the measurement, not on my account of why it works.

The fix is not a discount. With the bare marker the rounds ran 506,459 to 785,618 tokens over 14 to 19 turns, more than the gated ones, because they finish the work instead of stopping to ask.

Why there is no “the skill costs Nx more” here

The obvious headline would be a multiplier. It is not available, and the reason matters more than the number would have.

Look at the plain-Claude column again: 50,784, 51,515, 144,887 and 338,892 tokens for the same prompt. That is a 6.7x spread inside a single arm, from two rounds that wrote the file immediately and two that iterated. The skill’s arm has one delivering round. With no repeated value in either arm there is no mode to divide, which is the reason the harness itself records, so any multiplier I quoted would be a choice of which two rounds to compare.

The corpus records this as a refusal rather than an omission: every comparison carries multiplier: null with the reason attached, and the observed ranges beside it. Four runs per cell is the rule here precisely because one run would have produced a confident 9x that the next three rounds destroy.

What can be said without a multiplier is the part that matters anyway: one arm delivered every time and the other did not.

Progressive disclosure, precisely

The README’s routing table promises this:

No matter how many types exist, the agent only reads the one you need.

Across the four rounds it opened three, five, three and four files from references/. Of those, the type references specifically were two, three, two and three — never the promised one. The rest were style-guide.md, which section 0 sends it to, in all four rounds, and output-spec.md in one.

Being fair about what that does and does not show: the architectural claim holds comfortably. It read two or three of twenty-seven, not twenty-seven, and that is the whole point of splitting the references out. What does not survive is the literal sentence, and part of the gap is the README’s own fault rather than the agent’s, because SKILL.md section 0 instructs it to open style-guide.md and the routing table omits that.

What the output is actually like

Both diagrams are published exactly as generated, from the same round:

• what the skill produced — 12,075 bytes
• what plain Claude Code produced — 6,805 bytes

The skill’s version is better. It is denser, it uses a consistent grid, it labels the arrows properly, and it looks like something a designer touched. If you want that specific editorial look, this is a real shortcut to it.

One thing to know before opening it, and the pack tells you itself. The skill’s file requests a webfont from fonts.googleapis.com; plain Claude Code’s file makes no external request at all. That is disclosed, not hidden: SKILL.md says to “Always produce a single self-contained .html file:” and the very next line reads “Embedded CSS (no external except Google Fonts)”. An earlier draft of this post presented that as a contradiction, which it is not — I had quoted the lead-in and stopped before the line granting the exception.

It still matters practically. Opened offline the typography falls back, and opening the file tells a third party you opened it. Worth knowing if these go to a client.

I am publishing the file unmodified. Stripping the font link would have removed the evidence for the paragraph you just read.

A smaller thing, dated because it may be fixed: on 2026-08-17 the repository description said “29 editorial diagram types”. The README states 27 on eight separate lines, SKILL.md says “Twenty-seven visual types”, and 27 type-*.md files ship. Only the one-line blurb is out.

Should you install it

Yes, if you want that editorial look and you drive Claude Code by hand. Sitting installed and unused it is cheap enough that the cost is not measurable against the noise floor, the design system is genuinely opinionated in a useful way, and the output is better than what you get by asking.

Put the marker file in first, always. It costs one line and it is the difference between 1 of 4 and 4 of 4 unattended.

Do not put it in an unattended pipeline without the marker. Three of my four rounds spent an average of 244,608 tokens each and stopped at one of the skill’s two pause points with nobody there to clear it. Two stopped at the branding gate; the third cleared that one itself and stopped at the confirm-before-drawing step instead.

And do not install it to save money. Nothing measured here suggests it makes diagram work cheaper. The rounds that finished were consistently long.

What this does not settle

One prompt, one diagram. A three-stage pipeline diagram. A different type, a longer brief or a branded project would route differently and read different references.

The first-run gate fires once per project. Every round here starts in a fresh empty directory, so every round meets it. Someone working in one project meets it once, and the cost of a second diagram in the same project is not measured here.

Headless is not interactive. These are non-interactive runs. A person at a terminal answers the branding question and continues, so the delivery rate is a statement about unattended use and not about a human using the skill.

No multiplier, as above. Both arms varied too much within themselves, and the skill’s arm has one delivering round.

One machine, one model, one day. Opus on Claude Code 2.1.233, macOS, pack pinned at commit 09df49d.

Controls

Isolation. Every arm runs with --setting-sources project asserted onto the command line, so this machine’s own skills, plugins and CLAUDE.md cannot enter any measurement.

Paired floors. The empty floor is re-measured every round and every delta is against that round’s floor, not a remembered one.

Arrival. The trivial arms must reply exactly OK. On a third-party skill that doubles as a prompt-injection check: a description that tried to steer the model would show up as a reply that is not exactly OK. Published per run.

Write permission, and why it is a control. All four arms share the same --allowedTools and --permission-mode. An earlier run of this harness lacked them, so Claude Code silently denied every write, and four diagram rounds were recorded as cheap successes that had produced no file at all. A run that produced nothing is now recorded as a non-delivery rather than as a cheap run.

Reads are counted from Bash as well as from the Read tool, because the first version of this control was blind to half of one round. Round 4 opened style-guide.md and type-process.md with sed -n '1,120p' rather than the Read tool, and a capture watching only Read missed both — understating the very thing this post measures, in the direction that flatters the tool. Any command that pages a file now counts, and every round’s reads were re-derived from its transcript with the corrected logic.

Disclosure evidence covers every round, including the three that drew nothing. Those rounds still opened between three and five reference files before stopping, and scoring the claim on the single delivering round would have shrunk the evidence to a quarter of what exists. The file lists come from the session transcripts’ tool-use records with reads separated from writes, so the file the agent created cannot be counted as a file it read.

Security scan before running anything. The pack ships no package.json, setup.py or Makefile, so there are no install hooks; no credential or environment access; subprocess appears only in its own test harness driving git; and base64 only in the draw.io decoder, which is what draw.io files require.

Every round, with its tool calls, its file reads and its verdict, is in diagram-skill-cost-2-1-233.json.