{
 "measurement": "Whether JetBrains' go-modern-guidelines skill changes the code on a task where its own ruleset says it should",
 "claude_code_version": "2.1.247 (Claude Code)",
 "why": "The companion corpus gomg-skill-fires measured this skill tripling the cost of a Go task and leaving the code identical apart from a doc comment. It named its own gap: that task summed ints with a range loop, which is ALREADY idiomatic, so it could show the cost being paid but not the benefit being delivered. This is that missing arm, and it changes the conclusion.",
 "the_idiom_is_the_tools_own": {
  "rule_id": "slices_sort_func",
  "rule_text_verbatim": "Use slices.SortFunc with cmp.Compare instead of sort.Slice for typed comparisons.",
  "how_it_was_obtained": "printed by the CLI's own `list` subcommand, so the task is taken from the tool's published rules rather than invented"
 },
 "method": {
  "task": "Write a Go file called sortpeople.go with a Person struct having Name string and Age int, and a function SortByAge(people []Person) that sorts the slice by Age ascending. Just write the file. Do not explain.",
  "population": "one project, one prompt, a constant directory created and removed per run",
  "rounds": 6,
  "detector": "which symbols appear in the written file: sort.Slice is the pre-1.21 idiom, slices.SortFunc with cmp.Compare the modern one. Machine-checked, not judged, and the full source of one run per arm is published so the classification can be checked.",
  "controls": [
   "paired: both arms in the same round, same prompt, same constant directory",
   "--setting-sources project asserted onto the command line",
   "BOTH arms get the same --permission-mode bypassPermissions, because -p denies Write and Bash and an earlier corpus was invalidated by exactly that",
   "SKILL.md asserted byte-identical to upstream every run",
   "the file must exist and be non-empty; recorded per run and true in 12 of 12",
   "the CLI's own cache directory recorded per run",
   "result_guard refuses an unusable result rather than recording a zero"
  ]
 },
 "results": {
  "headline": {
   "verdict": "THE SKILL DELIVERS, AND IT IS NOT SUBTLE.",
   "no_skill_wrote_sort_Slice": "6 of 6",
   "skill_wrote_slices_SortFunc": "6 of 6",
   "behavioural_arms_separate_completely": true,
   "cost_without_skill": [
    45733,
    45960
   ],
   "cost_with_skill": [
    146181,
    146691
   ],
   "cost_arms_overlap": false,
   "multiplier": [
    3.18,
    3.21
   ],
   "plain_statement": "Given a task the tool's own ruleset covers, the arm without the skill wrote the pre-1.21 sort.Slice in 6 of 6 runs and the arm with it wrote slices.SortFunc with cmp.Compare in 6 of 6. No run went the other way. The cost was 45,733-45,960 tokens without and 146,181-146,691 with, about 3.2x, arms not overlapping."
  },
  "arms": {
   "no_skill": {
    "rounds": 6,
    "context_min": 45733,
    "context_max": 45960,
    "wrote_file_every_round": true,
    "used_sort_Slice": 6,
    "used_slices_SortFunc": 0,
    "used_cmp_Compare": 0,
    "example_source": "package main\n\nimport \"sort\"\n\n// Person represents a named individual with an age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
   },
   "skill": {
    "rounds": 6,
    "context_min": 146181,
    "context_max": 146691,
    "wrote_file_every_round": true,
    "used_sort_Slice": 0,
    "used_slices_SortFunc": 6,
    "used_cmp_Compare": 6,
    "example_source": "package main\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person is a named person with an age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by ascending Age.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
   }
  }
 },
 "known_limits": {
  "RETIRES THE GAP IN THE COMPANION CORPUS": "gomg-skill-fires could not show a benefit because its task had no modern idiom to apply. That limit is RETIRED by this corpus, and anyone reading the two together should read this one second. The pair says: the skill costs about 3.2x whether or not there is anything to fix, and when there is, it fixes it every time.",
  "ONE RULE OF TWENTY-FIVE": "The CLI's list printed roughly two dozen rules. Exactly one is tested here. Nothing establishes that the other rules fire as reliably, and a rule the model already follows would show no difference by construction.",
  "the_old_idiom_is_not_wrong": "sort.Slice compiles, works, and is not deprecated. The no-skill arm produced correct code. What separates the arms is modernity, not correctness, and no claim is made that the modern form is faster or safer here.",
  "nothing_was_compiled_or_tested": "Neither arm's file was compiled or run. The classification is textual.",
  "the_cost_is_round_trips": "The skill adds tool calls, and each tool call re-sends the whole conversation, so the 3.2x is that mechanism rather than a large injected payload.",
  "bypass_permissions_is_not_the_default_experience": "Both arms bypass permission prompts so the task completes unattended. Interactively a user is asked to approve the CLI and may decline, in which case neither the cost nor the benefit lands.",
  "one_machine_one_model": "Opus on Claude Code 2.1.247, project scope, one machine."
 },
 "runs": [
  {
   "arm": "no_skill",
   "round": 0,
   "status": "ok",
   "context": 45930,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": true,
    "uses_slices_SortFunc": false,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": false,
    "imports_sort": true,
    "imports_slices": false
   },
   "source": "package main\n\nimport \"sort\"\n\n// Person represents a named individual with an age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
  },
  {
   "arm": "skill",
   "round": 0,
   "status": "ok",
   "context": 146181,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": false,
    "uses_slices_SortFunc": true,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": true,
    "imports_sort": false,
    "imports_slices": true
   },
   "source": "package main\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person is a named person with an age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by ascending Age.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
  },
  {
   "arm": "no_skill",
   "round": 1,
   "status": "ok",
   "context": 45960,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": true,
    "uses_slices_SortFunc": false,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": false,
    "imports_sort": true,
    "imports_slices": false
   },
   "source": "package main\n\nimport \"sort\"\n\n// Person holds a person's name and age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
  },
  {
   "arm": "skill",
   "round": 1,
   "status": "ok",
   "context": 146472,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": false,
    "uses_slices_SortFunc": true,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": true,
    "imports_sort": false,
    "imports_slices": true
   },
   "source": "package main\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person describes a named individual and their age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by ascending Age.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
  },
  {
   "arm": "no_skill",
   "round": 2,
   "status": "ok",
   "context": 45929,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": true,
    "uses_slices_SortFunc": false,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": false,
    "imports_sort": true,
    "imports_slices": false
   },
   "source": "package main\n\nimport \"sort\"\n\n// Person holds a person's name and age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
  },
  {
   "arm": "skill",
   "round": 2,
   "status": "ok",
   "context": 146658,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": false,
    "uses_slices_SortFunc": true,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": true,
    "imports_sort": false,
    "imports_slices": true
   },
   "source": "package main\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person is a named individual with an age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by ascending Age.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
  },
  {
   "arm": "no_skill",
   "round": 3,
   "status": "ok",
   "context": 45929,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": true,
    "uses_slices_SortFunc": false,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": false,
    "imports_sort": true,
    "imports_slices": false
   },
   "source": "package main\n\nimport \"sort\"\n\n// Person holds a person's name and age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
  },
  {
   "arm": "skill",
   "round": 3,
   "status": "ok",
   "context": 146501,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": false,
    "uses_slices_SortFunc": true,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": true,
    "imports_sort": false,
    "imports_slices": true
   },
   "source": "package main\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person is a named individual with an age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by ascending Age.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
  },
  {
   "arm": "no_skill",
   "round": 4,
   "status": "ok",
   "context": 45733,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": true,
    "uses_slices_SortFunc": false,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": false,
    "imports_sort": true,
    "imports_slices": false
   },
   "source": "package main\n\nimport \"sort\"\n\n// Person represents a person with a name and an age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
  },
  {
   "arm": "skill",
   "round": 4,
   "status": "ok",
   "context": 146691,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": false,
    "uses_slices_SortFunc": true,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": true,
    "imports_sort": false,
    "imports_slices": true
   },
   "source": "package sortpeople\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person is a named individual with an age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by ascending Age.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
  },
  {
   "arm": "no_skill",
   "round": 5,
   "status": "ok",
   "context": 45733,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": true,
    "uses_slices_SortFunc": false,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": false,
    "imports_sort": true,
    "imports_slices": false
   },
   "source": "package main\n\nimport \"sort\"\n\n// Person represents a person with a name and an age.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age in ascending order.\nfunc SortByAge(people []Person) {\n\tsort.Slice(people, func(i, j int) bool {\n\t\treturn people[i].Age < people[j].Age\n\t})\n}\n"
  },
  {
   "arm": "skill",
   "round": 5,
   "status": "ok",
   "context": 146645,
   "wrote_file": true,
   "cli_cache_exists_after": true,
   "refused_because": null,
   "classification": {
    "uses_sort_Slice": false,
    "uses_slices_SortFunc": true,
    "uses_slices_SortStableFunc": false,
    "uses_cmp_Compare": true,
    "imports_sort": false,
    "imports_slices": true
   },
   "source": "package main\n\nimport (\n\t\"cmp\"\n\t\"slices\"\n)\n\n// Person is a named individual with an age in years.\ntype Person struct {\n\tName string\n\tAge  int\n}\n\n// SortByAge sorts people in place by Age, ascending.\nfunc SortByAge(people []Person) {\n\tslices.SortFunc(people, func(a, b Person) int {\n\t\treturn cmp.Compare(a.Age, b.Age)\n\t})\n}\n"
  }
 ]
}