{
 "measurement": "Whether ponytail changes what the model writes, on one small task, 2.1.243",
 "claude_code_version": "2.1.243",
 "trial_date": "2026-08-25",
 "subject": {
  "repo": "DietrichGebert/ponytail",
  "commit": "2ed6c52",
  "version": "4.9.0",
  "its_claim": "The best code is the code you never wrote.",
  "cost_corpus": "/ponytail-session-cost-2-1-243.json"
 },
 "task": "Write a Python function in temp_convert.py that converts a temperature from Celsius to Fahrenheit. Create the file.",
 "headline": "Docstrings separate perfectly: the baseline wrote one in 8 of 8 runs, ponytail in 0 of 8. Line counts do NOT separate: they overlap, because one ponytail run added assert self-checks.",
 "the_headline_changed_at_round_six": "After FIVE rounds ponytail was 2 lines every time and the baseline 3 or 6, so the line counts did not overlap and that was going to be the headline. Three more rounds produced a single 7-line ponytail run. The five-round headline would have been false. This is recorded because a reader deserves to know the claim was nearly stronger than the data.",
 "method": {
  "task_choice": "A genuine one-liner with room to over-build. A task with no room could not have separated the arms, and a null would have been uninformative.",
  "arms": "baseline: empty project, no settings file. ponytail: same, with .claude/settings.json registering hooks/ponytail-activate.js on SessionStart, mode full.",
  "isolation": "--setting-sources project asserted onto the command line on every run.",
  "tools": "--allowedTools Write Read Edit, so the model can create the file and nothing else.",
  "metric": "DECLARED BEFORE ANY RUN. Primary: non-blank, non-comment-only lines across every file created, excluding .claude. Also recorded: file count, class present, docstring present, test file present.",
  "not_measured": "Whether shorter code is BETTER. This counts lines and features, not quality.",
  "controls": {
   "arrival": "Each round also asked, with the hook installed, which ponytail level was active. 8 of 8 replies named a level, so a null could not be confused with a hook that never ran.",
   "settings_presence": "Asserted in code per arm.",
   "usable": "result_guard rejects a run with empty modelUsage or a reported error."
  }
 },
 "results": {
  "baseline": {
   "runs": 8,
   "with_docstring": 8,
   "with_class": 0,
   "with_test_file": 0,
   "code_lines_range": [
    3,
    6
   ],
   "code_lines_histogram": {
    "3": 2,
    "6": 6
   },
   "distinct_outputs": 3,
   "outputs": [
    "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32",
    "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}C = {celsius_to_fahrenheit(c)}F\")",
    "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}\u00b0C = {celsius_to_fahrenheit(c)}\u00b0F\")"
   ]
  },
  "ponytail": {
   "runs": 8,
   "with_docstring": 0,
   "with_class": 0,
   "with_test_file": 0,
   "code_lines_range": [
    2,
    7
   ],
   "code_lines_histogram": {
    "2": 7,
    "7": 1
   },
   "distinct_outputs": 2,
   "outputs": [
    "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32",
    "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    assert celsius_to_fahrenheit(0) == 32\n    assert celsius_to_fahrenheit(100) == 212\n    assert celsius_to_fahrenheit(-40) == -40\n    print(\"ok\")"
   ]
  },
  "code_lines_overlap": true,
  "algorithm_identical_in_both_arms": true,
  "what_ponytail_removed": "The docstring, in every run. The baseline also added an if __name__ == \"__main__\" demo block in 6 of 8 runs.",
  "the_one_long_ponytail_run": "7 lines, and it added assert-based self-checks rather than a demo. ponytail's own ruleset says non-trivial logic leaves ONE runnable check behind, so that run is the tool following a different rule of its own, not the tool failing.",
  "arrival_control": {
   "named_a_level": 8,
   "of": 8
  },
  "runs_excluded": 0
 },
 "known_limits": {
  "one_task_one_shape": "A two-line arithmetic conversion. Nothing here shows what happens on a task with real architecture to choose, which is where a 'do not over-build' instruction would matter most and is untested.",
  "line_counts_do_not_separate": "Stated as a limit as well as a result, because the tempting summary is 'ponytail makes the model write less code' and on this task the line counts overlap.",
  "docstrings_may_be_the_easy_case": "Removing a docstring is the cheapest possible way to satisfy a terseness instruction. That it happened 8 of 8 does not show the same reliability on harder choices.",
  "quality_not_assessed": "Both arms produced the same correct formula. Whether losing the docstring is a good trade is a judgement this trial does not make.",
  "one_machine_one_release": "Claude Code 2.1.243, one machine, mode full, project scope only."
 },
 "runs": [
  {
   "reply": "Created `temp_convert.py:1` with `celsius_to_fahrenheit(celsius)`, which returns `celsius * 9 / 5 + 32`.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 3,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n",
   "cell": "baseline",
   "round": 0,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one-liner.\n\nSkipped: input validation, `__main__` self-check. Add when it takes untrusted input or grows a branch.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 0,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 0,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py` with `celsius_to_fahrenheit(celsius)` using the standard `C \u00d7 9/5 + 32` formula, plus a small `__main__` demo block printing a few sample conversions.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 6,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}C = {celsius_to_fahrenheit(c)}F\")\n",
   "cell": "baseline",
   "round": 1,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one line.\n\nskipped: validation and a test, it's a one-liner formula. Add if it starts parsing strings.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 1,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 1,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py:1` with `celsius_to_fahrenheit(celsius)` returning `celsius * 9 / 5 + 32`, plus a small `__main__` demo printing a few sample conversions.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 6,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}\u00b0C = {celsius_to_fahrenheit(c)}\u00b0F\")\n",
   "cell": "baseline",
   "round": 2,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py` created.\n\nSkipped: type hints, validation, `__main__` self-check \u2014 one-liner arithmetic, YAGNI. Add a test when it grows a branch.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 2,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 2,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py` with `celsius_to_fahrenheit(celsius)` returning `celsius * 9 / 5 + 32`, plus a small `__main__` demo printing a few sample conversions.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 6,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}\u00b0C = {celsius_to_fahrenheit(c)}\u00b0F\")\n",
   "cell": "baseline",
   "round": 3,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one line, no test (trivial arithmetic).",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 3,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 3,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py:1` with `celsius_to_fahrenheit(celsius)`, which returns `celsius * 9 / 5 + 32`.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 3,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n",
   "cell": "baseline",
   "round": 4,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one-liner.\n\nskipped: input validation, type hints, self-check (trivial arithmetic). Add a test when it grows a branch.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 4,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 4,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py` with `celsius_to_fahrenheit(celsius)` using the standard `C \u00d7 9/5 + 32` formula, plus a small `__main__` demo printing a few sample conversions.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 6,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}\u00b0C = {celsius_to_fahrenheit(c)}\u00b0F\")\n",
   "cell": "baseline",
   "round": 0,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one-liner.\n\nskipped: validation/tests, add when it takes untrusted input.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 0,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 0,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py` with `celsius_to_fahrenheit(celsius)` returning `celsius * 9 / 5 + 32`, plus a small `__main__` block printing a few sample conversions.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 6,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}\u00b0C = {celsius_to_fahrenheit(c)}\u00b0F\")\n",
   "cell": "baseline",
   "round": 1,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one line plus a `__main__` self-check.\n\nSkipped: type hints, input validation, F\u2192C reverse. Add when a caller passes non-numeric input or needs the round trip.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 7,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    assert celsius_to_fahrenheit(0) == 32\n    assert celsius_to_fahrenheit(100) == 212\n    assert celsius_to_fahrenheit(-40) == -40\n    print(\"ok\")\n",
   "cell": "ponytail",
   "round": 1,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 1,
   "status": "ok"
  },
  {
   "reply": "Created `temp_convert.py` with `celsius_to_fahrenheit(celsius)` \u2014 returns `celsius * 9 / 5 + 32`, plus a small `__main__` demo printing a few sample conversions.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 6,
   "has_class": false,
   "has_docstring": true,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(celsius):\n    \"\"\"Convert a temperature from Celsius to Fahrenheit.\"\"\"\n    return celsius * 9 / 5 + 32\n\n\nif __name__ == \"__main__\":\n    for c in (-40, 0, 37, 100):\n        print(f\"{c}C = {celsius_to_fahrenheit(c)}F\")\n",
   "cell": "baseline",
   "round": 2,
   "status": "ok",
   "hook_installed": false
  },
  {
   "reply": "`temp_convert.py:1` \u2014 one line.\n\nskipped: input validation, tests. Trivial arithmetic; add a check if it grows a unit-parsing or rounding branch.",
   "turns": 2,
   "files": [
    "temp_convert.py"
   ],
   "file_count": 1,
   "code_lines": 2,
   "has_class": false,
   "has_docstring": false,
   "has_test_file": false,
   "content": "def celsius_to_fahrenheit(c):\n    return c * 9 / 5 + 32\n",
   "cell": "ponytail",
   "round": 2,
   "status": "ok",
   "hook_installed": true
  },
  {
   "reply": "full",
   "turns": 1,
   "cell": "arrival",
   "round": 2,
   "status": "ok"
  }
 ]
}