{ "measurement": "Whether six free-tier chat models can count 120 working days from a fixed start date, asked twice each in separate chats", "ran_on": "2026-08-17", "surface": "duck.ai, DuckDuckGo's free AI chat. No account, no API, no payment. Six models from four vendors are offered there and all six were asked.", "prompt": "A project started on 3 March 2026 and ran for 120 working days, Monday to Friday, ignoring public holidays. What date did it finish?", "method": { "shape": "One ordinary prompt, typed by hand, into a New Chat per run so no previous answer sat in context. Two rounds of all six models, twelve runs total. Nothing scripted and nothing automated against the vendors' services.", "model_labels": "Recorded from the label the UI printed above each answer and screenshotted, rather than scraped from the DOM, because a product can change what sits behind a label without saying so.", "rubric_fixed_before_running": "The question has two defensible readings and BOTH were declared correct before any model was asked, so no answer could be graded after the fact to suit a conclusion. Reading A, the start date counts as working day 1, gives 2026-08-17. Reading B, 120 working days after the start, gives 2026-08-18.", "ground_truth_is_computed": "Both readings are produced by the assembler from Python's datetime, not typed in. The assembler also decides every weekday, every month's working-day count and every date-offset claim quoted from a run, so the scoring cannot be a second opinion.", "what_was_entered_by_hand": "Per run: the finishing date the model gave, the weekday, offset and month-count claims it made, and which method it showed. The verbatim answer is stored beside them so a reader can check every one of those readings. Where a run showed no working, its setup is recorded as unknown rather than as correct or incorrect." }, "ground_truth": { "start": "2026-03-03", "start_weekday": "Tuesday", "working_days": 120, "reading_a_start_counts_as_day_1": "2026-08-17", "reading_a_weekday": "Monday", "reading_b_working_days_after_start": "2026-08-18", "reading_b_weekday": "Tuesday", "why_reading_b_is_a_whole_number_of_weeks": "120 working days is exactly 24 five-day weeks, so reading B is 168 calendar days after the start and therefore lands on the same weekday as the start. Three runs used that shortcut and it is correct; two runs used it and then executed the addition wrongly." }, "results": { "runs_total": 12, "runs_correct": 7, "models_total": 6, "models_correct_on_both_runs": 3, "models_that_gave_the_same_answer_twice": 3, "by_model": { "GPT-5.4 nano": { "answers": [ "2026-07-02", "2026-09-17" ], "correct_runs": 0, "gave_the_same_answer_twice": false, "spread_days": 77 }, "GPT-5.4 mini": { "answers": [ "2026-08-25", "2026-08-22" ], "correct_runs": 0, "gave_the_same_answer_twice": false, "spread_days": 3 }, "Claude Haiku 4.5": { "answers": [ "2026-08-17", "2026-08-17" ], "correct_runs": 2, "gave_the_same_answer_twice": true, "spread_days": 0 }, "Mistral Small 4": { "answers": [ "2026-08-27", "2026-08-18" ], "correct_runs": 1, "gave_the_same_answer_twice": false, "spread_days": 9 }, "gpt-oss 120B": { "answers": [ "2026-08-18", "2026-08-18" ], "correct_runs": 2, "gave_the_same_answer_twice": true, "spread_days": 0 }, "Gemma 4 31B": { "answers": [ "2026-08-18", "2026-08-18" ], "correct_runs": 2, "gave_the_same_answer_twice": true, "spread_days": 0 } }, "wrong_answers": 5, "wrong_answers_a_reader_could_refute_without_knowing_the_right_one": 4, "runs_by_method": { "month_by_month": { "count": 2, "runs": [ "Claude Haiku 4.5 r1", "Claude Haiku 4.5 r2" ] }, "none_shown": { "count": 4, "runs": [ "GPT-5.4 mini r1", "GPT-5.4 mini r2", "gpt-oss 120B r1", "gpt-oss 120B r2" ] }, "setup_only": { "count": 1, "runs": [ "GPT-5.4 nano r1" ] }, "shortcut_24_weeks": { "count": 5, "runs": [ "GPT-5.4 nano r2", "Mistral Small 4 r1", "Mistral Small 4 r2", "Gemma 4 31B r1", "Gemma 4 31B r2" ] } }, "runs_that_showed_a_setup": 8, "runs_that_showed_nothing": 4, "runs_that_showed_a_setup_and_got_the_setup_right": 8, "wrong_answers_that_showed_a_setup": 3, "wrong_answers_that_showed_nothing": 2, "runs_that_said_in_words_whether_the_start_counts": { "count": 2, "runs": [ "GPT-5.4 nano r1", "GPT-5.4 nano r2" ], "of_those_correct": 0 }, "what_cannot_be_said": "2 of the 5 wrong answers printed a date and no working. Whether those runs misread the question is not decidable from this trial, and no claim is made about them.", "the_one_wrong_answer_that_survives_every_cheap_check": [ "GPT-5.4 mini round 1: 2026-08-25" ], "stability_and_correctness_coincided_here": true, "runs": [ { "model_label": "GPT-5.4 nano", "round": 1, "method_the_run_showed": "setup_only", "setup_stated_correctly": true, "answer": "2026-07-02", "answer_weekday": "Thursday", "correct": false, "matches_reading": null, "said_in_words_whether_start_counts": true, "reading_implied_by_its_method": "A", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 88, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-07-02", "said": "Friday", "holds": false, "check": "2026-07-02 is a Thursday; the run said Friday" } ], "false_quoted_claims": 1, "refutable_without_knowing_the_answer": true, "raw_answer_verbatim": "Project started 3 March 2026.\n\n120 working days from the start date, counting 3 March 2026 as day 1.\nThat means 119 working days after 3 March.\nSkipping only weekends (Mon\u2013Fri) and ignoring public holidays, the finish falls on Friday, 2 July 2026.\n\nSo, it finished on 2 July 2026." }, { "model_label": "GPT-5.4 nano", "round": 2, "method_the_run_showed": "shortcut_24_weeks", "setup_stated_correctly": true, "answer": "2026-09-17", "answer_weekday": "Thursday", "correct": false, "matches_reading": null, "said_in_words_whether_start_counts": true, "reading_implied_by_its_method": "A", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 143, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Wednesday", "holds": false, "check": "2026-03-03 is a Tuesday; the run said Wednesday" }, { "kind": "day_offset", "from": "2026-03-03", "days": 168, "said": "2026-09-17", "holds": false, "check": "2026-03-03 + 168 days is 2026-08-18; the run said 2026-09-17" } ], "false_quoted_claims": 2, "refutable_without_knowing_the_answer": true, "raw_answer_verbatim": "120 working days starting from 3 March 2026 (counting that day as working day), Monday\u2013Friday, skipping public holidays.\n\nFirst, the day-of-week for 3 March 2026 is Wednesday. So we can count full weeks:\n\n120 working days = 24 weeks + 0 working days (since 24 weeks \u00d7 5 = 120)\n\nThat means the end date is exactly 24 weeks after 3 March 2026:\n\n24 weeks = 168 calendar days\n3 March 2026 + 168 days = 17 September 2026\n\nSo it finished on 17 September 2026." }, { "model_label": "GPT-5.4 mini", "round": 1, "method_the_run_showed": "none_shown", "setup_stated_correctly": null, "answer": "2026-08-25", "answer_weekday": "Tuesday", "correct": false, "matches_reading": null, "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": null, "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 126, "quoted_claims_checked": [], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "25 August 2026." }, { "model_label": "GPT-5.4 mini", "round": 2, "method_the_run_showed": "none_shown", "setup_stated_correctly": null, "answer": "2026-08-22", "answer_weekday": "Saturday", "correct": false, "matches_reading": null, "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": null, "finish_falls_on_a_weekend": true, "working_days_its_answer_implies": 124, "quoted_claims_checked": [], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": true, "raw_answer_verbatim": "22 August 2026" }, { "model_label": "Claude Haiku 4.5", "round": 1, "method_the_run_showed": "month_by_month", "setup_stated_correctly": true, "answer": "2026-08-17", "answer_weekday": "Monday", "correct": true, "matches_reading": "A", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": "A", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 120, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Tuesday", "holds": true, "check": "2026-03-03 is a Tuesday; the run said Tuesday" }, { "kind": "weekday_of", "date": "2026-08-03", "said": "Monday", "holds": true, "check": "2026-08-03 is a Monday; the run said Monday" }, { "kind": "weekday_of", "date": "2026-08-17", "said": "Monday", "holds": true, "check": "2026-08-17 is a Monday; the run said Monday" }, { "kind": "working_days_in_range", "from": "2026-03-03", "to": "2026-03-31", "said": 21, "holds": true, "check": "2026-03-03 to 2026-03-31 inclusive is 21 working days; the run said 21" }, { "kind": "working_days_in_range", "from": "2026-04-01", "to": "2026-04-30", "said": 22, "holds": true, "check": "2026-04-01 to 2026-04-30 inclusive is 22 working days; the run said 22" }, { "kind": "working_days_in_range", "from": "2026-05-01", "to": "2026-05-31", "said": 21, "holds": true, "check": "2026-05-01 to 2026-05-31 inclusive is 21 working days; the run said 21" }, { "kind": "working_days_in_range", "from": "2026-06-01", "to": "2026-06-30", "said": 22, "holds": true, "check": "2026-06-01 to 2026-06-30 inclusive is 22 working days; the run said 22" }, { "kind": "working_days_in_range", "from": "2026-07-01", "to": "2026-07-31", "said": 23, "holds": true, "check": "2026-07-01 to 2026-07-31 inclusive is 23 working days; the run said 23" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "I need to calculate when a project that started on March 3, 2026, and ran for 120 working days (Monday to Friday) would finish.\n\nLet me work through this systematically:\n\nStarting point: March 3, 2026 (Tuesday)\n\nCalculation approach: I need to count forward 120 working days, excluding weekends and public holidays.\n\nStarting from March 3, 2026, and counting 120 working days:\n\nMarch 2026: 3rd is a Tuesday. From March 3\u201331 = 21 working days (Tue\u2013Fri = 4 days, then 4 full weeks = 20 days, totaling 21)\n\nRemaining after March: 120 - 21 = 99 working days\n\nApril 2026: 22 working days (30 days \u00f7 5 \u00d7 7)\n\nRemaining after April: 99 - 22 = 77 working days\n\nMay 2026: 21 working days (31 days \u00f7 5 \u00d7 7)\n\nRemaining after May: 77 - 21 = 56 working days\n\nJune 2026: 22 working days (30 days \u00f7 5 \u00d7 7)\n\nRemaining after June: 56 - 22 = 34 working days\n\nJuly 2026: 23 working days (31 days \u00f7 5 \u00d7 7)\n\nRemaining after July: 34 - 23 = 11 working days\n\nAugust 2026: Starting August 3 (Monday), counting 11 working days brings us to August 17, 2026 (Monday)\n\nThe project finished on August 17, 2026.\n\n(Note: This calculation assumes no public holidays fall within the working period. If your organization observes specific public holidays during this timeframe, the end date would shift accordingly.)" }, { "model_label": "Claude Haiku 4.5", "round": 2, "method_the_run_showed": "month_by_month", "setup_stated_correctly": true, "answer": "2026-08-17", "answer_weekday": "Monday", "correct": true, "matches_reading": "A", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": "A", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 120, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Tuesday", "holds": true, "check": "2026-03-03 is a Tuesday; the run said Tuesday" }, { "kind": "weekday_of", "date": "2026-08-17", "said": "Monday", "holds": true, "check": "2026-08-17 is a Monday; the run said Monday" }, { "kind": "working_days_in_range", "from": "2026-03-03", "to": "2026-03-31", "said": 21, "holds": true, "check": "2026-03-03 to 2026-03-31 inclusive is 21 working days; the run said 21" }, { "kind": "working_days_in_range", "from": "2026-04-01", "to": "2026-04-30", "said": 22, "holds": true, "check": "2026-04-01 to 2026-04-30 inclusive is 22 working days; the run said 22" }, { "kind": "working_days_in_range", "from": "2026-05-01", "to": "2026-05-31", "said": 21, "holds": true, "check": "2026-05-01 to 2026-05-31 inclusive is 21 working days; the run said 21" }, { "kind": "working_days_in_range", "from": "2026-06-01", "to": "2026-06-30", "said": 22, "holds": true, "check": "2026-06-01 to 2026-06-30 inclusive is 22 working days; the run said 22" }, { "kind": "working_days_in_range", "from": "2026-07-01", "to": "2026-07-31", "said": 23, "holds": true, "check": "2026-07-01 to 2026-07-31 inclusive is 23 working days; the run said 23" }, { "kind": "working_days_in_range", "from": "2026-08-03", "to": "2026-08-17", "said": 11, "holds": true, "check": "2026-08-03 to 2026-08-17 inclusive is 11 working days; the run said 11" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "I need to calculate when a project that started on March 3, 2026, and ran for 120 working days (Monday to Friday) would finish.\n\nLet me work through this:\n\nStarting point: March 3, 2026 (which was a Tuesday)\n\nCalculation: I need to count forward 120 working days, excluding weekends and public holidays.\n\nLet me break this down by months:\n\nMarch 2026: Starting March 3 (Tuesday), the working days remaining are: 3, 4, 5, 6, 9, 10, 11, 12, 13, 16, 17, 18, 19, 20, 23, 24, 25, 26, 27, 30, 31 = 21 working days\nApril 2026: 22 working days (30 days - 8 weekend days)\nMay 2026: 21 working days (31 days - 10 weekend days)\nJune 2026: 22 working days (30 days - 8 weekend days)\nJuly 2026: 23 working days (31 days - 8 weekend days)\n\nRunning total: 21 + 22 + 21 + 22 + 23 = 109 working days\n\nRemaining: 120 - 109 = 11 working days into August\n\nAugust 2026: Starting from August 3 (Monday), counting 11 working days:\n\nAugust 3 (1), 4 (2), 5 (3), 6 (4), 7 (5), 10 (6), 11 (7), 12 (8), 13 (9), 14 (10), 17 (11)\n\nThe project finished on August 17, 2026 (which was a Monday).\n\nNote: This calculation assumes no public holidays during this period. If your region observes specific public holidays between March 3 and August 17, 2026, you would need to subtract those days from the total to get the precise finish date." }, { "model_label": "Mistral Small 4", "round": 1, "method_the_run_showed": "shortcut_24_weeks", "setup_stated_correctly": true, "answer": "2026-08-27", "answer_weekday": "Thursday", "correct": false, "matches_reading": null, "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": "B", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 128, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Tuesday", "holds": true, "check": "2026-03-03 is a Tuesday; the run said Tuesday" }, { "kind": "weekday_of", "date": "2026-08-27", "said": "Thursday", "holds": true, "check": "2026-08-27 is a Thursday; the run said Thursday" }, { "kind": "day_offset", "from": "2026-03-03", "days": 168, "said": "2026-08-27", "holds": false, "check": "2026-03-03 + 168 days is 2026-08-18; the run said 2026-08-27" } ], "false_quoted_claims": 1, "refutable_without_knowing_the_answer": true, "raw_answer_verbatim": "To determine the finish date of the project, we'll calculate 120 working days (Monday to Friday) starting from 3 March 2026, while ignoring public holidays.\n\nAssumptions:\nWorking days: Monday to Friday (no weekends).\nPublic holidays: None are specified, so we'll assume no holidays are skipped.\nLeap year check: 2026 is not a leap year (2024 was the last leap year).\nCalculation:\nTotal weeks: 120 working days \u00f7 5 days/week = 24 weeks.\nTotal days: 24 weeks \u00d7 7 days/week = 168 calendar days.\nStart date: 3 March 2026.\nEnd date: 3 March 2026 + 168 days = 27 August 2026.\nVerification:\n3 March 2026 (Tuesday) + 168 days lands on 27 August 2026 (Thursday).\nNo weekends or holidays are skipped in this calculation.\nFinal Answer:\n\nThe project finished on 27 August 2026." }, { "model_label": "Mistral Small 4", "round": 2, "method_the_run_showed": "shortcut_24_weeks", "setup_stated_correctly": true, "answer": "2026-08-18", "answer_weekday": "Tuesday", "correct": true, "matches_reading": "B", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": "B", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 121, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Tuesday", "holds": true, "check": "2026-03-03 is a Tuesday; the run said Tuesday" }, { "kind": "weekday_of", "date": "2026-08-18", "said": "Tuesday", "holds": true, "check": "2026-08-18 is a Tuesday; the run said Tuesday" }, { "kind": "day_offset", "from": "2026-03-03", "days": 168, "said": "2026-08-18", "holds": true, "check": "2026-03-03 + 168 days is 2026-08-18; the run said 2026-08-18" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "To determine the finish date of the project, we'll calculate 120 working days (Monday to Friday) starting from 3 March 2026, while ignoring public holidays.\n\nAssumptions:\nPublic Holidays: Since no specific country/region is mentioned, we'll assume no public holidays (as none were provided).\nWorking Days: Only Monday\u2013Friday count (weekends are excluded).\nCalculation:\nStart Date: 3 March 2026 (Tuesday)\nTotal Working Days: 120\nStep-by-Step:\nWeeks in 120 Working Days:\n120 working days \u00f7 5 days/week = 24 weeks.\n24 weeks \u00d7 7 days/week = 168 calendar days.\n\nAdding 168 Days to 3 March 2026:\n\nMarch has 31 days \u2192 31 - 3 = 28 days remaining in March.\n168 - 28 = 140 days remaining.\nApril (30 days) \u2192 140 - 30 = 110 days remaining.\nMay (31 days) \u2192 110 - 31 = 79 days remaining.\nJune (30 days) \u2192 79 - 30 = 49 days remaining.\nJuly (31 days) \u2192 49 - 31 = 18 days remaining.\nAugust (18 days) \u2192 18 August 2026.\nVerification:\n3 March 2026 (Tue) + 168 days = 18 August 2026 (Tue).\nSince 120 working days (Mon\u2013Fri) are exactly 24 weeks, the finish date is 18 August 2026.\nFinal Answer:\n\nThe project finished on 18 August 2026." }, { "model_label": "gpt-oss 120B", "round": 1, "method_the_run_showed": "none_shown", "setup_stated_correctly": null, "answer": "2026-08-18", "answer_weekday": "Tuesday", "correct": true, "matches_reading": "B", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": null, "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 121, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-08-18", "said": "Tuesday", "holds": true, "check": "2026-08-18 is a Tuesday; the run said Tuesday" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "The 120 working-day schedule ends on Tuesday, 18 August 2026." }, { "model_label": "gpt-oss 120B", "round": 2, "method_the_run_showed": "none_shown", "setup_stated_correctly": null, "answer": "2026-08-18", "answer_weekday": "Tuesday", "correct": true, "matches_reading": "B", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": null, "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 121, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-08-18", "said": "Tuesday", "holds": true, "check": "2026-08-18 is a Tuesday; the run said Tuesday" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "The project would finish on Tuesday, 18 August 2026." }, { "model_label": "Gemma 4 31B", "round": 1, "method_the_run_showed": "shortcut_24_weeks", "setup_stated_correctly": true, "answer": "2026-08-18", "answer_weekday": "Tuesday", "correct": true, "matches_reading": "B", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": "B", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 121, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Tuesday", "holds": true, "check": "2026-03-03 is a Tuesday; the run said Tuesday" }, { "kind": "weekday_of", "date": "2026-08-18", "said": "Tuesday", "holds": true, "check": "2026-08-18 is a Tuesday; the run said Tuesday" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "To calculate the finish date, you add 120 working days to the start date of 3 March 2026.\n\nSince there are 5 working days per week, 120 working days equals exactly 24 full weeks (120 \u00f7 5 = 24).\n\nAdding 24 weeks to Tuesday, 3 March 2026, brings the date to Tuesday, 18 August 2026." }, { "model_label": "Gemma 4 31B", "round": 2, "method_the_run_showed": "shortcut_24_weeks", "setup_stated_correctly": true, "answer": "2026-08-18", "answer_weekday": "Tuesday", "correct": true, "matches_reading": "B", "said_in_words_whether_start_counts": false, "reading_implied_by_its_method": "B", "finish_falls_on_a_weekend": false, "working_days_its_answer_implies": 121, "quoted_claims_checked": [ { "kind": "weekday_of", "date": "2026-03-03", "said": "Tuesday", "holds": true, "check": "2026-03-03 is a Tuesday; the run said Tuesday" }, { "kind": "weekday_of", "date": "2026-08-18", "said": "Tuesday", "holds": true, "check": "2026-08-18 is a Tuesday; the run said Tuesday" }, { "kind": "day_offset", "from": "2026-03-03", "days": 168, "said": "2026-08-18", "holds": true, "check": "2026-03-03 + 168 days is 2026-08-18; the run said 2026-08-18" } ], "false_quoted_claims": 0, "refutable_without_knowing_the_answer": false, "raw_answer_verbatim": "To find the finish date, you calculate 120 working days (Monday through Friday) starting from 3 March 2026.\n\n120 working days is exactly 24 full weeks (120 divided by 5).\n\nSince 3 March 2026 is a Tuesday, 24 full weeks from that start date would bring the project to the same day of the week (Tuesday) exactly 168 calendar days later.\n\nAdding 168 days to 3 March 2026 results in a finish date of Tuesday, 18 August 2026." } ] }, "known_limits": { "twelve_runs_is_not_a_benchmark": "Two runs of six models on one question on one day. It cannot rank the models and no ranking is offered. What twelve runs can show is that a single answer from any of them is not evidence, which is the only claim made from the tally.", "stability_is_not_a_correctness_check": "In this trial every model that answered the same twice was right and every model that changed its answer was wrong at least once. That is a coincidence of twelve runs, not a rule, and this site has already published the counterexample: a model that gave the same wrong answer eight times out of eight. Asking twice catches an unstable error and tells you nothing about a stable one.", "silent_runs_cannot_be_scored_for_comprehension": "Four of the twelve runs printed a date and no working, two of them wrong. Nothing in this trial says whether those two misread the question, and the post makes no claim that they did or did not. The comprehension finding covers only the runs that showed a route to their answer.", "one_question_one_shape": "Working-day arithmetic on a fixed calendar. Nothing here transfers to a different kind of question, and the failures found were arithmetic slips rather than misunderstandings of the task, which may be specific to this shape of problem.", "free_tier_only": "These are the models a public free chat surface offers, several of them small or fast variants. A vendor's larger paid model is not represented and no claim is made about one.", "only_two_runs_said_which_reading_they_used": "Exactly two of the twelve runs stated in words whether the start date counts as working day one, both of them from GPT-5.4 nano, and both of those runs were wrong. Every other run left it to be inferred from its method or its enumeration. A reader who needs one convention has to put it in the prompt.", "the_reading_split_is_real": "Answers landed on both 17 and 18 August and both are counted correct. A reader who needs one specific convention has to say which in the prompt; none of the twelve runs asked, and only half of them stated which reading they had used." } }