720 of 720 table cells read correctly. My scorer was the thing that kept failing
The widely repeated claim is that models read table screenshots across instead of down, misaligning the numbers. Across nine runs on three table sizes, not one cell was misplaced. Three separate times my own checker reported failures that were not there.
The claim, repeated across support forums and extraction-tool blogs, is that models read multi-column table screenshots left to right across columns rather than down each column, producing misaligned numbers.
That is precisely testable, so I built tables where every single value is unique. With no repeated numbers, a wrong cell can be classified rather than merely counted: if the value appears elsewhere in the table it was misplaced, and if it appears nowhere it was misread. “78% accurate” hides which of those happened. This does not.
The result
Browser screenshots of HTML tables, read by Claude Sonnet 5, scored cell by cell against the exact source data.
| Table | Cells | Correct | Misplaced | Misread | Missing |
|---|---|---|---|---|---|
| 5 x 4 | 20 | 20, 20, 20 | 0 | 0 | 0 |
| 10 x 6 | 60 | 60, 60, 60 | 0 | 0 | 0 |
| 20 x 8 | 160 | 160, 160, 160 | 0 | 0 | 0 |
Three rounds each. 720 of 720 cells correct. Zero misplaced.
The transposition claim did not reproduce even once. On a 20-row, 8-column table the output was byte-identical to the source CSV.
Messy financial formatting did not break it either
The plain tables were bare four-digit integers, which is the easy case. So I built a second table with the formatting real spreadsheets actually carry: currency symbols, thousands separators, explicit signs, decimals and percentages.
| Month | Revenue | Cost | Variance | Units | Margin % |
|---|---|---|---|---|---|
| 2026-01 | $47,999 | $2,368 | -4,949 | 414 | 13.1% |
Asked to strip the symbols and separators and return plain numbers, it returned all twelve rows cell-perfect, verified in two separate captures. A third run could not be scored because my own parser rejected its formatting, so it is excluded rather than counted as a failure.
The part that actually matters
My scorer was wrong three times, always for the same reason, and twice it would have produced a confident, completely false headline.
The first version reported 20 missing on a table the model had read perfectly. The number-matching regex allowed commas inside numbers, to handle 1,234. Applied to a CSV line, 5185,6874,9684,1475 was captured as one token instead of four.
I fixed that, added controls, and the controls passed. Then the 20 x 8 table came back 96 correct, 16 misread, 48 missing, identically across all three rounds. Reproducible failure, plausible shape, obvious story: big tables break it.
The output was byte-identical to ground truth. The row labels were the problem: Q2, W01, Q4. In the line Q2,6190,... the pattern matched 2,619 as a grouped thousands number, silently shifting every field.
Then a third variant failed on the messy table, again at the parser.
A comma is both the CSV separator and the thousands separator. Every one of these bugs was that single ambiguity wearing a different hat, and the second one was the dangerous kind: it did not crash, it did not look like noise, and it reproduced perfectly across three runs.
What saved it
Two habits, and only two.
A zero is a failed measurement, not a result. Twenty missing cells from a model whose raw output I could see was correct is not a finding, it is a broken pipeline. That is what prompted the first look.
Diff against the truth directly, not through the scorer. Running diff on the model’s raw output against the source CSV took one command and returned identical, which settled it instantly. The scorer sat between me and the evidence, and going around it was the whole fix.
The final parser does not hunt for numbers in text at all. It splits on the delimiter structurally, discards the row label, then parses each field, and it is control-tested against digit-bearing row labels, zero-padded labels, pipe tables and spaced numbers, precisely the cases that broke the earlier versions.
If you are extracting tables programmatically
- Do not ask for CSV when the data contains commas. Ask for tab-separated output and for separators to be stripped. The ambiguity that broke my scorer three times is the same one that will break yours.
- Make every value unique in your test data. It is the only way to tell a misplaced number from a misread one, and they call for completely different fixes.
- Check the raw output against the truth by hand once, before trusting any accuracy figure your harness produces.
What this does not establish
One model, one renderer, clean screenshots. These are crisp browser captures of simple HTML tables. Phone photos, scanned PDFs, merged headers, multi-line cells and stacked sub-rows are all untested and are exactly where the reported problems cluster.
Three sizes up to 20 x 8. 160 cells is a real table but not a large one. I did not find the size where it breaks, so I cannot tell you where that is.
Nine runs is nine runs. No accuracy rate is claimed. What reproduced is the absence of misplacement, which is the specific claim being tested.
Method
Tables rendered as HTML and captured with headless Chrome at 1100px wide, then trimmed. Every data value unique, drawn without replacement, so any wrong cell is attributable. Read by Claude Sonnet 5 via Claude Code 2.1.226 headless, asked for CSV, scored cell by cell against the generating data with misplaced, misread and missing counted separately. The scorer was control-tested against a perfect grid, a transposed grid, an invented value and a truncated grid, plus five parsing cases, before any model output was scored, and was still wrong twice before the version reported here.