The plateau, and why it lasted so long
Classical OCR was a stack of narrow models stitched together: page deskew, layout analysis, text-line detection, character recognition, post-OCR language correction. Each stage was a specialist that didn't know what the next stage was going to do with its output. The engineering quality of those specialists was real — Tesseract, ABBYY, AWS Textract and the rest were not lazy systems — but the architecture had a ceiling that wasn't a model problem. It was a context problem.
A character recognizer that has never seen the surrounding paragraph doesn't know that the smudged token after "amount" is a number, not a word. A layout analyzer that emits boxes without semantics can't tell a table cell from a footnote and gives the downstream extractor no help when the layout is novel. The pipeline could be tuned, but the joints between stages were where the accuracy was bleeding out.
The numbers reflected that. Across the last decade of vendor benchmarks the same shape repeated: high-90s on the cleanest printed-text subset, mid-80s on receipts and invoices in the wild, low-70s on faxes and degraded scans, and a steep drop into the 40s for cursive handwriting. Improvements were incremental and got harder year over year.
What multimodal models actually changed
A 2026 multimodal model treats the page as a single object. It sees the pixels, the layout, the surrounding context, and the schema it's being asked to fill — at the same time, in the same forward pass. That's not a faster pipeline. It's a different architecture.
Three things follow from that, and the accuracy jump is mostly explained by the combination.
Context is no longer something the next stage adds
When the model that recognizes the smudged digit also has the line "Total amount due:" directly above it in its visual context, the disambiguation is free. The classical pipeline had to reconstruct that relationship through bounding boxes and post-processing rules. The multimodal pipeline never broke it apart.
Handwriting stopped being a special case
Cursive recognition was a fundamentally different problem from printed-text OCR in the classical stack — different training data, different models, different tooling, different accuracy ceiling. The vision encoders inside modern multimodal models are trained on enough handwritten content that the same model that reads a typed invoice reads a doctor's margin note. The 46–70% historical band on handwriting moved to 95% in benchmarks that would have been considered impossible two years ago.
Tables and structured layouts became readable as structure
The classical pipeline either flattened a table into rows of text and lost the geometry, or used a specialized table-extraction model that worked on layouts it had been trained for and broke on layouts it hadn't. A multimodal model returns the table as structured rows and columns directly, including merged cells, hierarchical headers, and the kind of irregular layouts spreadsheet-as-PDF tends to produce.
Why the deterministic stack didn't go away
The temptation, given accuracy numbers like that, is to route everything through the multimodal model. The teams that did this in early 2025 mostly came back. Three reasons keep the deterministic stack alive in production.
Cost. A multimodal call against a frontier model is roughly 50–200x the cost of running deterministic OCR on the same page. For a high-volume invoice pipeline processing millions of pages a month, that is the difference between a viable unit economic and one that breaks the second the volume grows.
Latency. Deterministic OCR returns in tens of milliseconds. A multimodal extraction returns in seconds. For interactive flows — a user uploading an ID and waiting for the form to fill in — the difference is felt.
Predictability. Deterministic OCR fails in deterministic ways. A multimodal model can fail confidently on a page it would have nailed yesterday because the provider quietly shipped a model update. Production pipelines that auto-approve high-volume decisions are not comfortable with that failure mode for the parts of the workload that the deterministic stack handles well.
The pipeline that actually shipped in 2026 is hybrid. The split is not "old vs. new" — it is volume vs. complexity.
| Document segment | What it looks like | Where it routes |
|---|---|---|
| Predictable structure | Standardized invoice templates, government forms with fixed layouts, structured statements. | Deterministic OCR + parser. Cheap, fast, audit-friendly. |
| Semi-structured | Receipts, varied invoice layouts, statements from many issuers. | Deterministic OCR + small classifier; multimodal fallback when confidence drops. |
| Unstructured / visually complex | Medical reports with handwritten margins, complex tables, scanned legal exhibits, claim packets. | Multimodal model directly. The cost premium is paid back by avoiding human review. |
The router that decides which path a page takes is a small classifier — itself often a deterministic model with handcrafted features — that predicts which side of the cost/quality curve the page lives on. Getting the router right is roughly half of the value of the hybrid design.
The 98% accuracy number is real. The 75% cost reduction next to it is real only if the routing is.
The economics, made concrete
A worked example that mirrors what we see across customer pipelines. Take a backoffice processing 1,000,000 pages a month, mixed across the three buckets above.
- Predictable structure (~70% of volume): 700,000 pages routed to deterministic OCR at roughly $0.0005/page. Monthly cost: $350.
- Semi-structured (~22% of volume): 220,000 pages through deterministic OCR with a small percentage falling through to a mid-tier multimodal model. Blended cost around $0.012/page. Monthly cost: $2,640.
- Visually complex (~8% of volume): 80,000 pages routed directly to a frontier multimodal model at roughly $0.04/page. Monthly cost: $3,200.
Total: about $6,200/month for the inference layer alone, processing a million pages with a quality bar that previously required a 12–15 person ops team to hit. The same workload run entirely through the frontier multimodal model would cost around $40,000/month. The same workload run entirely through deterministic OCR would either fail the visually complex bucket outright or push that work back to humans, where the loaded cost is materially higher than either inference path.
The savings live in the routing. The teams quoting "we saved 80% on document costs" are usually not telling you the routing is what's working — they are quoting the post-routing result and crediting the model.
Where the new accuracy bites back
A 98% extraction accuracy on benchmark printed text is not the same as a 98% production accuracy on your documents. Three failure modes to plan for.
Confident wrongness on plausible-looking content
Multimodal models will, on a small but non-zero fraction of pages, return a value that is formatted correctly, internally consistent, and wrong. A date that doesn't match the source but parses cleanly. An amount with the right currency symbol but the wrong digit. The classical OCR stack would have flagged a low-confidence character. The multimodal model returns the field with no signal that anything was hard. This is the failure mode that pure-multimodal pipelines miss without an explicit validation stage — the same architectural argument we made when looking at the shift from extraction to decision.
Layout reasoning errors on tables
Tables with merged cells, hierarchical headers, or non-rectangular shapes are where the new models reach for the strongest improvements — and also where they generate the most creative wrong answers. A model that correctly transcribes the cells of a table can still decide that two header rows are one row, or attribute a footnote value to the column above it. Detect these by validating column sums, by cross-referencing totals to the page totals, and by sampling the layout decisions, not just the cell contents.
Provider drift between model versions
The "98% on benchmark" number applies to the model you tested against. The model the provider serves you next quarter may shift that number up or down by a percent or two on the public benchmark and by considerably more on a long-tailed distribution like the one you actually run in production. Pin the model identifier, log it on every call, and benchmark new versions against your golden set before promoting them. The audit-trail schema we run at Cogneris captures the model identifier and prompt at every decision so this kind of regression has a place to surface.
What this changes for ops teams
The work the new accuracy reshuffles is not the day-to-day extraction work. It is the quality discipline around the pipeline.
- Golden sets become a primary artifact. The team needs a frozen set of a few thousand pages with verified ground truth, refreshed quarterly, that every model change has to pass before it goes live. Without it, accuracy regressions are invisible until a customer escalates.
- Confidence calibration replaces threshold-tuning. The classical pipeline let you tune a confidence threshold per field and call it done. Multimodal models tend to return uniformly high confidence even when wrong; the resilience has to come from cross-field validation, not from confidence scores alone.
- The exception queue gets smaller and weirder. The cases the system flags for human review are no longer the cases where OCR couldn't read a digit. They are the cases where the document was internally inconsistent, where the policy didn't have a clean rule, or where the multimodal model returned something that looked plausible but failed validation. The training material for human reviewers needs to change with it.
- Latency budgets need a routing line. If a synchronous flow needs a response in under 300ms, that flow can't tolerate routing to a multimodal model on the critical path. The router has to know which calls are interactive and degrade gracefully when the cheap path can't handle the page.
The honest take on the headline number
The 98% number is accurate as benchmarks go. It is also the answer to a specific question — "what is the per-character or per-token accuracy of this model on its strongest document class" — that is not the question most production teams are actually asking. The question teams are actually asking is closer to "what fraction of documents flowing through my pipeline come out with every field correct, without human touch, and without the system being confidently wrong about any of them." That number is materially lower than 98%, even with the new models. It can still be high enough to change the unit economics of the backoffice — that part is real — but only when the pipeline around the model carries its weight.
The interesting design decisions in 2026 aren't about which multimodal model to pick. They are about where in the pipeline to spend the multimodal model's accuracy and where to keep the cheaper, more predictable stack. The teams that are getting the headline cost reductions are the ones that treated this as a routing problem, not a model problem.
Closing thought
Classical OCR isn't gone. It got good enough, fast enough, cheap enough that nobody notices it inside a hybrid pipeline — and that's the point. The multimodal model handles the long tail that used to require human review, and the deterministic stack handles the volume that never needed a frontier model in the first place. The pipeline split that won is the one that takes both seriously.
For the routing logic the Cogneris pipeline uses on hybrid OCR — and the validation layer that catches confident-but-wrong extractions before they reach a downstream system — see our product page or talk to our team. We're happy to benchmark a sample of your documents through both paths and share the cost/accuracy breakdown by document class.