A question that keeps turning up on vendor forums goes roughly like this: is there a service that tells me whether a PDF is scanned, so I only send scans to OCR? The answers are usually right. Then the follow-up gives the game away: I'll check the first page, and if it's a scan I'll OCR the whole document.
That shortcut is the bug. Page 1 is not a sample of the file.
Why the file is the wrong unit
Real document piles are lumpy:
- a 40-page contract that's born-digital except for the signature pages, which someone printed, signed, scanned and put back in;
- a report whose appendix is photographs of a whiteboard;
- a deck exported to PDF where four slides are screenshots.
Ask "is this file a scan" and you get one answer for the whole thing, wrong in both directions. The contract's page 1 is text, so the signature pages never get OCR'd and the one part you cared about is missing from your index. Flip it, and a scanned cover page sends 39 pages of perfectly good text through an image pipeline.
Ask per page and you can send 36 pages down the cheap lane and 4 down the expensive one.
What a per-page classifier tells you
We make pdf2md, so here's the disclosure first: it has no OCR in it. It extracts the text that's already in a PDF, and when there isn't any, it says so and stops. That makes it a cheap gate in front of whatever OCR you already run.
I built a five-page test file: a short title page, two pages of contract text, one page that is only a picture of text (a stand-in for a scan), and one more text page. Then I asked the command-line tool to classify it:
$ npx @dudko.dev/pdf-to-md-cli mixed.pdf --detect
{"pdfType":"Mixed","pageCount":5,"pagesNeedingOcr":[4],
"ocrReasonsByPage":[{"page":4,"reasons":["scanned"]}],
"confidence":0.74,"hasEncodingIssues":false, …}
(Trimmed; that's version 0.3.6.) The document is Mixed, and exactly one page, the picture, is flagged as scanned. The title page, two short lines on an otherwise empty page, was not. A two-page all-picture file came back Scanned with both pages listed, and an all-text one came back TextBased with an empty list.
Converting the same mixed file gives you the Markdown for the four text pages and a summary line that still says 1 page(s) need OCR. Ask for page 4 alone with --pages 4 and the tool exits with code 2 and no text could be extracted. That's the routing signal: nothing to retry, the page goes to OCR.
The catch: one list is a scan verdict, the other is a hint
Here's the part worth knowing before you wire this to a budget. I also made a one-page PDF with nothing on it but a heading and one line (Quarterly report, Revenue rose 4 percent.). Real text, no images.
--detectsaysTextBased,pagesNeedingOcr: [].- A full conversion with
--jsonsaysTextBased, andpagesNeedingOcr: [1], with no entry for that page inocrReasonsByPage.
So, on 0.3.6, the conversion result lists sparse pages as "needing OCR" even when nothing on them is an image, and doesn't give a reason. Title pages, section dividers, a page that's one chart and a caption will show up there. The real scanned page came with "reasons": ["scanned"] from both calls.
That gives a simple rule:
Route on the reason, not on the list. A page whose reasons include scanned needs OCR. A page that's in the list with no reason has text on it, just not much of it. Whether that's worth a second pass is a call about your documents (for contracts, maybe, since a signature page is mostly empty; for a research archive full of chapter dividers, you'd be paying to OCR the word "Appendix").
Routing, concretely
This is the script I ran against the test files. It calls the published CLI, so there's no server and no key involved:
import json, subprocess, sys
CLI = ["npx", "--yes", "@dudko.dev/pdf-to-md-cli@0.3.6"]
def classify(path):
out = subprocess.run(CLI + [path, "--detect"],
capture_output=True, text=True, check=True).stdout
return json.loads(out)
def route(path):
d = classify(path)
if d["hasEncodingIssues"]:
return "quarantine", [] # text would come out garbled
if d["pdfType"] == "Scanned":
return "ocr-all", []
scanned = [r["page"] for r in d["ocrReasonsByPage"] if "scanned" in r["reasons"]]
return "extract", scanned # OCR only these pages
for p in sys.argv[1:]:
print(p, *route(p))
$ python3 route.py text.pdf mixed.pdf scan.pdf report.pdf
text.pdf extract []
mixed.pdf extract [4]
scan.pdf ocr-all []
report.pdf extract []
The sparse report.pdf stays in the cheap lane, which is what you want.
Don't skip the hasEncodingIssues check. It catches the failure that doesn't announce itself: text comes out, nothing errors, and the fonts declare an encoding the bytes don't honour, so what you index is plausible-looking garbage.
In production you'd call the core library from Node rather than spawning npx per file. The shape stays the same.
The same thing over HTTP
If you'd rather not run anything locally, the API has a classification-only endpoint, POST /api/v1/detect, and the conversion responses carry the verdict as headers:
| header | what it carries |
|---|---|
X-Pdf-Type |
TextBased, Scanned, ImageBased or Mixed |
X-Pdf-Needs-Ocr |
comma-separated 1-indexed pages, empty when none |
X-Pdf-Encoding-Issues |
true when the text is probably garbled |
A fully scanned document answers 422 from /convert with X-Pdf-Type: Scanned and no Markdown. Treat it as a routing signal; retrying won't change it. Every account starts with a free allowance, and the API needs a key. I tested the local tool for this article, not the hosted API, so if the reasons matter to you there, ask for ?format=json and check the body yourself.
The local tool is free for noncommercial use under PolyForm Noncommercial 1.0.0, with a 32-day trial for evaluation at work. The web app runs the same parser in your browser, so a file you drop there doesn't get uploaded anywhere.
When this is the wrong tool
If your pile is mostly scans (court filings, older medical records, anything from the fax era), a classifier in front of OCR saves very little, because almost everything goes down the expensive lane anyway. Run your OCR across the lot and skip the extra hop.
Where this pays is a mixed pile that you suspect is mostly born-digital and are currently OCR'ing in full to be safe. Classify a thousand of your own files first and count the pages with a scanned reason. Either it's a small minority and you've found the size of your OCR bill's waste, or it's most of them and you should go buy good OCR. Both are worth knowing before you build the pipeline.