Chandra OCR: what it is, how it works, and when to use it

Direct answer: Chandra is Datalab’s vision-language model for document extraction — a single model that reads a full page (text, tables, forms, handwriting, math) in one decoding pass instead of chaining separate layout-detection and recognition stages. It’s open-weight (Apache-2.0 code, modified OpenRAIL-M weights), available via pip install chandra-ocr or a hosted API, and on the independent olmOCR benchmark it scores 85.8 overall, ahead of the pipeline-based tools it was built to replace. It’s a model, not a pipeline — it doesn’t classify document types, route between specialized extractors, or produce a per-page confidence signal, which is what tools built around it (including pdfmux) add on top.

What Chandra actually is

Chandra is Datalab’s successor to Marker, their earlier OCR tool. Marker worked as a pipeline: detect layout, recognize text, reconstruct tables, stitch the results together, with each stage a separate model. Chandra replaces that pipeline with one end-to-end vision-language model that takes a rendered page image and decodes it directly into structured markup — text, table cells, form fields, and image captions in a single forward pass, rather than a chain of specialized components handing off to each other.

That architectural shift is the actual news here, not just an accuracy bump. A pipeline’s failure modes are localized and often debuggable — you can tell whether the layout detector or the table-structure model produced a bad output. A single end-to-end model’s failure modes are harder to isolate: it either reads the page correctly or it doesn’t, and there’s no intermediate stage to inspect when it’s wrong. What you gain is a model tuned holistically on the actual task (reading a page) instead of stitched together from components optimized for narrower sub-tasks.

What it handles

Per Datalab’s own description, Chandra is built for the harder end of document extraction:

• Handwriting, including handwritten math
• Forms with checkboxes
• Complex tables, including cells with text spanning multiple columns
• Mathematical equations, including older/unusual fonts
• Images with captions, extracted and described in context
• General layout awareness — identifying what a region is before reading it, which is what makes image extraction and captioning possible in the same pass

That list overlaps heavily with the failure modes that break simpler extraction tools: a checkbox form or a merged-cell table is exactly where a naive text-extraction pass produces garbled or missing output. Chandra’s pitch is handling those cases as well as the easy ones, in the same call.

Running it

The open-weight release installs as a standard Python package:

pip install chandra-ocr
# with a HuggingFace inference backend:
pip install chandra-ocr[hf]
# every optional backend:
pip install chandra-ocr[all]

From the command line, it runs against a single PDF or a directory of documents:

chandra input.pdf ./output --method vllm
chandra ./documents ./output --method hf

The --method flag picks the inference backend — vllm for throughput on a GPU serving setup, hf for a plain HuggingFace Transformers backend. Quantized 8B and 2B parameter variants are available for on-premises deployment where a full-size model isn’t practical, and Datalab’s hosted API is the option if you’d rather not run inference yourself — it comes with a free credit tier for testing before you commit either way.

Benchmark numbers, with the source attached

On the independent olmOCR benchmark, Chandra 2 scores 85.8 ± 0.8 overall, breaking down to 86.9 on arXiv documents, 92.1 on tables, and 89.1 on math — measured against other OCR approaches including dots.ocr, Mistral OCR, and GPT-4o on the same benchmark set. On a separate 90-language multilingual benchmark, Chandra 2 averages 72.7% versus 60.8% for Gemini 2.5 Flash on the same test.

Those are Datalab’s own reported numbers on their chosen benchmark, not an independent third-party run — worth the same caveat that applies to any vendor-published score: run your own representative sample before trusting a benchmark result to predict your document mix specifically. The olmOCR benchmark is at least a fixed, checkable public test set rather than a cherry-picked internal corpus, which is more than most vendor claims in this space offer.

What Chandra doesn’t do

Chandra is a model, not an orchestration layer, and being a strong single model doesn’t give you the things an orchestration layer provides:

No document-type routing. Chandra reads whatever page you send it the same way every time. It doesn’t classify a page as “clean digital text, skip the vision model, use the free text layer instead” — every page, including ones that would extract perfectly with a free text-layer read, goes through the full model.

No per-page confidence signal. A response comes back whether the page was read correctly or not, with no built-in flag distinguishing a confident correct read from a low-confidence guess on a badly scanned page. If a pipeline needs to know which pages to route for human review, that logic has to be built separately.

No multi-backend fallback. If a specific page is genuinely outside what the model handles well, there’s no automatic retry with a different extraction approach — you get Chandra’s read or nothing, same as calling any single model directly.

These aren’t criticisms of Chandra as a model — a narrow, well-executed single-model tool doesn’t need to be an orchestration platform to be the right choice for a uniform document mix. They’re the reasons a tool like pdfmux exists as a layer that can call Chandra (or any other backend) per page rather than for every document unconditionally: classify first, route to the cheapest backend that will work, and fall back automatically when a page fails an audit check. The full pdfmux-vs-Chandra comparison breaks down exactly when each is the better fit — a uniform document mix and a need for Datalab’s bundled compliance posture favor Chandra directly; a mixed document mix where silent failures cost money favors an orchestrator with an audit layer on top.

When to reach for Chandra directly

Chandra is the right call when your document mix is genuinely uniform — the same kind of hard document (dense scanned tables, handwritten forms, complex layouts) repeated at volume — and you want one specialized model rather than a routing layer you’d have to configure and maintain. It’s also the clear pick if you’re under the OpenRAIL-M revenue threshold and want a strong open-weight model without a licensing cost, or if you’re already using Datalab’s Marker or Surya and want the in-family upgrade.

FAQ

Is Chandra OCR free to use? The code is Apache-2.0. The model weights ship under a modified OpenRAIL-M license: free for research, personal use, and startups under $2M in funding or revenue, with a clause preventing you from running it competitively against Datalab’s own hosted API. Datalab also offers a hosted API with a free credit tier for testing before committing to self-hosting.

How is Chandra different from traditional OCR? Traditional OCR pipelines chain separate stages — layout detection, text recognition, table structure recovery — each a distinct model with its own failure modes. Chandra is a single vision-language model that decodes a full page in one pass, using layout awareness to identify what each region is (text, table, image, form field) and read it in the same step rather than handing off between specialized stages.

Does Chandra run locally or only through an API? Both. The weights are open (HuggingFace and GitHub), with quantized 8B and 2B variants for on-premises deployment, and Datalab also runs a hosted API if you’d rather not manage inference infrastructure yourself. Running it locally requires a GPU for reasonable throughput; the hosted API removes that requirement at a per-page cost.