TinyDoc Engine 3.0 · v0.3.2

Document extraction you can check.

Receipts and invoices become schema-valid JSON, with the exact OCR span and a confidence score behind every field. Local Qwen VLMs. No API key, no cloud, no telemetry.

$ pip install tinydoc
vendor_name 0.96 invoice_date 0.93 total 0.61 · review the page is the source of truth
Each value keeps its page, bbox, and score. Low scores route to review.
Receipt field F1
0.9475
Qwen2.5-VL-3B · clean-100
Fast tier
0.9200
Qwen3-VL-2B · 7.0 s median
Schema valid
1.00
clean-100 · zero errors
Multipage fully correct
10/10
VLM+OCR totals · VLM-only 6/10

TinyDoc is an open-source Python SDK and CLI that turns receipt and invoice images into schema-validated JSON, with the exact source span and a confidence score attached to every field. It runs a local vision-language model through Ollama and adds JSON Schema validation, OCR-span evidence, and review routing. It is not a fine-tuned TinyDoc checkpoint: the reader is a third-party Qwen model and the product is the trust layer around it.

Install: pip install tinydoc · License: Apache 2.0 · Runtime: local Ollama · Python 3.9+

How it works

A local reader, then four checks that make the output usable.

TinyDoc architecture: document input, local tier router with Qwen3-VL 2B and Qwen2.5-VL 3B, schema prompt, trust layer, and cited JSON output
The tier router chooses a reader; the trust layer runs on every result. No document leaves the host.
  1. Schema validationReject or repair broken JSON before it reaches your code.
  2. OCR-span evidenceAlign RapidOCR word boxes to each field: quote, page, and bbox.
  3. Field confidenceCalibrated for review routing, not for probability claims.
  4. Review routingSend low-confidence fields to a human instead of silently accepting them.

What ships in every result

DocumentResult on a clean receipt
FieldValueConfidence
companyNordhafen Workshop0.96
date2026-04-300.93
total6702.900.61

Evidence coverage on clean-100 is 0.798 of non-empty fields. Bboxes are OCR spans, not model predictions.

Measured results

Two tiers, two verticals, one scorer. Every published number recomputes from committed artifacts.

Measured results: receipt field F1 94.8% for Qwen2.5-VL-3B, 92.0% for Qwen3-VL-2B, 60.3% first-party hybrid, 37.6% OCR; invoice key-header 91.9% and 93.5%; multipage 10/10 key fields with OCR reconciliation vs 6/10 VLM-only; DocVQA 0.829 single-reference ANLS; latency 7.0 to 26.5 seconds
Receipts: SROIE clean-100. Invoices: public synthetic benchmark, page 1 only. Neither is an external benchmark. DocVQA is a separate single-reference reading probe, not a leaderboard score.
BenchmarkMetricScoreStatus
SROIE receipts — clean-100, Qwen2.5-VL-3BField F10.9475n=100 · schema 1.00
SROIE receipts — clean-100, Qwen3-VL-2B-InstructField F10.92002× faster
SROIE receipts — first-party hybrid v2Field F10.603superseded
SROIE receipts — PP-OCR + heuristicsField F10.376baseline
SROIE receipts — Tesseract + regexField F10.227baseline
Invoice benchmark v1 — 181 synthetic invoices, page 1, 3BKey-header91.9%65.7% fully correct
Invoice benchmark v1 — page 1, 2BKey-header93.5%71.8% fully correct
Invoice multipage, all pages — 10 synthetic docs, 46 pages, 2BFully correct4/100/10 page 1 · 71.1% vs 63.3% key-header
Invoice multipage, page-wise — same 10 docs, 2BFully correct6/1093.3% key-header · 0.848 line F1 · ~2× slower
Invoice multipage, page-wise + OCR totals — same 10 docs, 2BFully correct10/1090/90 key fields · pipeline, not pure VLM · 95% CI 69–100%
Real Portuguese receipts/invoices — held-out 100 real photos, 5 fields, 2BField accuracy72.0%22/100 fully correct · merge fix shipped in 0.3.1 · development 74.2% / 18 · 0.3.0 merge 64.8% / 3/100
Lone-tax total fallback — seed-44 replay + 28-doc toll/parking stress, 2BField accuracy404/570 · 114/139+3 fields each · 6/6 triggers correct · targeted, not population
Targeted tax retry — fresh 100 real photos (seed 44), 5 fields, 2BField accuracy72.1%first pass 70.4% · tax 51/82 → 61/82 · 0 fields lost · opt-in
DocVQA reading probe — 200 public images, 2BANLS0.829single-reference · not a leaderboard number
Invoice flags — inconsistent + multi-invoiceRecall0/7not usable
Evidence coverage — RapidOCRRate0.798gate 0.90
VLM bbox grounding — hit@0.52B / 3B0.20 / 0.04gate 0.70
CORD / real B2B supplier invoices——not measured
Provenance. clean-100 is a duplicate-filtered split of the public SROIE train pool, not an external test; Qwen may have seen it in pretraining. The original 100-doc set is 60% leaked. The invoice set is synthetic, vendor-made, and page-1 only; real B2B supplier invoices are unmeasured, and flagging fails (0/7). A separate real Portuguese receipt/invoice probe (five fields) reaches 72.0% field accuracy and 22/100 fully correct on a held-out 100-document set with the merge fix shipped in 0.3.1; the development set gets 74.2% / 18, and the 0.3.0 merge gets 64.8% / 3/100 because it double-counts VAT on tax-inclusive receipts. The opt-in targeted tax retry in 0.3.1 moves a fresh 100-document set from 401/570 = 70.4% to 411/570 = 72.1% (tax 51/82 to 61/82, fully correct 19 to 21) with no field regression. The 0.3.2 lone-tax total fallback moves the seed-44 path from 401/570 to 404/570 and fully 19 to 22, and a targeted 28-document toll/parking set from 111/139 to 114/139 and fully 4 to 7; a representative seed-45 sample triggered it 0 times. The multipage probe is 10 synthetic documents at 1400 px, 2B only, and descriptive: all-pages gets 4/10 fully correct and page-wise VLM gets 6/10; deterministic reconciliation plus OCR-read printed totals reaches 10/10 on the nine key-header fields, but that is a pipeline result, not a pure VLM score, with a 69.2–100% exact 95% CI at n=10. Line items remain weak (0.848 F1, 1/10 all right), page-wise is ~2× slower, the OCR parser was not held out from these pages, and the 181-document page-1 numbers remain VLM-only. DocVQA is a 200-image public reading probe with one reference answer per question; it is not comparable to leaderboard numbers. These numbers are for same-document engine comparisons, not generalization claims. Recompute with python3 evaluation/phase0/recompute_scores.py.

What you get back

Not just fields: the page, the span, and a score for every value.

TinyDoc result table showing field name, value, exact evidence quote with page and bbox, and confidence; a 0.61 total is routed to review
Evidence is OCR-span provenance. Confidence is for review routing; document-level scores are not calibrated.

Quick start

One pip install, one local model, one call.

# 1. install
pip install tinydoc==0.3.2

# 2. pull a local reader (choose one)
ollama pull qwen2.5vl:3b            # quality
ollama pull qwen3-vl:2b-instruct    # fast

# 3. extract
tinydoc extract ./receipts/ \
  --engine ollama:qwen2.5vl:3b \
  --out results.jsonl --overlay overlays/
from tinydoc import ReceiptPipeline

pipe = ReceiptPipeline("ollama:qwen2.5vl:3b")
result = pipe.extract("receipt.jpg")

print(result.fields)
print(result.schema_valid)

for field in result.field_results:
    print(field.name, field.value,
          field.confidence)
    print(field.evidence.quote,
          field.evidence.bbox)

Large scans: keep TINYDOC_OLLAMA_NUM_CTX=8192 (default). Do not send Ollama format:"json"; constrained decoding truncates long addresses.

Questions

Short answers, with the caveats attached.

What is TinyDoc?

TinyDoc is an open-source Python SDK and CLI that turns receipt and invoice images into schema-validated JSON, with exact OCR-span evidence and per-field confidence. It runs local Qwen2.5-VL-3B or Qwen3-VL-2B-Instruct models through Ollama. The SDK is Apache 2.0 and available as pip install tinydoc.

Is there a local UI?

Yes. 0.3.2 is on PyPI and adds tinydoc ui, a local Gradio app for receipt and multipage invoice extraction, and tinydoc invoice, a page-wise PDF/CLI path with deterministic OCR total reconciliation, an opt-in --retry-tax-px high-resolution tax retry, and a default lone-tax total fallback for toll/parking receipts. It is local-only.

Does TinyDoc ship its own model?

No. The reader is a third-party Qwen vision-language model. TinyDoc adds schema prompts, JSON validation and salvage, OCR-span evidence, confidence, overlays, and the CLI/SDK. The first-party 256M and SmolVLM2 tracks were measured, failed to beat the free base model, and were closed.

Does it work on invoices?

On a synthetic public benchmark of 181 invoices (page 1 only), it reaches 91.9% key-header accuracy and 65.7% fully correct with the 3B, and 93.5% / 71.8% with the 2B. A separate real Portuguese receipt/invoice probe (five fields) reaches 72.0% field accuracy and 22/100 fully correct on a held-out 100-document set with the merge fix shipped in 0.3.1; the development set gets 74.2% / 18, and the 0.3.0 merge gets 64.8% / 3/100 because it double-counts VAT on tax-inclusive receipts. The opt-in targeted tax retry in 0.3.1 moves a fresh 100-document set from 401/570 = 70.4% to 411/570 = 72.1% (tax 51/82 to 61/82, fully correct 19 to 21) with no field regression. The 0.3.2 lone-tax total fallback moves the seed-44 path from 401/570 to 404/570 and fully 19 to 22, and a targeted 28-document toll/parking set from 111/139 to 114/139 and fully 4 to 7; a representative seed-45 sample triggered it 0 times. Real B2B supplier invoices are unmeasured and flagging fails (0/7); multipage is a separate 10-document probe. The 0.3.2 SDK exposes the page-wise method, but that is probe-quality.

How well does it read documents?

On a 200-image public DocVQA subset (one question per image, 768 px mirror, one reference answer), Qwen3-VL-2B-Instruct scores 0.829 ANLS and 0.765 exact match with zero errors. Official DocVQA uses up to 10 references, so this single-reference number is not comparable to leaderboard scores; the set is public and may be in pretraining.

Does it handle multipage invoices?

The evaluation harness can send every page, or read one page per request and merge deterministically. On 10 synthetic multipage invoices (46 pages, 2B), all-pages gets 4/10 fully correct and 71.1% key-header, with two 8-page runaway failures; page-wise VLM gets 6/10 fully correct and 93.3% key-header. Adding deterministic reconciliation — last-page currency, arithmetic repairs, and OCR-read printed totals accepted only when subtotal + tax == total — reaches 10/10 on the nine key-header fields, with an exact 95% CI of 69.2–100% at n=10. That is a pipeline result, not a pure VLM score: line-item F1 is 0.848, all lines are right on 1/10, page-wise is ~2× slower, and the 0.3.2 SDK exposes the page-wise path, but that is probe-quality. A deterministic degraded copy of the same 46 pages completed 8 of 10 documents, with 7 of 8 fully correct, 71 of 72 key-header fields, and 2 page failures.

Does my data leave my machine?

No. Inference runs against a local Ollama server. There is no hosted API, no telemetry, and no API key in the SDK.

Is the confidence calibrated?

Field-level scores are usable for review routing after document-grouped isotonic out-of-fold calibration: ECE 0.005–0.014. Document-level confidence is not calibrated (ECE 0.09–0.10) and is not a probability. Treat every score as a routing signal, not a decision.

Can it predict bounding boxes?

It returns OCR-span boxes as page-derived provenance. VLM-predicted boxes were measured at hit@0.5 of 0.04–0.20 against a 0.70 gate, so they are not claimed or shipped.

How fast is it?

Median latency on an M4 through Ollama is 7.0 s (2B) and 14.1 s (3B) per receipt, and 16.1 s / 26.5 s per invoice. That is above the 4 s target; it is a batch tier until MLX or narrower prompts land.

What is the license?

The TinyDoc SDK is Apache 2.0. The Qwen base models are Apache 2.0; check their model cards for terms.