TinyDoc-VLM
The World's Smallest Document-Specialist VLM — 256M parameters, runs on a Raspberry Pi, open-source.
🔥 Highlights
256M Parameters
SigLIP-B/16 vision encoder + Pixel-Shuffle compressor + SmolLM2-135M decoder. Optimized for edge.
<1GB VRAM
Runs on Raspberry Pi 5, MacBook Air, or any CPU with ONNX export. Over 100 tok/s on CPU.
Structured Output
JSON extraction, key-value pairs, table parsing, OCR, and VQA — all from a single unified model.
3-Stage Training
Layout pretrain → Document understanding → Instruction tuning on 10K+ synthetic documents.
One pip Install
pip install tinydoc — Python SDK with Pydantic-typed results. PyTorch and ONNX backends.
Open Weights
Pre-trained weights on HuggingFace Hub. Apache 2.0 license — free for commercial use.
🚀 Quick Start
pip install tinydoc
# Use it in Python
from PIL import Image
from tinydoc import TinyDocExtractor
extractor = TinyDocExtractor(device="cpu")
img = Image.open("invoice.png")
# Ask a question
result = extractor.ask(img, "What is the total?")
print(result.answer)
# Extract JSON fields
result = extractor.extract(img, output_format="json")
print(result.fields)
# Extract tables
result = extractor.extract_table(img)
print(result.markdown)
🏗️ Architecture
Image (384×384)
↓
SigLIP Vision Encoder (93M) ← 576 patches × 768 dim
↓
Pixel-Shuffle Compressor (scale=3) ← 9× compression → 64 tokens
↓
Visual Position Embeddings
↓
SmolLM2 Decoder (135M) ← 30 layers, GQA (9:3 heads), 8192 ctx
↓
Multi-Task Output Heads
↓
JSON / KV Extraction / Table / OCR / QA
📊 Benchmarks
| Benchmark | Metric | Score | Status |
|---|---|---|---|
| DocVQA | ANLS | — | ⏳ Running |
| FUNSD | F1 | — | ⏳ Running |
| CORD | F1 | — | ⏳ Running |
| SROIE | F1 | — | ⏳ Running |
| PubTabNet | TEDS | — | ⏳ Running |
🔗 Ecosystem
🐍 PyPI
pip install tinydoc — pypi.org/project/tinydoc
🤗 Model Hub
Pre-trained weights — eulogik/TinyDoc-VLM-256M
🤗 Space Demo
Try it live — eulogik/TinyDoc-VLM Space
📖 Documentation
Full docs — GitHub README
📄 Paper
Technical report — coming soon
Follow — @eulogik