Jina AI's 3.4B MoE Parser Runs on Budget GPUs

Jina AI, the Berlin-based search and embeddings company now under Elastic, released jina-ocr-v1 this week, a document parser built to run on GPUs most labs can actually afford. The model takes a PDF, a scanned invoice or a chart-heavy report and turns it into clean Markdown in a single pass, tables rendered as HTML, formulas in LaTeX. That's the pitch, anyway: 3.4 billion parameters total, but only about 570 million decoder parameters active per token, thanks to a mixture-of-experts setup that routes each token through 6 of 64 experts. Jina AI says it targets hardware like the NVIDIA L4, a card built for inference budgets rather than flagship training rigs.

The company built jina-ocr-v1 on top of DeepSeek-OCR, keeping its two-stage encoder-decoder design rather than starting from scratch. Weights are open at roughly 6.8 GB in BF16, compatible with Transformers or vLLM, though the CC BY-NC 4.0 license restricts commercial use without contacting Jina directly. On benchmarks, the technical report cites scores on OmniDocBench v1.6 and olmOCR-Bench. What makes the release notable isn't just the size class, it's a decoding trick baked into the checkpoint itself.