Skip to content
docstack-ocr

Languages & internationalization

The platform is multilingual by virtue of its models — PaddleOCR-VL-1.6 (≈109 languages), GLM-OCR, and whichever extraction LLM you configure. The orchestration prompts are written in English, but every stage is made intelligent about the document’s language: the platform detects it once and threads the verdict into the OCR-trust gate, the extraction and judge prompts, and grounding.

Detection

Section titled “Detection”

At canonical assembly (so single- and multi-page documents are covered) the worker computes a detected_language from two signals:

  • Script + reading direction — a deterministic Unicode histogram over the OCR text. Zero dependency; this is what drives right-to-left handling and a script-appropriate language candidate set.
  • Language — py3langid (offline, model bundled in the wheel) disambiguates same-script languages (es/fr/pt/de within Latin, ru/sr/bg within Cyrillic) with a 0–1 confidence.

On multi-page documents the characteriser also names the primary_language; it refines a close same-script sibling that py3langid confuses on flat text (e.g. Galician vs Spanish), never flipping script.

The verdict — language, script, reading direction, confidence, source — rides on the canonical and shows as a chip on the document detail view. Metric: docstack_document_language_total{language,script,source}.

What the language signal drives

Section titled “What the language signal drives”
  • OCR trust — Arabic is a day-one language and GLM Arabic output is trusted by default (the other risk gates still apply). Opt out per tenant with glm_risk_policy.glm_arabic_supported=false.
  • Extraction / judge / classifier prompts — a generic English === Document Language === preamble names the language, script, and reading direction, and adds a one-line guidance fragment for languages with a known pitfall (RTL + Eastern-Arabic digits, German compounding). The model is told to interpret foreign labels by meaning and output values verbatim — it never translates extracted values.
  • Grounding — the per-field confidence check recognises printed prose dates across the day-one + campaign languages (including year-first orderings like Hungarian 2030 március 15), so a correctly-extracted foreign date is not penalised.

Structured output is template-driven

Section titled “Structured output is template-driven”

The product has no generic named-entity mode. output_mode=document returns faithful OCR in the source language; output_mode=extraction adds only the fields requested by the selected template. The extraction LLM receives the detected language, but every emitted value remains grounded in the canonical source evidence and is not silently translated.

Configuring

Section titled “Configuring”

language_policy (platform or tenant scope, editable in the admin UI):

  • detection_enabled (default true) — detect and thread the language.
  • language_hints_enabled (default true) — inject the prompt preamble.
  • language_allowlist (default empty = all 97) — restrict py3langid’s candidate ISO-639-1 codes. Narrowing to your corpus’s languages cleanly separates close same-script siblings (set ["en","es","fr","de"] and a Spanish invoice stops being labelled Galician).