# Languages & internationalization

The platform is multilingual by virtue of its models — PaddleOCR-VL-1.6
(≈109 languages), GLM-OCR, and whichever extraction LLM you configure.
The orchestration prompts are written in English, but every stage is
made **intelligent about the document's language**: the platform detects
it once and threads the verdict into the OCR-trust gate, the extraction
and judge prompts, and grounding.

## Detection

At canonical assembly (so single- *and* multi-page documents are
covered) the worker computes a `detected_language` from two signals:

- **Script + reading direction** — a deterministic Unicode histogram
  over the OCR text. Zero dependency; this is what drives right-to-left
  handling and a script-appropriate language candidate set.
- **Language** — `py3langid` (offline, model bundled in the wheel)
  disambiguates same-script languages (es/fr/pt/de within Latin,
  ru/sr/bg within Cyrillic) with a 0–1 confidence.

On multi-page documents the characteriser also names the
`primary_language`; it refines a close same-script sibling that
`py3langid` confuses on flat text (e.g. Galician vs Spanish), never
flipping script.

The verdict — language, script, reading direction, confidence, source —
rides on the canonical and shows as a chip on the document detail view.
Metric: `docstack_document_language_total{language,script,source}`.

## What the language signal drives

- **OCR trust** — Arabic is a day-one language and GLM Arabic output is
  **trusted by default** (the other risk gates still apply). Opt out per
  tenant with `glm_risk_policy.glm_arabic_supported=false`.
- **Extraction / judge / classifier prompts** — a generic English
  `=== Document Language ===` preamble names the language, script, and
  reading direction, and adds a one-line guidance fragment for languages
  with a known pitfall (RTL + Eastern-Arabic digits, German
  compounding). The model is told to interpret foreign labels by meaning
  and output values **verbatim** — it never translates extracted values.
- **Grounding** — the per-field confidence check recognises printed
  prose dates across the day-one + campaign languages (including
  year-first orderings like Hungarian `2030 március 15`), so a
  correctly-extracted foreign date is not penalised.

## NER is English-primary

The optional `entities` processing mode (GLiNER2) and its Universal
Entity Taxonomy are **English**. Entity quality on non-English documents
is best-effort and not a v1 guarantee — it is intentionally not swapped
for a multilingual model. Multilingual support lives in the
**template-extraction** path described above. When the entities stage
runs on a detected non-English document it logs an advisory and bumps
`docstack_entities_non_english_total{language}`.

## Configuring

`language_policy` (platform or tenant scope, editable in the admin UI):

- `detection_enabled` (default `true`) — detect and thread the language.
- `language_hints_enabled` (default `true`) — inject the prompt preamble.
- `language_allowlist` (default empty = all 97) — restrict `py3langid`'s
  candidate ISO-639-1 codes. Narrowing to your corpus's languages
  cleanly separates close same-script siblings (set
  `["en","es","fr","de"]` and a Spanish invoice stops being labelled
  Galician).