Integration Guide

Integration Guide

This is the definitive reference for system integrators embedding docstack-ocr — a self-hosted, dual-VLM document understanding platform — into a larger system.

It documents every public HTTP endpoint, both authentication modes (bearer and trusted_headers), the CanonicalOcrDocument response shape, the template + validation DSL, the review workflow, and the operational surfaces (infra endpoints, audit log, health, metrics) you will need to operate the platform in production.

The companion deep-dive specifications live in docs/architecture.md (pipeline + serving layer) and docs/admin-iam-design.md (identity model). This guide is consciously narrower: it tells an integrator how to call the API. It does not assume you have read either of those documents.

Audience. Backend engineers writing the integration glue between docstack and an upstream business system (workflow engine, ERP, ECM, RPA orchestrator, internal portal). Familiarity with HTTP, JSON, and Bearer auth is assumed.

Source of truth. Every endpoint, parameter, and field below was extracted from api/routers/, api/schemas/, api/db/models.py, ocr/base.py, and validation/rules.py at the commit this document ships against. When in doubt, the code wins — please file an issue if you spot drift.

---

1. Platform overview

docstack-ocr ingests a document (raster image, PDF, vector, or Office file) and returns a rich, model-agnostic JSON description of its content.

The pipeline runs in two stages:

1. OCR — Every input region is OCR’d by PaddleOCR-VL-1.6 and GLM-OCR in parallel, then merged into a single CanonicalOcrDocument. The winner is selected per region by tenant policy (PaddleOCR priority, GLM-OCR priority, or confidence-based), with GLM risk gates preventing suspect GLM output from displacing usable Paddle output. This always runs.
2. Extraction (optional) — When a tenant has authored a template (a JSON Schema + validation DSL), the OCR markdown is fed to a model-agnostic LLM endpoint that emits structured fields conforming to the template, then validated against the DSL.

Both stages are tenant-scoped and run inside an in-process SAQ worker. Job submission is asynchronous: POST /v1/documents returns 202 Accepted with a job_id; clients poll GET /v1/documents/{job_id} until status is terminal (completed, needs_review, rejected, or failed).

Why integrators pick this platform

• Self-hosted, single-binary deploy. All components ship as a Docker Compose stack — no managed cloud dependency, no per-document SaaS billing.
• Model-agnostic LLM. Any OpenAI-compatible chat-completions server (vLLM, Ollama — local or Ollama Cloud — gpt-oss, qwen3, deepseek-r1, Gemma) plugs in via runtime configuration, and Google Gemini and Anthropic are first-class via their native SDKs (Anthropic adds provider-side enforced structured output via tool-use). No code changes.
• Tenant-authored templates. Document types are defined at runtime, not in source. Each tenant gets its own catalog.
• First-class review workflow. Failed validation routes to a review queue with approve / reprocess / field-level override flows, all idempotent.
• Soft-delete + audit log everywhere. Every mutation is auditable; every job is recoverable until hard-purged.
• Two auth modes. Use the built-in bearer + session model, or front the platform with your own gateway and forward identity via trusted headers.

What is not in scope

• No webhooks (yet). Status changes are discoverable only via polling GET /v1/documents/{job_id}. The platform is designed for 5–15 second job latencies where polling is acceptable. See §16 Webhooks for the rationale.
• No batch streaming results. Results are read job-by-job; batch endpoints return aggregates and a job list, not concatenated payloads.
• No multi-region deployment. A single deployment is one logical region. Operators run multiple deployments for geo-distribution.

---

2. Quickstart

The fastest path from “I cloned the repo” to “my first extracted document”:

Step 1 — Boot the stack

# Required env (in .env):
#   POSTGRES_PASSWORD=<random>
#   MINIO_ROOT_USER=<random>
#   MINIO_ROOT_PASSWORD=<random>
#   ARTIFACT_SIGNING_SECRET=<random>
#   SESSION_SECRET=<random — python -c "import secrets; print(secrets.token_urlsafe(32))">
#   BOOTSTRAP_PLATFORM_ADMIN_ENABLED=true
#   BOOTSTRAP_PLATFORM_ADMIN_EMAIL=admin@yourcompany.com
#   BOOTSTRAP_PLATFORM_ADMIN_PASSWORD=<choose one>
#   LLM_URL=https://your-vllm.internal:8000  (or any OpenAI-compatible server)
#   LLM_MODEL=qwen3-8b-instruct              (or whatever the server hosts)

# Application + infra (no GPU model servers):
docker compose -f docker/docker-compose.yml up -d

# Or include the bundled local PaddleOCR-VL + GLM-OCR servers
# (single-GPU, 24 GiB):
docker compose -f docker/docker-compose.yml \
               -f docker/docker-compose.local-models.yaml up -d

POSTGRES_PASSWORD, MINIO_ROOT_USER, MINIO_ROOT_PASSWORD, ARTIFACT_SIGNING_SECRET, and SESSION_SECRET are hard-required by the compose file (:? fallback). The API container will refuse to start without them.

Step 2 — Confirm health

curl http://localhost:8080/v1/health
# → { "status": "ok", "components": { ... }, "queue_depth": 0 }

Step 3 — Create a tenant + invite a user

Once BOOTSTRAP_PLATFORM_ADMIN_ENABLED=true provisions the first platform_admin, log into the admin UI at http://localhost:3010 (or use the CLI):

docstack-admin tenant create --id acme --name "Acme Corp" \
    --initial-admin-email admin@acme.com
docstack-admin user invite --tenant-id acme \
    --email integrator@acme.com --role tenant_admin

The invite emits an accept-invite URL the user opens to set a password.

Step 4 — Mint an API key

docstack-admin api-keys create --tenant-id acme \
    --user-email integrator@acme.com --name backlog-sync
# → printed once: rk_<public>_<secret>

Capture the full key. The secret is unrecoverable.

Step 5 — Submit a document

API="http://localhost:8080"
KEY="rk_<public>_<secret>"

JOB=$(curl -s -X POST "$API/v1/documents" \
    -H "Authorization: Bearer $KEY" \
    -F "file=@invoice.pdf" \
    | jq -r .job_id)
echo "submitted: $JOB"

Step 6 — Poll until terminal

while true; do
  RESPONSE=$(curl -s -H "Authorization: Bearer $KEY" \
      "$API/v1/documents/$JOB")
  STATUS=$(echo "$RESPONSE" | jq -r .status)
  echo "status: $STATUS"
  case "$STATUS" in
    completed|needs_review|rejected|failed) break ;;
  esac
  sleep 2
done
echo "$RESPONSE" | jq .canonical.markdown

That’s the full happy path. The remainder of this guide details every endpoint, every failure mode, and every advisory signal you can wire into your downstream system.

---

3. URL conventions

Concept	Convention
Base URL	Whatever you bind the API container to. The default compose maps 8080:8080. The internal canonical URL is set via PUBLIC_BASE_URL and is used to mint signed artifact URLs.
API version prefix	/v1/…. There is one active version. Breaking changes will land under /v2/… alongside /v1/… with a deprecation window.
ID prefixes	doc_ (documents), art_ (artifacts), tpl_ (templates), tpla_ (template audits), rq_ (reviews), cred_ (credentials), batch_ (batches), draft_ (template drafts), conn_ (connections). IDs are otherwise opaque — do not parse beyond the prefix.
Pagination	Offset + limit on every list endpoint. limit ∈ [1, 200], default 50. offset ≥ 0, default 0. Responses include total so callers can compute page counts.
Datetimes	ISO 8601 with explicit Z suffix for UTC. Timestamps are server-side; the platform does not honor client-supplied timestamps.
Tenant scoping	Every read and write is filtered by the resolved tenant. Cross-tenant access returns 404 (not 403) so existence is not leaked.
Idempotency	Mutations marked idempotent below can be safely retried. Non-idempotent mutations should be retried only on 5xx or network failure, not on 4xx.

---

4. Authentication

The platform supports two mutually exclusive auth modes selected at boot via the AUTH_MODE environment variable.

4.1 Bearer mode (default)

Set AUTH_MODE=bearer. Two credential types are accepted:

API keys (machine-to-machine)

Format: rk_<public_id>_<secret>

• public_id — 16 hex chars (≈ 64 bits of entropy)
• secret — base64url, 32 bytes (≈ 256 bits)

Send via Authorization: Bearer rk_<public_id>_<secret> header. API keys are owned by a user and inherit that user’s role; revoking the user revokes all their keys.

API keys are CSRF-exempt. They are the recommended credential for backend integrations.

Session cookies (interactive humans)

Issued by POST /v1/auth/login. Two cookies are set:

Cookie	Purpose	Flags
dsx_session	Signed session token (HMAC over user_id + expiry)	HttpOnly, Secure (production), SameSite=lax
dsx_csrf	Random CSRF token for double-submit	Secure (production), SameSite=lax, not HttpOnly (JS reads it)

Mutating requests (POST / PUT / PATCH / DELETE) carrying a session cookie must also send the CSRF token in the X-CSRF-Token header (configurable via csrf_header_name). The middleware checks dsx_csrf cookie value matches the header value (constant-time compare).

Sessions roll on every request: TTL extends by session_max_age_seconds. Idle sessions expire; active sessions stay alive indefinitely.

Cookie names and the CSRF header name are all configurable via api/config.py (session_cookie_name, csrf_cookie_name, csrf_header_name).

4.2 Trusted-headers mode

Set AUTH_MODE=trusted_headers. Use this when the platform sits behind an upstream gateway (Kong, Envoy, custom) that already authenticates the caller and forwards identity via headers.

Header	Required	Purpose
X-Tenant-Id	yes	Opaque tenant identifier. The platform performs no further check — your gateway is fully trusted.
X-Actor-Id	no	User or credential identifier (used for audit log only).
X-Actor-Roles	no	Comma-separated or JSON array of roles. Recognised values: platform_admin, tenant_admin, tenant_user. Legacy admin/user are normalised to the new tier names. Unknown values pass through (forward-compat).
X-Request-Id	no	Correlation ID; echoed in the response and emitted in structured logs.

In this mode /v1/auth/* endpoints return 501 Not Implemented. Sessions and password flows do not apply.

Security warning. In trusted_headers mode, anyone who can reach the API can impersonate any tenant. Bind the platform to a private network and front it with a gateway you control.

4.3 Roles

Three roles, hierarchical:

platform_admin > tenant_admin > tenant_user

• platform_admin — Operator. Reads and writes any tenant; manages tenants, infra endpoints, platform defaults, and the cross-tenant audit log. Cannot be created or promoted via HTTP — use the CLI (docstack-admin platform-admin create) for D19 safety.
• tenant_admin — Manages users, API keys, templates, validation rules, tenant policies, and connections within their tenant. Can include_deleted=true on list endpoints.
• tenant_user — Submits and reads documents; views own API keys; can author template drafts but not accept them.

Each endpoint declares its minimum role (User+, Admin+, or Platform_admin) in the reference below.

4.4 Rate limits

Two layers:

1. Per-tenant write rate limit. Configurable via tenant_policies.rate_limit_per_minute (default 100/min). All mutating requests count. Responses include X-RateLimit-Limit and X-RateLimit-Remaining headers. Exceeding returns 429 with Retry-After.
2. Per-IP auth rate limit. Sliding window for /v1/auth/login, /v1/auth/accept-invite, /v1/auth/reset-password, and /v1/auth/change-password. Threshold: login_rate_limit_per_15min (default 5 per 15 min, shared across the four endpoints). Returns 429 with Retry-After.

4.5 CORS and security headers

CORS is configured via the CORS_ORIGINS env var (JSON array of allowed origins). Default: ["http://localhost:3000"]. Allowed methods are GET, POST, PUT, PATCH, DELETE, OPTIONS; allowed headers include Authorization, Content-Type, X-Request-Id, X-Tenant-Id, and the configured CSRF header. Credentials are allowed (cookies + auth header).

Every response carries a fixed security header set:

X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Security-Policy: default-src 'none'
Referrer-Policy: no-referrer
Strict-Transport-Security: max-age=31536000; includeSubDomains

---

5. Documents

The core endpoint surface. Submit a document, poll for status, retrieve results, and manage the job lifecycle.

5.1 Submit a document

POST /v1/documents
Authorization: Bearer rk_...
Content-Type: multipart/form-data

Form fields

Field	Type	Required	Notes
file	binary	yes	The document. See Supported content types below.
template_id	string	no	Pin extraction to a specific template version.
template_name	string	no	Pin to the latest active version of a template name. template_id wins if both are supplied.
processing_mode	string	no	How far the pipeline runs: extraction (default), entities, or canonical. See §5.10 Processing modes.
page_range	string	no	Print-style PDF page selection, 1-indexed (e.g. 1-3,5,12-14). The last token resolves to the final page, so 1-10,last is “first 10 + last” without knowing the page count. Omitted / empty / all = every page. Only the selected pages are rendered + OCR’d, and the page limit below applies to the selected count — so a range can pull a few pages out of a very large PDF. Original page numbers are preserved on the result (canonical.pages[].source_page_number). Malformed ranges return 400.

If neither template selector is supplied, the engine resolves a template automatically (auto-classify, opt-in per tenant via tenant_policies.auto_classify_enabled) or returns canonical OCR only.

processing_mode is optional and defaults to extraction (the OCR → template-extraction pipeline). Set it to entities for OCR + named-entity recognition instead, or canonical to stop after OCR. NER and template extraction are isolated pipelines: an extraction job never runs the NER stage (its canonical.entities is always empty), and an entities job never runs template extraction. A non-extraction mode combined with template_id / template_name is 400 Bad Request — those modes never run template extraction, so the combination is contradictory.

Response — 202 Accepted

{
  "job_id": "doc_01HX...",
  "deduplicated": false,
  "status": "queued"
}

Deduplication. If the same content hash was submitted within dedupe_window_hours, the second call returns deduplicated: true and the original job_id; no new job is enqueued. Use POST /v1/documents/{job_id}/reprocess if you intentionally want to re-run.

Supported content types

Family	MIME types	Size limit	Page limit
Raster	image/jpeg, image/jpg, image/png, image/tiff, image/webp, image/bmp, image/gif	MAX_UPLOAD_SIZE_BYTES (default 10 MiB)	n/a
Vector	image/svg+xml, image/emf, image/x-emf, image/vnd.ms-emf, image/wmf, image/x-wmf, image/vnd.ms-wmf	same as raster	n/a
PDF	application/pdf	PDF_MAX_FILE_SIZE_BYTES (default 50 MiB)	the PDF page limit (default 50) applies to the selected pages — use page_range to process specific pages of a larger PDF. The limit is a runtime policy (pdf_max_pages, platform or tenant scope, editable in the admin UI without restart; PDF_MAX_PAGES is the env fallback).
Office	DOCX, PPTX, XLSX (full OOXML MIME types)	OFFICE_MAX_FILE_SIZE_BYTES (default 100 MiB)	selected pages of the rendered PDF must be ≤ PDF_MAX_PAGES

Office files are converted to PDF via an embedded Gotenberg service (LibreOffice under the hood) before OCR. Operators can run a remote/managed Gotenberg by overriding GOTENBERG_URL.

Failure responses

Status	When
413 Payload Too Large	Content-Length exceeds the per-format size limit.
415 Unsupported Media Type	MIME type not in the allowlist.
400 Bad Request	Both template_id and template_name set with conflicting values, template not found, processing_mode is not one of extraction / entities / canonical, or a non-extraction processing_mode was paired with a template selector.
429 Too Many Requests	Per-tenant write rate limit exceeded.

5.2 List documents

GET /v1/documents?status=completed&limit=50&offset=0

Query param	Type	Default	Notes
status	enum	(all)	One of queued, processing, completed, needs_review, rejected, failed.
template_id	string	(any)	Filter by resolved template ID.
template_name	string	(any)	Filter by template name.
since	ISO 8601	(none)	Lower-bound on created_at.
until	ISO 8601	(none)	Upper-bound on created_at.
q	string	(none)	Exact prefix match on job_id.
include_deleted	bool	false	Admin+ only. Surfaces soft-deleted rows; non-admin callers get 403 if set to true.
limit	int	50	1–200.
offset	int	0	≥ 0.

Response — 200 OK

{
  "items": [
    {
      "job_id": "doc_01HX...",
      "status": "completed",
      "source_format": "image | pdf | vector | office",
      "processing_mode": "extraction | entities | canonical",
      "page_count": 1,
      "requested_template_id": "tpl_... | null",
      "requested_template_name": "acme_invoice | null",
      "resolved_template_id": "tpl_... | null",
      "resolved_template_version": 3,
      "review_decision": "accept | review | reject | null",
      "review_reasons": ["..."],
      "failure_code": null,
      "failure_message": null,
      "created_at": "2026-04-21T08:30:00Z",
      "completed_at": "2026-04-21T08:30:14Z",
      "deleted_at": null
    }
  ],
  "total": 1234,
  "limit": 50,
  "offset": 0
}

The list response is intentionally thin: no canonical, extraction, validation_flags, or artifacts. Use GET /v1/documents/{job_id} to load full details.

5.3 Fetch a document

GET /v1/documents/{job_id}

Returns the full document job response — see §7 CanonicalOcrDocument schema for the complete shape.

While status is queued or processing, canonical and extraction are null and validation_flags is []. Soft-deleted jobs are still readable (with deleted_at set) so audit access remains intact.

Cross-tenant access returns 404.

5.4 Retry a failed job

POST /v1/documents/{job_id}/retry

Re-enqueue a failed or rejected job with the same input and the same template. Returns 409 Conflict if the job is in any other status.

Response — 202 Accepted

{ "job_id": "doc_01HX...", "status": "queued", "failure_code": null }

Side effects: retry_count incremented; failure_code, failure_message, started_at, completed_at cleared. The same artifact is re-used; the job ID does not change. Bypasses dedup.

5.5 Reprocess a terminal job

POST /v1/documents/{job_id}/reprocess
Content-Type: application/json

{
  "template_id": "tpl_new",      // optional
  "clear_template": false,        // optional
  "processing_mode": "entities",  // optional
  "page_range": "1-3"             // optional
}

Re-enqueue any terminal job (completed, needs_review, rejected, failed) with optional template, processing-mode, or page-range change.

Body	Effect
Empty / both null	Re-run with the existing template assignment.
template_id set	Swap to the new template (must exist + be active).
clear_template: true	Drop the template; re-run as canonical OCR + auto-classify.
page_range set	Re-render a different page selection — e.g. narrow a job that failed “all pages” over the cap to 1-3, or all to widen back. Omitted keeps the current selection; malformed → 400.
Both template selectors supplied	400 Bad Request.

processing_mode is optional. When omitted, the job keeps its current mode. When supplied it must be one of extraction, entities, or canonical (see §5.10 Processing modes) — switching to a non-extraction mode while also passing template_id is 400 Bad Request, since those modes never run template extraction.

5.5b Re-extract from the stored canonical (no re-OCR)

POST /v1/documents/{job_id}/reextract
Content-Type: application/json

{
  "template_id": "tpl_new"        // optional; or "template_name"
}

The cheap path for “the template (or LLM endpoint) changed — re-run extraction on this document”. Unlike /reprocess, the document is not re-OCR’d: the worker rebuilds the canonical from the stored result and re-runs only the extraction stage (LLM call + judge + validation), then updates the result, status, and review queue in place. Typical latency is one LLM round-trip instead of the full pipeline.

Constraints (violations return 409 unless noted):

• The job must be terminal with a stored canonical: completed, needs_review, or rejected. For failed jobs (which may have no canonical) use /retry.
• The job’s processing_mode must be extraction (400 otherwise — other modes never run template extraction; use /reprocess to change mode).
• Body is optional. {} re-applies the job’s current template; pass template_id or template_name (not both — 400) to swap. If the job has no template association at all and none is supplied, the call is 409.

Returns 202 Accepted with the job’s current (unchanged) status; the status flips to the new outcome (completed / needs_review / rejected) when the worker finishes. Use /reprocess instead when the file itself, the page selection, or the processing mode changed — those need a fresh OCR pass.

5.6 Cancel a queued job

POST /v1/documents/{job_id}/cancel

Cancel a queued job. Returns 409 Conflict for any other status — in-flight cancellation is not supported because the worker pipeline is not interruptible mid-region.

Response — 200 OK

{
  "job_id": "doc_01HX...",
  "status": "failed",
  "failure_code": "cancelled",
  "failure_message": "Job cancelled by user"
}

Idempotent: cancelling an already-cancelled job returns 200 with the same payload. Implementation uses an atomic conditional UPDATE paired with the worker’s claim guard so exactly one of {user cancel, worker claim} wins.

5.7 Soft-delete a job

DELETE /v1/documents/{job_id}

Marks deleted_at. Refuses queued / processing jobs (cancel first). Idempotent: deleting an already-deleted job returns 200 with the same payload.

Side effects:

• Open review_queue rows for the job are closed with status='closed_by_delete'.
• Artifacts remain in storage (no hard purge).
• GET /v1/documents/{job_id} still returns the row; list endpoints hide it unless include_deleted=true is passed by an admin.

5.8 Restore a soft-deleted job

POST /v1/documents/{job_id}/restore

Admin+ only. Clears deleted_at. Idempotent.

5.9 Re-upload context (UI helper)

GET /v1/documents/{job_id}/reupload-context

Returns the minimal metadata an “upload corrected version” UI needs to prefill: template_id, template_name, batch_id, source_format, processing_mode. Readable even on soft-deleted jobs.

5.10 Processing modes

processing_mode is a per-job choice of how far the pipeline runs. It is accepted on POST /v1/documents, POST /v1/documents/{job_id}/reprocess, and POST /v1/batches, and is echoed back on every document, list item, and batch object. Three values:

Mode	Pipeline	Result
extraction (default)	OCR → template-based structured extraction. The GLiNER2 NER stage does not run — NER and template extraction are isolated modes.	Canonical markdown and a populated extraction block (when a template resolved). canonical.entities is empty.
entities	OCR → GLiNER2 generic named-entity recognition. Stops before template resolution / extraction.	Canonical markdown and canonical.entities. extraction is null.
canonical	OCR only. Stops at the canonical document.	Canonical markdown. No NER, no template extraction; canonical.entities is empty and extraction is null.

When to use each:

• extraction — you have (or auto-classify resolves) a template and want structured fields validated against its DSL. This is the default; omit processing_mode to get it.
• entities — you want canonical OCR plus document-agnostic typed entities (people, organisations, dates, amounts, …) but have no template, or do not need template-shaped output. This is the only mode that runs NER; if you need both typed entities and template-shaped output for the same document, submit one job per mode.
• canonical — you want OCR output only (markdown, layout, tables, formulas) and nothing downstream. The fastest mode; skips both NER and template extraction.

Making “OCR only” and “OCR + NER only” an explicit per-job choice means a no-template job no longer silently falls through the auto-classify / default-template path — the depth is deliberate, not an accident of tenant config.

Mode / template conflict. A non-extraction mode (entities or canonical) combined with a template_id or template_name returns 400 Bad Request with an actionable message — entities and canonical never run template extraction, so pairing them with a template selector is contradictory. Either drop the template selector or set processing_mode to extraction.

The mode is fixed for the life of a job. To change it, POST /v1/documents/{job_id}/reprocess with a new processing_mode (see §5.5).

---

6. Job lifecycle

queued ──┬─ processing ──┬─ completed
         │               ├─ needs_review ─ (review action) ─ completed | (re-enqueue) ─ queued
         │               ├─ rejected     (catastrophic schema parse failure)
         │               └─ failed       (any other terminal failure)
         │
         └─ failed[failure_code=cancelled]    (POST /cancel before claim)

Status	Meaning	Terminal?
queued	Submitted, awaiting worker claim.	no
processing	Worker is executing the pipeline.	no
completed	Pipeline finished; validation passed (or no template).	yes
needs_review	Pipeline finished; validation flagged at least one severity: error or hit the warn-count threshold. A review_queue row exists.	yes
rejected	Catastrophic JSON parse failure (LLM produced output that won’t decode). Rare.	yes
failed	Any other terminal failure. failure_code carries the reason.	yes

Transitions in detail:

• queued → processing: SAQ worker claims the job. started_at set.
• processing → completed | needs_review | rejected | failed: worker writes the result. completed_at set.
• failed | rejected → queued: via POST /retry. retry_count incremented; timestamps cleared.
• * → queued (terminal except cancelled failure): via POST /reprocess. Optional template swap.
• needs_review → completed: via POST /reviews/{id}/approve or POST /reviews/{id}/override.
• needs_review → queued: via POST /reviews/{id}/reprocess.

---

7. CanonicalOcrDocument schema

The full shape returned by GET /v1/documents/{job_id}. Source: ocr/base.py + api/dependencies.py::build_document_job_response.

{
  "job_id": "doc_01HX...",
  "status": "completed",
  "source_format": "pdf",
  "processing_mode": "extraction",
  "page_count": 5,
  "original_filename": "2026-02-invoice-12345.pdf | null",
  "page_range": "1-3,5 | null",

  "requested_template_id": "tpl_acme_invoice_v3",
  "requested_template_name": "acme_invoice",
  "resolved_template_id": "tpl_acme_invoice_v3",
  "resolved_template_version": 3,

  "review_decision": "accept",
  "review_reasons": [],

  "failure_code": null,
  "failure_message": null,
  "warnings": ["characterisation timed out"],

  "batch_id": "batch_01HX... | null",
  "deleted_at": null,

  "created_at": "2026-04-21T08:30:00Z",
  "started_at": "2026-04-21T08:30:01Z",
  "completed_at": "2026-04-21T08:30:14Z",

  "canonical":      { /* see §7.1 */ },
  "extraction":     { /* see §7.2 */ },
  "validation_flags": [ /* see §7.3 */ ],

  "artifacts": {
    "original":     "http://.../v1/artifacts/art_...?expires=...&tenant=...&token=...",
    "rendered_pdf": "http://.../v1/artifacts/art_...?expires=...&tenant=...&token=..."
  },

  "ocr_confidence": 0.95,
  "ocr_agreement":  0.98
}

canonical, extraction, and validation_flags are populated only after a terminal status. While the job is queued or processing, they are null / [].

processing_mode echoes the per-job pipeline-depth choice (§5.10). When it is canonical or entities, extraction is always null — those modes never run template extraction. Only entities populates canonical.entities from the GLiNER2 NER stage; extraction and canonical leave it empty (NER and template extraction are isolated modes).

warnings is a free-form list of advisory messages for non-fatal anomalies (e.g. characterisation timed out, fast-path fallback fired). They never block downstream consumption.

artifacts.rendered_pdf is only present for Office uploads — it is the Gotenberg-rendered PDF that was actually OCR’d. The signed URLs use HMAC and expire; embed them directly in browsers (no auth required to follow the URL).

7.1 canonical (the OCR layer)

{
  "document_id": "doc_01HX...",
  "provider": "dual-vlm",
  "pages": [ /* see §7.1.1 */ ],
  "markdown": "# Invoice\n\n...",
  "document_context": { /* see §7.1.4, null when characterisation disabled */ },
  "detected_language": { /* see §7.1.5, omitted when undetected */ }
}

markdown is the human-readable, single-string serialisation of the entire document, suitable for display, downstream LLM consumption, or embedding into a search index. Produced by deterministic per-page rendering of pages[].blocks + pages[].tables in reading order.

7.1.1 pages[]

Each page:

{
  "page_index": 0,
  "width": 1240,
  "height": 1754,
  "rotation": 0,
  "blocks":   [ /* §7.1.2 */ ],
  "tables":   [ /* §7.1.3 */ ],
  "formulas": [ { "bbox": [...], "latex": "E = mc^2" } ],
  "charts":   [ { "bbox": [...], "caption": "...", "data": {...}, "raw": "..." } ],
  "seals":    [ { "bbox": [...], "text": "OFFICIAL SEAL" } ],
  "metadata": {
    "orientation_degrees": 0,
    "skew_degrees": 0.0,
    "has_header": false,
    "has_footer": false,
    "source_class": "scanned_pdf | raster_photo | born_digital_pdf_text_only | born_digital_pdf_hybrid | office_render | vector_render | unknown",
    "applied_policies": { "layout": {...}, "preprocessing": {...} }
  }
}

7.1.2 blocks[] (OCRBlock)

{
  "id": "b_1",
  "type": "text | title | table | formula | chart | seal",
  "text": "Invoice",
  "confidence": 0.98,
  "bbox": [0.05, 0.05, 0.3, 0.1],
  "polygon": [[0.05, 0.05], [0.3, 0.05], [0.3, 0.1], [0.05, 0.1]],
  "reading_order": 0,
  "language": "en",
  "tokens": [
    { "text": "Invoice", "confidence": 0.98, "bbox": [0.05, 0.05, 0.3, 0.1] }
  ],
  "native_label": null,
  "sources": {
    "paddleocr-vl": {
      "text": "Invoice",
      "confidence": 0.96,
      "latency_ms": 45.2,
      "error": null
    },
    "glmocr-vl": {
      "text": "Invoice",
      "confidence": 1.0,
      "latency_ms": 67.8,
      "error": null
    }
  },
  "chosen_source": "glmocr-vl",
  "agreement": 0.99,
  "risk_flags": [],
  "non_authoritative": false
}

Coordinates (bbox, polygon) are normalised to [0, 1] against page width/height. reading_order indexes the linear reading sequence within the page.

sources retains the raw per-VLM attempts for audit. chosen_source records which VLM’s text won the merge. agreement is the cross-VLM similarity (0–1).

risk_flags is an advisory list set by the merger when GLM’s response looks suspect:

Flag	Meaning
unsupported_script	GLM returned text in a script it doesn’t reliably support (e.g. Arabic).
length_growth_suspect	GLM output is dramatically longer than Paddle’s — possible hallucination.
hallucination_suspect	Heuristic match for fabricated content patterns.
repetition_suspect	High n-gram repetition rate; the VLM degenerated.

When risk_flags is non-empty, non_authoritative is true and the extraction prompt builder label-prefixes the block as advisory so the downstream LLM treats it as low-trust evidence.

7.1.3 tables[]

{
  "bbox": [0.05, 0.2, 0.95, 0.6],
  "rows": [
    ["Item", "Qty", "Price"],
    ["Widget A", "2", "$100"]
  ],
  "html": "<table>...</table>",
  "markdown": "| Item | Qty | Price |\n|------|-----|-------|\n| Widget A | 2 | $100 |",
  "spans_pages": [],
  "merge_confidence": null
}

For cross-page tables, spans_pages lists the 1-indexed page numbers and merge_confidence is the LLM arbiter’s confidence in the merge (0–1). Single-page tables have spans_pages: [] and merge_confidence: null.

7.1.4 document_context (optional Stage 5.5)

When CHARACTERISATION_ENABLED=true and the document has at least CHARACTERISATION_MIN_PAGES (default 2), one extra LLM call produces a document-level summary:

{
  "document_type_hint": "invoice",
  "summary": "Invoice for Acme Corp, dated 2026-04-21, total $500",
  "section_map": [
    { "title": "Header",  "pages": [1] },
    { "title": "Items",   "pages": [2, 3] },
    { "title": "Footer",  "pages": [4] }
  ],
  "page_roles": {
    "1": "cover",
    "2": "body",
    "3": "body",
    "4": "signature"
  },
  "cross_page_tables": [
    { "table_id": "t_5", "pages": [2, 3], "confidence": 0.95 }
  ],
  "entities": [
    {
      "canonical": "Acme Corporation",
      "aliases": ["Acme", "Acme Corp"],
      "role": "vendor",
      "pages": [1, 2, 3]
    }
  ],
  "characterisation_latency_ms": 2345.0,
  "raw_output": "...",
  "error": null
}

page_roles values: cover, toc, body, section_divider, appendix, signature, notes, blank.

7.2 extraction (the LLM extraction layer)

Populated only when a template was applied:

{
  "template_id": "tpl_acme_invoice_v3",
  "template_version": 3,
  "parsed": {
    "invoice_number": "INV-2026-001",
    "vendor_name": "Acme Corporation",
    "invoice_date": "2026-04-21",
    "subtotal": 500.00,
    "tax": 75.00,
    "total": 575.00,
    "items": [
      { "description": "Widget A", "quantity": 2, "unit_price": 250.00, "line_total": 500.00 }
    ]
  },
  "raw_output": "The extracted data is...",
  "prompt_length": 4567,
  "latency_ms": 1234.5,
  "error": null,
  "error_kind": null,
  "schema_errors": [],
  "field_confidence": { "invoice_number": 1.0, "total": 0.67 },
  "repair": null,
  "chunked": null
}

schema_errors[] carries per-field Pydantic validation entries when the LLM’s output partially conformed to the template schema. Each entry escalates the job to needs_review (one error-severity validation flag per entry) but does not trigger rejected. Only a catastrophic JSON parse failure (no parseable output at all) yields rejected.

error_kind is the machine-readable discriminator for error: budget_exceeded (the assembled prompt exceeded the extraction budget — routed to needs_review, never rejected), transport (the LLM call failed after a bounded retry), parse (output was not a JSON object after the repair pass), or model_build (broken template schema). null when there is no error.

field_confidence maps each top-level template field to a deterministic confidence in [0, 1] — the mean of available signals: schema validity, document grounding (was the value’s text found in the canonical markdown the LLM saw), absence of an error-severity judge flag (a judge warn/info advisory is non-escalating and does not lower confidence), and repair stability. Fields whose confidence falls below the template’s review_thresholds[field] emit an error-severity confidence_threshold flag and force needs_review.

repair records the engine’s bounded corrective re-ask, when one ran: {"attempted", "trigger" ("parse" | "schema_errors"), "accepted", "schema_errors_before", "schema_errors_after", "changed_fields"}. The repaired payload is adopted only when it strictly improved on the original.

chunked records page-windowed extraction for documents that exceeded the prompt budget: {"windows", "window_pages", "failed_window_pages", "conflict_fields", "dropped_pages"}. Values merge deterministically (first non-null scalar in document order; arrays concatenate with exact-duplicate dedupe); window conflicts surface as warn-severity chunk_conflict flags, and failed/dropped pages force needs_review — information loss is never silent.

If the field has been overridden via the review queue:

{
  ...,
  "overrides": [
    {
      "actor_id": "cred_01HX...",
      "patch": { "/total": 575.00 },
      "note": "corrected tax calculation",
      "at": "2026-04-21T10:30:00Z"
    }
  ],
  "original_parsed": { "total": 575.01 }
}

The original LLM output is preserved under original_parsed for audit; parsed reflects the post-override state.

7.3 validation_flags[]

Each entry:

{
  "rule_id": "rule_0",
  "kind": "required | regex | arithmetic | date_plausible | enum | checksum | cross_agreement",
  "severity": "info | warn | error",
  "message": "$subtotal + $tax != $total (500.00 + 75.00 != 576.00)",
  "target": "$total",
  "paths": ["total"]
}

paths lists the JSON paths into extraction.parsed that produced the flag, so a UI can highlight the offending fields.

Decision logic (drives review_decision):

• Any severity: error flag → review_decision: review (job goes to needs_review).
• warn flags accumulate; threshold is tenant_policies.review_warn_count_threshold (default 3) — at or above, review_decision: review.
• info flags never escalate; they are advisory.
• reject is only set on catastrophic JSON parse failure (the LLM emitted no usable output).

---

8. Templates

Templates declare what to extract from a document. The engine is fully generic — zero templates ship in source. Three sample templates ship under sample_templates/ (universal-invoice-v1, tendam-invoice-v1, universal-receipt-v16) as starter material; operators import them via docstack-admin templates import.

A template is the union of:

1. A JSON Schema (Draft 2020-12) describing the desired extraction.
2. A prompt overlay — optional free-form text injected after the platform’s scaffold prompt.
3. A validation rules array — declarative rules in the DSL described in §9.
4. Review thresholds — per-field confidence/agreement floors.

Templates are versioned: PUT /v1/templates/{template_id} creates a new version (immutable history). Resolution order:

1. Explicit template_id on the document submission.
2. Explicit template_name (resolves to latest active version).
3. Auto-classification (opt-in per tenant).
4. Canonical OCR only.

8.1 List templates

GET /v1/templates?active_only=true

Auth: User+ (any role; the upload form needs a template picker).

Response:

{
  "templates": [
    {
      "id": "tpl_01HX...",
      "tenant_id": "acme",
      "name": "acme_invoice",
      "description": "Acme standard invoice",
      "version": 3,
      "prompt_overlay": "...",
      "json_schema": { /* Draft 2020-12 schema */ },
      "validation_rules": [ /* see §9 */ ],
      "review_thresholds": { "$total": 0.90 },
      "is_active": true
    }
  ]
}

8.2 Create a template

POST /v1/templates

Auth: Admin+.

{
  "name": "acme_invoice",
  "description": "Acme standard invoice",
  "prompt_overlay": "",
  "json_schema": { "type": "object", "properties": { ... }, "required": [ ... ] },
  "validation_rules": [
    { "kind": "required",   "target": "$subtotal" },
    { "kind": "regex",      "target": "$po_number", "pattern": "^[0-9]{6}$" },
    { "kind": "arithmetic", "expression": "$subtotal + $tax == $total", "tolerance": 0.01 },
    { "kind": "enum",       "target": "$status", "values": ["draft", "final", "paid"] },
    { "kind": "checksum",   "target": "$iban", "algorithm": "iban" }
  ],
  "review_thresholds": { "$total": 0.90, "$subtotal": 0.85 }
}

Constraints

• name: lowercase alphanumeric + underscore/dash, ≤ 128 chars (^[a-z0-9][a-z0-9_\-]*$).
• json_schema: must be valid Draft 2020-12 with top-level type: object.
• validation_rules[].pattern (for regex): must compile with the Python regex package.

Response — 201 Created — the template DTO with version: 1.

Created templates start with is_active: true. A templates:invalidate message is published to redis so workers refresh their in-memory cache (~50 ms propagation).

8.3 Fetch a template

GET /v1/templates/{template_id}

8.4 Bump version

PUT /v1/templates/{template_id}

Creates a new immutable version. Body shape is the same as POST except name is fixed (derived from the existing template). Response: 201 Created with incremented version. Audit event: TEMPLATE_VERSION_BUMPED.

8.5 Activate / deactivate

POST /v1/templates/{template_id}/activate
POST /v1/templates/{template_id}/deactivate

Both Admin+, both empty bodies. Deactivation stops new jobs from resolving the template; historical jobs keep referencing it.

8.6 Soft delete

DELETE /v1/templates/{template_id}

Admin+. Marks the template inactive and retains it for audit. Returns { "deleted": true, "id": "tpl_...", ... }.

8.7 Dry-run a template

POST /v1/templates/{template_id}/test
Content-Type: application/json

{ "ocr_markdown": "..." }

Admin+. Runs the template against supplied OCR markdown without persisting. Useful for validating template changes before bumping the version.

Response:

{
  "template_id": "tpl_...",
  "template_version": 3,
  "extraction": { "parsed": {...}, "raw_output": "...", "latency_ms": 1234.5, "error": null, "schema_errors": [] },
  "validation": { "flags": [...], "decision": "accept | review | reject" },
  "persisted": false
}

8.8 Unified test (for editors)

POST /v1/templates/test

Admin+. Same response as above, but the request body picks exactly one mode:

{
  "ocr_markdown": "...",
  "template_id": "tpl_...",
  "draft_id": "draft_...",
  "ad_hoc_template": {
    "prompt_overlay": "",
    "json_schema": { ... },
    "validation_rules": [ ... ],
    "review_thresholds": { ... }
  }
}

The ad-hoc mode is what the frontend’s live-preview editor uses to evaluate in-progress changes without committing them.

8.9 Classify (preview)

POST /v1/templates/classify
Content-Type: application/json

{ "ocr_markdown": "..." }

Admin+. Runs the classifier on the supplied markdown and returns the picked template plus all candidates. Use this to preview classifier behaviour before enabling tenant_policies.auto_classify_enabled.

{
  "template_id": "tpl_acme_invoice_v3 | null",
  "raw_output": "...",
  "latency_ms": 1234.5,
  "candidates": [
    { "id": "tpl_...", "name": "acme_invoice", "description": "..." }
  ]
}

8.10 Versions

GET /v1/templates/{template_id}/versions

Admin+. Lists every version of the template’s name (no pagination — > 200 versions is pathological).

8.11 Audit

GET /v1/templates/{template_id}/audit?limit=50&offset=0

Admin+. Paginated audit events. Event kinds: created, version_bumped, activated, deactivated, deleted.

8.12 Rule-kind metadata

GET /v1/templates/meta/rule-kinds

Auth: User+. Static metadata describing every supported validation rule kind, used by the template editor to render input forms. The full payload is shown in §9.7.

8.13 Template drafts

Drafts are pending template versions awaiting admin approval — the intelligent-extraction “draft on miss” path uses these.

POST   /v1/templates/drafts                    # Tenant_user+: create a draft
GET    /v1/templates/drafts                    # Tenant_user+: list drafts for tenant
GET    /v1/templates/drafts/{draft_id}         # Tenant_user+: fetch one draft
PATCH  /v1/templates/drafts/{draft_id}         # Tenant_user+: update (mutable until accepted)
POST   /v1/templates/drafts/{draft_id}/accept  # Admin+: convert to active template
DELETE /v1/templates/drafts/{draft_id}         # Tenant_user+ or admin: discard (idempotent)

POST /accept returns { "template_id": "tpl_...", "draft_id": "draft_..." }.

8.14 Template forge

The forge automates the iterative loop of refining a template against a hand-curated corpus + ground truth. The tenant uploads N source files and a GT JSON keyed by filename; the forge runs Claude Code as a subprocess per iteration against the existing OCR + extraction stack as the scoring oracle, and produces a template_drafts row pointing at the winning iteration’s templates row. Per-job dollar / iteration / wall-time caps gate spend.

Prerequisites. A tenant must have tenant_policies.forge_enabled=true (a platform_admin flips this; default false) and at least one forge_endpoints row configured with the Claude Code backend credentials (see §14.2 for the CRUD).

Operational footprint. The forge worker runs in its own container (docker-compose.forge.yaml override, opt-in) that requires FORGE_API_INTERNAL_TOKEN to be set. Without the override, all forge endpoints respond but POST /jobs will succeed only to leave jobs queued — set up the worker before enabling forge for a tenant.

POST /v1/templates/forge/jobs                                  # Admin+: submit
GET  /v1/templates/forge/jobs                                  # Admin+: list
GET  /v1/templates/forge/jobs/{job_id}                         # Admin+: detail
POST /v1/templates/forge/jobs/{job_id}/abort                   # Admin+: cooperative abort
GET  /v1/templates/forge/jobs/{job_id}/iterations              # Admin+: leaderboard
GET  /v1/templates/forge/jobs/{job_id}/iterations/{iter_id}    # Admin+: iteration detail
POST /v1/templates/forge/jobs/{job_id}/promote                 # Admin+: override winner

7.1.5 detected_language (optional, L-package)

Document-level language verdict. Omitted from the payload when the OCR text was too short / unscriptable to classify.

{
  "language": "es",            // ISO 639-1, or null when undetermined
  "language_name": "Spanish",  // English display name
  "script": "Latin",           // dominant Unicode script (or "Unknown")
  "direction": "ltr",          // "rtl" | "ltr"
  "confidence": 0.97,          // 0–1 detector confidence
  "source": "reconciled"       // "deterministic" | "llm" | "reconciled"
}

Deterministic Unicode script + py3langid language, reconciled with the characteriser on multi-page documents. The orchestration prompts are English but are made language-aware from this verdict (OCR-trust gate, extraction/judge/classifier preambles, grounding). NER is English-primary regardless. Configure via the language_policy (detection_enabled, language_hints_enabled, language_allowlist).

8.14.1 Submit a forge job

POST /v1/templates/forge/jobs is a multipart request:

Form field	Type	Required	Notes
body	JSON string	yes	Forge config (see below).
files	repeated upload	yes	Corpus files (≥ 1). PDF / image MIME types accepted.
ground_truth	upload	yes	JSON object keyed by corpus filename → operator’s expected extraction. Every corpus filename MUST have a GT entry; missing keys 422.

body shape:

{
  "fork_name": "forge-receipts-20260430",     // optional, auto-suggested
  "starter": {
    "template_id": "tpl_...",                  // mutually XOR with .uploaded
    "uploaded": null
  },
  "schema_evolution_policy": "additive_from_gt",  // locked | additive_from_gt | additive_type_corrective
  "train_holdout_split_ratio": 0.8,           // (0, 1); deterministic per filename
  "forge_endpoint_id": "fep_...",             // optional; defaults to tenant default → platform default
  "caps": {                                    // all optional; clamped by tenant + platform layers
    "max_iterations": 25,
    "max_wall_time_seconds": 14400,
    "max_total_input_tokens": 5000000,
    "max_total_output_tokens": 1000000,
    "max_dollars": 20.0
  },
  "plateau_window": 5,
  "plateau_epsilon": 0.005
}

Response 201:

{
  "job": {
    "id": "frg_...",
    "status": "pending",
    "fork_name": "forge-receipts-20260430",
    "corpus_size": 47,
    "endpoint_summary": { "id": "fep_...", "backend": "anthropic", "model": "claude-opus-4-7", "has_credentials": true },
    "caps": { "max_iterations": 25, "max_dollars": 20.0, ... },
    "created_at": "...",
    "...": "..."
  }
}

Errors:

HTTP	Cause
403	forge_enabled=false for the tenant (status_detail surfaces the kill-switch).
422	Validation failure: bad fork_name, missing GT entries, unknown schema policy, no endpoint configured.
409	Fork name already in use (concurrent submit collision).

8.14.2 Status, leaderboard, iteration detail

curl -H "Authorization: Bearer $TOKEN" \
     http://localhost:8080/v1/templates/forge/jobs/frg_xxx

Returns the job DTO with status ∈ {pending, discovering_corpus, seeding_template, iterating, plateau, completed, aborted, failed, quota_exceeded}, winning_iteration_id (set on terminal), result_template_draft_id (set on a successful terminal), and aggregate token + dollar tallies.

curl -H "Authorization: Bearer $TOKEN" \
     'http://localhost:8080/v1/templates/forge/jobs/frg_xxx/iterations?only_scored=true'

Returns one entry per iteration with composite_score_train, composite_score_holdout, field_breakdown_*, outcome ∈ {scored, rejected_schema_gate, rejected_leak, failed_batch, failed, aborted}, and gate_violation / leak_hits for rejected ones.

8.14.3 Abort

curl -X POST -H "Authorization: Bearer $TOKEN" \
     http://localhost:8080/v1/templates/forge/jobs/frg_xxx/abort

Sets status_detail="abort_requested". The runtime checks this at every iteration boundary; mid-iteration abort waits until the boundary. Already-terminal jobs return 409.

8.14.4 Promote (override the auto-pick winner)

By default the forge auto-promotes the iteration with the highest composite_score_holdout (ties broken by train). Override with:

curl -X POST -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "iteration_index": 7 }' \
     http://localhost:8080/v1/templates/forge/jobs/frg_xxx/promote

Only valid post-terminal (plateau / completed); rewrites winning_iteration_id and points the existing template_drafts row at iteration 7’s body. The operator then accepts the draft via the existing POST /v1/templates/drafts/{id}/accept path.

8.14.5 Schema-evolution policies

Per design §8.8:

• locked — schema is immutable; the agent edits only prompt_overlay / validation_rules / review_thresholds. Description / title / examples edits are tolerated.
• additive_from_gt — fields may be added when every GT entry populates them with type-compatible values; no removals, no renames, no type changes.
• additive_type_corrective — additions as above, plus type changes when every GT value parses cleanly into the new type AND at least one fails under the prior type (the change is justified by GT evidence). Pure widenings without a prior-side failure are rejected as unjustified.

Schema-gate rejections cost an iteration without scoring. Three consecutive rejections fail the job.

8.14.6 Leak audit

The runtime builds a forbidden-token set per job from GT scalar values + filename stems (the distinctive-n-gram pass over canonical OCR text is documented in design §4.4 step 2 and ships in a follow-up). After every agent edit, the runtime greps the proposed template’s overlay + schema descriptions / examples / enum values / field names against the set; any hit rejects the iteration with outcome="rejected_leak" and the offending tokens are surfaced to the agent on the next turn. Three consecutive leak rejections fail the job.

The audit is mechanical and final — operators cannot disable it. The set is built from train-split items only so a token unique to holdout doesn’t produce false-positive rejections.

---

9. Validation rules DSL

Validation rules are declarative JSON objects on template.validation_rules[]. Every rule shares two common fields:

Common param	Type	Required	Default
id	string	no	rule_<index>
severity	info | warn | error	no	warn

Field paths use a leading $ and dot notation: $total, $items.0.line_total, $invoice.dates.due. The arithmetic rule additionally supports base.*.attr array iteration in expressions (translated to a _iter_attr marker before AST parsing).

9.1 required

Flag when the target value is null or empty.

{ "kind": "required", "target": "$subtotal", "severity": "warn" }

9.2 regex

Flag when the target value doesn’t match the pattern (Python regex package syntax).

{
  "kind": "regex",
  "target": "$po_number",
  "pattern": "^[A-Z]{2}\\d{4}$",
  "severity": "error"
}

9.3 arithmetic

Evaluate a boolean expression over numeric fields using a safe AST. Supported operators: + - * /, comparisons == != <= >= < >, and a single function: sum($items.*.line_total).

{
  "kind": "arithmetic",
  "expression": "$subtotal + $tax == $total",
  "tolerance": 0.01,
  "severity": "error"
}

tolerance (default 0) is the absolute tolerance applied to equality comparisons. Skip-on-indeterminate: if any referenced field resolves to None, no flag is emitted (optional fields stay the responsibility of required).

9.4 date_plausible

Flag when a parsed date falls outside [min, max].

{
  "kind": "date_plausible",
  "target": "$invoice_date",
  "min": "-10y",
  "max": "+1y",
  "severity": "warn"
}

min / max are either ISO dates (2020-01-01) or relative offsets (-90d, +1y, -10y).

9.5 enum

Flag when the target value isn’t in the allowed set.

{
  "kind": "enum",
  "target": "$status",
  "values": ["draft", "final", "paid"],
  "severity": "error"
}

9.6 checksum

Flag when a structured identifier fails its checksum.

{
  "kind": "checksum",
  "target": "$iban",
  "algorithm": "iban | mrz | luhn",
  "severity": "error"
}

9.7 cross_agreement

Flag when the dual-VLM cross-agreement score for the supporting block falls below a threshold.

{
  "kind": "cross_agreement",
  "target": "$total",
  "threshold": 0.85,
  "severity": "warn"
}

If threshold is omitted, it falls back to template.review_thresholds[target].

9.8 Decision flow

• Any error flag → review_decision: review (job → needs_review).
• warn flags accumulate; if the count reaches tenant_policies.review_warn_count_threshold (default 3), review_decision: review.
• info flags never escalate; pure telemetry.
• reject is reserved for catastrophic JSON parse failure (no usable LLM output).

9.9 Rule-kind metadata endpoint

For UI-driven template editors:

GET /v1/templates/meta/rule-kinds

{
  "kinds": [
    { "kind": "required",        "summary": "...", "params": [...] },
    { "kind": "regex",           "summary": "...", "params": [...] },
    { "kind": "arithmetic",      "summary": "...", "params": [...] },
    { "kind": "date_plausible",  "summary": "...", "params": [...] },
    { "kind": "enum",            "summary": "...", "params": [...] },
    { "kind": "checksum",        "summary": "...", "params": [...] },
    { "kind": "cross_agreement", "summary": "...", "params": [...] }
  ],
  "common_params": [
    { "name": "id",       "type": "string", "required": false, "description": "..." },
    { "name": "severity", "type": "enum",   "required": false, "values": ["info", "warn", "error"], "default": "warn" }
  ]
}

The metadata is generated from validation/rules/RULE_KINDS_META + RULE_COMMON_PARAMS; no DB query, no caching needed.

---

10. Review queue

When a job lands in needs_review, the worker creates a review_queue row. Reviewers list, claim, and resolve from these endpoints.

10.1 List reviews

GET /v1/reviews?status=pending&limit=50&offset=0

Query param	Type	Default	Notes
status	enum	(all)	pending, in_review, approved, overridden, reprocessed, closed_by_delete.
batch_id	string	(none)	Filter to reviews for a specific batch.
limit	int	50	1–200.
offset	int	0	≥ 0.

{
  "items": [
    {
      "review_id": "rq_doc_01HX...",
      "job_id": "doc_01HX...",
      "job_status": "needs_review",
      "batch_id": "batch_01HX... | null",
      "original_filename": "2026-02-invoice-12345.pdf | null",
      "status": "pending",
      "reasons": ["arithmetic constraint failed: $subtotal + $tax == $total"],
      "created_at": "...",
      "updated_at": "...",
      "resolved_at": null,
      "resolved_by": null,
      "resolution": null,
      "resolution_note": null
    }
  ],
  "total": 12,
  "limit": 50,
  "offset": 0
}

10.2 Fetch one review

GET /v1/reviews/{review_id}

Returns { "review": {...}, "job": {...} } — the full underlying job document is included so a reviewer UI can render with one round trip.

10.3 Approve

POST /v1/reviews/{review_id}/approve
Content-Type: application/json

{ "note": "looks fine" }   // optional

Side effects: review row → approved; underlying job → completed with review_decision: accept. Publishes to the destination connection (if the batch has one). Returns 409 Conflict if the review is already terminal.

10.4 Reprocess (re-enqueue with optional template swap)

POST /v1/reviews/{review_id}/reprocess
Content-Type: application/json

{ "template_id": "tpl_new" }   // optional

Side effects: job → queued (timestamps cleared, retry_count incremented); review row → reprocessed. If the next run again lands in needs_review, a new review row is created (upsert-by-job_id resets resolved_at and clears resolution fields). 202 Accepted.

10.5 Override (apply field corrections)

POST /v1/reviews/{review_id}/override
Content-Type: application/json

{
  "patch": { "/total": 132.45, "/items/0/quantity": 2 },
  "note": "fixed total typo"
}

patch keys are RFC 6901 JSON Pointers into extraction.parsed. Value-replacement only — no structural changes (insert / remove / move).

Side effects, atomically:

1. Apply the patch to extraction.parsed.
2. Append an overrides[] entry: { actor_id, patch, note, at }.
3. Re-run validation; validation_flags reflect the post-override state.
4. Job → completed with review_decision: accept.
5. Review row → overridden.

Original LLM output is preserved under extraction.original_parsed for audit. Publishes to the destination connection only if post-override validation passes.

Failures:

• 409 Conflict if the review is terminal or extraction is null.
• 400 Bad Request if any patch path doesn’t resolve in the parsed extraction.

---

11. Batches and connections

Use batches to submit many documents at once or to source-pull from external storage; use connections to wire batches to external sources (SFTP, S3) and destinations (S3, email, webhooks).

11.1 Submit a batch

POST /v1/batches

Two content types are accepted, dispatched by header:

File upload (multipart/form-data)

name: 2026-04-invoices
template_id: tpl_acme_invoice_v3            (optional)
template_name: acme_invoice                 (optional)
processing_mode: extraction                 (optional)
page_range: 1-3,5,last                       (optional; applies to every file)
page_cap_policy: {"over_pages":50,"first_pages":10,"include_last":true}  (optional)
destination_connection_id: conn_s3_...      (optional)
destination_config: { ... }                 (optional)
file: <binary>                              (1 or more)

page_range and page_cap_policy are mutually exclusive (422 if both). page_cap_policy is the per-file alternative to a uniform range: each PDF whose page count exceeds over_pages is capped to its first first_pages pages plus the last page (include_last, default true) — expanded server-side into that file’s page_range. Smaller files run in full. Use it to cap long documents across a heterogeneous batch in one control. It is echoed on the batch object as page_cap_policy (or null) and also applies to source-pull batches (expanded as each remote file is discovered).

Source-pull (application/json)

{
  "name": "2026-04-invoices",
  "source_connection_id": "conn_sftp_...",
  "source_pull_params": {
    "prefix": "/invoices/2026-04/",
    "extension_filter": "pdf"
  },
  "destination_connection_id": "conn_s3_...",
  "destination_config": { ... },
  "template_id": "tpl_acme_invoice_v3",
  "processing_mode": "extraction",
  "page_cap_policy": { "over_pages": 50, "first_pages": 10, "include_last": true },
  "scan_config": {
    "max_files": 100,
    "include_pattern": "^[A-Z]+.*\\.pdf$"
  }
}

processing_mode (the multipart form field on file upload, the processing_mode key in the source-pull JSON body) is optional and defaults to extraction. It is applied uniformly to every job in the batch — see §5.10 Processing modes. A non-extraction mode combined with template_id / template_name is 400 Bad Request, since entities and canonical never run template extraction.

Response — 202 Accepted

{
  "id": "batch_01HX...",
  "tenant_id": "acme",
  "name": "2026-04-invoices",
  "template_id": "tpl_acme_invoice_v3",
  "processing_mode": "extraction",
  "page_cap_policy": null,
  "source_connection_id": null,
  "destination_connection_id": null,
  "destination_config": {},
  "scan_config": {},
  "status": "queued",
  "counts": { "total": 5, "completed": 0, "failed": 0, "needs_review": 0 },
  "created_by": "cred_01HX...",
  "created_at": "...",
  "started_at": null,
  "completed_at": null,
  "error_message": null
}

processing_mode on the batch object is the mode seeded onto every job the batch spawns. Each spawned job carries its own processing_mode (identical to the batch’s) and surfaces it via GET /v1/documents/{job_id}.

Limits: BATCH_MAX_FILES_PER_REQUEST (default 100), BATCH_MAX_AGGREGATE_UPLOAD_BYTES (default 500 MiB).

11.2 List, fetch, jobs, cancel, retry-failed, reprocess, reconcile

GET    /v1/batches?status=running&limit=50
GET    /v1/batches/{batch_id}
GET    /v1/batches/{batch_id}/jobs?limit=50&offset=0
POST   /v1/batches/{batch_id}/cancel              # idempotent
POST   /v1/batches/{batch_id}/retry-failed
POST   /v1/batches/{batch_id}/reprocess           # re-run every settled job
POST   /v1/batches/{batch_id}/reconcile           # tenant_admin+: recount counters

GET /v1/batches/{batch_id}/jobs returns the same thin job items as the documents list, plus progress_pct, progress_stage, destination_published_at, and destination_error.

POST /v1/batches/{batch_id}/reprocess re-enqueues every settled job (completed / needs_review / failed / rejected) through the full pipeline — the sanctioned way to regenerate a whole corpus after a pipeline or template upgrade. Existing job rows are re-queued in place, so submission dedup never interferes; queued / processing jobs are skipped (the call is idempotent and safe while a drain is in flight); open review entries close as reprocessed. Returns {"batch": {...}, "reprocessed_jobs": N} with 202.

POST /v1/batches/{batch_id}/reconcile (tenant_admin+) recomputes the batch’s settled counters from the true job statuses — a self-heal for historically skewed counts. Returns {"batch": {...}, "counters": {...}}.

POST /v1/admin/jobs/recover-stale (tenant_admin+) sweeps jobs stranded in processing by a dead worker and re-queues them (or fails them as processing_interrupted after repeated interruptions). The worker runs the identical sweep automatically at startup; this endpoint is the break-glass variant for operators who don’t want to bounce the worker. Returns {"requeued": n, "failed": m}.

GET /v1/admin/tenant-policy/registry-spec returns the machine-readable policy-registry schema (field types, bounds, UI hints) that drives the admin policy editor.

11.3 Connections

Connections store per-tenant credentials for external systems. Type-discriminated by kind.

GET    /v1/admin/connections?kind=destination
POST   /v1/admin/connections
GET    /v1/admin/connections/{connection_id}
PATCH  /v1/admin/connections/{connection_id}
DELETE /v1/admin/connections/{connection_id}
POST   /v1/admin/connections/{connection_id}/test

Auth: Tenant_admin+.

Credentials are encrypted at rest using CONNECTION_ENCRYPTION_KEYS (versioned keyset). Test response: { "ok": true|false, "message": "...", "latency_ms": 123.45 }.

---

12. Artifacts

Signed downloads for original files, rendered PDFs, and other persisted artifacts.

GET /v1/artifacts/{artifact_id}?expires=...&tenant=...&token=...

Auth: none (all three query params verify an HMAC signature minted by the API).

URLs are minted by the API via api/services/storage.py::generate_download_url and surfaced inside the document response (artifacts.original, artifacts.rendered_pdf). Embed them directly in browsers — they’re self-authenticating until expires.

PUBLIC_BASE_URL must match the hostname browsers will use, otherwise the signed URL points at the wrong origin.

---

13. Identity, access, and tenancy (IAM)

The IAM model is detailed in docs/admin-iam-design.md. The HTTP surface below is the integrator-facing slice.

Body shapes. The inline annotations in this section (e.g. # { id, name, ... }) summarise the request fields a Pydantic model requires at the time of writing. The authoritative shape is always the live OpenAPI schema served at GET /openapi.json and rendered interactively at /api-reference/. If an inline summary disagrees with the schema, trust the schema — and please file an issue.

13.1 Identity (whoami)

GET /v1/me

The authoritative call to discover the caller’s tenant, roles, auth method, and rate-limit info:

{
  "user_id": "...",
  "tenant_id": "acme",
  "tenant_name": "Acme Corp",
  "tenant_status": "active | suspended | deleted",
  "actor_id": "cred_... | null",
  "email": "user@example.com | null",
  "display_name": "John Doe | null",
  "role": "tenant_admin | null",
  "actor_roles": ["tenant_admin"],
  "auth_mode": "bearer | trusted_headers",
  "auth_method": "session | api_key | trusted_headers | null",
  "must_change_password": false,
  "rate_limit": { "limit": 100, "remaining": null }
}

rate_limit.remaining is always null here — the live value lives in the X-RateLimit-Remaining response header.

13.2 Auth endpoints (bearer mode only)

These return 501 Not Implemented in trusted_headers mode.

Method	Path	Auth	Purpose
POST	/v1/auth/login	none	Email + password → session cookie + CSRF cookie.
POST	/v1/auth/logout	session	Revoke current session; clear cookies.
POST	/v1/auth/change-password	session	Change own password (current_password + new_password).
GET	/v1/auth/invites/{raw_token}	none	Peek at an invite token without consuming.
POST	/v1/auth/accept-invite	none	Consume invite, set password, auto-login.
GET	/v1/auth/reset-tokens/{raw_token}	none	Peek at a password-reset token.
POST	/v1/auth/reset-password	none	Consume reset token, set new password.
GET	/v1/auth/sessions	session	List active sessions for the caller.
POST	/v1/auth/sessions/{session_id}/revoke	session	Revoke a specific session.

The four mutating unauthenticated endpoints (login, accept-invite, reset-password, change-password) share the per-IP rate limit (login_rate_limit_per_15min, default 5/15 min).

POST /v1/auth/login body:

{
  "email": "user@example.com",
  "password": "secretpass"
}

(tenant_id is accepted but ignored — kept for client back-compat.)

Invite + reset tokens are single-use by construction: the consumer query is UPDATE auth_tokens SET consumed_at = NOW() WHERE id = ? AND consumed_at IS NULL RETURNING * so two concurrent accept-invite calls serialise on the row lock and the loser sees None (and surfaces 410 Gone).

13.3 User management

GET    /v1/admin/users?limit=50&offset=0
POST   /v1/admin/users                              # invite
GET    /v1/admin/users/{user_id}
DELETE /v1/admin/users/{user_id}                    # hard delete
POST   /v1/admin/users/{user_id}/promote            # → tenant_admin
POST   /v1/admin/users/{user_id}/demote             # → tenant_user
POST   /v1/admin/users/{user_id}/suspend
POST   /v1/admin/users/{user_id}/activate           # resume after suspend
POST   /v1/admin/users/{user_id}/reset-password     # admin-initiated reset
POST   /v1/admin/users/{user_id}/resend-invite
POST   /v1/admin/users/{user_id}/revoke-invite

Auth: Tenant_admin+. All idempotent except DELETE (returns 404 on a missing user).

Invite body: { "email": "newuser@example.com", "role": "tenant_user", "display_name": "John Doe" }.

reset-password mints a single-use reset token for the target user (the same flow as POST /v1/auth/reset-password); resend-invite re-issues the invite token; revoke-invite cancels a pending invite without deleting the user record.

There is no HTTP path to promote to platform_admin — by D19 (see docs/admin-iam-design.md), promotion and demotion of platform_admin go through the CLI. Calling demote / suspend on a platform_admin returns 409 Conflict pointing the caller at the CLI.

13.4 API credentials

Two router prefixes:

• /v1/admin/api-keys — admin manages credentials for any user in the tenant.
• /v1/account/api-keys — user manages own credentials.

Both expose GET (list), POST (create), POST /{public_id}/rotate, POST /{public_id}/revoke. The api_key secret is returned exactly once on create or rotate; it is never echoed in list responses.

Create response — 201 Created

{
  "credential": {
    "id": "cred_01HX...",
    "public_id": "abcd1234",
    "user_id": "...",
    "is_active": true,
    "created_at": "..."
  },
  "api_key": "rk_abcd1234_sEcR3Tsecr3tsecr3t"
}

Rotate invalidates the old secret immediately. Revoke is idempotent.

User profile self-service (session-only):

PATCH /v1/account/profile           # body: { display_name }

The PATCH endpoint accepts only display_name (1–128 chars). To change your password, use POST /v1/auth/change-password (§13.2) — passwords are not mutated through the profile endpoint.

13.5 Tenant management (platform admin)

GET    /v1/platform/tenants?status=active&limit=50
POST   /v1/platform/tenants                      # { id, name, initial_admin_email }
GET    /v1/platform/tenants/{tenant_id}
PATCH  /v1/platform/tenants/{tenant_id}          # { name }
POST   /v1/platform/tenants/{tenant_id}/suspend
POST   /v1/platform/tenants/{tenant_id}/activate
DELETE /v1/platform/tenants/{tenant_id}          # soft delete

Auth: Platform_admin. Tenant id is opaque, chosen by the creator (no auto-generation).

13.6 Cross-tenant user search (platform admin)

GET /v1/platform/users?search=alice&tenant_id=acme&role=tenant_admin&status=active

Standard pagination. Use this for support tooling.

13.7 Tenant policy

Per-tenant overrides on platform defaults — every field is nullable; null means “inherit platform default”.

GET    /v1/admin/tenant-policy/{tenant_id}
PATCH  /v1/admin/tenant-policy/{tenant_id}        # partial; explicit null clears
DELETE /v1/admin/tenant-policy/{tenant_id}        # drop the row entirely
GET    /v1/admin/tenant-policy/defaults-view

Auth: Admin+ for own tenant; platform admin can edit any.

Editable fields:

{
  "tenant_id": "acme",
  "auto_classify_enabled": true,
  "default_template_id": "tpl_...",
  "rate_limit_per_minute": 200,
  "review_agreement_floor": 0.85,
  "review_confidence_floor": 0.90,
  "review_warn_count_threshold": 3,
  "pptx_notes_mode": "embed_notes | skip_notes",
  "characterisation_enabled": true,
  "generic_entities_enabled": true,
  "judge_mode_enabled": true,
  "table_aware_extraction_enabled": true,
  "draft_on_miss_enabled": true,
  "ocr_selection_mode": "paddle_priority | glm_priority | confidence_based",
  "ocr_priority_margin": 0.08,
  "feature_flags": { "flag_a": true, "flag_b": false }
}

Platform-wide defaults are at /v1/platform/defaults (platform_admin only): same shape plus the read-only upload limits (max_upload_size_bytes, pdf_max_pages, pdf_max_file_size_bytes, office_max_file_size_bytes).

---

14. Runtime-editable endpoints

Two endpoint families both run on the runtime-editable model:

• §14.1 Infra endpoints — where the platform sends its OCR + LLM traffic (paddleocr_vl, glmocr_vl, llm).
• §14.2 Forge endpoints — Claude Code backend selection for the template forge (anthropic, bedrock, vertex, custom_url).

14.1 Infra endpoints (model server config)

Three runtime-editable endpoints control where the platform sends its OCR + LLM traffic. Editing requires no restart. Two surfaces share the same row table (infra_endpoints): the platform path (below, platform_admin) manages the global defaults; the tenant path (§14.1.1) lets tenant_admin users author per-tenant overrides that fall back to the platform default for any type they leave unset.

GET  /v1/admin/platform/infra
GET  /v1/admin/platform/infra/{endpoint_type}
PUT  /v1/admin/platform/infra/{endpoint_type}
POST /v1/admin/platform/infra/{endpoint_type}/test

Auth: Platform_admin. endpoint_type is one of paddleocr_vl, glmocr_vl, llm.

PUT body

{
  "backend_kind": "vllm",
  "base_url": "http://glm:8000",
  "served_model": "glmocr-large",
  "auth_header": "Bearer token123 | null | \"\"",
  "max_concurrent": 4,
  "timeout_seconds": 60.0,
  "max_model_len": 512000
}

backend_kind is one of vllm, ollama, openai, gemini, anthropic. Set it to match the server at base_url. The first three are OpenAI-wire dialects (they differ only in the max_tokens vs max_completion_tokens field the client emits); gemini and anthropic select native vendor SDKs and are LLM-only (the two OCR-VLM endpoints stay on an OpenAI-compatible server). Structured output is vendor-agnostic: each backend enforces the template’s JSON Schema with its own native mechanism — json_schema response_format (OpenAI / vLLM guided_json), response_schema (Gemini), or a forced tool call (Anthropic) — with the repair loop as the universal fallback for backends that only guide the schema.

Gemini (backend_kind="gemini"). auth_header carries the Google API key (the native SDK is used, not Gemini’s OpenAI-compat shim — that’s beta and silently drops unlisted params). base_url is optional: leave it at https://generativelanguage.googleapis.com (or empty) for the public Developer API, or set a proxy / private endpoint. served_model is a Gemini model id (e.g. gemini-2.5-flash, gemini-2.5-pro). Sampling maps natively — LLM_REASONING_EFFORT → thinking budget (disabled by default for the constrained-JSON call sites; gemini-2.5-pro is floored at 128), and safety defaults to BLOCK_NONE so document extraction isn’t silently filtered (set LLM_GEMINI_SAFETY=default to opt out). First-boot bootstrap seeds it from LLM_BACKEND_KIND=gemini + LLM_API_KEY.

Anthropic (backend_kind="anthropic"). The native anthropic SDK against the Messages API (/v1/messages). Its purpose is enforced structured output: the extraction call maps the template’s JSON Schema onto a single forced tool, so the model’s answer is constrained to the schema provider-side — no repair round-trip in the happy path. This is the reliable enforcement path on gateways that merely guide response_format (notably Ollama Cloud, which silently returns prose for a json_schema request). Auth is host-based: the public API (https://api.anthropic.com) uses x-api-key; an Anthropic-compatible gateway (e.g. Ollama Cloud at https://ollama.com) uses Authorization: Bearer — store the token in auth_header (a bare key or a Bearer <key> value both work; the client unwraps it). served_model is the model id served at that endpoint. LLM-only, like Gemini.

Base URL & /v1. For OpenAI-wire backends, enter the server root (http://host:11434, http://host:8000) — the client appends /v1. A base that already ends in /v1 is handled idempotently, so Ollama Cloud’s documented https://ollama.com/v1 works as-is (backend_kind="ollama", auth_header="Bearer <key>").

auth_header semantics:

Value	Effect
omitted / null	Keep existing.
"" (empty string)	Clear.
Any other string	Set / overwrite.

Auth headers are encrypted at write time (Fernet, via CONNECTION_ENCRYPTION_KEYS) and never echoed in responses (only has_auth_header: bool is exposed). Storing any credential requires CONNECTION_ENCRYPTION_KEYS to be set on the api + worker — an Ollama Cloud token or a Gemini API key; without it the PUT returns 503 with an actionable message (keyless local servers carry no header and are unaffected). base_url must include scheme + hostname.

A endpoint_audit row is written for every PUT and TEST. A redis pub/sub message invalidates worker caches.

TEST response

The PUT and TEST surfaces both run a three-stage schema-aware validation probe: (1) GET /v1/models reachability, (2) served_model listed, (3) chat-completions shape probe — for llm a response_format={"type":"json_object"} call that asserts the body decodes; for paddleocr_vl / glmocr_vl a tiny synthetic PNG over the multimodal chat surface that asserts the response shape matches OpenAI chat-completions. For the native-SDK backends (backend_kind="gemini" and "anthropic") the same three stages run through the vendor SDK instead of /v1/... HTTP — models.list() for reachability + served-model presence, then a tiny native completion JSON probe for the shape check. PUT rejects the save on failure (HTTP 422) when INFRA_VALIDATION_HARD_GATE=true (default); the row’s last_test_* columns are populated either way so the UI can render “validated 3 days ago” without re-probing. On success the response includes a neutral informational note that performance characteristics depend on the operator’s hardware/model/load.

{
  "ok": true,
  "served_models": ["glmocr-large"],
  "latency_ms": 123.45,
  "error": null,
  "stages": {
    "reachable": { "ok": true, "latency_ms": 12.0 },
    "served_model_present": { "ok": true, "served_models": ["glmocr-large"] },
    "shape_probe": { "ok": true, "latency_ms": 8.0, "probe": "chat_completions_image" }
  },
  "informational": "Performance characteristics will depend on your endpoint's hardware, model, and load. Run a few representative documents to confirm latency and throughput meet your needs."
}

14.1.1 Tenant overrides

Tenants on tenant_policies.tenant_infra_overrides_enabled=true (a platform_admin flips this; default false) can author per-tenant overrides for any subset of the three endpoint types. Resolution falls back to the platform default for any type the tenant has not overridden, so a tenant that overrides only llm still uses the platform’s paddle/glm.

GET    /v1/admin/tenants/{tenant_id}/infra                     # all three endpoints with effective + override + platform_default
GET    /v1/admin/tenants/{tenant_id}/infra/{endpoint_type}
PUT    /v1/admin/tenants/{tenant_id}/infra/{endpoint_type}     # upsert tenant override; same body shape + validation as platform PUT
DELETE /v1/admin/tenants/{tenant_id}/infra/{endpoint_type}     # delete override; tenant returns to platform fallback
POST   /v1/admin/tenants/{tenant_id}/infra/{endpoint_type}/test

Auth: Tenant_admin+. Tenant admins are pinned to their own tenant_id; platform_admin may pass any.

Kill switch. PUT / DELETE / POST .../test return 403 with status_detail="tenant_infra_overrides_disabled" when the tenant policy flag is off. GET paths stay open regardless so a tenant can always see which endpoints their jobs are using. platform_admin bypasses the kill switch — needed during tenant onboarding to pre-stage overrides before the flag is flipped.

GET response shape (one entry per endpoint type, three total):

{
  "tenant_id": "acme",
  "overrides_enabled": true,
  "endpoints": [
    {
      "endpoint_type": "llm",
      "is_override": true,
      "effective":         { "endpoint_type": "llm", "tenant_id": "acme", "backend_kind": "ollama", "base_url": "http://acme-llm", ... },
      "tenant_override":   { "endpoint_type": "llm", "tenant_id": "acme", "backend_kind": "ollama", "base_url": "http://acme-llm", ... },
      "platform_default":  { "endpoint_type": "llm", "tenant_id": null,   "backend_kind": "vllm",   "base_url": "http://platform-llm", ... }
    },
    { "endpoint_type": "paddleocr_vl", "is_override": false, "effective": { ... platform_default ... }, "tenant_override": null, "platform_default": { ... } }
  ]
}

The PUT and TEST routes run the same three-stage validation probe described above; PUT hard-gates save under INFRA_VALIDATION_HARD_GATE. Audit events for tenant-side changes use the tenant.infra.endpoint.{updated,deleted,tested} event names and audit under the affected tenant’s id rather than the platform tenant id, so the tenant audit log surfaces the change.

14.2 Forge endpoints (Claude Code backend config)

The template forge (§8.14) spawns Claude Code as a subprocess per iteration; the backend it talks to (Anthropic Direct, Amazon Bedrock, Google Vertex, or any Anthropic-format custom URL proxy) is configured per tenant via these endpoints. A tenant_id IS NULL row is a platform default available to every tenant; only platform_admin callers create / mutate those.

GET    /v1/admin/forge-endpoints                      # list (own tenant ∪ platform defaults)
POST   /v1/admin/forge-endpoints                      # create
GET    /v1/admin/forge-endpoints/{id}                 # detail
PUT    /v1/admin/forge-endpoints/{id}                 # update; null on optional fields = clear
DELETE /v1/admin/forge-endpoints/{id}                 # soft-delete
POST   /v1/admin/forge-endpoints/{id}/set-default     # set as tenant default (clears prior)
GET    /v1/admin/forge-endpoints/backends/list        # surface ["anthropic","bedrock","vertex","custom_url"]

Auth: Tenant_admin+. Tenants see their own rows + platform defaults; only platform_admin may mutate platform-default rows or act cross-tenant.

POST body

{
  "name": "default-anthropic",
  "backend": "anthropic",                              // one of: anthropic | bedrock | vertex | custom_url
  "model": "claude-opus-4-7",
  "base_url": "https://proxy.internal/v1",            // required for custom_url
  "aws_region": "us-east-1",                           // required for bedrock
  "gcp_project": "my-project",                         // required for vertex
  "gcp_region": "us-central1",                         // required for vertex
  "credentials": {
    "api_key": "sk-ant-...",                           // anthropic / custom_url
    "access_key_id": "AKIA0000",                       // bedrock
    "secret_access_key": "...",                        // bedrock
    "session_token": "...",                            // bedrock (optional STS)
    "service_account_json": "..."                      // vertex (full SA JSON as a string)
  },
  "default_max_budget_usd": 5.0,
  "default_max_turns": 50,
  "is_default_for_tenant": true,
  "tenant_id": null                                    // platform_admin: pass null for platform default
}

Credentials are encrypted at write time using the same Fernet keyset as connections (CONNECTION_ENCRYPTION_KEYS); responses NEVER echo the raw key, only has_credentials: bool and encryption_key_id. base_url (custom_url only) must include scheme + hostname.

PUT body

Same shape as POST; fields absent from the request body are KEPT, fields explicitly set to null are CLEARED. credentials MAY be omitted to keep the existing ciphertext, or supplied as a full new object to atomically replace. Setting tenant_id is not allowed via PUT — endpoints don’t migrate between tenants. Backend itself is also immutable post-create (delete + recreate to switch).

The full /test round-trip (spawn claude --bare -p against the configured endpoint) is a Phase 3 follow-up that lands with the forge-worker container’s HTTP surface; until then, operators verify endpoints empirically by submitting a small forge job. last_tested_at / last_test_ok columns stay null.

---

15. Audit log

Every IAM, template, infra, and policy change is recorded.

GET /v1/admin/audit?limit=50&offset=0          # tenant-scoped (Admin+)
GET /v1/platform/audit?limit=50&offset=0       # all tenants (Platform_admin)

Both return:

{
  "events": [
    {
      "id": "...",
      "tenant_id": "acme",
      "event": "user_invited | user_promoted | template_created | policy_updated | endpoint_updated | ...",
      "actor_id": "cred_...",
      "target_type": "user | template | endpoint | tenant | policy | ...",
      "target_id": "...",
      "payload": { /* before / after snapshots, rule kind, etc. */ },
      "created_at": "..."
    }
  ],
  "total": 500,
  "limit": 50,
  "offset": 0
}

/v1/platform/audit accepts optional tenant_id and event filters.

---

16. Health, metrics, and operational surfaces

16.1 Health

GET /v1/health

Auth: none. Returns:

{
  "status": "ok | degraded | unhealthy",
  "components": {
    "database": true,
    "redis": true,
    "worker": true,
    "storage": true,
    "providers": {
      "paddleocr-vl": true,
      "glmocr-vl": false,
      "llm": true
    }
  },
  "queue_depth": 42
}

status: degraded means at least one non-database component is unhealthy but the API can still serve reads. status: unhealthy means database or storage is down.

16.2 Metrics

GET /metrics

Auth: none. Prometheus exposition format. Notable gauges:

• provider_health{provider="paddleocr-vl|glmocr-vl|llm"} — 1 healthy, 0 unhealthy.
• queue_depth — SAQ pending count.
• docstack_layout_* counters for the layout robustness ladder.
• docstack_preprocessing_source_class_total{class} — per-class document classification counter.
• Standard ASGI request latency / status histograms via api/observability.py.

16.3 Service info

GET /

Returns { "service": "docstack", "version": "1.0.0", "docs": "/docs" }.

16.4 OpenAPI + Swagger

FastAPI auto-generated docs are exposed:

Path	Purpose
/openapi.json	OpenAPI 3.1.0 schema covering every routed endpoint and Pydantic model.
/docs	Swagger UI explorer.
/redoc	ReDoc alternative renderer.

These are the FastAPI defaults — there is no current setting to disable them. Operators who do not want them publicly accessible should restrict via gateway or by patching api/main.py to construct FastAPI(docs_url=None, redoc_url=None, openapi_url=None).

The OpenAPI spec drives every part of the docs experience that an integrator might care about:

• The interactive API Reference in the docs site is rendered from a snapshot of /openapi.json (refreshed via npm run sync:openapi from docs-site/).
• Code generators (openapi-typescript, openapi-python-client, oapi-codegen, etc.) work directly against the live URL.
• Contract testing runs the spec back against the live API to catch any drift between code and contract — see §16.5 below.

Endpoints absent from /openapi.json. A handful of routes are exposed but not listed in the spec because they are not registered as FastAPI route handlers: /metrics (Prometheus exposition wired into the ASGI app via api/observability.py), /docs, /redoc, and /openapi.json itself (FastAPI internals). Any integrator that wants the spec to be the complete surface should treat these four paths as out-of-band by convention.

16.5 Contract testing

The spec is the contract; the contract should be tested.

We recommend schemathesis — a property-based fuzz tester driven by an OpenAPI spec. It walks every documented endpoint, generates request payloads that satisfy the declared schemas, and asserts that responses match the declared shapes and status codes.

Bring up the stack, mint a test API key, then:

pip install schemathesis
schemathesis run \
    --base-url http://localhost:8080 \
    -H "Authorization: Bearer rk_..." \
    http://localhost:8080/openapi.json \
    --checks all \
    --workers 4 \
    --hypothesis-deadline=10000

Common drift this catches in practice:

1. A handler raises a status code (e.g. 404) that the route’s responses= declaration doesn’t list.
2. A Pydantic model was renamed but the response still uses the old field shape.
3. A path parameter accepts values outside the declared regex (e.g. tenant_id="../../../etc/passwd").
4. A response_model= was forgotten on a new route — schemathesis sees the spec doesn’t predict the actual fields.
5. Multipart upload size or page-count limits aren’t reflected in the spec.

Treat each schemathesis failure as a contract bug — fix the code or fix the spec; never silence the failure. The full how-to (CI integration, spec linters, generator tooling) lives on the docs site under Contract testing.

---

17. Errors

17.1 Envelope

All error responses use the FastAPI default envelope:

{ "detail": "human-readable message" }

Pydantic / FastAPI validation errors (status 422) use the framework default — an array of {loc, msg, type} items.

17.2 HTTP status code conventions

Status	When
200 OK	Successful read or non-creating mutation.
201 Created	Successful resource creation (templates, API keys, tenants, etc.).
202 Accepted	Asynchronous work enqueued (document submit, retry, reprocess, batch).
204 No Content	Resource removed (template draft delete).
400 Bad Request	Application-level validation failure (invalid json_schema, regex doesn’t compile, JSON Pointer doesn’t resolve, conflicting body fields).
401 Unauthorized	No credential, expired session, bad token.
403 Forbidden	Authenticated but not allowed (role insufficient).
404 Not Found	Resource not found or belongs to another tenant — existence is not leaked.
409 Conflict	State conflict (job not in required status, review already terminal, template name collision, attempting to demote a platform_admin).
410 Gone	Single-use token already consumed.
413 Payload Too Large	Upload exceeds the per-format size limit.
415 Unsupported Media Type	Upload MIME type not in the allowlist.
422 Unprocessable Entity	FastAPI / Pydantic schema validation failed (most validation surfaces as 400; this fires for shape errors).
429 Too Many Requests	Per-tenant or per-IP rate limit exceeded. Retry-After header included.
5xx	Server fault. No detail leaked; structured logs carry the trace.

17.3 Failure codes (document_jobs.failure_code)

Populated by the worker on status: failed. Use these to differentiate retriable from permanent failures in your downstream:

Family	Code	Retriable?
Input validation	invalid_format, unsupported_format, file_too_large	no
Office conversion (Gotenberg)	office_conversion_rejected, office_conversion_failed, office_conversion_timeout, office_conversion_unreachable, office_conversion_empty, office_conversion_route_missing, office_conversion_unauthorised	situational
PDF processing	pdf_encrypted, pdf_corrupted, pdf_invalid, pdf_empty, pdf_too_many_pages	no
Pipeline	ocr_processing_failed, extraction_failed, characterisation_failed, processing_failed	yes (via POST /retry)
User action	cancelled	n/a

office_conversion_unreachable typically means the Gotenberg container is unhealthy — once it’s back, POST /retry should succeed.

---

18. Webhooks and async notifications

Status: not implemented.

The platform is designed for 5–15 second job latencies where polling GET /v1/documents/{job_id} is acceptable. Webhooks were considered and explicitly deferred (docs/api-design.md §8).

Recommended polling pattern:

import time, requests

def wait_for_completion(api, key, job_id, timeout_s=300, interval_s=2):
    deadline = time.time() + timeout_s
    while time.time() < deadline:
        r = requests.get(f"{api}/v1/documents/{job_id}",
                         headers={"Authorization": f"Bearer {key}"})
        r.raise_for_status()
        body = r.json()
        if body["status"] in ("completed", "needs_review", "rejected", "failed"):
            return body
        time.sleep(interval_s)
    raise TimeoutError(f"Job {job_id} did not complete within {timeout_s}s")

For high-throughput integrations, scale this by polling GET /v1/documents?status=completed&since=<last_seen> instead of one job at a time.

---

19. Versioning policy

• One active API version: /v1/…. There is no /v2/… yet.
• Backward-compatible changes (new fields, new endpoints, new optional query params) ship under /v1 without notice.
• Breaking changes will land under /v2/… alongside /v1/… for an explicit deprecation window. The sunset date and migration guide will appear in docs/CHANGELOG.md (TBD when /v2 ships).
• Per-resource versioning (e.g. /v1/documents and /v2/documents simultaneously) is not done — the version is a global router prefix.

Internal IDs (template versions, audit revisions) are independent of the HTTP API version and may bump freely.

---

20. CLI reference

docstack-admin is the operator CLI for IAM and infra surfaces that have no HTTP analogue (or where running locally on the API host is operationally safer).

# First platform admin (only when no platform_admin exists)
docstack-admin platform-admin create --email you@example.com

# Tenants + users
docstack-admin tenant create --id acme --name "Acme" --initial-admin-email admin@acme.com
docstack-admin tenant list
docstack-admin user invite --tenant-id acme --email new.user@acme.com --role tenant_user
docstack-admin user reset-password --email user@acme.com

# API keys
docstack-admin api-keys create --tenant-id acme --user-email admin@acme.com --name ingest-cli
docstack-admin api-keys list --tenant-id acme
docstack-admin api-keys rotate --public-id abcd1234
docstack-admin api-keys revoke --public-id abcd1234

# Templates
docstack-admin templates import --tenant-id acme --path ./my_templates/

# Infra endpoints (runtime-editable)
docstack-admin infra list
docstack-admin infra set llm --url https://my-vllm.internal:8000 --model qwen3-8b --timeout 300
docstack-admin infra test llm
docstack-admin infra describe llm

# Tenant-scoped overrides — alembic 20260507_0001. The same
# subcommands gain a --tenant-id flag that scopes to a tenant
# override row instead of the platform-default layer. Operators use
# this when the admin UI is unreachable; it talks directly to the
# infra_endpoints table and publishes redis pub/sub on commit so
# running api/worker replicas pick up the change without a restart.
docstack-admin infra list  --tenant-id acme
docstack-admin infra set   --tenant-id acme llm --url https://acme-llm.internal --model qwen3-8b
docstack-admin infra reset --tenant-id acme llm
docstack-admin infra test  --tenant-id acme llm

The infra commands operate on the same infra_endpoints table the HTTP API exposes; either path works.

---

21. Endpoint index

Quick navigation to every endpoint in this guide.

Resource	Method	Path
Documents	POST	/v1/documents
	GET	/v1/documents
	GET	/v1/documents/{job_id}
	POST	/v1/documents/{job_id}/retry
	POST	/v1/documents/{job_id}/reprocess
	POST	/v1/documents/{job_id}/reextract
	POST	/v1/documents/{job_id}/cancel
	DELETE	/v1/documents/{job_id}
	POST	/v1/documents/{job_id}/restore
	GET	/v1/documents/{job_id}/reupload-context
Reviews	GET	/v1/reviews
	GET	/v1/reviews/{review_id}
	POST	/v1/reviews/{review_id}/approve
	POST	/v1/reviews/{review_id}/reprocess
	POST	/v1/reviews/{review_id}/override
Templates	GET	/v1/templates
	POST	/v1/templates
	GET	/v1/templates/{template_id}
	PUT	/v1/templates/{template_id}
	POST	/v1/templates/{template_id}/activate
	POST	/v1/templates/{template_id}/deactivate
	DELETE	/v1/templates/{template_id}
	POST	/v1/templates/{template_id}/test
	POST	/v1/templates/test
	POST	/v1/templates/classify
	GET	/v1/templates/{template_id}/versions
	GET	/v1/templates/{template_id}/audit
	GET	/v1/templates/meta/rule-kinds
Template drafts	POST / GET / GET / PATCH / POST / DELETE	/v1/templates/drafts[/{id}[/accept]]
Template forge	POST / GET	/v1/templates/forge/jobs
	GET	/v1/templates/forge/jobs/{job_id}
	POST	/v1/templates/forge/jobs/{job_id}/abort
	GET	/v1/templates/forge/jobs/{job_id}/iterations
	GET	/v1/templates/forge/jobs/{job_id}/iterations/{iteration_id}
	POST	/v1/templates/forge/jobs/{job_id}/promote
Forge endpoints	GET / POST	/v1/admin/forge-endpoints
	GET / PUT / DELETE	/v1/admin/forge-endpoints/{id}
	POST	/v1/admin/forge-endpoints/{id}/set-default
	GET	/v1/admin/forge-endpoints/backends/list
Batches	POST / GET	/v1/batches
	GET	/v1/batches/{batch_id}
	GET	/v1/batches/{batch_id}/jobs
	POST	/v1/batches/{batch_id}/cancel
	POST	/v1/batches/{batch_id}/retry-failed
	POST	/v1/batches/{batch_id}/reprocess
	POST	/v1/batches/{batch_id}/reconcile
Admin (ops)	POST	/v1/admin/jobs/recover-stale
	GET	/v1/admin/tenant-policy/registry-spec
Connections	GET / POST	/v1/admin/connections
	GET / PATCH / DELETE	/v1/admin/connections/{connection_id}
	POST	/v1/admin/connections/{connection_id}/test
Artifacts	GET	/v1/artifacts/{artifact_id}
Identity	GET	/v1/me
Auth	POST	/v1/auth/login
	POST	/v1/auth/logout
	POST	/v1/auth/change-password
	GET / POST	/v1/auth/invites/{raw_token} / /v1/auth/accept-invite
	GET / POST	/v1/auth/reset-tokens/{raw_token} / /v1/auth/reset-password
	GET	/v1/auth/sessions
	POST	/v1/auth/sessions/{session_id}/revoke
Users	GET / POST	/v1/admin/users
	GET / DELETE	/v1/admin/users/{user_id}
	POST	/v1/admin/users/{user_id}/promote
	POST	/v1/admin/users/{user_id}/demote
	POST	/v1/admin/users/{user_id}/suspend
	POST	/v1/admin/users/{user_id}/activate
	POST	/v1/admin/users/{user_id}/reset-password
	POST	/v1/admin/users/{user_id}/resend-invite
	POST	/v1/admin/users/{user_id}/revoke-invite
API keys (admin)	GET / POST	/v1/admin/api-keys
	POST	/v1/admin/api-keys/{public_id}/rotate
	POST	/v1/admin/api-keys/{public_id}/revoke
API keys (self)	GET / POST	/v1/account/api-keys
	POST	/v1/account/api-keys/{public_id}/rotate
	POST	/v1/account/api-keys/{public_id}/revoke
Profile	PATCH	/v1/account/profile
Tenants	GET / POST	/v1/platform/tenants
	GET / PATCH / DELETE	/v1/platform/tenants/{tenant_id}
	POST	/v1/platform/tenants/{tenant_id}/suspend
	POST	/v1/platform/tenants/{tenant_id}/activate
Platform users	GET	/v1/platform/users
Tenant policy	GET / PATCH / DELETE	/v1/admin/tenant-policy/{tenant_id}
	GET	/v1/admin/tenant-policy/defaults-view
Platform defaults	GET / PATCH	/v1/platform/defaults
Infra (platform)	GET	/v1/admin/platform/infra
	GET / PUT	/v1/admin/platform/infra/{endpoint_type}
Infra (tenant)	GET	/v1/admin/tenants/{tenant_id}/infra
	GET / PUT / DELETE	/v1/admin/tenants/{tenant_id}/infra/{endpoint_type}
	POST	/v1/admin/tenants/{tenant_id}/infra/{endpoint_type}/test
	POST	/v1/admin/platform/infra/{endpoint_type}/test
Audit	GET	/v1/admin/audit
	GET	/v1/platform/audit
Health	GET	/v1/health
Service info	GET	/
OpenAPI	GET	/openapi.json, /docs, /redoc (not listed in the spec)
Metrics	GET	/metrics (not listed in the spec)

---

22. Further reading

• docs/architecture.md — pipeline internals, serving layer GPU budget, prompt-budget regime.
• docs/admin-iam-design.md — full IAM spec: tenants, users, API keys, sessions, invites, role normalisation, bootstrap.
• docs/intake-and-context-design.md — Office conversion (Gotenberg) and Stage 5.5 characterisation.
• docs/intelligent-extraction-design.md — auto-classify and draft-on-miss flows.
• docs/template-forge-design.md — the forge methodology, schema-evolution policies, leak audit, scoring contract.
• docs/environment-variables.md — every operator-tunable env var and its default.
• sample_templates/ — operator-importable starter templates.