# DocStack vision-first OCR architecture # DocStack vision-first OCR architecture ## Full PaddleOCR-VL, full Docling with Granite Docling, and GLM-OCR coexistence **Status:** research record, updated with the implemented August 2026 technology review **Research date:** 3–4 August 2026 **Product:** DocStack 2.4.3 **DocStack repository snapshot:** commit 8e1c1553d244f142b39c4d9536af82044767f5e1 **Decision scope:** product boundaries, rendering, OCR, visual document understanding, provider-neutral infrastructure control, orchestration, canonical output, structured extraction, evaluation, deployment, migration, and removal of GLiNER2 --- ## Executive decision DocStack should remain an OCR and faithful visual document-understanding product. Its primary truth must be what a person can see on a rendered page, slide, sheet, or image. That rule applies to scanned documents and born-digital documents alike. The original review selected two independent full visual-document lanes: 1. **Paddle lane:** the complete PaddleOCR/PaddleX document-parsing pipeline on the PaddleOCR 3.7 and PaddleX 3.7 lines with PaddleOCR-VL-1.6. 2. **Docling lane:** the complete Docling conversion framework using its VLM pipeline with Granite Docling and DocTags as an independent comparison profile. The August 2026 update changes the routing conclusion. Project use found Granite Docling substantially worse on real documents than the other available paths. The current Docling adapter has been corrected to the 2.124 API, but the lane remains explicit-only until a controlled corpus shows that the corrected integration adds value. It is not a default, fallback, or production-primary recommendation. The current target adds three distinct capabilities without conflating their contracts: - PP-OCRv6 medium for localized text recognition across its unified language set; - Granite Vision 4.1 for targeted tables, charts, captions, and key-value tasks; - HPD-Parsing as an explicitly selected high-throughput full-page lane whose lower quality is accepted only by the `hpd-throughput` profile. GLM-OCR coexists with those lanes through a different abstraction: a **visual recognition provider** for targeted text, table, and formula recognition on uncertain regions. Its complete SDK may also run as an experimental or shadow document lane, but it should not be the default third full pipeline. This separation is essential. A complete document pipeline and a crop-level VLM are not interchangeable: - Full Paddle owns page layout analysis, crop formation and merging, task routing, task prompts, task-specific decoding, page assembly, and optional cross-page restructuring. - Full Docling owns format conversion, the VLM execution contract, response-format parsing, construction of a DoclingDocument, serialization, and conversion status. - GLM-OCR can recognize regions very effectively, and its SDK also combines PP-DocLayoutV3 with GLM inference, but a raw GLM endpoint does not automatically implement either Paddle's or Docling's orchestration and output grammar. The product flow should therefore be: > render first; parse visually; retain engine-native evidence; normalize into a small DocStack-owned visual document model; assess coverage and uncertainty; invoke a secondary lane or GLM only where justified; produce faithful Markdown; optionally use an LLM and a template schema to produce validated JSON. The following decisions supersede the earlier design: - A native text layer is never sufficient reason to skip visual parsing. - DoclingDocument is an important lane-native artifact, not DocStack's sovereign cross-engine schema. - Docling's native Office or spreadsheet parsing can supply corroborating evidence, but it is not the visual source of truth. - The existing PDF text fast path remains disabled and should be retired from visual-fidelity profiles. - PowerPoint notes must not be inserted into a slide margin by default because that changes the slide geometry. Notes become separately identified surfaces. - Excel requires both author print-view surfaces and a coverage view with an overview and readable, overlapping tiles. - “OpenAI-compatible” describes transport, not document-pipeline, prompt, token, geometry, or response-format compatibility. - PaddleOCR-VL-1.6, Granite Docling, and GLM-OCR are code-owned semantic contracts; the organization that hosts each contract is an independently configurable provider binding, never a compile-time vendor dependency. - Platform defaults and tenant overrides must be editable, testable as unsaved drafts, and applicable from the web console. Apply creates a monotonic revision used by every subsequently planned job without restarting API, worker, or visual services; an already planned or in-flight job remains pinned to its prior revision. - GLiNER2 and the generic automatic NER mode are removed. Structured extraction remains template-driven and LLM-based. - The public outputs remain high-fidelity Markdown and, when requested, schema-constrained JSON. The recommended quality topology is Paddle Full as the complete lane, PP-OCRv6 for uncertain text crops, Granite Vision for structured visual crops, and GLM for formula recognition. HPD is a separate throughput choice. Granite Docling remains available for controlled comparison, not automatic routing. Dependency selectors follow one rule across the implementation: select X.Y and float Z. Resolved image digests and model/file hashes are provenance records, not dependency selectors. --- ## 1. Product concept and boundary ### 1.1 Product statement DocStack converts visually presented documents into: - honest, readable, high-accuracy Markdown that represents visible content and document structure as faithfully as Markdown permits; or - template-driven structured JSON derived from that grounded representation and validated against the requested schema and source evidence. DocStack is not trying to recover the author's editable source model, reproduce every application behavior, or become a general document intelligence and retrieval platform. ### 1.2 The governing doctrine: visual presentation is primary evidence Most business documents are authored for appearance. Their internal structures are often incomplete, misleading, or optimized for a specific application rather than for semantic extraction: - A PDF page is a sequence of painting operations. Character emission order need not be reading order, and tagged logical structure is optional. - A slide is predominantly a shape tree. Text, images, charts, connectors, and WordArt can be freely positioned, overlapped, layered, grouped, and rotated. - A spreadsheet cell can contain a formula and a cached result while the printed cell displays a formatted value. Headers, tables, totals, and visual separators can be communicated through merged cells, whitespace, borders, color, chart placement, print areas, scaling, and page breaks. - Word-processing pagination depends on renderer behavior, fonts, page setup, floating objects, sections, and field evaluation. - Prepress PDFs may look correct while their text objects have an unusable logical order. The consequence is architectural, not cosmetic: > Every supported document must enter a visual-document path. Native structures may improve exactness or explain ambiguity, but they may not silently replace what is visibly presented. ### 1.3 In scope - PDF, including born-digital, scanned, hybrid, prepress, forms, and photographed pages. - Raster images and supported vector images. - DOCX, PPTX, and XLSX through deterministic rendering. - Page, slide, and worksheet-surface layout understanding. - Visible text, tables, formulas, figures, captions, charts, seals, headers, footers, and spatial grouping. - Reading order grounded in visual organization. - Multi-scale rerendering and crop recognition. - Faithful Markdown plus referenced visual assets. - Template-selected, LLM-produced structured JSON with schema and grounding checks. - Engine routing, cascades, parallel execution, shadow evaluation, provenance, uncertainty, and human-review signals. ### 1.4 Explicitly out of scope - Competing with LlamaCloud, LlamaParse, or a universal parsing platform. - General-purpose RAG ingestion, semantic chunking, embeddings, search, question answering, or agent workflows. - Reconstructing editable PPTX, DOCX, XLSX, or PDF files. - Treating an XLSX workbook as a general ETL or calculation engine. - Executing macros or fetching arbitrary external workbook or document resources. - Inventing descriptions for figures and presenting them as original source text. - Claiming exact visual equivalence when the rendering application, fonts, linked resources, or dynamic content differ from the author's environment. ### 1.5 Output honesty “High-quality Markdown” must not mean cosmetically fluent text that conceals uncertainty. It means: - visible content is not knowingly omitted; - content is not paraphrased in the canonical transcription; - generated descriptions are explicitly labeled; - reading order and grouping follow the rendered artifact; - tables preserve cell and span structure when reliably recognized; - page or surface provenance remains available; - unreadable material is marked as unreadable or uncertain rather than guessed; - important disagreements remain inspectable in a sidecar evidence record. --- ## 2. Corrections to the previous architecture The earlier proposal underestimated visual interpretation in born-digital files and placed too much authority in native conversion and a shared document model. The corrected decisions are: | Earlier direction | Corrected direction | |---|---| | Native parsing can be the healthy born-digital path | All formats use a visual path; native data is auxiliary evidence | | DoclingDocument can be the central cross-engine representation | Each engine retains its native artifact; DocStack owns a minimal CanonicalVisualDocument | | One parser interface can abstract everything | Full document lanes and crop-level recognizers require different contracts | | A VLM endpoint can be injected wherever transport matches | Adapter compatibility includes task semantics, prompts, preprocessing, special tokens, output grammar, postprocessing, and geometry | | Fixed-resolution page rendering is adequate | Rendering is multi-scale and adaptive, with immutable coordinate transforms | | PowerPoint notes can share the slide page | Slide and notes are distinct surfaces | | Excel print pages represent the workbook sufficiently | Print-view pages and a coverage overview/tile set are both required | | A text-bearing PDF may bypass vision | Text layers may corroborate, but do not bypass visual layout analysis | | Generic NER remains an optional product mode | GLiNER2 and automatic NER are removed | This is not a rejection of native information. Native information can be uniquely valuable: exact characters, hyperlinks, formulas, cached values, comments, object anchors, accessibility tags, and document metadata. The correction is about authority. Where visual and native interpretations disagree, the output must first represent what was visible, record the discrepancy, and use native evidence only through explicit rules. --- ## 3. Research method and accuracy conventions ### 3.1 Source hierarchy This report relies on: 1. selected upstream minor lines and configuration; 2. official upstream documentation and model cards; 3. primary technical reports and benchmark repositories; 4. the current DocStack source, tests, configuration, and evaluation report. Vendor benchmark results are labeled as reported results, not as proof of DocStack quality. Architectural claims based on code inspection name the inspected version line because upstream branches can change. ### 3.2 Pinned research snapshots | Component | Revision inspected | |---|---| | DocStack | 8e1c1553d244f142b39c4d9536af82044767f5e1 | | PaddleOCR 3.7.0 | b03f46425e8ff4442b268ce449e3eef758146cd4 | | PaddleX 3.7.0 | e0068ce0bfe75b2992e5b38d06a0393c70f887f7 | | Docling | 9bdb2af94e93cab47fe96105850438eb663d7953 | | Docling Serve | 7f2b890c5538d5ad511dce42a69af5840452b1c4 | | Granite Docling model | 982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe | | GLM-OCR repository | cef4d0ea120d1741f5cefe8985eee45f6c8eff1d | | GLM-OCR model | ca5d8b3e287e52589e37c28385d9655ee4372f9d | ### 3.3 Paddle version terminology “Paddle 3.7.0” is ambiguous and should not appear in implementation plans. - **PaddleOCR 3.7.0** is the OCR package release. - **PaddleX 3.7.0** supplies the pipeline implementation used by PaddleOCR. - **PaddlePaddle** is the separate deep-learning framework. For the inspected PaddleOCR 3.7.0 release, published on 11 June 2026, the project dependency selects the PaddleX 3.7 line. The official compatibility guidance permits PaddlePaddle 3.x rather than defining a PaddlePaddle 3.7.0 release as part of the same version set. PaddleOCR-VL-1.6 arrived with PaddleOCR 3.6.0 on 28 May 2026 and is the current default PaddleOCR-VL model in the inspected 3.7 line; the major new model highlighted by PaddleOCR 3.7.0 itself is PP-OCRv6. ### 3.4 Meaning of “full integration” In this report: - **full Paddle integration** means using the supported PaddleOCR/PaddleX pipeline rather than calling only the PaddleOCR-VL model server; - **full Docling integration** means using Docling's conversion and VLM pipeline contracts, DoclingDocument artifact, conversion status, serializers, and supported deployment mechanisms rather than calling only Granite Docling; - **GLM provider integration** means using an explicit GLM task adapter with the official prompt and response semantics; - **full GLM SDK** means the official GLM page loader, PP-DocLayoutV3, region scheduling, GLM inference, and result formatting path. --- ## 4. Current DocStack baseline ### 4.1 Existing visual path The current implementation already has an important architectural strength: Office documents are not directly trusted as semantic source models. - DOCX, PPTX, and XLSX are submitted to Gotenberg's LibreOffice conversion route. - The produced PDF is persisted as an exact rendered derivative. - PDF pages are rasterized with PDFium. - Raster and vector images enter image preprocessing. - PP-DocLayoutV3 performs layout detection. - PaddleOCR-VL and GLM-OCR process selected regions. - Provider-specific results are merged into canonical Markdown. - Template-driven LLM extraction and judging occur after OCR. The redesign should preserve this visual backbone while making render surfaces, transforms, lanes, evidence, and quality decisions explicit. ### 4.2 Baseline quality that must not be lost The checked-in 62-document diagnosis reports: - invoice-number accuracy: 98.4%; - invoice-date accuracy: 98.4%; - total accuracy: 100%; - extraction composite: 0.9724. These measurements are not broad state-of-the-art evidence, but they are the relevant migration baseline. The existing behavior must be frozen as a versioned profile, provisionally named **legacy-dual-vlm-v1**, and included in every paired evaluation until a replacement passes the agreed gates. ### 4.3 Current rendering findings The local audit identified several concrete concerns: 1. The PDF text fast path is disabled by default because PDFium text extraction exposes drawing order rather than dependable visual reading order. That default is correct. A conflicting docstring that describes native text as strictly better should be corrected. 2. The default page rasterization is 150 DPI. A single universal resolution is too weak for fine print, dense tables, spreadsheets, stamps, and formulas, and wasteful for pages that are already easy. 3. PowerPoint currently defaults to notes in the margin. This changes the page geometry, shrinks or relocates the slide, and mixes non-slide content into the OCR plane. 4. Excel correctly does not force every sheet to one page, so author print areas and page breaks are respected. That is necessary but not sufficient: content outside the print area or scaled to unreadability still requires a coverage representation. 5. Gotenberg uses a floating patch tag. The resolved image digest is recorded as run provenance, while the image selector, LibreOffice packages, and font packages remain on selected version lines. 6. The code sends maxImageResolution without enabling LibreOffice's reduce-image-resolution behavior. Under current Gotenberg documentation the limit does not take effect by itself; configuration and comments should be aligned. 7. The code and older design document use exportNotePages, while current Gotenberg documentation describes exportNotesPages. This must be tested against the selected image line and corrected only after confirming the accepted form field. It is a validation item, not evidence that production conversion currently fails. ### 4.4 Renderer limitations that must be surfaced Gotenberg documents that LibreOffice conversion is not guaranteed to be identical to Microsoft Office. Complex styling and SmartArt may differ, missing fonts can change layout and pagination, macros are disabled, and linked external content can be absent even if conversion succeeds. Those conditions cannot be solved by a better OCR model. They require a render manifest, validation warnings, and an honest distinction between: - faithful OCR of the artifact DocStack actually rendered; and - guaranteed equivalence to the artifact the author saw in a different application environment. --- ## 5. Why born-digital documents still require vision ### 5.1 PDF The PDF imaging model defines page appearance through a sequence of text, graphics, and image painting operations. Logical structure is a separate and optional facility. Adobe's accessibility guidance explicitly distinguishes rendering order from reading order and warns that inferred structure in untagged files may be unsatisfactory. Therefore: - a selectable text layer is evidence that characters exist, not evidence of correct reading order; - text extraction may miss clipping, overlap, rotation, form appearance streams, figures, visual grouping, and page furniture; - layout analysis must inspect the rendered page; - native glyphs and tags may be used for exact spelling or accessibility semantics only after visual alignment. ### 5.2 PPTX PresentationML slide content is principally a shape tree. Its order determines stacking and navigation behavior, not a guaranteed semantic reading sequence. Authors can place and layer text boxes, pictures, tables, charts, connectors, groups, and WordArt freely. Therefore: - the slide surface at its actual aspect ratio is the primary unit; - z-order, overlap, whitespace, alignment, arrows, and proximity are semantic evidence; - a source XML walk cannot reliably determine the intended reading order; - notes, comments, hidden slides, animations, and alternate states require explicit policies rather than implicit inclusion. ### 5.3 XLSX SpreadsheetML separates formula, cached value, format, style, merge, drawing, print, and page-setup data. A workbook can contain several visually distinct tables on one sheet with nothing more than blank rows, blank columns, borders, fill colors, or aligned headings separating them. Charts and drawings have spatial anchors. Print areas may contain multiple nonadjacent ranges. Page breaks and scaling affect what a user sees. Therefore: - reading cells in row-major order is not document understanding; - formula text is not necessarily the value visible to the reader; - print view is meaningful evidence but can omit or shrink important sheet content; - the sheet must also be viewed as a spatial canvas; - native cells and formulas are useful corroboration only after mapping them to rendered geometry. ### 5.4 DOCX WordprocessingML has a stronger logical flow than presentation or spreadsheet formats, but visible pagination still depends on page settings, fields, floating shapes, tables, headers and footers, fonts, sections, and the rendering engine. Therefore: - print-view rendering remains primary; - native paragraphs, links, fields, and styles can assist exact recovery; - renderer substitutions and pagination changes must be recorded. ### 5.5 Scientific support The DocLayNet work treats accurate document layout analysis as a prerequisite for high-quality PDF conversion. OmniDocBench includes reading-order and element-structure evaluation rather than measuring only plain text. Real5 evaluates physical distortions such as scanning, warping, screen photography, illumination, and skew, illustrating the gap between clean benchmark pages and real visual inputs. These sources do not prove a particular DocStack engine choice. They support the broader requirement that document OCR quality is multidimensional: recognition, coverage, geometry, reading order, tables, formulas, and honest failure handling all matter. --- ## 6. Full PaddleOCR-VL-1.6: actual system boundary ### 6.1 What the official v3.x documentation establishes The PaddleOCR v3.x documentation distinguishes the complete PaddleOCR-VL pipeline from a bare document-VLM service. The complete flow includes: 1. page or document-image loading; 2. optional document preprocessing; 3. full-page layout detection with PP-DocLayoutV3; 4. region formation, crop processing, and block merging; 5. task selection for recognized block types; 6. PaddleOCR-VL inference with task-specific instructions; 7. task-specific output postprocessing; 8. page result assembly and Markdown/JSON export; 9. optional multi-page restructuring. The documentation warns that directly calling the model through a generic HTTP or OpenAI-style service does not reproduce the complete parsing workflow and can produce incomplete or hallucinated results if the caller does not implement the surrounding logic. ### 6.2 Full-pipeline stages and ownership | Stage | Paddle full lane owns | |---|---| | Layout | PP-DocLayoutV3 prediction and supported layout labels | | Geometry | page-to-model scaling, block bounding boxes, crops, ordering | | Task routing | selection of OCR, table, chart, formula, spotting, or seal task | | Prompting | Paddle-defined task prompts | | Decoding | model-output token and grammar interpretation | | Postprocessing | table OTSL conversion, formula cleanup, spotting parsing, repetition handling | | Assembly | page blocks, images, Markdown, JSON | | Multipage | optional page restructuring, including table and title reconstruction behavior | These are coherent parts of one upstream stack. DocStack should configure and observe them, not reimplement or selectively replace them in the first integration. ### 6.3 Pipeline defaults requiring a DocStack fidelity profile The selected PaddleX configuration has defaults that are reasonable for generic document conversion but not automatically correct for faithful OCR: - layout detection enabled; - document preprocessing disabled; - chart recognition disabled; - seal recognition disabled; - layout blocks merged; - queues enabled; - Markdown ignores number, footnote, header, header image, footer, footer image, and aside text labels. DocStack must supply an explicit profile. It should not silently discard visible headers, footers, footnotes, page numbers, or marginal text. Chart and seal treatment must be deliberate. If a visible chart is not converted into reliable data, its image and caption should still be preserved rather than disappearing. ### 6.4 Multipage behavior The pipeline exposes page restructuring after prediction. This can merge cross-page tables, reconstruct title levels, and concatenate page content. That is valuable but potentially destructive if it erases page boundaries or joins unrelated structures. DocStack should: - store the raw page-level Paddle result first; - run restructuring as a versioned, optional step; - record every cross-page join; - retain original surface identifiers and geometry; - allow a fidelity profile to preserve explicit page breaks. ### 6.5 Supported VLM runtimes are not the full service Paddle supports multiple inference arrangements for the VLM stage, including native execution and documented server backends such as vLLM, SGLang, FastDeploy, MLX-VLM, and llama.cpp where applicable. This is operational flexibility inside the full pipeline. It does not make those model servers complete document parsers. DocStack should expose two different deployments: - **paddle-full:** the supported full pipeline behind a DocStack lane facade; - **paddle-vlm:** an optional task-level recognition service used only through a Paddle-aware crop adapter. ### 6.6 The legitimate crop-mode extension The official pipeline supports operation without layout detection when a prompt label is supplied. The inspected implementation supports task families including OCR, table, formula, chart, spotting, and seal. This makes a PaddleVlmProvider feasible, but the adapter must own: - the supported task-to-prompt mapping; - image preprocessing and limits; - response grammar; - Paddle OTSL-to-table conversion; - spotting token interpretation; - formula cleanup; - truncation and repetition detection; - model and prompt-profile versions. It is not a generic chat adapter. ### 6.7 Why GLM cannot simply replace PaddleOCR-VL inside the full pipeline The inspected PaddleX implementation is coupled beyond transport: - the full pipeline asserts supported PP-DocLayout model families; - task prompts are fixed to Paddle-recognized labels; - tables use Paddle's OTSL representation and decoder; - spotting and formula tasks have special preprocessing and postprocessing; - repetition and figure token behavior are model-specific; - remote model binding recognizes a PaddleOCR-VL model allowlist. An OpenAI-compatible GLM server can accept an HTTP request, but it does not thereby emit Paddle's expected grammar or satisfy Paddle's model binding. Initial integration must not fork PaddleX to force this substitution. A fork would create a high-maintenance, poorly testable hybrid that is neither the supported Paddle pipeline nor the supported GLM SDK. ### 6.8 Reported benchmark evidence Paddle reports 96.33 percent on OmniDocBench v1.6 for PaddleOCR-VL-1.6 and strong performance on Real5 physical-distortion scenarios, using a 0.9-billion-parameter model. This is relevant evidence that the system is capable and compact. It is not a substitute for a DocStack corpus containing its target languages, prepress PDFs, complex slides, and difficult spreadsheets. ### 6.9 Recommended Paddle role Paddle should be the initial primary lane because: - the current DocStack pipeline already uses PP-DocLayoutV3 and PaddleOCR-VL; - the full stack restores upstream-coherent crop, task, postprocessing, and page-assembly behavior; - the migration can be evaluated against the successful legacy profile; - the lane can be kept independently upgradeable by pinning PaddleOCR and PaddleX together. ### 6.10 Paddle doc2md and native Office conversion The version 3.x documentation also exposes doc2md conversion support for formats such as DOCX, PPTX, and XLSX through native Python libraries including python-docx, python-pptx, and openpyxl. This can provide useful exact-text, formula, cell, and source-structure evidence. It is not the production layout path for DocStack. A native object walk cannot establish the visible semantics of arbitrary slides or sheets and does not reproduce renderer-dependent Word pagination. Use it only as: - an optional NativeEvidenceExtractor; - a diagnostic comparison; - a source of exact strings or coordinates after visual alignment; - a test aid for detecting render omissions. It may never suppress creation or visual parsing of the corresponding RenderBundle. --- ## 7. Full Docling with Granite Docling: actual system boundary ### 7.1 What “full Docling” contributes Docling is more than Granite Docling inference. A full integration contributes: - format and backend selection; - standard and VLM pipeline choices; - page and image handling; - accelerator and runtime configuration; - model-spec registration; - response-format parsers; - DoclingDocument construction; - partial-success and failure status; - serialization and rich provider-native artifacts; - service deployment through Docling Serve or a DocStack-owned facade. DocStack should integrate those capabilities without adopting unrelated RAG, chunking, search, or agent features. ### 7.2 The Docling VLM pipeline The VLM pipeline renders or receives page images, executes a configured model, interprets the response according to its declared format, and assembles a DoclingDocument. The inspected model-spec contract includes: - model repository and revision; - prompt; - response format; - supported inference engines; - engine- and API-specific overrides; - stop strings; - generation options. This is a genuine extension point, but it is a **model profile** extension point, not a guarantee that all models are semantically equivalent. ### 7.3 Granite Docling profile The inspected Granite Docling profile uses: - the instruction “Convert this page to docling.”; - DocTags output; - model-specific stop strings; - an 8,192-token output budget; - a page-image scale factor of 2.0. The 258-million-parameter model card describes full-page conversion and bounding-box-guided inference, with text, layout, table, and formula capabilities encoded through DocTags. English is the primary validated language; Japanese, Arabic, and Chinese are described as experimental. The model is Apache-2.0 licensed. The adapter must preserve required special tokens. Treating the response as ordinary prose or stripping structural tokens would destroy the geometry and document grammar that make the profile useful. ### 7.4 Why DocTags matter DocTags carries typed document elements and spatial structure. That allows Docling to construct a richer document than generic page Markdown. By contrast, Docling's inspected GLM-OCR preset requests full-page Markdown. When generic Markdown is assembled into a DoclingDocument, its elements cannot recover real source geometry; placeholder or zero geometry may be used. The preset is useful as a baseline, but it is not equivalent to Granite Docling plus DocTags and it is not the official GLM SDK. ### 7.5 Docling runtimes Current Docling supports model execution through profiles that can use local engines and API-backed engines, including generic OpenAI-compatible service routes when the model profile supports them. Granite Docling can be served remotely when generation, special-token, stop, and response handling are kept consistent. The safe conclusion is: - transport and compute are replaceable behind a verified model profile; - prompt and response semantics are not automatically replaceable; - runtime parity must be tested before promotion. ### 7.6 Standard PDF pipeline and native backends Docling's StandardPdfPipeline and native-format backends remain useful: - as an independent visual/deterministic comparison profile; - to recover exact embedded text, links, metadata, or structure; - to diagnose a VLM omission; - to generate auxiliary evidence for reconciliation. They must not become a visual bypass. For Office and spreadsheet input, the production lane must still receive DocStack's rendered surfaces. A source-native Docling conversion may run as an evidence sidecar or explicit diagnostic profile. ### 7.7 Recommended Docling role The production Docling lane should: 1. consume the same immutable visual surfaces as Paddle; 2. run Docling's VlmPipeline with the declared Granite Docling profile; 3. retain the complete DoclingDocument and conversion status; 4. normalize it into DocStack's canonical visual model; 5. optionally attach standard-pipeline or native-backend evidence; 6. act initially as a shadow or conditional secondary lane; 7. become eligible for primary routing only after corpus-specific promotion. For image materialization, the cleanest first implementation is to submit each page, slide, sheet page, or tile as an image document and assemble results using immutable surface identifiers. A later custom Docling input backend can eliminate per-surface overhead while still using exactly the same pixels. Passing the original Office file directly to Docling would violate the common-render requirement unless its output is explicitly classified as auxiliary evidence. ### 7.8 Docling capability-adoption matrix | Docling capability | DocStack treatment | |---|---| | VlmPipeline with Granite Docling | production visual lane | | Granite DocTags response parsing | production structure and geometry | | DoclingDocument | retained lane-native artifact | | serializers and conversion status | production artifact and failure handling | | accelerator and remote-runtime profiles | production after parity qualification | | StandardPdfPipeline | optional visual comparison or deterministic recovery profile | | native DOCX/PPTX/XLSX backends | auxiliary evidence and diagnostics | | picture or chart enrichment | optional only when output is labeled and source-grounded | | chunking, embeddings, RAG, search, and agents | outside DocStack product scope | --- ## 8. GLM-OCR: compatible role without false equivalence ### 8.1 Official capabilities The inspected GLM-OCR model supports explicit tasks for: - text recognition; - formula recognition; - table recognition. The official repository also provides a complete document SDK: 1. page loading; 2. PP-DocLayoutV3 layout detection; 3. parallel region inference with GLM-OCR; 4. result formatting; 5. Markdown and JSON output. This means GLM is not “only a model.” It has a full SDK. The architectural question is whether DocStack benefits from maintaining three complete, overlapping page pipelines. ### 8.2 Recommended primary role: targeted visual recognition GLM should initially operate through VisualRecognitionProvider for: - a low-confidence text crop; - a disputed number or identifier; - an invalid or incomplete table region; - a formula region; - a region missed by one full lane but detected by visual coverage analysis; - adjudication between Paddle and Docling candidates. This role preserves GLM's strengths without rerunning its PP-DocLayoutV3 path on every page. ### 8.3 Optional role: full SDK shadow lane The complete GLM SDK should remain available as a versioned experimental profile for: - benchmark comparison; - failure analysis; - documents where both primary lanes fail; - future promotion if evidence justifies a third full lane. It must produce its own engine-native artifact and pass the same normalization and quality contracts. It should not be presented as equivalent to Paddle full or Docling full merely because all can emit Markdown. ### 8.4 Docling's GLM preset is a separate integration Docling includes a GLM-OCR model profile using the full-page prompt “Text Recognition:” and Markdown response. That profile exercises Docling's VLM pipeline, not GLM's official PP-DocLayout SDK. It lacks Granite's DocTags geometry and may be useful for controlled experiments, but it should not be confused with: - GLM as a crop recognizer; or - GLM's full document SDK. ### 8.5 Information extraction GLM examples may support direct key information extraction, but DocStack should not create a second extraction architecture around it. All document lanes first produce grounded document evidence. The single product extraction path then uses the selected template, an LLM, schema validation, business rules, and grounding. --- ## 9. Comparative system anatomy | Capability | Full Paddle lane | Full Docling/Granite lane | GLM crop provider | Full GLM SDK | |---|---|---|---|---| | Primary input | page/document images | page images or supported documents | one image/crop | page images/PDF | | Layout ownership | PP-DocLayoutV3 | VLM/DocTags and Docling assembly | none | PP-DocLayoutV3 | | Task routing | Paddle pipeline | model profile/pipeline | DocStack adapter | GLM SDK | | Main response grammar | Paddle task formats, including OTSL | DocTags | GLM task-specific text/table/formula | GLM SDK results | | Geometry | rich page/block geometry | rich with DocTags | request crop geometry | region geometry | | Multipage assembly | Paddle result/restructure | DoclingDocument/DocStack assembly | none | SDK formatter | | Best initial role | primary full lane | shadow or secondary full lane | targeted rescue | experimental shadow | | Safe generic model swap | no | only through verified model spec | provider registry | no | | Native artifact retained | yes | yes | raw and normalized response | yes | The two full lanes deliberately overlap. Their independent failure modes are valuable. The provider plane deliberately does not reproduce a full layout pipeline. --- ## 10. Target architecture ### 10.1 High-level topology SourceArtifact | v RendererAdapter registry | v Immutable RenderBundle / Document Canvas | +---------------------+----------------------+ | | | v v v Full Paddle lane Full Docling lane Native evidence PP-DocLayoutV3 Granite + DocTags extractors PaddleOCR-VL-1.6 DoclingDocument auxiliary only | | | +----------+----------+----------------------+ | v provider-native artifact store | v CanonicalVisualDocument + EvidenceLedger | visual quality assessment | +----------+------------------+ | | localized uncertainty systemic failure | | v v GLM crop provider secondary full lane | | +-------------+---------------+ | v alignment and reconciliation | +---------+---------+ | | v v faithful Markdown template-driven LLM | schema-grounded JSON ### 10.2 Architectural invariants 1. Every output token can be traced to a surface and an observation. 2. Every surface has an immutable transform to source visual coordinates. 3. Full lanes remain upstream-coherent and independently upgradeable. 4. Crop providers never impersonate full document pipelines. 5. Native evidence never silently overwrites visual evidence. 6. Engine-native outputs are retained before normalization. 7. Routing and model revisions are fixed in an immutable execution plan. 8. Unresolved material disagreement is reviewable, not averaged away. 9. Canonical Markdown is transcription, not summary. 10. Structured JSON is produced only after OCR and is governed by a template schema. 11. Model semantics are pinned independently from the provider, deployment alias, endpoint, and credential mechanism that supply them. 12. Provider configuration is a runtime control plane with platform defaults, tenant overrides, validation-before-apply, revision fencing, audit, and secret-safe execution lineage. ### 10.3 Provider-neutral model infrastructure control plane The platform may require specific load-bearing model families without requiring a specific hosting vendor. DocStack therefore separates two kinds of identity: - **semantic model contract**, owned by code and the immutable execution profile: PaddleOCR-VL-1.6 for the Paddle full lane, Granite Docling with its DocTags prompt/stop/codec contract for the Docling full lane, GLM-OCR with task-specific prompts/codecs for recognition, and the selected extraction-LLM contract; - **provider binding**, owned by configuration: protocol adapter, exact API base, provider deployment or routing alias, approved credential-header name and encrypted value, concurrency, timeout, monotonic configuration revision, and tenant/platform source. Four logical endpoint roles are first-class: `paddleocr_vl`, `granite_docling_vl`, `glmocr_vl`, and `llm`. A platform administrator defines the defaults. When enabled by tenant policy, a tenant administrator may define a role-specific override; resolution is tenant override first and platform default second. `local://baked` is an explicit Granite/Docling binding, not a hidden special case. Remote bindings are protocol descriptions such as `vllm`, `ollama`, or `openai`; vendor names do not appear in the execution contracts. Both consoles provide two distinct operations: 1. **Test draft** validates the unsaved form through the production validation path and does not mutate active state. 2. **Apply** uses compare-and-swap against the displayed revision, repeats validation as a hard gate, commits and audits the new revision, then publishes eager cache invalidation. “Hot reload” has a precise consistency contract. An Apply does not mutate an in-flight plan. Every new execution resolves all four effective bindings using a fresh database-backed cache check, snapshots them once, and places only sanitized binding IDs, revisions, public fingerprints, aliases, and endpoint metadata in the immutable `ExecutionPlan`. Plaintext credentials exist only in the in-memory worker-to-isolated-service request. Workers rebuild role-specific clients when fingerprints change, even if Redis Pub/Sub invalidation was missed; Redis remains the low-latency signal and the database/revision check is the correctness fence. Role-specific adapter allowlists preserve flexibility without false equivalence. For example, a generic Gemini or Anthropic text adapter cannot impersonate the Paddle full-lane contract, and a transport-compatible endpoint cannot substitute for Granite unless the Docling prompt, response grammar, and model-revision checks also match. Provider portability changes who serves the model; it does not weaken what model behavior the lane requires. Credential headers are configurable (`Authorization`, Azure-style `api-key`, or another syntactically valid non-hop-by-hop name) and are never returned by APIs, persisted in execution plans, echoed by visual services, or included in fingerprints. Platform and tenant remote destinations undergo DNS/IP/CIDR SSRF validation before Test or Apply and are re-resolved on runtime cache retrieval. Runtime egress remains default-deny and must be explicitly enabled for approved provider CIDRs and ports. Self-hosted endpoints, Azure AI Foundry deployments, and other compatible services consequently share the same binding plane and lifecycle. --- ## 11. Document Canvas and rendering contracts ### 11.1 SourceArtifact A source artifact contains: - immutable original bytes or object-store reference; - content hash; - declared and detected MIME type; - original filename; - source and tenant identifiers; - upload and security metadata; - optional page or range selection. The original is never modified in place. ### 11.2 RenderBundle RenderBundle is the durable output of rendering. It contains: - original artifact identity and hash; - ordered logical surfaces; - a render manifest; - renderer warnings; - optional NativeEvidenceBundle; - relationships among overview, detail, notes, and tiled surfaces. ### 11.3 VisualSurface Each surface represents a human-visible plane: - PDF page; - slide; - speaker-notes page; - worksheet print page; - worksheet overview; - worksheet detail tile; - raster image or frame; - vector viewport. A surface records: - stable surface ID; - kind and sequence; - pixel dimensions; - physical or logical dimensions where known; - color profile and alpha handling; - image object reference or a deterministic lazy-render recipe; - source-to-surface and surface-to-source transforms; - crop, overlap, and parent relationships; - sheet cell-range mapping where applicable; - renderer and scale; - visibility status; - warnings and exclusions. ### 11.4 RenderManifest The manifest must include: - renderer name, exact version, container digest, and command/options; - source hash and generated artifact hashes; - DPI or scale and image encoding; - font inventory and substitutions when available; - locale, timezone, paper size, and application settings; - conversion timestamp; - macro and external-resource policy; - page, sheet, or slide selections; - hidden-content policy; - detected omissions, conversion warnings, and timeouts. The manifest makes a result reproducible and explains why an apparently correct source may render differently in another environment. ### 11.5 NativeEvidenceBundle Native evidence may contain: - PDF characters, glyph boxes, tags, links, and form metadata; - DOCX paragraphs, styles, fields, links, tables, and object anchors; - PPTX shape IDs, text runs, z-order, hyperlinks, chart data, and notes; - XLSX formulas, cached values, displayed formats, merges, styles, defined print areas, drawings, charts, hidden states, and cell coordinates. Every item has: - provenance; - visibility status; - a confidence or proof state for its mapping to a visual region; - raw value and optionally displayed value; - an explicit rule controlling whether it may corroborate an OCR candidate. Native evidence is never concatenated blindly into Markdown. --- ## 12. Format-specific visual rendering ### 12.1 PDF For every selected PDF page: 1. generate a moderate-resolution overview; 2. run layout and coverage analysis; 3. materialize a recognition image within a pixel budget; 4. rerender small, dense, or uncertain regions at a higher scale; 5. preserve one coordinate plane so all resolutions map back to the page; 6. retain native glyphs and tags as auxiliary evidence. The present 150-DPI default can be the first overview for ordinary pages, but it cannot be the only recognition resolution. Resolution should be driven by estimated character height, line density, table density, and provider input limits. ### 12.2 Raster and photographed pages Keep both: - the original image; and - the selected derivative after orientation, deskew, dewarp, denoise, contrast, or crop operations. Every operation must add an invertible transform where possible. A provider result expressed in derivative coordinates must map back to the original image. ### 12.3 Vector images Render the explicit viewport at adaptive scale and store its viewBox transform. Preserve the original vector asset. Do not assume XML element order is semantic reading order. ### 12.4 DOCX Use the selected Gotenberg/LibreOffice print conversion to PDF, then treat the generated pages like PDF surfaces. Additionally: - extract native data only as sidecar evidence; - record missing fonts and field-update behavior; - exclude hidden or revision content unless a profile explicitly requests it; - preserve headers, footers, footnotes, floating objects, and page boundaries visually; - record external links or content that was not fetched. ### 12.5 PPTX Create one primary surface per visible slide at the slide's own aspect ratio. Do not place notes in a margin around the slide. Speaker notes, if requested, become separate surfaces: - linked to their slide; - clearly labeled as notes; - ordered by policy after the slide or in a sidecar section; - never allowed to change slide coordinates. Also: - preserve hidden-slide policy explicitly; - detect font substitutions; - preserve visible overlap and z-order in pixels; - store native shape IDs and anchors only as evidence; - declare animation states, video, audio, and interactive transitions out of visual-static scope unless a future renderer captures them. ### 12.6 XLSX: dual visual representation XLSX needs a stronger rendering contract than a single PDF conversion. #### A. Author print view Render the author's print areas, page breaks, scaling, margins, repeated rows or columns, orientation, headers, and footers through the selected Office renderer line. These pages show the intended printed presentation. #### B. Coverage view For each visible worksheet: 1. determine the visually occupied extent using both native anchors and a rendered inspection; 2. create a downscaled overview of the whole occupied canvas; 3. generate overlapping detail tiles at a readable text scale; 4. map every tile to worksheet coordinates and neighboring tiles; 5. include visible cells, shapes, charts, images, and objects outside the print area; 6. deduplicate observations using surface relationships and cell or object anchors. This covers sheets where: - several tables share one sheet; - blank rows or columns are the only separators; - multirow headers are positioned above values; - print scaling makes text unreadable; - content exists outside the formal print area; - drawings or charts communicate the central information. #### C. Formula and displayed-value policy The visible, formatted value is the transcription target. Formula text, cached value, format code, and calculation state are retained as auxiliary evidence. If a workbook is stale or cannot be recalculated safely: - do not invent a fresh result; - record the visible rendered value; - flag a formula/cache discrepancy; - optionally expose the formula in the evidence sidecar or a template field when requested. #### D. Boundary DocStack does not promise workbook recalculation parity, pivot refresh, macro execution, external-data refresh, or semantic reconstruction of arbitrary spreadsheet models. It promises visual OCR of the rendered workbook with explicit limitations and supplemental cell evidence. --- ## 13. Engine contracts The contracts below are conceptual public interfaces. Exact class names may change during implementation, but the separation must remain. ### 13.1 RendererAdapter render(source, render_profile) -> RenderBundle Responsibilities: - validate supported input; - isolate unsafe conversion; - create deterministic visual surfaces; - record transforms and provenance; - return warnings rather than hiding fidelity risks. ### 13.2 VisualDocumentLane parse(VisualParseRequest) -> LaneResult VisualParseRequest contains: - RenderBundle reference and selected surfaces; - lane deployment and model revisions; - language or script hints; - fidelity options; - page and token limits; - timeout and retry policy; - output and artifact requirements. LaneResult contains: - status: success, partial, failed, or rejected; - complete provider-native result; - normalized observations; - per-surface and per-block geometry; - warnings, truncation, repetition, and omissions; - timings and resource use; - exact engine, model, prompt, runtime, and configuration versions. Implementations: - PaddleFullLane; - DoclingGraniteLane; - optional GlmFullSdkLane; - LegacyDualVlmLane during migration. ### 13.3 VisualRecognitionProvider recognize(RecognitionRequest) -> RecognitionResult Supported task vocabulary: - TEXT; - TABLE; - FORMULA; - CHART; - SEAL; - SPOTTING; - PAGE_TO_MARKDOWN; - PAGE_TO_DOCTAGS. Not every provider advertises every task. A request includes: - exact source surface or crop; - crop box and coordinate transform; - task; - language/script hints; - prompt-profile revision; - pixel and output-token limits; - required response grammar; - fidelity intent. A result includes: - raw response; - normalized candidate; - response codec; - stop reason; - truncation and repetition flags; - provider confidence if meaningful; - provenance and latency; - validation errors. Initial providers: - GlmOcrProvider for text, formula, and table; - PaddleVlmProvider using the official no-layout/prompt-label mode; - future providers only after a conformance suite. ### 13.4 Provider adapters own semantics Each provider adapter owns: - image preparation; - task and prompt mapping; - special-token policy; - output parser and grammar; - table and formula normalization; - retry-safe request identity; - validation and failure interpretation; - model-specific limits. The orchestration layer owns neither hidden model prompts nor undocumented parsing tricks. ### 13.5 NativeEvidenceExtractor extract(source, evidence_profile) -> NativeEvidenceBundle This contract is optional and format-specific. Its output can trigger a visual review or resolve exact character ambiguity only when a geometry or identity mapping is defensible. --- ## 14. Provider-neutral canonical model ### 14.1 Why DocStack needs its own small model Paddle results and DoclingDocument represent different provider assumptions. Choosing either as the universal schema would: - leak one provider's release cadence into every consumer; - make another provider lossy by construction; - tempt the system to coerce unsupported semantics; - complicate rollback and side-by-side evaluation. DocStack should own a minimal **CanonicalVisualDocument** containing only what its product outputs and quality logic require. ### 14.2 CanonicalVisualDocument The model contains: - document identity and execution-plan ID; - surfaces and source transforms; - ordered spatial blocks; - block type and semantic label; - visible text and spans; - tables with grid, row, column, and span structure; - formulas with source image and normalized representation; - figures, charts, captions, and asset references; - headers, footers, notes, and marginalia; - cross-surface links and cross-page table links; - selected observation ID for every canonical value; - uncertainty and review state. It deliberately does not attempt to be a universal knowledge graph or editable Office object model. ### 14.3 EvidenceLedger Every engine observation is retained separately: - engine and adapter version; - surface and geometry; - raw response and parsed value; - task and prompt profile; - confidence or validation status; - candidate relationships; - selection, rejection, or unresolved reason; - native corroboration; - crop and rendered-image hashes. The canonical document points to selected evidence. It does not erase alternatives. ### 14.4 Provider-native artifacts The object store retains: - raw Paddle page results and restructured results; - DoclingDocument and serialized DocTags/JSON; - raw GLM responses and normalized crop results; - legacy output during migration; - all surface images or reproducible render recipes; - rendered PDFs from Office conversion; - quality and reconciliation reports. This is critical for future upgrades: a new normalizer can be tested against historical provider output without rerunning expensive inference, while a new model can be evaluated against identical stored pixels. ### 14.5 Coordinate contract All geometry is represented in: 1. a provider-local plane; 2. a surface pixel plane; 3. the stable source visual plane. Adapters must supply transforms among them. Geometry is never discarded merely because Markdown cannot express it. --- ## 15. Workflow orchestration ### 15.1 Durable state machine RECEIVED -> RENDERING -> SURFACE_VALIDATION -> PLANNED -> PRIMARY_LANE -> NORMALIZATION -> VISUAL_QUALITY -> OPTIONAL_RESCUE_OR_SECONDARY -> RECONCILIATION -> CANONICALIZATION -> MARKDOWN -> OPTIONAL_STRUCTURED_EXTRACTION -> VALIDATION_AND_JUDGE -> COMPLETED | NEEDS_REVIEW | FAILED Every transition is durable and idempotent. Retries reuse immutable input hashes and do not silently change model or prompt versions. ### 15.2 Immutable ExecutionPlan The plan records before model execution: - render profile and version; - selected surfaces; - primary lane; - conditions for secondary lane; - crop providers and allowed tasks; - model and deployment revisions; - resolved platform/tenant provider-binding IDs, aliases, configuration revisions, and secret-free fingerprints; - quality thresholds; - reconciliation policy; - Markdown policy; - extraction template revision; - resource and latency budget. Operational failover may select another healthy deployment of the same declared engine contract. It may not silently change model families. ### 15.3 Execution modes | Mode | Behavior | Initial use | |---|---|---| | paddle-only | full Paddle lane | standard production candidate | | docling-only | full Docling/Granite lane | explicit profile and evaluation | | paddle-primary cascade | Paddle, then GLM crops or Docling when quality fails | recommended production | | docling-primary cascade | Docling, then GLM crops or Paddle | eligible after validation | | parallel high-assurance | Paddle and Docling on all selected surfaces, then reconcile | high-value documents | | shadow | one lane serves output; another is measured but cannot affect it | rollout and upgrades | | legacy-dual-vlm-v1 | frozen current custom pipeline | regression baseline | | glm-full-shadow | complete GLM SDK, no production authority | research and failure analysis | ### 15.4 Initial routing policy - Default documents: Paddle primary. - Paddle output with localized uncertainty: GLM targeted crops. - Paddle output with systemic layout, reading-order, or coverage failure: full Docling secondary. - High-value or regulated profiles: Paddle and Docling in parallel. - Granite language outside the validated corpus: prefer Paddle primary and treat Docling confidence conservatively. - Explicit operator selection: Paddle-only or Docling-only. - No healthy visual surface: render failure, not an OCR-model fallback. ### 15.5 Source class is telemetry, not a bypass The present text-only, hybrid, and scanned classification can remain useful for: - expected difficulty; - render scale; - OCR resource planning; - benchmark stratification; - quality thresholds. It must not decide that a born-digital document can bypass visual parsing. ### 15.6 Queueing and scheduling The orchestrator should schedule work at surface or bounded surface-group granularity while retaining document-level assembly: - primary lane has a document-wide execution identity; - pages can execute in bounded parallelism; - spreadsheet overview must precede tile planning; - GPU admission uses predicted pixels and output tokens; - secondary work is created only after quality assessment; - retries preserve the same crop and image hashes. --- ## 16. Visual quality assessment and escalation ### 16.1 Quality is not one model confidence VLM confidence is often absent, uncalibrated, or incomparable across providers. DocStack needs observable, provider-neutral checks. ### 16.2 Surface-coverage checks - page or surface unexpectedly empty despite visible ink; - unparsed connected components or high-entropy regions; - layout boxes leaving material visual regions uncovered; - cropped text touching a region boundary; - detected image, chart, or table without a canonical representation; - header, footer, footnote, or page number suppressed by a provider default; - worksheet tile or print page not represented; - duplicate coverage from overlapping tiles not reconciled. ### 16.3 Recognition checks - truncation or repetition stop; - invalid Unicode or broken script sequence; - improbable character substitutions in numeric identifiers; - formula syntax failure; - table grammar or cell-count failure; - malformed OTSL, DocTags, HTML, or Markdown; - output text unsupported by visible evidence; - significant native-text mismatch in a geometrically aligned region. Native mismatch is a trigger for inspection, not permission to replace the visual output automatically. ### 16.4 Structure checks - overlapping blocks with incompatible reading order; - missing captions or orphan figures; - table rows with implausible alignment; - multirow spreadsheet header disconnected from its values; - cross-page table merge without compatible columns; - repeated or skipped surface sequence; - slide element order inconsistent with spatial grouping; - clipped worksheet regions. ### 16.5 Escalation policy Use the least expensive action that addresses the observed failure: 1. rerender the same region at a better scale if resolution is inadequate; 2. retry the same provider only for transient or bounded decoding failure; 3. invoke GLM for a localized text, table, or formula uncertainty; 4. run the secondary full lane for systemic layout, coverage, or reading-order failure; 5. route to review when material disagreement remains. Do not invoke all models on all pages by default. Doing so increases cost, latency, and reconciliation risk without guaranteeing honesty. ### 16.6 Partial success Docling can report partial success, and VLM output can truncate while appearing fluent. Paddle and GLM can also return partially usable regions. Lane adapters must map provider states into a common status without converting partial output into unconditional success. --- ## 17. Reconciliation ### 17.1 Reconciliation is evidence selection Never merge documents by concatenating or unioning strings. The process is: 1. align surfaces by immutable ID; 2. align blocks by geometry, type, and visual overlap; 3. compare normalized content; 4. validate candidates against the source crop; 5. select, retain alternatives, or mark unresolved; 6. regenerate canonical reading order only after block selection. ### 17.2 Selection rules - Prefer a candidate that covers visible content without unsupported additions. - Prefer a valid structured table over a flattened text approximation when the grid is visible. - Prefer exact native spelling only when aligned geometry proves it corresponds to the same visible text. - Prefer a high-resolution crop result for a small region over a downscaled full-page guess. - Do not use majority vote when two models share the same upstream layout detector or training lineage. - Do not allow a fluent secondary lane to erase a visible element retained by the primary. ### 17.3 Tables Table reconciliation is cell- and span-aware: - align table regions; - infer grid correspondence; - compare cell text and row/column spans; - preserve empty visible cells; - validate totals and repeated headers without “correcting” source values; - retain the table image for audit. ### 17.4 Reading order Reading order should be represented as a graph or ordered edges over visual blocks, not as provider string order alone. Disagreement is resolved using: - geometry; - columns and grouping; - headings and captions; - connectors and alignment; - provider evidence; - format-specific surface relationships. ### 17.5 Unresolved material disagreement If candidates disagree on a material number, formula, table structure, or reading order and the source image does not support a confident selection: - preserve both observations; - place an uncertainty marker in the sidecar; - mark the job needs_review when policy requires; - never choose merely to make the output look complete. --- ## 18. Faithful Markdown ### 18.1 Canonical serialization rules Markdown generation operates from CanonicalVisualDocument, not directly from an engine's prose output. It must: - preserve surface boundaries through stable comments or headings where configured; - serialize headings according to supported visual hierarchy; - preserve lists and paragraphs without paraphrase; - serialize tables when their structure is supported; - reference extracted figures and charts as assets; - include formulas in a stable notation when recognized; - keep headers, footers, notes, and marginalia according to explicit policy; - mark illegible or uncertain content; - avoid invented connective prose. ### 18.2 Visible and nonvisible content Default canonical Markdown contains visible content. Examples: - a slide note is included only as a separately labeled notes surface; - hidden spreadsheet cells or slides are excluded unless requested; - a PDF text object fully clipped from view is not emitted merely because it exists natively; - alternative text can be retained as accessibility evidence and included only under an explicit profile; - formula source is not substituted for the displayed cell value. ### 18.3 Figures and charts If chart data cannot be recovered reliably: - retain the chart crop; - retain visible labels and caption; - do not fabricate a data table; - label any generated visual description as generated metadata, not source transcription. ### 18.4 Markdown limitations Markdown cannot preserve arbitrary geometry, overlapping slide elements, exact table layout, z-order, or all spreadsheet relationships. The Markdown is the human-readable projection. CanonicalVisualDocument, EvidenceLedger, assets, and transforms are the audit representation. --- ## 19. Template-driven structured extraction ### 19.1 Target flow CanonicalVisualDocument | page-tagged faithful Markdown + selected table structures + source evidence references | template resolution | LLM constrained by JSON Schema | deterministic schema validation | business-rule and type validation | grounding against source spans/cells | bounded repair or judge | JSON result or needs_review ### 19.2 No automatic NER stage GLiNER2 provides no required intermediate for this flow. The LLM already receives the template semantics and should extract only requested fields. Removing generic NER: - reduces a large worker dependency and model footprint; - removes an English-primary, schema-independent stage; - simplifies modes and UI; - prevents users from confusing generic entity detection with validated template extraction; - keeps one source of truth for structured output. ### 19.3 Grounding requirements Every extracted field should retain: - source surface; - selected block, span, cell, or table reference; - source text excerpt or normalized cell value; - extraction confidence or validation state; - transformations applied; - judge or repair history. The LLM may normalize a date or amount according to the template, but the raw visible value remains available. ### 19.4 Extraction modes after removal The product needs two primary output modes: - **document:** OCR and faithful Markdown; - **extraction:** document output plus template-based structured JSON. Internal execution profiles select engines and quality policies; they should not be confused with user-facing output modes. --- ## 20. Complete GLiNER2 removal ### 20.1 Behavioral removal - stop all new GLiNER2 inference; - remove the entities output mode from upload, batch, reprocess, API schemas, and frontend controls; - remove generic entity policy and concurrency controls; - ensure extraction never reads GLiNER entities; - retain only document and extraction behavior. ### 20.2 Code and dependency removal The current repository contains direct GLiNER references in: - extraction/entities.py; - extraction/judge.py comments and legacy assumptions; - worker/pipeline_runner.py; - ocr/base.py typed-entity structures and defaults; - ocr/offload.py; - api/config.py; - API models, repositories, observability, job and policy services; - worker and API dependency images; - pyproject.toml and uv.lock; - Helm values and environment examples; - frontend job, upload, batch, entities panel, and type definitions; - tests for entity extraction, processing modes, jobs, policies, and reprocessing; - README, architecture, environment, integration, and intelligent-extraction documentation. An implementation pass must use repository-wide searches for both GLiNER terms and indirect entities-mode references. Several frontend selectors and API branches may refer only to processing mode and will not appear in a search for the model name. ### 20.3 Persistence migration Do not edit historical applied Alembic migrations. Add a new forward migration that: - changes allowed/default output modes according to the final schema strategy; - maps active entities jobs or batches to document mode, or blocks migration until terminal according to deployment policy; - preserves historical row values if required for audit; - removes obsolete policy fields only after application code no longer reads them. Whether the physical processing_mode column remains is a separate schema decision. It may be renamed to output_mode, constrained to document/extraction, or replaced by an output request field. Avoid reusing old values with new meanings. ### 20.4 Backward compatibility Historical results can contain TypedEntity or canonical.entities. Read-only deserialization should remain for an announced compatibility window so old jobs can still be displayed or exported. Compatibility does not mean: - accepting new entities-mode submissions; - loading GLiNER; - writing new generic entities; - keeping the entities UI as an active feature. The optional LLM characterizer may use terms such as entity aliases as contextual hints. Those are LLM prompt metadata, not automatic NER, and should be renamed if the terminology creates ambiguity. ### 20.5 Verification - clean installation contains no gliner2 package; - worker image does not download GLiNER weights; - no runtime import or configuration variable remains; - API rejects the retired entities mode with a migration-oriented message; - legacy entity-bearing jobs remain readable; - document and extraction modes pass end-to-end tests; - memory, image size, startup time, and dependency-license reports improve as expected. --- ## 21. Deployment topology ### 21.1 Services | Service | Responsibility | |---|---| | DocStack API | ingestion, job/profile selection, status, results | | orchestrator/worker | state machine, quality, routing, reconciliation, Markdown, extraction | | render service | selected Gotenberg/LibreOffice and PDF/image renderers | | paddle-full | supported PaddleOCR/PaddleX full pipeline | | paddle-vlm | optional Paddle task-model service | | docling-full | Docling VlmPipeline, Granite profile, DoclingDocument | | glm-recognition | GLM text/table/formula crop tasks | | glm-full-shadow | optional complete GLM SDK | | object store | originals, surfaces, rendered PDFs, native and canonical artifacts | | database | plans, attempts, manifests, evidence indexes, results | | queue | durable stage and surface work | ### 21.2 Dependency isolation Paddle, Docling, and GLM should not share one monolithic Python/CUDA image. Separate services provide: - independent dependency and CUDA lifecycles; - smaller failure blast radius; - independent horizontal scaling; - controlled model upgrades; - reproducible builds; - simpler rollback; - clear license and vulnerability inventories. ### 21.3 GPU scheduling - assign per-deployment GPU requirements; - cap concurrent pixel and token budgets; - use full-page batching only where the upstream engine supports it; - reserve GLM capacity for targeted rescue; - avoid loading all models in every generic worker; - support CPU or platform-specific Granite profiles only after performance validation; - expose queue time separately from inference time. ### 21.4 Service contracts Internal service requests include: - execution-plan and attempt IDs; - idempotency key; - surface object references and hashes; - model/profile revision; - an ephemeral provider binding whose required semantic model ID/revision must match the service's declared contract; - deadline and resource limits; - callback or result location; - tenant/security context. Services return structured status and artifact references rather than only Markdown. The provider credential is an execution secret, not a domain attribute. It may cross the authenticated worker-to-visual-service boundary only in the ephemeral request envelope; the service removes it before constructing provider-neutral lane/recognition requests and never returns it in status, artifacts, diagnostics, or errors. ### 21.5 Security - treat every Office and PDF source as untrusted; - disable macros; - deny uncontrolled external-resource fetching; - sandbox renderers; - authenticate internal model services; - validate platform and tenant provider hosts with DNS/IP/CIDR SSRF policy before probing, applying, and runtime use; - default-deny visual-service egress, then allow only explicitly configured provider CIDRs and ports (or a policy-enforcing internal gateway); - reject redirects and forbidden credential-header names where the direct protocol adapter controls transport; - encrypt and ACL all source and crop artifacts; - avoid document content in general logs and metrics labels; - enforce page, pixel, token, time, and archive-expansion limits; - pin images by digest and scan dependencies; - define retention and deletion across all derived artifacts. --- ## 22. Versioning and upgradeability ### 22.1 EngineDeployment An engine deployment record contains: - adapter kind and version; - upstream package and commit/tag; - model repository and immutable revision; - runtime and container digest; - accelerator class; - prompt and codec profiles; - supported tasks, formats, scripts, and limits; - health and promotion state. The engine deployment describes semantic and runtime capability; it is not the mutable provider account. Each execution plan joins it to a resolved provider-binding revision. This keeps Azure AI Foundry, a private vLLM/Ollama service, and a baked local model interchangeable at the infrastructure layer while retaining exact lane semantics. ### 22.2 ExecutionProfile An execution profile contains: - renderer profile; - primary and secondary lanes; - recognition-provider rules; - quality thresholds; - reconciliation policy; - Markdown policy; - latency/cost class; - permitted language and format strata. Profiles are immutable after use. A change creates a new version. ### 22.3 Upgrade process For every engine or model update: 1. read the upstream release notes and source-level compatibility changes; 2. build a new immutable deployment; 3. run adapter conformance tests; 4. replay stored visual surfaces; 5. compare native and normalized artifacts; 6. run the stratified golden corpus; 7. shadow production traffic; 8. examine regressions, disagreement, latency, and resource use; 9. promote by format/language/profile rather than globally; 10. retain immediate rollback. ### 22.4 Adapter conformance Every recognition adapter must prove: - task capability declaration; - deterministic request construction; - correct special-token handling; - known good and known malformed response parsing; - truncation and repetition handling; - table and formula validation; - geometry transform preservation; - stable provenance; - idempotent retry behavior. - provider-binding schema validation, secret non-disclosure, supported credential headers, and exact semantic-model contract enforcement. Every full lane must additionally prove: - complete-surface coverage; - page/surface order; - partial and failure mapping; - lossless provider-native artifact retention; - no hidden provider defaults that suppress required visible content. - local and remote provider conformance for every advertised protocol adapter. --- ## 23. Observability and auditability ### 23.1 Metrics - render time, pages, pixels, and warnings by format/profile; - primary and secondary lane latency; - crop-rescue count and area; - engine failure and partial-success rate; - coverage, truncation, repetition, and invalid-structure signals; - disagreement by field and element type; - review rate; - output omission and hallucination rate on audited samples; - GPU/CPU memory, utilization, queue time, and cost; - extraction schema, grounding, and judge outcomes. Do not place document content, extracted values, filenames, or tenant-sensitive text in metric labels. ### 23.2 Trace A job trace should answer: - exactly which bytes were received; - how each surface was rendered; - which model saw which image or crop; - what raw result it returned; - what normalizer changed; - why a secondary action was invoked; - why one candidate was selected; - how each Markdown span and JSON field is grounded. ### 23.3 Operator diagnostics The review interface should be able to overlay: - visual surface; - Paddle blocks; - Docling blocks; - GLM crop and response; - native evidence; - canonical blocks and reading order; - selected and rejected candidates. This is far more actionable than displaying three Markdown strings. --- ## 24. Scientific evaluation framework ### 24.1 Research questions 1. Does full Paddle improve or preserve DocStack's current accuracy and visual coverage? 2. Does full Docling/Granite add independent value on layouts where Paddle fails? 3. Does targeted GLM rescue improve local accuracy without increasing hallucination? 4. Which routing policy provides the best quality/latency trade-off? 5. Does the render-first multi-surface strategy materially improve PPTX and XLSX fidelity? 6. Can either new lane replace the legacy pipeline without regressing the 62-document baseline? ### 24.2 Corpus strata The evaluation set must include: - born-digital PDF with simple, multicolumn, prepress, tagged, untagged, form, and overlaid content; - scanned and photographed PDF with skew, warping, blur, illumination, compression, handwriting, and stamps; - DOCX with sections, columns, floating objects, tables, headers, footers, fields, and missing-font cases; - PPTX with overlapping shapes, arbitrary text boxes, charts, connectors, SmartArt, WordArt, rotation, speaker notes, hidden slides, and nonstandard aspect ratios; - XLSX with multiple visually separated tables, blank-column separators, merged/multirow headers, formulas and cached values, conditional formatting, charts, drawings, print areas, content outside print areas, page breaks, scaled sheets, and multiple visible sheets; - raster and vector images; - the product's target language and script mix, especially Arabic and mixed-direction layouts; - long documents and cross-page tables; - adversarial blank, clipped, corrupted, and resource-limited cases. ### 24.3 Ground truth Ground truth must include more than plain text: - rendered reference surfaces and fidelity notes; - text spans and exact values; - layout boxes and labels; - reading-order edges; - table grids and spans; - formulas; - figure/caption associations; - visible versus nonvisible classification; - Markdown reference with accepted alternatives; - structured-template field truth and source grounding. For Office files, capture a reference rendering from the authoritative authoring application where licensing and infrastructure permit. Compare it with the production LibreOffice rendering before scoring OCR, otherwise renderer error is incorrectly attributed to the OCR engine. ### 24.4 Metrics #### Rendering - perceptual image difference against reference rendering; - page/slide/sheet extent and count; - font substitution and missing-resource rate; - clipped or omitted object rate; - print-area and coverage-view completeness. #### Text and honesty - character and word error rate by language/script; - exact match for identifiers, dates, amounts, and critical fields; - omission rate; - unsupported or hallucinated token rate; - illegible-region detection precision and recall. #### Layout and reading order - block detection mAP or IoU; - label accuracy; - reading-order edge accuracy or normalized sequence distance; - figure-caption and header/footer association; - coverage of visible ink or annotated elements. #### Tables and formulas - GriTS for table topology and content; - TEDS where HTML comparison is appropriate; - cell text accuracy; - row/column span accuracy; - formula exact or normalized-tree match; - cross-page table merge accuracy. #### Product outputs - Markdown structural accuracy and completeness; - asset retention; - schema-valid JSON rate; - field precision, recall, exact match, and normalization accuracy; - grounding accuracy; - review, false-accept, and false-review rates. #### Operations - end-to-end and per-stage latency; - pages or surfaces per second; - CPU/GPU RAM and utilization; - tokens and pixels processed; - retry/failure rate; - cost per page and per completed job. ### 24.5 Systems to compare - legacy-dual-vlm-v1; - full PaddleOCR-VL-1.6; - full Docling VLM with Granite Docling; - optional Docling StandardPdfPipeline visual profile; - Paddle plus targeted GLM rescue; - Paddle-to-Docling conditional cascade; - Docling-to-Paddle conditional cascade; - Paddle and Docling parallel reconciliation; - optional full GLM SDK shadow. ### 24.6 Experimental protocol - predeclare primary metrics and noninferiority margins; - separate development, validation, and held-out test sets; - stratify by format, layout, language, and degradation; - use identical RenderBundle surfaces for engine comparisons; - calculate paired differences and bootstrap confidence intervals; - report mean and tail latency; - inspect all hallucinations and material numeric disagreements; - keep model, renderer, prompt, and codec revisions fixed; - do not tune thresholds on the held-out set; - record failure and partial-success cases rather than dropping them. ### 24.7 Initial promotion gates A replacement primary lane should: - meet predeclared noninferiority on the 62-document exact-field and extraction composite baseline; - reduce no critical-field metric beyond its margin; - not increase unsupported-content or omission rate; - pass PDF, PPTX, XLSX, and target-language minimums; - preserve page and surface provenance; - stay within operational budgets; - support immediate profile rollback. Parallel or cascade mode must demonstrate that its quality gain justifies its additional latency, GPU use, and reconciliation complexity. --- ## 25. Migration plan ### Phase 0 — Freeze the successful baseline - name and pin legacy-dual-vlm-v1; - freeze the 62-document set and expand it with visual-layout cases; - store current raw and canonical artifacts; - add omission, hallucination, layout, and reading-order metrics; - record current latency and resource use. **Exit:** the current system is reproducibly measurable. ### Phase 1 — Introduce Document Canvas without changing OCR behavior - add SourceArtifact, RenderBundle, VisualSurface, and RenderManifest; - make current Office/PDF/image rendering produce these records; - persist transform and artifact hashes; - wrap the legacy pipeline as a VisualDocumentLane; - add immutable ExecutionPlan and attempt records. **Exit:** existing output is unchanged within deterministic tolerance, and every result has visual provenance. ### Phase 2 — Harden visual rendering - pin Gotenberg/LibreOffice image digest and fonts; - validate the current notes field names against the selected image line; - make slide notes separate surfaces; - add PDF adaptive resolution; - add spreadsheet author print pages, overview, and tiles; - retain native evidence as a sidecar; - retire or permanently disable the PDF text bypass in fidelity profiles; - add renderer fidelity warnings. **Exit:** the same surfaces can be replayed identically across lanes. ### Phase 3 — Add canonical and evidence contracts - implement CanonicalVisualDocument and EvidenceLedger; - normalize legacy results; - store engine-native output separately; - implement coordinate and reading-order contracts; - generate existing Markdown from the canonical model. **Exit:** changing a lane no longer changes the public result contract implicitly. ### Phase 4 — Remove GLiNER2 as an independent workstream - disable new entity inference and submissions; - remove dependency, configuration, UI, policies, and runtime stage; - add a forward database migration; - retain read compatibility for historical entities; - consolidate document and extraction output modes; - update tests and documentation. This work can proceed alongside lane integration after the canonical contract exists. It should be deployed separately from a primary-engine switch to keep rollback and diagnosis clear. **Exit:** no new job loads or writes GLiNER output, while historical results remain readable. ### Phase 5 — Integrate full Paddle - deploy the selected PaddleOCR/PaddleX minor lines behind PaddleFullLane; - create a DocStack fidelity configuration that retains visible page furniture; - store raw page and optional restructured results; - normalize Paddle geometry, blocks, tables, formulas, and assets; - run in shadow mode against legacy; - promote by corpus stratum after gates pass. **Exit:** full Paddle can be selected as a production primary without changing public output contracts. ### Phase 6 — Add the visual recognition plane - implement task/capability manifests; - implement GlmOcrProvider first; - preserve current GLM endpoint behavior through the new adapter; - add targeted rerender and rescue logic; - implement PaddleVlmProvider only through official no-layout mode; - add codec and conformance fixtures. **Exit:** a localized uncertainty can be resolved without rerunning or modifying a full lane. ### Phase 7 — Integrate full Docling/Granite - deploy the selected Docling line and Granite Docling model channel; - use VlmPipeline and DocTags; - process immutable DocStack surfaces; - retain DoclingDocument and conversion status; - normalize into CanonicalVisualDocument; - run as shadow, then conditional secondary; - evaluate language and runtime profiles independently; - add optional standard/native evidence profiles without visual bypass. **Exit:** Docling supplies an independent, full visual interpretation and can serve selected primary profiles. ### Phase 8 — Quality-aware coexistence - activate visual coverage and structure checks; - enable localized GLM rescue; - enable systemic Paddle-to-Docling cascade; - implement geometry-aware reconciliation; - add high-assurance parallel profile and review overlays; - tune thresholds only on development/validation data. **Exit:** routing decisions are measurable, explainable, and reversible. ### Phase 9 — Controlled retirement - retire legacy primary status only after all mandatory gates pass; - keep a rollback window and historical artifact reader; - remove legacy-only dependencies after rollback expiry; - promote future engines through the same contracts and evaluation process. --- ## 26. Suggested implementation structure docstack/ rendering/ contracts.py registry.py pdf.py office.py image.py spreadsheet_canvas.py manifests.py lanes/ base.py legacy.py paddle_full.py docling_granite.py glm_full_shadow.py recognition/ base.py registry.py paddle_vlm.py glm_ocr.py codecs/ paddle_otsl.py paddle_spotting.py glm_table.py formula.py canonical/ visual_document.py evidence.py geometry.py normalize_paddle.py normalize_docling.py normalize_glm.py quality/ coverage.py recognition.py structure.py escalation.py reconciliation/ align.py reading_order.py tables.py select.py serialization/ markdown.py assets.py extraction/ templates.py llm.py grounding.py validation.py judge.py orchestration/ plans.py state_machine.py scheduler.py attempts.py This is a target boundary map, not a request for a disruptive repository rename. Introduce it incrementally around existing modules. --- ## 27. Major risks and mitigations | Risk | Consequence | Mitigation | |---|---|---| | Renderer differs from Microsoft Office | OCR is faithful to the wrong visual artifact | selected renderer/font lines, resolved-artifact provenance, reference-render tests, explicit warnings | | Excel surface explosion | excessive pages and cost | occupied-range limits, overview-driven tiles, overlap deduplication, budgets | | Full lanes suppress visible furniture | dishonest Markdown omissions | fidelity profiles, coverage checks, raw artifact retention | | Provider output appears fluent when truncated | silent data loss | stop/repetition validation, ink coverage, partial status | | Cross-engine geometry differs | bad reconciliation | stable surface plane and mandatory transforms | | DocTags or OTSL changes upstream | parser breakage | selected codec lines, fixtures, conformance tests, resolved-artifact provenance | | Arbitrary VLM substituted by URL | malformed output and hallucination | capability/codec registry; transport never implies semantic support | | Two lanes double latency | poor throughput | conditional cascade; shadow sampling; high-assurance opt-in | | Shared CUDA environment conflicts | unstable deployment | separate service images and upgrade cycles | | Native evidence overrides pixels | invisible or stale data emitted | evidence policy, alignment proof, audit ledger | | GLiNER removal breaks old jobs | historical data unreadable | legacy deserializer and forward-only migration | | Markdown hides geometry limits | users overtrust linear output | canonical/evidence sidecar and explicit uncertainty | | Benchmarks do not represent production | misleading promotion | DocStack stratified corpus and held-out gates | --- ## 28. Decisions and rejected alternatives ### 28.1 Accepted - Visual presentation is the source of truth for all supported formats. - Two independent full lanes: Paddle and Docling/Granite. - GLM as the initial targeted recognition provider. - Full GLM SDK retained as an optional shadow candidate. - DocStack-owned minimal canonical visual model and evidence ledger. - Native evidence as an auxiliary channel. - Conditional cascade as the standard quality/cost strategy. - Configuration-driven provider bindings with platform defaults, tenant overrides, unsaved-draft testing, monotonic hot-reload revisions, and immutable per-job snapshots. - Model-family contracts declared independently from Azure AI Foundry, self-hosted, or other compatible API infrastructure. - One LLM/template structured-extraction path. - Complete GLiNER2 retirement. ### 28.2 Rejected #### Native-first routing for born-digital files Rejected because internal order and visual intent diverge too frequently, especially in PDF, PPTX, and XLSX. #### DoclingDocument as the universal cross-engine schema Rejected because it would couple DocStack's public semantics to one lane and force lossy mappings from other engines. #### A single generic “OCR provider” interface Rejected because a full document pipeline and a region VLM have materially different responsibilities and failure modes. #### Inject GLM into Paddle's internal VLM slot Rejected initially because model allowlists, prompts, task routing, OTSL, spotting, formula handling, and postprocessing are coupled to PaddleOCR-VL. #### Treat any OpenAI-compatible endpoint as a Docling-equivalent model Rejected because response grammar, special tokens, geometry, prompt, stop behavior, and model profile determine compatibility. #### Bind a required model family to one infrastructure vendor Rejected because hosting location, provider routing alias, credentials, capacity, and endpoint lifecycle are operational policy. The required semantic contract remains fixed while its provider binding is runtime configuration. #### Run Paddle, Docling, and GLM fully on every page Rejected as the default because cost and reconciliation complexity grow without a demonstrated proportional quality gain. #### Make full Docling's native Office parsing the production truth Rejected because the product doctrine requires common rendered surfaces. Native conversion is retained as evidence. #### Retain automatic generic NER Rejected because template-constrained LLM extraction is the intended structured-output path and GLiNER adds cost and product ambiguity without being required. --- ## 29. Deferred engine candidates HunyuanOCR, Surya, Chandra, Unlimited-OCR, LiteParse, and other emerging systems are **deferred, not rejected**. The present design deliberately avoids adding more production engines before the Paddle, Docling, and GLM boundaries are proven. A future candidate must first be classified as one of: - a complete VisualDocumentLane; - a task-bounded VisualRecognitionProvider; - a renderer or NativeEvidenceExtractor; - an evaluation-only system. It must then declare its task grammar, geometry, languages, license, hardware, limits, runtime, failure states, and model revision and pass the same conformance and held-out evaluation. A GitHub repository, impressive benchmark, or OpenAI-compatible endpoint is not enough to enter production routing. Deferred projects include: - [Tencent HunyuanOCR](https://github.com/Tencent-Hunyuan/HunyuanOCR); - [Surya](https://github.com/datalab-to/surya); - [Chandra](https://github.com/datalab-to/chandra); - [Baidu Unlimited-OCR](https://github.com/baidu/Unlimited-OCR); - [LlamaIndex LiteParse](https://github.com/run-llama/liteparse). This deferral keeps the architecture open without expanding the initial deployment, evaluation matrix, and maintenance burden prematurely. --- ## 30. Primary research sources ### 30.1 PaddleOCR, PaddleX, and PaddleOCR-VL - [PaddleOCR version 3.x documentation tree](https://github.com/PaddlePaddle/PaddleOCR/tree/main/docs/version3.x) - [Pinned PaddleOCR 3.7.0 source](https://github.com/PaddlePaddle/PaddleOCR/tree/b03f46425e8ff4442b268ce449e3eef758146cd4) - [PaddleOCR 3.7.0 release](https://github.com/PaddlePaddle/PaddleOCR/releases/tag/v3.7.0) - [PaddleOCR-VL pipeline usage](https://www.paddleocr.ai/main/en/version3.x/pipeline_usage/PaddleOCR-VL.html) - [PaddleOCR-VL-1.6 algorithm documentation](https://www.paddleocr.ai/main/en/version3.x/algorithm/PaddleOCR-VL/PaddleOCR-VL-1.6.html) - [Document VLM module usage](https://www.paddleocr.ai/main/en/version3.x/module_usage/doc_vlm.html) - [PaddleOCR and PaddleX relationship](https://www.paddleocr.ai/main/en/version3.x/paddleocr_and_paddlex.html) - [Local inference engines](https://www.paddleocr.ai/latest/en/version3.x/inference_deployment/local_inference/inference_engine.html) - [Pinned PaddleOCR dependency declaration](https://github.com/PaddlePaddle/PaddleOCR/blob/b03f46425e8ff4442b268ce449e3eef758146cd4/pyproject.toml) - [Pinned PaddleX 3.7.0 source](https://github.com/PaddlePaddle/PaddleX/tree/e0068ce0bfe75b2992e5b38d06a0393c70f887f7) - [Pinned PaddleOCR-VL pipeline implementation](https://github.com/PaddlePaddle/PaddleX/blob/e0068ce0bfe75b2992e5b38d06a0393c70f887f7/paddlex/inference/pipelines/paddleocr_vl/pipeline.py) - [Pinned PaddleOCR-VL-1.6 pipeline configuration](https://github.com/PaddlePaddle/PaddleX/blob/e0068ce0bfe75b2992e5b38d06a0393c70f887f7/paddlex/configs/pipelines/PaddleOCR-VL-1.6.yaml) - [Pinned document-VLM task constants](https://github.com/PaddlePaddle/PaddleX/blob/e0068ce0bfe75b2992e5b38d06a0393c70f887f7/paddlex/inference/models/doc_vlm/constants.py) - [PaddleOCR-VL remote VLM server configuration](https://www.paddleocr.ai/main/en/version3.x/pipeline_usage/PaddleOCR-VL.html#3-3-2-use-the-general-purpose-server-side-inference-model-deployment-method) - [PaddleOCR-VL technical report](https://arxiv.org/abs/2510.14528) ### 30.2 Docling and Granite Docling - [Pinned Docling source](https://github.com/docling-project/docling/tree/9bdb2af94e93cab47fe96105850438eb663d7953) - [Pinned Docling minimal VLM pipeline example](https://github.com/docling-project/docling/blob/9bdb2af94e93cab47fe96105850438eb663d7953/docs/examples/minimal_vlm_pipeline.py) - [Docling vision-model documentation](https://docling-project.github.io/docling/usage/vision_models/) - [Docling model catalog](https://docling-project.github.io/docling/usage/model_catalog/) - [Pinned Docling API VLM model example](https://github.com/docling-project/docling/blob/9bdb2af94e93cab47fe96105850438eb663d7953/docs/examples/vlm_pipeline_api_model.py) - [Docling API VLM engine example](https://docling-project.github.io/docling/_generated/examples/vlm_pipeline_api_model/) - [DoclingDocument concepts](https://docling-project.github.io/docling/concepts/docling_document/) - [Docling serialization](https://docling-project.github.io/docling/concepts/serialization/) - [Pinned Docling model-spec source](https://github.com/docling-project/docling/blob/9bdb2af94e93cab47fe96105850438eb663d7953/docling/datamodel/stage_model_specs.py) - [Pinned Docling VLM pipeline source](https://github.com/docling-project/docling/blob/9bdb2af94e93cab47fe96105850438eb663d7953/docling/pipeline/vlm_pipeline.py) - [Pinned Docling Serve source](https://github.com/docling-project/docling-serve/tree/7f2b890c5538d5ad511dce42a69af5840452b1c4) - [Pinned Granite Docling model card](https://huggingface.co/ibm-granite/granite-docling-258M/tree/982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe) - [Docling technical report](https://arxiv.org/abs/2408.09869) ### 30.3 GLM-OCR - [Pinned GLM-OCR repository](https://github.com/zai-org/GLM-OCR/tree/cef4d0ea120d1741f5cefe8985eee45f6c8eff1d) - [Pinned GLM-OCR model](https://huggingface.co/zai-org/GLM-OCR/tree/ca5d8b3e287e52589e37c28385d9655ee4372f9d) - [GLM-OCR technical report](https://arxiv.org/abs/2601.15496) ### 30.4 Provider infrastructure - [Azure AI Foundry vision chat-completions guidance](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/gpt-with-vision) - [Azure AI Foundry model inference REST API](https://learn.microsoft.com/en-us/rest/api/aifoundry/modelinference/) ### 30.5 Visual-format evidence - [Adobe Acrobat SDK accessibility and reading-order guidance](https://opensource.adobe.com/dc-acrobat-sdk-docs/library/accessibility/index.html) - [Adobe PDF Reference 1.5](https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.5_v6.pdf) - [Microsoft PresentationML document structure](https://learn.microsoft.com/en-us/office/open-xml/presentation/structure-of-a-presentationml-document) - [Microsoft Open XML ShapeTree](https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.presentation.shapetree?view=openxml-3.0.1) - [Microsoft guidance for aligning and arranging visual objects](https://support.microsoft.com/en-us/office/align-or-arrange-objects-bfd91078-2078-4b35-8672-f6270690b3b8) - [Microsoft Open XML spreadsheet Cell](https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.spreadsheet.cell?view=openxml-3.0.1) - [Microsoft formula handling in SpreadsheetML](https://learn.microsoft.com/en-us/office/open-xml/spreadsheet/working-with-formulas) - [Microsoft Open XML Worksheet](https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.spreadsheet.worksheet?view=openxml-3.0.1) - [Microsoft Excel print areas](https://support.microsoft.com/en-us/office/set-or-clear-a-print-area-on-a-worksheet-27048af8-a321-416d-ba1b-e99ae2182a7e) - [Microsoft Excel page breaks](https://support.microsoft.com/en-us/office/insert-move-or-delete-page-breaks-in-a-worksheet-ad3dc726-beec-4a4c-861f-ed640612bdc2) - [Gotenberg LibreOffice conversion](https://gotenberg.dev/docs/convert-with-libreoffice/convert-to-pdf) - [Gotenberg configuration](https://gotenberg.dev/docs/configuration) - [LibreOffice Calc print ranges](https://help.libreoffice.org/latest/en-US/text/scalc/01/05080000.html) ### 30.6 Evaluation - [DocLayNet paper](https://arxiv.org/abs/2206.01062) - [OmniDocBench repository](https://github.com/opendatalab/OmniDocBench) - [Real5 paper](https://arxiv.org/abs/2603.16776) - [GriTS table metric paper](https://arxiv.org/abs/2203.12555) --- ## 31. Current recommended shape - Pixels remain the visual source of truth; native PDF and Office structures are separately retained corroborating evidence. - Paddle Full is the complete quality-oriented document lane. - PP-OCRv6 medium handles localized text recognition. - Granite Vision 4.1 handles targeted tables, charts, captions, and key-value extraction. - GLM-OCR handles formula recognition and declared fallback tasks. - HPD-Parsing is selected only by `hpd-throughput`; it is not an automatic fallback because its throughput advantage comes with lower parsing quality. - Granite Docling is explicit-only pending a corpus result that overturns the poor real-document behavior observed in this project. - Full lanes and recognition providers consume the same immutable Canvas and preserve their native outputs before normalization. - Markdown remains the faithful human-readable projection. Template-driven, schema-checked extraction remains the structured JSON path. - Dependencies select X.Y lines with floating patches. Builds and executions record resolved hashes without using those hashes as dependency selectors. ### 31.1 August 2026 sources - [PaddleOCR 3.7 release](https://github.com/PaddlePaddle/PaddleOCR/releases/tag/v3.7.0) - [PP-OCRv6 recognition documentation](https://www.paddleocr.ai/main/en/version3.x/algorithm/PP-OCRv6/PP-OCRv6.html) - [HPD-Parsing usage](https://github.com/PaddlePaddle/PaddleOCR/blob/main/docs/version3.x/pipeline_usage/HPD-Parsing.en.md) - [HPD-Parsing model card](https://huggingface.co/PaddlePaddle/HPD-Parsing) - [Granite Vision 4.1 model card](https://huggingface.co/ibm-granite/granite-vision-4.1-4b) - [Granite Docling model card](https://huggingface.co/ibm-granite/granite-docling-258M)