Summary
DoclingPdfParser.get_page() parses and materialises every path in a page's content stream before any ContentConfig level or create_words/create_textlines flag can bound the work. On CAD / wiring-schematic pages drawn as hundreds of thousands of vector paths this costs multiple GiB of native memory per page, and there is no configuration that limits it. Downstream this is the OOM class tracked in docling-project/docling#4058.
Measurements
Synthetic path-storm page (dense ruled grid of N line segments, generator in docling#4058), Docling 2.121 / docling-parse 7.14.0, amd64, 4 CPU:
| segments |
peak RSS |
| 100k |
~0.6 GiB |
| 1M |
3.49 GiB (default backend, tables on or off identical) |
1M, all ContentLevel SKIP + compute_shapes=False |
2.75 GiB — the parse itself dominates, not materialisation |
| 1M, pypdfium2 backend |
1.42 GiB, 3× faster |
Real documents (equipment manuals, 337–480 pages, worst pages 466k–970k paths, 12 GiB cgroup, 40-page batches, Docling 2.121/2.122 with docling-parse 7.14.0):
| document |
docling-parse default |
OcrMode.FULL_PAGE (decode skipped since docling#4061) |
pypdfium2 backend |
| 352 pp, worst page 970k paths |
OOMKilled at 12 GiB |
6.55 GiB peak |
6.16 GiB peak |
| 337 pp, worst page 660k paths |
OOMKilled at 12 GiB (2.41) |
6.19 GiB |
6.03 GiB |
| 480 pp, worst page 466k paths |
11.34 GiB peak, no headroom |
7.25 GiB |
4.98 GiB |
Full details and the docling-side discussion: docling-project/docling#4058 (this issue is the docling-parse half that thread left open).
Ask
A per-page budget or degradation threshold inside get_page() — for example a maximum path/segment count (or a byte budget on the decoded content stream) above which the parser stops materialising paths for that page and reports it as degraded, so callers can fall back (OCR-only, or a different backend) without the process being killed first. The docling side is already able to consume a parsed_page=None-style outcome since docling#4061.
Happy to contribute the change if maintainers can point at the preferred place for such a budget in the C++ parser.
Summary
DoclingPdfParser.get_page()parses and materialises every path in a page's content stream before anyContentConfiglevel orcreate_words/create_textlinesflag can bound the work. On CAD / wiring-schematic pages drawn as hundreds of thousands of vector paths this costs multiple GiB of native memory per page, and there is no configuration that limits it. Downstream this is the OOM class tracked in docling-project/docling#4058.Measurements
Synthetic path-storm page (dense ruled grid of N line segments, generator in docling#4058), Docling 2.121 / docling-parse 7.14.0, amd64, 4 CPU:
ContentLevelSKIP +compute_shapes=FalseReal documents (equipment manuals, 337–480 pages, worst pages 466k–970k paths, 12 GiB cgroup, 40-page batches, Docling 2.121/2.122 with docling-parse 7.14.0):
OcrMode.FULL_PAGE(decode skipped since docling#4061)Full details and the docling-side discussion: docling-project/docling#4058 (this issue is the docling-parse half that thread left open).
Ask
A per-page budget or degradation threshold inside
get_page()— for example a maximum path/segment count (or a byte budget on the decoded content stream) above which the parser stops materialising paths for that page and reports it as degraded, so callers can fall back (OCR-only, or a different backend) without the process being killed first. The docling side is already able to consume aparsed_page=None-style outcome since docling#4061.Happy to contribute the change if maintainers can point at the preferred place for such a budget in the C++ parser.