Skip to content

[Bindings] Expose positioned OCR spans in OcrPageResult #513

Description

@yitom486

Use case

Inkdown is an Electron ebook reader. For scanned PDF pages it draws a selectable OCR text layer over the rendered page image and stores highlight/note geometry against that layer. Native pages already expose positioned text through extractTextWithPositions; OCR pages currently return Markdown and provenance only, so the recognized text cannot be placed back onto the page.

Current behavior

processPdfWithOcr already recognizes positioned OCR lines, maps them into PDF coordinates during fusion, and uses them to assemble Markdown. The public OcrPageResult drops those individual recognition results at the binding boundary.

Proposal

Add a backward-compatible spans field to the Node.js and Python OcrPageResult types:

spans: Array<{
  text: string
  x: number
  y: number
  width: number
  height: number
  confidence: number
}>

Each entry represents one accepted OCR recognition line. Its rectangle is axis-aligned, measured in PDF points, and uses the same visible-page coordinate frame as TextItem. This proposal is deliberately line-level; word-level boxes can be considered separately if a future OCR engine exposes reliable word geometry.

Compatibility and behavior

  • spans is additive: existing Markdown, provenance, routing, and OCR model behavior are unchanged.
  • spans is empty when OCR did not run for a page.
  • When OCR did run, spans remain available even if fusion ultimately selects native Markdown for the page.
  • No second OCR pass, model download, or layout-model change is introduced.

Why it matters

Without positioned OCR output, applications that need text selection, page overlays, highlight/note anchors, or confidence visualization must run a second OCR engine only to obtain geometry. Exposing the already-computed spans makes processPdfWithOcr a complete single-pass result for scanned pages.

Acceptance criteria

  • Rust fusion retains accepted OCR spans in FusedPageMarkdown.
  • Node.js and Python expose the fields above on OcrPageResult.
  • The Node/Python API documentation defines the coordinate frame and empty-array behavior.
  • Tests cover geometry passthrough, invalid/empty-span filtering, and binding conversion without requiring OCR model files.

A patch is proposed in #514. I can validate it against a 340-page Chinese scanned-book sample after review.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions