Skip to content

Detection/OCR: annotation-based training with each family's standard loss, and metric-improvement tests #2170

Description

@ooples

Summary

The detection and OCR models train only through Train(Tensor<T> input, Tensor<T> expectedOutput): mean-squared error between the raw network output and a tensor of the same shape. That proves the networks are differentiable end to end (#2152), but it is not how any of these models is trained, and it cannot drive the task metrics. Nothing today can show that training improves mAP, H-mean or CER.

This is the last acceptance item of #2152, split out by decision on #2154.

What is missing

A training entry point that takes annotations, not raw-output tensors, and applies each family's standard loss with its standard target assignment:

family annotation standard loss (reference)
YOLOv8 / v9 / v11 boxes + classes task-aligned assignment, BCE cls + CIoU + DFL (YOLOv8, TOOD)
YOLOv10 boxes + classes consistent dual assignment: one-to-many + one-to-one heads (YOLOv10)
DETR / DINO / RT-DETR boxes + classes Hungarian set loss: CE + L1 + GIoU (DETRSetLoss exists)
Faster / Cascade R-CNN boxes + classes RPN: anchor IoU sampling, BCE + smooth-L1; RoI head: CE + smooth-L1, per cascade stage with rising IoU thresholds
DBNet polygons BCE on shrunk probability map + L1 threshold map + dice on binary map (Liao et al. 2020)
CRAFT character/affinity heatmaps MSE on region and affinity Gaussian maps
EAST quads balanced BCE score + IoU/angle geometry (Zhou et al. 2017)
CRNN text CTC (CTCLoss exists)
TrOCR text cross-entropy with teacher forcing

Acceptance

  • A public training API on the three family bases taking image(s) plus annotations (boxes/classes, polygons, or text), exposed through the facade.
  • Each model uses its paper's loss and target assignment (table above).
  • Family fixtures gain an overfit test: training on a small synthetic set (for example one to four images with known boxes, polygons or strings) raises mAP@0.5 / H-mean, or lowers CER, measured with ObjectDetectionMetrics, TextDetectionMetrics and TextRecognitionMetrics, against the untrained model.

🤖 Generated with Claude Code

https://claude.ai/code/session_016jgqTmscEnkgmAp1TkNFpG

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions