feat(vision): NCNN pose models + live keypoint rendering#34
Merged
Conversation
Add support for NCNN-exported YOLO models in the live camera CV path and draw pose keypoints as dots on the preview. - cv_processor: detect NCNN models (dir / .param), read task from the export's metadata.yaml so pose models load as task=pose (Ultralytics otherwise guesses detect and misreads keypoint channels), and skip the torch .to() move for the CPU-only NCNN backend. - cv_processor: normalize Ultralytics keypoints to a flat (K, C) array (fixes a latent shape bug that also affected vision cones) and render them via new _draw_keypoints, gated on keypoint_min_confidence. - CVSettings: add show_keypoints / keypoint_radius / keypoint_color / keypoint_min_confidence (+ serialization). - camera settings dialog: model picker accepts .param; add Keypoints overlay toggle. - docs: NCNN export/selection guide for faster inference on the Pi. - tests: 20 new unit tests (settings round-trip, keypoint drawing across shapes, NCNN path/task resolution, device-move skip).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add support for NCNN-exported YOLO models in the live camera CV path and draw pose keypoints as dots on the preview.