feat(viser): add scan-from-here and saved viewpoints - #3836
Conversation
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## main #3836 +/- ##
==========================================
- Coverage 77.58% 77.57% -0.02%
==========================================
Files 1315 1315
Lines 124697 124929 +232
Branches 10867 10882 +15
==========================================
+ Hits 96748 96915 +167
- Misses 24816 24873 +57
- Partials 3133 3141 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 3 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Greptile SummaryThis change adds scan-from-here controls, planner obstacle refreshes from scan results, and saved camera viewpoints in the Viser panel. A delayed result from a timed-out scan can be applied to a later scan and refresh the planning world using an earlier camera pose. Saved viewpoints are shared across connected clients, allowing one client to overwrite a name and cause another client to restore that camera position. The scan timeout path can also permit the backend operation to continue beyond the panel timeout, potentially refreshing obstacles after the panel enters a failed state and prevents retries. T-Rex validation blockedThe late-completion timing path for the scan timeout mismatch could not be conclusively executed because the available validation tool execution capacity was exhausted after a scheduler-sensitive timing attempt. Category: tool. Missing item: additional validation execution capacity. Confidence Score: 2/5The change should not merge until scan responses are correlated, scan completion is bounded by one operation deadline or cancellation mechanism, and saved viewpoints are isolated per client. Two failures were reproduced with executable harnesses against the reviewed behavior. The remaining timeout lifecycle failure is supported by the independent backend and UI time budgets, but its exact late-refresh timing could not be conclusively executed. Files Needing Attention: dimos/manipulation/manipulation_module.py needs request correlation and a single end-to-end scan deadline; dimos/manipulation/visualization/viser/gui.py needs client-scoped viewpoint state and timeout recovery.
 T-Rex LogsWhat T-Rex did
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| self.scan_requests.publish(normalized) | ||
| try: | ||
| result = self._scan_result_queue.get(timeout=timeout) |
There was a problem hiding this comment.
If a timed-out scan responds after a later scan drains the shared queue, the later call accepts that stale response without checking request identity or timestamp, causing planner obstacles to be refreshed from the earlier camera pose.
Artifacts
Focused executable stale scan response reproduction script
- This script AST-loads and executes the current target method bodies, simulates a timeout and delayed callback, and proves that the later scan accepts the earlier response.
Timed-out first scan followed by delayed response queueing
- The executed before run shows the first scan timing out and its delayed old-pose response subsequently occupying the shared queue.
Later scan accepts stale old-pose response and refreshes obstacles
- The executed after run shows the new-pose request accepting `old-pose-response` without a fresh response and performing one obstacle refresh, proving the bug.
| except queue.Empty as error: | ||
| raise TimeoutError(f"Scan timed out after {timeout:.1f}s") from error | ||
| expected_ids = {str(detection.id) for detection in result.detections if detection.id} | ||
| self._wait_for_scan_objects(expected_ids, time.monotonic() + timeout) |
There was a problem hiding this comment.
When the result consumes most of scan_timeout and object propagation takes over one additional second, the GUI times out while this fresh timeout remains active; its uncancelled operation can later refresh the planning world after the panel reports failure, while the panel remains in FAILED and blocks further scans and plans.
| self._viewpoints[name] = CameraViewpoint( | ||
| position=self._camera_vector(camera.position), | ||
| look_at=self._camera_vector(camera.look_at), | ||
| up_direction=self._camera_vector(camera.up_direction), | ||
| fov=float(camera.fov), | ||
| ) | ||
| choices = self._handles.get("viewpoint_choices") | ||
| self._set_optional_handle_attr(choices, "options", list(self._viewpoints)) | ||
| self._set_optional_handle_attr(choices, "value", name) |
There was a problem hiding this comment.
If multiple clients use this panel, saving writes one client's camera into the shared _viewpoints dictionary and shared dropdown, so another client can overwrite or restore that entry and have its camera jump to a viewpoint it did not save.
Artifacts
Executed two-client viewpoint reproducer source
- This Python harness imports the production GUI class and invokes its save and restore callbacks for two distinct clients using the same viewpoint name, demonstrating the client-crossing path; takeaway.
Executed two-client viewpoint reproducer output
- This command output records a successful execution where client B overwrote `shared-entry` and client A restored B’s camera position with `cross_client_jump_reproduced= True`; takeaway.
github-actions
Bot
added
the
ready-to-merge
a60a617 to
058d935
Compare
github-actions
Bot
removed
the
ready-to-merge
1 participant
Summary
ManipulationOperatorand typed module-to-OSR request/response streamsxarm-room-sim