Skip to content

Add relation solver geometry to env runner - #990

Open
qianl-nv wants to merge 5 commits into
mainfrom
qianl/feature/collsion-overlay-environment-runner
Open

Add relation solver geometry to env runner#990
qianl-nv wants to merge 5 commits into
mainfrom
qianl/feature/collsion-overlay-environment-runner

Conversation

@qianl-nv

@qianl-nv qianl-nv commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Visualize relation-placement geometry optionally in environment_runner.py

Detailed description

  • Add an opt-in --show_placement_overlay flag to environment_runner.py.
  • Draw solver AABBs and MESH collision wireframes with entity-specific colors.
  • Render fixed background meshes once with full edges while updating movable geometry from live poses.
  • Add lifecycle, geometry, fallback-pose, and warning-deduplication tests.

Command to run:
python isaaclab_arena/scripts/environment_runner.py --show_placement_overlay --env_graph_spec_yaml isaaclab_arena_environments/kitchen_bench/droid_pick_and_place_lightwheel_kitchen.yaml

image

@qianl-nv
qianl-nv force-pushed the qianl/feature/collsion-overlay-environment-runner branch from 02fb64f to 820b66d Compare August 3, 2026 09:58
@qianl-nv
qianl-nv changed the base branch from main to qianl/feature/replicator-example August 3, 2026 10:00
@qianl-nv
qianl-nv force-pushed the qianl/feature/collsion-overlay-environment-runner branch from 820b66d to 939aafb Compare August 3, 2026 10:34

@qianl-nv qianl-nv left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self review 1

Comment thread isaaclab_arena/utils/placement_debug_overlay.py Outdated
Comment thread isaaclab_arena/scripts/environment_runner.py Outdated
Comment thread isaaclab_arena/scripts/environment_runner.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py
@qianl-nv
qianl-nv force-pushed the qianl/feature/collsion-overlay-environment-runner branch from 939aafb to a8f02f9 Compare August 4, 2026 07:44
@qianl-nv
qianl-nv changed the base branch from qianl/feature/replicator-example to main August 4, 2026 07:44
@qianl-nv
qianl-nv marked this pull request as ready for review August 4, 2026 07:44
@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds an optional placement-geometry overlay to the interactive environment runner and exposes placement-pool geometry needed to construct it.

  • Adds entity-colored solver AABBs and mesh collision wireframes.
  • Keeps fixed background wireframes as persistent USD curves while redrawing movable geometry from live poses.
  • Moves and expands the Isaac Sim debug-draw helpers.
  • Adds overlay lifecycle, geometry, fallback, and warning-deduplication tests.

Confidence Score: 4/5

The live-pose conversion must be fixed before merging because enabling the new overlay can terminate the environment runner on its first scene-backed asset.

The overlay calls PyTorch-only tensor methods directly on Isaac Lab's Warp-backed root pose values, while established consumers explicitly convert those buffers with warp.to_torch first.

Files Needing Attention: isaaclab_arena/visualization/placement_geometry_draw.py

Important Files Changed

Filename Overview
isaaclab_arena/scripts/environment_runner.py Adds the opt-in CLI flag and correctly owns overlay creation, per-step redraw, and cleanup.
isaaclab_arena/visualization/placement_geometry_draw.py Implements placement-pool visualization, but live-pose reads apply PyTorch methods directly to Warp-backed scene data and can terminate the runner.
isaaclab_arena/visualization/isaac_sim_debug_draw.py Relocates the debug-draw wrapper and adds reusable box, quaternion-transform, mesh-edge, and wireframe helpers.
isaaclab_arena/relations/pooled_object_placer.py Adds defensive-copy access to fixed collision objects and exposes the solver's default collision mode.
isaaclab_arena/tests/test_placement_geometry_draw.py Covers geometry and fallback behavior but does not exercise the actual Warp-backed live scene pose buffers.
isaaclab_arena/tests/test_environment_runner.py Covers argument defaults and the overlay lifecycle through a test double.

Sequence Diagram

sequenceDiagram
  participant Runner as environment_runner
  participant Overlay as PlacementGeometryDraw
  participant Scene as Isaac Lab scene
  participant Draw as Debug draw / USD
  Runner->>Overlay: from_env(env)
  Overlay->>Runner: overlay
  Runner->>Overlay: redraw(env)
  Overlay->>Draw: create fixed background curves once
  loop Every simulation step
    Runner->>Scene: env.step(zero_actions)
    Runner->>Overlay: redraw(env)
    Overlay->>Scene: read live root pose
    Overlay->>Draw: clear transient lines
    Overlay->>Draw: draw AABBs and movable meshes
  end
  Runner->>Overlay: close()
  Overlay->>Draw: clear lines and remove static root
Loading

Reviews (1): Last reviewed commit: "Harden placement overlay fallback handli..." | Re-trigger Greptile

Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
Comment thread isaaclab_arena/scripts/environment_runner.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

This PR adds an opt-in --show_placement_overlay flag to environment_runner.py that draws relation-solver AABBs and MESH collision wireframes in Kit, and refactors the debug-draw helper from utils/ into a new visualization/ package. The change is nicely conservative — default-off, best-effort (the visualizer is guarded so it never crashes the runner), and well tested without needing Kit. No correctness or boundary blockers; a few readability/maintenance points below.

Findings

🟡 placement_geometry_draw.py:249_solver_world_aabb re-derives the solver's box-construction math and its docstring admits it mirrors ObjectPlacer._rotate_candidate_bboxes. If the solver changes, the overlay silently diverges and misleads. Worth a single shared helper both call.

🔵 placement_geometry_draw.py:153getattr(asset, "name", kind) on a name-declaring contract; use asset.name.

🔵 placement_geometry_draw.py:279 — visualizer calls the private PlaceableAsset._get_initial_pose_as_pose(); a public accessor would decouple it.

🔵 environment_runner.py:142 — module import deferred into run_environment with no sim-init or cycle reason; hoist or add a # NOTE: if intentional.

Test Coverage

Good. New pure-Python geometry tests (test_placement_geometry_draw.py) cover corners/edges/rotation, mesh decimation, entity classification, the ranged/per-env fallback-pose collapse, world-AABB construction for fixed vs. live placeables, and the warning-dedup path. The runner lifecycle test asserts from_env → print_legend → redraw → redraw → close, and fixtures are kept in sync with the new CLI arg. No sim-dependent imports leak to module top.

Verdict

Ship it (minor cleanups optional).

@qianl-nv qianl-nv left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self review 2

Comment thread isaaclab_arena/scripts/environment_runner.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
Comment thread isaaclab_arena/visualization/placement_geometry_draw.py Outdated
Visualize relation-solver collision geometry in the environment runner
and keep the overlay synchronized with live asset poses. Move debug-draw
helpers into the visualization package and cover overlay lifecycle and
fallback behavior.

Signed-off-by: Qian Lin <qianl@nvidia.com>
Persist static AABBs and collision meshes as USD curves so live redraws
only update movable assets.

Signed-off-by: Qian Lin <qianl@nvidia.com>
Convert Warp-backed scene pose buffers before reading them and simplify
overlay entry metadata and static-asset classification.

Signed-off-by: Qian Lin <qianl@nvidia.com>
Keep lossy pose collapsing private and draw a missing scene-backed asset
only when it has a concrete configured pose.

Signed-off-by: Qian Lin <qianl@nvidia.com>
Signed-off-by: Qian Lin <qianl@nvidia.com>
@qianl-nv
qianl-nv force-pushed the qianl/feature/collsion-overlay-environment-runner branch from a8f02f9 to 063ff13 Compare August 5, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant