Modernize camera benchmark for current sensor APIs

[StafaH]

Description

Modernizes benchmark_cameras.py for the current develop APIs and fixes its current camera-output failure.

The original reset failure in #4334 came from the old Fabric cloning path exposing only one camera prim to the sensor. The current clone-plan architecture has resolved that cardinality problem, but the stale benchmark then fails during depth-to-point-cloud conversion because camera outputs and intrinsics are now Warp-first ProxyArray objects. This change uses explicit cached zero-copy .torch views at that Torch math boundary.

The cleanup also:

• replaces the separate standard/tiled paths with the unified Camera API while retaining deprecated CLI aliases;
• migrates launching, task injection, standalone scene creation, Fabric defaults, and ray-caster mounts to current APIs;
• fixes standalone environment cardinality and ray-caster resolution;
• makes warmup and measured step counts exact and synchronizes CUDA around timing;
• avoids retaining frame outputs and reuses the zero-action tensor;
• makes autotune monitoring lifecycle-safe and validates incompatible CLI arguments early;
• updates the camera benchmarking guide and adds simulator-free CLI regression tests.

Fixes #4334

Type of change

• Bug fix (non-breaking change which fixes an issue)
• Documentation update

Release backport

• Backport this pull request to the active release branch after it merges into develop

Screenshots

Not applicable.

Validation

• uv run --frozen --extra test pytest -q scripts/benchmarks/test/test_benchmark_cameras_cli.py (4 passed)
• uv run --frozen isaaclab -f
• uv run --isolated --extra dev -- make -C docs current-docs
• Original legacy command shape with 20 tiled cameras in Isaac-Cartpole on Newton + RTX
• Task benchmark with Camera on PhysX + RTX
• Standalone benchmark with Camera on RTX
• Task and standalone benchmarks with RayCasterCamera

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with uv run isaaclab -f
• I have made corresponding changes to the documentation
• My changes generate no new unexpected warnings
• I have added tests that prove the CLI validation and compatibility behavior
• No changelog fragment is required because no source package is changed
• My name already exists in CONTRIBUTORS.md

[greptile-apps]

Confidence Score: 4/5

The PR should not merge until autotuning records metrics for the same passing camera count it reports to the user.

Findings

1.  Autotune Records Failing Trial ▶

Summary

The PR modernizes the camera benchmark around the unified Camera and RayCasterCamera APIs, updates runtime and scene creation, adds explicit ProxyArray-to-Torch views at the point-cloud boundary, revises autotuning and measurement lifecycle handling, and adds simulator-free CLI regression tests and updated documentation.

• Consolidates legacy standard and tiled camera options behind the current Camera API while retaining deprecated aliases.
• Uses configuration-driven standalone and task scenes with current launcher, Fabric, physics, and sensor APIs.
• Makes warmup and measurement counts explicit, synchronizes CUDA timing, and avoids retaining frame outputs.
• Adds CLI validation tests for compatibility aliases and invalid argument combinations.
• One autotune result-selection issue remains: recorded metrics can belong to the threshold-exceeding trial rather than the reported largest passing count.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  CLI[Parse and validate CLI] --> Select{Camera kind}
  Select -->|RTX| Camera[Create Camera configuration]
  Select -->|Ray caster| Ray[Create RayCasterCamera configuration]
  Camera --> Mode{Task supplied?}
  Ray --> Mode
  Mode -->|No| Scene[Create standalone vectorized scene]
  Mode -->|Yes| Task[Inject sensors into task configuration]
  Scene --> Launch[Launch simulation runtime]
  Task --> Launch
  Launch --> Warmup[Run exact warmup steps]
  Warmup --> Measure[Run measured steps and sample utilization]
  Measure --> Threshold{Autotune and within limits?}
  Threshold -->|Yes| Increase[Increase camera count]
  Increase --> Measure
  Threshold -->|No| Record[Record benchmark results]

Loading

_{Reviews (1) · Last reviewed commit: "Modernize camera benchmark"}

[greptile-apps]

Autotune Records Failing Trial

When a trial exceeds a utilization threshold, final_analysis has already been replaced with that trial's results before the loop exits. The script then reports last_passing_count, but records timing and utilization from the threshold-exceeding count, so the saved benchmark does not represent the advertised autotune result. Preserve the previous passing analysis with its count, or record every trial with matching metadata.

[isaaclab-review-bot]

Isaac Lab Review Bot

The modernization coherently unifies the Camera and RayCasterCamera paths, adopts config-first launching, and handles ProxyArray-to-Torch conversion at the depth-processing boundary. Three compatibility and benchmark-contract issues remain: explicit zero-valued legacy counts are rejected, the default autotune interval invalidates the newly documented two-cameras-per-environment command, and standalone num_objects now applies per environment despite being documented as a scene-wide count.

• Design and architecture: The pre-launch configuration and post-launch instantiation split is coherent, as are the unified CameraSelection flow, per-environment camera cardinality, and ray-caster mounts. However, placing standalone objects below {ENV_REGEX_NS} makes object count scale with camera count, materially changing the benchmark workload unless this per-environment behavior is explicitly intended and documented.
• API: Deprecated aliases do not fully preserve accepted legacy command shapes because an explicit zero count is treated as a selected camera kind. Autotune validation also conflicts with the default interval and the guide's task_num_cameras_per_env=2 example. The num_objects CLI description and guide should state that the value is per environment if that new behavior is intentional.
• Implementation: The exact warmup and measurement loops, CUDA synchronization, NVML cleanup, zero-action reuse, and non-retained camera outputs are well structured. Before merge, adjust legacy zero-count resolution, make autotune interval handling compatible with the documented command, and align the standalone object-count implementation or documentation with the intended workload.

Minor fixes needed. Posted 3 actionable findings inline.

Automated review; human maintainers own approval decisions.

[isaaclab-review-bot]

🟡 Warning · Api — Explicit zero legacy counts now rejected

Legacy counts default to None and any supplied value marks the option as selected, so --num_tiled_cameras 100 --num_standard_cameras 0 now errors as mutually exclusive. The previous script defaulted both to 0 and only conflicted on two positive counts, and its own warning text told users to set the unused counts to 0. Treat non-positive legacy counts as unselected while keeping the negative-value rejection.

[isaaclab-review-bot]

🟡 Warning · Api — Autotune interval default rejects documented example

The default --autotune_camera_count_interval is 25, so this check fails for any even --task_num_cameras_per_env. The autotune example added in the same PR (docs lines 76-78: --task_num_cameras_per_env 2 --autotune) therefore exits with status 2 before launching. Either round the interval up to a multiple of the per-env count, or change the default/documented example so the guide's command runs.

[isaaclab-review-bot]

🔵 Suggestion · Implementation — num_objects is now per environment

The standalone scene uses num_envs=selection.count and spawns objects under {ENV_REGEX_NS}/Objects/Obj_XX, so the total is num_objects * num_cameras. The help text ("Number of objects in the sample scene") and the guide example (--num_cameras 2 --num_objects 100, described as 100 objects) both understate the spawned count, which changes what the benchmark measures. Reword both to state "per environment".