Isaac Sim 6.0.1: understanding an apparently under-utilized headless sensor workload

[Z3ZEL]

I'm building a simulation application with Isaac Sim 6.0.1. I have reduced the issue to a small, repeatable sensor workload and would appreciate advice on where to look next.

The scene is deliberately light: 30 prims, a ground environment, and one articulated vehicle. The simulated sensor rig is:

• Two RGB cameras, grouped in one TiledCameraSensor render product, at 640x400 and 30 Hz.
• One RTX lidar at 10 Hz, using a 128-channel / 512-horizontal-sample configuration.
• One IMU at 200 Hz.

The physics rate is 200 Hz to support the IMU and rendering is 30 Hz. The application loop is uncapped and headless. My target is to run faster than real time the current result is normally around 1.3–1.5x RTF.

What seems counterintuitive is that neither aggregate CPU nor GPU utilization appears saturated during the run. I understand that aggregate utilization is not proof of spare capacity: the Kit main loop has ordering constraints, and a single serial wait can determine the frame time. I am trying to determine whether I have reached an expected synchronization boundary for this workload or whether I am overlooking a configuration or architectural change.

Hardware and runtime environment

• Lenovo ThinkPad P14s Gen 6 laptop running Ubuntu 24.04.4 LTS, kernel 6.17.
• Intel Core Ultra 7 265H (16 cores / 16 logical cores), 64 GB RAM.
• NVIDIA RTX PRO 1000 Blackwell Laptop GPU with 8 GB VRAM, NVIDIA driver 580.167.08 / CUDA 13.0.
• The integrated GPU drives the X11 desktop Isaac Sim renders on the NVIDIA GPU with Vulkan in a headless/no-window configuration.
• CPU governor is performance no swap was in use during the benchmark.

Repeatable benchmark results

Each normal run uses 120 warm-up frames followed by 600 measured application frames. The 200 Hz runs advance 18 simulated seconds with 3,600 physics steps. Results below are from the same machine and same scene values labelled live include my normal sensor-consumption path.

Experiment	What changed	App FPS	Frame time	Physics throughput	RTF	What it suggests
lean-base-kit-fixed	100 Hz physics, smaller base Kit configuration	56.94	17.56 ms	113.89 Hz	1.898	Reference for the 100 Hz configuration.
baseline	100 Hz physics, normal base configuration	53.75	18.60 ms	107.50 Hz	1.792	Extra Kit configuration costs about 1 ms/frame, but this is not the main 200 Hz issue.
sensor-profile-200hz	200 Hz physics, instrumented sensor path	52.90	18.90 ms	317.38 Hz	1.587	Near the best 200 Hz result with sensor consumption minimized/instrumented.
sensor-profile-200hz-fixed	Same, corrected benchmark harness	53.22	18.79 ms	319.33 Hz	1.597	Best observed 200 Hz baseline.
sensor-profile-live-200hz	200 Hz with normal live sensor consumption	50.75	19.70 ms	304.53 Hz	1.523	Enabling normal consumers costs about 0.9 ms/frame or 4.6% RTF.
sensor-profile-live-detailed-200hz	Same, with more detailed per-sensor timing	50.72	19.71 ms	304.34 Hz	1.522	Extra lightweight timing did not materially change the result.
live-baseline-200hz	Normal live workload	50.97	19.62 ms	305.84 Hz	1.529	Representative live baseline.
live-tasking-4-200hz	carb.tasking.plugin.threadCount=4, TBB max=4	50.42	19.83 ms	302.60 Hz	1.513	Lower tasking parallelism did not improve the result.
live-tasking-12-200hz	carb.tasking.plugin.threadCount=12, TBB max=12	49.56	20.18 ms	297.49 Hz	1.487	More tasking threads also did not improve the result.
live-physics-4-200hz	persistent.physics.numThreads=4	49.83	20.07 ms	299.30 Hz	1.497	PhysX thread count was not the limiting factor in this scene.
live-physics-12-200hz	persistent.physics.numThreads=12	49.18	20.34 ms	295.15 Hz	1.476	More PhysX threads slightly regressed performance.
live-gather-render-results-off-200hz	gatherRenderResults=false	49.84	20.07 ms	299.07 Hz	1.495	This setting did not help this single tiled-render-product workload.
live-lidar-host-buffer-200hz	Lidar output buffer on host instead of GPU	40.81	24.50 ms	244.87 Hz	1.224	The only large regression: forcing lidar data to host adds about 4.9 ms/frame and costs 20% RTF.
headless-no-ui	Headless/no UI configuration	43.10	23.20 ms	258.60 Hz	1.293	Earlier headless configuration, included as a sanity check.
headless-no-ui-final	Final headless/no UI configuration	45.95	21.76 ms	275.69 Hz	1.378	Improved headless configuration, still below the live baseline above.
cpu-profile-200hz	CPU profiler enabled	27.99	35.72 ms	167.97 Hz	0.840	Profiling overhead is substantial (roughly 40% lower RTF), so I am cautious about interpreting profiler timing in isolation.

There is normal run-to-run variation, particularly in the multi-sample runs (roughly 1–4% RTF standard deviation), so I do not interpret the small differences between the CPU-thread experiments as meaningful. The useful conclusions seem to be:

1. Changing the tasking or PhysX thread limits from 4 to 12 did not improve throughput. That looks more like a latency-bound serial dependency than a CPU-throughput limit.
2. Normal sensor consumption is measurable but modest: comparing the best 200 Hz sensor baseline (1.597 RTF) with the normal live path (1.529 RTF) puts its maximum recoverable benefit around 4–5%.
3. Device-to-host lidar transfer is expensive. Keeping the lidar output buffer on the GPU improves RTF from 1.224 to 1.529. This is the clearest indication that synchronization/readback matters here.
4. Increasing the physics rate from 100 Hz to 200 Hz does have a material cost, but it alone does not explain the apparent low aggregate resource utilization.

Sensor consumption architecture

I author the RTX sensors first, then create the experimental runtime wrappers (TiledCameraSensor, lidar runtime sensor, and IMU sensor). The significant throughput drop happens when the runtime wrapper is created and get_data() begins to be called authoring the sensor alone is much cheaper.

The relevant camera paths appear to reshape GPU data and use Warp copies/kernels. In simplified form, the tiled path performs a Warp copy or a tiled-image reshape kernel:

wp.copy(out, data[:, :, :output_channels])

wp.launch(
    kernel=_wk_reshape_tiled_image,
    dim=(camera_count, height, width),
    inputs=[data.flatten(), height, width, input_channels, output_channels, tiles_per_row, 0],
    outputs=[out],
    device=data.device,
)

while the single-camera path can reshape a view and optionally copy into an output buffer:

data = data.reshape((*resolution, input_channels))
if out is None:
    out = data[:, :, :output_channels]
else:
    wp.copy(out, data[:, :, :output_channels])
return out, info

My Python code is loaded through Kit's interactive Python environment. An async main method builds the scene, subscribes to new-frame and physics-step events, calls get_data() at each sensor's desired rate, and hands data to downstream processing that does not need to remain on the main thread.

I know the render/physics/Kit frame itself cannot safely be moved to a Python subprocess. My narrower question is whether there is a supported way to decouple consumption of already-produced GPU sensor buffers from the Kit main loop without introducing a forced synchronization or host copy. A subprocess is not necessarily the right mechanism it may make the data-transfer problem worse.

Relevant Kit settings

This is the performance-relevant portion of the experience file. I removed unrelated application settings.

[dependencies]
"isaacsim.core.deprecation_manager" = { order = -100 }
"isaacsim.core.experimental.utils" = {}
"isaacsim.core.rendering_manager" = {}
"isaacsim.core.simulation_manager" = {}
"isaacsim.sensors.experimental.physics" = {}
"isaacsim.sensors.experimental.rtx" = {}
"isaacsim.sensors.rtx.nodes" = {}
"isaacsim.storage.native" = {}
"omni.kit.loop-isaac" = {}
"omni.physics.physx" = {}
"omni.physics.stageupdate" = {}
"omni.warp.core" = {}
"omni.hydra.rtx" = {}

[settings]
renderer.asyncInit = true
renderer.gpuEnumeration.glInterop.enabled = false
rtx-transient.dlssg.enabled = false
rtx.hydra.mdlMaterialWarmup = true
rtx.hydra.supportMultiTickRate = true
rtx.post.dlss.execMode = 0
omni.replicator.asyncRendering = false
app.sensors.nv.lidar.outputBufferOnGPU = true
persistent.physics.numThreads = 8

[settings.plugins."carb.tasking.plugin"]
threadCount = 12

[settings.plugins."omni.tbb.globalcontrol"]
maxThreadCount = 12

[settings.app]
gatherRenderResults = true
settings.fabricDefaultStageFrameHistoryCount = 3
settings.persistent = false
useFabricSceneDelegate = true
runLoops.main.rateLimitEnabled = false
runLoops.main.manualModeEnabled = true
player.useFixedTimeStepping = false
window.enabled = false
vulkan = true
asyncRendering = false
asyncRenderingLowLatency = false

[settings.persistent]
omnihydra.parallelHydraSprimSync = true

A few environment details

The log notes that multi-tick mode is enabled without Motion BVH, DLSS is scaling the small tiled render product up to its minimum input size (weird knowing that it has been deactivated in the kit file), the GPU is operating at PCIe x8 despite x16 capability, and IOMMU is enabled. I will test these independently, but they are secondary context rather than the central question.

Questions

1. For a small scene with this sensor workload and hardware, is approximately 1.5x RTF an expected Isaac Sim 6.0.1 performance limit, or does the benchmark point to remaining optimization opportunities?
2. If there is still meaningful headroom, which area is most likely to matter: Kit loop scheduling, RTX sensor/render synchronization, multi-rate configuration, GPU-buffer consumption, or another setting I have not tested?
3. Is there a supported zero-copy or least-synchronizing pattern for consuming CameraSensor/TiledCameraSensor and RTX-lidar data when downstream work can run asynchronously? In particular, can the GPU work returned by get_data() be handed to another process without blocking the Kit frame and sharing data throughout cuda shared memory for instance exempting the main thread from extracting data to cpu?

I would especially value advice from anyone who has profiled a comparable headless multi-rate RTX sensor setup, or guidance on a lower-overhead profiling approach than the CPU profiler used above.

Thank you in advance 🙏

Here is a frame from the profiler:

[PeterL-NV]

Hi, thanks for the post. A few references might be helpful to your work.

For Isaac Sim performance optimizations, please check out the Isaac Sim performance optimization handbook.
For the profiler, please check out the doc at Profiling Performance

[Z3ZEL]

Thanks @PeterL-NV ,

Yes I've gone through those two pages multiple times in order to optimize as much as I can.

How consumption is wired

Sensors are authored first, then runtime wrappers are created exactly once and retained. Capture is driven by two callbacks registered on the Kit main loop:

• RenderingManager.register_callback(RenderingEvent.NEW_FRAME, ..., order=100) drives the two RGB cameras (one shared TiledCameraSensor) and the RTX lidar.
• SimulationManager.register_callback(..., event=SimulationEvent.PHYSICS_POST_STEP, order=100) drives the IMU at 200 Hz.
  Each binding has a rate gate keyed on SimulationManager.get_simulation_time(), so a sensor only calls get_data() when its own period has elapsed, the render callback itself fires at the render rate. I do know that multi-tick rendering is preventing rendering sensors from rendering useless frames but as I'm subscribing to the rendering event I still need to gate this.

The camera path (simplified)

# One TiledCameraSensor shared by the two 640x400 cameras

self._buffers = [wp.empty((n_cams, h, w, 4), dtype=wp.uint8, device="cuda")
                 for _ in range(2)]          # double buffered

def get_image(self, sensor, timestamp_ns):
    if self._last_timestamp_ns != timestamp_ns:
        batch, _ = self._sensor.get_data("rgba", tiled=False,
                                         out=self._buffers[self._next_buffer])
        self._last_timestamp_ns = timestamp_ns
        self._last_batch = batch.numpy()      # <-- single D2H for the whole tile
        self._next_buffer = (self._next_buffer + 1) % len(self._buffers)
    return self._last_batch[self._indices[sensor.sensor_id]]

The tiled product is fetched once per render (camera render tick) and sliced per logical camera, so the device to host is one ~2 MB transfer per frame, not one per camera. But it is a blocking .numpy() executed inline in the NEW_FRAME callback, which I believe makes the double buffering ineffective, the copy is consumed before the next get_data() is ever issued.

The lidar path (simplified)

raw, _ = self._sensor.get_data("generic-model-output")
host_raw = raw.numpy()                        # <-- D2H, on the main thread
gmo = parse_generic_model_output_data(host_raw)
xyz = np.column_stack((gmo.x, gmo.y, gmo.z)).astype(np.float32, copy=False)
valid = np.isfinite(xyz).all(axis=1)
xyz = xyz[valid]

with de-duplication on (gmo.frameId, gmo.timestampNs) so a repeated GMO buffer is dropped rather than re-published. That's why I used app.sensors.nv.lidar.outputBufferOnGPU=true (1.224 → 1.529 RTF) : the buffer staying on device is worth ~4.9 ms/frame even though I then immediately pull it to host anyway.

Downstream

Committed samples go into a per-sensor latest-value stream, subscribers serialize and push them over a socket from the same interpreter's asyncio loop. Downstream consumers do not need main-thread ordering they only need the sample.

Hope it helps to understand better the architecture of my codebase, thank you in advance 🙏

[PeterL-NV]

I am closing this, feel free to contribute/re-open if you have other optimization ideas!