AGENTS.md

Agent notes — IsaacTeleop src/core (index)

CRITICAL: The mandatory multi-file AGENTS.md preflight in ../../AGENTS.md applies here too. Before changing anything under src/core/, you must have read this file, the repo root AGENTS.md, and every other AGENTS.md on the directory paths you will touch. Do not skip any of them.

To see all AGENTS.md files in the IsaacTeleop repo, use the find command (or **/AGENTS.md glob) documented in the repo root AGENTS.md—do not rely on a hand-maintained list in this file.

If work under src/core/ went wrong—user correction, pre-commit/CI failure, or repeated same-class mistakes—you must follow the repo root AGENTS.md Mandatory learning loop: distill a short rule and update the nearest relevant AGENTS.md (this file or a package file) or source comments in the same session (including delta vs main scope).

• Async retargeting pacing behavior belongs on the pacing config objects; keep the worker focused on scheduling mechanics and avoid adding concrete pacing-mode or subclass branches there.
• Prefer coarse-grained async boundaries around an existing synchronous step before splitting DeviceIO/source polling away from graph execution; split internals only when a measured correctness or performance need justifies the extra thread-safety surface.
• In pipelined TeleopSession, last_context follows the returned completed frame; reset/control-transition events travel with that frame and must not force exact-current-frame waits. Use sync mode for exact current-frame behavior.
• Keep async retargeting comments short and local to invariants; user-facing pacing tuning guidance belongs in docs rather than long code docstrings.
• When changing TeleopSession retargeting execution defaults, update config, docs, and default-behavior tests together so opt-in vs. default semantics stay aligned.
• Preserve existing TeleopSession lifecycle flag semantics unless changing the public/context-manager contract intentionally; use tests to lock down cleanup details before altering them.
• After Python test or session-manager edits, let ruff format/pre-commit own wrapping and rerun the hook when it modifies files.
• IDeviceIOSource leaves are only discovered when reachable from a declared graph root — the pipeline combiner's outputs, the teleop_control_pipeline, or a subgraph registered in TeleopSessionConfig(sinks=[...]). TeleopSession._discover_sources unions the leaves of all three. An input source whose only purpose is a side effect (e.g. message-channel send) must therefore expose at least one output (a heartbeat boolean is the established pattern) and be reachable from one of those roots; silent no-discovery is the recurring footgun. Output IDeviceIOSink nodes are different: they are registered via TeleopSessionConfig(sinks=[...]) and the session runs and flushes them explicitly each frame, so a sink needs no heartbeat output and no OutputCombiner reachability.
• All C++ diagnostics go through isaaccapture::Logger (log_bridge/cpp/inc/log_bridge/logger.hpp), never std::cout/std::cerr/printf, and the logger name is always isaaccapture.<module>.<ClassName>. The env-var contract and wire format are shared with the Python half, so a change to either must land in both — see log_bridge/AGENTS.md before touching the logging machinery, and the repo root AGENTS.md "Logging" section for the rules that apply to every call site.
• Run clang-format on touched C++ before pushing — CI rejects unformatted C++ and pre-commit does not catch it. See the formatting instructions in the repo root AGENTS.md ("Pre-commit — match CI before you stop") for the exact commands; the source of truth lives there, not here.
• Authored .py under src/python/isaaccapture/ carries no build-system registration. The CONFIGURE_DEPENDS glob in src/python/CMakeLists.txt stages every .py there and [tool.setuptools.packages.find] discovers the packages, so a subpackage is a directory with an __init__.py and nothing else — no packages list, no DEPENDS entry. Targets that produce files in the staging tree (pybind .so/.pyd, isaacteleop_python copies, stage_generated_tracker_exports, viz_py, vendored runtimes, …) must be dependencies of python_package (DEPENDS / add_dependencies(python_package …) in src/core/python/CMakeLists.txt), not the other way around — otherwise Windows ninja races the wheel build and setuptools fails with error: package directory 'isaaccapture\\<name>' does not exist. python_package waits for viz_py when BUILD_VIZ=ON (.pyd link race). Central stubgen in src/core/python/CMakeLists.txt (not a viz_py POST_BUILD) DEPENDS on python_package, so stage_generated_tracker_exports must stay on python_package's DEPENDS — otherwise stubbing isaaccapture.viz._viz (or any module that loads isaaccapture/__init__.py) fails on the missing export. For pip install -e . (scikit-build-core), _generated_tracker_exports.py must also be install(FILES ...) under isaacteleop_wheel — the directory install excludes *.py so authored sources stay redirect-resolved, but that module is not under src/python/.
• Editing a .fbs under schema/fbs/ can make every recording already on disk unreadable. Recorded MCAP embeds the schema it was written under; schema_tests compares the current schemas against schema/golden/ and McapTrackerViewers re-checks at replay time. Read schema/AGENTS.md and schema/README.md before changing one.
• Schema-based tracker sources exist only in a configured build tree. They are generated at CMake configure time from deviceio_trackers/trackers.toml into ${CMAKE_BINARY_DIR}/generated/trackers/, so grepping src/ for a tracker such as Se3Tracker finds the manifest entry and the generator, not a .cpp. Searching, debugging, and clangd all need cmake -B build to have run first (already true for the flatc output). See deviceio_trackers/AGENTS.md and codegen/AGENTS.md before adding or editing a tracker.
• OpenXRSession waits for the headset by default (wait_for_system = true). xrGetSystem answers XR_ERROR_FORM_FACTOR_UNAVAILABLE (-35) until a system exists, and the retry loop is unbounded, holds the GIL for the whole wait (the pybind constructor releases nothing), and defers Ctrl-C until a headset turns up. Anything that constructs a session where no client will ever connect — CI, a headless smoke test — must impose its own timeout; deps/cloudxr/docker-compose.test.yaml wraps the CloudXR GPU tests in one for that reason.
• CloudXR join-main (Orin): after the native service is up, do not start additional Python threads (including a sigwait helper). That re-triggers _PyGILState_NoteThreadState: Couldn't create autoTSSkey mapping. Keep nv_cxr_service_join on the main thread only; rely on the launcher process teardown for shutdown while join blocks.