Skip to content

px4_events: add optional PX4 event decoding plugin - #2359

Merged
vooon merged 22 commits into
mavlink:ros2from
AscendEngineering:feat/px4-external-event-decoding
Nov 15, 2026
Merged

vooon merged 22 commits into
mavlink:ros2from
AscendEngineering:feat/px4-external-event-decoding

Conversation

@Sean-Hickey1

@Sean-Hickey1 Sean-Hickey1 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds optional PX4 EVENT decoding through a dedicated mavros_px4_events package and px4_events plugin.

The plugin translates event IDs and arguments into readable messages using firmware event definitions. It supports local definitions, automatic metadata discovery and acquisition over MAVLink FTP, and a persistent metadata cache.

Resolves #1990. This replaces the earlier core-integrated implementation in #2324 with the separate-package structure requested during review.

Depends on #2361. Its FTP fix is now removed, with tests updated accordingly. Please merge #2361 first.

Package structure

  • Provides its own shared library and pluginlib registration.
  • Keeps mavros and mavros_extras independent of the new package, allowing it to be omitted from builds and installations.
  • Leaves sys_status unchanged.
  • Handles EVENT, COMPONENT_METADATA, and COMPONENT_INFORMATION messages in the new plugin.
  • Uses MAVROS command and FTP ROS services instead of implementing another MAVLink FTP client.
  • Moves the decoder, discovery, acquisition, cache, and associated tests into the new package.

Core raw EVENT logging and status_event publication remain unchanged. The new plugin adds decoded logging, so raw and decoded messages can both appear. It does not introduce a replacement event topic. Sys Status will "double log" raw EVENTS which is something that can be revisited in a future PR.

Dependencies

libevents

libevents is pinned as a git submodule under mavros_px4_events/vendor/libevents and built into the plugin.

This removes the previous build-time-download vendor package. CI initializes submodules recursively. The submodule arrangement also follows the release-packaging approach suggested during review.

XZ decompression

Uses system liblzma through liblzma-dev, replacing xz_embedded_vendor.

Compressed metadata input is capped at 1 MiB, and uncompressed JSON at 8 MiB per file. liblzma has a separate 128 MiB decoder memory limit (as it briefly requests a large dictionary that it doesn't fill when interacting with most PX4 systems)

Operation

Automatic mode

Automatic mode is the default because decoding now lives in an optional package.

The plugin:

  1. Checks that the connected autopilot is PX4.
  2. Requests COMPONENT_METADATA, with legacy COMPONENT_INFORMATION fallback.
  3. Acquires and validates general metadata.
  4. Acquires and validates the referenced event definitions.
  5. Enables decoded logging when definitions are ready.

Acquisition uses cmd/command, ftp/open, ftp/read, and ftp/close. The command, ftp, and normal sys_status plugins should remain enabled.

Disconnect cancels automatic acquisition and clears automatically acquired definitions. Reconnection starts discovery again. Stale asynchronous completions are discarded.

Local mode

Local mode loads a supplied JSON or JSON.xz event-definition file without discovery, FTP acquisition, or persistent cache use.

The file is validated, but its correspondence to the connected firmware is the user's responsibility. Local definitions remain loaded across reconnects.

Failure behavior

Metadata discovery, validation, acquisition, or persistence failures leave core raw event logging available.

Unknown events remain visible through the unchanged core path. Unknown enum/bitmask values are retained in decoded output rather than silently discarded.

Parameters

Parameters belong to the plugin node, normally /mavros/px4_events. They are read-only after startup and require a restart to change.

Parameter Default Purpose
event_decoding_mode auto Select auto, local, or disabled.
event_metadata_file "" Firmware-matching local JSON or JSON.xz definitions; valid only in local mode.
event_metadata_download_delay 30.0 Delay FTP downloads after connection, in seconds. Cache lookup can complete before this delay.
event_metadata_cache_enabled true Enable persistent metadata reuse in automatic mode.
event_metadata_cache_dir "" Empty selects $HOME/.cache/mavros/px4_events.
event_metadata_cache_max_bytes 16777216 Best-effort finalized cache budget, in bytes. Nonpositive values disable persistence while allowing automatic decoding.

Cache behavior

  • Stores validated original metadata bytes, keyed by role, advertised URI, and checksum.
  • Reuses complete cache hits without FTP.
  • Downloads only the missing file for partial hits.
  • Rejects corrupt entries and metadata with changed advertised checksums.
  • Uses atomic replacement when saving entries.
  • Applies best-effort eviction based on save time.
  • Keeps validated definitions usable in memory if persistence fails.

Container users can mount the cache directory on persistent storage to retain entries across container replacement.

Compatibility and validation

Metadata discovery supports both current and legacy advertisement messages.

FTP paths preserve advertised absolute paths. Relative paths are tried as advertised, with a single leading-slash retry after ENOENT for compatibility with the older PX4 behavior encountered during vehicle testing.

Metadata validation includes file-size limits, checksums, supported definitions, argument-layout checks, and a floating-point precision limit before rendering.

Automated tests

The package includes focused tests for:

  • Decoder behavior and XZ handling.
  • Metadata parsing and validation.
  • Discovery and fallback behavior.
  • Acquisition, retries, cancellation, and service failures.
  • Complete, partial, corrupt, and changed-checksum cache entries.
  • Persistence failures.
  • Plugin configuration and connection lifecycle.

CI initializes the libevents submodule and includes the new package in build/test coverage. Coverage reporting excludes vendored libevents sources.

Manual testing

I tested this against PX4 sim, as well as a ModalAI starling. Testing on previous implementation was also successful against a ModalAI Seeker.

Testing included cold/warm starts, local file mode, and various attempts to test systems edge cases including disconnecting halfway through discovery, intentionally corrupting a cache, disconnecting/reconnecting both vehicle and mavros, and injected false ftp file names. All tests successful, it failed gracefully (logged and self-disabled) in each situation.

ASan/UBSan testing was also performed during development.

Documentation

Adds package documentation covering installation, plugin loading, configuration, local mode, automatic acquisition, cache behavior, resource limits, and logging.

Extends the existing documentation generator with:

  • A separate docs/plugins/px4_events/ group.
  • Plugin-index and QoS-link support for the new package.
  • Extraction of typed declare_parameter() calls.

The extractor improvement also discovers the existing setpoint_accel.send_force parameter, which was previously absent from generated documentation. Its generated page and standard-plugin index are updated accordingly; this does not change runtime behavior.

@Sean-Hickey1

Copy link
Copy Markdown
Contributor Author

Thanks for your feedback, moving everything over to an dedicated package with its own plugin. I agree on the structure and points.

One detail behind choosing XZ Embedded: the PX4 metadata I tested advertises a 64 MiB dictionary despite decompressing to ~151 KiB. liblzma requests a dictionary sized allocation, so its decoder memory limit needs to allow 64 MiB plus overhead, separately from the output-size cap. Since we test the output anyway, it won't use anywhere near this much permanently , but it does reserve it temporarily. So long as linux overcommit is on (like it is by default), or it's a computer not running at absolute limits it's almost certinatly fine, especially for larger companion computers, but just wanted to highlight.

XZ embedded reuses the output buffer as it's dictionary.

Given that context, would you want to use XZ Embedded, or would you prefer liblzma for reasons listed above? I'm pretty split on it, and happy to follow your recommendation either way.

Linking comment from old PR here.

@vooon vooon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, skeleton looks right.
except, i'd probably place libevents into a vendor subdirectory.

Comment thread mavros_px4_events/package.xml Resolved
@Sean-Hickey1 Sean-Hickey1 changed the title PX4 Events: Skeleton px4_events: add optional PX4 event decoding plugin Nov 4, 2026
@Sean-Hickey1
Sean-Hickey1 marked this pull request as ready for review November 4, 2026 23:32
@Sean-Hickey1
Sean-Hickey1 requested a review from vooon November 4, 2026 23:34

@vooon vooon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should not touch FTP code.

@Sean-Hickey1
Sean-Hickey1 force-pushed the feat/px4-external-event-decoding branch from 62a44ba to 0f9953d Compare November 5, 2026 17:58
@Sean-Hickey1

Copy link
Copy Markdown
Contributor Author

Fixed failing CI/CD case, and removed FTP code. This PR depends on #2361, it will not work without it.

@Sean-Hickey1
Sean-Hickey1 requested a review from vooon November 5, 2026 18:06

@vooon vooon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, a separate package is the right direction. A few things need to change before merge, the rest can be follow-ups.

Must fix

  1. Acquisition does not recover after a dropped link (src/plugins/px4_events/acquisition.cpp:169). On cancellation or an "uncertain" failure (timeout, disconnect) ftp/close is skipped on purpose, but the FTP plugin keeps the path in session_file_map. After reconnect ftp/open on the same path returns EALREADY, which is neither EBUSY nor EIO, so it is not retried. Decoding stays off until mavros restarts or someone calls ftp/reset, and the PX4-side session leaks too. Please close (or reset) on these paths, or handle EALREADY on open.

  2. Configure hard-fails without the submodule (CMakeLists.txt:39). FATAL_ERROR when vendor/libevents is not checked out breaks colcon build for everyone who pulls ros2 without --recurse-submodules, and for release tarballs / git archive exports, which carry an empty directory. Please make this not break the workspace build, and update the build instructions (AGENTS.md, README) for the submodule.

  3. No lint gate (CMakeLists.txt:65). The package does not enable ament_lint_auto like mavros and mavros_extras do, so uncrustify/cpplint never run on it. px4_events.cpp:262 and :301 are already over 100 columns.

  4. Unrelated docs churn (docs/plugins/std/index.json, docs/plugins/index.md). About 110 lines of regenerated output are not related to this PR, and the std index does not match the PR's sources (ftp ~/list at line 308 vs 309 in ftp.cpp). Please rebase on current ros2, regenerate, and keep the regenerated outputs in a separate commit.

Should fix (here or in a follow-up)

  • Events from any component are decoded (px4_events.cpp:307). handle_event uses SystemAndOk, so an EVENT from a camera or companion with the same sysid is decoded with the autopilot's definitions and logged as "FCU: ...". Discovery already uses ComponentAndOk.
  • One bad event definition rejects the whole file (px4_events/decoder.cpp:243). The decoder throws on the first definition it cannot handle (float enum, arguments over 40 bytes, bad precision, duplicate id), so nothing is decoded for that firmware. Skipping that one event with a warning would be more robust. The cache validate() path has the same all-or-nothing behaviour.
  • Only the first decode failure is logged (px4_events.cpp:149). RCLCPP_WARN_ONCE at a single call site is shared by all event ids and all decoders, while the message reads as per-id.
  • Metadata is decoded twice under the mutex (px4_events/cache.cpp:192). Each file is XZ-decoded, JSON-parsed and built into a Decoder in MetadataCache::validate and again in Acquisition::file_finished_locked, with mutex_ held. With up to 8 MiB of JSON this can stall handle_event on the MAVLink thread.

Simplifications

  • px4_events/xz_metadata.cpp:76: about 100 lines of hand-rolled XZ container parsing only to learn the output size. lzma_stream_decoder + lzma_code with avail_out capped at the limit gives the same bound, and accepts valid streams this parser rejects (stream padding, concatenated streams).
  • px4_events/metadata.cpp:106: the bitwise CRC32 can be replaced by liblzma's lzma_crc32, which is already linked: ~lzma_crc32(data, n, 0xFFFFFFFF) gives the MAVLink variant.

@Sean-Hickey1
Sean-Hickey1 force-pushed the feat/px4-external-event-decoding branch from f2e0952 to 1053122 Compare November 12, 2026 22:04
@Sean-Hickey1

Sean-Hickey1 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks for the review, Addressed the review feedback as follows:

Must fix

  • Reconnect/session cleanup: acquisition now attempts bounded cleanup after cancellation and uncertain failures, including late successful opens. Added one delayed retry for EMFILE, reflecting failed FTP close leaving to recieving EMFILE from the Autopilot (PX4 releases FTP session after 10 seconds, so our bounded wait will retry after that period in case earlier close failed due to disconnect). Tested via disconnecting in the middle of Acqusition, and then autopilot reconnecting. Reproduced failure without this change, and then success with this change.
  • Missing submodule: configuration now warns and skips the optional plugin instead of failing the workspace. Updated README and AGENTS.md instructions. Tested builds with and without libevents.
  • Lint gate: enabled ament_lint_auto, corrected reported lint issues, and excluded vendored sources where appropriate. Excluded PEP257 because the package’s only Python files belong to libevents.
  • Generated docs: rebased onto current ros2, regenerated the affected documentation, and kept regenerated output in a separate commit. There is one non-related px4_events change (recognition of parameter send_force), due to an addition to gendoc that finds parameters that were previously missed.

Should fix

  • Component filtering: changed EVENT handling to ComponentAndOk, matching discovery and restricting decoding to the target component.
  • Bad event definitions: unsupported types, float enums, excessive argument sizes, and unsafe precision now skip only the affected event. Duplicate IDs are excluded entirely. File-level failures remain fatal. Updated decoder/cache coverage and manually verified an intentionally invalid definition against SITL while other events continued decoding.
  • Decode warnings: replaced the global once-only warning with reporting once per full event ID per metadata load. Tested against SITL to ensure validity.
  • Mutex/duplicate validation — TBD: unchanged. Planned follow-up to reduce lock scope without compromising cancellation/reconnect ordering, want to take some more time to think about best implementation here.

Simplifications

  • XZ decoding: replaced manual container parsing with bounded lzma_stream_decoder / lzma_code processing, retaining the separate decoder-memory limit. Added coverage for padding, concatenated streams, and output limits.
  • CRC32: replaced the bitwise implementation with liblzma’s CRC32 function while preserving the MAVLink variant and existing reference-vector tests.

@Sean-Hickey1
Sean-Hickey1 requested a review from vooon November 12, 2026 22:04

@vooon vooon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, all four must-fix items from the previous review are resolved at 1053122, along with the component filter, per-event skipping, per-id warnings and both simplifications. This pass covered the previous findings and the main sources; I did not build it locally.

On item 1: sending ftp/close on cancel and on uncertain failures is enough, because the FTP plugin's close_file() erases its session entry even when the reply times out, so the next open does not hit EALREADY.

Non-blocking notes:

  1. gendoc.sh now requires the submodule. It fails unless mavros_px4_events has compile commands, and the package produces none when vendor/libevents is missing. Fine for now, but worth a line in tools/README.md if it is not there already.

  2. Skipped definitions log one WARN each at load. Firmware with many unsupported definitions will produce a burst; a single summary line would be quieter.

  3. A missing message or events key still rejects the whole file. These throw from the JSON accessors in decoder.cpp instead of skipping the one event.

Follow-up, as you planned: metadata is still decoded twice with mutex_ held (MetadataCache::validate and Acquisition::file_finished_locked).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Will mavros support mavlink event interface?

2 participants