fs/xipfs: writable execute-in-place file system for downloadable modules by casaroli · Pull Request #19536 · apache/nuttx

casaroli · 2026-07-25T15:08:34Z

Summary

Adds XIPFS, a writable file system that can serve execute-in-place
mappings, so a loadable module can arrive on the board at run time and still
be executed out of flash rather than copied into RAM.

Today ROMFS is the only in-tree file system able to serve the XIP mappings
the NXFLAT loader needs (Documentation/components/nxflat.rst says so
explicitly). A ROMFS image is built on the host and is read-only, so on a
NOMMU target with memory-mapped NOR there is no way to download a module and
run it in place. XIPFS closes that gap; this PR also updates the NXFLAT page,
which no longer has only one answer to point at.

How it works:

Contiguous extents. Each file is one physically contiguous,
erase-block-aligned run, so mmap() resolves to flash_base + offset and
the loader executes the file where it already lies. This needs the MTD
driver underneath to answer BIOC_XIPBASE.
Write once. A file is created, its size declared, written sequentially,
closed, and is immutable until deleted — the whole life cycle of a
downloaded module. That is what licenses reserving the exact extent up
front, which makes intra-file fragmentation impossible by construction.
Random writes, appends and truncation of a written file are refused.
Manual, best-effort, pin-aware defragmentation. Allocation returns
-ENOSPC and never compacts on its own; the caller decides, via
XIPFSIOC_DEFRAG on a descriptor for the mountpoint directory (using the
ioctldir method added in fs/vfs: Add ioctldir for volume ioctls via the mountpoint directory. #19512). A pass is a loop of atomic
single-extent relocations, so every stop point is a consistent layout, and
it reports the largest contiguous run it achieved — which is what tells the
caller whether a retry will fit.
Power-safe metadata. A generation is committed as write the body, then
write the header that carries the sequence number and CRC. That single
header program is the commit point, so a torn write costs the interrupted
operation and nothing else; mount selects the last fully valid generation.
Pinning. A mapping that aliases flash takes a pin on the extent, not on
the descriptor, so N running instances of one module hold N pins and the
extent becomes movable only when the last goes. Defragmentation skips
pinned extents, which is what stops it relocating code that is executing.
Pins are released by munmap() or by task teardown, so a module that
faults without unmapping does not leak one.
Directories are records in the same metadata generation as files, never
objects in the data region. That placement is deliberate: mkdir and
rmdir add or remove a record and commit one generation, exactly as create
and unlink do, so there is no multi-object update to journal and no orphan
to collect at mount. An empty directory therefore exists and survives a
remount, and costs one entry out of the volume's fixed supply and no flash
blocks. Mount rebuilds the tree and verifies it is a tree — unique
identities, unique names per directory, every parent a live directory, and
parents reaching the root — because a cycle on the medium would otherwise
hang a path walk rather than merely answer wrongly.

MAP_XIP_STRICT is added to sys/mman.h: with it a mapping either resolves
in place or fails with -ENXIO, never silently falling back to the
CONFIG_FS_RAMMAP copy. A module loader must not get a RAM copy by accident,
since that defeats the entire point.

The applications that go with this — the xipfs compaction command, the test
suite whose results are quoted below, and the NXFLAT execute-in-place demo —
are in the companion PR:

Companion PR: apache/nuttx-apps#3665

Both are needed together: the test suite and demo in that PR are what exercise
this one, and its Kconfig options depend on CONFIG_FS_XIPFS from here.

Impact

New feature, off by default. CONFIG_FS_XIPFS defaults to n and
depends on MTD and !DISABLE_MOUNTPOINT. Nothing changes for existing
configurations.
Users: a second answer to "how do I execute a module in place", and the
first writable one. The Limitations section of the docs is explicit about
what XIPFS is not: no random writes, no appends, no growth, no rename, and a
file occupies a whole number of erase blocks.
Existing code touched:
- include/sys/mman.h — one new flag, MAP_XIP_STRICT (bit 27).
- include/sys/statfs.h and fs/mount/fs_gettype.c — a magic and a case so
  df names the file system instead of printing "Unrecognized".
- fs/mount/fs_mount.c — registers xipfs among the MTD-backed file
  systems.
- Documentation/components/nxflat.rst — the "ROMFS is the only XIP-capable
  file system" limitation is now two, and says which one is writable.
- The rp23xx board Make.defs files gain MKNXFLAT/LDNXFLAT, which every
  other ARM board with NXFLAT support already names.
Hardware: no change to any driver. The RP2350 flash MTD driver this was
developed against is already upstream (rp23xx: add an MTD driver over the unused QSPI flash #19531).
Build: new files only under fs/xipfs/; the fs CMakeLists.txt picks
the directory up automatically.
Security/compatibility: the on-media format carries a version and mount
refuses anything else with -EFTYPE, which autoformat turns into a
reformat where the mount asked for it.

Testing

Host: macOS 15 (arm64), arm-none-eabi-gcc 14.2.rel1.

Targets:

sim:xipfs — new configuration, XIPFS on rammtd (which answers
BIOC_XIPBASE, so the in-place path is exercised without flash).
pimoroni-pico-2-plus:xipfs — new configuration, XIPFS on the real QSPI
NOR via /dev/rpflash, i.e. genuine erases and programs throughout.

Both were verified with the test suite from the companion PR. Every run below
was bracketed by a uname -a check of the git revision and board name, before
the first command and after the last, so the logs cannot be from a different
build.

`pimoroni-pico-2-plus:xipfs` — full suite, real flash

[identity ok before] rev 0e9e189444, board pimoroni-pico-2-plus
-- basic VFS --                     12 checks
-- write-once enforcement --         5 checks
-- strict mapping semantics --       3 checks
-- XIP mapping --                    9 checks
-- concurrent mappings --            4 checks
-- cross-task pin refcounting --     4 checks
-- pin release on task exit --       3 checks
-- directories --                   28 checks
-- defragmentation --                9 checks
-- power loss during create (clean) --
  PASS  filesystem stays consistent across every injection point (clean)
-- power loss during create (torn) --
  PASS  filesystem stays consistent across every injection point (torn)
-- power loss during unlink --
    swept 4 injection points, covering the whole unlink
-- power loss during mkdir and rmdir --
  PASS  mkdir and rmdir stay consistent across every injection point
-- power loss during defrag --
    swept 110 injection points, covering every operation of the whole pass
==== 90 passed, 0 failed ====
[identity ok after] rev 0e9e189444, board pimoroni-pico-2-plus

The power-loss sections fail the Nth flash write or erase, remount, and assert
the volume is consistent and every committed file byte-for-byte intact — with
the failing operation left torn (half a page programmed, half a sector
erased) as well as cleanly refused, because a torn generation is what forces
the mount-time CRC to do real work.

`sim:xipfs`

==== 90 passed, 0 failed ====

Execute-in-place, end to end on hardware

An NXFLAT module written into the volume at run time and run twice
concurrently:

staged 268 bytes to /mnt/xipfs/xipmod
extent: block 97 x1, size 268, flash addr 0x10161000
[while running] pin count on the shared text = 2
  instance seed 1: text 0x10161070, stack 0x20006980, sum 2080 -- data private and intact
  instance seed 2: text 0x10161070, stack 0x20007588, sum 4160 -- data private and intact

Both instances report the same text address — inside the flash window, equal
to where the file lies on the media — and different stacks.

Defragmentation on real NOR

[3] requesting a 209-block (836 KB) contiguous file
    result: FAILED -ENOSPC (free space is not contiguous)
[4] compacting...
    elapsed          : 1230 ms
    extents relocated: 27
    blocks reclaimed : 27
    largest free run : 913408 bytes (223 blocks)
    stop reason      : 0 (done)
[5] retrying, sized to the reported largest run (223 blocks)
    result: OK -- allocation now fits
[6] verifying every relocated file byte-for-byte
    all survivors intact
[7] remounting to prove it is durable
    all files intact after remount

Directories from the shell

nsh> mkdir /mnt/xipfs/bin
nsh> mkdir /mnt/xipfs/bin/sub
nsh> echo deep > /mnt/xipfs/bin/sub/deep
nsh> mkdir /mnt/xipfs/empty
nsh> ls -lR /mnt/xipfs
/mnt/xipfs:
 dr-xr-xr-x           0 empty/
 dr-xr-xr-x           0 bin/
/mnt/xipfs/empty:
/mnt/xipfs/bin:
 -r--r--r--           6 greet
 dr-xr-xr-x           0 sub/
/mnt/xipfs/bin/sub:
 -r--r--r--           5 deep
nsh> rmdir /mnt/xipfs/bin
nsh: rmdir: rmdir failed: 39          <- ENOTEMPTY
nsh> df -h
  Filesystem      Size      Used  Available Mounted on
  xipfs          1000K        4K       996K /mnt/xipfs

tools/checkpatch.sh -c -u -m -g passes on every commit in the branch.

github-actions · 2026-07-25T15:19:35Z

MemBrowse Memory Report

esp32-devkitc

ROM: .flash.text -112 B (-0.1%, 124,400 B / 4,194,272 B, total: 3% used)
irom0_0_seg: .flash.text -112 B (-0.1%, 88,608 B / 3,342,304 B, total: 3% used)

qemu-armv8a

Code: .text.readline_common -56 B (-0.0%, 318,078 B)

qemu-intel64

Code: .text -18 B (-0.0%, 8,657,992 B)

s698pm-dkit

Code: .text -64 B (-0.0%, 363,168 B)
No memory changes detected for:
arduino-mega2560
hifive1-revb
mirtoo
rx65n-rsk2mb
stm32-nucleo-f103rb

Laczen · 2026-07-26T07:10:10Z

@casaroli this is a very nice addition. Is there some place where the xipfs system is documented more thoroughly. I was thinking that a integration of the metadata into the "exe" data region would be simple and remove limitations that might be introduced by the number of metadata regions.

casaroli · 2026-07-26T08:48:07Z

@casaroli this is a very nice addition. Is there some place where the xipfs system is documented more thoroughly. I was thinking that a integration of the metadata into the "exe" data region would be simple and remove limitations that might be introduced by the number of metadata regions.

thank you for your comment. You mean Documentation/components/filesystem/xipfs.rst is not enough? There is a small section about the on-disk layout and in fact it is very simple. I can add more detailed technical documentation if you think it is necessary.

Regarding the directory metadata location, I considered having it in the data region, however this would make the compacting (defrag) mechanism and power-loss mitigation much more complex, and also less efficient as we would waste entire blocks with metadata. So I preferred to keep this as is.

In my rp2350, this gives me 60 possible directories. It actually depends on the system flash erase block size. Making those larger than the block size.

In real use cases where I have seen people use XIP, they just slot the flash into a few (4 to 16) file slots and do it without a filesystem or directory structure, so supporting directories and file creation is already overkill for these common use cases.

So I tried to keep it as simple as possible, while still usable and efficient.

Let me know if you like me to detail the documentation or investigate/explore other on-disk formats to overcome some of the limitations.

xiaoxiang781216 · 2026-07-26T13:45:32Z

 AFLAGS := $(CFLAGS) -D__ASSEMBLY__

+MKNXFLAT = mknxflat
+LDNXFLAT = ldnxflat


should we move to arch/ common cmake/make file?

I think so, but since this would touch 13 other files, we might want to do this generalization in a separate PR.

I just copied the pattern from the other Make.defs into ours.

xiaoxiang781216 · 2026-07-26T17:30:16Z

+ * statfs reports it as, in f_namelen.
+ */
+
+#define XIPFS_NAME_MAX      31


why not use NAME_MAX directly

because this is not a user-definable policy parameter. it is an format/ABI constant defined by the current version of the on-disk format.

xiaoxiang781216 · 2026-07-26T17:30:49Z

+ * which keeps the part that identifies the file.
+ */
+
+#define XIPFS_PATH_MAX      127


why not use PATH_MAX

this could be set to PATH_MAX as it is not a on-disk-format parameter, however this has nothing to do with nuttx PATH_MAX as XIPFS_PATH_MAX is just part of the ioctl ABI for the LISTPINNED ioctl.

I suggest we keep XIPFS_PATH_MAX and XIPFS_NAME_MAX as constants, as they are now.

xiaoxiang781216 · 2026-07-26T17:35:34Z

+ */
+
+#define XIPFS_FAULT_CLEAN 0
+#define XIPFS_FAULT_TORN  1


move to macro section

xiaoxiang781216 · 2026-07-26T17:37:37Z

+  uint32_t size;
+  uint32_t pincount;
+  uint32_t data_start;    /* First block of the data region                */
+  uint32_t data_nblocks;  /* Length of the data region                    */


keep the */ align each other

xiaoxiang781216 · 2026-07-26T17:44:00Z

+ * straddling entries.
+ */
+
+#define XIPFS_DIRENT_SIZE   64


move macro to Pre-processor section

xiaoxiang781216 · 2026-07-26T17:46:37Z

+  uint32_t size;                      /* File size in bytes; 0 for a dir */
+  uint32_t start_block;               /* Extent block; 0 for a directory */
+  uint32_t nblocks;                   /* Extent length; 0 for a directory */
+  uint32_t flags;                     /* XIPFS_DIRENT_DIR               */


xiaoxiang781216 · 2026-07-26T17:51:34Z

+
+  uint16_t id;              /* Identity, unique, never 0              */
+  uint16_t parent;          /* Containing directory; XIPFS_ROOT_ID    */
+  bool     isdir;           /* A directory record, with no extent     */


move after line 184

xiaoxiang781216 · 2026-07-26T17:55:45Z

+#ifdef CONFIG_FS_XIPFS_FAULT_INJECT
+  int32_t  fault_countdown; /* Negative disables injection */
+  uint8_t  fault_mode;      /* XIPFS_FAULT_CLEAN or XIPFS_FAULT_TORN */
+  bool     media_dead;


move before line 245

ROMFS is the usual way to carry executables on a NOMMU target with memory mapped NOR flash: it can hand out a real flash pointer from mmap(), so the NXFLAT loader maps a module's text in place instead of copying it into RAM. But a ROMFS image is built on the host and is read only, so a module cannot be downloaded onto the board at run time. xipfs is a writable file system with the same in-place property. Each file is stored as one physically contiguous, erase-block aligned extent, so an mmap() of it resolves to flash_base + extent_offset and a loader can execute the file where it already lies. This needs the underlying MTD driver to answer BIOC_XIPBASE; on the RP2350 rp23xx_flash_mtd.c does. Files are write once. A file is created, its size is declared, it is written sequentially, closed, and is thereafter immutable until it is deleted. That is the whole life cycle of a downloaded module, and it is what licenses the design: the exact extent is reserved at create time, so no file ever grows, moves, or fragments internally. Random writes, appends and truncation of a written file are not supported and are refused. The only source of fragmentation is therefore free space holes left by deletes. Allocation fails with -ENOSPC when no single contiguous run is large enough, and never defragments on its own; the caller decides whether to compact and retry, through XIPFSIOC_DEFRAG. Defragmentation is manual, best effort and interruptible: it is a loop of atomic single-extent relocations, each one copy, commit, erase, so every stop point -- a time budget, a pinned extent, an erase error -- leaves a consistent layout that is simply less compact. It reports the largest contiguous run it achieved, which is what tells the caller whether the retry will fit. Metadata is committed power safely. Two metadata block sets are used in ping-pong, each generation carrying a sequence number and a CRC, and every state change is ordered as write the new data, flip the metadata reference, then erase what the old one referenced. Mount scans both sets and selects the last fully valid generation, so a torn write costs the interrupted operation and nothing else. A mapping takes a pin on the extent, and the pin lives on the extent rather than on the file descriptor, so three running instances of one module hold three pins and the extent becomes movable only when the last one goes. Defragmentation skips pinned extents, which is what stops it relocating code that is executing. The pin is released by munmap() or by the task teardown walk, so a task that dies without unmapping does not leak it. Directories are records in that same generation, carrying their own identity and the identity of the directory holding them; the root is implicit and owns identity zero. They are deliberately NOT objects in the data region, which is what keeps the commit story in one piece: mkdir and rmdir add or remove a record and commit one generation, exactly as create and unlink do, so there is never a multi-object update to journal or an orphan to collect at mount. An empty directory therefore exists, survives a remount, and costs one entry out of the volume's fixed supply and no flash blocks at all. A name is one path component; depth comes from the parent, so XIPFS_NAME_MAX bounds a component, which is what statfs reports it as. Mount rebuilds the tree and checks that it is one: identities unique, names unique within a directory, every parent a live directory, and following parents reaching the root -- a cycle on the medium would otherwise hang a path walk rather than merely answering wrongly. '.' and '..' are refused as components, since an entry stored under either could never be reached again. The commands that act on the volume rather than on one file -- XIPFSIOC_DEFRAG and XIPFSIOC_LISTPINNED -- are reached through the ioctldir method, on a descriptor for the mountpoint directory. They are accepted on a descriptor for a file inside the volume too, but that route holds the file open for the duration and an open extent cannot be relocated, so a pass asked for that way is obstructed by the act of asking. mmap() falls back to the generic RAM copy for ordinary readers when the media cannot be addressed directly. A module loader must not silently get a RAM copy, so MAP_XIP_STRICT is added: with it the mapping either resolves in place or fails with -ENXIO, which the caller can turn into defragment and retry. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

A file system that answers statfs with a magic nothing maps to shows up as "Unrecognized" in df. Give xipfs its constant alongside the others in sys/statfs.h and the case in fs_gettype that turns it into a name. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

The flash MTD device answers BIOC_XIPBASE, which is what xipfs needs to serve mappings straight out of the memory mapped QSPI flash. Mount it at /mnt/xipfs when both are configured, formatting on first boot, so a board comes up with somewhere to download and run a module from. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

rammtd answers BIOC_XIPBASE with the base of its RAM buffer, so it is a usable stand-in for memory mapped NOR: extents are directly addressable and the in-place mmap path can be exercised end to end without any flash. Mount xipfs on it when it is the configured file system, and add a configuration that runs the xipfs test suite, fault injection included. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

Give the QEMU boards the same directly addressable media the sim configuration has: rammtd answers BIOC_XIPBASE with the base of its buffer, so xipfs layered on it hands out real pointers and the in-place mmap path can be exercised on an ARM target with no flash present. Registered as /dev/rammtd and mounted at /mnt/xipfs when xipfs is configured. Both mps2-an500 (Cortex-M7, armv7e-m) and mps2-an521 (Cortex-M33, armv8-m) get it, which is what makes the filesystem testable on two different core generations without either one needing flash. Each board also gets a xipfs configuration that runs the test suite, so the bringup above is exercised rather than only compiled. Both run the suite to completion under QEMU 10.1, 90 checks apiece, the power loss sweeps included. The an521 configuration carries CONFIG_CMSDK_UART0_RX_IRQ=48 and _TX_IRQ=49 rather than the reversed pair the an521 nsh configuration uses. That is the SSE-200 order, receive first, and the one consistent both with the overflow interrupt at 63 that nsh already has and with mps2-an500, which puts RX at 16 and TX at 17. With the pair reversed the TX interrupt reaches uart_cmsdk_rx_interrupt, which acknowledges only UART_INTSTATUS_RX, so the board live-locks on its first console write. Correcting nsh is left to a separate change. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

Describe the write-once usage model, the strict in-place mmap and what MAP_XIP_STRICT is for, extent pinning, manual defragmentation and how to read its result, the power-loss ordering, the on-media layout, and the limitations. The NXFLAT page said ROMFS was the only file system able to serve the XIP mappings its loader needs. That is now one of two, so point at both, and at what the writable one adds: a module can arrive at run time instead of being baked into a host-built image. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

A configuration for the Pimoroni Pico Plus 2 that mounts xipfs on the flash MTD and builds everything that exercises it: the test suite with fault injection, the xipfs command, and the NXFLAT execute-in-place demo. Building an NXFLAT module needs mknxflat and ldnxflat, so name them in the rp23xx board Make.defs files, which already carry the rest of the NXFLAT flags but not these. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

Pages for the three applications that come with xipfs: the command that compacts a volume and prints its block map, the test suite and what each of its sections covers, and the demo that downloads an NXFLAT module into a volume and runs two instances of it in place. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

casaroli · 2026-07-26T23:21:37Z

@xiaoxiang781216 i addressed all your formatting suggestions. please check the answer to the design and build system questions. thank you!

casaroli requested review from Donny9, GUIDINGLI, acassis, gustavonihei, hartmannathan, jerpelea, masayuki2009, raiden00pl and xiaoxiang781216 as code owners July 25, 2026 15:08

casaroli mentioned this pull request Jul 25, 2026

xipfs: compaction command, test suite and NXFLAT execute-in-place demo apache/nuttx-apps#3665

Open

github-actions Bot added Area: Documentation Improvements or additions to documentation Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. Board: arm Board: simulator labels Jul 25, 2026

acassis previously approved these changes Jul 25, 2026

View reviewed changes

casaroli dismissed acassis’s stale review via 216111f July 25, 2026 16:40

casaroli force-pushed the fs-xipfs branch from 0e9e189 to 216111f Compare July 25, 2026 16:40

casaroli requested a review from acassis July 25, 2026 16:45

casaroli force-pushed the fs-xipfs branch 2 times, most recently from 71bf982 to 71364ac Compare July 25, 2026 19:10

acassis previously approved these changes Jul 26, 2026

View reviewed changes

casaroli closed this Jul 26, 2026

casaroli reopened this Jul 26, 2026

xiaoxiang781216 reviewed Jul 26, 2026

View reviewed changes

casaroli added 3 commits July 27, 2026 01:18

casaroli added 5 commits July 27, 2026 01:18

casaroli dismissed acassis’s stale review via 8facba6 July 26, 2026 23:20

casaroli force-pushed the fs-xipfs branch from 71364ac to 8facba6 Compare July 26, 2026 23:20

Uh oh!

Conversation

casaroli commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Impact

Testing

pimoroni-pico-2-plus:xipfs — full suite, real flash

sim:xipfs

Execute-in-place, end to end on hardware

Defragmentation on real NOR

Directories from the shell

Uh oh!

github-actions Bot commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

Uh oh!

Laczen commented Jul 26, 2026

Uh oh!

casaroli commented Jul 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

casaroli commented Jul 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

casaroli commented Jul 25, 2026 •

edited

Loading

`pimoroni-pico-2-plus:xipfs` — full suite, real flash

`sim:xipfs`

github-actions Bot commented Jul 25, 2026 •

edited

Loading