Skip to content

usb: add IsoEndpointIn and IsoInSlot for scheduled isochronous IN - #7559

Merged
leftger merged 1 commit into
embassy-rs:mainfrom
leftger:usb-iso-packet-api
Nov 19, 2026
Merged

leftger merged 1 commit into
embassy-rs:mainfrom
leftger:usb-iso-packet-api

Conversation

@leftger

@leftger leftger commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Rebased onto current main. All backwards-compatibility shims and default fallback trait methods have been removed.

An isochronous IN packet that the host does not poll for in its service interval should be dropped rather than sent late, so the writer can move on to fresh data. EndpointIn::write_transfer is the wrong shape for that: needs_zlp and multi-packet chunking do not apply to isochronous transfers, and it has no way to schedule by service interval or report intervals left empty by a late writer.

API (embassy-usb-driver)

  • IsoEndpointIn: Endpoint (no fallback defaults):
    • type Slot<'a>: IsoInSlot where Self: 'a
    • fn queue_depth(&self) -> u8
    • async fn ready(&mut self) -> Result<Self::Slot<'_>, EndpointDisabledError> — asynchronously reserves a borrowing slot for the next service interval. Cancel-safe: dropping the future or slot leaves nothing armed.
  • IsoInSlot:
    • fn skipped(&self) -> Option<NonZeroU32> — empty service intervals left by the writer before this slot (uncollected packets dropped by the controller are not counted).
    • fn write(self, data: &[u8]) -> Result<(), IsoWriteError> — synchronously queues one packet for the slot, returning TooLate, BufferOverflow, or Disabled.
  • iso::IsoSchedule: helper for drivers that tracks wraps, skipped intervals, and host polling phase lock for interval > 1.

Drivers & Classes

  • embassy-usb-synopsys-otg (STM32 OTG FS/HS, nRF54 USBHS): ready() waits until the previous packet has been sent or dropped, TX FIFO has space, and the target tick is one tick away; IsoSlot::write() arms synchronously. For interval > 1, XFRC locks the host's polling phase and IISOIXFR probes remaining ticks of an unlocked interval with 0-byte ZLPs.
  • embassy-rp (RP2040 / RP2350): uses DEV_SOF + SOF_RD, disarms uncollected packets on expiry at SOF, arms FS ISO IN with DATA0, and locks phase on buff_status.
  • embassy-stm32 (non-OTG USB usb_v1–usb_v4): uses FNR.FN + ISTR.SOF, zeroes both PMA lengths when idle or expired so uncollected packets are dropped and unarmed polls receive a ZLP, and locks phase on CTR_TX.
  • embassy-nrf (USBD EP8): uses FRAMECNTR + EVENTS_SOF with synchronous EasyDMA load in IsoSlot::write(); hardware latches ISOIN at SOF and sends ZeroData ZLPs when unarmed.
  • embassy-usb (UAC): Stream, Feedback, and LineInStream expose ready() and update write_packet() to return Result<Option<NonZeroU32>, IsoWriteError> without mapping TooLate to Disabled. UAC examples updated accordingly.

@embassy-bot

embassy-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown

This pull request is a draft, so it isn't in the review queue yet. Mark it as ready for review when you'd like someone to look at it.

@github-actions github-actions Bot added e-nrf Issues for the nRF family of chips e-stm32 Issues for the STM32 family of chips e-rp Issues for the RP family of chips e-usb labels Nov 17, 2026
@leftger
leftger force-pushed the usb-iso-packet-api branch 2 times, most recently from 7b9b7f0 to 18b1795 Compare November 17, 2026 03:54
@leftger

leftger commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Tested on a NUCLEO-WBA65RI (synopsys OTG HS, high speed), head 18b1795, which sits on #7558 and #7561. The device writes one 64-byte packet per microframe with write_iso and puts a sequence number in each. The host tool polls the endpoint with libusb.

  • Host polling all the time for 3 s: all 24000 packets arrived, with no gaps. The device reported 35 missed out of 24037 writes. I think that's the packets it wrote while the host was starting and stopping its transfers, since that is 37 more writes than the host received, but I didn't test it separately.
  • Host sending no IN tokens for 2 s: the device wrote 16023 packets and write_iso reported 16023 missed, one on every call.
  • Host polling again: the first packet was seq 41716 while the device was at 41684 when polling resumed, so about 32 packets later, and nothing older. No stale packet from the starve period was sent.

The same head also passes the control checks I ran on #7558, with transfers up to 400 bytes.

Not covered: full speed, iso OUT, and the other drivers, which use the default write_iso and report nothing missed.

@leftger

leftger commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@GrantM11235 let me know what you think... I'll look at this again after I get some rest 😪

@cryptographix

cryptographix commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

May want to look at #7440, which is RP specific but seems to be an optimization for packet streaming, removing 1ms of latency per packet and avoiding delivery of stale buffered packets. Alters the contract for write_iso and so can be generalized.

Also: does isochronous need a way to access/sync to the frame clock, for UAC for example?

@github-actions github-actions Bot removed e-nrf Issues for the nRF family of chips e-stm32 Issues for the STM32 family of chips e-rp Issues for the RP family of chips labels Nov 18, 2026
@leftger

leftger commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

@cryptographix both of these are the right questions.

#7440. That change returns from an isochronous IN write() only after the host has collected the packet. A streaming loop then prepares the next packet in the frame it goes out, instead of while the previous one is still armed, which is the frame of latency it measured. write_iso did the opposite: it waited for the previous packet, queued the new one, and returned immediately, so the caller prepared the next packet a frame early.

write_iso now queues the packet and returns once it has been sent or dropped, with nothing left armed. On synopsys the incomplete-isochronous interrupt still drops a packet the host did not poll for, and IsoCompletion::missed counts it, including the packet just written. Dropping the future while it waits leaves that packet armed.

#7440 can become an RP write_iso that waits until the buffer-available bit clears, instead of a special case in write(). RP still uses the default, which calls write_transfer, may return before the host collects the packet, and reports nothing missed. RP also does not drop a packet the host simply skipped: it stays armed until the host takes it or the endpoint is disabled. Synopsys already drops at end of frame, so that part does not move the other way.

The cost is the same one #7440 describes. The next packet has to be queued within the frame after write_iso returns, or that frame goes out empty.

Frame clock. Not in this PR, on purpose. IsoCompletion is non_exhaustive, so a frame number can be added later, but the counters are not the same width: USB 2.0's frame number is 11 bits, a microframe needs 3 more, and synopsys exposes DSTS.FNSOF. I would rather agree on the width, and on frames versus microframes, before picking a type.

UAC pacing wants that clock while the packet is being prepared, not only when write_iso returns. That is a read of the current frame, on Bus, separate from the completion. The feedback endpoint is the other direction: it tells the host the device rate.

@leftger

leftger commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Re-ran the same NUCLEO-WBA65RI test (synopsys OTG HS, high speed) on head 1138229, which returns from write_iso once the packet has been sent or dropped. The device writes one 64-byte packet per microframe, with a sequence number in each. The host tool polls the endpoint with libusb. 10 checks, all passed.

  • Host polling all the time for 3 s: all 24000 packets arrived, with no gaps and no empty microframes. The device reported 30 missed out of 24032 writes. Same shape as the earlier run (35 of 24037): the extra writes are the ones queued while the host was starting and stopping its transfers.
  • Host sending no IN tokens for 2 s: the device wrote 16042 packets and write_iso reported 16042 missed, one on every call, including the packet just written.
  • Host polling again: the first packet was seq 41755 while the device was at 41725 when polling resumed, so about 30 packets later, and nothing older. No stale packet from the starve period was sent. The following second was 8000 packets, no gaps.

@leftger
leftger marked this pull request as ready for review November 18, 2026 04:23
@leftger
leftger added this pull request to the merge queue Nov 18, 2026
@GrantM11235

Copy link
Copy Markdown
Contributor

I'm still not a fan of putting this in the existing Endpoint traits, and I especially don't like the default impl that just doesn't really work right.

I also think it's a little weird that you can't tell what happened to the last packet without sending a new packet.

How does this work with intervals other than 1 (micro)frame? In the linux host side api, transfers are scheduled to happen at the next interval, not the next (micro)frame.

What happens to a packet that is submitted too late? In the linux host side api, it depends on whether the URB_ISO_ASAP flag is set. If it's not set, the packet won't be transfered and an error will be reported. If the flag is set, the packet will be scheduled for the next interval.

@leftger
leftger removed this pull request from the merge queue due to a manual request Nov 18, 2026
@leftger

leftger commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

@GrantM11235:

On the default impl I see two ways I see to fix it:
(a) Keep write_iso in EndpointIn with no default, so drivers that can't do it must say so. Breaking, but this is already the 0.3 wishlist.
(b) Move it to a separate trait, e.g. IsoEndpointIn: implemented only by drivers that can drop and count. Bulk and interrupt endpoints don't change.
I'm leaning on (b)

On the of the last packet: that's fair. Would an accessor on the endpoint or a status returned with the next scheduled packet fit better?

On the Intervals: you're right. Periodic iso should schedule at the next service interval, derived from the endpoint's interval, not the next microframe.

On late submission: I'd propose Linux's URB_ISO_ASAP behaviour as the model: by default a packet that misses its interval is dropped and counted, and the caller can opt in to sending it at the next interval. Is that the policy you want?

I'll hold this out of the queue for now....

@cryptographix

cryptographix commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

@GrantM11235

I also think it's a little weird that you can't tell what happened to the last packet without sending a new packet.

With the revised write_iso contract, I think we already get the status from current packet on return, since "write_iso now queues the packet and returns once it has been sent or dropped", and missing is 0/1.

@leftger

(b) Move it to a separate trait, e.g. IsoEndpointIn: implemented only by drivers that can drop and count. Bulk and interrupt endpoints don't change. I'm leaning on (b)

Seems cleaner. You could then add frame_number() and a setter for the drop policy.

Are you thinking about including a scheduler / multiple buffers in the new trait? Using existing EndpointConfig.buffers as a hint? How would app know how many buffers were allocated. Some targets have very limited HW buffer space, would a ram ring-buffer be needed?

@GrantM11235

Copy link
Copy Markdown
Contributor

I don't think the missing packet count is actually required, and it conflates two distinct errors:

  • The endpoint not was polled at the service interval, because the host was too slow or the IN token was corrupted
  • The device did not have a packet ready in time for the service interval

The first case is hard to detect and handle because it happens some time after the write call returns. But we don't actually need to handle it aside from silently dropping the packet, which is perfectly normal for isochronous transfers.

The second case is much easier because it can be detected before the write call returns, so we can just return an error.

enum IsoError {
    /// The data was not sent because it was submitted too late. It would have been sent at this frame in the past
    TooLate(FrameNum)
    /// The data to be sent is larger than the maximum transfer size for the endpoint
    BufferOverflow,
    /// The endpoint is disabled
    Disabled,
}

trait IsoEndpointIn {
    /// Schedules a packet to be sent at the next service interval.
    /// `Ok(frame)` means the packet has been sent at this frame or is scheduled to be sent at this frame in the future.
    async fn write(&mut self, buf: &[u8]) -> Result<FrameNum, IsoError>;
}

If the device gets behind, it can catch up by repeatedly trying to send packets until it gets Ok instead of Err(TooLate). If the device gets so far behind that that isn't practical, the device will need to be re-initialized. I don't think that URB_ISO_ASAP behavior is necessary, except for the first packet after the endpoint is enabled.

I'm not sure if Ok and Err(TooLate) should be treated differently in most cases, so maybe it is better to move it out of the error enum and just return something like enum IsoRes { frame: FrameNum, on_time: bool } in the Ok result.

Queue Depth

  • Zero means that write doesn't return until the packet has actually been sent (or the (micro)frame ended without the endpoint being polled)
  • One means that write waits until the previous packet has been sent and queues this packet to be sent next
  • n means that write returns when the packet is in the queue with n-1 packets ahead of it

As far as I understand it, #7440 just changes the queue depth from one to zero.

Because it's a tradeoff between slack, latency, and memory usage, I think that the queue depth should be configurable by the user. I don't think that class drivers need to care about the queue depth, but maybe there are cases where they need to be able to query it.

@GrantM11235

Copy link
Copy Markdown
Contributor

This is what I think should happen when the endpoint is enabled:

There needs to be a way to await for the first IN token, like async fn open(&mut self); (maybe this could be combined with wait_enabled?). During this time, the hardware will respond to any IN tokens with a zero-length packet, which is the closest thing iso IN transfers have to a NAK.

After we see the first IN token, we know the initial frame number and we can start accepting packets into the queue, with the first one scheduled to be sent at frame number initial_frame + interval(queue_depth + 1). The next packet will be scheduled exactly one interval later, and so on.

Note that we can't start queuing packets before we get the first IN token, because we don't know what frame they will be scheduled for. This is a problem because we are reporting the frame number when the packet is queued instead of after it is sent.

@cryptographix

cryptographix commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

@GrantM11235 and @leftger - the proposed scheduled model works for me, so I prototyped it on RP2350, with a synth streaming UAC1 audio to macOS:

The API shape I protoed abstracts the frame counter into Slots, via a Ready then Write sequence, as in:

pub trait IsoEndpointIn: Endpoint {
    /// Waits until the next packet can be queued, and numbers its slot. Cancel-safe.
    async fn ready(&mut self) -> Result<IsoSlot, EndpointError>;
    /// Queues one packet for `slot`, without waiting.
    fn write_slot(&mut self, slot: IsoSlot, data: &[u8]) -> Result<(), IsoWriteError>;
}

impl IsoSlot {
    pub const fn index(&self) -> u32;   // service interval number, from the bus clock
    pub const fn skipped(&self) -> u32; // slots the writer left unfilled
}

pub enum IsoWriteError { TooLate, BufferOverflow, Disabled }

Details:

  • A slot is one service interval, numbered from the frame counter.
  • Slots advance whether or not the host polls. A packet the host doesn't collect is dropped when its slot ends, and is never sent late.
  • ready() identifies the next slot to be served, before the data is made
  • a late writer jumps ahead using skipped() instead of building packets to throw away.
  • TBD: a disabled endpoint still counts slots? for my audio app, yes, for general use maybe not.

Doing it this way seems to eliminate the need for open() since the slot number no longer depends on the first IN token. In my case, the synth's clock needs to keep running before the host starts polling. Also not sure open() could be implemented on all stacks: on RP, an unarmed iso IN goes back to idle without sending anything (as per datasheet.

Results on RP2350 at a 1 ms interval:

  1. 20 s stream: 0 skipped slots, 0 underruns
  2. USB task blocked for 30 ms: 29 skipped + 1 TooLate, then it recovered
  3. Arming a packet during its own slot missed the poll every time on macOS, so ready() always offers the next slot

Tested only on RP - FS with 1 queued packet and no intervals above 1 ms

Does this work for you?

@github-actions github-actions Bot added e-stm32 Issues for the STM32 family of chips e-rp Issues for the RP family of chips labels Nov 19, 2026
@leftger

leftger commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

@GrantM11235 @cryptographix: IsoEndpointIn replaces write_iso, and the missed count is gone.

ready() gives the next service interval before the packet is built. write_slot() queues that packet. TooLate means the slot already started. A packet the host doesn't poll for is still dropped silently. skipped counts intervals a late writer left empty.

Synopsys arms one packet at a time. There's no open(): the first slot comes from the frame counter, not the first IN token. RP and STM32 use the default and don't schedule.

@cryptographix

Copy link
Copy Markdown
Contributor

@leftger, looking good!

Do we really want "incorrect" defaults in trait or should drivers only implement IsoEndpointIn if they really support it? Or is this just temporary until we get the other drivers up to speed?

Two x TooLate disables the endpoint. Is that what we really want?

We map TooLate to Disabled, maybe UAC::write_packet should return IsoWriteError and handle the TooLate itself, then we remove the error mapping.

Reworked and tested my RP impl. with no issues. Can send as a follow-on PR when this lands.

Test Results:
Setup: full speed, 1 ms interval, UAC1 48 kHz 16-bit stereo (192-byte packets), queue depth 1, macOS host. A 4-voice synth on core 1 plays a 125 BPM riff; core 0 writes one packet per slot via ready() / write_slot().

| Stream open | 300.1 s |
| Slots | 300,112 (1000.0 per second) |
| Packets written | 300,106 |
| Packets collected by host | 300,100 |
| Uncollected (dropped, never sent late) | 6: 5 while the host started polling, 1 at close |
| skipped | 0 |
| TooLate | 0 |
| Render underruns | 0 |

@GrantM11235

Copy link
Copy Markdown
Contributor

I really like the idea of asynchronously reserving a slot, then writing to it synchronously. By changing the API a bit, it is possible to prevent some errors like using a slot with a different endpoint or trying to reserve two slots and using them out of order.

pub enum IsochronousWriteError {
    TooLate,
    BufferOverflow,
    Disabled,
}

pub trait IsochronousInSlot {
    // TODO: should it be possible to reuse the slot if this returns `BufferOverflow`?
    fn write(self, data: &[u8]) -> Result<(), IsochronousWriteError>;
    // Even better, we can write directly into the HALs buffer, which is the size of the max packet.
    // This may eliminate a copy, and we can get rid of `BufferOverflow`.
    // (Unless the closure returns a nonsense length, but we can just panic in that case)
    fn write_with(self, f: impl FnOnce(&mut [u8]) -> usize) -> Result<(), IsochronousWriteError>;
    // `if slot.skipped.is_some()` is slightly nicer than `if slot.skipped != 0`
    fn skipped(&self) -> Option<NonZeroU32>;
    // the index number is an implementation detail, it doesn't need to be exposed
}

/// Isochronous IN Endpoint trait.
pub trait IsochronousEndpointIn: Endpoint {
    // TODO: does this need a lifetime?
    type Slot: IsochronousInSlot;

    /// # Cancel Safety
    /// 
    /// If the future or the slot is dropped, the slot should become available again
    async fn ready(&mut self) -> Result<Self::Slot, EndpointDisabledError>;
    // or the function could also wait for the endpoint to be enabled
    async fn ready(&mut self) -> Self::Slot;
}

I've changed my mind about exposing the frame number to the class driver. As long as the HAL handles the scheduling and can report any skipped slots, the class driver doesn't need to know the exact frame number. That means the HALs don't need to agree on how many bits to use, or how to handle frames vs microframes.

Doing it this way seems to eliminate the need for open() since the slot number no longer depends on the first IN token.

I don't think that works for intervals other than 1. For an interval of 2, the host can decide whether to use the even or odd frames, and there is no way to know that until the first IN token comes in. I think it's possible to work around that though, as long as we don't tell the class driver which frame we will use. We would schedule the first packet for frame 0, then reschedule it for frame 1 if there wasn't an IN token. If there is still no IN token, we need to drop the first packet and schedule the second packet for frame 2, then frame 3, etc. After we get an IN token, we can lock in and only schedule for the correct even or odd frame. This shouldn't change as long as the endpoint stays enabled.

on RP, an unarmed iso IN goes back to idle without sending anything (as per datasheet.

In that case, maybe the behavior of skipped slots needs to be implementation defined. Based on the datasheet for the STM32F411 USB OTG (synopsys?), it sounds like the hardware always responds to IN tokens for disabled/unarmed endpoints with an empty packet, and it doesn't look like there is a way to change that. Maybe the RP should try to match that behavior in software for consistency?

Do we really want "incorrect" defaults in trait or should drivers only implement IsoEndpointIn if they really support it? Or is this just temporary until we get the other drivers up to speed?

I don't think we should have any bad default impls, I think we should plan to have proper support in all the HAL drivers (there aren't that many, right?) by the time we release the new version. If any of the major impls can't support it, it's better if we know about it before the release so we can change the API to fit.

I haven't looked very hard at the actual changes in this PR yet, partly because it currently has a lot of backwards compatibility stuff that I am hoping will be removed, if possible.

Two x TooLate disables the endpoint. Is that what we really want?

I don't think so

Add IsoEndpointIn and borrowing IsoInSlot traits in embassy-usb-driver with no
fallback defaults, and implement them across all four embassy device drivers:
synopsys-otg (STM32 OTG / nRF54 USBHS), embassy-rp, embassy-stm32 (non-OTG USB),
and embassy-nrf (USBD EP8).

- ready() asynchronously reserves a borrowing Slot<'_> for the next service
  interval and returns EndpointDisabledError when disabled.
- IsoInSlot::write(self, data) synchronously queues one packet for the slot,
  returning IsoWriteError (TooLate, BufferOverflow, Disabled).
- IsoInSlot::skipped() returns Option<NonZeroU32> for empty intervals left by
  a late writer; uncollected packets are dropped silently.
- IsoSchedule locks the host polling phase on the first completed IN transfer
  for intervals > 1 instead of assuming frame % interval == 0.
- UAC IN classes expose ready() and update write_packet() to return
  Result<Option<NonZeroU32>, IsoWriteError> without mapping TooLate to Disabled.
@github-actions github-actions Bot added the e-nrf Issues for the nRF family of chips label Nov 19, 2026
@leftger leftger changed the title usb: add EndpointIn::write_iso for isochronous IN packets usb: add IsoEndpointIn and IsoInSlot for scheduled isochronous IN Nov 19, 2026
@leftger

leftger commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

@GrantM11235 @cryptographix Rebased onto main, dropped all backwards-compatibility shims and default trait impls, and updated the PR description.

  • Borrowing slot API: IsoEndpointIn::ready(&mut self) -> Result<Self::Slot<'_>, EndpointDisabledError> returns a linear IsoInSlot with skipped() -> Option<NonZeroU32> and synchronous write(self, data) -> Result<(), IsoWriteError>. index() is gone.
  • No write_with(&mut [u8]): Synopsys OTG uses a 32-bit streaming FIFO, STM32 non-OTG PMA isn't byte-contiguous, and nRF EasyDMA reads caller RAM, so a &mut [u8] buffer would require allocating up to 1024 B of staging RAM per endpoint.
  • interval > 1 phase locking: IsoSchedule now locks onto the host's polling phase on the first completed IN transfer instead of assuming frame % interval == 0 (Synopsys probes remaining unlocked ticks with 0-byte ZLPs; RP and STM32 non-OTG keep the packet armed across the interval).
  • All 4 HALs implemented: Proper IsoEndpointIn support in synopsys-otg, embassy-rp, embassy-stm32, and embassy-nrf—no fallback defaults.
  • TooLate: Removed From<IsoWriteError> for EndpointError and the 2× TooLate → Disabled mapping; UAC write_packet now returns Result<Option<NonZeroU32>, IsoWriteError>.

@leftger
leftger enabled auto-merge November 19, 2026 18:29
@leftger
leftger added this pull request to the merge queue Nov 19, 2026
@GrantM11235

Copy link
Copy Markdown
Contributor

Hold on, I have a few more suggestions

@leftger

leftger commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

I think we've covered the major overarching issues. I think we're getting into the realm of diminishing returns.

Any other issues can be covered by a follow-up PR.

Merged via the queue into embassy-rs:main with commit 3638971 Nov 19, 2026
8 checks passed
@leftger
leftger deleted the usb-iso-packet-api branch November 19, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

e-nrf Issues for the nRF family of chips e-rp Issues for the RP family of chips e-stm32 Issues for the STM32 family of chips e-usb

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants