Skip to content

qualcommax: qca_ppe: gate switch-fabric TX-MAC across carrier flaps - #154

Closed
JuliusBairaktaris wants to merge 15 commits into
Ansuel:qca-edma-reworkfrom
JuliusBairaktaris:qca-ppe-carrier-flap-txmac
Closed

JuliusBairaktaris wants to merge 15 commits into
Ansuel:qca-edma-reworkfrom
JuliusBairaktaris:qca-ppe-carrier-flap-txmac

Conversation

@JuliusBairaktaris

@JuliusBairaktaris JuliusBairaktaris commented Jul 8, 2026 •

Copy link
Copy Markdown

Hi @Ansuel — thanks again for the qca-edma rework. Along the way I hit a silent per-port egress wedge that traces back to qca_ppe link handling; this PR fixes it.

The fix (one commit, qca_ppe_main.c only)

On a carrier flap, qca_ppe_mac_link_down() disables only the port MAC (ppe_port_gmac_set(..., false, false)) and leaves the PPE switch-fabric TX-MAC (PPE_PORT_BRIDGE_CTRL_TXMAC_EN) forwarding into the port. At the down→up edge the fabric can then dequeue into a MAC that is still being re-clocked, which latches the port's Queue-Manager egress scheduler into a state that no longer drains: the link comes back at full speed and the host still hands frames to the port, but nothing egresses the wire — a silent, per-port egress black-hole that clears only on reboot.

The reference ssdk switch driver gates the bridge TX-MAC across every link change — disabling it first on link-down (its own comment: "release ppe port egress packets when link down") and re-enabling it last on link-up, after the MAC is back up (qca_hppe_mac_sw_sync_task, src/adpt/hppe/adpt_hppe_portctrl.c, tag NHSS.QSDK.14.0.r8-00026-O). The qca_ppe rewrite dropped that gating. This PR restores it, reusing the existing ppe_port_bridge_txmac_set() helper: disable it before the MAC teardown in mac_link_down, and re-enable it as the final step of mac_link_up. No new API; device-agnostic (flat per-port PPE_PORT_BRIDGE_CTRL register, no port mask); covers both the GMAC and XGMAC cases.

Scope — where it actually bites

I hit this on the NSS offload path, where the firmware injects into a port's egress queue decoupled from the host's MAC-ready state, so it bursts into the cold-MAC window right at link-up. On the plain EDMA datapath (no NSS) I could not reproduce it across many flaps at up to ~420 Mbit host-forwarded load — there the host TX is gated by netif_carrier_on, which follows mac_link_up, so it never bursts into a re-clocking MAC. So for a pure-EDMA user this is mostly defensive / ssdk-parity; it's load-bearing for NSS offload. Either way qca_ppe's link path is objectively missing the fabric gating ssdk does, and the change is harmless on plain EDMA too (I confirmed the bit gates host egress cleanly and is re-enabled last).

How to reproduce (NSS offload build)

  1. Drive a sustained high-rate download to a wired client on an offloaded LAN port (~40+ accelerated flows, ~350 Mbit; e.g. iperf3 -u -R -P 8 from a few servers).
  2. Force a long carrier-down on that port for ≥40–60 s while the load runs — a real cable pull works, or an MDIO PHY power-down (write BMCR 0x1800 to the port PHY, then 0x1200 to bring it back).
  3. On link-up, watch the port's QM per-queue occupancy (AC_UNI_QUEUE_CNT_TBL, 0x3a84e000 + qid*0x10, low 12 bits) for the port's base queue: without the fix it pins at its ceiling and the client's egress is dead (0 bytes) until reboot; with the fix it stays ~0 and egress survives. PPE_PORT_BRIDGE_CTRL(port) (0x3a060300 + port*4, bit 16) reads 0 during the down and 1 again after up, confirming the gating fires.

Validated on a Xiaomi AX3600 (IPQ8074): the unpatched driver wedges on the first flap; the patched driver survives 8/8 flaps at 68 accelerated flows.

Thanks!

blogic and others added 15 commits July 4, 2026 17:45
Add kernel patch for the DSA out-of-band tagging protocol used by the
PPE switch driver, and enable CONFIG_NET_DSA_TAG_OOB.

Signed-off-by: John Crispin <john@phrozen.org>
Add pending patch that introduce a new HW CLK OP to trigger recalculation
of the rate.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
PCS driver for UNIPHY SerDes blocks on Qualcomm IPQ platforms.

Signed-off-by: John Crispin <john@phrozen.org>
[ rework Makefile for external repository, dependency ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
[ make it in-tree under qualcommax ]
Signed-off-by: Robert Marko <robimarko@gmail.com>
PPE switch driver for Qualcomm IPQ platforms, depends on EDMA and
UNIPHY PCS.

Signed-off-by: John Crispin <john@phrozen.org>
[ rework Makefile for external repository, dependency ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
[ bring PPE in-tree ]
Signed-off-by: Robert Marko <robimarko@gmail.com>
EDMA dataplane ethernet driver for Qualcomm IPQ platforms.

Signed-off-by: John Crispin <john@phrozen.org>
[ rework Makefile for external repository, dependency ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
[ rework for in-tree ]
Signed-off-by: Robert Marko <robimarko@gmail.com>
Add IPQ5018 DWMAC driver. IP version of this Synopsys DWMAC is 3.7.

This Qualcomm IPQ5018 specific MAC implementation supports link speeds
of 10HD/FD, 100HD/FD, 1,000HD/FD, and 2500FD and SGMII and 2500BASEX
interface modes.

The driver supports the MAC be attached directly to a PHY or via an
optional PCS to a switch or PHY.

Signed-off-by: George Moussalem <george.moussalem@outlook.com>
Now that we are not using SSDK anymore, we must set the USXGMII MAC autoneg
bit manually, as otherwise MAC autonegotiation does not work.

Signed-off-by: Robert Marko <robimarko@gmail.com>
Clocks are disabled by default for IPQ5018 GEPHY, but they are required
for the PHY to function properly. So let's enable the RX and TX clocks.

Signed-off-by: George Moussalem <george.moussalem@outlook.com>
Add DTSI files defining EDMA, PPE, and UNIPHY nodes for the new PPE
driver bindings on IPQ5018, IPQ6018 and IPQ8074 platforms.

IPQ5018 requires a patch for UNIPHY node as its cmn PLL node is upstream.

These replace the existing NSS-DP ones.

Signed-off-by: John Crispin <john@phrozen.org>
[IPQ5018]
Signed-off-by: George Moussalem <george.moussalem@outlook.com>
[IPQ6018 and IPQ8074]
Signed-off-by: Robert Marko <robimarko@gmail.com>
Convert IPQ60xx and IPQ807x devices from the old NSS dataplane to the new
PPE stack.

IPQ50xx is not yet supported.

Signed-off-by: Robert Marko <robimarko@gmail.com>
Import the interface renaming script from filogic so we can use DTS
label or openwrt,netdev-name to name interfaces from DTS on non DSA
ports.

Signed-off-by: Robert Marko <robimarko@gmail.com>
Convert to UNIPHY PCS and DWMAC ethernet stack from qca-ssdk.

Since we are not using SSDK anymore, we dont need to patch in the UNIPHY
clock names anymore so drop the patch.

Signed-off-by: George Moussalem <george.moussalem@outlook.com>
Now that all targets using NSS-DP and SSDK have been converted, we can
finally drop these.

Signed-off-by: Robert Marko <robimarko@gmail.com>
… module"

This reverts commit 14beb34.

It turned out not to be required.

Signed-off-by: Robert Marko <robimarko@gmail.com>
On a carrier flap qca_ppe_mac_link_down disabled only the port MAC and
left the PPE switch fabric forwarding into the port. At the down->up edge
the fabric could then dequeue into a MAC that was still being re-clocked,
which latches the port's queue-manager egress scheduler on IPQ807x into a
state that no longer drains: the link comes back at full speed and the
host still hands frames to the port, but nothing egresses the wire - a
silent, per-port egress black-hole that clears only on reboot. It shows
up with a high-rate flow egressing a port across a long carrier-down.

The reference ssdk switch driver gates the bridge TX-MAC across every
link change - disabling it first on link-down ("release ppe port egress
packets when link down") and re-enabling it last on link-up, after the
MAC is back up (qca_hppe_mac_sw_sync_task, src/adpt/hppe/
adpt_hppe_portctrl.c, NHSS.QSDK.14.0.r8-00026-O). The qca_ppe rewrite
dropped that gating. Restore it: disable the fabric TX-MAC before the MAC
teardown in qca_ppe_mac_link_down and re-enable it as the final step of
qca_ppe_mac_link_up.

The register and helper already exist (ppe_port_bridge_txmac_set); no new
API. The change is device-agnostic (flat per-port PPE_PORT_BRIDGE_CTRL
register, no port mask) and covers both the GMAC and XGMAC port cases.

Signed-off-by: Julius Bairaktaris <julius@bairaktaris.de>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a Qualcomm PPE (qca_ppe) link-handling corner case where, on carrier flaps, the PPE switch-fabric can continue dequeuing into a port MAC while it is being torn down / re-clocked. The change restores the “bridge TX-MAC” gating behavior used by the reference SSDK driver to prevent silent per-port egress wedges (notably observed on NSS offload).

Changes:

  • Disable PPE_PORT_BRIDGE_CTRL_TXMAC_EN on mac_link_down() before MAC teardown.
  • Re-enable PPE_PORT_BRIDGE_CTRL_TXMAC_EN on mac_link_up() as the final step after MAC bring-up.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@robimarko
robimarko force-pushed the qca-edma-rework branch 3 times, most recently from cf81f32 to 412526f Compare July 11, 2026 20:26
@JuliusBairaktaris

Copy link
Copy Markdown
Author

The qca-edma rework is merged into openwrt/main now, so this moves upstream to openwrt#24188. Closing here — thanks @Ansuel.

@JuliusBairaktaris
JuliusBairaktaris deleted the qca-ppe-carrier-flap-txmac branch July 11, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants