Skip to content

[BUG] Vendored axi_riscv_lrsc: write bursts invalidate reservations only at the burst base address; SC succeeds on burst-overwritten data (+ LR can overtake in-flight writes) #3532

Description

@john-kearney

Code of Conduct

  • I have searched the existing bug issues.
  • I am a human engaging in an interpersonal interaction. During this interaction, my words are my own and are not generated. If relevant, I provide links to my sources.

CVA6 commit affected

Any commit since #3183 (2026-01-13, "Vendorize PULP 'axi_riscv_atomics' submodule to support
CVA6-specific changes."), including current master (verified at e643a39).
The same defects exist in the pre-vendoring submodule pin corev_apu/src/axi_riscv_atomics @550881f
(used by e.g. the pulp-v2 branches and Ara's cva6 dependency).

Bug Description

The vendored vendor/pulp-platform/axi_riscv_atomics is a verbatim copy of upstream
pulp-platform/axi_riscv_atomics rev 550881f12e22dfae405612fc1df6368f4c003e68 (v0.2.2,
2019-02-28), i.e. 7 years / 6 minor versions behind. It is live on the DRAM port of the FPGA
wrappers (corev_apu/altera/src/cva6_altera.sv:890, corev_apu/fpga/src/ariane_xilinx.sv:1190)
and in the corev_apu testbenches. The upstream LR/SC shim in this old revision has two defects,
both fixed upstream long ago and both demonstrable with a small Verilator testbench
(reproducer + logs linked below).

Defect 1 — burst invalidation only at the base address (upstream issue #30 / PRs #26+#31,
fixed upstream in v0.8.0/v0.8.2).

The write path latches only the AW base address
(vendor/pulp-platform/axi_riscv_atomics/src/axi_riscv_lrsc.sv:299-305,
w_addr_d = slv_aw_addr_i) and, after the last W beat, issues a single reservation clear at
that one address (:373-385, wr_clr_addr = w_addr_q in W_FORWARD, retried in
W_WAIT_ART_CLR at :395-401). A write burst covering [base, base + (len+1)*size) therefore
invalidates reservations only at base; any other master's burst beat that overwrites a reserved
word leaves the reservation intact. The victim's subsequent SC then succeeds on
foreign-overwritten data:

hart B: LR  X            -> EXOKAY, reservation at X
master A: SW burst base=X-8, len=1 (beats at X-8 and X)   # beat 2 overwrites X
hart B: SC  X            -> EXOKAY  <-- must FAIL (memory at X changed between LR and SC)

This breaks the exclusivity contract of LR/SC (riscv-isa-manual Zalrsc: "An SC may succeed only
if no store ... to the reservation set can be observed to have occurred between the LR and the
SC") and is memory-corruption-grade for lock-free code on multi-master FPGA systems
(CVA6 core + DMA/other masters behind the interconnect on the DRAM port).

A second aspect of the same mechanism: reservations are stored at the exact AXI address with
exact-equality matching, so a sub-word reservation (e.g. LR.W at ...34 on the 64-bit DRAM
port) is not invalidated even by a full-word write that covers its bytes. Upstream fixed this in
v0.8.0 by tracking reservations at word granularity (AXI_ADDR_LSB).

Defect 2 — LR can overtake an in-flight write (RVWMO ordering; upstream issue #4, fixed by the
v0.3.0 in-flight-write checks).

The R and W FSMs are independent. An exclusive read is forwarded downstream (and sets its
reservation) while a write covering the same word is still in flight (AW accepted, B not yet
received). The LR can then be served by the downstream slave before the write lands, return
pre-write data, and re-establish the reservation after the write's reservation clear, so the
following SC succeeds — a lost update:

master A: SW  X   (AW/W accepted, B not yet issued)
hart B:   LR  X   -> served immediately: returns PRE-write data, EXOKAY
master A: (write commits)
hart B:   SC  X   -> EXOKAY  <-- SC succeeds although a foreign write to X occurred between
                                 B's LR and B's SC, and B computed on stale data

Upstream fixed this class in v0.3.0 (2022-03-11, "axi_riscv_lrsc now orders SWs and SCs in
accordance with RVWMO (#4)") with in-flight-write exists-checks.

Reproducer / evidence

  • Focused Verilator testbench (drives the vendored shim's flat ports directly; commit-on-B
    memory model): tb/tb_lrsc_burst.sv in the linked package.
    • logs/run-old-red.log: current vendored code — T1 SC EXOKAY after burst overwrite of the
      reserved non-base word; T1b SC.W EXOKAY after full-word overwrite; T2 LR returns stale
      (aaaaaaaa...) data while the write is uncommitted and the following SC succeeds.
    • logs/run-new-green.log: with the minimal backport — T1/T1b SC fail (OKAY), T2 LR waits
      for the write to complete and returns fresh (beef...) data; all regression checks
      (plain LR/SC pairing, single-beat SW invalidation, burst with reservation at base,
      SC-success consumes reservation, bypass writes) unchanged.
  • Minimal fix: 0001-lrsc-burst-invalidation-rvwmo-backport.patch — ports the upstream v0.8.x
    word-walker (one reservation clear per covered word, termination per upstream PR icache corruption #31) and a
    word-granular reservation check/set, plus an exclusive-AR stall while an overlapping write is
    in flight (upstream Make verilator  #4 semantics adapted to the single-write-in-flight 2019 FSM). Only
    axi_riscv_lrsc.sv is touched; no interface or dependency changes; the existing cva6 local
    big-endian patch to axi_riscv_amos.sv (Add big endian support #3057) is unaffected.

Related

Checklist part 2

  • CVA6 commit affected: since Vendorize PULP 'axi_riscv_atomics' submodule to support CVA6-specific changes. #3183 incl. master e643a39 (2026-09-04).
  • Expected behavior: SC fails after any foreign write overlapping the reservation between LR
    and SC; LR does not observe data older than writes that completed before it was forwarded.
  • Observed: SC succeeds on burst-overwritten reserved words (non-base beats, sub-word
    reservations) and on LR-vs-in-flight-write races.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Component:RTLFor issues in the RTL (e.g. for files in the rtl directory)Status:NewNewly created issue, nobody has looked at it yet.Type:BugFor bugs in the RTL, Documentation, Verification environment or Tool and Build system

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions