arch/risc-v: optimize riscv backtrace by hitHuang · Pull Request #19528 · apache/nuttx

hitHuang · 2026-07-25T02:02:52Z

Summary

risc-v's up_backtrace() currently has the following problems:

Under CONFIG_BUILD_KERNEL, dumpstack on a user task only dumps its user stack, never its kernel stack.
Under CONFIG_BUILD_FLAT/CONFIG_BUILD_PROTECTED, backtrace output contains duplicate frames.
The existing implementation piles up branches, macros, and conditionals to handle the various combinations, which makes the code hard to follow and makes it increasingly difficult to keep fixing bugs or adding support for new scenarios on top of the existing structure.

Problem 1 was already noted in #19439, but never actually fixed in code. Problem 2 is caused by an implementation bug that dumps the same range of the user stack twice.

This change reworks up_backtrace() around a table of stack ranges (interrupt stack / kernel stack / user stack) that a given fp chain may legitimately cross into, walked by one shared procedure that switches ranges as needed, instead of hand-writing a separate branch with one or two hardcoded ranges per scenario.

Problem 1 repro

rv-virt:knsh, dumpstack on a user task only shows frames on the user stack (tid 2/3/4 partially only show kernel-stack addresses 0xc0..., missing the corresponding user-stack frames):

NuttShell (NSH) NuttX-13.0.0
nsh> dumpstack 0 10
[3951369912320000.001000] sched_dumpstack: backtrace| 0: 0x8020cf92 0x80608730 0x802011d6 0x8020004a
[3977139716096000.001000] sched_dumpstack: backtrace| 1: 0x8020cafc 0x80218ffa 0x80208cfc 0x80208d20 0x8020312c 0x80202a94
[3990024617984000.001000] sched_dumpstack: backtrace| 3: 0xc000c478 0xc000561e 0xc0002d4e 0xc0003cbc 0xc0003d5c 0xc0002722 0xc0002542 0xc0000b4a
[4002909519872000.001000] sched_dumpstack: backtrace| 3: 0xc0000b08
[4011499454464000.001000] sched_dumpstack: backtrace| 4: 0xc000289a 0xc0000b44 0xc0000aca 0xc0000a8c

Problem 2 repro

rv-virt:nsh, mb 0xdeadbeef 1 faults on an invalid address, and the backtrace for nsh_main (tid 2) itself contains duplicate frames. This happens any time up_backtrace() tries to backtrace self from up_interrupt_context().

NuttShell (NSH) NuttX-13.0.0
nsh> ps
  TID   PID  PPID PRI POLICY   TYPE    NPX STATE    EVENT     SIGMASK            STACK    USED FILLED COMMAND
    0     0     0   0 FIFO     Kthread   - Ready              0000000000000000 0002016 0000668  33.1%  Idle_Task
    2     2     0 100 RR       Task      - Running            0000000000000000 0001984 0001984 100.0%! nsh_main
nsh> mb 0xdeadbeef 1
riscv_exception: EXCEPTION: Load access fault. MCAUSE: 00000005, EPC: 8000b450, MTVAL: deadbeef
riscv_fault_handler: PANIC!!! Exception = 00000005
dump_assert_info: Current Version: NuttX  13.0.0 2f73fe2267-dirty Jul 20 2026 09:26:46 risc-v
dump_assert_info: Assertion failed panic: at file: :0 task: nsh_main process: nsh_main 0x80008ce2
up_dump_register: EPC: 8000b450
......
stack_dump: 0x80039160: 00000000 00000000 80039180 80002288 00000000 00000000 00000000 00000000
stack_dump: 0x80039180: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
sched_dumpstack: backtrace| 2: 0x80025234 0x80018dc8 0x80015590 0x80008566 0x80001918 0x8000156a 0x8000150c 0x80000186
sched_dumpstack: backtrace| 2: 0x8000b450 0x8000b3b6 0x8000973a 0x8000a490 0x8000a530 0x80008f42 0x80008d62 0x80008d1e
sched_dumpstack: backtrace| 2: 0x80006248 0x80002288 0x8000b450 0x8000b3b6 0x8000973a 0x8000a490 0x8000a530 0x80008f42
sched_dumpstack: backtrace| 2: 0x80008d62 0x80008d1e 0x80006248 0x80002288
dump_tasks:    PID GROUP PRI POLICY   TYPE    NPX STATE   EVENT      SIGMASK          STACKBASE  STACKSIZE      USED   FILLED    COMMAND
dump_tasks:   ----   --- --- -------- ------- --- ------- ---------- ---------------- 0x80034380      2048      1100    53.7%    irq
dump_task:       0     0   0 FIFO     Kthread -   Ready              0000000000000000 0x80037180      2016       668    33.1%    Idle_Task
dump_task:       2     2 100 RR       Task    -   Running            0000000000000000 0x800389c0      1984      1984   100.0%!   nsh_main
sched_dumpstack: backtrace| 0: 0x80008bc0 0x80037950 0x800014a0 0x80000048
sched_dumpstack: backtrace| 2: 0x80025234 0x80018dc8 0x800151d8 0x80015960 0x80015624 0x80008566 0x80001918 0x8000156a
sched_dumpstack: backtrace| 2: 0x8000150c 0x80000186 0x8000b450 0x8000b3b6 0x8000973a 0x8000a490 0x8000a530 0x80008f42
sched_dumpstack: backtrace| 2: 0x80008d62 0x80008d1e 0x80006248 0x80002288 0x8000b450 0x8000b3b6 0x8000973a 0x8000a490
sched_dumpstack: backtrace| 2: 0x8000a530 0x80008f42 0x80008d62 0x80008d1e 0x80006248 0x80002288

The duplicated part (0x8000b450 0x8000b3b6 0x8000973a 0x8000a490 0x8000a530 0x80008f42 0x80008d62 0x80008d1e 0x80006248 0x80002288) resolved via addr2line:

huang@ubuntu:~/workspace/nuttx-fork/nuttx$ riscv64-unknown-elf-addr2line -f -i -e nuttx 0x8000b450 0x8000b3b6 0x8000973a 0x8000a490 0x8000a530 0x80008f42 0x80008d62 0x80008d1e 0x80006248 0x80002288
cmd_mb
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_dbgcmds.c:150
nsh_command
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_command.c:1321
nsh_execute
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_parse.c:759
nsh_parse_command
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_parse.c:2911
nsh_parse
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_parse.c:3116
nsh_session
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_session.c:249
nsh_consolemain
/home/huang/workspace/nuttx-fork/apps/nshlib/nsh_consolemain.c:81
nsh_main
/home/huang/workspace/nuttx-fork/apps/system/nsh/nsh_main.c:82
nxtask_startup
/home/huang/workspace/nuttx-fork/nuttx/libs/libc/sched/task_startup.c:72 (discriminator 1)
nxtask_start
/home/huang/workspace/nuttx-fork/nuttx/sched/task/task_start.c:72

The same duplication reproduces under rv-virt:pnsh as well.

Along the way, several related but independent issues were found and fixed and are tracked as separate PRs: #19439, #19468, #19471, #19479, #19487.

This PR also picks up two small, related fixes/improvements along the way:

up_idle() is a leaf function that doesn't spill ra, which produced garbage frames at the tail of a backtrace that unwound through it.
Under CONFIG_BUILD_KERNEL, when a user app is killed due to a fault, its backtrace is now dumped proactively to help with debugging.

Since these two are small in scope, the rest of this description focuses on the up_backtrace() fix/rework described above.

Impact

Only affects up_backtrace() in arch/risc-v/src/common/riscv_backtrace.c. Fixes the missing-kernel-stack-frames bug under CONFIG_BUILD_KERNEL and the duplicate-frame bug under CONFIG_BUILD_FLAT/CONFIG_BUILD_PROTECTED described above, and replaces the previous scenario-specific hardcoded branches with a general multi-range registration + relay walk. Behavior for already-covered scenarios should stay equivalent or become more complete (recovering previously missing frames); no new Kconfig options or API changes.

Testing

Only one SiFive U74 risc-v SoC is available as real hardware, so most of the verification below is on QEMU.

Config dimensions covered:

rv32 / rv64
BUILD_FLAT / BUILD_PROTECTED / BUILD_KERNEL
no-smp / smp
CONFIG_ARCH_INTERRUPTSTACK > 15 / < 15

The full cross product is 2 * 3 * 2 * 2 = 24 combinations, too many to cover individually. Testing mainly relied on the existing nsh/pnsh/knsh/smp/ksmp64 configs, with CONFIG_ARCH_INTERRUPTSTACK toggled manually to cover the remaining combination. These configs cover the combinations of interest:

Config	Arch	Build mode	SMP	INTERRUPTSTACK > 15	KERNEL_STACK + ADDRENV	LIBC_SYSCALL
nsh	RV32	FLAT	N	Y	N	N
pnsh	RV32	PROTECTED	N	Y	N	Y
knsh	RV32	KERNEL	N	Y	Y	Y
smp	RV64	FLAT	Y	Y	N	N
ksmp64	RV64	KERNEL	Y	Y	Y	Y
nsh (no intstack)	RV32	FLAT	N	N	N	N
pnsh (no intstack)	RV32	PROTECTED	N	N	N	Y
knsh (no intstack)	RV32	KERNEL	N	N	Y	Y

Scenarios covered (the most complex case involves interrupt stack + kernel stack + user stack together; there's also an independent dimension of backtracing self vs. a different task, where cross-task backtracing under BUILD_KERNEL additionally has to cross address spaces):

ostest
dumpstack (application) on self / another task
mb on an invalid address triggering a panic with backtrace output
backtrace from an interrupt handler that preempted a kernel thread
backtrace from an interrupt handler that preempted a user thread running in user mode
backtrace from an interrupt handler that preempted a user thread that had trapped into the kernel via a syscall
NULL pointer access inside an interrupt handler that preempted a kernel thread, triggering a panic with backtrace output
NULL pointer access inside an interrupt handler that preempted a user thread running in user mode, triggering a panic with backtrace output
NULL pointer access inside an interrupt handler that preempted a user thread that had trapped into the kernel via a syscall, triggering a panic with backtrace output
a user thread trapping into the kernel and accessing a NULL pointer, triggering a panic with backtrace output

Notes:

Under BUILD_FLAT/BUILD_PROTECTED, scenarios 4/5/6 are treated as a single case, since these build modes don't distinguish user stack from kernel stack; likewise for 7/8/9. Scenario 10 only applies to BUILD_KERNEL.
Under SMP, interrupt-handler scenarios only test backtracing self (cross-CPU backtracing from interrupt context isn't recommended).

Test matrix and results (- means not applicable for that config):

Config	1	2	3	4	5	6	7	8	9	10
nsh	Y	Y	Y	Y			Y			-
pnsh	Y	Y	Y	Y			Y			-
knsh	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
smp	Y	Y	Y	Y			Y			-
ksmp64	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
nsh (no intstack)	Y	Y	Y	Y			Y			-
pnsh (no intstack)	Y	Y	Y	Y			Y			-
knsh (no intstack)	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y

All the above test cases have been covered, and the backtrace output matched expectations in every case.

github-actions · 2026-07-25T02:09:48Z

MemBrowse Memory Report

No memory changes detected for:

acassis · 2026-07-25T12:13:48Z

@hitHuang please add: Assisted-by: in the commit message to indicate which AI model you used to create this code

acassis · 2026-07-25T17:10:03Z

@hitHuang you need to add in all commits

up_idle() is a leaf function; with frame pointers enabled the compiler skips spilling ra, corrupting the fp chain. Clobber "ra" in the WFI inline asm so backtrace can walk past the idle frame correctly. Assisted-by: Claude Code:claude-sonnet-5 Signed-off-by: liang.huang <liang.huang@houmo.ai>

When riscv_fault_handler() kills a user task with SIGSEGV, it previously produced no stack trace, making the crash hard to diagnose. Call sched_dumpstack() on the killed task when SCHED_BACKTRACE is enabled. Assisted-by: Claude Code:claude-sonnet-5 Signed-off-by: liang.huang <liang.huang@houmo.ai>

Bugs fixed: 1. BUILD_FLAT/BUILD_PROTECTED: the interrupt-context self backtrace printed the same frames twice. 2. BUILD_KERNEL: a cross-tcb backtrace never included the target task's kernel stack, so frames on it were missing. Also reworked the implementation around a list of independent starting points, each carrying every stack range its fp chain may cross into, splicing them into one backtrace. Assisted-by: Claude Code:claude-sonnet-5 Signed-off-by: liang.huang <liang.huang@houmo.ai>

xiaoxiang781216 · 2026-07-26T18:04:43Z

 #include "riscv_internal.h"

+/****************************************************************************
+ * Private Types


macro section

xiaoxiang781216 · 2026-07-26T18:08:05Z

 ****************************************************************************/

-static inline uintptr_t getfp(void)
+always_inline_function static uintptr_t getfp(void)


static always_inline_function...

xiaoxiang781216 · 2026-07-26T18:10:23Z

+                         tcb->adj_stack_size);
+
+#ifdef CONFIG_LIB_SYSCALL
+  bool insyscall = (tcb->flags & TCB_FLAG_SYSCALL) != 0;


move to the beginning

xiaoxiang781216 · 2026-07-26T18:14:23Z

+
+/* One candidate stack range a segment's fp chain may be walking. */
+
+struct bt_range_s


let change bt_/BT_ to backtrace_/BACKTRACE_

hitHuang requested review from Ouss4, gustavonihei, jerpelea, tmedicci, xiaoxiang781216 and yf13 as code owners July 25, 2026 02:02

github-actions Bot added Arch: risc-v Issues related to the RISC-V (32-bit or 64-bit) architecture Size: M The size of the change in this PR is medium labels Jul 25, 2026

hitHuang force-pushed the optimize/riscv_backtrace branch from 6c11cb7 to a97eac8 Compare July 25, 2026 14:15

hitHuang added 3 commits July 26, 2026 17:10

hitHuang force-pushed the optimize/riscv_backtrace branch from a97eac8 to 47298a5 Compare July 26, 2026 09:11

xiaoxiang781216 reviewed Jul 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

arch/risc-v: optimize riscv backtrace#19528

arch/risc-v: optimize riscv backtrace#19528
hitHuang wants to merge 3 commits into
apache:masterfrom
hitHuang:optimize/riscv_backtrace

hitHuang commented Jul 25, 2026

Uh oh!

github-actions Bot commented Jul 25, 2026 •

edited

Loading

Uh oh!

acassis commented Jul 25, 2026

Uh oh!

acassis commented Jul 25, 2026

Uh oh!

xiaoxiang781216 Jul 26, 2026

Uh oh!

xiaoxiang781216 Jul 26, 2026

Uh oh!

xiaoxiang781216 Jul 26, 2026

Uh oh!

xiaoxiang781216 Jul 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		/* One candidate stack range a segment's fp chain may be walking. */

		struct bt_range_s

Uh oh!

Conversation

hitHuang commented Jul 25, 2026

Summary

Problem 1 repro

Problem 2 repro

Impact

Testing

Uh oh!

github-actions Bot commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

Uh oh!

acassis commented Jul 25, 2026

Uh oh!

acassis commented Jul 25, 2026

Uh oh!

xiaoxiang781216 Jul 26, 2026

Choose a reason for hiding this comment

Uh oh!

xiaoxiang781216 Jul 26, 2026

Choose a reason for hiding this comment

Uh oh!

xiaoxiang781216 Jul 26, 2026

Choose a reason for hiding this comment

Uh oh!

xiaoxiang781216 Jul 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Jul 25, 2026 •

edited

Loading