Skip to content

fix: Address review findings across the async concurrency stack - #2552

Merged
schloerke merged 42 commits into
jat255/2508-windows-busy-indicator-checkfrom
schloerke/2508-stack-review-fixes
Oct 15, 2026
Merged

schloerke merged 42 commits into
jat255/2508-windows-busy-indicator-checkfrom
schloerke/2508-stack-review-fixes

Conversation

@schloerke

@schloerke schloerke commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #2525. Fixes the findings from two reviews of the whole stack (#2549, #2545, #2546, #2520, #2547, #2548), so that the fixes can be seen in one place. One commit per finding, so each can be inspected (or squashed down into its PR) on its own.

First review

#2549 (non-waiting flush)

  • reactive.flush() could return before a session's outputs were sent. run_until_idle() treated "no running task, empty effect queue" as idle, but ignored _round_requested: a round requested via call_soon (e.g. by an effect's last step, to start its session's output flush) hadn't started a task yet. The idle check now also requires no pending request. Test: test_flush_waits_for_a_round_requested_by_an_effects_last_step.
  • Errors in the output flush task lost their traceback. An error in an input update, an on_flush/on_flushed callback, or the restore context flush was passed only to _unhandled_error(), which prints the message. The traceback is now printed first (the _start_cycle() print moved there, so an action's error is printed once). Test: test_flush_callback_error_closes_the_session_with_a_traceback.

#2545 (cancel effects on destroy)

  • A scope's values were destroyed before its effects. session.destroy(id) runs destroy callbacks in registration order, and a value is usually declared before the effects that read it. That only worked because nothing yielded between callbacks: an async on_destroy callback that awaits let a running effect resume, read the destroyed value, and close the whole session with DestroyedReactiveError. AsyncCallbacks.register() takes first=True, effects register their destroy callback with it (AppSession._on_destroy_first(fn, ns=), reached from a module effect through its proxy's root session; not a Session method), so effects are cancelled before any other destroy callback runs. The existing test now has an async on_destroy callback and releases the effect before the destroy, which made it fail before the fix.

#2546 (run_once_when_idle)

  • The base Session.run_once_when_idle() ran fn synchronously, against its own docstring. It now raises NotImplementedError (all three subclasses implement it). It isn't abstract, so a Session subclass outside shiny, such as a test mock, can still be instantiated.

#2520 (deprecate reactive.lock())

  • The changelog now says that the lock no longer provides mutual exclusion either, and that code using it as a plain mutex should create its own asyncio.Lock.
  • The warning stays once per process (as decided); the changelog and docstring now say so. The v1.9.0 wording stays.

#2547 (per-session reactive_update span)

  • With bookmarking on, the first cycle's span was a root trace. bookmark._create_effects() ran before the session_start span was set; the effects it creates invalidate, which turns the session busy and starts the cycle span with no parent. The effects are now created inside the span, before the server function. test_effect_spans_are_children_of_their_sessions_reactive_update runs with bookmark_store="url" too.
  • test_session_ending_mid_cycle_ends_its_reactive_update passed without the _end_otel_cycle_span() call in _run_session_ended_tasks, because the cancelled effect's finally ended the span anyway. It now asserts that the span ended before the cancelled effect's finally ran.
  • The changelog entry cites feat(otel): Give each session's cycle its own reactive_update span #2547 instead of feat(otel): Give each session's cycle its own reactive_update span #2522.

#2548 (bookmark restore gate)

  • Effects queued before their session ended hung forever. If a session ended while its effects were still queued behind another session's open round, its restore effect saw _destroyed and skipped its run, so it never opened the gate, and every lower-priority effect of the session waited on it forever (leaking the tasks and keeping reactive.flush() from returning). A destroyed effect no longer waits on the gate. Test: test_session_ending_before_its_restore_effect_runs_releases_its_effects.

Second review

Correctness

  • A second Effect_.destroy() interrupted a cancelled run's cleanup (feat(reactive): Cancel an effect's in-progress run when it is destroyed #2545). A session registers destroy for both its end and its destroy, and a cancelled task stays in _run_tasks until its finally completes, so the second call cancelled the cleanup mid-await. destroy() is now idempotent. Test: test_destroying_an_effect_twice_does_not_interrupt_its_cleanup.
  • run_once_when_idle() after the session ended grew the queue forever (feat(session): Add session.run_once_when_idle() #2546). Nothing drains _cycle_start_action_queue once the session has ended, so a custom timer that kept calling it leaked one closure per call. Calls after the session ends are now dropped right away. Test: test_run_once_when_idle_after_the_session_ends_is_dropped_right_away.
  • An extra, empty round per cycle (feat: Run reactive effects #2549). When an effect requested a round while one was running, the round's finally reset _round_requested and requested again, so two callbacks each spawned a start_round(); the second set _rerun_round, and the first ran one more round with nothing in it, firing every reactive.on_flushed() callback (and the app's output-flush hook) an extra time. The early-return path now clears the request flag, and the finally just requests (a no-op if a request is pending). Test: test_round_requested_during_a_round_runs_exactly_one_more_round.
  • NotifyException(close=True) from a destroyed effect closed the session (feat(reactive): Cancel an effect's in-progress run when it is destroyed #2545), against the rule that errors from a destroyed effect are only logged. Test: test_notify_exception_from_a_destroyed_effects_cleanup_keeps_session_open.
  • An effect task's error lost its traceback (feat: Run reactive effects #2549): the outer except in on_flush_cb only called _unhandled_error(), which prints the message. It prints the traceback first now.
  • A crashing cycle reported OK (feat(otel): Give each session's cycle its own reactive_update span #2547). _end_otel_cycle_span() always set StatusCode.OK, even when _unhandled_error() closed the session. The cycle span now records the exception and ends with ERROR. Test: test_reactive_update_reports_the_error_that_closed_the_session.

Docs

Tests and cleanups

  • AsyncCallbacks entries are a NamedTuple instead of an index-addressed 3-tuple (fix: Address review findings across the async concurrency stack #2552).
  • RestoringClient builds on RecordingClient (bookmark_store=) instead of copying its constructor (fix(bookmark): Run async on_restore callbacks before the session's effects #2548).
  • The busy-indicator Playwright test asserts every message arrived before computing gaps, so a missing one fails with the collected times rather than a KeyError; the express-session-start test waits for a count of 0 or 1 rather than any count (fix(session): Send each output's progress message before it renders #2525).
  • MockTime.advance_time() yields until no round is requested, running, or queued instead of a fixed ten passes. (reactive.flush() can't be used there: a mocked sleep inside an effect calls advance_time, which would then wait on that same effect.)
  • Timing-based assertions are gone from test_concurrency.py and test_otel_reactive_update.py: about 40 tests that slept and then asserted "nothing happened", or ordered effect runs by sleep durations, now wait on state (_run_tasks, _busy_count, _run_waiters, _cycle_start_action_queue) or asyncio.Events. The remaining sleeps only make bodies async or space requests; no assertion depends on their duration. test_otel_reactive_update.py's TIMEOUT is 10 s like test_concurrency.py's.
  • A regression test checks that the app's output-flush hook is dropped after its last session ends (a suspected re-registration leak did not reproduce: nothing yields between removing the session and disabling its output flush).

Not changed

Testing

Each new or changed test was run without its fix (fails) and with it (passes); the converted timing tests were run three times under xdist. pytest tests/pytest passes (1400 tests), as do uv run make format check-lint check-types check-pyright. The bookmark and otel Playwright suites pass on Chromium.

Fixes found while reviewing #2549, #2545, #2546, #2520, #2547, and #2548:

- `reactive.flush()` could return before a session's outputs were sent.
  `run_until_idle()` ignored a round requested via `call_soon` that had
  not started a task yet (#2549).
- Errors in the output flush task (input updates, `on_flush`/`on_flushed`
  callbacks) lost their traceback; only the message was printed (#2549).
- `session.destroy(id)` ran destroy callbacks in registration order, so a
  scope's values were destroyed before its effects were cancelled. An
  async `on_destroy` callback that yields let a running effect read a
  destroyed value, which closed the whole session (#2545).
- The base `Session.run_once_when_idle()` ran `fn` synchronously, against
  its docstring; it is now abstract (#2546).
- The changelog and docstring now say that `reactive.lock()` no longer
  provides mutual exclusion and warns once per process (#2520).
- With bookmarking on, the bookmark effects turned the session busy before
  the `session_start` span existed, so the first cycle's `reactive_update`
  span was a root trace. They are now created inside the span (#2547).
- The session-end test for the cycle span passed without the code it
  covered; it now checks that the span ends before the cancelled effect
  does (#2547).
- Effects queued before their session ended hung forever on the bookmark
  restore gate, because the destroyed restore effect never opened it.
  A destroyed effect no longer waits on the gate (#2548).
… test

On Windows, `time.time_ns()` is too coarse to separate the span's end time
from the cancelled effect's `finally`, so the two timestamps were equal
and the test failed. Record whether the session's cycle span had already
ended when the `finally` ran instead.
@jat255
jat255 added this pull request to stack #2550 October 14, 2026 16:13
Comment thread shiny/session/_session.py Outdated
Comment thread shiny/session/_session.py Outdated
Comment thread shiny/session/_session.py
…der_output

Its two header lines were missing, so its body sat inside the preceding
test (pyright: "server" obscured) and it never ran on its own.
…ations

Tests that relied on one sleep being longer than another (an older effect
run finishing after a newer one, a stale calc run finishing after the fresh
one, a background task starting after its effect's run) now release each
step with an asyncio.Event, so the order no longer depends on the clock.
Adds settle(), a bounded number of loop passes, for checking that a parked
run stays parked.
… waits

CI machines are slow, so a test can't assume a sleep of N ms was long
enough for something (not) to happen. Each negative assertion now waits for
internal state that proves the thing is parked (the update in the session's
cycle-start queue, the effect's run task registered, the calc's waiter list,
the session's busy count), then checks after a bounded number of loop
passes. Start-up settling uses reactive.flush(), which also fails fast when
a flush loop never goes idle.
Also raise the wait limit to 10s like test_concurrency's: waits return as
soon as their condition holds, so a long limit costs passing tests nothing.
…d of making it abstract

An abstract method stops any Session subclass outside shiny (e.g. a downstream
test mock) from being instantiated, even if it never calls run_once_when_idle().
…e/2508-stack-review-fixes

# Conflicts:
#	CHANGELOG.md
An effect of a module scope reaches it through the proxy's root session, passing
the scope's namespace. Sessions that aren't an AppSession (or a proxy of one) fall
back to on_destroy().
@schloerke
schloerke merged commit 549b78c into main Oct 15, 2026
176 checks passed
@schloerke
schloerke deleted the schloerke/2508-stack-review-fixes branch October 15, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants