Background task queue with admin API, WebSocket updates, and a Maintainer dashboard page - #54
Open
JustSamuel wants to merge 7 commits into
Open
Background task queue with admin API, WebSocket updates, and a Maintainer dashboard page#54JustSamuel wants to merge 7 commits into
JustSamuel wants to merge 7 commits into
Conversation
Adds a generic database-backed task queue (Laravel-style) and uses it to replace the existing BullMQ + Redis email queue. The `task` row is the source of truth for state (pending, processing, completed, failed) and carries enough JSON payload to run the task independently. Workers concurrently drain the table via an atomic SELECT-then-UPDATE claim: a candidate is picked, then `UPDATE task SET status='processing' WHERE id=? AND status='pending'` accepts it only if affected_rows == 1. That makes concurrent workers (in this process or across replicas) safe without explicit row locking or SKIP LOCKED tricks, and works across SQLite, MySQL/MariaDB, and Postgres. BullMQ, ioredis, the redis-connection helper, the mail-queue worker, and the dual-mode Mailer all go away. The Mailer is now a thin nodemailer wrapper that the task worker calls directly; emails flow through the same `task` row that admins see in the UI, which is the unit failure reporting + retry operate on. A `send-notification` task type is registered so Notifier callers have a handler ready to dispatch into in the next commit. Existing tests that asserted on the BullMQ mail-queue add now assert on the dispatched task row in the DB, and a new race-condition test verifies that two parallel `claim()` calls produce exactly one winner. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the /v1/tasks controller (list, get, retry, stats) gated behind the new `Task` RBAC resource. The retry endpoint resets a failed task back to pending so an admin can re-run it after fixing the underlying cause. Surfaces `failedTaskCount` on the existing /ping response so monitoring tools can treat a non-zero queue of failed tasks as a health signal -- the issue calls this out as an explicit acceptance criterion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits Notifier.notify into a synchronous pre-check that resolves eligible channels and a `notifySync` worker path that owns the actual rendering and channel dispatch. The pre-check keeps the original behaviour (throwing when no channel applies) so call-sites and tests that assert on that branch are unaffected, while the heavy work -- template rendering, SMTP -- now happens inside the `send-notification` task handler. The big win is fine-notify and other fan-out flows no longer block the HTTP request handler on N rendering passes; they enqueue N tasks and return immediately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Registers a `tasks:all` room (policy gated on `get:all:Task`) and a `task:updated` event in the event registry. TaskService already emits through the generic `WebSocketService.emit` on every state transition; this commit gives that emit somewhere to land so subscribed admin clients refresh in real time instead of polling /v1/tasks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets the dashboard call into the new /v1/tasks admin endpoints (list, get, retry, stats) through the standard authenticated axios instance, matching every other API on the service. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…s work Dashboard `setupWebSocket()` was being called without a `getToken` callback, so the socket connected anonymously and any room with a policy (e.g. the new `tasks:all`) returned "Authentication required for this room." Mirror the POS app's pattern: pass a getter that reads the JWT from storage on every (re)connect, so token refreshes are picked up automatically. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New /maintainer/tasks page (renamed Maintainer to a dropdown with Maintenance and Background tasks) for admins with the `Task` RBAC resource. The page shows: - A stats grid (pending / processing / completed / failed) wired to /v1/tasks/stats. - A paginated, filterable DataTable of tasks. Filters: status (multi- select) and type (free-text). Each row exposes a (i) info button that opens a modal with the full task -- id, type, status tag, attempts, every timestamp, pretty-printed JSON payload, full last error. Failed rows additionally show a Retry button that calls POST /v1/tasks/:id/retry. - A "LIVE" indicator dot. The view subscribes to the `tasks:all` WebSocket room on mount and listens for `task:updated` events -- rows update in place, new dispatches appear at the top, and a debounced fetchStats keeps the counters fresh without polling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SudoSOS Coverage ReportCommit: 72c83ea
Details (changed files)
The full per-file breakdown is omitted to stay under GitHub's 65,536-character comment limit. Download the |
Coverage Report for CI Build 31796994125Coverage increased (+0.3%) to 92.535%Details
Uncovered Changes
Coverage Regressions11 previously-covered lines in 4 files lost coverage.
Coverage Stats
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ported from GEWIS/sudosos-backend#918 and GEWIS/sudosos-frontend#830.
These two PRs were companions stuck in separate repos, connected only by a published
@gewis/sudosos-clientnpm package. Backend#918 was sitting CONFLICTING against old backend develop, and frontend#830 was a draft waiting on it. Building both together for the first time, in one repo, with the OpenAPI client generated in-repo instead of published separately, turned up a real drift: the frontend's task detail dialog renderedtask.payload, but the backend deliberately never puts payload in the API response (notification params can contain reset tokens). I removed the payload viewer from the detail dialog (template, theformatPayloadhelper, and thedetails.payloadkey in en/nl/pl) instead of inventing a backend field to feed it.Other things that came up during the port:
packages/sudosos-clientoff the merged backend to addTasksApi, folded into the "expose task admin API" commit.websocket-service.tsand its test had already picked up an unrelatedTerminalPaymentResponseroom/event registration on develop since this PR's branch was cut. Merged both features' room registrations and emit methods side by side.backend/package.json/pnpm-lock.yamlconflicts from droppingbullmq/iorediswere just a lockfile regeneration. Thepnpmoverrides block lives at the repo root now, not per-package, so there was nothing to re-add there.@typescript-eslint/no-unnecessary-type-assertionfailure showed up in the newtasks.store.ts-- the monorepo's eslint config is stricter here than either old repo's was. Fixed it.pnpm buildandpnpm run backend:testboth pass (2579 tests passed, 38 skipped, 0 failed).backend:lintandfrontend:lintare clean too.