DevTrack is a monorepo with a React SPA frontend, an Express API, and standalone BullMQ worker processes. Background work (platform sync, resume intelligence, XP) is isolated from the HTTP thread to keep API latency predictable.
graph TD
subgraph Client["Presentation Layer"]
SPA["React 19 + Vite SPA"]
ZS["Zustand (local UI state)"]
RQ["TanStack Query (server cache)"]
SSEc["SSE Singleton Client"]
end
subgraph API["API Layer (Express 5)"]
RT["/api/v1 Router"]
AUTH["Clerk Auth Middleware"]
SSEh["SSE Handler"]
end
subgraph Workers["Background Workers (BullMQ)"]
SYNC["Platform Sync"]
XPw["XP Processing"]
RES["Resume Intelligence"]
MAINT["System Maintenance"]
end
subgraph Data["Persistence"]
MONGO[(MongoDB 7)]
REDIS[(Redis 7)]
end
SPA -->|REST| RT
SSEc <-->|SSE /events| SSEh
RT --> AUTH
AUTH --> MONGO
RT --> REDIS
Workers --> REDIS
Workers --> MONGO
SSEh <-->|PubSub| REDIS
| Package | Entry | Role |
|---|---|---|
frontend/ |
src/main.tsx |
React SPA, Clerk auth, SSE client |
backend/ |
src/index.ts |
Express HTTP server |
backend/ |
src/worker-entrypoint.ts |
BullMQ worker dispatcher |
dev-orchestrator/ |
src/index.ts |
Local dev boot sequence |
| Root | docker-compose.yml |
Container orchestration |
- Canonical path:
/api/v1 - Legacy alias:
/api(returns deprecation header) - Infrastructure (unversioned):
/health,/metrics,/api/system/*
Defined in backend/src/config/constants.ts:
export const API_VERSION = 'v1';
export const API_BASE_PATH = '/api/v1';DevTrack uses Clerk for authentication — not custom JWT.
| Layer | Mechanism |
|---|---|
| Frontend | @clerk/clerk-react — sign-in/sign-up UI |
| Backend | @clerk/express middleware + CLERK_SECRET_KEY |
| User sync | ClerkUserSyncService creates/updates MongoDB user records on first request |
| SSE | POST /api/v1/auth/sse-handshake issues a short-lived token; GET /api/v1/events?token= |
Protected routes use authMiddleware. Admin and ops routes use adminMiddleware / opsAuditorMiddleware.
Browser tab limits make per-component SSE connections expensive. DevTrack uses a singleton coordinator:
- Client: One shared connection managed via
useSyncExternalStore; all tabs share the channel. - Server: Express registers keep-alive SSE clients in
sseHandler.ts. - PubSub: Job completions broadcast through a Redis channel; the SSE handler pushes events to connected clients.
Event types include platform sync completion, XP updates, notification delivery, and runtime state changes.
Powered by BullMQ on Redis. Workers run in separate processes (or containers in production).
| Queue | Purpose |
|---|---|
platform-sync |
LeetCode, Codeforces, CodeChef, GitHub crawlers |
xp-processing |
XP and level calculations |
streak-recalc |
Streak reconciliation |
notifications |
Notification dispatch |
profile-rebuild |
Profile stat reconstruction |
readiness-* |
Readiness snapshot and domain rebuilds |
resume-* |
Upload, ATS, semantic, embedding, credibility, variant, export |
system-maintenance |
Scheduled cleanup and reconciliation |
dataset-ingestion |
Dataset scan and ingest (ops) |
Development: npm run dev:workers or npm run dev:all
Production (docker-compose.prod.yml):
| Container | WORKER_TYPE |
Role |
|---|---|---|
api |
— (ENABLE_WORKERS=false) |
HTTP + SSE only |
worker-sync |
sync |
Platform crawlers |
worker-xp |
xp |
Gamification |
worker-orch |
orch |
Orchestration compensation |
worker-maint |
maint |
Maintenance |
Platform sync is idempotent: each external submission maps to platform + externalId to prevent duplicates.
23 route modules mounted in backend/src/routes/index.ts:
| Module | Mount | Description |
|---|---|---|
auth |
/auth |
Clerk user sync, SSE handshake |
dashboard |
/dashboard |
Stats, streaks, missions, GitHub, activity |
dsa |
/dsa |
Problems, stats, heatmap, submissions, contests |
activity |
/activity |
Feed, heatmap, focus sessions |
projects |
/projects |
Projects, tasks, GitHub sync |
profile |
/profile |
User profile, platforms, public profiles |
settings |
/settings |
User preferences |
platform-sync |
/platforms |
Manual and scheduled sync |
xp |
/xp |
Experience points |
streak |
/streak |
Streak tracking and freeze |
ops |
/ops |
Admin console, queues, DLQ, trust scores |
runtime-state |
/runtime-state |
Unified runtime state rebuild |
observation |
/observation |
Session replay, momentum, retention |
missions |
/missions |
Daily/weekly missions |
notifications |
/notifications |
In-app notifications |
onboarding |
/onboarding |
Onboarding step tracking |
daily-challenge |
/daily-challenge |
Daily challenge generation |
coaching |
/coaching |
AI coaching insights |
readiness |
/readiness |
Snapshots, intent, skill progress, DSA sub-routes |
resume (upload) |
/resume |
File upload and session status |
resume-intelligence |
/resume-intelligence |
ATS, variants, credibility, export |
recommendations |
/recommendations |
Intelligence reports |
analytics |
/analytics |
Event tracking, feedback, beta dashboard |
Beyond route modules, these internal engines power resume and readiness features:
intelligence-runtime/— Parsing, retrieval, credibility, replaymodules/ai/— Provider adapters (OpenAI/Gemini), embeddings, retrievalmodules/ml/— Ranking, governance, training pipelinesmodules/validation/— Load testing, scenario replay, benchmarking
60+ Mongoose models in backend/src/db/models/, grouped by domain:
| Domain | Models (examples) |
|---|---|
| User & Auth | user, userProfile, userSettings |
| DSA | dsaProblem, dsaSubmission, dsaProfile, dsaContest |
| Resume | resumeProfile, resumeSession, atsAnalysis, embedding |
| Readiness | readinessCore, readinessRoadmap, careerIntent |
| Activity | activityEvent, focusSession, dailyActivity |
| Gamification | userXp, userStreakLog, mission, dailyChallenge |
| Platform | connectedPlatform, platformStats, syncJob |
| Ops | deadLetterJob, aiResponseAuditLog, behavioralTelemetry |
Indexes are set up via backend/src/db/migrations/index-setup.ts.
React Router v7 with lazy-loaded pages and ErrorBoundary wrappers. Protected routes sit behind ProtectedLayout + AuthSyncGate (Clerk → /auth/me sync).
| Type | Tool | Examples |
|---|---|---|
| Server state | TanStack Query | Dashboard, DSA, readiness data |
| Client state | Zustand | Missions, gamification, UI overlays |
| Real-time | SSE hook | useSse, RealtimeLayer |
admin, ai, coaching, dashboard, dsa, gamification, intelligence-experience, notifications, onboarding, projects, readiness, realtime, resume-tracker, settings
16 service modules using axiosClient with Clerk token injection. Base URL from VITE_API_BASE_URL.
| Platform | Method | Coverage |
|---|---|---|
| Codeforces | REST API | Full submission history |
| GitHub | REST/GraphQL API | Repos, commits, stats |
| LeetCode | GraphQL (public) | Partial — API limitations |
| CodeChef | Web scraping | Stats only |
Optional GITHUB_TOKEN raises GitHub API rate limits.
| Endpoint | Auth | Purpose |
|---|---|---|
GET /health |
Public | Liveness |
GET /health/detailed |
Admin | Deep health |
GET /metrics |
Admin | Prometheus text |
GET /api/v1/ops/* |
Admin/ops | Queues, DLQ, cache, trust, AI audits |
Structured logging via Winston (backend/src/shared/logger.ts). Failed jobs route to the dead-letter queue (dlq.service.ts).
.github/workflows/pr-checks.yml on PRs to main/develop:
- TypeScript compile + ESLint
- Backend + frontend unit tests
- Integration tests (Mongo 6 + Redis 7 service containers)
- Docker image build verification
Production deploy workflows target Railway (backend) and Vercel (frontend).
- DEPLOYMENT.md — Production env vars and scaling
- CONTRIBUTING.md — Module conventions
- PROJECT_STATUS.md — Current gaps and health