This document describes the system as it exists today. Where something is a placeholder, a constant, or a known gap, it says so. Nothing here is aspirational.
Audience: contributors and maintainers who need to know what runs, where the boundaries are, and what they must not break.
PARTHA is a monorepo: a React frontend, a FastAPI backend, and a local filesystem plus relational database for persistence. Durable analysis uses the database as its queue behind an explicit control-plane boundary, driven by a daemon worker thread inside the API process; there is no external message queue or separate analysis service.
flowchart LR
UI["<b>Frontend</b><br/>React 18 · Vite · TypeScript<br/>apps/frontend"]
subgraph Server["Backend — FastAPI · apps/backend"]
direction TB
MW["<b>Middleware</b><br/>rate limit · security headers<br/>CORS · request ID"]
Routes["<b>Routes</b><br/>app/api/routes/"]
Services["<b>Services</b><br/>app/services/"]
Queue["<b>Control plane</b><br/>app/workers/control_plane.py<br/>claim · lease · reclaim"]
Worker["<b>Analysis worker</b><br/>app/workers/ · daemon thread"]
Extract["<b>Extractors</b><br/>app/extraction/"]
RI["<b>Repository Intelligence</b><br/>app/intelligence/<br/>sealed read model"]
Consumers["<b>Consumers</b><br/>analysis · graph · review · insights<br/>documentation · ai · reports"]
MW --> Routes --> Services
Services -->|"enqueue"| Queue
Queue -->|"lease"| Worker
Worker --> Extract --> RI
Services --> Consumers
Consumers -->|"read only"| RI
end
subgraph Store["Persistence"]
direction TB
DB[("Relational DB<br/>SQLite local<br/>PostgreSQL configured")]
Disk[("Filesystem<br/>STORAGE_PATH")]
end
subgraph Ext["External"]
direction TB
GH["GitHub<br/>git clone over HTTPS"]
LLM["AI providers<br/>OpenAI · Anthropic · Gemini<br/>OpenRouter · Ollama"]
end
UI --> MW
RI --> DB
Queue --> DB
Worker --> DB
Services --> Disk
Services --> GH
Consumers --> LLM
| Area | Responsibility |
|---|---|
src/app/ |
Shell, router, pages, and global stores. Every application route sits behind the RequireAuth guard. Exactly four routes are public: / (landing, which hosts the waitlist modal), /login, /register, and /oauth/complete — the provider redirect target, which must be reachable before the tab holds any access token. |
src/features/ |
Domain features with colocated hooks, components, and stores: repositories, upload, explorer, architecture, dependencies, review, documentation, AI workspace, insights, auth, settings, plus the public landing and waitlist surfaces that sit outside the auth guard. |
src/shared/ |
API clients, error mapping, feature-state helpers, reusable UI, config, and types. All backend calls go through the shared client, which owns access-token attachment and 401 handling. |
| Module | Responsibility | Must not do |
|---|---|---|
api/ |
HTTP boundary. Routes stay thin; deps.py wires every dependency. |
Contain business logic. |
services/ |
Application services: repository import, analysis orchestration, documentation, AI. | Parse repository files directly. |
intelligence/ |
The Repository Intelligence engine. Builds, persists, and reloads reusable repository facts. | Render API response shapes. |
parsers/ |
RepositoryParser walks the extracted tree and produces the file tree plus basic metadata. Legacy ingestion still uses the parser's heuristic symbol path. |
Produce feature-specific output. |
extraction/ |
The producer set behind the engine: syntax-aware python.py and typescript.py, dependency manifests.py and lockfiles.py, outbound service interactions (http.py), Docker Compose resources (iac.py), and the support_matrix.py that generates the public capability registry. |
Persist snapshots or answer product queries. |
analysis/ |
Architecture model — modules, layers, edges, request-flow hints. Consumer. | Read the filesystem. |
graph/ |
Dependency graph response model. Consumer. | Re-read dependency manifests. |
review/ |
Deterministic engineering-review.v2 findings and category assessment over one sealed snapshot. An unresolved reference becomes a finding only if the shared classifier in insights/relationship_diagnostics.py (also used by Insights and Architecture, #467) calls it a genuine gap. Consumer. |
Read the legacy JSON model, invent scores/grades, or emit a finding without exact same-snapshot evidence. |
insights/ |
Defined repository-insights.v1 metrics and breakdowns over one sealed snapshot. Consumer. |
Read legacy metadata, infer trends without history, or publish undefined health metrics. |
ai/ |
Context builder, prompt builder, orchestrator, provider registry/factory, five provider implementations, and the central egress policy/pinned sender. Consumer. | Parse repositories, read source files, or let a tenant expand provider destinations. |
reports/ |
ReportDocument intermediate representation, builders, and JSON/Markdown/HTML/PDF renderers. Second-order consumer — renders analysis output that already exists. |
Re-analyse a repository. |
auth/ |
Argon2 password hashing, HS256 access tokens, rotating refresh tokens with reuse detection. | — |
core/ |
Settings and validation, database engine, structured logging with redaction, request IDs, metrics, rate limiting, security headers. Timestamp columns use models/types.py::UTCDateTime: stored as UTC and always read back timezone-aware, so the API emits an explicit Z on SQLite as on PostgreSQL (#473). |
— |
storage/ |
Local filesystem storage for uploads and extracted/cloned repositories. Enforces path safety on extraction. | — |
workers/control_plane.py |
The queue boundary: which jobs are eligible, and who owns them — claiming, lease renewal, expiry, reclaim, and every ownership guard. | Analyse a repository, or read anything but analysis_jobs. |
workers/runner.py |
The in-process runner: worker identity, the poll/sweep loop, and shutdown. The same loop a standalone worker process would run. | Contain queue policy of its own, or be imported by the API for anything but start/stop. |
workers/analysis_worker.py |
Execution of an already-claimed job: the extraction pipeline, snapshot sealing, bounded retry, cancellation and stale-job reconciliation. During the extract stage it commits between facts (about every 0.25 s) so other writers, notably a cancel request, are never locked out of SQLite's single writer for the whole run (#469); it never does this while sealing. On failure or cancellation it discards the build's partial facts (intelligence/retention.py, #485). |
Decide who owns a job, or serve request-specific data. |
Both entry points converge on the same import path: land the source on disk,
compute immutable revision identity, parse the bounded file tree, and persist the
repository revision. A separate durable job then builds and seals the normalized
ri.v1 snapshot off the request path.
sequenceDiagram
participant UI as Frontend
participant API as FastAPI route
participant Repo as RepositoryService
participant Store as LocalStorage
participant Parser as RepositoryParser
participant Queue as Control plane
participant Worker as AnalysisWorker
participant RI as Extraction + Intelligence
participant DB as Database
UI->>API: POST /repositories/upload (archive)<br/>or POST /repositories/github (URL)
API->>Repo: import
Repo->>Store: save + extract archive, or shallow-clone
Note over Store: path traversal and symlink escape rejected on upload<br/>upload and clone size caps enforced
Store-->>Repo: repository root on disk
Repo->>Parser: parse(root)
Parser-->>Repo: FileTreeNode[] + RepositoryMeta + size
Repo->>DB: insert row (revision kind/value/ref + metadata + file_tree)
DB-->>UI: RepositoryResponse
UI->>API: POST /analysis/{id}/start
API->>DB: insert queued analysis_jobs row
API-->>UI: queued + job id
Worker->>Queue: claim
Queue->>DB: compare-and-swap queued -> running + lease
Queue-->>Worker: JobLease
Worker->>Queue: renew lease by stage (reports cancellation)
Worker->>RI: extract and resolve repository facts
Worker->>DB: persist and seal normalized ri.v1 snapshot
Clone/archive extraction and initial file-tree parsing run synchronously during
import. POST /analysis/{id}/start durably enqueues the analysis and returns
immediately. A daemon worker thread in the API process claims jobs through the
control plane, reports progress at completed stage boundaries, seals the
normalized snapshot, and serves every product consumer from that immutable read
model.
The API process hosts that worker but does not own the queue. Claiming, leases,
expiry, reclaim and ownership guards live in app/workers/control_plane.py, and
the poll/sweep loop lives in app/workers/runner.py, so the same components
would drive a standalone worker process without changing the claim/lease
contract. Running analysis in a separate process is not implemented today.
| Store | Holds | Notes |
|---|---|---|
| Relational DB | users, refresh_tokens, repositories, repository_lineages, analysis_jobs, ai_provider_configs, ai_conversation_messages, the account-access tables (approved_emails, invite_tokens, oauth_identities, oauth_flow_states, oauth_pending_links, account_deletion_audits), waitlist_entries, and the normalized ri_* snapshot tables |
SQLite by default for local development; PostgreSQL is supported through DATABASE_URL. Analysis jobs and their worker leases are durable database state. |
account_deletion_audits |
A minimal, non-PII record that an account deletion happened: the deleted user id, status, timestamps, and any failure reason. | Retained after the user row and its data are removed, so a deletion remains auditable without preserving the account. |
approved_emails |
The registration gate: which email addresses may create an account. | Registration is allowlisted in every environment except development (the default APP_ENV when running from source), where any address is auto-approved so local use stays frictionless. Outside development the one exception is a genuinely empty instance: the first account registered is auto-approved and becomes that instance's owner. No address is pre-approved by a migration (the seed row 0016 once inserted was removed in #465). Registration does not verify email ownership, so an approved address can be claimed by whoever registers it first, and there is no admin role: rows are added with scripts/approve_email.py. Account requests accept private-network domains such as corp.local. invite_tokens is the superseded invite-code table, retained for historical continuity and no longer a gate. |
oauth_identities, oauth_flow_states, oauth_pending_links |
Linked Google/GitHub provider identities, in-flight authorization state, and pending link confirmations. | OAuth is inert unless provider credentials are configured. It never creates an account — an unknown identity is returned to the invite-gated registration flow. |
repository_lineages |
Owner-scoped grouping of repeated imports of the same repository and branch, with 1-based never-reused sequence allocation (RFC-0002). | Uploads and unresolved GitHub imports stay unlineaged: their lineage fields are null and no synthetic lineage is created. |
waitlist_entries |
Landing-page waitlist signups: normalized email and optional name. | Written by POST /waitlist, the one public write route in the API. Deliberately unauthenticated; guarded by the auth rate-limit class and an existence-concealing response. |
repositories.revision_kind, revision_value, revision_ref |
Exact imported source identity: Git commit + resolved ref, or upload archive hash. | revision_value is indexed and immutable; a moving branch name is metadata, never identity. |
repositories.repo_metadata (JSON column) |
Import/parser metadata; historical rows may also retain a legacy/unverified intelligence value. |
New analysis does not write the legacy value, and executable product consumers ignore it. New imports no longer stash commitSha here. |
ri_snapshots, ri_nodes, ri_edges, ri_assertions, ri_observations, ri_evidence, ri_derivations, ri_diagnostics |
Revision-addressed normalized ri.v1 artifacts, provenance, lifecycle state, and canonical hash. |
Durable analysis runs the Python/TypeScript/manifest producers and resolver, then seals a snapshot. Architecture, authentication explanation, Dependencies, Engineering Review, Insights, Documentation, exports, and AI context consume it. |
repositories.file_tree (JSON column) |
The parsed file tree. | Serves the explorer. |
ai_provider_configs |
One row per user: provider, model, base URL, and the Fernet-encrypted API key plus its last four characters. | Owner-scoped; the plaintext key is never stored or returned. A stored endpoint remains unusable unless it satisfies the current deployment egress policy. |
ai_conversation_messages |
One row per AI Workspace turn: role, content, optional citations, and an explicit sequence. |
The AI Workspace thread is durable, not ephemeral. One ordered thread per owner per repository, so history survives navigating away and returning. UNIQUE(owner_id, repository_id, sequence) is the concurrency guard; the repository foreign key cascades, so deleting a repository deletes its turns. A cross-owner read resolves to the same 404 as a missing repository. |
Filesystem (STORAGE_PATH) |
Extracted archives and cloned repositories; uploaded archives (deleted after extraction). | Repository source is read from here on demand for file preview. What accumulates, what is discarded automatically and the operator cleanup script are in Storage and retention. |
AI provider configuration is per-user and encrypted at rest. Each user's API key is encrypted with a Fernet key from AI_ENCRYPTION_KEY (required outside development/test), decrypted only in-process at request time, and injected per request — so a query runs against the caller's own key and bill, never a shared one.
sequenceDiagram
participant UI as Frontend
participant API as /auth
participant Auth as AuthService
participant DB as Database
UI->>API: POST /auth/register or /auth/login
API->>Auth: verify (Argon2)
Auth->>DB: persist rotating refresh token (hashed)
API-->>UI: access token (HS256, in body)<br/>+ refresh token (httpOnly cookie, path=/auth)
Note over UI: access token held in memory,<br/>attached as Authorization: Bearer
UI->>API: POST /auth/refresh (cookie)
Note over Auth: reuse of a spent token revokes<br/>the entire token family
API-->>UI: new access token + rotated refresh cookie
The access token is short-lived (15 min default); the refresh token lasts 14 days and rotates on every use. Refresh-token reuse revokes the whole family. AUTH_SECRET_KEY is required and length-checked outside development/test; in dev it falls back to a fixed insecure value.
Cookie scope and the same-site deployment requirement. The refresh cookie is HttpOnly, Path=/auth, SameSite=Lax, and Secure outside development/test (_set_refresh_cookie in app/api/routes/auth.py). SameSite=Lax is a deliberate CSRF control, and it means the frontend and the API must be served from the same site (same registrable domain) in any deployment — e.g. one origin behind a path prefix, or app.example.com + api.example.com. If they are cross-site, the browser withholds the cookie from the background POST /auth/refresh, so the session cannot be re-established and the user is bounced to login on every reload. Locally this only bites if the frontend origin and VITE_API_URL disagree on host (localhost vs 127.0.0.1); keep both on the same host.
Enforcement. Every non-public route requires a valid access token. The /repositories, /analysis, /ai, /documentation, and /export routers each apply get_current_user at the router level, so a request with no token — or an invalid one — is rejected with 401 before reaching a handler, and a newly added route under those prefixes is protected by default. The pre-auth get_current_user_or_default fallback and its X-Dev-User header were removed in E1.3; there is no anonymous seed-user bucket. Data is additionally owner-scoped in the service layer (below), so authentication and authorization are enforced independently.
The public surface, stated exactly. Three groups of routes are reachable without a token, deliberately: the session-establishing /auth routes (register, login, refresh), the OAuth start/callback pair, and POST /waitlist — the only public write route in the API. It exists because the landing page must accept a signup before anyone has an account or an invite. It writes one row to waitlist_entries, returns an identical response for a new signup, a repeat signup, and a concurrent collision (so membership of the list is never disclosed), and its abuse guard is a per-minute budget in the auth rate-limit class rather than authentication.
OAuth sign-in. Google and GitHub sign-in and account linking live under /auth/oauth and are inert unless provider credentials are configured — GET /auth/oauth/providers returns only providers with a real client id and secret, so an unconfigured deployment offers none. OAuth authenticates an existing account; it never creates one. A provider identity that matches no account is returned to the invite-gated registration flow, because AuthService.register() is the only account-creating path in the backend and it enforces the approved_emails allowlist.
This is the one architectural invariant of the system.
Repository Intelligence is the single repository-understanding boundary. Consumers must not independently parse repositories or construct a second source of repository truth.
The AI subsystem gets no exemption from this:
AI is a consumer of Repository Intelligence and must not independently parse or reinterpret repositories.
Repository source is read only by the bounded import/analysis pipeline
(RepositoryParser and AnalysisWorker) and by
RepositoryService.read_file, which serves the explorer's path-checked preview
and feeds no analysis.
Every repository-derived consumer uses the owner-scoped SnapshotQueryService
and requires a sealed snapshot for the current repository revision. Historical
legacy JSON may remain stored but is ignored. See
REPOSITORY_INTELLIGENCE.md for what each path
extracts and what it does not.
| Consumer | Reads | Produces |
|---|---|---|
analysis/ |
sealed ri.v1 nodes, edges, assertions, diagnostics, evidence |
Architecture nodes, edges, layers, request-flow hints |
review/ |
sealed ri.v1 diagnostics plus exact evidence and manifest identity |
Supported findings and assessed/not-assessed category matrix; no scores |
insights/ |
sealed ri.v1 nodes, edges, diagnostics, evidence and extractor set |
Defined counts, ratios and breakdowns; no inferred trends |
graph/ |
sealed dependency nodes, declarations, resolved edges, diagnostics, evidence | Direct dependency graph with explicit not-computed assessments |
services/documentation_service.py |
shared sealed structural projection and snapshot/revision identity | Markdown / HTML documentation |
ai/repository_context.py |
shared sealed structural projection, without source bytes, plus the recent turns of the stored conversation thread | Preview RepositoryContext → PromptBundle for a configured provider |
reports/ |
snapshot-backed analysis and documentation output | JSON / Markdown / HTML / PDF |
Two different things answer "what language / framework is this repository", and they must not be confused.
RepositoryResponse.meta (RepositoryMeta) |
Repository Intelligence | |
|---|---|---|
| Source | RepositoryParser, at import time; language, framework and entry point are then refreshed from the snapshot when analysis completes (ArchitectureAnalyzer.stack_summary, #475), so an analysed repository's meta and its Architecture summary agree |
sealed ri.v1 snapshot, from durable analysis |
| How | file-extension Counter; _detect_framework / _detect_entry_point / _detect_package_manager / _detect_license filename heuristics |
evidence-backed extractors + deterministic resolver |
| Provenance | none | path + span + producer/version on every fact |
| Owner-scoped | via the repository row | yes, in the query service |
| Can be wrong / disagree with the other | yes | it is the product's source of truth |
RepositoryMeta exists so the repository list, dashboard, and detail header can show something
before analysis has run or when it is stale. It is not Repository Intelligence. Any
intelligence-oriented surface — Architecture, Dependencies, Insights, the authentication
explanation, AI context — reads the snapshot query API, never meta.
RepositoryParser is not redundant with the engine: it does the bounded tree walk, produces
file_tree for the Explorer, enforces the file-count and symlink limits, and computes the size the
import path persists. Its heuristic meta is a byproduct of that walk, not a parallel analysis
engine, and no consumer under the Repository Intelligence boundary reads it.
Until analysis completes, meta.language / meta.framework remain the import-time heuristics, and
the repository pages do not label them as such. Once analysis has completed, the worker replaces them
with the snapshot-derived values where the snapshot supplies one (a value it cannot supply, such as an
unknown framework, keeps the import-time one). Repositories analysed before that change keep their
old values until re-analysed.
| Dependency | Used for | Failure mode |
|---|---|---|
git (system binary) |
Shallow-cloning public GitHub repositories. | Import fails with a normalized external-service error; the partial clone is cleaned up. |
| GitHub (HTTPS) | Source for public repository import. Only https://github.com/owner/repo URLs are accepted, optionally with /tree/<branch> (split off as the branch, #481); other page URLs are refused. No authentication, so no private repositories. |
Timeout and size caps abort and clean up. |
| AI providers | Answering repository questions. Configured per user with an encrypted API key; destinations are centrally policy-checked and DNS-pinned. | A missing, stale, or policy-denied configuration produces a normalized error; the rest of the system is unaffected. |
| PostgreSQL, Redis | Optional configured services and CI integration. Redis backs the rate limiter when RATE_LIMIT_BACKEND=redis. |
Local development uses SQLite and the in-memory rate limiter; neither service is required. |
flowchart TB
subgraph Untrusted["Untrusted input"]
Archive["Uploaded archive"]
RepoURL["GitHub URL / branch"]
Source["Repository source content"]
end
subgraph Backend["Backend process — trusted"]
Validate["<b>Validation</b><br/>URL allowlist · branch charset · size caps<br/>path traversal + symlink rejection (upload)"]
Engine["<b>Parse + Repository Intelligence</b>"]
Thread[("Stored conversation<br/>ai_conversation_messages<br/>both turns of every query")]
end
subgraph Out["Egress"]
Providers["AI providers"]
Logs["Logs"]
end
Archive --> Validate
RepoURL --> Validate
Validate --> Engine
Source --> Engine
Engine -->|"structure + file paths only —<br/>never source content"| Providers
Engine -->|"redacted; never repository content"| Logs
Engine --> Thread
Thread -->|"recent turns replayed as context"| Providers
- Uploaded archives and cloned repositories are untrusted input. Upload extraction rejects path traversal and symlink escape; the GitHub clone tree walk rejects symlinked files and directories the same way (#182); upload and clone sizes are capped; only allowlisted archive suffixes are accepted. Repository content is never executed — it is only read as text.
- Repository source never leaves the process. The AI context builder passes structure and metadata only: languages, frameworks, modules, dependency names, and file paths. No file contents and no line numbers are sent to any provider, and the system prompt explicitly tells the model not to claim line numbers or quote code it was not given.
- Conversation turns do leave the process, and are retained. Alongside that structural context, the most recent turns of the AI Workspace thread are replayed to the provider so a follow-up question resolves. Those turns are user-authored text, not repository source, but they are egress and they are persisted in
ai_conversation_messagesrather than discarded at the end of a session. Interface copy must describe this accurately: the workspace does not forget, and telling a user otherwise would be a false privacy assurance. - Logs are redacted. Keys containing
api_key,apikey,authorization,password,secret, ortokenare redacted from structured log extras. Repository contents and credentials must never be logged. - The authenticated-user boundary is enforced. Every non-public route requires a valid token, and every repository lookup is owner-scoped in the service layer, so a caller sees only their own data. The deliberate exceptions are the session-establishing
/authroutes, the OAuth start/callback pair, andPOST /waitlist— the only unauthenticated write, which touches nothing butwaitlist_entriesand is rate-limited rather than authenticated. Provider API keys are encrypted at rest and injected per user. Provider destinations are selected by a deployment-owned egress policy, not by a tenant: configured endpoints must match an exact allowlist, DNS answers are checked again immediately before the request and pinned to the connection, redirects are denied, and policy errors omit destination details. Rate-limit budgets are keyed on the validated user id for authenticated requests (falling back to the client IP otherwise), so one user cannot spend another's budget. See AI provider egress policy.
These are properties of the system as built, not a wish list.
- Production classification remains partly heuristic. File roles, derived modules, and layers are inferred from observed path segments and filenames and are labelled as heuristic.
- Line-level evidence is surface-dependent. Syntax-aware Python and TypeScript facts can carry validated spans, but Documentation uses structural facts and free-form AI receives no source bytes, so provider answers have no automatic citations.
- The graph store is the sole product read model. Durable analysis populates immutable normalized snapshots, and every product consumer requires the latest owner-scoped snapshot matching the current revision. Missing or stale snapshots return 404 without fallback.
- Analysis is whole-repository, and its worker is still in-process. It runs in a durable, cancellable background job with bounded retry and stale-worker recovery, claimed through an explicit control plane, but incremental re-analysis is not implemented and no standalone worker process is deployed — one API process runs one worker. Import extraction and file-tree parsing remain synchronous.
- The rate limiter trusts only the direct socket peer for unauthenticated requests.
X-Forwarded-Foris deliberately ignored, so behind a reverse proxy every unauthenticated client shares one IP budget until a trusted-proxy allowlist is designed. Authenticated requests are keyed per user and unaffected. - Dependency coverage is narrow. Three manifest formats plus two lockfile formats (
package-lock.json,poetry.lock), whose exact pins are recorded as resolutions on the same dependency identity rather than as direct edges. There is no transitive resolution and no vulnerability or outdated-version scanning. The API exposes explicitnot_computedassessment statuses and does not emit a clean result or count without a scanner. - Frontend assurance remains focused. Vitest covers shared and feature behavior, and a disposable Playwright acceptance suite exercises the defined Architecture, Engineering Review, and Insights browser journeys. This is not comprehensive end-to-end product coverage.
These are missing guarantees in the system as built. They are not scheduled work, and this document does not commit to when or whether any of them change.