feat: the URL-first pipeline, phases 2 to 4

[ralyodio]

feat: the URL-first pipeline, phases 2 to 4

Paste a company URL — or a hundred — and get approval cards. Phases 2,
3 and 4 of docs/url-first-pipeline.md, on top of the queue from #19.

Phase 2, the site provider (packages/providers/src/site/).

Hybrid extraction, as decided. The deterministic pass reads what a site
published about itself — JSON-LD, OpenGraph, rel=me, outbound links to
known networks — and the model runs only where that came back empty. A
page with decent markup never costs a token, and the grounding rule is
satisfied by construction: a value lifted from a site's own structured
data has evidence attached to it.

robots.txt is read and obeyed, the user agent names the product and
carries a contact URL, redirects are followed by hand so each hop is
re-checked against that origin's rules, and body size, redirect depth
and timeouts are all bounded. That is in the first commit rather than a
later hardening pass because the policy engine permits website/observe
as "permitted public web retrieval" — and what makes a retrieval
permitted is partly that the site said it was.

Phase 3, company-first intake.

POST /prospects/by-url takes one URL or up to a hundred and answers
202 with a batch id; GET /batches/:id reports each URL's state rather
than a bare count, because after a hundred URLs the question is which
ones failed and why. URLs dedupe by host, so example.com, www. and the
bare domain are one company. A URL already queued is reported as a
duplicate, never an error: pasting the same address twice is a normal
thing for a human to do and must not fail the other ninety-nine.

runPipeline splits: runPipelineForCandidate is the chain from a
candidate onwards, and the GitHub path is now one caller that produces
such a candidate. Provenance and identity source types follow the
provider that actually supplied the value instead of being hardcoded to
GitHub's — with a crawl in the mix, that would have labelled a scraped
name as an API fact and mis-classified what may be retained (PRD §35).
The old GitHub anchor rule generalises to anchorNetwork: a crawl has
no anchor, because nobody named on a company page is proven to be
that person.

Phase 4, fan-out — partial, deliberately.

findIdentities asks every configured provider what else it can vouch
for, then hands the lot to the existing resolver. It gathers claims and
merges nothing; keeping those apart is what stops a provider's
confidence becoming the product's. A provider is only asked about a
network we already hold a handle for — handing a display name to a
network with no verification and taking the first hit is how the wrong
human reaches an approval queue.

Bluesky ships because its AppView needs no key and no contract, so it
can be built and tested without a commercial decision attached. X and
the enrichment vendors are not here: both need credentials I cannot
test against, and an adapter that has never made a real call is code
pretending to be a feature.

62 new tests. Verified 0008 applies incrementally on an existing
database, as the container does at boot.