Break the research loop: an untitled name is still a fact worth citing

[ralyodio]

The root cause was one condition

The queue held 73 pending cards, all refresh_research, none with a signal. That reads as a broken composer. It was a closed cycle.

storeSiteSignal returned early when a crawled person had no job title. The correlation across all 208 people in production is exact:

	has signal	no signal
has title	131	0
no title	1	76

Every one of the 73 pending cards was an untitled person. No signal → generateRecommendation refuses outbound (outreach with no reason is what the product exists to avoid) → falls back to refresh_research → which nothing executed. The card that existed because there was nothing to say could never produce anything to say.

The gate was never a policy decision. "Listed as Office Manager on the company website" and "named on the company website" are equally unrelated to buying intent — the title was load-bearing only because the summary string needed it to be grammatical.

What changed

An untitled listing is a signal, at relevance 0.35 against 0.5 for a stated role. Weight is decay × confidence × relevance against a 0.15 floor, so 0.315 fresh and 0.205 at thirty days will trigger, and by sixty days it stops — the right shape for a thinner pretext.

The crawl follows the pages that name people. A homepage describes the company, links to /team, and names nobody. Now up to three same-origin links whose path or anchor text looks like a team/about/contact page. Bounded deliberately: a fixed shortlist rather than a spider, robots.txt fetched once and reused, and the model asked at most once for the whole site after the cheap deterministic reads are merged — so four pages cost four fetches, and the expensive pass runs only if all four together named nobody. Merging prefers the titled record.

Approving research now causes research. refresh_research had no executor anywhere; the runner knows four job kinds and this was not one. It now queues a crawl_site for the company's own domain. A person with no company on file says so instead of queueing a job that cannot run.

Recommendations are reconciled, not just inserted. Re-crawling would otherwise stack a second pending card on the same person. Only pending internal actions are superseded — an outbound card may carry a drafted message and a half-made human decision, and replacing that silently would discard real work.

A signal without a channel keeps the person visible. Reachable only on website means no permitted engagement action, and returning nothing made the lead vanish entirely. It now falls back to the passive set, which is the honest description: we know something, and have nowhere to use it.

Migration 0015

Backfills the signal the fixed code would have written, for the 75 already-crawled people who have none. Dry-run against production: 75 rows, 30 of those people reachable via a company inbox.

Dated to the person's created_at rather than now, so decay treats the evidence as the age it really is instead of resetting the clock on something nobody re-checked.

It writes no recommendations, so it sends nothing. Autopilot's candidate query only ever selects pending action = 'send_email' rows; the 73 are refresh_research. Cards only become sendable when a crawl regenerates them — which is a separate, deliberate step.

Blast radius, stated plainly

All 73 cards sit in trusted_automation (autopilot) campaigns, one workspace has a connected mailbox, and the autopilot daily cap defaults to 25. So once re-crawls run, up to 30 of these become real cold emails sent without further human approval. That is the intended product behaviour and it is worth knowing before the re-crawls are triggered, which is why this PR does not trigger them.

Verification

• bun run check — format, typecheck, 867 tests pass, 0 fail (up from 855)
• 8 new tests in research-loop.test.ts: untitled signal, evidence content, relevance, team-page title promotion, inbox on a followed page, footer social kept, dead sub-link tolerated, supersession
• 4 new tests in research-approve.test.ts: crawl queued with the right campaign, no-domain path, outbound queues nothing, card still approved
• Migrations run in every seeded test, so 0014/0015 are proven to apply

🤖 Generated with Claude Code