Repository navigation
Stop outreach mailing the same inbox once per colleague - #34
Merged
Merged
Conversation
Every rate limit was keyed on person_id, which is the right key for "how often do we contact this human" and the wrong one for "how much mail does this mailbox get". A prospect with no personal address falls back to their company's shared inbox, so N prospects at one company are N separate people — each comfortably inside its own weekly limit — and one support@ mailbox receives N messages. In production that meant 24 outbound emails reaching 6 distinct addresses, with support@canny.io alone receiving 14. Nothing was ever contacted twice by its own key, which is why no existing gate saw it. Three things were missing, not one: - The delivered address was never recorded, so it could not be counted. `interactions` now stores contact_address and shared_inbox, written by both the automated and the by-hand send paths. Migration 0011 backfills it using the sender's own resolution order; all 24 existing rows resolve to a shared inbox, so the limits protect those mailboxes immediately. - The policy engine had no address-keyed limits. Adds rate_limit_address and an address cooldown alongside the per-person ones. A personal mailbox is unaffected — for it the two keys are the same. - Nothing ever wrote an inbound interaction, so a reply could not stop anything and the "replied" figure in analytics was structurally zero. Adds POST /people/:id/replied and a conversation_open gate: a contact who has answered is out of cold outreach, and a follow-up needs a human. A reply to a shared inbox protects that person's colleagues too. Also, because both are needed to use this queue at all: - MODEL_CHAIN orders the provider fallback from the environment, so the cheap model can lead without a deploy and an omitted provider is not built at all. - POST /recommendations/drafts/backfill writes the drafts a queue is missing, bounded by `limit` because every card is a model call. The approvals page had 74 pending recommendations and no drafts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The approvals page looked like a wall of blank cards, and the obvious reading — "drafting is broken" — was wrong. Production has 77 recommendations without a draft, of which 75 are `refresh_research`: internal actions that have no message by definition and never will. Only 2 are genuinely undrafted emails. So the expensive mistake was not failing to draft, it was drafting everything. As first written this route would have spent 75 model calls producing nothing and reported each one as a refusal — on a queue whose whole problem is that it costs too much to run. Restricts the backfill to OUTBOUND_ACTION_KINDS, and tests that a research card produces no model call at all rather than a cheap one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ralyodio
added a commit
that referenced
this pull request
Aug 16, 2026
PR #34 added a `conversation_open` gate that refuses to write to a contact who has answered — but nothing could tell it that anyone had. The product sent over SMTP and read no mailbox, so every reply was invisible, the funnel's `replied` count was structurally zero, and the queue kept offering prospects who were already mid-conversation. SMTP cannot close that gap. Its whole vocabulary is MAIL FROM / RCPT TO / DATA; there is no verb for "what arrived". Reading is IMAP, on a different port, with the same credentials the mailbox was already connected with — so this adds no second password, and one provider choice fills in both halves. - `packages/email/src/imap.ts` turns a mailbox into a list of messages. - `packages/pipeline/src/receive-email.ts` decides what they mean. - Presets carry IMAP coordinates for Gmail, Microsoft 365, Fastmail, Forward Email and Zoho; the generic entry lets both be typed. - The worker polls on its own five-minute clock, *before* the send sweep — a reply noticed this tick has to stop a message that would otherwise go out on the same tick. Most of the care is in what must not be recorded. The two failure modes are not symmetric: missing a reply means we mail someone who answered, which is visible and recoverable, while inventing one silently removes a prospect from outreach for good on the strength of an out-of-office. So auto-replies, bounces and bulk mail are classified and skipped, mail from an address we never wrote to is left alone, and migration 0012 adds a Message-ID key so an overlapping poll window cannot double-count. A reply to a shared inbox is recorded against that address, which — because `conversationOpen` matches on address as well as person — protects every colleague written to there. Attributing it to one of them is a guess; that the mailbox answered is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ralyodio
added a commit
that referenced
this pull request
Aug 16, 2026
* feat: read replies over IMAP, so the reply gate has an input PR #34 added a `conversation_open` gate that refuses to write to a contact who has answered — but nothing could tell it that anyone had. The product sent over SMTP and read no mailbox, so every reply was invisible, the funnel's `replied` count was structurally zero, and the queue kept offering prospects who were already mid-conversation. SMTP cannot close that gap. Its whole vocabulary is MAIL FROM / RCPT TO / DATA; there is no verb for "what arrived". Reading is IMAP, on a different port, with the same credentials the mailbox was already connected with — so this adds no second password, and one provider choice fills in both halves. - `packages/email/src/imap.ts` turns a mailbox into a list of messages. - `packages/pipeline/src/receive-email.ts` decides what they mean. - Presets carry IMAP coordinates for Gmail, Microsoft 365, Fastmail, Forward Email and Zoho; the generic entry lets both be typed. - The worker polls on its own five-minute clock, *before* the send sweep — a reply noticed this tick has to stop a message that would otherwise go out on the same tick. Most of the care is in what must not be recorded. The two failure modes are not symmetric: missing a reply means we mail someone who answered, which is visible and recoverable, while inventing one silently removes a prospect from outreach for good on the strength of an out-of-office. So auto-replies, bounces and bulk mail are classified and skipped, mail from an address we never wrote to is left alone, and migration 0012 adds a Message-ID key so an overlapping poll window cannot double-count. A reply to a shared inbox is recorded against that address, which — because `conversationOpen` matches on address as well as person — protects every colleague written to there. Attributing it to one of them is a guess; that the mailbox answered is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(replies): move the funnel, not just the interaction row The funnel is built from campaign_people.status and lead_stage_events, so setting only interaction_state recorded the reply everywhere except the chart that exists to show replies — the prospect sat in Contacted having answered. Routes through recordStatus, per campaign membership. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(migrations): renumber inbound dedupe to 0013 0012 is taken by listening-per-campaign's 0012_listening_targets.sql. The ledger is keyed on filename so both would apply, but two 0012s is exactly the ambiguity that makes an out-of-order apply hard to reason about later. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ralyodio
added a commit
that referenced
this pull request
Aug 17, 2026
…ly sends (#48) #34 added address-keyed rate limits to the policy engine to stop one `support@` receiving a message per colleague. The gates are right. Only half the product passes them. `evaluatePolicy` takes `actionsToThisAddressThisWeek`, `hoursSinceLastActionToAddress` and `addressShared` as **optional** inputs, documented as "omitted when the caller could not resolve an address, in which case these gates simply do not fire". `apps/api/src/app.ts` — the human approval route — fills them in. `runAutopilot` never did. So the limits protected the path where a person is already looking at the message, and not the unattended one that sends at volume. Production shows the seam exactly. Every duplicate predates the fix except one: `info@thebarcelonafeeling.com` was written to at 14:03, #34 merged at 17:45, and autopilot wrote to it again at 18:39 — hours after the fix was live, through the half of the code the fix never reached. Optional inputs are why this was quiet. Omitting them is not a type error and not a runtime error; it is a gate that silently evaluates to "no opinion". The same shape will hide the next one, so the fields are now passed unconditionally rather than spread in behind a check — there is always an address by this point, since `pickEmailRecipient` returning nothing already skipped the row. Counted from `interactions.contact_address` — the mailbox a message reached — rather than from `actions`, which records who it was addressed to. For a shared inbox those are different questions and only the first can see fourteen colleagues arriving at one address. It also means a message sent by hand from the approval queue binds the automated path and vice versa: a mailbox does not care which half of the product wrote to it. Three tests, built from the production shape — two colleagues at one company, neither with a personal address, both inside their own per-person limits: - one message goes out, not two, and the one held back explains itself in terms of the address rather than the person - two people with their own addresses still get two messages, so this does not quietly become "one email per company" - an interaction recorded by the manual route stops the automated one 933 tests pass. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened
You sent the same email twice to a lead who had already replied. The real number is worse: 24 outbound emails reached 6 distinct addresses, and
support@canny.ioalone received 14 of them.Every rate limit is keyed on
person_id— the right key for "how often do we contact this human", the wrong one for "how much mail does this mailbox get". A prospect with no personal address falls back to their company's shared inbox, so 14 prospects at Canny were 14 separate people, each comfortably inside its own weekly limit, all delivering to one mailbox. Nothing was ever contacted twice by its own key, which is why no existing gate saw it.Three gaps, not one
The delivered address was never recorded, so it could not be counted.
interactionsnow storescontact_addressandshared_inbox, written by both the automated and the by-hand send paths. Migration0011backfills it using the sender's own resolution order — all 24 existing rows resolve to a shared inbox, so the limits protect those mailboxes the moment it runs.The policy engine had no address-keyed limits. Adds
rate_limit_addressand an address cooldown beside the per-person ones. A personal mailbox is unaffected: for it the two keys are the same thing.Nothing ever wrote an inbound interaction, so a reply could not stop anything — the
repliedfigure in analytics was structurally zero. AddsPOST /people/:id/repliedand aconversation_opengate. A contact who has answered is out of cold outreach; a follow-up is denied unless explicitly flagged, and then still needs human approval. A reply to a shared inbox protects that person's colleagues too.Also here, because the queue is unusable without them
MODEL_CHAINorders the provider fallback from the environment (e.g.openai,anthropic), so the cheap model can lead without a deploy. A provider left out of the list is not built at all, so the cheap chain cannot quietly fall through to an expensive one.POST /recommendations/drafts/backfillwrites the drafts a queue is missing, bounded bylimitbecause every card is a model call. The approvals page had 74 pending recommendations and zero drafts.Not done
No inbound polling — the product still sends over SMTP and reads no mailbox, so replies have to be recorded through the new route until IMAP ingestion exists. That is the one thing standing between this and the gate being automatic.
Verification
apps/api/src/shared-inbox.test.tsreproduces the production scenario end to end: two colleagues sharing one inbox, second send refused, nothing on the wire.🤖 Generated with Claude Code