Skip to content

Email that sends, multiple products, and listening on public feeds - #33

Merged
ralyodio merged 3 commits into
mainfrom
worktree-email-connect
Aug 16, 2026
Merged

ralyodio merged 3 commits into
mainfrom
worktree-email-connect

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

Three things, in the order they were hit.

1. Email could not send at all

No connected email account, so this must be done manually. was not a configuration problem. The policy engine asks hasConnectedAccount before permitting an outbound action; the API answered by counting rows in integration_accounts, and nothing in the product could write one. Every email recommendation evaluated to manual_only, approving returned 409, and the product spent its time drafting messages it had already decided it could never send.

Autopilot asked a different question — "is a mailer configured?" — and sent the identical message unattended. Two paths, two answers, and the one a human used was the broken one.

  • A workspace can connect its own SMTP mailbox. Verified against the real server before storing, encrypted at rest (AES-256-GCM, SECRET_ENCRYPTION_KEY), never returned by any route.
  • Approving an email sends it. There was nothing for a reviewer to do between approving and sending. The send is inline so a rejected login is visible, not an approved action rotting in a queue.
  • One set of send mechanics, shared by autopilot and the approval queue, so the two cannot drift apart again.

Also fixes a rate-limit bug the tests found: re-checking policy at execution time counted the action being executed against its own per-prospect weekly limit, so with the default of one action per prospect per week every explicit send was refused by its own existence.

2. Setup could only describe one product

The schema always allowed several — offerings is keyed by workspace and every campaign names the offering it sells — but every read was ORDER BY created_at ASC LIMIT 1. Describing a second product overwrote the first and repointed the single campaign at the new offering. A company with two products could sell one, and the switch was silent.

Each product now owns its offering, its voice (a security tool and a design tool sounded identical, sharing one voice profile) and its campaign + ICP filters (a second product was scored against the first product's titles).

Archiving rather than deleting: an offering is referenced by campaigns → recommendations → actions → interactions, i.e. the record of every message already sent.

3. Outreach only knew website + GitHub

Every intake started from a company. That finds buyers of developer tooling and almost nobody else. Listening inverts it: a campaign's keywords (plus competitor names already captured at setup) are searched against public feeds, and each match becomes the person who wrote it and the signal that made them worth noticing.

Four sources chosen for having no gatekeeper: Reddit (best non-technical coverage; scoping to trade subreddits is the whole targeting story), RSS/Atom (rss was in the network list since the first commit with nothing behind it), Bluesky post search, Nostr (new network + matrix entries).

Classification is deterministic regex, not a model call — a listening run that produces nothing whenever a provider is capped is a feature that appears broken at random.

Listening does not make strangers contactable. A handle is not an identity, so these people are written at 0.35 confidence, below the 0.85 outreach floor, and the policy engine refuses outbound against them. Intended, not a gap.

Facebook and Nextdoor are absent on purpose: neither exposes a public search, and scraping violates their terms. LinkedIn stays research-only for the same reason.

Deploy notes

  • Set SECRET_ENCRYPTION_KEY (32 bytes, base64) or connecting a mailbox is refused and outreach falls back to the platform sender.
  • Listening is off unless LISTEN_SOURCES names sources. See .env.example.

Checks

format:check, typecheck, bun test (726 pass, +102 new) and the PWA build all pass locally.

🤖 Generated with Claude Code

ralyodio and others added 3 commits August 16, 2026 14:50
"No connected email account, so this must be done manually." was not a
configuration problem. The policy engine asks `hasConnectedAccount` before
permitting an outbound action, the API answered by counting rows in
`integration_accounts`, and nothing in the product could ever write one. So
every email recommendation evaluated to `manual_only`, approving returned 409,
and the product spent its time drafting messages it had already decided it
could never send.

Autopilot asked a different question — "is a mailer configured?" — and sent the
identical message unattended. Two paths, two answers, and the one a human used
was the broken one.

Three changes, in the order they matter:

- **A workspace can connect its own SMTP mailbox.** Verified against the real
  server before it is stored, encrypted at rest with AES-256-GCM under
  `SECRET_ENCRYPTION_KEY`, and never returned by any route. This is what the
  capability matrix has always meant by calling email `customer_managed`, and
  what makes `integration_accounts` a table with a writer.

- **Approving an email sends it.** There was nothing for a reviewer to do
  between approving and sending — they had already read the evidence, read the
  words and said yes. The send is inline and its outcome is part of the same
  answer, so a rejected login is visible instead of being an approved action
  sitting in a queue nobody watches.

- **One set of send mechanics, shared.** Recipient choice, subject fallback and
  the five writes a completed send implies now live in `outreach-email` and are
  used by both autopilot and the approval queue, so the two paths cannot drift
  apart again.

Also fixes a rate-limit bug the tests found: re-checking policy at execution
time counted the action being executed against its own per-prospect weekly
limit, so with the default of one action per prospect per week every explicit
send was refused by its own existence.

Deploy note: set `SECRET_ENCRYPTION_KEY` (32 bytes, base64) or connecting a
mailbox is refused and outreach falls back to the platform sender.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Setup could only ever see one. The schema always allowed several — `offerings`
is keyed by workspace and every campaign names the offering it sells — but
every read in the profile module was `ORDER BY created_at ASC LIMIT 1`. So
describing a second product overwrote the first, and the single campaign was
repointed at the new offering: a company with two products could sell one, and
the switch was silent.

A product now owns the three things that make its outreach specific:

- its **offering**, the claims a draft may ground itself in,
- its **voice**, because a security tool and a design tool should not sound the
  same — and they did, sharing the workspace's first voice profile,
- its **campaign and ICP filters**, because different products have different
  buyers. The ICP was read unscoped, so a second product was scored against the
  first product's titles.

`PUT /onboarding/profile?new=1` adds one; the same route with an `offeringId`
edits one; `GET /products` lists them; `GET /onboarding/profile?offeringId=`
loads one. Setup takes `?product=`, so each product is a URL — the back button
works and a half-finished edit to one is never carried into another.

Archiving rather than deleting: an offering is referenced by campaigns, which
are referenced by recommendations, actions and interactions — the record of
every message already sent. Tidying a settings page must not delete that. The
pipeline and autopilot already skip archived campaigns.

An offering id arriving in a request body is scoped to the workspace before it
is trusted; unchecked, it would let one workspace rewrite another's product.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every intake until now started from a company: a keyword named domains, the
domains were crawled, the pages named people, and GitHub said what those people
had been doing. That finds the buyers of developer tooling and almost nobody
else. A trade supplier's buyers have no engineering blog and no GitHub profile
— they are posting "can anyone recommend a decent…" in a subreddit, a forum, or
the comments of a trade publication.

Listening inverts it. A campaign's own keywords (plus the competitor names
already captured during setup — someone naming a competitor is the strongest
listening signal there is) are searched against public feeds, and every match
becomes the person who wrote it and the signal that made them worth noticing.

Four sources, chosen for having no gatekeeper:

- **Reddit** — the best non-technical coverage available without a commercial
  agreement. Scoping to a few trade subreddits is the whole targeting story.
- **RSS/Atom** — `rss` has been in the network list and the capability matrix
  since the first commit with nothing behind it. Trade press, local news, job
  boards, forum and podcast feeds are all just XML at a URL.
- **Bluesky post search** — distinct from the existing identity provider, which
  runs the other way. The one network where finding and replying are both
  already permitted.
- **Nostr** — new network, new matrix entries. Small and crypto-skewed, but no
  company can revoke access to it.

Classification is deterministic regex, not a model call: a listening run that
produces nothing whenever a provider is capped is a feature that appears broken
at random, so the cheap half stands alone and the model can only improve on it.

**Listening does not make strangers contactable.** A handle is not an identity,
so people found this way are written at 0.35 confidence — below the 0.85
outreach floor — and the policy engine refuses outbound against them until
something else raises it. That is the intended behaviour, not a gap: the honest
path from a public post to a message is to work out who the person is first.

Off unless `LISTEN_SOURCES` names sources. Facebook and Nextdoor are absent on
purpose — neither exposes a public search, and scraping them violates their
terms. LinkedIn stays research-only for the same reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​types/​nodemailer@​8.0.11001007687100
Addednodemailer@​9.0.5961009794100

View full report

@ralyodio
ralyodio marked this pull request as ready for review August 16, 2026 16:02
@ralyodio
ralyodio merged commit 8cd3d23 into main Aug 16, 2026
4 checks passed
ralyodio added a commit that referenced this pull request Aug 16, 2026
PR #33 landed on main while this branch was open, and it built the same
feature: a workspace connecting its own SMTP server. Two implementations
of that survived the textual merge, which is the actual conflict —
`packages/email/src/smtp.ts` existed on both sides as an add/add.

Main's is the one kept, and not because it was merged first. It stores the
mailbox on `integrations` + `integration_accounts`, which is the pair
`packages/policy` reads for `hasConnectedAccount`. This branch's parallel
`email_accounts` table answered the same question in a second place, and a
policy engine that can get two different answers about whether a mailbox
exists is how "no connected email account, so this must be done manually"
came back — the exact bug #33 was written to fix.

Removed here as duplicates of shipped code:

  packages/email/src/smtp.ts            main's nodemailer client instead
  packages/email/src/secret-box.ts      packages/secrets, SECRET_ENCRYPTION_KEY
  apps/api/src/email-account.ts         packages/pipeline/src/email-account.ts
  packages/pipeline/src/sending.ts      mailerForWorkspace
  apps/web/components/mail-server-form   mailbox-form
  migrations/0010 email_accounts table

What this branch actually adds is untouched: prefilled composers for the
twelve manual-only networks, several campaigns with their own controls, and
the `workflow_events` ledger streamed over SSE.

Two seams needed re-fitting rather than deleting. `runAutopilot` now resolves
the sender itself, so `via` is read from `sender.ownMailbox` instead of being
passed down from the server loop — the send/success and send/error events
still record which transport carried the message. And the status panel's
sending snapshot reads the integrations pair; a row there is verified by
construction, because `connectEmailAccount` authenticates against the real
server before it writes anything.

773 tests pass, typecheck and format clean, PWA builds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ralyodio
ralyodio deleted the worktree-email-connect branch August 16, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant