500k MAU on communal pgvector — bank-per-user vs bank-per-assistant?

[Simon-Druchinin]

Hi Hindsight team,

We’re planning ~500k MAU on one Hindsight instance and shared PostgreSQL (pgvector) — no sharding, no AlloyDB/ScaNN.

Q1. Is bank-per-user viable at this scale?
From v0.8.4 code: 3 partial vector indexes per bank on pgvector. 500k banks ≈ 1.5M indexes — matches our postmortem (~10k banks already hurt PostgreSQL). pgvectorscale doesn’t reduce index count. We conclude bank-per-user is not fit for communal pgvector at this scale.

Q2. Is bank-per-assistant + user:{id} tags (all_strict, observation_scopes=per_tag) sound?
Our PoC: 1 bank → 3 indexes; no cross-user recall with all_strict; consolidation doesn’t merge users. We know tags ≠ bank-level isolation (shared entity graph) and need a strict client wrapper.

Q3. Is Hindsight suitable for 500k MAU overall?
Yes, if we use few banks + tags + wrapper — not bank-per-user. Index cardinality looks solved; we still need to validate recall/consolidation at full data volume inside large banks.

Any guidance on practical bank limits on pgvector and tag-filtered recall at millions of facts/bank would help.

Best regards,
Simon

[chrislatimer]

Hi Simon, yes, Hindsight can definitely scale to support that level of users. You start getting into the Postgres scaling challenges that are fairly well known. We've built some solutions to this that aren't part of the open source project. Happy to set up a time to talk about the strategies we use and potential solutions if you'd like.

[Simon-Druchinin]

Hi, thanks for the reply! At this stage, we are strictly looking to stay within the open-source ecosystem and do not have a budget for commercial solutions. If you have any advice or strategies that can be implemented using the open-source version of Hindsight, I’d be happy to hear them here in the chat. Thanks again!