Firehose Notifications

A lightweight, long-running Bun service that monitors web pages for content changes via the Firehose API and sends batched Slack notifications when updates are detected.

How it works

1. Connects to the Firehose API and fetches all configured taps (monitored queries)
2. Opens SSE (Server-Sent Events) streams for each tap to receive real-time updates
3. Buffers incoming update events and flushes them to Slack on a configurable interval
4. Periodically refreshes the tap list to pick up new taps and disconnect removed ones

Features

• Multi-tap streaming — monitors multiple queries concurrently via independent SSE streams
• Event batching — buffers events and sends aggregated Slack messages to reduce noise
• LRU caching — caches tap and rule metadata for fast lookups without extra API calls
• Auto-reconnect — streams automatically reconnect on failure
• Rate-limited Slack posts — prevents exceeding Slack's webhook rate limits; failed posts are re-queued

Configuration

Variable	Default	Description
FIREHOSE_MANAGEMENT_KEY	—	Your Firehose API management key (required)
SLACK_WEBHOOK_URL	—	Slack incoming webhook URL for notifications (required)
BATCH_WINDOW_MS	10000	How often (ms) buffered events are flushed to Slack
TAP_REFRESH_INTERVAL_MS	300000	How often (ms) the tap list is refreshed from Firehose
LRU_CACHE_MAX_SIZE	100	Max entries in the LRU cache

Getting started

Install dependencies:

bun install

Run the service:

bun run dev

Run tests:

bun test