Autonomous Adaptive Music Intelligence System
Orpheus is a self-driving music system that connects to your Spotify account and learns how you listen. It observes your skips, completions, and feedback to build a continuous model of your preferences, then autonomously selects tracks that match your current mood, time of day, and listening context. An optional AI layer powered by Ollama provides session naming, monthly listening recaps, and natural-language reasoning about what to play next.
Orpheus runs as a local server that controls Spotify playback on your behalf. When a session starts, the system:
- Infers initial state from time-of-day preferences, recent history, and (optionally) an LLM context inference
- Builds a candidate pool from your Spotify library plus real-time external recommendations (similar tracks seeded by the current track, discovery tracks seeded by top artists) — external picks are saved to the local DB automatically
- Scores candidates against the current state vector using a weighted multi-dimensional distance function
- Selects a track via weighted random sampling (top candidates weighted by inverse distance)
- Observes your response — completions, skips, likes, dislikes — and blends feedback into the state using exponential moving averages
- Repeats — each track shifts the state, and the next selection adapts accordingly
The result is a continuously adapting stream of music that responds to you in real time without requiring manual playlist curation.
Flutter Client (Dart) Node.js Server (TypeScript)
+-----------------+ +---------------------------+
| Screens | REST/WS | Fastify API |
| Home | <--------> | Auth, Playback, Steering |
| Session | | Sessions, Analytics, AI |
| Analytics | +---------------------------+
| Intelligence | | Intelligence Pipeline |
| Settings | | State Vector -> Steering |
+-----------------+ | Candidate Pool -> Scorer |
| -> Weighted Selection |
+---------------------------+
| Playback Engine |
| Spotify Web API control |
| Session lifecycle mgmt |
+---------------------------+
| SQLite (node:sqlite) |
| Tracks, Interactions, |
| Preferences, Sessions, |
| State History, Analytics |
+---------------------------+
| AI Knowledge (RAG) |
| Genre aliases, mood maps |
| Library context injection|
+---------------------------+
| Ollama (optional) |
| Session naming, recaps, |
| context inference |
+---------------------------+
- Runtime: Node.js 24+ with built-in
node:sqlite - Framework: Fastify 5 with CORS, WebSocket support
- Language: TypeScript (ESM)
- Database: SQLite via
node:sqlite(zero native dependencies) - Spotify: Official
@spotify/web-api-ts-sdkwith PKCE auth flow - AI: Ollama REST client (optional, graceful degradation)
- Scheduler:
node-cronfor periodic tasks - Validation: Zod for config and request schemas
- Logging: Pino with pretty-print
- Framework: Flutter 3.5+
- State Management: Riverpod
- HTTP: Dio
- Real-time: WebSocket via
web_socket_channel - Charts: fl_chart
- Routing: go_router with shell navigation
- SVG: flutter_svg for Spotify brand assets
- Design: Custom dark theme, Lyre Gold (#D4A843) accents, Cinzel headings, Inter body text, JetBrains Mono for data
- Accessibility: Semantics widgets, Tooltips, semantic slider formatters, WCAG AA contrast compliance
- Branding: Spotify attribution (icon + text) on all metadata screens per Developer Terms
- PKCE authorization flow (no server-side secret exposure in redirects)
- Full library sync with audio feature caching (energy, valence, tempo, danceability, etc.)
- Playback control: play, pause, skip, resume, device selection
- Background polling to track what's currently playing
- State Vector: 8-dimensional representation of current listening context (energy, valence, tempo, genre, familiarity, vocalness, aggressiveness, fatigue)
- EMA Blending: New observations blend into the state via exponential moving average (alpha = 0.2). Null audio features are skipped to prevent state drift.
- Steering Controls: User-adjustable sliders that bias the candidate scoring (energy, mood, familiarity, vocal preference, aggressiveness, genre openness, focus/party)
- Candidate Pool: Genre-filtered, energy-windowed subset of cached tracks with recency penalties
- Scorer: Multi-dimensional distance with configurable weights per attribute
- Coherence Analysis: Evaluates queue coherence against the intelligence pipeline state for graceful startup decisions
- Natural Language Requests: Process requests like "play something energetic" via LLM-guided library search
- Weighted Random: Top 3 candidates selected probabilistically by inverse distance (avoids always picking the same "best" track)
- Interactive lock chips in the flow indicator — tap to pin playback to the current genre or artist
- Genre chip shows the current dominant genre (e.g., "ELECTRONIC"), artist chip shows the current artist
- Locked state: gold border + closed lock icon; unlocked: muted border + open lock icon
- Genre lock uses the
targetGenremechanism (hard override in scorer and candidate pool) - Artist lock uses a new
targetArtistmechanism with scoring multiplier and candidate pre-filtering - AI prompt integration: phrases like "only play kpop" or "just Daft Punk" auto-activate the corresponding lock
- Genre Openness slider auto-disables when genre is locked
- All locks clear automatically when the session ends
- Like/dislike buttons with immediate preference score adjustment
- Skip detection with partial-listen analysis
- Completion tracking with preference reinforcement
- Time-of-day preference learning (incremental averaging across sessions)
- Per-track familiarity scoring that evolves with play count
- Engine detects current Spotify playback on start — never interrupts what's already playing
- Analyzes the existing Spotify queue for coherence against the intelligence pipeline
- Adopts coherent tracks seamlessly, seeding state from the adopted queue
- Takes over after the current track if the queue is incoherent
- Auto-start: configurable delay before engine begins playback
- Quiet hours: suppresses auto-start during specified hours
- Configurable via settings API and client UI
- Automatic session lifecycle (start on play, end on stop)
- Session history with energy curves, track lists, and state snapshots
- AI-generated session names (post-session)
- AI-generated session recaps (narrative summary of what was played and why)
- Total listening time, tracks played, skip rate, completion rate, discovery rate
- Genre distribution (pie chart)
- Daily energy trends (line chart)
- Listening hours heatmap (24-hour bar chart)
- Top tracks by play count
- Daily sparklines for quick trend visualization
- Comprehensive listening stats combining Orpheus and Spotify data
- AI-enhanced playlist creation from natural language prompts ("chill jazz for a long flight", "high energy workout mix")
- Configurable parameters: duration (5–180 min), discovery rate (0–100% new songs), energy arc (steady, build up, wind down, peak & fade), transition smoothness, max tracks per artist, source preference (library only or library + Spotify discovery)
- Reuses the full 8-dimension scoring pipeline — candidates scored per energy arc segment with synthetic scoring contexts
- Greedy nearest-neighbor track ordering for smooth transitions (BPM, energy, valence, key compatibility)
- Optional seed track anchoring from currently playing track
- AI prompt parsing extracts artists, genres, moods, BPM range from natural language (keyword fallback when AI unavailable)
- AI-generated creative playlist names and descriptions
- Automatic Spotify playlist creation and track population
- Real-time WebSocket progress events during generation
- Playlist history with full track lists, scores, and metadata
- Build a Spotify playlist from your Liked Songs by genre family (e.g. every K-pop track among thousands of likes) — no track cap
- Families group Spotify sub-genres (K-pop = k-pop girl group, k-pop boy group, k-rap…); uncheck sub-genres individually
- A track matches through any of its artists, so featurings count
- Preview before creating: uncheck tracks, exclude or force-add artists, filter by liked date, limit the count, sort (date liked, artist, title, shuffle, smooth transitions when audio features exist)
- Side-panel layout on desktop/web, stacked on mobile
- Artist genres sync in the background (one artist at a time, resumable) with live progress
- Powered by Ollama running locally (default model: llama3.2)
- Rich System Prompt: Three-level system prompt architecture (
full/light/minimal) with music expert persona, genre normalization, mood mappings, and output format rules - RAG Context Injection: Full-level prompts include the user's actual library stats — top genres, favorite artists, preferred tracks, listening hours, discovery rate, and time-of-day patterns
- Genre Aliases: Extensible
genre_aliases.json(~40 canonical genres with aliases) loaded at startup; normalizes variant spellings (e.g., "kpop" → "k-pop") across AI and keyword parsers - Mood Mappings: Extensible
mood_mappings.json(~13 moods mapped to genres + energy/valence ranges) for mood-to-music translation - Session Naming: LLM generates evocative session names from track lists
- Session Recaps: Narrative summaries of each listening session
- Monthly Recaps: End-of-month listening personality analysis with stats
- Context Inference: LLM suggests initial state values based on time, weather context, and recent patterns
- Advisory Suggestions: Periodic in-session analysis recommending energy/genre shifts
- Playlist Prompt Parsing: Extracts structured parameters from natural language playlist descriptions
- Playlist Naming: Generates creative 2–5 word names and descriptions for generated playlists
- Graceful degradation: all AI features are no-ops when Ollama is unavailable
- When Ollama is available, an LLM picks the next 3 tracks at a time and generates DJ patter (short commentary) for each transition
- 4 preset DJ personas (The Curator, Late Night Radio, Hype DJ, Chill Host) + fully custom persona via free-text description
- Chattiness control: Silent / Minimal / Balanced / Chatty — governs how often patter is shown
- Discovery appetite: Comfort (65% familiar / 25% similar / 10% new), Balanced (50/35/15), Adventurous (40/30/30)
- Steering awareness: genreOpenness slider blends appetite ratios toward explore or comfort in real time
- Proactive scheduling: curator fires when < 30s of audio remains across current track + lookahead queue
- Fallback: on LLM timeout or error, falls back to the existing algorithmic selector; retries every 2 minutes automatically
- Spam-skip detection: 3 skips in 10s triggers an immediate rapid-skip curator call
- 3-step onboarding wizard on first launch captures appetite, chattiness, and persona preference
- Home screen shows: DJ patter banner (speech-bubble style, hidden when chattiness is Silent), pick reason below track info, Up Next queue with source badges (library / similar / new), persona chip or "AUTO" badge in status bar
- Settings screen: DJ Personality section with persona chip row, chattiness segmented toggle, appetite segmented toggle with live mini ratio bar, custom persona textarea; DJ Voice section with Piper TTS enable toggle, voice dropdown with preview button, and music duck-volume slider
- Text-to-speech converts DJ patter into spoken audio using Piper (optional; degrades gracefully when binary absent)
- Client-orchestrated volume ducking: Flutter ducks Spotify volume → fetches WAV from server → plays via
just_audio→ restores volume - On-demand generation: server spawns
piper.exeper request (stdin←text, stdout→WAV bytes), ~100ms latency on CPU - Configurable: enable/disable toggle, voice model selection (dropdown populated from
PIPER_VOICES_DIR), duck volume (0–100%) - Full graceful degradation: 503 from server when Piper absent → client silently skips audio, patter text still displayed in banner
- WebSocket push for playback state changes, track transitions, AI events, curator picks (
curator_update,curator_fallback,curator_restored) - Client receives live updates without polling
All routes are prefixed with /api.
| Group | Endpoint | Method | Description |
|---|---|---|---|
| Health | /health |
GET | Server health check |
| Auth | /auth/status |
GET | Authentication status |
| Auth | /auth/login |
GET | Get Spotify auth URL |
| Auth | /auth/callback |
GET | OAuth callback handler |
| Playback | /playback/current |
GET | Current playback state |
| Playback | /playback/start |
POST | Start the engine |
| Playback | /playback/stop |
POST | Stop the engine |
| Playback | /playback/skip |
POST | Skip current track |
| Playback | /playback/pause |
POST | Pause playback |
| Playback | /playback/resume |
POST | Resume playback |
| Playback | /playback/previous |
POST | Play previous track |
| Playback | /playback/devices |
GET | Available Spotify devices |
| Playback | /playback/request |
POST | Natural language music request |
| Steering | /steering |
GET | Current steering values |
| Steering | /steering |
PUT | Update steering controls |
| Steering | /steering/target-genre |
GET | Get genre lock state |
| Steering | /steering/target-genre |
PUT | Lock to a genre |
| Steering | /steering/target-genre |
DELETE | Clear genre lock |
| Steering | /steering/target-artist |
GET | Get artist lock state |
| Steering | /steering/target-artist |
PUT | Lock to an artist |
| Steering | /steering/target-artist |
DELETE | Clear artist lock |
| Feedback | /feedback/like |
POST | Like current track |
| Feedback | /feedback/dislike |
POST | Dislike current track |
| Sessions | /sessions |
GET | Paginated session history |
| Sessions | /sessions/active |
GET | Active session with tracks |
| Sessions | /sessions/:id |
GET | Session detail by ID |
| Analytics | /analytics/overview |
GET | Aggregate stats (cached) |
| Analytics | /analytics/genres |
GET | Genre distribution |
| Analytics | /analytics/energy |
GET | Daily energy trends |
| Analytics | /analytics/hours |
GET | Listening hours heatmap |
| Analytics | /analytics/top-tracks |
GET | Top tracks by plays |
| Analytics | /analytics/daily |
GET | Daily listening totals |
| Analytics | /analytics/listening-stats |
GET | Combined listening statistics |
| Settings | /settings |
GET | All settings |
| Settings | /settings |
PUT | Update settings |
| Context | /context/state |
GET | Current intelligence state |
| Context | /context/time-preferences |
GET | Learned time preferences |
| AI | /ai/status |
GET | AI system status |
| AI | /ai/suggestions |
GET | AI suggestion history |
| AI | /ai/analyze |
POST | Trigger AI analysis |
| AI | /ai/recaps |
GET | All monthly recaps |
| AI | /ai/recaps |
POST | Generate monthly recap |
| AI | /ai/infer |
POST | AI context inference |
| Playlists | /playlists/generate |
POST | Generate AI-enhanced playlist |
| Playlists | /playlists |
GET | List playlist history (paginated) |
| Playlists | /playlists/:id |
GET | Get playlist with tracks |
| Playlists | /playlists/:id |
DELETE | Delete playlist from local DB |
| Library | /library/genres |
GET | Genre families + counts over Liked Songs, sync progress |
| Library | /library/genres/sync |
POST | Start artist-genre sync (409 if running) |
| Library | /library/artists?q= |
GET | Search artists on liked tracks |
| Library | /library/artists/:artistId/liked-tracks |
GET | All liked tracks of an artist |
| Genre Playlists | /genre-playlists/preview |
POST | Uncapped liked-track preview for genre filters |
| Genre Playlists | /genre-playlists/order |
POST | Smooth-transition ordering of track ids |
| Genre Playlists | /genre-playlists/export |
POST | Create a Spotify playlist from ordered track ids |
| DJ | /dj/preferences |
GET | Get DJ preferences |
| DJ | /dj/preferences |
PUT | Update DJ preferences (partial) |
| DJ | /dj/onboarding |
GET | Get onboarding completion status |
| DJ | /dj/onboarding/complete |
POST | Mark onboarding complete |
| Playback | /playback/volume |
PUT | Set Spotify volume (0–100) |
| TTS | /tts/speak?text= |
GET | Generate WAV for given text |
| TTS | /tts/voices |
GET | List available Piper voice models |
| TTS | /tts/preview |
POST | Preview a specific voice |
| TTS | /tts/settings |
PUT | Update TTS preferences |
| WebSocket | /ws |
WS | Real-time event stream |
Music/
server/
src/
ai/ # Ollama client, prompts, personas, AI service, knowledge/RAG
api/
middleware/ # Error handler
routes/ # auth, playback, steering, sessions,
# analytics, settings, context, ai, playlist, dj, tts
server.ts # Fastify setup and route registration
websocket.ts # WebSocket broadcast infrastructure
database/
connection.ts # SQLite connection (node:sqlite)
migrations.ts # Schema v1-v9
repositories/ # track, interaction, session, preference,
# state-history, analytics, settings,
# time-preferences, ai-suggestion,
# monthly-recap, listening-stats, top-artists,
# playlist, dj-preferences, artist, genre-playlist
types.ts # Row type interfaces
intelligence/
state-vector.ts # 8D state with EMA blending
steering.ts # User steering control application
candidate-pool.ts # Genre/energy/recency filtering
scorer.ts # Multi-dimensional distance scoring
selector.ts # Pipeline coordinator
feedback.ts # Like/dislike/skip processing
context-learning.ts # Time-of-day preference learning
coherence.ts # Queue coherence analysis
request-handler.ts # Natural language request processing
playlist-generator.ts # AI-enhanced playlist generation pipeline
genre-families.ts # Genre family classification
genre-playlist.ts # Genre builder service
track-ordering.ts # Greedy nearest-neighbor ordering
curator.ts # DJ curator LLM orchestrator
listener-context.ts # LLM context assembler
pool-builder.ts # Library/similar/discovery ratio pool
types.ts # Intelligence type definitions
playback/
engine.ts # Session lifecycle, track advancement
queue.ts # Track queue management
session.ts # Session start/end logic
types.ts # Playback type definitions
scheduler/
scheduler.ts # Cron task registration
tasks/
sync-library.ts # Spotify library + audio features sync
poll-player.ts # Playback state polling
compute-analytics.ts # Daily analytics precomputation
monthly-recap.ts # Monthly AI recap generation
shared/
constants.ts # Application-wide constants
errors.ts # Custom error classes
logger.ts # Pino logger configuration
utils.ts # Shared utility functions
spotify/
auth.ts # PKCE flow + token management
client.ts # Authenticated Spotify SDK wrapper
library.ts # Library sync operations
player.ts # Playback control operations (includes setVolume)
artist-genre-sync.ts # Artist genre background sync
types.ts # Spotify type definitions
tts/
adapter.ts # TtsAdapter interface + Voice type
piper.ts # PiperAdapter — subprocess wrapper, singleton
config.ts # Zod-validated environment config
index.ts # Application entry point
data/
orpheus.db # SQLite database (auto-created)
genre_aliases.json # Genre alias map (~40 genres)
mood_mappings.json # Mood-to-genre/energy/valence mappings
.env # Environment variables (not committed)
.env.example # Environment template
client/
lib/
config/
constants.dart # API base URL, app constants
routes.dart # go_router configuration
theme.dart # Orpheus dark theme, colors, typography
models/
genre_builder_models.dart # Genre playlist builder data models
providers/
session_provider.dart # Session state management
steering_provider.dart# Steering control state
playlist_provider.dart# Playlist generation state + WS progress
dj_provider.dart # DJ curator preferences + patter/Up Next state
genre_builder_provider.dart # Genre playlist builder state
screens/
home_screen.dart # Now playing, DJ patter, Up Next, persona chip
session_screen.dart # Session history, energy curves
analytics_screen.dart # Charts and stats dashboard
intelligence_screen.dart # AI status, suggestions, recaps
settings_screen.dart # Automation, AI, and DJ personality settings
playlist_screen.dart # Playlist creation form + result view
auth_screen.dart # Spotify login
onboarding_screen.dart# 3-step DJ setup wizard (first launch)
shell_screen.dart # Bottom navigation shell
services/
api_service.dart # Dio HTTP client for all endpoints
websocket_service.dart# WebSocket connection management
tts_service.dart # TtsService singleton — duck, fetch WAV, play, restore
widgets/
energy_arc.dart # Semi-circular energy gauge
feedback_buttons.dart # Like/dislike controls
now_playing_card.dart # Current track display
orpheus_app_bar.dart # Branded app bar
session_timeline.dart # Horizontal energy curve chart
stat_card.dart # Metric card with sparkline
spotify_attribution.dart # Spotify branding attribution widget
steering_slider.dart # Steering control slider
genre_builder/ # Genre playlist builder UI components
main.dart # Flutter app entry point
pubspec.yaml # Flutter dependencies
SQLite with 9 migration versions:
- v1: Core tables —
tracks,interactions,sessions,preferences,state_history,steering_history,time_preferences,analytics_cache,settings,auth_tokens - v2: AI support —
ai_suggestionstable, AI-related settings - v3: Reasoning layer —
session_namecolumn on sessions,monthly_recapstable - v4: Spotify top artists —
spotify_top_artiststable, listening stats cache support - v5: Playlist generation —
playliststable (prompt, parameters, Spotify link, stats),playlist_trackstable (track positions, scores, segments with CASCADE delete) - v6: AI genre inference —
genre_sourcecolumn ontracks(spotify|ai|ai_failed) - v7: DJ Curator —
dj_preferencessingle-row table (persona, chattiness, discovery_appetite, custom_persona, onboarding_completed).CHECK (id = 1)enforces single-row invariant. - v8: DJ Voice —
tts_enabled,tts_voice,tts_duck_volumecolumns ondj_preferences. - v9: Genre playlists —
tracks.liked_at(Liked Songs date, cleared when un-liked),tracks.features_source(spotify|default),artists,artist_genres(all Spotify genres per artist),track_artists(every artist of a track, in order)
| Component | Technology |
|---|---|
| Server Runtime | Node.js 24+ |
| Server Framework | Fastify 5 |
| Language (Server) | TypeScript (ESM) |
| Database | SQLite via node:sqlite |
| Spotify SDK | @spotify/web-api-ts-sdk |
| Config Validation | Zod |
| Scheduler | node-cron |
| Logging | Pino |
| AI (Optional) | Ollama (llama3.2) |
| Client Framework | Flutter 3.5+ |
| State Management | Riverpod |
| HTTP Client | Dio |
| Charts | fl_chart |
| Routing | go_router |
| Containerization | Docker + Docker Compose |
Run the full stack with Docker Compose:
# Server + Client (no AI)
docker compose up --build
# Server + Client + Ollama AI
docker compose --profile ai up --buildIf using the ai profile, pull a model after Ollama starts:
docker exec orpheus-ollama ollama pull llama3.2The client is served on port 80 (nginx), the server API on port 3000, and Ollama on port 11434.
| Service | Container | Port | Notes |
|---|---|---|---|
| Server | orpheus-server |
3000 | Node.js API + WebSocket |
| Client | orpheus-client |
80 | Flutter web via nginx (proxies /api and /ws to server) |
| Ollama | orpheus-ollama |
11434 | Optional — enable with --profile ai |
The SQLite database is persisted in the server-data Docker volume. Ollama models are persisted in ollama-data.
Note: You still need a
server/.envfile with your Spotify credentials before building. See Setup Guide.
Deploy to a self-hosted k3s cluster using the included Helm chart:
# Build and import images into k3s
docker build -t orpheus-server:latest ./server
docker build -t orpheus-client:latest ./client
docker save orpheus-server:latest | sudo k3s ctr images import -
docker save orpheus-client:latest | sudo k3s ctr images import -
# Deploy with Helm
helm install orpheus ./helm/orpheus \
--set spotify.clientId=YOUR_ID \
--set spotify.clientSecret=YOUR_SECRET \
--set global.domain=orpheus.local
# Enable Ollama AI
helm upgrade orpheus ./helm/orpheus --set ollama.enabled=true
# Pull the AI model
kubectl exec -it orpheus-ollama-0 -- ollama pull llama3.2The chart includes: server Deployment (with PVC for SQLite), client Deployment (nginx with templated reverse proxy), optional Ollama StatefulSet (with GPU support), Traefik Ingress, and Spotify credentials Secret.
See the full setup guide: helm/README.md
This project is for personal use.