Feature: Voice Channel — local microphone adapter with wake word detection

[ar1vit0r]

Problem Statement

OpenJarvis currently supports text-based channels (Telegram, Discord, Slack, etc.) but lacks a native voice channel for local microphone interaction. Users who want voice interaction must rely on external tools or build custom integrations.

A built-in voice channel would:

• Enable hands-free interaction with the AI assistant
• Support wake-word detection for always-listening capability
• Provide a complete voice pipeline (wake word → record → STT → agent → TTS)
• Work offline with local models (openWakeWord, faster-whisper)

Proposed Solution

Add a VoiceChannel class that implements the BaseChannel interface with the following components:

Core Architecture:

VoiceChannel
├── Wake Word Detection (openWakeWord)
│   └── Frame-by-frame streaming detection
├── Audio Capture (sounddevice/PipeWire)
│   └── Microphone input with fallback
├── Speech-to-Text (pluggable)
│   └── OpenAI Whisper, Deepgram, faster-whisper
├── Text-to-Speech (pluggable)
│   └── Kokoro, OpenAI, Cartesia
└── Event Bus
    └── Publish/subscribe for UI integration

Key Features:

• Wake word detection with configurable keyword and threshold
• Audio capture with sounddevice (PortAudio) and PipeWire fallback
• Pluggable STT/TTS backends via registry pattern
• Event-driven architecture for UI integration
• Voice sample capture for model improvement (opt-in)
• Conversation mode with follow-up wake word detection

Dependencies (all optional):

• sounddevice — microphone input
• openwakeword — wake word detection
• numpy — audio processing
• webrtcvad — voice activity detection

Acceptance Criteria

• VoiceChannel implements BaseChannel interface
• Wake word detection works with openWakeWord
• Audio capture works with sounddevice (and PipeWire fallback)
• STT/TTS backends are pluggable via registry
• Events are published for UI integration (VOICE_LISTENING_START, VOICE_SPEAKING_START, etc.)
• Voice sample capture is opt-in and non-blocking
• Conversation mode supports follow-up wake word detection
• Unit tests for core functionality
• Documentation with usage examples

Alternatives Considered

1. External voice tools — Users could use external STT/TTS services, but this adds complexity and latency
2. Plugin system — Could be implemented as a plugin, but voice is a core use case for an AI assistant
3. Minimal MVP — Start with just wake word + STT, add TTS later — but a complete pipeline is more useful

Additional Context

I have a working implementation in my personal fork (ar1vit0r/OpenJarvis) with:

• 1,093 lines of production code
• openWakeWord integration
• PipeWire audio capture fallback
• Voice sample capture
• Conversation mode

The implementation has been tested on real hardware and is running in my daily-use deployment.

Contributor: @ar1vit0r (9 merged PRs in OpenJarvis)