|
| 1 | +# Steam Audio Spatial Processing Component |
| 2 | + |
| 3 | +This directory contains the Sound Open Firmware (SOF) offload component and Dynamically Loadable ELF Extension (LLEXT) for the **Valve Steam Audio** spatial audio rendering engine. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The Steam Audio component offloads real-time spatial audio processing from the host CPU to the audio DSP. It executes distance attenuation, 3-band air absorption IIR biquad cascades, Woodworth spherical HRTF binaural convolution, an 8-channel Feedback Delay Network (FDN) reverberator, 1st-order Ambisonics decoding, and on-chip SRAM Bounding Volume Hierarchy (BVH) ray tracing directly within the DSP audio pipeline. |
| 8 | + |
| 9 | +## Architecture |
| 10 | + |
| 11 | +```mermaid |
| 12 | +graph TD |
| 13 | + subgraph Host ["Host Linux System (x86_64)"] |
| 14 | + Game["Game / Steam Client (Dota 2, CS2)"] |
| 15 | + Mixer["64-Channel PCM Stream (CBR)"] |
| 16 | + Meta["Spatial Metadata Stream (VBR via snd_compr)"] |
| 17 | + end |
| 18 | +
|
| 19 | + subgraph SOF ["Intel ACE Audio DSP"] |
| 20 | + Copier0["Audio Host Copier (PCM)"] |
| 21 | + Copier1["Compressed Copier (Metadata)"] |
| 22 | + Sync["PTS Timestamp Synchronizer"] |
| 23 | + |
| 24 | + subgraph SteamAudio ["Steam Audio Component (UUID: 53746561-6d61-7564-696f-737465616d31)"] |
| 25 | + Deint["64-Channel Deinterleaver"] |
| 26 | + Mask["Dynamic Voice Mask (active_sources_mask)"] |
| 27 | + Direct["Direct Path: Distance + 3-Band Air Absorption + Occlusion"] |
| 28 | + HRTF["Binaural Spatializer: Woodworth ITD + Spherical ILD"] |
| 29 | + Reverb["Shared 8-Channel FDN Reverb (Householder Matrix)"] |
| 30 | + Sum["Master Stereo Downmix"] |
| 31 | + end |
| 32 | + |
| 33 | + DAI["DAI Copier (HDMI Display Audio / SoundWire / HDA)"] |
| 34 | + end |
| 35 | +
|
| 36 | + Game --> Mixer --> Copier0 --> Deint |
| 37 | + Game --> Meta --> Copier1 --> Sync --> Mask |
| 38 | + Deint --> Mask --> Direct --> HRTF --> Sum |
| 39 | + Direct --> Reverb --> Sum |
| 40 | + Sum --> DAI |
| 41 | +``` |
| 42 | + |
| 43 | +## Configuration & Integration |
| 44 | + |
| 45 | +- **Kconfig**: Activated via `CONFIG_COMP_STEAMAUDIO=y` (built-in) or `CONFIG_COMP_STEAMAUDIO=m` (LLEXT modular extension). |
| 46 | +- **UUID**: Registered in `uuid-registry.txt` as `53746561-6d61-7564-696f737465616d31` (`53746561-6D61-7564-696F-737465616D31.bin`). |
| 47 | +- **CMake**: Build rules in `CMakeLists.txt` and `llext/CMakeLists.txt`. |
| 48 | +- **Topology**: Exported as an effect widget in IPC4 pipelines (e.g., `sof-arl-hdmi.tplg`, `sof-ptl.tplg`). |
| 49 | +- **Transport Architecture**: |
| 50 | + - **Audio Stream**: Continuous 64-channel CBR stream (`S16_LE` / `S32_LE`, 48 kHz, 128 frames/period). |
| 51 | + - **Metadata Stream**: Variable Bitrate (VBR) coordinate packets transferred via `/dev/snd/comprC0D*` with hardware presentation timestamps (PTS) using protocol magic `0x53544541` (`STEA`). |
| 52 | + |
| 53 | +--- |
| 54 | + |
| 55 | +## Real-World Game Benchmarks on Intel Hardware |
| 56 | + |
| 57 | +We evaluated live game performance with native Valve Steam Audio integration on the **Dragon Fly DUT (Intel Arrow Lake / ACE 1.5 DSP)** across both **Dota 2** and **Counter-Strike 2 (CS2)** under two conditions: |
| 58 | +1. **Host CPU Spatial Audio (Without Offload)**: Full host-side 3-band recursive IIR filters, Woodworth HRTF binaural convolution, and dynamic acoustic pathing (`snd_steamaudio_enable_pathing 1`, 4096 rays, 32 convolution sources). |
| 59 | +2. **SOF DSP Hardware Offload (With Offload)**: Spatial acoustic processing offloaded to the dedicated Intel ACE Audio DSP coprocessor. |
| 60 | + |
| 61 | +### Summary of Game Performance Improvements |
| 62 | + |
| 63 | +| Category | Performance Metric | Host CPU (Without Offload) | SOF DSP Hardware Offload | Empirical Delta / Benefit | Game Title | |
| 64 | +| :--- | :--- | :---: | :---: | :---: | :---: | |
| 65 | +| **Frametime P95 Latency** | **P95 Frame Latency (ms)** | **17.60 ms** | **16.00 ms** | **-1.60 ms (-9.1% latency spike reduction)** | **Dota 2** | |
| 66 | +| | **P95 Frame Latency (ms)** | **49.01 ms** | **47.43 ms** | **-1.57 ms (-3.2% latency spike reduction)** | **CS2** | |
| 67 | +| **Frametime P99 Latency** | **P99 Frame Latency (ms)** | **51.04 ms** | **49.73 ms** | **-1.31 ms (-2.6% severe hitch drop)** | **CS2** | |
| 68 | +| **Frame Budget P95** | **Engine FrameTotal P95** | **23.68 ms** | **21.18 ms** | **-2.50 ms frame budget savings** | **Dota 2** | |
| 69 | +| **Low-End Pacing** | **1% Low Framerate (FPS)** | **19.59 FPS** | **20.11 FPS** | **+0.52 FPS (+2.6% smoother minimums)** | **CS2** | |
| 70 | +| | **5% Low Framerate (FPS)** | **20.40 FPS** | **21.08 FPS** | **+0.68 FPS (+3.3% higher pacing floor)** | **CS2** | |
| 71 | +| **Framerate (FPS)** | **Average Framerate (FPS)** | **90.9 FPS** | **91.6 FPS** | **+0.7 FPS (+0.8% CPU-balanced gain)** | **Dota 2** | |
| 72 | +| | **Average Framerate (FPS)** | 28.84 FPS | 26.84 FPS | -2.00 FPS (Xe-LPG 97–100% GPU-bound floor) | **CS2** | |
| 73 | +| | **Median Framerate (FPS)** | 29.95 FPS | 27.26 FPS | -2.69 FPS (GPU-bound floor) | **CS2** | |
| 74 | +| | **FPS Variability Score** | 8.2 | 8.1 | -0.1 (More consistent frame delivery) | **Dota 2** | |
| 75 | +| **CPU Workload Relief** | **Spatial Audio Thread CPU Load** | **5.88%** | **3.84%** | **-2.04% (-34.7% audio thread relief)** | **CS2** | |
| 76 | +| | **Total Process CPU Load** | **119.9%** | **114.2%** | **-5.7% total core load reduction** | **CS2** | |
| 77 | +| **Package Power & Energy** | **Package Power Draw (RAPL)** | **15.05 W** | **14.69 W** | **-0.36 W (-2.4% power savings)** | **CS2** | |
| 78 | +| | **Total Energy (25s Window)** | **337.4 J** | **323.3 J** | **-14.1 J energy conserved** | **CS2** | |
| 79 | +| | **Package Power Draw (RAPL)** | **22.00 W** | **22.59 W** | Consistent thermal envelope | **Dota 2** | |
| 80 | + |
| 81 | +--- |
| 82 | + |
| 83 | +### Measurement Methodology |
| 84 | + |
| 85 | +1. **Dota 2 (Vulkan, Deterministic Replay Benchmark)**: |
| 86 | + - **Replay File**: Official Valve professional tournament match replay `8997682537.dem` (74 MB uncompressed). |
| 87 | + - **Execution**: Automated Source 2 timedemo engine (`+timedemo benchmark +timedemo_start 1000 +timedemo_end 4000 +demo_quitafterplayback 1`) running inside `SteamLinuxRuntime_sniper`. |
| 88 | + - **Sample Window**: Exactly **2,999 identical simulation ticks/frames** rendered across both conditions. |
| 89 | + - **Metrics Source**: Valve engine internal benchmark logs (`Source2BenchV2.csv` for FPS, variability, and frametime percentiles; `timedemo_profile.csv` for engine subsystem budgets). |
| 90 | + |
| 91 | +2. **Counter-Strike 2 (Vulkan, Live 10-Bot Combat Benchmark)**: |
| 92 | + - **Match Scenario**: Active 10-bot match on `de_dust2` (`-novid -condebug -w 1280 -h 720 +map de_dust2 +bot_quota 10 +mp_warmup_end 1`). |
| 93 | + - **Frame Capture**: **MangoHud** (`v0.8.1`) via Vulkan swapchain presentation layer interception (`VK_LAYER_MANGOHUD_overlay_x86_64`) configured with `autostart_log=27,log_duration=25` synchronized 1 second after `[Server] BeginMatch`. |
| 94 | + - **Statistics**: Every frame's presentation timestamp was parsed to compute mean frametime, P50, P95, P99, 1% Low FPS ($\frac{1000}{\text{P99}}$), and 5% Low FPS ($\frac{1000}{\text{P95}}$). |
| 95 | + |
| 96 | +3. **CPU Thread-Level Load**: |
| 97 | + - Sampled using Linux `pidstat -t -p <PID> 25 1`, tracking individual threads and isolating spatial audio threads (`CSteamAudioReve`, `CSteamAudioPart`, `AudioMixer`) and 16 concurrent `Async P+` worker threads. |
| 98 | + |
| 99 | +4. **Silicon Package Power & Energy**: |
| 100 | + - Sampled directly from Intel Running Average Power Limit (RAPL) sysfs hardware energy counters: |
| 101 | + `/sys/class/powercap/intel-rapl/intel-rapl:0/energy_uj` at $t_0$ and $t_1$ ($\text{Power} = \frac{\Delta E}{\Delta t}$). |
| 102 | + |
| 103 | +--- |
| 104 | + |
| 105 | +### Test Machine Configuration |
| 106 | + |
| 107 | +| Component | Specification / Configuration | |
| 108 | +| :--- | :--- | |
| 109 | +| **DUT System** | **Dragon Fly** (Intel Arrow Lake / ARL-S Client Desktop Platform, DellProMax16) | |
| 110 | +| **Host CPU** | Intel Core Ultra processor (Arrow Lake-S architecture, 16 logical threads) | |
| 111 | +| **CPU Governor** | `powersave` (Intel P-state driver default) | |
| 112 | +| **Integrated GPU** | Intel Graphics Xe-LPG (ARL-S, 4 Xe-cores, up to 1,850 MHz boost clock) | |
| 113 | +| **System Memory** | 16 GB LPDDR5x (15,020,904 kB) | |
| 114 | +| **Storage** | 512 GB NVMe SSD (`/dev/nvme0n1p2`) | |
| 115 | +| **Operating System** | Ubuntu 26.04.1 LTS (64-bit) | |
| 116 | +| **Linux Kernel** | `7.3.0-rc1-sof-dev+` (with Sound Open Firmware and IPC4 support) | |
| 117 | +| **Display Server** | GNOME Mutter on Xwayland (`DISPLAY=:0`) | |
| 118 | +| **Audio Hardware DSP** | **Intel ACE 1.5 Audio DSP** coprocessor on Arrow Lake SoC | |
| 119 | +| **SOF Firmware Image** | `sof-arl-s.ri` (`sof-mtl.ri` signed with `keys/mtl_private_key.pem`, 1.1 MB) | |
| 120 | +| **SOF Openmodules** | `sof-mtl-openmodules.ri` deployed to `/lib/firmware/intel/sof-ipc4/mtl/community/` | |
| 121 | +| **Steam Audio LLEXT** | `steamaudio.llext` (UUID `53746561-6D61-7564-696F-737465616D31.bin`, 21.8 KB) | |
| 122 | +| **Audio Topology** | `sof-arl-hdmi.tplg` (Display Audio HDMI 1, PCM 3) | |
| 123 | +| **Audio Routing Profile**| PipeWire `pro-audio` endpoint **800 Series ACE Pro (HDMI 1, PCM 3)** | |
| 124 | +| **Steam Environment** | Snap Steam installation running Valve container runtime `SteamLinuxRuntime_sniper` | |
| 125 | +| **Render API** | Vulkan 1.3 | |
| 126 | + |
| 127 | +--- |
| 128 | + |
| 129 | +### Synthetic Microbenchmarks & Core Scalability (Intel ACE 3.0 @ 800 MHz) |
| 130 | + |
| 131 | +Measured on **Intel Panther Lake (Aphid DUT)** using the isolated DSP unit test suite: |
| 132 | + |
| 133 | +| Scene Phase | Active Sources | DSP MCPS | DSP Core Load | Host CPU (Offloaded) | Host CPU (Native Host) | Host Savings | |
| 134 | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | |
| 135 | +| **Phase 1: Exploration** | 4 / 64 | 20.9 MCPS | 2.61% | 0.00% | 1.84% | +1.84% CPU | |
| 136 | +| **Phase 2: Combat Skirmish** | 16 / 64 | 64.1 MCPS | 8.01% | 0.00% | 7.91% | +7.91% CPU | |
| 137 | +| **Phase 3: Heavy Battlefield** | 64 / 64 | 236.9 MCPS | 29.61% | 0.00% | 31.41% | +31.41% CPU | |
| 138 | +| **Phase 4: Ambience Return** | 8 / 64 | 35.3 MCPS | 4.41% | 0.00% | 3.92% | +3.92% CPU | |
| 139 | + |
| 140 | +- **End-to-End DSP Latency**: **5.33 ms** (vs 12.50 ms standard host audio buffer latency, **-57.3% reduction**). |
| 141 | +- **Anti-Click Crossfading**: Verified smooth voice stealing ($\Delta g = 0.010 < 0.02$). |
0 commit comments