[BUG]: in-tree FFmpeg corrupts RGB-to-VP9 colors with --disable-x86asm

[Jont828]

Describe the Bug

Dynamo's in-tree FFmpeg 8.1.2 build corrupts colors when converting RGB frames to VP9 on x86 with --disable-x86asm. A synthetic red quadrant becomes approximately magenta. This reproduces without a model or GPU through both the FFmpeg CLI and dynamo.common.utils.video_utils.encode_to_video_bytes().

This issue isolates the shared codec/root-cause defect from the two example-level trackers:

• [BUG]: vllm-omni-video-agg on 1.4.2 drops base64 delivery and corrupts video colors #14667 — T2V (vllm-omni-video-agg)
• [BUG]: vllm-omni-i2v-agg on 1.4.2 drops base64 delivery and corrupts video colors #14668 — I2V (vllm-omni-i2v-agg)

Those issues also describe a separate response_format=b64_json forwarding bug. Merged PR #14844 fixes that delivery bug, not this codec bug. Retain the original issues' end-to-end acceptance criteria; this tracker does not assert that either complete workflow is fixed.

Environment

The original reports document a September 10, 2026 reproduction with public Dynamo 1.4.2 frontend/worker images, including nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.2. The worker reports FFmpeg 8.1.2, configured with --disable-x86asm.

A separate model-free CPU investigation built official FFmpeg 8.1.2 and 9.0.1 on Linux/amd64 under emulation, using the same restricted Dynamo configure flags and libvpx 1.14.1. No additional codec implementation, GPL/nonfree option, model, or GPU was required.

Steps to Reproduce

In an affected runtime, generate reference quadrants and encode them twice: once normally and once with the diagnostic MMXEXT override.

import os
from pathlib import Path
import subprocess
import tempfile

import numpy as np
from PIL import Image

out = Path(tempfile.mkdtemp(prefix="dynamo-ffmpeg-colors-"))
ffmpeg = os.environ.get("IMAGEIO_FFMPEG_EXE", "/usr/local/bin/ffmpeg")
frame = np.zeros((128, 128, 3), dtype=np.uint8)
frame[:64, :64] = (255, 0, 0)
frame[:64, 64:] = (0, 255, 0)
frame[64:, :64] = (0, 0, 255)
frame[64:, 64:] = (255, 255, 255)
Image.fromarray(frame).save(out / "reference.png")
frames = np.repeat(frame[None], 8, axis=0)

for name, flags in [("default", []), ("no-mmxext", ["-cpuflags", "-mmxext"])]:
    subprocess.run(
        [ffmpeg, "-v", "error", "-y", *flags,
         "-f", "rawvideo", "-vcodec", "rawvideo", "-s", "128x128",
         "-pix_fmt", "rgb24", "-r", "8", "-i", "-", "-an",
         "-vcodec", "libvpx-vp9", "-pix_fmt", "yuv420p",
         str(out / f"{name}.mp4")],
        input=frames.tobytes(), check=True,
    )
print(out)

Decode the generated files with an independent decoder and compare quadrant interiors against reference.png, allowing a small lossy-codec tolerance. Do not add a rawvideo encoder or an unrestricted FFmpeg/PyAV wheel to the shipping runtime just to perform this check. The CPU investigation recovered YUV planes with libavcodec and performed the YUV-to-RGB inverse separately, avoiding the suspect libswscale conversion for the oracle.

Expected Behavior

Default RGB → VP9 conversion preserves red/green/blue/white quadrants within a small encoding tolerance. MP4 and WebM produced by the shared Dynamo helper have sensible colors.

Actual Behavior / Validation

Maximum absolute error among the quadrant-mean RGB channels, on a 0–255 scale:

Build/path	Maximum error	Result
8.1.2 CLI → MP4	253.580	Corrupt
8.1.2 shared Dynamo helper → MP4	247.370	Corrupt
8.1.2 shared Dynamo helper → WebM	235.957	Corrupt
8.1.2 CLI with -cpuflags -mmxext	0.930	Diagnostic control passes
9.0.1 CLI/shared helper → MP4/WebM	0.930 for each	Pass

The override is a diagnostic control, not a performance-qualified production fix. These measurements do not claim patched/full-model T2V or I2V acceptance.

Root Cause and Upstream Status

FFmpeg commit 62285be0096319310d7012bbab1739db79926c4d, committed April 26, 2026, corrects selection of use_mmx_vfilter when the matching external-assembly plane writers are unavailable. In the affected source, inline MMX initialization can select the filter layout while --disable-x86asm removes the corresponding writers.

The correction is present in official 9.x releases. Direct source inspection also confirms it in 9.0.2, the current official release. 8.1.3, released September 21, still contains the old logic in its official tarball; do not assume an 8.x maintenance release includes the fix.

The executed color comparisons above used 9.0.1, not 9.0.2. The selected release must receive its own build/runtime qualification. Official release information: https://ffmpeg.org/download.html.

Acceptance Criteria

• The shipped x86 runtime passes a model-free RGB-quadrant regression through the real CLI and shared Dynamo helper for MP4/WebM.
• The regression asserts pixel/color correctness, not merely nonempty output or a VP9 codec identifier; it demonstrably fails on the affected 8.1.2 build.
• Existing codec/license restrictions and required NVDEC bitstream filters remain intact.
• The fix does not introduce a Rust/PyNvVideoCodec native-library loading regression.
• Rebuilt, version-identified images are used for the remaining T2V/I2V acceptance checks in [BUG]: vllm-omni-video-agg on 1.4.2 drops base64 delivery and corrupts video colors #14667 and [BUG]: vllm-omni-i2v-agg on 1.4.2 drops base64 delivery and corrupts video colors #14668.

Tracked Fix

Proposed fix: #15199 — upgrade the in-tree FFmpeg to official 9.0.2 with native-library isolation and regression qualification. That implementation issue builds on existing PR #14925. This bug remains open until the rebuilt runtime meets the acceptance criteria above; the link is not a claim that the fix has shipped.