Skip to content

Commit 578efa3

Browse files
ralyodioclaude
andauthored
Declare the codecs the renditions actually have (#280)
The multivariant playlist hardcoded CODECS="avc1.640028,mp4a.40.2" for every rendition. Both halves were wrong, and I only found it by fetching a shipped playlist and probing the file it points at. The level: shipped renditions probe as H.264 High @ 3.1, which is avc1.64001F. 0x28 is level 4.0. A player reads CODECS to choose a decoder configuration before it fetches a segment, so the declaration is a promise about the bitstream, and this one was not kept. The audio: every ad rendered so far is silent and carries a video stream only, yet each rendition advertised an AAC-LC track. An audio codec named in CODECS is an audio track the player expects; a strict one (Safari/AVPlayer) can allocate a decoder and wait for media that never arrives. Both are now derived from ffprobe on the encoded rendition rather than written down in advance: avcCodecString maps the reported profile name back to its profile_idc and formats avc1.PPCCLL, and mp4a.40.2 is appended only when the file actually has an audio stream. The recorded codecs on the hls asset row match what the playlist declares, instead of a third hardcoded copy. The pipeline test now asserts against the real encode that every declared codec matches /^avc1\\.[0-9A-F]{6}$/ and that no silent rendition mentions mp4a. Already-rendered revisions keep their incorrect playlist. Nothing reads it yet — there is no decision endpoint and no property serving HLS — and re-rendering them means bumping RENDERER_VERSION, which by design invalidates every cached encode. Worth doing before anything streams these, not worth seven hours of re-encoding today. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 64ba690 commit 578efa3

5 files changed

Lines changed: 95 additions & 6 deletions

File tree

‎lib/ads/video/encode.ts‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,11 +152,48 @@ export function hlsArgs(o: HlsOptions): string[] {
152152
];
153153
}
154154

155+
/**
156+
* RFC 6381 codec string for an H.264 stream: avc1.PPCCLL.
157+
*
158+
* PP is profile_idc, CC the constraint flags, LL the level, each as two hex
159+
* digits. ffprobe reports the profile by name and the level as an integer, so
160+
* the name is mapped back to its idc.
161+
*
162+
* This is derived from the encoded file rather than hardcoded because the
163+
* declaration is load-bearing: a player reads CODECS to choose a decoder
164+
* configuration before it fetches a segment, and a wrong level is a promise
165+
* about the bitstream that the bitstream does not keep.
166+
*/
167+
const PROFILE_IDC: Record<string, number> = {
168+
"Constrained Baseline": 66,
169+
Baseline: 66,
170+
Main: 77,
171+
Extended: 88,
172+
High: 100,
173+
"High 10": 110,
174+
"High 4:2:2": 122,
175+
"High 4:4:4 Predictive": 244,
176+
};
177+
178+
export function avcCodecString(profile: string | undefined, level: number | undefined): string {
179+
const idc = PROFILE_IDC[profile ?? ""] ?? 100;
180+
const lvl = typeof level === "number" && level > 0 ? level : 31;
181+
const hex = (n: number) => n.toString(16).padStart(2, "0").toUpperCase();
182+
// Constraint flags are 0 for the profiles we produce; libx264 sets none that
183+
// belong in this field for High.
184+
return `avc1.${hex(idc)}00${hex(lvl)}`;
185+
}
186+
187+
/** AAC-LC, appended only when a rendition actually carries an audio track. */
188+
export const AAC_LC_CODEC = "mp4a.40.2";
189+
155190
/**
156191
* The multivariant playlist.
157192
*
158193
* Written by hand rather than by ffmpeg's var_stream_map because the bandwidth
159-
* and codec declarations have to describe the renditions we actually produced,
194+
* and codec declarations have to describe the renditions we actually produced
195+
* — including not advertising an audio codec for a silent ad, which is a
196+
* promise a player will wait on,
160197
* and because EXT-X-INDEPENDENT-SEGMENTS is deliberately absent: our segments
161198
* open on a keyframe but are not independently decodable in the sense that tag
162199
* asserts, and claiming it would be a lie a player acts on.

‎lib/ads/video/render.ts‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ import {
1818
import { composeDocument, type ComposeAssets } from "./compose";
1919
import { validateSnapshot, type VideoDesignSnapshot } from "./snapshot";
2020
import {
21+
AAC_LC_CODEC,
22+
avcCodecString,
2123
encodeAudioCompanion,
2224
encodeMp4,
2325
extractPoster,
@@ -232,19 +234,30 @@ export async function renderPreroll(args: {
232234
for (const profile of ["mp4_720p", "mp4_480p"] as const) {
233235
const spec = videoProfile(profile);
234236
const name = `${spec.height}p`;
235-
await packageHls({ inPath: path.join(outDir, `${profile}.mp4`), outDir: hlsDir, name });
237+
const renditionPath = path.join(outDir, `${profile}.mp4`);
238+
await packageHls({ inPath: renditionPath, outDir: hlsDir, name });
236239

237240
const playlist = await readFile(path.join(hlsDir, `${name}.m3u8`), "utf8");
238241
problems.push(...validateMediaPlaylist(playlist));
239242

240-
const mp4Size = statSync(path.join(outDir, `${profile}.mp4`)).size;
243+
// Declare what this rendition actually is. Both halves used to be
244+
// hardcoded and both were wrong: the level said 4.0 for a stream encoded
245+
// at 3.1, and every silent ad advertised an AAC track it did not have.
246+
// A player reads CODECS before fetching a segment, so an audio codec named
247+
// here is an audio track it will wait for.
248+
const probe = await probeMedia(renditionPath);
249+
const v = probe.streams.find((st) => st.codec_type === "video");
250+
const hasAudio = probe.streams.some((st) => st.codec_type === "audio");
251+
const codecs = [avcCodecString(v?.profile, v?.level), ...(hasAudio ? [AAC_LC_CODEC] : [])].join(",");
252+
253+
const mp4Size = statSync(renditionPath).size;
241254
renditions.push({
242255
name,
243256
width: spec.width!,
244257
height: spec.height!,
245258
// Peak bandwidth, derived from the rendition we actually produced.
246259
bandwidth: Math.round((mp4Size * 8) / 5),
247-
codecs: "avc1.640028,mp4a.40.2",
260+
codecs,
248261
});
249262
}
250263

@@ -263,7 +276,7 @@ export async function renderPreroll(args: {
263276
width: null,
264277
height: null,
265278
durationMs: null,
266-
codecs: "avc1.640028,mp4a.40.2",
279+
codecs: renditions.map((r) => r.codecs).join(" "),
267280
validation: { ok: true, problems: [] },
268281
extraFiles: hlsFiles.filter((f) => f !== masterPath),
269282
});

‎lib/ads/video/validate.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ export type ProbeStream = {
3030
avg_frame_rate?: string;
3131
/** Present only with -count_frames; it is a decode, not a header read. */
3232
nb_read_frames?: string;
33+
/** h264 level as an integer, e.g. 31 for level 3.1. */
34+
level?: number;
3335
duration?: string;
3436
sample_rate?: string;
3537
channels?: number;

‎tests/ads-video-compose-validate.test.ts‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import {
77
timeline,
88
} from "@/lib/ads/video/compose";
99
import type { VideoDesignSnapshot } from "@/lib/ads/video/snapshot";
10-
import { GOP_FRAMES, hlsArgs, mp4Args, multivariantPlaylist, posterArgs } from "@/lib/ads/video/encode";
10+
import { AAC_LC_CODEC, avcCodecString, GOP_FRAMES, hlsArgs, mp4Args, multivariantPlaylist, posterArgs } from "@/lib/ads/video/encode";
1111
import {
1212
evaluateProbe,
1313
parseRational,
@@ -195,6 +195,32 @@ describe("encoder arguments carry the media contract", () => {
195195
expect(poster[poster.indexOf("-ss") + 1]).toBe("2");
196196
});
197197

198+
it("derives the avc1 codec string from what was actually encoded", () => {
199+
// Shipped renditions probe as High @ 3.1. The hardcoded string said level
200+
// 4.0 (0x28), which is a promise about the bitstream it does not keep.
201+
expect(avcCodecString("High", 31)).toBe("avc1.64001F");
202+
expect(avcCodecString("High", 40)).toBe("avc1.640028");
203+
expect(avcCodecString("Main", 31)).toBe("avc1.4D001F");
204+
expect(avcCodecString("Constrained Baseline", 30)).toBe("avc1.42001E");
205+
// Unknown input falls back to High @ 3.1 rather than emitting nonsense.
206+
expect(avcCodecString(undefined, undefined)).toBe("avc1.64001F");
207+
});
208+
209+
it("never advertises audio for a silent rendition", () => {
210+
// A player reads CODECS before fetching a segment, so an audio codec named
211+
// here is an audio track it will wait for. Every ad is silent today.
212+
const silent = multivariantPlaylist([
213+
{ name: "720p", width: 1280, height: 720, bandwidth: 700000, codecs: avcCodecString("High", 31) },
214+
]);
215+
expect(silent).toContain('CODECS="avc1.64001F"');
216+
expect(silent).not.toContain(AAC_LC_CODEC);
217+
218+
const narrated = multivariantPlaylist([
219+
{ name: "720p", width: 1280, height: 720, bandwidth: 700000, codecs: `${avcCodecString("High", 31)},${AAC_LC_CODEC}` },
220+
]);
221+
expect(narrated).toContain(AAC_LC_CODEC);
222+
});
223+
198224
it("does not claim independent segments in the multivariant playlist", () => {
199225
const m = multivariantPlaylist([
200226
{ name: "720p", width: 1280, height: 720, bandwidth: 2_400_000, codecs: "avc1.640028" },

‎tests/ads-video-pipeline.test.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,17 @@ describe("a snapshot renders to validated media", () => {
148148
// Never claimed, because we do not guarantee it.
149149
expect(master).not.toContain("EXT-X-INDEPENDENT-SEGMENTS");
150150

151+
// The CODECS declaration must describe what was actually encoded. These
152+
// renditions are silent, so naming an audio codec would be an audio
153+
// track a player waits for and never receives; and the level has to be
154+
// the one in the bitstream, not a guess.
155+
expect(master).not.toContain("mp4a");
156+
const declared = [...master.matchAll(/CODECS="([^"]+)"/g)].map((m) => m[1]);
157+
expect(declared.length).toBe(2);
158+
for (const codec of declared) {
159+
expect(codec, `codecs=${codec}`).toMatch(/^avc1\.[0-9A-F]{6}$/);
160+
}
161+
151162
expect(byProfile.poster.byteSize).toBeGreaterThan(0);
152163
expect(byProfile.poster.contentType).toBe("image/webp");
153164
},

0 commit comments

Comments
 (0)