music-video-gen

Author SHA1 Message Date

Author	SHA1	Message	Date
Dejvino	b7c6fd1c5a	Fix export frame loss: reject codec profiles that reorder frames The exporter was losing roughly three of every four frames. A 20s render muxed 319 samples instead of 1200 and played at 15.9 fps, with no error reported and a file that looked superficially fine. The cause was B-frames. WebCodecs delivers chunks in decode order, but EncodedVideoChunk carries only a presentation timestamp — there is no decode timestamp to recover the real order from. Handed presentation timestamps as if they were decode timestamps, mp4-muxer saw DTS run backwards and rejected every reordered chunk. That throw happened inside the encoder's output callback, where it could not reach the export loop, so it surfaced as an uncaught error and the render carried on. Only the I/P frames survived, one per 4-frame GOP, which is exactly the stts pattern the files showed. Writing the correct timeline instead is not available to us: it needs negative composition offsets, and mp4-muxer emits ctts as a version-0 box, which is unsigned. isConfigSupported says nothing about reordering, and measurement showed latencyMode: 'realtime' does not prevent it either. So pickVideoConfig now test-encodes 12 frames per candidate and checks the order they come back in, taking the first profile that does not reorder. Candidates stay in quality order, high profile down to baseline, so browsers that never reorder keep the better profiles; baseline forbids B-slices by spec and is the guaranteed floor. If every supported profile reorders the export fails up front rather than after minutes of rendering. Two guards so this class of loss cannot be silent again: - The output callback catches, routing muxer rejections to the error list the loop actually checks. - Frames in and chunks accepted are counted and compared after flush, with the reordering count and a gap histogram alongside. The count deliberately tracks chunks the muxer took, not chunks that arrived — counting arrivals reports success for frames rejected a line later. Failures now raise a toast over the stage that stays until dismissed. An export that dies after a long render should not sit unread in a panel. Also corrects the record from `d31d0fc`, which claimed VideoEncoder.encode() silently drops frames once its queue saturates. It does not: the queue grows without bound and the only cost is memory. That commit's dequeue-gated backpressure addressed a mechanism that does not exist and is reverted here; the queue poll it replaced is restored, described honestly as a memory bound. The opus resampling from that commit was a real fix and is untouched. tools/probe-mp4.js reports per-track timescale, sample count and the stts table, which is what identified the fault and what verifies a good export: one row of [N x 1]. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-06 13:10:24 +02:00

Dejvino

b7c6fd1c5a

Fix export frame loss: reject codec profiles that reorder frames

The exporter was losing roughly three of every four frames. A 20s render
muxed 319 samples instead of 1200 and played at 15.9 fps, with no error
reported and a file that looked superficially fine.

The cause was B-frames. WebCodecs delivers chunks in decode order, but
EncodedVideoChunk carries only a presentation timestamp — there is no decode
timestamp to recover the real order from. Handed presentation timestamps as
if they were decode timestamps, mp4-muxer saw DTS run backwards and rejected
every reordered chunk. That throw happened inside the encoder's output
callback, where it could not reach the export loop, so it surfaced as an
uncaught error and the render carried on. Only the I/P frames survived, one
per 4-frame GOP, which is exactly the stts pattern the files showed.

Writing the correct timeline instead is not available to us: it needs
negative composition offsets, and mp4-muxer emits ctts as a version-0 box,
which is unsigned. isConfigSupported says nothing about reordering, and
measurement showed latencyMode: 'realtime' does not prevent it either.

So pickVideoConfig now test-encodes 12 frames per candidate and checks the
order they come back in, taking the first profile that does not reorder.
Candidates stay in quality order, high profile down to baseline, so browsers
that never reorder keep the better profiles; baseline forbids B-slices by
spec and is the guaranteed floor. If every supported profile reorders the
export fails up front rather than after minutes of rendering.

Two guards so this class of loss cannot be silent again:

- The output callback catches, routing muxer rejections to the error list
  the loop actually checks.
- Frames in and chunks accepted are counted and compared after flush, with
  the reordering count and a gap histogram alongside. The count deliberately
  tracks chunks the muxer took, not chunks that arrived — counting arrivals
  reports success for frames rejected a line later.

Failures now raise a toast over the stage that stays until dismissed. An
export that dies after a long render should not sit unread in a panel.

Also corrects the record from d31d0fc, which claimed VideoEncoder.encode()
silently drops frames once its queue saturates. It does not: the queue grows
without bound and the only cost is memory. That commit's dequeue-gated
backpressure addressed a mechanism that does not exist and is reverted here;
the queue poll it replaced is restored, described honestly as a memory bound.
The opus resampling from that commit was a real fix and is untouched.

tools/probe-mp4.js reports per-track timescale, sample count and the stts
table, which is what identified the fault and what verifies a good export:
one row of [N x 1].

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-06 13:10:24 +02:00

1 Commits