b7c6fd1c5a
The exporter was losing roughly three of every four frames. A 20s render
muxed 319 samples instead of 1200 and played at 15.9 fps, with no error
reported and a file that looked superficially fine.
The cause was B-frames. WebCodecs delivers chunks in decode order, but
EncodedVideoChunk carries only a presentation timestamp — there is no decode
timestamp to recover the real order from. Handed presentation timestamps as
if they were decode timestamps, mp4-muxer saw DTS run backwards and rejected
every reordered chunk. That throw happened inside the encoder's output
callback, where it could not reach the export loop, so it surfaced as an
uncaught error and the render carried on. Only the I/P frames survived, one
per 4-frame GOP, which is exactly the stts pattern the files showed.
Writing the correct timeline instead is not available to us: it needs
negative composition offsets, and mp4-muxer emits ctts as a version-0 box,
which is unsigned. isConfigSupported says nothing about reordering, and
measurement showed latencyMode: 'realtime' does not prevent it either.
So pickVideoConfig now test-encodes 12 frames per candidate and checks the
order they come back in, taking the first profile that does not reorder.
Candidates stay in quality order, high profile down to baseline, so browsers
that never reorder keep the better profiles; baseline forbids B-slices by
spec and is the guaranteed floor. If every supported profile reorders the
export fails up front rather than after minutes of rendering.
Two guards so this class of loss cannot be silent again:
- The output callback catches, routing muxer rejections to the error list
the loop actually checks.
- Frames in and chunks accepted are counted and compared after flush, with
the reordering count and a gap histogram alongside. The count deliberately
tracks chunks the muxer took, not chunks that arrived — counting arrivals
reports success for frames rejected a line later.
Failures now raise a toast over the stage that stays until dismissed. An
export that dies after a long render should not sit unread in a panel.
Also corrects the record from d31d0fc, which claimed VideoEncoder.encode()
silently drops frames once its queue saturates. It does not: the queue grows
without bound and the only cost is memory. That commit's dequeue-gated
backpressure addressed a mechanism that does not exist and is reverted here;
the queue poll it replaced is restored, described honestly as a memory bound.
The opus resampling from that commit was a real fix and is untouched.
tools/probe-mp4.js reports per-track timescale, sample count and the stts
table, which is what identified the fault and what verifies a good export:
one row of [N x 1].
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
86 lines
3.9 KiB
HTML
86 lines
3.9 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>flow-state</title>
|
|
<link rel="stylesheet" href="/src/ui/style.css">
|
|
</head>
|
|
<body>
|
|
<div id="app">
|
|
<div id="stage">
|
|
<canvas id="canvas"></canvas>
|
|
<div id="overlay">
|
|
<div id="dropzone">
|
|
<div class="dz-inner">
|
|
<div class="dz-title">flow-state</div>
|
|
<div class="dz-sub">drop an audio file, or click to choose</div>
|
|
<div class="dz-hint">mp3 · flac · wav · ogg</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div id="hud" hidden></div>
|
|
<div id="toast" hidden></div>
|
|
</div>
|
|
|
|
<div id="transport">
|
|
<div id="timeline">
|
|
<canvas id="timeline-canvas"></canvas>
|
|
</div>
|
|
<div id="controls">
|
|
<button id="btn-play" title="Play / pause (space)">▶</button>
|
|
<button id="btn-prev-section" title="Previous section (←)">⇤</button>
|
|
<button id="btn-next-section" title="Next section (→)">⇥</button>
|
|
<button id="btn-loop" title="Loop current section (L)">↻</button>
|
|
<span id="time-display">0:00 / 0:00</span>
|
|
<!-- The section readout is the row's only flexible item: it
|
|
absorbs all the free space and centres its own text, which
|
|
is what the two spacers used to do. They competed with it
|
|
for that space and squeezed it to nothing. -->
|
|
<span id="section-display"></span>
|
|
<label class="ctl">quality
|
|
<select id="sel-quality">
|
|
<option value="draft">draft</option>
|
|
<option value="full" selected>full</option>
|
|
</select>
|
|
</label>
|
|
<button id="btn-reroll" title="New seed for the whole track">reroll</button>
|
|
<button id="btn-reroll-section" title="New scene for this section only">reroll section</button>
|
|
<button id="btn-lock" title="Lock this section against rerolls">lock</button>
|
|
<button id="btn-hud" title="Toggle debug HUD (D)">hud</button>
|
|
<button id="btn-osd" title="Toggle the title plate in the corner (O)">osd</button>
|
|
<button id="btn-segment" title="Render 20s around the playhead at export quality">test render</button>
|
|
<button id="btn-export" class="primary" title="Export the full video">export</button>
|
|
</div>
|
|
</div>
|
|
|
|
<aside id="panel">
|
|
<div id="track-header">
|
|
<div class="th-top-row">
|
|
<div class="th-info">
|
|
<span id="th-label" class="th-label">no track loaded</span>
|
|
<span id="th-name" class="th-name" hidden>no track</span>
|
|
</div>
|
|
<button id="btn-change-track" class="th-btn" title="Choose or drop an audio file">choose track</button>
|
|
</div>
|
|
<div id="th-progress" class="th-progress" hidden>
|
|
<div id="th-step" class="th-step">decoding…</div>
|
|
<div class="an-bar"><div id="th-fill" class="an-fill"></div></div>
|
|
</div>
|
|
</div>
|
|
<div id="panel-tabs">
|
|
<button data-tab="look" class="active">look</button>
|
|
<button data-tab="scene">scene</button>
|
|
<button data-tab="post">post</button>
|
|
<button data-tab="export">export</button>
|
|
</div>
|
|
<div id="panel-body"></div>
|
|
</aside>
|
|
</div>
|
|
|
|
<input type="file" id="file-input" accept="audio/*" hidden>
|
|
<audio id="audio" hidden></audio>
|
|
<script type="module" src="/src/main.js"></script>
|
|
</body>
|
|
</html>
|