The exporter was losing roughly three of every four frames. A 20s render
muxed 319 samples instead of 1200 and played at 15.9 fps, with no error
reported and a file that looked superficially fine.
The cause was B-frames. WebCodecs delivers chunks in decode order, but
EncodedVideoChunk carries only a presentation timestamp — there is no decode
timestamp to recover the real order from. Handed presentation timestamps as
if they were decode timestamps, mp4-muxer saw DTS run backwards and rejected
every reordered chunk. That throw happened inside the encoder's output
callback, where it could not reach the export loop, so it surfaced as an
uncaught error and the render carried on. Only the I/P frames survived, one
per 4-frame GOP, which is exactly the stts pattern the files showed.
Writing the correct timeline instead is not available to us: it needs
negative composition offsets, and mp4-muxer emits ctts as a version-0 box,
which is unsigned. isConfigSupported says nothing about reordering, and
measurement showed latencyMode: 'realtime' does not prevent it either.
So pickVideoConfig now test-encodes 12 frames per candidate and checks the
order they come back in, taking the first profile that does not reorder.
Candidates stay in quality order, high profile down to baseline, so browsers
that never reorder keep the better profiles; baseline forbids B-slices by
spec and is the guaranteed floor. If every supported profile reorders the
export fails up front rather than after minutes of rendering.
Two guards so this class of loss cannot be silent again:
- The output callback catches, routing muxer rejections to the error list
the loop actually checks.
- Frames in and chunks accepted are counted and compared after flush, with
the reordering count and a gap histogram alongside. The count deliberately
tracks chunks the muxer took, not chunks that arrived — counting arrivals
reports success for frames rejected a line later.
Failures now raise a toast over the stage that stays until dismissed. An
export that dies after a long render should not sit unread in a panel.
Also corrects the record from d31d0fc, which claimed VideoEncoder.encode()
silently drops frames once its queue saturates. It does not: the queue grows
without bound and the only cost is memory. That commit's dequeue-gated
backpressure addressed a mechanism that does not exist and is reverted here;
the queue poll it replaced is restored, described honestly as a memory bound.
The opus resampling from that commit was a real fix and is untouched.
tools/probe-mp4.js reports per-track timescale, sample count and the stts
table, which is what identified the fault and what verifies a good export:
one row of [N x 1].
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Grain was in every video. It was added twice unconditionally — every scene
called sigGrain, and the grade added its own on top — so the only thing that
varied between two tracks was how much of it there was. That makes grain the
renderer's fingerprint rather than a decision about one video.
It is now described rather than dialled (look/grain.js): a mode (off /
constant / swell / sections / transient), a cell size in pixels, a refresh
rate in frames, a mask (uniform, shadows, highlights, edges, bands) and a
chroma amount. Roughly 45% of tracks get none at all. The non-constant modes
carry a per-frame envelope computed in Show._postAt from frame and features
only, so preview and export still agree. Scene-side grain is gated the same
way, and a module can decline it outright with `texture: 0` — crisp line work
should stay crisp. The post tab grew a real grain block so any of it can be
forced per track.
Slow songs got fast scenes. `motion` bias was mostly section energy with
tempo as a small correction, so a 70bpm track's drop asked for nearly as much
speed as a 150bpm one. Motion is now tempo-dominated, and every `rate: true`
param is additionally scaled by a per-track rateScale — measured, 84bpm now
samples its rate params at 0.276 of range against 148bpm's 0.571.
Parameter sampling also commits harder: extremity starts at 0.45 rather than
0.25 and shapes the draw more aggressively. This was first pushed to 0.82 and
backed off to 0.72, because the gates caught the overshoot — seeds began
collapsing onto the same range ends and a sparse scene sampled at its low end
rendered effectively black.
Fallout worth recording: turning the default grade grain off exposed two
scenes that were never really animating. Dust Chamber and Eclipse Field
passed the Phase 7 movement gate only because per-pixel noise was moving
underneath them; both now breathe on their own fixed clock, and Dust Chamber
needed a brightness floor as well. Four scenes (Classic Wave, Silk Ribbon,
Kaleido Tunnel, Slow Orb) express the style trait ONLY through grain and so
cannot opt out yet; they hold a reduced share at 0.35 pending real edge and
softness response.
Phase 3's look-space check now measures its closest pair relative to image
brightness, the same correction Phase 10 already documents for sparse scenes:
the absolute number was being propped up by grain rather than by look-space
width. Five new Phase 10 checks cover grain distribution, treatment variety,
envelope range, the texture opt-out, and tempo. 93/93 pass.
Two transport bugs, both stale state surfacing in the UI:
- Loading a track replaces audio.src, which stops playback silently, so
state.playing stayed true and the play button stayed on pause — the first
click after a track change only flipped the flag back. Added stopPlayback().
- The controls row wrapped mid-song because two readouts that change length
while playing were sized by their content: the clock crossing ten minutes
and the section label whenever a scene name is long or a crossfade appears.
The clock is now fixed-width with tabular figures and the section label is
the row's only flexible item, laying out at zero width and ellipsising into
whatever space is left. The two spacers competed with it for that space and
are gone; its own text-align does their job.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Engine core: Timeline (fixed dt, audio-mastered in realtime), seeded Rng,
Renderer, Layer/ShaderLayer/SceneLayer, Compositor with blend modes,
feedback and post chain.
Shader scenes are compiled against a fixed uniform contract and define only
`vec4 scene(vec2 uv, vec2 p)`, so adding a scene costs a shader plus a
params block. Deep Nebula ported from party-stage as the first one.
Gate passes, 7/7 in checks.html:
- 300 frames rendered twice are bit-identical
- a fresh Engine reproduces the same frames
- simulated dropped frames change nothing (proves dt is fixed)
- seek matches sequential playback
- 320x180 vs 1280x720 agree within 0.010 (limit 0.06)
- seeded rng reproducible, forked streams independent
- compositor reset clears feedback history
Static gates: no wall-clock or unseeded randomness in deterministic
directories; scene schemas and shader sources agree in both directions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
