b7c6fd1c5a
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Dejvino
|
b7c6fd1c5a |
Fix export frame loss: reject codec profiles that reorder frames
The exporter was losing roughly three of every four frames. A 20s render
muxed 319 samples instead of 1200 and played at 15.9 fps, with no error
reported and a file that looked superficially fine.
The cause was B-frames. WebCodecs delivers chunks in decode order, but
EncodedVideoChunk carries only a presentation timestamp — there is no decode
timestamp to recover the real order from. Handed presentation timestamps as
if they were decode timestamps, mp4-muxer saw DTS run backwards and rejected
every reordered chunk. That throw happened inside the encoder's output
callback, where it could not reach the export loop, so it surfaced as an
uncaught error and the render carried on. Only the I/P frames survived, one
per 4-frame GOP, which is exactly the stts pattern the files showed.
Writing the correct timeline instead is not available to us: it needs
negative composition offsets, and mp4-muxer emits ctts as a version-0 box,
which is unsigned. isConfigSupported says nothing about reordering, and
measurement showed latencyMode: 'realtime' does not prevent it either.
So pickVideoConfig now test-encodes 12 frames per candidate and checks the
order they come back in, taking the first profile that does not reorder.
Candidates stay in quality order, high profile down to baseline, so browsers
that never reorder keep the better profiles; baseline forbids B-slices by
spec and is the guaranteed floor. If every supported profile reorders the
export fails up front rather than after minutes of rendering.
Two guards so this class of loss cannot be silent again:
- The output callback catches, routing muxer rejections to the error list
the loop actually checks.
- Frames in and chunks accepted are counted and compared after flush, with
the reordering count and a gap histogram alongside. The count deliberately
tracks chunks the muxer took, not chunks that arrived — counting arrivals
reports success for frames rejected a line later.
Failures now raise a toast over the stage that stays until dismissed. An
export that dies after a long render should not sit unread in a panel.
Also corrects the record from
|
||
|
Dejvino
|
44e1826fcb |
Six more visualizers, built through the scaffolder
Seven per family now, 42 scenes. Each one exists because of the second
line of its header comment — what makes it different from the scenes it
sits beside — since "no two scenes render the same image" is a gate with a
numeric floor:
Smoke Column (flow) a plume with a source, a body and a
dissipating head. Everything moves one way and
widens; an isotropic field has no up.
Cell Divide (organic) a partition, not objects. Every pixel belongs
to a cell, boundaries are hard, no background.
Eclipse Field (minimal) the occluder nearly fills the frame and is not
the subject — the corona around it is. The
library's one high-contrast minimal scene.
Girder Lattice (structural) the only scene whose subject is ABOVE the
camera; perspective converges downward.
Quasicrystal (geometric) five plane waves at incommensurate angles, so
the pattern has local symmetry and no tile,
no cell and no centre.
Time Smear (glitch) nothing is displaced. Each band shows the same
image at a different age — a slit-scan built
from one feedback buffer by giving each band
its own persistence.
The scaffolder produced six skeletons that passed lint and every gate
before a line of shader was written, and all six passed their per-scene
gate first time once written. That is what it was for.
Two real faults, both found by gates rather than by eye:
Cell Divide coloured each pixel by its nearest seed, and exactly on a tie
which seed is nearest comes down to the last bit of a distance. Two renders
disagreed by a whole palette step: 6/255 against a ceiling of 1, and it
also broke Phase 6's preview/export parity because that look casts it.
Blending the nearest tint with the runner-up across the membrane makes the
two answers agree in the limit — and reads better, as membranes rather than
cuts.
The lint's own "prev() with no base image" rule fired on a scene whose
comment explained why it does NOT rely on prev(). Rules that ask "does the
code do X" now run against a comment-stripped copy; the fixed-cost opt-out
still reads the original, since it is a comment.
Phase 6's parity tolerance goes from 1/255 to 2. Measured in order:
unprimed, the first pass differed on frames 0/2/3 — fixed earlier by
priming. Primed and isolated: 0/40 at delta 0, three times over. Primed,
range-warmed and run at the end of the full suite: three frames at delta 2.
The warm-up stays because it is correct, but the residue is GPU load, not
logic, and it is the same 1-2/255 Phases 5 and 7 already account for. A
real divergence scores in the tens.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Dejvino
|
2806ef1386 |
Phase 10: variety, twelve scenes, and tooling to write the next one
Watching several finished tracks side by side turned up the problem neither Phase 8 (too few cuts) nor Phase 9 (no through-line) addressed: the same scene cast in two different videos looked like the same footage twice. Section bias is nearly identical between two tracks' drops, so both sampled their parameters around the same centre, and the library's own averageness did the rest. Three answers, none of them a new scene: Temperament — a per-track hand on every parameter dial: intensity, pace, detail, and an extremity that decides how far toward the ends of a range the track is willing to sample. Bias comes from the section and is shared between tracks; temperament comes from the track and is not. Overlays — sometimes a second full scene composited over the shot, from a different family, in a blend that preserves what is underneath and never above 0.6 opacity. Not always: a stack that always doubled up would read as permanently cluttered rather than as occasionally layered. A wider palette — hue now derives from SPECTRAL TILT, the log ratio of treble to body. The centroid is a number most masters sit in the middle of, and the plain body/(body+treble) fraction is worse: low frequencies carry most of the energy in all music, so it read 0.98-1.00 for everything and four different battery tracks came out within 0.02 of each other. The ratio is multiplicative, so its logarithm is what spreads — the same four measure -9.3, -5.0, -4.1, -3.8. Also both ways round the wheel (violet, magenta and pink were unreachable by construction), four new schemes, and seeded chroma profile and lightness curve. Closest battery pair went from 0.005 to 0.113. Twelve scenes take the library to 36, six per family: Aurora Veil, Vortex Drift, Tide Rings, Ink Bleed, Dust Chamber, Salt Flat, Cargo Belt, Gate Corridor, Circuit Bloom, Truchet Fold, Signal Decay, Storm Rift. Weighted toward the 'space' and 'shape' traits, which were thinnest and so the signatures most likely to run a track out of cast — the Phase 9 casting rule means the pool a track draws from is smaller than the library. Also fixes a real one in shots.js: heavy LRU weighting was not enough to make a section reach its whole roster, and a five-shot section still came out 0,2,0,2,0 about a fifth of the time. An unseen companion now wins outright; which one is still free, so only the coverage is guaranteed. Block Mosh declared the camera trait, assigned sigCamera(p) to a p it then never read, and passed the lint's evidence grep. The Phase 9 render gate measured its response to the camera at exactly zero. --- tooling --- Adding a scene was mostly boilerplate and round-trips, which is expensive in both senses. The irreducible cost is the shader body; everything around it is now mechanical: npm run new:scene -- "Name" --family=... --traits=... writes the module, registers it, and leaves a skeleton that already passes every gate, with name-derived constants so two skeletons are not twins. The lint grew the rules that previously needed a GPU to catch: the dead camera above, prev() with no base image, and large loops with no early break (with a `// lint: fixed-cost` opt-out for a genuinely fixed-cost sampling loop). checks.html?scene=Name runs the per-scene acceptance battery for one scene — ten lines and a verdict instead of rendering the whole library to find out whether one shader is alive. The same procedure is a repo skill under .claude/skills/build-visualizer/. --- checks changed, with the measurements --- P5 determinism compared two WebGL CONTEXTS, which is not what it is for. Measured: one context is bit-exact over 40 frames with feedback at 0.6; two contexts disagree by up to 2/255 whether feedback is on or off. It now asserts generation is byte-identical (hard) and rendering within 2/255, since feedback compounds single-level variance. P10's cross-track comparison measures distance RELATIVE to how much image there is. Most scenes are mostly dark, so two genuinely different renders — 25 bars against 53 — scored under 0.02 absolute purely because the black background agrees with itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Dejvino
|
ca5a68eb84 |
Phases 8 and 9: shots, and a production design per track
Both phases come out of the manual gate — watching whole tracks — and both
fix something no automated check was looking for.
Phase 8: shots. A section is a STAGE of the song and can run ninety
seconds; one scene held that long reads as a still image with a wobble on
it. Each section kind now gets a roster of three or four stage visuals
instead of one scene, and each section is cut into shots that rotate
between them on phrase lines, never holding past 22s. The roster stays per
kind, so a track's drops still cut between the same images and the video
keeps its identity; the anchor opens each section and the rotation returns
to it, and when a companion is due it is the least recently shown one.
The arc driver stopped working in sections and started working in cues, one
per shot, so a shot cut and a section change take the same code path and
differ only in transition length. The default transition is a slow
dissolve — two bars calm, one loud; a straight cut is reserved for
sections above the energy threshold, because on calm material a cut reads
as a glitch rather than as an edit.
Phase 9: production design. With cuts every fifteen seconds the next
problem was that the images being cut between shared nothing but the
palette. What a music video actually shares across shots is a location, a
cast, a camera operator and an art direction, so each track now generates a
personality in four traits (shape, camera, space, style) off the look seed.
The traits reach shaders as uniforms plus four helpers in the contract, and
each scene expresses them its own way: Classic Wave's rings take the
signature polygon, Metaballs merge as one, Floating Geometry no longer
picks between a box and a circle because the production already decided.
The part that makes it a design rather than a filter: scenes DECLARE which
traits they honour, a track is built on one or two, and a scene that does
not honour all of them is not cast in that track. The library shrinks per
track on purpose.
Two gates keep the declaration honest — lint greps each shader for evidence
of every trait it claims, and a render check measures that each declared
trait actually moves the image (41 scene/trait pairs, weakest response 64
of 255). A layer with no personality renders bit-identically to before,
which is what keeps every earlier sweep and regression valid.
Checks changed rather than added:
- P4 scene-change and drift checks now measure per shot, not per section;
the crossfade check reads its length off the cue.
- P5 flash sweep runs per shot, so the visuals that only appear
mid-section are measured too.
- P6 preview/export parity primes first (as both real paths do) and
compares at the one-LSB tolerance Phase 7 already uses. Measured over
four consecutive shows: 3 frames at delta 1, then bit-exact — GPU
variance on first render, not a divergence.
- P2's contract-uniform list is derived from the contract instead of
retyped, so the signature uniforms cannot fall out of sync.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Dejvino
|
9d15c3cf49 |
Phase 7: grow the scene library to 16
Ten new scenes, with 'minimal' first: the family was empty, so intros and breakdowns fell through to flow/organic and every track opened at full density. Quiet sections now land on a restful family 48/48 times across 24 seeds, 28 of them minimal. New: Horizon Lines, Spectrum Sculpture, Slow Orb (minimal); Curl Flow (flow); Plasma Bloom, Metaballs (organic); Kaleido Tunnel, Moiré Grid (geometric); Ridge Terrain (structural); Scan Tear (glitch). Four real bugs, three of which the existing gates could not have caught: 1. SHADER PROGRAMS LINK ASYNCHRONOUSLY. three.js uses KHR_parallel_shader_compile, so draws against an unlinked program render wrong. The heaviest scene had its first TEN frames differ from every later render of the same frames. Preview hides this entirely; export renders each frame once, so those frames would ship broken. Added Engine.prime() — WebGLRenderer.compile() plus a discarded warm frame — and the exporter now primes before encoding. Rendering a throwaway frame and reading it back is NOT sufficient; measured, it left 3-5 frames wrong. 2. Moiré Grid declared a param on u_width, which the shader contract already uses for stereo width. GLSL redefinition, and the only symptom was a black frame. Lint now rejects any param uniform colliding with the contract. 3. Spectrum Sculpture strobed at 4 flashes/s. Two causes: rotation measured in turns meant bar-crossing frequency was bars x rate (82 bars put a slow-looking 0.12 turns/s at 10 Hz), and hard band-tier boundaries made every bar switch band simultaneously. Rotation is now in segment units so the rate IS the crossing frequency, bands interpolate, and the range is capped where the flash meter measures zero. 4. Particle Field was being chosen as a primary background despite being mostly empty by design. Scenes now declare role: 'accent'; those are never primary and are judged on variance rather than luminance. Three checks were themselves wrong and were rebuilt: mean-distance metrics unfairly fail sparse scenes for being tasteful rather than static, so "animates" and "no duplicates" now use max channel delta. PLAN.md §1 gains two refinements: programs must be primed before the first frame, and even same-machine the heaviest shaders vary by one LSB under differing GPU load — so the per-scene criterion is max delta <= 1 rather than an identical hash. A real bug scores in the tens there. Full suite 67/67 across all seven phases. Worst 4K frame 3.8ms, worst flash rate 0/s, worst determinism delta 1/255. Adds README.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Dejvino
|
59180948d0 |
Phase 5: compositing depth ("C" complete)
Multi-layer stacks with blend modes, feedback, post chain, and a 3D particle layer proving the compositor is genuinely hybrid. Looks now generate accent layers from a different family, composited additively at low opacity, weighted by section energy so intros stay sparse. Adds flash-rate safety (engine/flash.js), which was not in the original plan and should have been. This generates beat-reactive video for publication, and rapid light-dark cycling is the photosensitive-epilepsy trigger; WCAG 2.3.1 caps it at three flashes per second. Classic Wave measured 7-8/s at every output resolution from 96x54 to 1920x1080, so it was a real hazard rather than a sampling artefact. Root cause was general, not one bad shader: `u_time * u_speed` where speed is reactively modulated. Phase is elapsed*rate, so changing the rate at time T jumps phase by T*delta — sixty seconds in, a 0.05 wobble throws the phase three whole units between consecutive frames, and it worsens as the track runs. Fixed by introducing rate params: - schema flag `rate: true` documents and marks them - Layer.resolveParams skips reactivity on them - ArcDriver skips drift on them - validateModule rejects a reactive entry on one - lint-scenes greps shaders for `u_time * u_X` and fails if X is unmarked, so no future scene can reintroduce it Every rate param across the six scenes is now marked. Two checks were needed to find this: a per-look flash check, and a per-SCENE sweep at aggressive params, since the look generator only samples part of the space and a scene can hide an unsafe region for a long time. Gate 10/10. Worst flash rate now 1/s. Feedback stable over 10,000 frames (luminance 0.17-0.59, no saturation or decay). 0.17ms/frame at 1280x720. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Dejvino
|
7e31c19d6e |
Phase 0: determinism spine
Engine core: Timeline (fixed dt, audio-mastered in realtime), seeded Rng, Renderer, Layer/ShaderLayer/SceneLayer, Compositor with blend modes, feedback and post chain. Shader scenes are compiled against a fixed uniform contract and define only `vec4 scene(vec2 uv, vec2 p)`, so adding a scene costs a shader plus a params block. Deep Nebula ported from party-stage as the first one. Gate passes, 7/7 in checks.html: - 300 frames rendered twice are bit-identical - a fresh Engine reproduces the same frames - simulated dropped frames change nothing (proves dt is fixed) - seek matches sequential playback - 320x180 vs 1280x720 agree within 0.010 (limit 0.06) - seeded rng reproducible, forked streams independent - compositor reset clears feedback history Static gates: no wall-clock or unseeded randomness in deterministic directories; scene schemas and shader sources agree in both directions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |