dejvino/music-video-gen
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Every shot stands on something, and the frame has two ends

Browse Source 
A section used to be one scene, and two thirds of the library is composable —
sparse by design, elements ON something. Cast as backgrounds anyway, they left
9 of 40 sampled frames under 20% painted, the darkest at 0.3%: a minute and a
half of a few bright things on black, invisible to every gate because every
gate on the stack was a limit rather than a floor.

Every section now stands on a GROUND: a canvas that fills the frame, cast per
section kind so a shot cut changes the shot and not the world. When the shot
fills the frame itself it IS the ground — two canvases stacked is two pictures
fighting. Above that, a coverage BUDGET: director appetite times the section's
energy times where the story is, capped at two frames' worth of material.

The measured facts move into the repo. scenes/metadata.json is generated from
the gallery — coverage as a shot, coverage as a bed, variety, the structural
profile — tracked in git, stamped with a fingerprint of the scenes and the
metric definitions, and refreshed from gallery.html. `surface` is derived from
it rather than declared; nine scenes claimed `canvas` while painting under a
third of the frame, and declaring it is now a lint error. The generator weights
every layering choice by measured structural distance, because family labels
and the render disagree: two `geometric` scenes can be 0.31 apart and a `flow`
and an `organic` scene 0.04.

The gallery's 0.1 red line is gone. It was right when a section was one scene
and wrong now — nineteen scenes were failing a bar for being consistent, which
is a virtue in an ingredient.

Chasing the numbers turned up four real faults:

  * A scene that reads prev() cannot be a ground. It returns the whole
    composited frame including the layers above it, so a datamosh under a shot
    is eating it: the render stopped reproducing from a seek and two WebGL
    contexts diverged by 91/255 against a tolerance of 4.
  * Screen was the wrong operator for a shot over a bed. It lightens, so a
    median quarter of every frame clipped to paper and whole sections rendered
    100% white. Replaced by a lumakey — the shot's brightness is its alpha.
  * Feedback was an accumulator: a still image settled at 2.3x its own
    brightness. Fine over black, fatal over a filled ground. Normalised at 0.6,
    plus a highlight shoulder so the top rolls off instead of clipping.
  * useTrack never prewarmed, so a fresh Show's first frame differed from every
    later render of it — the export-breaking hazard Compositor.prime documents.

Blazing is a decision now, not a side effect: directors declare an appetite for
it, a section must be loud and late in the story to earn one, and quiet kinds
never do. The ceiling gate matches that — a hard cap per section, and no more
than a fifth of them hot at all.

Rendered across twelve videos, middle of every section:

    painted        51% mean, darkest 0.3%  ->  87% mean, darkest 43%
    clipped white  24% median, worst 100%  ->   1% mean, worst 30%
    separation     0.10                    ->  0.44

Seven scenes can ground a section — five geometric, two organic — so every
quiet section of every video stands on one of two beds. That is the library's
largest hole and it is scene work: there is no minimal or flow canvas that
fills half the frame without reading prev().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dejvino
2026-08-20 09:02:01 +02:00
co-authored by Claude Opus 5
parent 7d151be6e1
commit 89e05459c0
68 changed files with 9896 additions and 293 deletions
Download Patch File Download Diff File Expand all files Collapse all files
flow-state/src/main.js
+58 -11
View File
@@ -7,6 +7,8 @@ import { toHex } from './look/palette.js';
import { applyGrainToPost, describeGrain, GRAIN_MASKS, GRAIN_MODES } from './look/grain.js';
import { renderClickTrack, audioBufferToWavBlob } from './audio/metronome.js';
import { Exporter, exportSegment, downloadBlob, PRESETS, isSupported } from './export/Exporter.js';
import { subjectOf, groundOf, overlaysOf, stackCoverage } from './look/stack.js';
import { coverageOf as sceneCoverage } from './scenes/surface.js';
const QUALITY = {
draft: 0.5, // half resolution — for scrubbing heavy stacks
@@ -56,7 +58,7 @@ const paramPanel = new ParamPanel(document.createElement('div'), onParamChange);
// ---------------------------------------------------------------- loading
async function loadFile(file) {
async function loadFile(file, { seed = null } = {}) {
if (state.busy) return;
state.busy = true;
stopPlayback();
@@ -76,7 +78,7 @@ async function loadFile(file) {
await state.show.load(file, (stage, fraction) => {
dom.thStep.textContent = stage;
dom.thFill.style.width = `${Math.round((fraction || 0) * 100)}%`;
});
}, { seed });
dom.audio.src = URL.createObjectURL(file);
dom.thLabel.textContent = 'loaded track';
@@ -113,6 +115,35 @@ if (dom.changeTrack) dom.changeTrack.addEventListener('click', () => dom.fileInp
dom.fileInput.addEventListener('change', (e) => {
if (e.target.files[0]) loadFile(e.target.files[0]);
});
/**
* `?song=centre` — open a bank song straight from a debug page.
*
* The filmstrip reduces a song to nine stills; the only way to argue with that
* reading is to watch the thing move, and until now that meant remembering
* which of seventeen near-identically-named wavs to drag in. `&seed=` carries
* the strip's seed across so the video you watch is the one it measured.
*
* Dev-only in practice: `test/songs/` is served by vite from the project root
* and is not part of a build. The name is matched against a bare word rather
* than the bank list so the app does not have to import the synth.
*/
async function loadBankSong(name, seed) {
if (!/^[a-z0-9_-]+$/i.test(name)) return;
const url = `/test/songs/${name}.wav`;
try {
const res = await fetch(url);
if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
const blob = await res.blob();
await loadFile(new File([blob], `${name}.wav`, { type: 'audio/wav' }), { seed });
} catch (err) {
dom.thLabel.textContent = 'no such song';
dom.thName.hidden = false;
dom.thName.textContent = name;
dom.thStep.textContent = `${url}: ${err.message} — run npm run build:songs`;
console.error(err);
}
}
document.addEventListener('dragover', (e) => e.preventDefault());
document.addEventListener('drop', (e) => {
e.preventDefault();
@@ -265,7 +296,8 @@ function renderPanel() {
if (state.tab === 'scene') {
paramPanel.container = dom.panelBody;
paramPanel.build(section.layers[0].module, section.layers[0].params);
const subject = subjectOf(section.layers);
paramPanel.build(subject.module, subject.params);
// The section's stage visuals, with the one currently on screen marked.
// Params above edit the anchor (variant 0) — the image the section opens
@@ -278,20 +310,29 @@ function renderPanel() {
list.innerHTML = `<div class="pp-sub">stage visuals · ${shots.length} shots</div>` +
section.variants.map((stack, v) =>
`<div class="pp-react-row${v === active ? ' current' : ''}">` +
`<span>${v === 0 ? '&#9679;' : '&#9675;'} ${stack[0].module.name}</span>` +
`<span>${v === 0 ? '&#9679;' : '&#9675;'} ${subjectOf(stack).module.name}</span>` +
`<span class="pp-feature">${shots.filter((s) => s.variant === v).length}&times;</span>` +
`</div>`).join('');
dom.panelBody.appendChild(list);
}
if (section.layers.length > 1) {
// The rest of the stack, named by the job each layer is doing rather
// than by its index: what is under the shot and what is over it are
// different questions, and the panel used to call both "layered over".
const ground = groundOf(section.layers);
const overlays = overlaysOf(section.layers);
if (ground || overlays.length) {
const row = (l, label) =>
`<div class="pp-react-row"><span>${l.module.name}</span>` +
`<span class="pp-feature">${label}</span>` +
`<span class="pp-amount">${(sceneCoverage(l.module) * 100).toFixed(0)}%</span></div>`;
const note = document.createElement('div');
note.className = 'pp-reactive';
note.innerHTML = `<div class="pp-sub">layered over (${section.layers.length - 1})</div>` +
section.layers.slice(1).map((l) =>
`<div class="pp-react-row"><span>${l.module.name}</span>` +
`<span class="pp-feature">${l.blend}</span>` +
`<span class="pp-amount">${l.opacity.toFixed(2)}</span></div>`).join('');
note.innerHTML =
`<div class="pp-sub">stack · ${(stackCoverage(section.layers) * 100).toFixed(0)}% painted</div>` +
(ground ? row(ground, 'ground') : '') +
row(subject, 'shot') +
overlays.map((l) => row(l, l.blend)).join('');
dom.panelBody.appendChild(note);
}
return;
@@ -326,7 +367,7 @@ function renderPanel() {
<div class="kv ${i === index ? 'current' : ''}">
<span>${s.kind}${s.locked ? ' &#128274;' : ''}
${s.shots ? `<i class="dim">${s.shots.length} shots</i>` : ''}</span>
<b>${(s.variants || [s.layers]).map((v) => v[0].module.name).join(' / ')}</b>
<b>${(s.variants || [s.layers]).map((v) => subjectOf(v).module.name).join(' / ')}</b>
</div>`).join('')}
<button id="btn-metronome" class="wide">download click track</button>
<div class="hint">Mixes clicks onto the detected beat grid. If they don't sit on
@@ -664,3 +705,9 @@ resize();
// "never going to start", and the failure it exists to catch is exactly the one
// that produces no error at all in the page.
window.__FLOW_STATE_READY__ = true;
const query = new URLSearchParams(location.search);
if (query.get('song')) {
const seed = query.get('seed');
loadBankSong(query.get('song'), seed === null ? null : (Number(seed) >>> 0));
}