dejvino/music-video-gen
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Make grain a treatment, tempo a governor, and params commit harder

Browse Source 
Grain was in every video. It was added twice unconditionally — every scene
called sigGrain, and the grade added its own on top — so the only thing that
varied between two tracks was how much of it there was. That makes grain the
renderer's fingerprint rather than a decision about one video.

It is now described rather than dialled (look/grain.js): a mode (off /
constant / swell / sections / transient), a cell size in pixels, a refresh
rate in frames, a mask (uniform, shadows, highlights, edges, bands) and a
chroma amount. Roughly 45% of tracks get none at all. The non-constant modes
carry a per-frame envelope computed in Show._postAt from frame and features
only, so preview and export still agree. Scene-side grain is gated the same
way, and a module can decline it outright with `texture: 0` — crisp line work
should stay crisp. The post tab grew a real grain block so any of it can be
forced per track.

Slow songs got fast scenes. `motion` bias was mostly section energy with
tempo as a small correction, so a 70bpm track's drop asked for nearly as much
speed as a 150bpm one. Motion is now tempo-dominated, and every `rate: true`
param is additionally scaled by a per-track rateScale — measured, 84bpm now
samples its rate params at 0.276 of range against 148bpm's 0.571.

Parameter sampling also commits harder: extremity starts at 0.45 rather than
0.25 and shapes the draw more aggressively. This was first pushed to 0.82 and
backed off to 0.72, because the gates caught the overshoot — seeds began
collapsing onto the same range ends and a sparse scene sampled at its low end
rendered effectively black.

Fallout worth recording: turning the default grade grain off exposed two
scenes that were never really animating. Dust Chamber and Eclipse Field
passed the Phase 7 movement gate only because per-pixel noise was moving
underneath them; both now breathe on their own fixed clock, and Dust Chamber
needed a brightness floor as well. Four scenes (Classic Wave, Silk Ribbon,
Kaleido Tunnel, Slow Orb) express the style trait ONLY through grain and so
cannot opt out yet; they hold a reduced share at 0.35 pending real edge and
softness response.

Phase 3's look-space check now measures its closest pair relative to image
brightness, the same correction Phase 10 already documents for sparse scenes:
the absolute number was being propped up by grain rather than by look-space
width. Five new Phase 10 checks cover grain distribution, treatment variety,
envelope range, the texture opt-out, and tempo. 93/93 pass.

Two transport bugs, both stale state surfacing in the UI:

- Loading a track replaces audio.src, which stops playback silently, so
  state.playing stayed true and the play button stayed on pause — the first
  click after a track change only flipped the flag back. Added stopPlayback().
- The controls row wrapped mid-song because two readouts that change length
  while playing were sized by their content: the clock crossing ten minutes
  and the section label whenever a scene name is long or a crossfade appears.
  The clock is now fixed-width with tabular figures and the section label is
  the row's only flexible item, laying out at zero width and ellipsising into
  whatever space is left. The two spacers competed with it for that space and
  are gone; its own text-align does their job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dejvino
2026-08-06 07:04:27 +02:00
co-authored by Claude Opus 5
parent 44e1826fcb
commit 14d1204e82
34 changed files with 756 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
flow-state/src/main.js
+115 -3
View File
@@ -4,6 +4,7 @@ import { ParamPanel } from './ui/ParamPanel.js';
import { formatTime } from './audio/decode.js';
import { describeLook } from './look/LookGenerator.js';
import { toHex } from './look/palette.js';
import { applyGrainToPost, describeGrain, GRAIN_MASKS, GRAIN_MODES } from './look/grain.js';
import { renderClickTrack, audioBufferToWavBlob } from './audio/clicktrack.js';
import { Exporter, exportSegment, downloadBlob, PRESETS, isSupported } from './export/Exporter.js';
@@ -57,6 +58,7 @@ const paramPanel = new ParamPanel(document.createElement('div'), onParamChange);
async function loadFile(file) {
if (state.busy) return;
state.busy = true;
stopPlayback();
if (dom.overlay) {
dom.overlay.hidden = true;
@@ -128,6 +130,21 @@ function seekTo(frame, options = {}) {
strip.setFrame(clamped);
}
/**
* Force the transport to a stopped state.
*
* Loading a track replaces `audio.src`, which stops playback without telling
* anyone — so `state.playing` stayed true, the button stayed on ❚❚, and the
* first click after a track change only toggled the flag back rather than
* starting anything. Every path that stops playback behind the UI's back has
* to come through here.
*/
function stopPlayback() {
state.playing = false;
dom.audio.pause();
dom.play.textContent = '▶';
}
function togglePlay() {
if (!state.show.ready) return;
state.playing = !state.playing;
@@ -321,7 +338,11 @@ function renderPanel() {
const post = show.look.post;
const fb = show.look.feedback;
const wide = new Set(['contrast', 'saturation', 'exposure']);
const rows = Object.entries(post).map(([key, value]) => `
// Grain has its own block below: its fields are a mode, a mask id and a
// pixel size, none of which are a 0..1 slider.
const rows = Object.entries(post)
.filter(([key]) => !key.startsWith('grain'))
.map(([key, value]) => `
<div class="pp-row">
<label class="pp-label">${key}</label>
<input class="pp-input" type="range" min="0" max="${wide.has(key) ? 2 : 1}"
@@ -338,7 +359,9 @@ function renderPanel() {
<span class="pp-value">${(+value).toFixed(3)}</span>
</div>`).join('');
dom.panelBody.innerHTML =
`<div class="pp-sub">post</div>${rows}<div class="pp-sub">feedback</div>${fbRows}`;
`<div class="pp-sub">post</div>${rows}` +
`<div class="pp-sub">grain</div>${grainRows(show.look.grain)}` +
`<div class="pp-sub">feedback</div>${fbRows}`;
dom.panelBody.oninput = (e) => {
const t = e.target;
if (t.dataset.post) {
@@ -349,7 +372,32 @@ function renderPanel() {
fb[t.dataset.feedback] = +t.value;
t.nextElementSibling.textContent = (+t.value).toFixed(3);
}
if (t.dataset.grain) {
const key = t.dataset.grain;
if (key === 'kinds') {
const kinds = new Set(show.look.grain.kinds);
if (t.checked) kinds.add(t.value); else kinds.delete(t.value);
show.look.grain.kinds = [...kinds];
} else {
show.look.grain[key] = t.tagName === 'SELECT' ? t.value : +t.value;
}
// Turning grain on for a track that was generated without it
// would otherwise select a mode and still show nothing.
if (show.look.grain.mode !== 'off' && show.look.grain.amount <= 0) {
show.look.grain.amount = 0.04;
}
applyGrainToPost(show.look.grain, post);
// The mode decides which of the other controls exist, so it is
// the one edit that has to rebuild the panel.
if (key === 'mode') { renderPanel(); return; }
if (t.nextElementSibling) {
t.nextElementSibling.textContent = (+t.value).toFixed(3);
}
const desc = dom.panelBody.querySelector('#grain-desc');
if (desc) desc.textContent = describeGrain(show.look.grain);
}
};
dom.panelBody.onchange = dom.panelBody.oninput;
return;
}
@@ -371,6 +419,70 @@ function renderPanel() {
}
}
/**
* Grain controls for the post tab.
*
* Grain is the one part of the grade with a shape rather than a level — when it
* is present, how coarse it is, how often it refreshes and where it lands — so
* it gets its own block instead of five sliders that all read 0..1.
*/
function grainRows(grain) {
const slider = (key, min, max, step, value) => `
<div class="pp-row">
<label class="pp-label">${key}</label>
<input class="pp-input" type="range" min="${min}" max="${max}" step="${step}"
value="${value}" data-grain="${key}">
<span class="pp-value">${(+value).toFixed(3)}</span>
</div>`;
const modeRow = `
<div class="pp-row">
<label class="pp-label">mode</label>
<select class="pp-input" data-grain="mode">
${GRAIN_MODES.map((m) =>
`<option ${m === grain.mode ? 'selected' : ''}>${m}</option>`).join('')}
</select>
<span class="pp-value"></span>
</div>`;
if (grain.mode === 'off') {
return modeRow + `<div class="hint" id="grain-desc">${describeGrain(grain)}</div>`;
}
const maskRow = `
<div class="pp-row">
<label class="pp-label">mask</label>
<select class="pp-input" data-grain="mask">
${Object.keys(GRAIN_MASKS).map((m) =>
`<option ${m === grain.mask ? 'selected' : ''}>${m}</option>`).join('')}
</select>
<span class="pp-value"></span>
</div>`;
const kindsRow = grain.mode !== 'sections' ? '' : `
<div class="pp-row">
<label class="pp-label">sections</label>
<span class="pp-input">
${['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro'].map((k) => `
<label class="dim"><input type="checkbox" data-grain="kinds" value="${k}"
${grain.kinds.includes(k) ? 'checked' : ''}>${k}</label>`).join(' ')}
</span>
</div>`;
const swellRows = grain.mode !== 'swell' ? ''
: slider('period', 4, 60, 1, grain.period) + slider('duty', 0.05, 1, 0.01, grain.duty);
return modeRow
+ slider('amount', 0, 0.25, 0.002, grain.amount)
+ slider('scale', 1, 8, 0.5, grain.scale)
+ slider('rate', 1, 8, 1, grain.rate)
+ maskRow
+ slider('chroma', 0, 1, 0.01, grain.chroma)
+ kindsRow
+ swellRows
+ `<div class="hint" id="grain-desc">${describeGrain(grain)}</div>`;
}
// ---------------------------------------------------------------- export
function currentPreset() {
@@ -468,7 +580,7 @@ function frame(now) {
}
}
show.timeline.syncToAudio(dom.audio.currentTime);
if (dom.audio.ended) { state.playing = false; dom.play.textContent = '▶'; }
if (dom.audio.ended) stopPlayback();
}
if (state.playing || show.timeline.frame !== lastRenderedFrame) {