dejvino/music-video-gen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Give the sections shapes, and the songs an opinion about their own edges

Browse Source 
Two complaints, one cause: stages were flat blocks and the noise bed was glued
to the pad level, so the bed was a constant hiss under a track that never went
anywhere.

Sections have contours now — rise, fall, swell, dip, surge — sampled per note
rather than per stage, so a build builds through its chords instead of stepping
between two flat halves. The noise gets its own per-stage level instead of
following the pad, which makes it an arrangement element: a riser through a
build with its filter sweeping up, a wash under a drop, nearly absent in a
breakdown. That is also the better test signal, since the segmenter classifies a
section partly on its energy slope and a build that does not build is one it has
to guess at.

Transition hardness is a per-song axis. Genres differ on this more than they
differ on tempo — an ambient record dissolves between its sections and a club
record cuts — and until now every song in the bank cut. Soft songs crossfade
across a couple of bars; the hardest get the pre-drop trick, where everything
stops for most of a beat before the loud stage lands. Five soft, three mid, nine
hard, and the builder fails if the bank ever loses either end.

The contours immediately broke the axis they were layered onto: section-to-
section contrast is dynamic range, so adding it put a floor of 0.50 under a
statistic that had reached 0.18. That is what compression IS, so contour depth
now scales with the track's dynamics — a limitered master has shallow section
contrast as well as a shallow crest. Back to 0.29, which is as low as enveloped
notes and hard cuts will go.

One regression accepted rather than fixed: the soft fades cost the beatless
entries their tempo detection, since the attack that gave a drone a pulse is
exactly what a crossfade removes. `drone` reads 170bpm for a 62bpm source. A
beatless track has no tempo and the detector is guessing either way; `ember`
exists to anchor the low end with a beat that is actually there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dejvino 2026-08-17 07:43:59 +02:00
parent d59dd9d5dd
commit e0c0dd664a
3 changed files with 185 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
flow-state/src/audio/songbank.js
View File

@ -35,43 +35,43 @@ import { synthesizeSong } from './synth.js';
*/ */
export const SONGS = [ export const SONGS = [
// --- slow, quiet, spacious ------------------------------------------- // --- slow, quiet, spacious -------------------------------------------
{ name: 'drone', bpm: 62, arrangement: 'ambient', brightness: 0.05, noise: 0.02, dynamics: 0.95, { name: 'drone', bpm: 62, arrangement: 'ambient', brightness: 0.05, noise: 0.02, dynamics: 0.95, transition: 0.05,
covers: 'beatless · darkest · most dynamic' }, covers: 'beatless · darkest · most dynamic' },
{ name: 'air', bpm: 76, arrangement: 'ambient', brightness: 0.90, noise: 0.25, noiseColour: 0.15, dynamics: 0.85, { name: 'air', bpm: 76, arrangement: 'ambient', brightness: 0.90, noise: 0.25, noiseColour: 0.15, dynamics: 0.85, transition: 0.1,
covers: 'beatless but bright — brightness without a beat' }, covers: 'beatless but bright — brightness without a beat' },
{ name: 'ember', bpm: 68, arrangement: 'club', brightness: 0.20, noise: 0.05, dynamics: 0.55, { name: 'ember', bpm: 68, arrangement: 'club', brightness: 0.20, noise: 0.05, dynamics: 0.55, transition: 0.8,
covers: 'slowest track with a detectable beat — anchors the low tempo end' }, covers: 'slowest track with a detectable beat — anchors the low tempo end' },
{ name: 'elegy', bpm: 84, arrangement: 'ballad', brightness: 0.10, noise: 0.03, dynamics: 0.90, { name: 'elegy', bpm: 84, arrangement: 'ballad', brightness: 0.10, noise: 0.03, dynamics: 0.90, transition: 0.12,
covers: 'slow, dark, one build and one payoff' }, covers: 'slow, dark, one build and one payoff' },
{ name: 'hymn', bpm: 92, arrangement: 'ballad', brightness: 0.80, noise: 0.20, dynamics: 0.70, { name: 'hymn', bpm: 92, arrangement: 'ballad', brightness: 0.80, noise: 0.20, dynamics: 0.70, transition: 0.25,
covers: 'slow and bright — separates tempo from brightness' }, covers: 'slow and bright — separates tempo from brightness' },
{ name: 'still', bpm: 100, arrangement: 'sparse', brightness: 0.35, noise: 0.06, dynamics: 0.75, { name: 'still', bpm: 100, arrangement: 'sparse', brightness: 0.35, noise: 0.06, dynamics: 0.75, transition: 0.2,
covers: 'the degenerate two-section case, which must not break' }, covers: 'the degenerate two-section case, which must not break' },
// --- mid tempo, the bulk of the space --------------------------------- // --- mid tempo, the bulk of the space ---------------------------------
{ name: 'dusk', bpm: 118, arrangement: 'classic', brightness: 0.02, noise: 0.01, dynamics: 0.60, { name: 'dusk', bpm: 118, arrangement: 'classic', brightness: 0.02, noise: 0.01, dynamics: 0.60, transition: 0.45,
covers: 'darkest and most tonal with a full arrangement' }, covers: 'darkest and most tonal with a full arrangement' },
{ name: 'centre', bpm: 124, arrangement: 'classic', brightness: 0.50, noise: 0.20, dynamics: 0.55, { name: 'centre', bpm: 124, arrangement: 'classic', brightness: 0.50, noise: 0.20, dynamics: 0.55, transition: 0.6,
covers: 'the middle of every axis — the null hypothesis' }, covers: 'the middle of every axis — the null hypothesis' },
{ name: 'glare', bpm: 128, arrangement: 'classic', brightness: 0.95, noise: 0.55, dynamics: 0.50, { name: 'glare', bpm: 128, arrangement: 'classic', brightness: 0.95, noise: 0.55, dynamics: 0.50, transition: 0.9,
covers: 'bright and noisy together' }, covers: 'bright and noisy together' },
{ name: 'slab', bpm: 126, arrangement: 'club', brightness: 0.45, noise: 0.25, dynamics: 0.05, { name: 'slab', bpm: 126, arrangement: 'club', brightness: 0.45, noise: 0.25, dynamics: 0.05, transition: 0.95,
covers: 'limitered — lowest dynamic range, longest sections' }, covers: 'limitered — lowest dynamic range, longest sections' },
{ name: 'chrome', bpm: 132, arrangement: 'club', brightness: 0.85, noise: 0.06, noiseColour: 0.1, dynamics: 0.35, { name: 'chrome', bpm: 132, arrangement: 'club', brightness: 0.85, noise: 0.06, noiseColour: 0.1, dynamics: 0.35, transition: 0.85,
covers: 'bright and tonal at club tempo' }, covers: 'bright and tonal at club tempo' },
{ name: 'murk', bpm: 88, arrangement: 'classic', brightness: 0.12, noise: 0.85, noiseColour: 0.95, dynamics: 0.70, { name: 'murk', bpm: 88, arrangement: 'classic', brightness: 0.12, noise: 0.85, noiseColour: 0.95, dynamics: 0.70, transition: 0.35,
covers: 'dark tonal content under bright hiss — the quadrant that breaks the brightness/noise correlation' }, covers: 'dark tonal content under bright hiss — the quadrant that breaks the brightness/noise correlation' },
// --- fast, dense, broken ---------------------------------------------- // --- fast, dense, broken ----------------------------------------------
{ name: 'lattice', bpm: 140, arrangement: 'breaks', brightness: 0.60, noise: 0.35, dynamics: 0.45, { name: 'lattice', bpm: 140, arrangement: 'breaks', brightness: 0.60, noise: 0.35, dynamics: 0.45, transition: 0.92,
covers: 'most sections — makes rosters actually rotate' }, covers: 'most sections — makes rosters actually rotate' },
{ name: 'grit', bpm: 150, arrangement: 'breaks', brightness: 0.15, noise: 0.90, noiseColour: 0.85, dynamics: 0.40, { name: 'grit', bpm: 150, arrangement: 'breaks', brightness: 0.15, noise: 0.90, noiseColour: 0.85, dynamics: 0.40, transition: 0.88,
covers: "noisiest and dark — the corrupt director's home ground" }, covers: "noisiest and dark — the corrupt director's home ground" },
{ name: 'plate', bpm: 138, arrangement: 'club', brightness: 0.25, noise: 0.70, noiseColour: 0.9, dynamics: 0.10, { name: 'plate', bpm: 138, arrangement: 'club', brightness: 0.25, noise: 0.70, noiseColour: 0.9, dynamics: 0.10, transition: 0.95,
covers: 'noisy AND compressed — two extremes at once' }, covers: 'noisy AND compressed — two extremes at once' },
{ name: 'runner', bpm: 174, arrangement: 'breaks', brightness: 0.70, noise: 0.40, dynamics: 0.50, { name: 'runner', bpm: 174, arrangement: 'breaks', brightness: 0.70, noise: 0.40, dynamics: 0.50, transition: 0.9,
covers: 'fastest with a broken arrangement' }, covers: 'fastest with a broken arrangement' },
{ name: 'flare', bpm: 168, arrangement: 'classic', brightness: 0.95, noise: 0.85, dynamics: 0.25, { name: 'flare', bpm: 168, arrangement: 'classic', brightness: 0.95, noise: 0.85, dynamics: 0.25, transition: 0.7,
covers: 'brightest and noisiest — the far corner' }, covers: 'brightest and noisiest — the far corner' },
]; ];
@ -97,6 +97,7 @@ export function song(name, { fps = 60 } = {}) {
noise: spec.noise, noise: spec.noise,
noiseColour: spec.noiseColour ?? null, noiseColour: spec.noiseColour ?? null,
dynamics: spec.dynamics, dynamics: spec.dynamics,
transition: spec.transition ?? 0.5,
seed: 1 + SONGS.indexOf(spec), seed: 1 + SONGS.indexOf(spec),
}), { fps }), }), { fps }),
}; };

204
flow-state/src/audio/synth.js
View File

@ -130,30 +130,63 @@ export function synthesizeSectioned({
* Brightness and noisiness have to be independently controllable or the bank * Brightness and noisiness have to be independently controllable or the bank
* cannot tell the two apart when it reports coverage. * cannot tell the two apart when it reports coverage.
*/ */
function addNoise(data, sampleRate, from, to, gain = 0.05, tilt = 0.5, seed = 12345) { function addNoise(data, sampleRate, from, to, gainAt, tiltAt, seed = 12345) {
const start = Math.max(0, Math.round(from * sampleRate)); const start = Math.max(0, Math.round(from * sampleRate));
const end = Math.min(data.length, Math.round(to * sampleRate)); const end = Math.min(data.length, Math.round(to * sampleRate));
if (end <= start) return;
let s0 = seed >>> 0; let s0 = seed >>> 0;
const rnd = () => { const rnd = () => {
s0 = (Math.imul(s0 ^ (s0 >>> 15), s0 | 1) + 0x6d2b79f5) >>> 0; s0 = (Math.imul(s0 ^ (s0 >>> 15), s0 | 1) + 0x6d2b79f5) >>> 0;
return ((s0 >>> 14) & 0xffff) / 0xffff - 0.5; return ((s0 >>> 14) & 0xffff) / 0xffff - 0.5;
}; };
// Two cascaded one-poles rather than one. A single pole is a 6dB/octave // Gain and tilt are functions of position through the stage, not constants.
// slope, which leaves so much top on "dark" noise that a noisy track always // Held constant, the bed was a continuous hiss over the entire track — the
// measured bright — the dark-and-noisy corner, which is the corrupt // one thing every listener noticed first. As a sweep it becomes a riser into
// director's entire home ground, was not reachable at all. // a drop and a wash that recedes, which is both easier to hear past and a
const a = 0.004 + tilt * 0.9; // better test signal: a rising filter is real spectral movement for the
// analyser to find.
let lp1 = 0, lp2 = 0; let lp1 = 0, lp2 = 0;
const span = end - start;
for (let s = start; s < end; s++) { for (let s = start; s < end; s++) {
const u = (s - start) / span;
const tilt = tiltAt(u);
// Two cascaded one-poles rather than one. A single pole is a 6dB/octave
// slope, which leaves so much top on "dark" noise that a noisy track
// always measured bright — the dark-and-noisy corner, which is the
// corrupt director's entire home ground, was not reachable at all.
const a = 0.004 + tilt * 0.9;
const white = rnd(); const white = rnd();
lp1 += a * (white - lp1); lp1 += a * (white - lp1);
lp2 += a * (lp1 - lp2); lp2 += a * (lp1 - lp2);
// Below half tilt the lowpass IS the signal; above it, what the lowpass // Below half tilt the lowpass IS the signal; above it, what the lowpass
// removed is. // removed is.
data[s] += (tilt > 0.5 ? white - lp2 : lp2 * 3) * gain * 2; data[s] += (tilt > 0.5 ? white - lp2 : lp2 * 3) * gainAt(u) * 2;
} }
} }
/**
* How a stage's energy moves across its own span.
*
* Stages were flat blocks: a drop was thirty seconds at one level, and the only
* shape available was a `ramp` flag on the build. That is both dull to listen to
* and a worse test signal than it looks, because the segmenter classifies a
* section partly on its energy SLOPE — a build that does not build is a build
* the classifier has to guess at.
*
* `u` runs 0..1 through the stage.
*/
const CONTOURS = {
flat: () => 1,
rise: (u) => 0.22 + 0.78 * u ** 1.5,
fall: (u) => 1 - 0.72 * u ** 1.2,
swell: (u) => 0.35 + 0.65 * Math.sin(Math.PI * u),
dip: (u) => 1 - 0.55 * Math.sin(Math.PI * u),
// Loud on the downbeat, easing off, then lifting back at the very end into
// whatever comes next.
surge: (u) => 0.72 + 0.28 * Math.cos(Math.PI * 2 * u),
};
const contourFor = (shape) => CONTOURS[shape] || CONTOURS.flat;
// --- notes ---------------------------------------------------------------- // --- notes ----------------------------------------------------------------
// The bank started out as sustained sine stacks at a fixed root, and a stage // The bank started out as sustained sine stacks at a fixed root, and a stage
// multiplier of 4 put that root above 1kHz with ten harmonics on top of it. // multiplier of 4 put that root above 1kHz with ten harmonics on top of it.
@ -236,9 +269,12 @@ function addNote(data, sampleRate, at, duration, freq, gain, harmonics = 3, atta
* measurements, an octave of headroom instead of a shriek. * measurements, an octave of headroom instead of a shriek.
*/ */
function addTonalBed(data, sampleRate, from, to, { function addTonalBed(data, sampleRate, from, to, {
gain, root, scale, progression, harmonics, beat, register, rng, sustain = 0, gainAt, root, scale, progression, harmonics, beat, register, rng, sustain = 0,
}) { }) {
const bar = beat * 4; const bar = beat * 4;
// The contour is sampled per note rather than per stage, so a rise actually
// rises through the chords instead of stepping between two flat halves.
const gainOf = (t) => gainAt((t - from) / Math.max(1e-6, to - from));
const degrees = SCALES[scale]; const degrees = SCALES[scale];
const chordAt = (index) => { const chordAt = (index) => {
const degree = progression[index % progression.length]; const degree = progression[index % progression.length];
@ -254,12 +290,14 @@ function addTonalBed(data, sampleRate, from, to, {
// Bass: the chord root, one per bar, low and simple. // Bass: the chord root, one per bar, low and simple.
for (let b = 0; b < 2 && t + b * bar < to; b++) { for (let b = 0; b < 2 && t + b * bar < to; b++) {
addNote(data, sampleRate, t + b * bar, Math.min(bar, to - t - b * bar), addNote(data, sampleRate, t + b * bar, Math.min(bar, to - t - b * bar),
semitone(root, chord[0]) / 2, gain * 1.1, Math.max(2, harmonics - 2), 0.02, sustain); semitone(root, chord[0]) / 2, gainOf(t + b * bar) * 1.1,
Math.max(2, harmonics - 2), 0.02, sustain);
} }
// Chord: held across the two bars, mid register. // Chord: held across the two bars, mid register.
for (const note of chord) { for (const note of chord) {
addNote(data, sampleRate, t, span, semitone(root, note), gain * 0.5, harmonics, 0.25, sustain); addNote(data, sampleRate, t, span, semitone(root, note),
gainOf(t) * 0.5, harmonics, 0.25, sustain);
} }
// Motif: only once the arrangement has opened up, so quiet stages stay // Motif: only once the arrangement has opened up, so quiet stages stay
@ -271,7 +309,8 @@ function addTonalBed(data, sampleRate, from, to, {
if (rng() < 0.3) continue; // rests, so it phrases if (rng() < 0.3) continue; // rests, so it phrases
const pick = degrees[Math.floor(rng() * degrees.length)]; const pick = degrees[Math.floor(rng() * degrees.length)];
addNote(data, sampleRate, t + n * step, step * 0.9, addNote(data, sampleRate, t + n * step, step * 0.9,
semitone(root, pick) * octave, gain * 0.30, Math.max(2, harmonics - 1), 0.008); semitone(root, pick) * octave, gainOf(t + n * step) * 0.30,
Math.max(2, harmonics - 1), 0.008);
} }
} }
} }
@ -285,59 +324,64 @@ function addTonalBed(data, sampleRate, from, to, {
* contains one arrangement cannot exercise the casting logic no matter how * contains one arrangement cannot exercise the casting logic no matter how
* widely it spreads tempo and brightness. Spans are fractions of the track. * widely it spreads tempo and brightness. Spans are fractions of the track.
* *
* Stage fields: span, kick, hat, noise, pad, root multiplier, ramp. * Stage fields: span, kick, hat, pad, noise, root (register), shape (contour).
*
* `noise` is per stage rather than following the pad level, which is what made
* the bed a constant hiss across the whole track. It is now an arrangement
* element: a riser through a build, a wash under a drop, nearly absent in a
* breakdown.
*/ */
export const ARRANGEMENTS = { export const ARRANGEMENTS = {
// Beatless. Segments into quiet kinds only — the case that must stay // Beatless. Segments into quiet kinds only — the case that must stay
// representable, since it is what an actual ambient track looks like. // representable, since it is what an actual ambient track looks like.
ambient: [ ambient: [
{ span: 0.30, kick: 0, hat: 0, pad: 0.10, root: 1.0 }, { span: 0.30, kick: 0, hat: 0, pad: 0.10, noise: 0.25, root: 1.0, shape: 'swell' },
{ span: 0.25, kick: 0, hat: 0, pad: 0.20, root: 1.5, ramp: true }, { span: 0.25, kick: 0, hat: 0, pad: 0.20, noise: 0.60, root: 1.5, shape: 'rise' },
{ span: 0.25, kick: 0, hat: 0, pad: 0.12, root: 1.0 }, { span: 0.25, kick: 0, hat: 0, pad: 0.12, noise: 0.35, root: 1.0, shape: 'fall' },
{ span: 0.20, kick: 0, hat: 0, pad: 0.07, root: 0.75 }, { span: 0.20, kick: 0, hat: 0, pad: 0.07, noise: 0.15, root: 0.75, shape: 'fall' },
], ],
// The standard shape: everything present, once. // The standard shape: everything present, once.
classic: [ classic: [
{ span: 0.12, kick: 0.00, hat: 0.00, pad: 0.09, root: 1.0 }, { span: 0.12, kick: 0.00, hat: 0.00, pad: 0.09, noise: 0.15, root: 1.0, shape: 'swell' },
{ span: 0.16, kick: 0.55, hat: 0.20, pad: 0.14, root: 1.5, ramp: true }, { span: 0.16, kick: 0.55, hat: 0.20, pad: 0.14, noise: 1.00, root: 1.5, shape: 'rise' },
{ span: 0.22, kick: 1.00, hat: 0.50, pad: 0.24, root: 4.0 }, { span: 0.22, kick: 1.00, hat: 0.50, pad: 0.24, noise: 0.35, root: 4.0, shape: 'surge' },
{ span: 0.14, kick: 0.05, hat: 0.02, pad: 0.07, root: 1.0 }, { span: 0.14, kick: 0.05, hat: 0.02, pad: 0.07, noise: 0.10, root: 1.0, shape: 'dip' },
{ span: 0.24, kick: 1.00, hat: 0.55, pad: 0.26, root: 4.0 }, { span: 0.24, kick: 1.00, hat: 0.55, pad: 0.26, noise: 0.40, root: 4.0, shape: 'surge' },
{ span: 0.12, kick: 0.25, hat: 0.08, pad: 0.08, root: 1.0 }, { span: 0.12, kick: 0.25, hat: 0.08, pad: 0.08, noise: 0.20, root: 1.0, shape: 'fall' },
], ],
// Long and level: a club tool that states its groove and stays there. The // Long and level: a club tool that states its groove and stays there. The
// case that produces `sustain`, which the classic shape barely reaches. // case that produces `sustain`, which the classic shape barely reaches.
club: [ club: [
{ span: 0.10, kick: 0.45, hat: 0.15, pad: 0.10, root: 1.0 }, { span: 0.10, kick: 0.45, hat: 0.15, pad: 0.10, noise: 0.30, root: 1.0, shape: 'rise' },
{ span: 0.30, kick: 0.90, hat: 0.45, pad: 0.20, root: 2.0 }, { span: 0.30, kick: 0.90, hat: 0.45, pad: 0.20, noise: 0.30, root: 2.0, shape: 'surge' },
{ span: 0.12, kick: 0.35, hat: 0.15, pad: 0.12, root: 1.0 }, { span: 0.12, kick: 0.35, hat: 0.15, pad: 0.12, noise: 0.20, root: 1.0, shape: 'dip' },
{ span: 0.34, kick: 1.00, hat: 0.55, pad: 0.22, root: 3.0 }, { span: 0.34, kick: 1.00, hat: 0.55, pad: 0.22, noise: 0.40, root: 3.0, shape: 'surge' },
{ span: 0.14, kick: 0.50, hat: 0.20, pad: 0.10, root: 1.0 }, { span: 0.14, kick: 0.50, hat: 0.20, pad: 0.10, noise: 0.20, root: 1.0, shape: 'fall' },
], ],
// Many short sections. Stresses the segmenter and produces the highest // Many short sections. Stresses the segmenter and produces the highest
// section counts, which is what makes rosters rotate. // section counts, which is what makes rosters rotate.
breaks: [ breaks: [
{ span: 0.10, kick: 0.30, hat: 0.10, pad: 0.08, root: 1.0 }, { span: 0.10, kick: 0.30, hat: 0.10, pad: 0.08, noise: 0.20, root: 1.0, shape: 'rise' },
{ span: 0.12, kick: 0.95, hat: 0.55, pad: 0.20, root: 3.0 }, { span: 0.12, kick: 0.95, hat: 0.55, pad: 0.20, noise: 0.35, root: 3.0, shape: 'surge' },
{ span: 0.10, kick: 0.05, hat: 0.02, pad: 0.06, root: 1.0 }, { span: 0.10, kick: 0.05, hat: 0.02, pad: 0.06, noise: 0.10, root: 1.0, shape: 'dip' },
{ span: 0.14, kick: 1.00, hat: 0.60, pad: 0.24, root: 4.0 }, { span: 0.14, kick: 1.00, hat: 0.60, pad: 0.24, noise: 0.40, root: 4.0, shape: 'surge' },
{ span: 0.10, kick: 0.10, hat: 0.03, pad: 0.07, root: 0.75 }, { span: 0.10, kick: 0.10, hat: 0.03, pad: 0.07, noise: 0.15, root: 0.75, shape: 'fall' },
{ span: 0.16, kick: 0.60, hat: 0.30, pad: 0.16, root: 2.0, ramp: true }, { span: 0.16, kick: 0.60, hat: 0.30, pad: 0.16, noise: 1.00, root: 2.0, shape: 'rise' },
{ span: 0.16, kick: 1.00, hat: 0.65, pad: 0.26, root: 4.0 }, { span: 0.16, kick: 1.00, hat: 0.65, pad: 0.26, noise: 0.40, root: 4.0, shape: 'surge' },
{ span: 0.12, kick: 0.20, hat: 0.05, pad: 0.08, root: 1.0 }, { span: 0.12, kick: 0.20, hat: 0.05, pad: 0.08, noise: 0.15, root: 1.0, shape: 'fall' },
], ],
// One long rise into one payoff, then gone. Slow material's shape. // One long rise into one payoff, then gone. Slow material's shape.
ballad: [ ballad: [
{ span: 0.28, kick: 0.00, hat: 0.00, pad: 0.10, root: 1.0 }, { span: 0.28, kick: 0.00, hat: 0.00, pad: 0.10, noise: 0.20, root: 1.0, shape: 'swell' },
{ span: 0.30, kick: 0.40, hat: 0.10, pad: 0.18, root: 1.5, ramp: true }, { span: 0.30, kick: 0.40, hat: 0.10, pad: 0.18, noise: 0.85, root: 1.5, shape: 'rise' },
{ span: 0.24, kick: 0.85, hat: 0.35, pad: 0.26, root: 3.0 }, { span: 0.24, kick: 0.85, hat: 0.35, pad: 0.26, noise: 0.35, root: 3.0, shape: 'surge' },
{ span: 0.18, kick: 0.15, hat: 0.03, pad: 0.09, root: 1.0 }, { span: 0.18, kick: 0.15, hat: 0.03, pad: 0.09, noise: 0.15, root: 1.0, shape: 'fall' },
], ],
// Two sections, both quiet. Kept because it is the degenerate case the old // Two sections, both quiet. Kept because it is the degenerate case the old
// bank produced by accident, and the generator must not break on it. // bank produced by accident, and the generator must not break on it.
sparse: [ sparse: [
{ span: 0.5, kick: 0.10, hat: 0.02, pad: 0.08, root: 1.0 }, { span: 0.5, kick: 0.10, hat: 0.02, pad: 0.08, noise: 0.20, root: 1.0, shape: 'swell' },
{ span: 0.5, kick: 0.35, hat: 0.15, pad: 0.16, root: 2.0 }, { span: 0.5, kick: 0.35, hat: 0.15, pad: 0.16, noise: 0.30, root: 2.0, shape: 'rise' },
], ],
}; };
@ -363,6 +407,7 @@ export function synthesizeSong({
noise = 0.15, noise = 0.15,
noiseColour = null, noiseColour = null,
dynamics = 0.6, dynamics = 0.6,
transition = 0.5,
seed = 1, seed = 1,
} = {}) { } = {}) {
const stages = ARRANGEMENTS[arrangement] || ARRANGEMENTS.classic; const stages = ARRANGEMENTS[arrangement] || ARRANGEMENTS.classic;
@ -423,41 +468,98 @@ export function synthesizeSong({
// quadrant was unreachable. Hiss over a sub-bass pad is an ordinary record. // quadrant was unreachable. Hiss over a sub-bass pad is an ordinary record.
const tilt = 0.05 + (noiseColour === null ? brightness : noiseColour) * 0.9; const tilt = 0.05 + (noiseColour === null ? brightness : noiseColour) * 0.9;
// How abruptly one stage becomes the next. Genres differ on this more than
// they differ on tempo: an ambient record dissolves between its sections and
// a club record cuts, and until now every song in the bank cut.
//
// It is not only a listening difference. The segmenter finds boundaries by
// looking for change, so a soft track genuinely has fewer of them — section
// count is the axis the bank covered worst, and this is the knob that moves
// it for a musical reason rather than by padding the stage list.
const bar = beat * 4;
const fade = (1 - transition) ** 1.5 * bar * 2.5;
// The hardest transitions get the pre-drop trick: everything stops for a
// moment before the loud stage lands. Nothing states "this is a cut" more
// plainly, and it is what an EDM master actually does.
const gapBeats = transition > 0.7 ? (transition - 0.7) / 0.3 * 0.9 : 0;
let at = 0; let at = 0;
let stageIndex = 0; let stageIndex = 0;
const bounds = [];
{
let cursor = 0;
for (const stage of stages) { for (const stage of stages) {
const from = at; const from = cursor;
const to = Math.min(duration, at + stage.span * duration); cursor = Math.min(duration, cursor + stage.span * duration);
bounds.push([from, cursor]);
}
}
for (const stage of stages) {
const [from, to] = bounds[stageIndex];
const next = stages[stageIndex + 1];
at = to; at = to;
stageIndex++; stageIndex++;
const span = Math.max(1e-6, to - from);
const contour = contourFor(stage.shape);
// Compression flattens the contrast BETWEEN sections, not just within
// them, so a limitered track gets shallower contours. Without this the
// contours alone put a floor of 0.50 under the measured dynamic range
// and the compressed end of the bank stopped existing — the section
// shapes were quietly undoing the axis they were layered onto.
const depth = 0.12 + dynamics * 0.88;
const fadeU = Math.min(0.45, fade / span);
// A hard cut into a LOUDER stage gets the gap; a fall into a quiet one
// does not, because silence before a breakdown is just silence.
const gapU = next && (next.kick + next.pad) > (stage.kick + stage.pad) * 1.3
? Math.min(0.2, (gapBeats * beat) / span) : 0;
/** The stage's own shape, with its edges softened or cut. */
const shapeAt = (u) => {
let v = 1 + (contour(Math.max(0, Math.min(1, u))) - 1) * depth;
if (fadeU > 1e-4) {
if (u < fadeU) v *= 0.5 - 0.5 * Math.cos(Math.PI * (u / fadeU));
if (u > 1 - fadeU) v *= 0.5 - 0.5 * Math.cos(Math.PI * ((1 - u) / fadeU));
}
if (gapU > 1e-4 && u > 1 - gapU) v *= 0.04;
return v;
};
let index = Math.round(from / beat); let index = Math.round(from / beat);
for (let t = from; t < to; t += beat, index++) { for (let t = from; t < to; t += beat, index++) {
const ramp = stage.ramp ? (t - from) / Math.max(1e-6, to - from) : 1; const u = (t - from) / span;
const gain = level(stage.kick, 'kick') * (0.25 + ramp * 0.75); const shaped = shapeAt(u);
const gain = level(stage.kick, 'kick') * shaped;
if (stage.kick > 0.02 && gain > 0.005) { if (stage.kick > 0.02 && gain > 0.005) {
addKick(left, sampleRate, t, gain * (index % 4 === 0 ? 1 : 0.8)); addKick(left, sampleRate, t, gain * (index % 4 === 0 ? 1 : 0.8));
} }
if (stage.hat > 0.02 && hatPresence > 0) { if (stage.hat > 0.02 && hatPresence > 0) {
const h = level(stage.hat, 'hat') * ramp * hatPresence * 1.4; const h = level(stage.hat, 'hat') * shaped * hatPresence * 1.4;
addHat(left, sampleRate, t + beat / 2, h, index + seed); addHat(left, sampleRate, t + beat / 2, h, index + seed);
if (h > 0.3) addHat(left, sampleRate, t + beat / 4, h * 0.6, index + seed + 7); if (h > 0.3) addHat(left, sampleRate, t + beat / 4, h * 0.6, index + seed + 7);
} }
} }
const padLevel = level(stage.pad, 'pad') * sustainBoost;
addTonalBed(left, sampleRate, from, to, { addTonalBed(left, sampleRate, from, to, {
gain: level(stage.pad, 'pad') * sustainBoost, gainAt: (u) => padLevel * shapeAt(u),
root, scale, progression, harmonics, beat, root, scale, progression, harmonics, beat,
register: stage.root, register: stage.root,
sustain: 1 - dynamics, sustain: 1 - dynamics,
rng, rng,
}); });
if (noise > 0.01) { if (noise > 0.01) {
// The bed follows the arrangement so it cannot flatten the dynamics const bed = noise * (stage.noise ?? 0.3) * padLevel * 6;
// it is layered over. // A riser sweeps its filter up as it goes; everything else holds its
// colour. This is the difference between a bed that builds tension
// and one that is just always there.
const rising = stage.shape === 'rise';
addNoise(left, sampleRate, from, to, addNoise(left, sampleRate, from, to,
noise * level(stage.pad, 'pad') * 3 * sustainBoost, (u) => bed * shapeAt(u),
tilt, (seed * 7919 + stageIndex) >>> 0); rising ? (u) => Math.min(0.98, tilt + u * 0.5) : () => tilt,
(seed * 7919 + stageIndex) >>> 0);
} }
} }

16
flow-state/tools/build-song-bank.js
View File

@ -89,6 +89,7 @@ for (const spec of SONGS) {
noise: spec.noise, noise: spec.noise,
noiseColour: spec.noiseColour ?? null, noiseColour: spec.noiseColour ?? null,
dynamics: spec.dynamics, dynamics: spec.dynamics,
transition: spec.transition ?? 0.5,
seed: 1 + SONGS.indexOf(spec), seed: 1 + SONGS.indexOf(spec),
}); });
const track = FeatureTrack.fromAudioBuffer(buffer, { fps: 60 }); const track = FeatureTrack.fromAudioBuffer(buffer, { fps: 60 });
@ -118,12 +119,13 @@ for (const spec of SONGS) {
const n2 = (x) => (typeof x === 'number' ? x.toFixed(2) : String(x)).padStart(7); const n2 = (x) => (typeof x === 'number' ? x.toFixed(2) : String(x)).padStart(7);
console.log('\n MEASURED — what the generator will actually read\n'); console.log('\n MEASURED — what the generator will actually read\n');
console.log(' song tempo bright noisy dynamic loud sections arrangement'); console.log(' song tempo bright noisy dynamic loud sections arrangement cuts');
console.log(' ' + '-'.repeat(86)); console.log(' ' + '-'.repeat(86));
for (const b of built) { for (const b of built) {
console.log(` ${b.name.padEnd(11)}${n2(b.measured.bpm)}${n2(b.measured.centroid)}` + console.log(` ${b.name.padEnd(11)}${n2(b.measured.bpm)}${n2(b.measured.centroid)}` +
`${n2(b.measured.flatness)}${n2(b.measured.dynamics)}${n2(b.measured.loudness)}` + `${n2(b.measured.flatness)}${n2(b.measured.dynamics)}${n2(b.measured.loudness)}` +
`${String(b.measured.sections).padStart(10)} ${b.arrangement}`); `${String(b.measured.sections).padStart(10)} ${b.arrangement.padEnd(9)}` +
` ${(b.transition ?? 0.5) >= 0.7 ? 'hard' : (b.transition ?? 0.5) <= 0.3 ? 'soft' : 'mid '}`);
} }
// --- coverage -------------------------------------------------------------- // --- coverage --------------------------------------------------------------
@ -197,6 +199,16 @@ for (let i = 0; i < AXES.length; i++) {
} }
} }
// Transition hardness is a spec input rather than a summary statistic, so it is
// reported rather than binned: how abruptly one section becomes the next is a
// genre difference the bank has to contain — an ambient record dissolves where a
// club record cuts — and a bank where every song cut would be testing the
// generator against one kind of music.
const soft = built.filter((b) => (b.transition ?? 0.5) <= 0.3).length;
const hard = built.filter((b) => (b.transition ?? 0.5) >= 0.7).length;
console.log(`\n transitions ${soft} soft · ${built.length - soft - hard} mid · ${hard} hard`);
if (!soft || !hard) problems.push('the bank has no ' + (soft ? 'hard' : 'soft') + ' transitions');
// Section kinds are the casting gate, so coverage of them is not optional: a // Section kinds are the casting gate, so coverage of them is not optional: a
// bank with no drop cannot reach the geometric, glitch or structural families // bank with no drop cannot reach the geometric, glitch or structural families
// no matter how widely its tempo spreads. // no matter how widely its tempo spreads.