dejvino/music-video-gen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

A region block: how differently the parts of a frame behave from each other

Browse Source 
The descriptor could say where a frame's energy sat and not whether one part of
the picture was doing something different from another — and that second thing
is what "nothing to watch" means. A pattern spread evenly over the screen has no
subject precisely because every region is the same region, and `layout`
normalises that away: a uniform field and a field with a form standing in it can
come out at nearly the same distribution.

So each ninth of the frame is now characterised in its own right — how much
detail, which way it runs, how many elements — and expressed as its deviation
from the frame's average. Heterogeneity rather than level, so a brighter or
busier frame does not register as a more varied one.

It is immediately the strongest block in the descriptor, and it credits scenes
the old one was underrating:

    Apollonian Gasket  0.039 -> 0.062   region 0.180
    Plasma Bloom       0.053 -> 0.082   region 0.182
    Moiré Grid         0.037 -> 0.047   region 0.097
    Droste Feedback    0.219 -> 0.324   region 0.446

Apollonian Gasket is the interesting one. It was called out as looking good and
scoring badly, and the answer turns out to be partly that the instrument was
missing the axis it is good on rather than that beauty and variety are simply
different things. Both were true; only one of them was the instrument's fault.

The metric's own gates still hold, which is the condition for believing any of
this: recolour moves structure 0.019 while moving colour 0.723, a quarter turn
moves it 0.0000, and the same scene against itself is 0.0000 against 0.0992 for
two different scenes.

EVERY SCORE HAS MOVED. The total is a mean over six blocks now rather than five,
and the 0.04 bar was derived under the old one. It needs re-deriving from a fresh
gallery before it is used to judge anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dejvino 2026-08-18 18:07:11 +02:00
parent 8defef3ed0
commit b879abe230
3 changed files with 86 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
flow-state/HOWTO-variety.md
View File

@ -37,11 +37,11 @@ the song.
Some reference points, measured:
```
0.219 Droste Feedback varies a lot
0.181 Spectrum Sculpture
0.049 Isometric Blocks after work — see below
0.039 Apollonian Gasket beautiful, and the same six times
0.031 Moiré Grid the same picture in six palettes
0.324 Droste Feedback varies a lot
0.082 Plasma Bloom
0.062 Voronoi Shatter after a focus
0.062 Apollonian Gasket
0.047 Moiré Grid after a subject; 0.031 before
```
---
@ -70,8 +70,20 @@ Design against this, because half of "why is my score low" is here.
| `orient` | grid vs radial vs stripes, rotation-blind | break or vary a regular lattice |
| `layout` | where in the frame structure sits | move the subject, change what is empty |
| `texture` | element count, sparsity, mirror and radial symmetry | vary how many things there are |
| `region` | how differently the parts of the frame behave from each other | give the frame a subject, so one region is unlike its neighbours |
| `motion` | what moves and where, not how much | vary the KIND of movement, not the rate |
**`region` is the block that rewards having something to look at.** Layout says
where the energy is, normalised, so a uniform field and a field with a subject in
it can normalise to nearly the same answer. Region characterises each part of the
frame in its own right — detail, direction, element count — and reports how far
each deviates from the frame's average. A pattern spread evenly deviates by
nothing everywhere, which is exactly why there is nothing to watch.
Measured when it was added: Apollonian Gasket 0.039 to 0.062 and Plasma Bloom to
0.082, both on region alone, because both have a subject the old blocks were not
crediting. It is the strongest single block in the descriptor.
**A known blind spot.** `layout` is nearly useless for full-frame fields. Voronoi
Shatter measures 0.004 there no matter what changes, because edge-to-edge cells
occupy the same frame however they fall. That is honest — there genuinely is no

60
flow-state/src/checks/variety/descriptors.js
View File

@ -279,6 +279,65 @@ function colourBlock(pixels) {
return [...Array.from(hist, (x) => x / total), sat / n, val / n];
}
/**
* Region block: how differently the parts of the frame behave from each other.
*
* The gap `layout` leaves. Layout says WHERE the energy is, normalised, so a
* uniform field and a field with a subject in it can normalise to nearly the
* same answer — measured, every full-frame scene sat at 0.004 there no matter
* what was done to it. What distinguishes them is not the distribution of
* energy but whether one part of the picture is doing something different from
* another. A pattern spread evenly has nothing to look at precisely because
* every region is the same region.
*
* So each region is characterised in its own right — how much detail, how
* anisotropic, how many elements — and then expressed as its DEVIATION from the
* frame's average. A uniform field deviates by nothing everywhere; a subject
* shows up as a region behaving unlike its neighbours, and which region and how
* is something the song can change.
*/
function regionBlock(band, cells = 3) {
const { data, w, h } = band;
const rw = Math.floor(w / cells), rh = Math.floor(h / cells);
const rows = [];
for (let gy = 0; gy < cells; gy++) {
for (let gx = 0; gx < cells; gx++) {
let energy = 0, gxx = 0, gyy = 0, cross = 0, n = 0;
for (let y = gy * rh + 1; y < (gy + 1) * rh - 1; y++) {
for (let x = gx * rw + 1; x < (gx + 1) * rw - 1; x++) {
const i = y * w + x;
const v = data[i];
energy += v * v;
const dx = data[i + 1] - data[i - 1];
const dy = data[i + w] - data[i - w];
gxx += dx * dx;
gyy += dy * dy;
if ((v > 0) !== (data[i - 1] > 0)) cross++;
n++;
}
}
n = n || 1;
rows.push([
Math.sqrt(energy / n), // how much detail
(gxx - gyy) / (gxx + gyy + 1e-6), // which way it runs
cross / n, // how many elements
]);
}
}
// Deviation from the frame's own average, so this measures HETEROGENEITY
// rather than overall level — a brighter or busier frame does not register
// as a more varied one.
const mean = [0, 1, 2].map((k) => rows.reduce((a, r) => a + r[k], 0) / rows.length);
const scale = [0, 1, 2].map((k) => Math.max(1e-4, Math.abs(mean[k])) );
return rows.flatMap((r) => [
(r[0] - mean[0]) / scale[0],
r[1] - mean[1],
(r[2] - mean[2]) / scale[2],
]);
}
/**
* Full descriptor for one frame.
*
@ -296,6 +355,7 @@ export function frameDescriptor(pixels, width, height) {
scale: scaleBlock(bands),
orient: orientBlock(band),
layout: layoutBlock(band),
region: regionBlock(band),
texture: textureBlock(band),
colour: colourBlock(pixels),
};

11
flow-state/src/checks/variety/signature.js
View File

@ -15,7 +15,7 @@
import { frameDescriptor, motionDescriptor } from './descriptors.js';
/** Blocks that count toward the structural score. Colour is measured, not counted. */
export const STRUCTURAL = ['scale', 'orient', 'layout', 'texture', 'motion'];
export const STRUCTURAL = ['scale', 'orient', 'layout', 'region', 'texture', 'motion'];
export const ALL_BLOCKS = [...STRUCTURAL, 'colour'];
/**
@ -26,7 +26,7 @@ export const ALL_BLOCKS = [...STRUCTURAL, 'colour'];
* block anyway — so the diagnosis survives even if the single number is
* weighted wrong.
*/
const WEIGHTS = { scale: 1, orient: 1, layout: 1, texture: 1, motion: 1 };
const WEIGHTS = { scale: 1, orient: 1, layout: 1, region: 1, texture: 1, motion: 1 };
/**
* Chi-square distance, 0..1, for the blocks that are normalised histograms.
@ -61,8 +61,15 @@ function scaledL1(a, b, spans) {
return d / a.length;
}
// Region entries are signed deviations from a frame mean, in three repeating
// kinds, so they need spans rather than a histogram distance.
const REGION_SPAN = [1.2, 1.0, 1.2];
function blockDistance(name, a, b) {
if (name === 'texture') return scaledL1(a, b, TEXTURE_SPAN);
if (name === 'region') {
return scaledL1(a, b, a.map((_, i) => REGION_SPAN[i % 3]));
}
if (name === 'colour') {
// Hue distribution, then saturation and brightness, weighted so a hue
// rotation reads as the large colour change it is.