Does sparse attention actually speed up video generation? — three conditions, measured

What we found

The three conditions

All three start from an identical first frame, with the same seed and the same prompt. Left to right: nothing removed, AI allocating per layer, and a uniform cut at the same average. The right two are matched on average compute, so the only difference is how it is distributed.

What to look forDoes each stride complete? Any skipped footfalls?

Play all threePauseRestartSpeed1.0x0.5x0.25x

① Nothing removed100%

baseline

② AI-allocatedkeep 6.755%

3%×67 / 5%×36 / 10%×97 (4 steps × 50 layers)

③ Uniform cutkeep 6.755%

6.755% on all 200 slots

What to look forDo the horses keep sweeping across while rising and falling? Any stalls or jumps?

Play all threePauseRestartSpeed1.0x0.5x0.25x

① Nothing removed100%

baseline

② AI-allocatedkeep 6.885%

3%×69 / 5%×28 / 10%×103 (4 steps × 50 layers)

③ Uniform cutkeep 6.885%

6.885% on all 200 slots

What to look forDo individual droplets survive? Does the foam flatten into sheets or vanish?

Play all threePauseRestartSpeed1.0x0.5x0.25x

① Nothing removed100%

baseline

② AI-allocatedkeep 6.88%

3%×67 / 5%×31 / 10%×102 (4 steps × 50 layers)

③ Uniform cutkeep 6.88%

6.88% on all 200 slots

What to look forDo individual flame tongues move separately, or only as one blob?

Play all threePauseRestartSpeed1.0x0.5x0.25x

① Nothing removed100%

baseline

② AI-allocatedkeep 6.795%

3%×73 / 5%×26 / 10%×101 (4 steps × 50 layers)

③ Uniform cutkeep 6.795%

6.795% on all 200 slots

What to look forDoes it stay the same person? Are blinks and mouth motion natural?

Play all threePauseRestartSpeed1.0x0.5x0.25x

① Nothing removed100%

baseline

② AI-allocatedkeep 6.685%

3%×74 / 5%×29 / 10%×97 (4 steps × 50 layers)

③ Uniform cutkeep 6.685%

6.685% on all 200 slots

What to look forDoes each person keep walking separately, or do they melt and swap?

Play all threePauseRestartSpeed1.0x0.5x0.25x

① Nothing removed100%

baseline

② AI-allocatedkeep 6.75%

3%×70 / 5%×32 / 10%×98 (4 steps × 50 layers)

③ Uniform cutkeep 6.75%

6.75% on all 200 slots

Audio is generated too, but these are muted by default since three play at once. Unmute one with its own volume control. We deliberately do not print a generation time on each clip. A single run of the same condition can vary by nearly 2× (see the measurement section), so a number next to each video would invite the wrong inference.

What is actually making it faster, and by how much

Every comparison up to here called "with the turbo LoRA already applied" the untouched baseline. So sparsification's 2× really meant 2× on top of something already sped up. We rebuilt the stack from nothing and measured what each piece contributes. Same subject, same seed, 1152×640, 124 frames. Sampling seconds are given alongside, to keep model loading out of the numbers.

Other things we tried and did not keep

These are not the only ways to go faster. Here is what else we tried, and why it is not in the stack.

Timing — where the seconds actually go

Our first conclusion was measured with the wrong ruler. Total generation time carries up to 11 seconds of variation unrelated to the condition (model reloading and so on) — the same size as the differences we were reasoning about. Isolating the sampling phase makes it clear.

What halving the keep rate buys you

Sampling moves −3 to −5% in all three. Total time scatters from +4.6% to −18.3%.

Every measurement

Allocation buys no speed

With average compute matched, we alternated the two allocations and measured repeatedly. Outliers near 2× show up — but they show up in both conditions alike. The keep rates actually applied are byte-identical on fast and slow runs, so this is the environment, not the method. By median: AI-allocated 40.22s / uniform 37.56s. Changing the allocation does not change generation time.

Cutting nudges it brighter

All three start from the same first frame, so how far each one drifts from it is directly comparable. We measured brightness, contrast and saturation at frame 0 and frame 123; the numbers are the change.

The AI looks like it is judging. It is not

The conclusion first. Ask an AI to decide where the compute should go and you get an answer that looks considered. But it was only tracing back the description we had written into the question ourselves. The AI never sees the thing it is budgeting for.

How we checked

Inside the model that makes the video there are 50 processing stages. We asked the AI, one stage at a time, how much compute that stage deserves. And in asking, we attach our own description — “this one is near the input”, “this one is near the output”.

What came back: less near the input, more near the output. Plausible enough.

So we swapped those descriptions and nothing else, then asked exactly the same thing again. The options, the stage numbers and every other word stayed identical. We simply told the input-side stages they were near the output, and the output-side stages they were near the input.

Take only the answers the AI gave confidently and you get 3 / 3 / 8.3 / 10 / 10 against 10 / 10 / 8.3 / 3 / 3. An exact mirror image.

So the answer was set by whoever wrote the question, not by the AI. It has no idea what those 50 stages actually do; all we can hand it is a position and a description we wrote ourselves.

It is not oblivious, though. Feed it a description that contradicts the stage number and its confidence drops noticeably (the share where it gave up and fell back to the default went from 17% to 43%). It still follows the description.

How this was measured

MiniMax H3 (1024×1792 and 1152×640, 124/192 frames, 4 steps, er_sde, a 4-step turbo LoRA) on an RTX 5080 16GB. Sparsification uses the SLA path already shipped in ComfyUI — the fraction of key blocks each query block keeps. No ComfyUI source was modified; we added one small node whose only job is to swap the ratio per step and per layer.

The three comparison clips are handed an identical first frame. Sparsification changes the computation itself, so the same seed no longer produces the same picture; without a shared starting point you are just looking at three different videos.

For what it is worth, a generative model returns the same output every time when the settings and the seed are all identical (we re-ran one condition unchanged and measured a pixel difference of 0.00). Change any one setting — sparsification, step count, anything — and what comes out is different. That is why the three clips are three different pictures.

The clips are not re-encoded. Re-compressing footage on a page about image quality would make it impossible to tell whether you are seeing sparsification or the codec. Only the container was rewritten (-c copy) so playback can start before the file finishes downloading.

Source data

Every number on this page comes from logs written automatically during generation. None were retyped by hand.

• Every run auto-saves the keep rates actually used (4 steps × 50 layers = 200 slots) and the sampling seconds as JSON
• Generation time comes from ComfyUI's own execution start/finish timestamps
• All six rounds of questions and answers to the AI (with confidence) are stored too
• The charts are built from those JSON files