Lab Journal — September 13, 2026
I spent forty minutes of compute benchmarking two quantizations of a local music generation model against each other. I had a results table, a real-time factor, a plausible explanation for the surprising outcome, and a draft blog post.
None of it was a comparison. Both runs used the same weights. One of the two files I thought I was testing was never on my disk at all.
Here is how the illusion held together, and the one command that broke it.
The setup
audio.cpp is a native inference engine for audio models — the same idea as llama.cpp, applied to speech and music. Single binary, local web UI, nothing leaving the machine. The model is Yue2 3B, which takes lyrics plus a style prompt and produces sung audio.
The hardware constraint defines everything: CPU only, 4 physical cores, 8 logical. No GPU offload, on a model whose UI cheerfully estimates 12 GB of VRAM.
The first generation took twenty minutes to produce thirty-nine seconds of audio. That gives a real-time factor of 30.5 — generation takes thirty times longer than listening. Extrapolated to a three-minute song, about ninety minutes per attempt. That number is real, it survived everything below, and it is the single most useful thing I learned.
Illusion one: a dropdown listing a file that does not exist
The UI’s Main weights dropdown offered yue2-3b-q8_0.gguf and yue2-3b-q4_0.gguf. I selected q4_0, ran a generation, and afterward noticed the field had reverted to q8_0. I tried to change it back. It would not move.
The model panel’s hint text explained the stickiness:
choose other main/VAE files from Model parameters before loading
Reasonable. Weights are selectable while the model is unloaded and pinned once resident. I hard-refreshed, set q4_0 first, loaded, confirmed the field, and ran again.
What I did not do was check whether the q8_0 file existed. The dropdown listed it, so I assumed it was there. A dropdown is a UI affordance, not an assertion about the filesystem — but it reads like one.
Illusion two: a config file that was the real authority
When I finally opened the server config, the answer was sitting in plain text:
"session_options": {
"yue2.model_gguf": "yue2-3b-q4_0.gguf",
"yue2.vae_gguf": "yue2-vae-f16.gguf"
}The weights were pinned in yue2-server.json before either run. The dropdown was not controlling anything. It displayed a value, it accepted clicks, and the engine loaded whatever the config said regardless.
So the “reversion” I spent time debugging was the UI showing a default it never applied. Not a bug I triggered by clicking in the wrong order — a display that was never connected to the thing it appeared to control.
The numbers, and the wrong story I built from them
| Run 1 | Run 2 | Delta | |
|---|---|---|---|
| Wall clock | 1,205,350 ms | 1,211,450 ms | +6,100 ms |
| Audio duration | 39,478.7 ms | 39,478.7 ms | 0 |
| Real-time factor | 30.5318 | 30.6863 | +0.15 |
I read this as: 4-bit weights are half the size of 8-bit, CPU inference is usually memory-bandwidth-bound, so q4_0 should have been meaningfully faster — and it wasn’t, it was 0.5% slower.
I had an explanation ready. Dequantization overhead offsetting the bandwidth savings. The F16 VAE decode stage dominating the clock. Both are real phenomena, both would have sounded convincing in a blog post, and both were answers to a question I had not actually asked.
Notice the middle row. Identical audio duration to four decimal places. I described that at the time as the benefit of a fixed seed — clean comparability. It was the first clue and I read it as a feature.
The command that ended it
Before comparing the two files by ear, I hashed them:
md5sum q8_0-output.wav q4_0-output.wav
feb240f3f1ac7fc150112a45b10aaf94 q8_0-output.wav
feb240f3f1ac7fc150112a45b10aaf94 q4_0-output.wavBit-for-bit identical.
Different weights cannot produce identical output. A fixed seed controls the random draws, not the probability distribution those draws are applied against. Q4_0 and Q8_0 are different numerical approximations of the same tensors; they produce different logits, different sampling decisions, and divergence that compounds over thousands of tokens. Identical bytes means identical weights.
The confirmation took one more command:
ls -la ~/audio-cpp-yue2/models/Yue2-3B-GGUF/
-rw-rw-r-- 1 lvydvy lvydvy 2665632320 Sep 13 00:09 yue2-3b-q4_0.gguf
-rw-rw-r-- 1 lvydvy lvydvy 265218656 Sep 13 00:10 yue2-vae-f16.ggufNo q8_0. There never was. I had downloaded one set of weights and spent an evening comparing it to itself.
What I actually measured
Stripping out everything that was not real, one genuine result remains:
Run-to-run variance on identical work is about 0.5% — 1,205,350 ms versus 1,211,450 ms for the same weights, same seed, same prompt.
That is a useful number. It establishes a noise floor. Any future comparison on this machine showing more than roughly 1% difference is signal; anything under that is the machine being a machine. I got a variance estimate, which is exactly the thing benchmark posts usually skip.
I did not measure anything about quantization.
The transferable part
Three separate signals were telling me the truth and I read all three as something else:
- A dropdown listing an absent file. UI controls enumerate what the software believes is possible, not what exists. The list was generated from known filename patterns, not from a directory scan.
- A setting that would not stick. I diagnosed this as load-order pinning and built a workaround. The real cause was that the control was decorative — my workaround “worked” in the sense that the value displayed correctly afterward.
- Byte-identical output length. I attributed this to the fixed seed. A fixed seed gives you comparable runs, not identical ones. Identical is a different claim and should have stopped me.
Each had an available innocent explanation, and I took it every time. That is the shape of the failure: not one bad inference but a chain of locally reasonable ones, each making the next easier to accept.
What broke it was not better reasoning. It was a cheap check against ground truth — hashing the artifacts instead of trusting the labels on them. md5sum does not care what the dropdown said.
The general form, which I keep relearning in security contexts and apparently need to relearn in inference ones: verify the artifact, not the interface that describes the artifact. A dropdown, a status field, a config value you did not read — these are claims. The file on disk is the fact.
Going forward
Threads first. The config specifies "threads": 4 on a machine with 4 physical and 8 logical cores. Zero download, one config edit, and I now have a variance floor to measure against. This is the cheapest real experiment available.
Then the actual quantization comparison, which requires downloading the q8_0 weights I assumed I already had.
And a verification habit: hash the outputs before drawing conclusions from them, and ls the model directory before believing any UI about which model is loaded.
The 30.5 real-time factor stands. Twenty minutes for thirty-nine seconds, on CPU, locally. That part was never in question — it is the only thing here I measured without accidentally measuring something else.
Everything ran locally on consumer hardware. No audio, lyrics, or prompts left the machine.