Updated 2026-09-08 · Athena / the LLM host, AMD Strix Halo, 128 GB unified · ollama 0.33.3, Vulkan, NUM_PARALLEL=3 · lm-eval-harness, harness d41297a
Quality & speed figures below are from the 2026-08-15 sweep on 0.32.13 and are not re-measured — only the 2026-09-08 box directly below is new.
NUM_PARALLEL was never the real problem.
A gemma4 model aborts once the full-attention K buffer for a SINGLE LAYER exceeds 1 GiB.
gemma4 interleaves 5 sliding-window layers per 1 full-attention layer, and only the full-attention layers scale with
context. K grows as num_ctx × NUM_PARALLEL × n_embd_k_gqa × 2, so the usable budget is a
product of context and parallelism — raise either and the other must fall:
| Model | Full-attn layers | n_embd_k_gqa | Budget: num_ctx × NP ≤ | Pin as of today | Headroom |
|---|---|---|---|---|---|
| gemma-12b | 8 of 48 | 512 | 1,048,576 | ctx131k × NP 3 | 38% |
| gemma-26b | 5 of 30 | 1024 | 524,288 | ctx131k × NP 3 | 75% |
| gemma-31b | 10 of 60 | 2048 | 262,144 | ctx200k × NP 1 separate instance | 78% |
The failure is nastier than a crash. Generation itself is fine — the request
completes and the answer is delivered at full speed. The abort happens afterwards, on slot release, when llama.cpp
serialises the slot into its prompt cache (server_slot::prompt_save →
GGML_ASSERT(tensor->data != NULL)). The runner then dies and reloads for ~14 s. A client sees
mostly-correct answers with intermittent errors and bad latency, not an outage — which is how
gemma-31b:ctx200k sat broken from 2026-09-05 until today unnoticed.
🔎 This retro-explains two entries already on this page.
The roster note that gemma-31b “was aborting two requests in three until NUM_PARALLEL went 3 → 2”
is this same cliff: at ctx131k × 3 the 31b sits at 150% of budget, and NP=2 put it at exactly 100%.
The global NP=2 has been a three-week workaround for one model’s context pin.
And gemma-31b-qat, dropped in the 2026-08-17 prune for “crashes the Vulkan runner”, is a gemma4 31B
that would have been over the same budget — likely removed for a config fault, not a bad model
(untestable now; the tag is gone).
What changed today: main host NUM_PARALLEL raised 2 → 3; pins are gemma-12b:ctx131k (38%) and gemma-26b:ctx131k (75%), both verified 30/30 clean at NP=3. gemma-31b:ctx200k relocated to the NUM_PARALLEL=1 instance, also 30/30 clean.
✅ gemma-31b keeps its full 200k — by moving hosts, not by lowering context.
OLLAMA_NUM_PARALLEL is box-wide, and there is no per-model override
(ollama create rejects num_parallel outright), so a single instance cannot serve the 31b at 200k
and give everything else 3 slots. Serialising callers is not an alternative: gemma4 reserves
full-attention K at load from num_ctx × NP, not per request, so even one lone request aborts
when the pin is over budget. The fix is a second ollama instance running NUM_PARALLEL=1,
where ctx200k is 204,800 = 78% of budget — verified 30/30 clean, zero aborts. The main host stays at
NP=3 for everything else.
Ruled out by measurement, so nobody re-derives them:
not memory pressure (76 GiB free at abort; NP=4 allocates more and fails no worse) · not the vision/CLIP
path (gemma-12b is vision too and is clean) · not the 4 GiB Vulkan allocation limit (a 4480 MiB K buffer runs
clean) · not total KV bytes or cells — glm-flash:ctx202k holds a 31,407 MiB KV
buffer at NP=3, 2.6× the gemma-26b buffer that fails, and is perfectly clean because deepseek2 spreads it over 40+
layers. Only the per-layer figure predicts. The 1 GiB constant is empirical — bisected to
1024 MiB/layer clean vs 1056 MiB/layer crash.
temperature: 0, was run twice over the same 308 MMLU-Pro items. 29 answers — 9.4% — flipped between runs. Greedy decoding on this stack is not deterministic. Aggregate percentages look stable only because roughly equal numbers flip each way and cancel out. Every single-run score below, and every stderr in the CSV, understates the real uncertainty. Treat any gap under ~2 points as no gap at all.
Removed in the 2026-08-17 prune (104 GB reclaimed, the LLM host rootfs 78% → 61%):
glm-flash-xl:ctx64k, qwen3.6-xl:ctx131k, nemotron-q5:ctx131k,
plus gemma-31b-qat (crashes the Vulkan runner) and ornith (third fine-tune of an
architecture already covered).
🚨 glm-flash-xl should probably not have gone.
This page's own conclusion is “only glm-flash-xl pays — smaller and identical in quality, use it”.
It was proposed for removal as a “duplicate tune”, which understated it. No live impact
(glm-flash is not currently on the forum roster — Explorer reverted to laguna),
but restoring it needs a re-download of Unsloth’s UD-Q4_K_XL build: its weights blob was
unique to that tag and is gone, so the preserved Modelfile alone cannot rebuild it.
New: gpt-oss:ctx131k — the 120B at its full architectural
context (131,072, its ceiling). 66 GiB measured on the host (/api/ps claimed 60.1 —
it under-reports), 36.5 tok/s, needle retrieved at 10/50/90% depth. Full context costs only
~4.7 GiB more than ctx64k. This doubles its per-result window (39,321 → ~78,643 chars);
it had never completed a second forum round at ctx64k, and did so first time at 131k in 4 minutes.
Forum roster now:
deep — Scholar gemma-31b · Theorist muse-glimmer-dflash ·
Empiricist qwen3.8 · Polymath gpt-oss:ctx131k.
fast — Analyst gemma-26b · Pragmatist nemotron-omni ·
Explorer laguna · Investigator agentworld.
Skeptic qwen3.6 and Editor gemma-26b in both.
Explorer and Investigator were reverted from glm-flash/qwen3.8 on 2026-08-17.
Token ceilings re-derived from measurement: 24,000 researchers / 32,000 Editor.
The previous 12,000 was sized from fast-roster output and would have truncated real work —
qwen3.8 had produced 16,124 tokens and gemma-26b 22,423. A trace over the cap returns
empty, so too low is the dangerous direction.
Full per-model provenance (upstream HF repo, quantiser, licence, author’s recommended sampling vs ours, what each tag actually costs on disk) is now generated rather than hand-kept: model inventory.
muse-glimmer (with a working DFlash drafter), qwen3.8, and the three higher-quantisation builds qwen3.6-xl, glm-flash-xl, nemotron-q5. The engine moved 0.32.4 → 0.32.13 and NUM_PARALLEL 3 → 2, so every speed and resident figure on this page was re-measured in one sweep on 2026-08-15 rather than carried over from mixed dates. The “is Unsloth’s XL tier actually better?” question is now answered by measurement instead of assumption.
What to compute before changing any num_ctx or NUM_PARALLEL on a gemma4 tag.
The constraint is on one layer’s K buffer, so it depends on the model’s attention geometry, not on its size in GB or on total KV. Two numbers give you everything:
budget_cells = 2^29 / n_embd_k_gqa # per model, fixed
max_num_ctx = budget_cells / NUM_PARALLEL # what you may pinn_embd_k_gqa is that of the full-attention layers. gemma4 interleaves 5 sliding-window
layers per 1 full-attention layer, and only the full-attention ones grow with context — they are the
small-head entries in the array. Read both off the load log:
journalctl -u ollama | grep -E 'n_embd_k_gqa|n_head_kv|llama_kv_cache: size'
print_info: n_embd_k_gqa = [2048, 2048, 2048, 2048, 2048, 1024, ...] <- 1024 = full-attn
llama_kv_cache: size = 7680.00 MiB (131072 cells, 5 layers, 3/3 seqs) <- 5 full-attn layersVerify after loading — the llama_kv_cache line for the full-attention cache is the
ground truth. That size covers K and V across all its layers, so:
per_layer_K_MiB = size_MiB / layers / 2 must be ≤ 1024
7680 / 5 / 2 = 768 MiB ✅ clean
12000 / 5 / 2 = 1200 MiB ❌ abortsnum_ctx per model| Model | Budget | NP=1 | NP=2 | NP=3 | NP=4 |
|---|---|---|---|---|---|
| gemma-12b | 1,048,576 | 262,144 capped | 262,144 capped | 262,144 capped | 262,144 |
| gemma-26b | 524,288 | 262,144 capped | 262,144 | 174,762 | 131,072 |
| gemma-31b | 262,144 | 262,144 | 131,072 | 87,381 | 65,536 |
“capped” = the budget exceeds the model’s trained context
(n_ctx_train = 262,144), so that is the real ceiling. Values are exact limits — a pin
at the limit is fine (ctx256k×NP2 on the 26b and ctx131k×NP2 on the 31b were both measured clean at
exactly 100%), but leaves no margin if NUM_PARALLEL is ever raised.
glm-flash:ctx202k at NP=3 holds a
31,407 MiB KV buffer, 2.6× the gemma-26b buffer that aborts, and is perfectly clean, because
deepseek2 spreads it across 40+ layers. Check any new hybrid or sliding-window model the same way; existing
non-gemma tags need no action.
Everything added since the 2026-08-08 edition of this page, and what each one is for.
| Tag | What it is | Speed tok/s | Knowledge MMLU-Pro n=308 | Verdict |
|---|---|---|---|---|
| muse-glimmer:ctx131k-dflash added 08-14 |
28B, its own muse-glimmer architecture, vision + tools + thinking, ships with a DFlash speculative drafter |
24.5 1.90× its own no-drafter arm |
75.3% gap 0.0 |
KEEP Third-best knowledge on the box with a zero extraction gap, and the cheapest KV of any large model here. Bottom of the speed band even with the drafter. |
| qwen3.6-xl:ctx131k added 08-09 |
Unsloth UD-Q4_K_XL of the same Qwen3.6-35B-A3B we already run |
52.2 | 77.9% | NO Bigger on disk, 30% slower, and not measurably better (77.9 vs 79.2 — inside the noise floor). The slowdown is its Modelfile’s MTP depth 4, not the quantisation. |
| glm-flash-xl:ctx64k added 08-08 |
Unsloth UD-Q4_K_XL of GLM-4.7-Flash |
68.6 | 61.7% | FREE WIN 0.7 GB smaller weights than the Q4_K_M build at identical quality (61.7 vs 61.4) and identical speed. The one case where XL pays. |
| nemotron-q5:ctx131k added 08-08 |
Q5_K_M of nemotron-omni — the Mamba-2 hybrid |
57.1 | 52.9% 62.7 flexible |
MARGINAL Real but small gain over the Q4 build (+5.2 custom, +3.3 flexible) for +5.1 GB resident and 3 tok/s. Buy it only if the extra 5 GB is free. |
| qwen3.8:ctx131k added 08-15 |
Qwen3.8-27B, dense (not MoE), vision + tools + thinking | 12.6 | deferred | REJECT 6× slower than qwen3.6 and the reason is architectural — a dense 27.3B reads 15 GiB per token on a 120 GB/s bus. Quality benchmark deliberately deferred: at 12.6 tok/s one n=308 run costs ~5 h. |
/api/ps lies about drafter models. muse-glimmer:ctx131k-dflash reports 3.6 GB resident against its own no-drafter arm’s 17.8 GB. The real figure, measured as host-memory delta, is 22.8 GiB — and host delta is the only trustworthy source here, because the GPU pool is charged to the host, not to the container. Never size a memory budget from /api/ps on a tag that carries a drafter.
Reasoning is think-mode BBH; knowledge is nothink MMLU-Pro custom-extract. Each cell is labelled with when it was measured — they are not all from the same run. Speed is: every figure comes from the 2026-08-15 sweep on 0.32.13.
| # | Model | Reasoning BBH CoT think |
Knowledge MMLU-Pro nothink |
GSM8K | IFEval | Speed tok/s 08-15 |
Verdict |
|---|---|---|---|---|---|---|---|
| 1 🥇 | qwen3.6 Hermes main | 94.4% 07-27 | 79.2% 08-15 · n=308 | 90.5% | 83.9% | 74.8 | The house default, and it earns it. Top reasoner, MTP-accelerated. Knowledge re-measured at n=308 and it went up (74.3 → 79.2). |
| 2 🥈 | agentworld | 92.6% 07-27 | 82.9% 07-27 · n=70 | 91.6% | 75.2% | 58.5 | Best knowledge and near-top reasoning — but its headline score is still the small-sample one. Weak at instructions. |
| 3 🥉 | ornith | 88.9% 07-27 | 74.3% n=70 | 88.0% | 83.7% | 68.8 | Quietly excellent all-rounder. Vision enabled. |
| 4 | gpt-oss 120B MoE | 87.0% 07-27 | 50.0% n=70 | 37.9% | 76.3% | 35.7 | +54pt thinking gain. Reasoning cannot be switched off — see caveats. 65 GB resident. |
| 5 | gemma-26b | 42.6% nothink | 81.4% n=70 | 83.0% 08-11 · n=200 | 86.9% | 65.4 | The low-risk pick: strong scores, zero extraction gap, no think machinery to misconfigure. Lightest of the majors at 16.8 GB. |
| 6 | qwen3.6-xl NEW | — | 77.9% 08-09 · n=308 | — | — | 52.2 | The XL repack of #1. Same answers, more disk, less speed — see the quantisation section. |
| 7 | muse-glimmer NEW | — | 75.3% 08-15 · n=308 | — | — | 24.5 dflash | Zero extraction gap at 75.3% — it answers in exactly the requested format, like gemma-26b and glm-flash. Vision + tools + thinking in one tag. |
| 8 | gpt-oss-20b | — | 73.1% 08-07 · n=308 | — | 89.6% | 50.9 | RE-READ Best knowledge in the roster, hidden behind the worst formatting on the box (+37.7pt gap, held at n=308). The headline 35.4% measures answer format, not ability. Treat 73.1% as an upper bound. |
| 9 | gemma-12b vision aux | 40.7% nothink | 71.4% n=70 | — | — | 27.9 | Punches above its size. No thinking mode. 10 GB — the cheapest vision on the box. |
| 10 | glm-flash | 79.2% think · n=313 | 61.4% 08-08 · n=308 | — | — | 67.4 | FIXED ×2 The broken quantisation was fixed 08-04; the runaway generation was fixed by ollama 0.32.11. It now stops on its own, 4/4. Zero extraction gap. |
| 11 | glm-flash-xl NEW | — | 61.7% 08-08 · n=308 | — | — | 68.6 | Statistically the same model as the row above, 0.7 GB lighter. Prefer this build. |
| 12 | laguna | 68.5% 07-27 | 57.1% n=70 | — | 87.4% 08-12 · n=541 | 88.2 🏆 | Fastest on the box, and a much better instruction-follower than we knew — 87.4 loose / 85.7 strict at n=541. Weakest reasoner. Latency-sensitive work. |
| 13 | qwen3-coder | 59.3% nothink | — | 89.3% | 80.6% | 84.5 | Strong at math and now the second-fastest tag. Think-mode reasoning never measured. Most expensive KV on the box. |
| 14 | nemotron-omni | 75.9% n=54 | 47.7% 08-08 · n=308 59.4 flexible |
79.7% | 80.8% | 59.8 | Multimodal, and by far the cheapest KV (6.4 MiB/1k). CORRECTED the previous edition of this page printed 61.7% here; the n=308 run is 47.7 custom / 59.4 flexible. |
| 15 | nemotron-q5 NEW | — | 52.9% 08-08 · n=308 62.7 flexible |
— | — | 57.1 | The Q5 build of the row above. Genuinely better, but most of the gain is formatting and it costs 5 GB. |
| 16 | gemma-31b / -qat | — | — | — | — | 11.9 | Dense 31B. Was aborting two requests in three at NUM_PARALLEL=3. Cause identified 2026-09-08: not NP itself but the gemma4 per-layer KV cliff — at ctx131k×3 it sits at 150% of its budget. Now pinned ctx80k and clean at NP=3. See the 2026-09-08 box at the top. |
| 17 | qwen3.8 NEW | — | deferred | — | — | 12.6 | REJECT on speed Dense 27.3B. Its MTP head exists in the GGUF and ollama discards it — there is no version to wait for. |
Three of the roster now run a drafter. It is worth more than any quantisation choice, and it is the single reason muse-glimmer is usable.
| Model | Drafter | Depth | Without | With | Gain |
|---|---|---|---|---|---|
| muse-glimmer NEW | draft-dflash | 3 | 12.9 | 24.5 | 1.90× |
| qwen3.6 | draft-mtp | 2 | — | 74.8 | acceptance 0.32, mean len 2.27 |
| qwen3.6-xl NEW | draft-mtp | 4 | — | 52.2 | −30% vs depth 2 |
| qwen3.8 NEW | MTP head present, ignored | — | 12.6 | 12.6 | none |
qwen3.6-xl ships with MTP depth 4 where our tuned qwen3.6 uses 2, and that alone accounts for most of the 74.8 → 52.2 gap. A rejected draft token is wasted work, so past the acceptance rate the drafter costs more than it saves. Never tune draft depth on a counting prompt — those invert the result.
qwen35.nextn_predict_layers = 1, and the runner log reads model has unused tensor blk.64.nextn.eh_proj.weight -- ignoring, then no implementations specified for speculative decoding and draft: 0.000 MiB. ollama has no MTP implementation for arch qwen35. This is not a wait-for-the-next-version situation — 0.32.12 is the release that added qwen3.8 support and 0.32.13 still ignores the head. Reviving it needs upstream llama.cpp work.
The 08-08 edition listed which UD-Q4_K_XL and Q5_K_M builds existed upstream. Three have since been pulled and run. The answer is “usually not, and never for the reason you expect.”
| Pair | Weights | Knowledge custom / flexible, n=308 | Speed | Resident | Worth it? |
|---|---|---|---|---|---|
glm-flash Q4_K_Mglm-flash-xl UD-Q4_K_XL |
17.3 GB 16.5 GB |
61.4 / 61.7 61.7 / 62.7 |
67.4 68.6 |
25.8 GB 25.0 GB |
YES smaller and identical — the XL build is strictly the better buy |
qwen3.6 Q4_K_Mqwen3.6-xl UD-Q4_K_XL |
20.4 GB 21.3 GB |
79.2 / 82.1 77.9 / 79.5 |
74.8 52.2 |
22.9 GB 23.9 GB |
NO bigger, slower, no better |
nemotron-omni UD-Q4_K_Mnemotron-q5 Q5_K_M |
22.4 GB 27.3 GB |
47.7 / 59.4 52.9 / 62.7 |
59.8 57.1 |
25.2 GB 30.3 GB |
MARGINAL +5.2pt custom for +5.1 GB |
qwen3.6-xl is slower than qwen3.6 because of a Modelfile setting (MTP depth 4), not because of its quantisation, and glm-flash-xl is smaller than its Q4_K_M sibling even though “XL” sounds larger. Read the file sizes and the Modelfile before assuming which way a tier moves anything.
The single most important finding of 2026-07-27. Thinking helps you work out an answer, not recall one.
| Task | Axis | nothink → think | Significance | Use |
|---|---|---|---|---|
bbh_cot_zeroshot | reasoning | qwen3.6 35–41% → 94.4% | +54pt | THINK |
| gpt-oss 33.3% → 87.0% | +54pt | |||
| laguna 39.9% 08-13 · n=313 → 68.5% | +28.6pt | |||
mmlu_pro | knowledge | laguna flexible 67.1% → 71.4% | +4.3pt · 0.56σ = noise | NOTHINK |
| laguna custom 57.1% → 41.4% | −15.7pt · 2.0σ | |||
| qwen3.6 custom 79.2% → 84.7% 08-10 · n=308 | +5.5pt · the exception |
think for reasoning and math; nothink for knowledge. Think mode also costs 6–7.5× wall time, so it is never a free upgrade. On knowledge tasks it usually hurts: the reasoning trace tripled laguna’s extraction gap (+10pt → +30pt), so its score fell even though its answers did not get worse. qwen3.6 is the documented exception — at n=308 thinking bought it +5.5pt on knowledge and closed its extraction gap to zero. The rule is a default, not a law; check the model you actually use.
custom-extract demands MMLU-Pro’s answer format; flexible-extract accepts a bare answer. A wide gap means the model knew the answer but wrote it the “wrong” way.
| Model | custom | flexible | gap | n | What the gap means |
|---|---|---|---|---|---|
| gemma-26b / gemma-12b | 81.4 / 71.4 | 81.4 / 71.4 | 0.0 | 70 | Perfectly clean. |
| muse-glimmer NEW | 75.3 | 75.3 | 0.0 | 308 | Clean at full sample size — the strongest zero-gap result on the page. |
| glm-flash / glm-flash-xl | 61.4 / 61.7 | 61.7 / 62.7 | +0.3 / +1.0 | 308 | Gap gone once the quantisation was fixed — the rambling was the bug, not the model. |
| agentworld | 82.9 | 84.3 | +1.4 | 70 | Clean. |
| qwen3.6 | 79.2 | 82.1 | +2.9 | 308 | Clean. Closes to 0.0 with thinking on. |
| qwen3.6-xl NEW | 77.9 | 79.5 | +1.6 | 308 | Clean. |
| laguna | 57.1 | 67.1 | +10.0 | 70 | Moderate formatting loss. |
| nemotron-q5 NEW | 52.9 | 62.7 | +9.8 | 308 | Inherits the family habit, slightly improved. |
| nemotron-omni | 47.7 | 59.4 | +11.7 | 308 | Simply ignores the answer format. |
| gpt-oss | 50.0 | 71.4 | +21.4 | 70 | Reasoning-trace residue. |
| gpt-oss-20b | 35.4 | 73.1 | +37.7 | 308 | Worst on the box — last place on score, first on ability. |
Speed and resident size from the 2026-08-15 sweep (ollama 0.32.13, NUM_PARALLEL=2, one call per tag, unloaded between models). KV cost measured 2026-08-16.
| Model tag | Speed tok/s | Arch / active | Resident GB | KV MiB per 1k | Ctx pin | Vision |
|---|---|---|---|---|---|---|
| laguna:ctx131k laguna 33.4B | 88.2 | 33.4B | 21.6 | 42.3 | 131K | — |
| qwen3-coder:ctx64k qwen3moe | 84.5 | MoE / ~3B | 31.9 | 96.0 | 65K | — |
| qwen3.6:ctx131k qwen35moe +MTP 2 · Hermes main | 74.8 | MoE / ~3B +MTP | 22.9 | 21.2 | 131K | ✓ |
| ornith:ctx131k qwen35moe | 68.8 | MoE / ~3B | 24.7 | 20.5 | 131K | ✓ |
| glm-flash-xl:ctx64k deepseek2 MLA NEW | 68.6 | MoE / ~4B | 25.0 | 52.9 | 65K | — |
| glm-flash:ctx64k deepseek2 MLA | 67.4 | MoE / ~4B | 25.8 | 52.9 | 65K | — |
| gemma-26b:ctx131k gemma4 sliding-window | 65.4 | MoE / ~4B | 16.8 | 24.7 | 131K | ✓ |
| nemotron-omni:ctx131k Mamba2+attn MoE | 59.8 | MoE / ~3B | 25.2 | 6.4 🏆 | 131K | ✓ |
| agentworld:ctx131k qwen35moe, no MTP | 58.5 | MoE / ~3B | 25.2 | 20.5 | 131K | — |
| nemotron-q5:ctx131k Mamba2+attn MoE, Q5 NEW | 57.1 | MoE / ~3B | 30.3 | 6.4 | 131K | ✓ |
| qwen3.6-xl:ctx131k qwen35moe +MTP 4 NEW | 52.2 | MoE / ~3B +MTP | 23.9 | 22.5 | 131K | ✓ |
| gpt-oss-20b:ctx131k gptoss | 50.9 | MoE / ~4B | 13.6 | 24.4 | 131K | — |
| gpt-oss:ctx64k 120B MoE | 35.7 | MoE / ~5B | 64.6 | 36.7 | 65K | — |
| gemma-12b:ctx131k dense, vision aux | 27.9 | DENSE / 12B | 10.1 | — | 131K | ✓ |
| muse-glimmer:ctx131k-dflash +DFlash drafter NEW | 24.5 | 28B | 22.8 ⚠ not 3.6 | 14.2 | 131K | ✓ |
| muse-glimmer:ctx131k no drafter — the A/B arm NEW | 12.9 | 28B | 17.8 | 14.2 | 131K | ✓ |
| qwen3.8:ctx131k qwen35 dense NEW | 12.6 | DENSE / 27.3B | 26.3 | 64.6 | 131K | ✓ |
| gemma-31b / -qat:ctx131k dense | 11.9 | DENSE / 31B | 23.8 | 89–155 sub-linear | 131K | ✓ |
Also resident on the box but not part of the reasoning roster: readerlm:ctx64k (113.9 tok/s, 5.9 GB — HTML→markdown), nuextract:ctx32k (42.4 tok/s, 6.1 GB — structured extraction) and bge-m3 (embeddings).
-c is num_ctx × NUM_PARALLEL, so the parallelism setting multiplies the whole KV bill.
NUM_PARALLEL is 2, not 3, since 2026-08-15. ollama allocates n_ctx = num_ctx × NP, so a :ctx131k tag at NP=3 asks for 393216. At NP=3 both gemma-31b tags aborted about two requests in three at their own pin — GGML_ASSERT(tensor->data != NULL) failed → HTTP 500 roughly 44 s into generation. NP=2 fixed them at no measurable speed cost, and a 20-tag sweep confirmed nothing else broke.repeat_penalty from 1.1 to 1.0. It applies to any tag that does not set the parameter explicitly — 11 of 16 here, including nuextract, which does client extraction work. Only readerlm (1.08) and a handful of others carry an explicit line. If a model’s output character changed this month and nothing else did, this is why.n=70. Only large separations are real./api/ps under-reports any tag carrying a drafter (see the muse-glimmer box above). Measure host-memory delta instead — the GPU pool is charged to the host, not to the container, so nothing inside it can see the real total.until/budget override, so they are not a like-for-like nothink baseline.softmax where this architecture needs sigmoid, so the expert router mis-routed), fixed 2026-08-04; the remaining runaway generation was fixed by the engine at 0.32.11, verified 4/4. Detect the quantisation bug by checking expert_gating_func is present and set to 2 in the GGUF metadata. A later download date is no guarantee of a good build.qwen3.8, whose card advertises 262144.Every tag here is a local ctxNNN build, so the name is one we invented and says nothing about the source. Eight of the roster are Unsloth repacks, not official builds.
ollama show --modelfile — that prints blob paths, which identify nothing. It lives in /api/show: details.parent_model for library pulls, and the GGUF header keys general.quantized_by / general.base_model.0.repo_url for file imports. Several local tags have neither and cannot be traced from the box at all.
| Model | Original author | Packaged by | Quant we run | Higher tier available? |
|---|---|---|---|---|
| qwen3.6 | Qwen | ollama library | Q4_K_M +MTP | pulled as qwen3.6-xl — not worth it |
| qwen3.6-xl NEW | Qwen Qwen3.6-35B-A3B | Unsloth | UD-Q4_K_XL reports as Q4_K_M | — |
| qwen3.8 NEW | Qwen Qwen3.8-27B, apache-2.0 | Unsloth | Q4_K_M | official library tag also exists, untested |
| muse-glimmer NEW | untraceable on box | ollama library | Q4_K_M + mmproj + DFlash drafter, one pull | — |
| gemma-26b | ollama library QAT | Q4_0 | none q8_0 only | |
| qwen3-coder | Qwen | ollama library | Q4_K_M | none |
| gpt-oss-20b | OpenAI | ollama library | MXFP4 | none |
| glm-flash | Zai Org GLM-4.7-Flash | Unsloth | Q4_K_M | pulled as glm-flash-xl — use it |
| glm-flash-xl NEW | Zai Org GLM-4.7-Flash | Unsloth | UD-Q4_K_XL | — |
| nemotron-omni | NVIDIA | Unsloth | UD-Q4_K_M | pulled as nemotron-q5 — marginal |
| nemotron-q5 NEW | NVIDIA | Unsloth | Q5_K_M | — |
| agentworld | Qwen | Unsloth | UD-Q4_K_M | yes not pulled |
| gemma-31b | Unsloth | Q4_K_M | yes not pulled | |
| ornith | deepreinforce-ai | untraceable on box | Q4_K_M | Q5_K_M |
| laguna | poolside | untraceable on box | Q4_K_M | none Q4_K_M or BF16 only |
| gemma-12b | untraceable on box | Q4_0 | source repo unknown | |
| gpt-oss 120B | OpenAI | re-tagged import | F16 | parent tag 404s upstream |
quantized_by, no base_model.repo_url, and its parent_model points at itself. The GGUF says size_label = 28B and 27.85B parameters, so even the “30B” in its name is approximate. We can measure what it does; we cannot say from this box who built it.
q4_K_M to q8_0 — roughly double the weights, well past the point where it pays on a bandwidth-bound box. Raising quantisation on qwen3.6 or gemma-26b means leaving the library or quantising yourself.
Measured 2026-08-08 by comparing per-item results, not aggregate scores. Both runs used temperature: 0 over identical prompts.
| Comparison | Both right | Both wrong | Disagreed | Agreement | McNemar p |
|---|---|---|---|---|---|
| Same weights, two runs the control | 176 | 103 | 29 | 90.6% | 0.711 |
| Q4_K_M vs UD-Q4_K_XL | 169 | 98 | 41 | 86.7% | 1.000 |
Two conclusions, and the second only became visible because of the first.
stderr suggests, because that figure assumes the model is deterministic. Differences smaller than roughly 2 points between two single runs should be treated as no difference at all. This is exactly why qwen3.6 vs qwen3.6-xl (1.3pt) is reported above as “no difference”.
The scores above rank models on axes that are already well covered. These are the gaps — mostly about character rather than score, which is what actually decides where a model belongs.
It has thinking, vision and tools, a zero extraction gap and the cheapest KV of any large model here — and no BBH number at all. Given that think-mode is worth up to 54 points, this is the largest single blank on the page, and it decides whether the new arrival is a niche pick or a genuine contender.
Nothing on this page measures it. The code panel is a deliberate hard error in the runner: it executes model-generated code and was deferred pending a sandbox. Only humaneval and mbpp are available locally, and both are weak — HumanEval dates from 2021 and is in every training set.
Better: replay real Kanban coding cards. Known-good outcomes already exist, so re-running the same card per model measures the thing that matters — did it drive the tool protocol unaided, did the tests pass, how many turns, how long — with zero contamination and no sandbox needed.
IFEval measures whether a model obeys formatting constraints on a single turn. It does not measure whether a model keeps obeying them after 10 turns, or when the instruction conflicts with what it wants to say. That is the failure mode that actually breaks agent workflows, and the extraction-gap column hints at who is vulnerable: gpt-oss-20b at +37.7pt knows the answer and still will not say it in the requested shape.
Completely unmeasured, and arguably the most useful property for research work. A model that says “I am not sure” on the 25% it gets wrong is far more valuable in a panel than one that is 5 points more accurate and confident throughout. Cheap to test: ask for an answer plus a confidence, and plot confidence against correctness.
Every model here is pinned at 65k–131k context, and none has been measured beyond a few thousand tokens. Benchmarks use short prompts; forum threads and document extraction do not. ✅ A harness now exists (claude-share/scripts/ctx_qualify.py, 2026-08-17) and is queued to run across the roster. It asks the narrower question that actually matters — not where a model breaks, but whether it works at the context we pin it to. Three tiers: retrieve a fact at 10/50/90% depth, synthesise two facts planted far apart, and resist a question that presupposes something false. That third tier is the one no standard needle benchmark tests, and it is where weaker models are expected to drift. Two traps it avoids: repetitive filler tokenises at ~7 chars/token and flatters the result (an earlier probe “proved” 131k while actually testing 51k), so it uses real prose and reports the true prompt_eval_count; and all three tiers share one haystack so the prefill KV cache is reused.
The 9.4% flip rate was measured on one model. If it varies by architecture — MoE routing is a plausible culprit, since a near-tie between experts can tip either way — then stability is itself a model characteristic worth knowing. This costs nothing but a re-run of existing panels.