LLM Model Roster — llm box

CT 100 · Strix Halo iGPU · 11 models · sorted fastest→slowest · updated 2026-07-18

Model tag	Decode speed (tok/s)	Arch / active	KV cost	ctx	Vision	Quality (quick)
qwen3-coder:ctx64kqwen3moe MoE · fastest	68	MoE / ~3B	exp +36GB	65K	—	—
glm-flash:ctx64kdeepseek2 MoE 4/64 · MLA	63	MoE / ~4B	exp +20GB	65K	—	—
ornith:ctx131kqwen35moe · Agents-A1 lineage	58	MoE / ~3B	cheap*	131K	✓	—
gemma-26b:ctx131kgemma4 MoE · sliding-window	57	MoE / ~4B	cheap +0.3GB	131K	✓	—
qwen3.6:ctx131kqwen35moe +MTP · Hermes main	53	MoE / ~3B +MTP	cheap +0.5GB	131K	✓	gsm8k .90 · ifeval .81
nemotron-omni:ctx131knemotron_h (Mamba2+attn) MoE	44	MoE / ~3B	cheap +1.4GB	131K	✓	—
agentworld:ctx131kqwen35moe · no MTP	44	MoE / ~3B	cheap*	131K	—	gsm8k .92 · ifeval .75
gpt-oss:ctx64kgpt-oss MoE 120B · councils	30	MoE / ~5B	cheap*	65K	—	—
gemma-12b:ctx131kgemma4 dense · Hermes vision aux	24	DENSE / 12B	cheap	131K	✓	—
gemma-31b:ctx131kgemma4 dense · unsloth Q4_K_M	11	DENSE / 31B	cheap	131K	✓	—
gemma-31b-qat:ctx131kgemma4 dense · Google QAT q4_0	11	DENSE / 31B	cheap	131K	✓	—

Key takeaways

Speed = MoE vs dense, nothing else. Every sparse MoE runs 30–68 tok/s; the 3 dense gemmas collapse to 11–24. On this bandwidth-bound iGPU, active-params-per-token is the only speed factor.
KV cost ≠ speed. The two fastest (qwen3-coder, glm-flash) are the two most KV-expensive (+36 / +20 GB) → pinned 65K. Cheap-KV comes from sliding-window (gemma, gpt-oss), Mamba (nemotron), or aggressive GQA (qwen3.6). Green KV = measured Δ 8K→131K; * = inferred by arch.
Modality: vision works on 7 models; audio works on none — even the omni model (ollama has no audio path).
Quality panel run for only 2 so far — agentworld edges qwen3.6 on math (.92 vs .90) but loses on instruction-following; qwen3.6 + MTP stays the best all-rounder. Next: quality panel on ornith / glm-flash / nemotron / qwen3-coder.

All tags standardized to name:ctx<N>k 2026-07-18 · source of truth: ollama list on llm + memory llm-model-roster