Inkling 2 Mini
Licence: Apache 2.0 — free for any use, including commercial. Just keep the licence notice.
Thinking Machines' small looped MoE, weights published Nov 20, 2026 on Hugging Face under Apache 2.0 (the card also links the lab's Model Acceptable Use Policy): 26.4B total parameters, 3.4B active per token — 5.4B effective once looping is counted, because the middle 12 of its 24 layers run three times per token (48 layer visits from 24 stored layers); each token routes to 8 of 128 experts; hybrid local/global attention; native vision input; context up to 1M tokens. It ships as a single checkpoint of about 20GB in native NVFP4 (last four layers MXFP8) — trained in the precision it ships in, so no conversion loss. The card's hardware floor: 1x DGX Spark, Apple silicon with 32GB+, or another GPU with 32GB+ VRAM recommended (the ~20GB file fits smaller memory on paper; the recommendation is the maker's, and KV cache at long context is why). Memory by arithmetic at full precision: about 53GB of weights — one 80GB-class card. Serving on day one: vLLM, SGLang, Transformers, TokenSpeed, NVIDIA Dynamo, Unsloth and llama.cpp GGUF, Ollama on Apple silicon. Watch-outs: every benchmark is the lab's own table (DeepSWE v1.1 48.2% and Artificial Analysis Index 20.1, tying Qwen3.8-27B, are lab-reported); the card itself puts multi-turn chat and constraint-following behind Qwen3.8-27B, and warns that looped-model support in local runtimes is new and speeds vary. A variable-loop research variant, TML-d24-varloop-research, was released alongside. Inkling 2 Small (594B/17B) is the big sibling, released Nov 12.
What it needs
How we work this out: memory = the model's total parameters × bytes per weight — 2 bytes at full precision (FP16), 1 byte at 8-bit, half a byte at 4-bit. Weights only — add room for the conversation memory (KV cache) and the serving software. A GPU counts as running a build when its memory covers the weights plus about 20% headroom. Real figures move with context length and software.
GPUs that can run it
Ranked by how well the model runs, then by price.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 95GB, with room left for the conversation memory.
This is the Max-Q version: it runs at lower power and is slower than the full card.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.
An older generation — no FP8, so the newer speed tricks for serving don't apply.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 96GB, with room left for the conversation memory.
No NVLink scale-out; strong all-round, but the Max-Q version (listed separately) runs slower.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.
No NVLink, and lower power than the SXM — slower when cards must work together.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.
Pays off only when the card stays busy; on bursty work you pay H100 rent for idle silicon.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 94GB, with room left for the conversation memory.
Sold as linked pairs, not as a lone card.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.
One provider's label, published alongside its plainer “H100 PCIe” — check which card you are actually booking.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 141GB, with room left for the conversation memory.
More card than most jobs need; the main trap is paying for it on small work.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.
Pays off only when kept busy — avoid it for bursty development work.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 192GB, with room left for the conversation memory.
Frontier training money. If your job fits on anything cheaper, a B200 is budget burned for a spec sheet.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 96GB, with room left for the conversation memory.
Its processor is ARM, not the usual x86 — some software needs changes to run on it.
Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 288GB, with room left for the conversation memory.
Very costly for small jobs — best value only on big work.
Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 48GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.
An older generation — no FP8, so the newer speed tricks for serving don't apply.
Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 32GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.
No NVLink, so two cards can't pool memory. A consumer card — driver and ECC behaviour matter for production serving.
Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 48GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.
70B at FP16 does not fit, and no flag fixes that. Weak at double-precision maths; no NVLink.
Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 48GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.
Weak at double-precision maths — not the card for scientific computing. No NVLink.
Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 40GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.
The 40GB fills up quickly on larger models.
Runs Inkling 2 Mini compressed — the 4-bit build needs 13GB, which fits this card's 24GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.
24GB fills up fast on big models. No NVLink, so two cards can't pool memory — and it is a gaming card at heart.
The cheapest verified way to run it
RTX 4090 at RunPod Community — $0.34/hr — runs it compressed — the 4-bit build.
Launch RunPod Community ↗ See the RTX 4090 →
Checked Nov 21, 2026
Select a GPU See these GPUs on the prices board, ranked for this model.