MODELS →

Inkling 2 Mini

Size
26.4B
3.4B active — only part of the model works on each answer, but all of it has to fit in memory
Context
1M
the longest conversation the model is built to hold
Lab
Thinking Machines
the lab that released it
Released
Nov 20, 2026

Licence: Apache 2.0 — free for any use, including commercial. Just keep the licence notice.

Worth knowing

Thinking Machines' small looped MoE, weights published Nov 20, 2026 on Hugging Face under Apache 2.0 (the card also links the lab's Model Acceptable Use Policy): 26.4B total parameters, 3.4B active per token — 5.4B effective once looping is counted, because the middle 12 of its 24 layers run three times per token (48 layer visits from 24 stored layers); each token routes to 8 of 128 experts; hybrid local/global attention; native vision input; context up to 1M tokens. It ships as a single checkpoint of about 20GB in native NVFP4 (last four layers MXFP8) — trained in the precision it ships in, so no conversion loss. The card's hardware floor: 1x DGX Spark, Apple silicon with 32GB+, or another GPU with 32GB+ VRAM recommended (the ~20GB file fits smaller memory on paper; the recommendation is the maker's, and KV cache at long context is why). Memory by arithmetic at full precision: about 53GB of weights — one 80GB-class card. Serving on day one: vLLM, SGLang, Transformers, TokenSpeed, NVIDIA Dynamo, Unsloth and llama.cpp GGUF, Ollama on Apple silicon. Watch-outs: every benchmark is the lab's own table (DeepSWE v1.1 48.2% and Artificial Analysis Index 20.1, tying Qwen3.8-27B, are lab-reported); the card itself puts multi-turn chat and constraint-following behind Qwen3.8-27B, and warns that looped-model support in local runtimes is new and speeds vary. A variable-loop research variant, TML-d24-varloop-research, was released alongside. Inkling 2 Small (594B/17B) is the big sibling, released Nov 12.

What it needs

Full precision (FP16)
53 GB
the weights, uncompressed
8-bit
26 GB
the weights, compressed to 8-bit
4-bit
13 GB
the weights, compressed to 4-bit

How we work this out: memory = the model's total parameters × bytes per weight — 2 bytes at full precision (FP16), 1 byte at 8-bit, half a byte at 4-bit. Weights only — add room for the conversation memory (KV cache) and the serving software. A GPU counts as running a build when its memory covers the weights plus about 20% headroom. Real figures move with context length and software.

GPUs that can run it

Ranked by how well the model runs, then by price.

1.RTX PRO 6000 Blackwell Max-Q Most recommended Runs it well95GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 95GB, with room left for the conversation memory.

This is the Max-Q version: it runs at lower power and is slower than the full card.

Lowest verified: nafnaf.ai — $0.75/hr Checked Nov 21, 2026 Launch ↗
2.A100 80GB Runs it well80GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.

An older generation — no FP8, so the newer speed tricks for serving don't apply.

Lowest verified: Thunder Compute — $1.09/hr Checked Nov 21, 2026 Launch ↗
3.RTX PRO 6000 Runs it well96GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 96GB, with room left for the conversation memory.

No NVLink scale-out; strong all-round, but the Max-Q version (listed separately) runs slower.

Lowest verified: RunPod Community — $1.69/hr Checked Nov 21, 2026 Launch ↗
4.H100 PCIe Runs it well80GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.

No NVLink, and lower power than the SXM — slower when cards must work together.

Lowest verified: RunPod Community — $1.99/hr Checked Nov 21, 2026 Launch ↗
5.H100 SXM Runs it well80GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.

Pays off only when the card stays busy; on bursty work you pay H100 rent for idle silicon.

Lowest verified: TensorDock — $2.25/hr Checked Nov 21, 2026 Launch ↗
6.H100 NVL Runs it well94GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 94GB, with room left for the conversation memory.

Sold as linked pairs, not as a lone card.

Lowest verified: RunPod Community — $2.59/hr Checked Nov 21, 2026 Launch ↗
7.H100 NVLink Runs it well80GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.

One provider's label, published alongside its plainer “H100 PCIe” — check which card you are actually booking.

Lowest verified: Hyperstack — $2.60/hr Checked Nov 21, 2026 Launch ↗
8.H200 Runs it well141GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 141GB, with room left for the conversation memory.

More card than most jobs need; the main trap is paying for it on small work.

Lowest verified: RunPod Community — $3.59/hr Checked Nov 21, 2026 Launch ↗
9.H100 Runs it well80GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 80GB, with room left for the conversation memory.

Pays off only when kept busy — avoid it for bursty development work.

Lowest verified: Nebius — $4.50/hr Checked Nov 21, 2026 Launch ↗
10.B200 Runs it well192GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 192GB, with room left for the conversation memory.

Frontier training money. If your job fits on anything cheaper, a B200 is budget burned for a spec sheet.

Lowest verified: RunPod Community — $5.98/hr Checked Nov 21, 2026 Launch ↗
11.GH200 Runs it well96GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 96GB, with room left for the conversation memory.

Its processor is ARM, not the usual x86 — some software needs changes to run on it.

Lowest verified: CoreWeave — $6.50/hr Checked Nov 21, 2026 Launch ↗
12.B300 Runs it well288GB

Runs Inkling 2 Mini at full precision — its 53GB of weights fit this card's 288GB, with room left for the conversation memory.

Very costly for small jobs — best value only on big work.

Lowest verified: Hyperstack — $7.40/hr Checked Nov 21, 2026 Launch ↗
13.RTX A6000 Runs it compressed48GB

Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 48GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.

An older generation — no FP8, so the newer speed tricks for serving don't apply.

Lowest verified: Thunder Compute — $0.35/hr Checked Nov 21, 2026 Launch ↗
14.RTX 5090 Runs it compressed32GB

Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 32GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.

No NVLink, so two cards can't pool memory. A consumer card — driver and ECC behaviour matter for production serving.

Lowest verified: RunPod Community — $0.69/hr Checked Nov 21, 2026 Launch ↗
15.L40S Runs it compressed48GB

Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 48GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.

70B at FP16 does not fit, and no flag fixes that. Weak at double-precision maths; no NVLink.

Lowest verified: RunPod Community — $0.79/hr Checked Nov 21, 2026 Launch ↗
16.L40 Runs it compressed48GB

Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 48GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.

Weak at double-precision maths — not the card for scientific computing. No NVLink.

Lowest verified: Thunder Compute — $0.79/hr Checked Nov 21, 2026 Launch ↗
17.A100 40GB Runs it compressed40GB

Runs Inkling 2 Mini compressed — the 8-bit build needs 26GB, which fits this card's 40GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.

The 40GB fills up quickly on larger models.

Lowest verified: Verda — $1.73/hr Checked Nov 21, 2026 Launch ↗
18.RTX 4090 Runs it compressed24GB

Runs Inkling 2 Mini compressed — the 4-bit build needs 13GB, which fits this card's 24GB. The compromise: slightly lower answer quality and a shorter conversation memory than the full-precision build.

24GB fills up fast on big models. No NVLink, so two cards can't pool memory — and it is a gaming card at heart.

Lowest verified: RunPod Community — $0.34/hr Checked Nov 21, 2026 Launch ↗

The cheapest verified way to run it

RTX 4090 at RunPod Community — $0.34/hr — runs it compressed — the 4-bit build.

Launch RunPod Community ↗  See the RTX 4090 →

Checked Nov 21, 2026

Select a GPU  See these GPUs on the prices board, ranked for this model.