Inkling 2 Mini is the rare new model that is a one-card job. Thinking Machines Lab released the weights on Nov 20, 2026 as a single checkpoint of about 20GB — a 26.4B-parameter mixture-of-experts whose middle layers loop three times per token — and the cheapest verified card that a serving framework has actually verified for it rents for $0.69 an hour. This guide works the two honest routes: run the checkpoint exactly as shipped on one 32GB card, or run it dequantized at full precision on an 80GB card — what each route needs, which serving commands are documented today, and what each costs at prices checked Nov 21, 2026. It also names the cheap cards that look like they fit and, as of today, are not verified to run it.
The short answer
Rent one RTX 5090 and run the checkpoint as shipped. The weights file is 19.53GB (plus a 0.64GB bundled draft head for speculative decoding), the lab's own hardware floor is a GPU with 32GB or more of VRAM, and SGLang's cookbook has a verified single-GPU recipe for exactly this card — with one honest limit: on 32GB it caps the context at 65,536 tokens (32,768 with speculative decoding on) to fit the KV cache. At RunPod Community's $0.69/hr, checked Nov 21, 2026, that is $503.70 for a 730-hour month. If you need long context or several long sessions at once, step up to the 96GB RTX PRO 6000 at $1.69/hr rather than to a data-centre card: same shipped precision, SGLang-verified on one GPU, and three times the card memory — about six times the KV headroom once the weights are loaded. If you need the model at full BF16 precision — for fine-tuning, evaluation against the lab's numbers, or because your stack dequantizes on load anyway — you are renting an 80GB card and paying $1.09–$1.99/hr, and you should read the full-precision section first: Thinking Machines does not publish a BF16 checkpoint, so that route starts with a conversion, not a download.
What you are actually downloading — model card and repo file listing, read Nov 21, 2026
| The fact | The detail |
|---|
| 26.4B total parameters, 3.4B active per token | A 24-layer decoder-only transformer with sparse MoE feed-forward layers: each token routes to 8 of 128 experts. The middle 12 layers (6–17) run three times with shared weights, so each token makes 48 layer visits, and the lab counts 5.4B effective active parameters once the extra passes are included. |
| One checkpoint file of 19.53GB | model-00001-of-00001.safetensors, plus mtp.safetensors at 0.64GB — the bundled DSpark draft head used for speculative decoding. The whole repo, tokenizers included, is 20.19GB. The card's “about 20GB” is exact. |
| Mixed precision, as trained | The routed experts of layers 0–19 are NVFP4 and layers 20–23 are MXFP8 (the repo config confirms a ModelOpt NVFP4 quantization setup). This is the model's native format, not a post-release squeeze — there is no higher-precision original to download. |
| Multimodal, 1M-token context | Text and image in, text out, via a 23-layer vision encoder. Context window up to 1M tokens (the config's maximum position count is 1,048,576). |
| Licence: Apache 2.0 | Declared on the card — the details, and one honest footnote, are in the licence table below. |
Route one: the 20GB checkpoint on a single card (prices checked Nov 21, 2026)
This is the route the model was built for. The lab lists three supported hardware classes on the card: one NVIDIA DGX Spark, Apple silicon with 32GB or more of memory, and other GPUs with 32GB or more of VRAM recommended. Note the floor is the maker's: the 20.19GB of files would physically squeeze onto a 24GB card, but the lab does not recommend it, and the frameworks' own context caps below show why — what is left after the weights is what your conversations live in.
| Card | Cheapest verified rental | $/hr | $/month | Verified for this model? |
|---|
| RTX 5090 — 32GB | RunPod Community (Secure: $1.39/hr) | $0.69 | $503.70 | Yes — SGLang, single GPU, TP1. Context capped at 65,536 tokens (Balanced) or 32,768 (DSpark) on this card. |
| RTX PRO 6000 — 96GB | RunPod Community (Nebius: $1.80/hr) | $1.69 | $1,233.70 | Yes — SGLang, single GPU, TP1. |
| H100 / H200 — 80GB / 141GB | H100 PCIe, RunPod Community $1.99/hr · H200, RunPod Community $3.59/hr | $1.99 / $3.59 | $1,452.70 / $2,620.70 | Yes — vLLM (both cards; it caps context at 262,144 tokens on H100 to fit memory); SGLang lists the H200. |
| RTX A6000 — 48GB | Thunder Compute | $0.35 | $255.50 | Not listed by vLLM's recipe or SGLang's matrix as of Nov 21, 2026. |
| L40S — 48GB | RunPod Community | $0.79 | $576.70 | Not listed by vLLM's recipe or SGLang's matrix as of Nov 21, 2026. |
Why the 5090 is the headline: it is the cheapest card on our board that clears the lab's 32GB floor and appears in a framework's verified matrix for this specific model. The A6000 at $0.35/hr is cheaper and bigger on paper — but “the lab says 32GB+ GPUs generally” is not the same as “a serving stack has run this looped, NVFP4-native model on Ampere.” Until a framework verifies it, treat the A6000 and L40S rows as an experiment you would be running, not a deployment you are buying.
The memory maths: why context, not weights, sets your card
Worked from the repo's own config, Nov 21, 2026. Inkling 2 Mini keeps 48 KV caches — one per layer visit, because every pass through the looped core keeps its own keys and values (the vLLM recipe and the Transformers documentation both state this).
| The figure | How it is derived |
|---|
| 4,096 bytes per token, per cache | Each cache costs, per token, 8 KV heads × 128 head dimension × 2 (keys and values) = 2,048 values, which at BF16 is 4,096 bytes per token per cache. |
| About 206GB — the impossible bound | If all 48 caches grew to the full sequence, a 1M-token context would need about 206GB of KV cache — impossible, and not what happens: 20 of the 24 stored layers are sliding-window layers whose caches stop growing at the config's 1,024-token window. Only the 4 full-attention layers' caches grow with your context. |
| Roughly 17GB to 35GB for one 1M-token sequence | Depending on how many of those 4 full-attention layers sit inside the looped core (the split is not published), the KV cache for one full 1M-token sequence works out at roughly 17GB to 35GB — arithmetic under a stated assumption, not a measurement. |
| About 11.8GB left on a 32GB card | On the RTX 5090 the weights and draft head take 20.19GB of the 32GB, leaving about 11.8GB for KV cache, activations and the runtime — plan your context around the caps, not around the 1M on the box. |
The operative numbers are the frameworks' own: vLLM fits about 1.8 concurrent 1M-token requests on one GB200 (192GB), caps H100 and DGX Spark at 262,144 tokens, and SGLang caps the 32GB RTX 5090 at 65,536 (32,768 with the draft head on).
Route two: full precision — the BF16 working set
There is no official BF16 checkpoint. The lab ships NVFP4/MXFP8 only, so a “full-precision deployment” means one of two things, and both start from the same arithmetic: 26.4B parameters × 2 bytes = 52.8GB of weights (script-computed from the card's parameter count; a community BF16 conversion measures 52.2GB on disk, which corroborates the derivation).
| The route | What it means |
|---|
| Transformers dequantizes on load | The Transformers documentation states plainly that both checkpoints' NVFP4 and MXFP8 layers are dequantized when loading — so a Transformers deployment carries a ~52.8GB working set even though the download is 20GB. That is the honest way to run it for research, fine-tuning prep and evaluation; it is not the fast serving path (attention runs in eager mode only in this first implementation, contributed to Transformers on Nov 21, 2026). |
| Community GGUF conversions | llama.cpp-format conversions include a BF16 build at 52.2GB — a third-party dequantization, not a lab artifact, and as of today the looped model needs an unmerged llama.cpp pull request (ggml-org/llama.cpp#32695) to load at all. Labelled here as community and unproven for production, because that is what it is. |
Memory consequence: 52.8GB of weights plus KV cache does not fit a 48GB card at all, and fits an 80GB card with roughly 27GB to spare. The cheapest 80GB rentals on our board, checked Nov 21, 2026: A100 80GB at Thunder Compute, $1.09/hr ($795.70/month), and A100 80GB PCIe at RunPod Community, $1.19/hr ($868.70/month). One caution that belongs with those prices: neither vLLM's recipe (Blackwell, Hopper and DGX Spark) nor SGLang's matrix (B300/GB300, B200/GB200, H200, RTX PRO 6000, RTX 5090, DGX Spark, AMD MI350X/MI355X, TPU7x) lists the A100 for this model — the A100 rows are the cheapest memory that fits the BF16 working set, with the framework check still on you. The cheapest full-precision home that a framework has verified is the H100: vLLM-verified, $1.99/hr at RunPod Community ($1,452.70/month), context capped at 262,144 tokens in its recipe. Who this route is for: fine-tuners and evaluators who need the weights at full precision, and teams whose existing stack is Transformers-shaped. Who it is not for: anyone serving traffic. The native checkpoint is the model as trained — serving it dequantized buys you memory bills, not accuracy the lab ever claimed.
Serving: what is documented today (framework pages read Nov 21, 2026)
| Framework | What is documented |
|---|
| vLLM — dedicated recipe | recipes.vllm.ai/thinkingmachines/Inkling-2-Mini. Until support lands in a vLLM release, the recipe specifies the pinned image vllm/vllm-openai:inkling-2-mini. Its documented serve command, with the bundled DSpark speculative decoding enabled: vllm serve thinkingmachines/Inkling-2-Mini --tensor-parallel-size 1 --reasoning-parser inkling --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'. The recipe verifies the model on GB200, B300, GB300, H100, H200 and DGX Spark; TP1 is its recommended shape (“the most efficient way to serve the model”), with the full 1,048,576-token context on one GB200 and a 262,144-token cap on H100 and DGX Spark. Drop the speculative-config line and you have the plain decode setup. |
| SGLang — full cookbook | A full cookbook page with verified launch recipes per platform, using the pinned image lmsysorg/sglang:inkling-2-mini. Its verified matrix is the widest of the three: B300/GB300, B200/GB200, H200, RTX PRO 6000, RTX 5090 and DGX Spark on one GPU each (TP1/EP1), plus AMD MI350X/MI355X and TPU7x images. The recipes pass --reasoning-parser inkling and --tool-call-parser inkling2 explicitly, and --enable-multimodal for image input. On the RTX 5090 its context caps are 65,536 tokens (Balanced) and 32,768 (DSpark). |
| Hugging Face Transformers — the research route | The architecture (Inkling2LoopForConditionalGeneration) was contributed to Transformers on Nov 21, 2026, with official model documentation and a pipeline example: pipeline("image-text-to-text", model="thinkingmachines/Inkling-2-Mini", dtype=torch.bfloat16, device_map="auto"). Proven as a documented route, with the documentation's own caveats attached: the quantized layers are dequantized on load (so budget the ~52.8GB working set, not 20GB), and attention runs in eager mode only. This is the research and integration route, not the throughput route. |
Named honestly — listed, community, or not yet verified
| Route | Status today |
|---|
| llama.cpp / GGUF | Listed by the lab for quantized GGUF versions, but the looped architecture loads only via an unmerged pull request (ggml-org/llama.cpp#32695) as of Nov 21, 2026, and the GGUF files themselves are community conversions. Fine for local experiments; not a production serving route today. |
| TokenSpeed and NVIDIA Dynamo | Named on the lab's card as supported frameworks. We did not verify a model-specific recipe for either first-hand today, so this guide asserts nothing about their setup or hardware. |
| Ollama and MLX builds | The lab lists Ollama on Apple silicon, and community MLX conversions exist — desktop routes, outside this guide's rental scope. |
| Ampere and Ada cards (A100 serving, A6000, L40S, RTX 4090) | No framework verification for this model found today. The 24GB RTX 4090 fails earlier anyway: the lab's floor is 32GB+ and the files alone are 20.19GB. |
The lab's benchmarks, in the lab's own frame
Every number in this table is Thinking Machines' own published table from the model card — the lab's harnesses and settings, not independent results, and not ours. The comparison is against the dense Qwen3.8-27B, the comparison the launch leans on.
| Measure (lab-reported) | Inkling 2 Mini | Qwen3.8-27B |
|---|
| Artificial Analysis Index (v5.0) | 20.1 | 20.1 — a tie |
| SWE-bench Verified (bash only) | 79.2% | 77.8% |
| DeepSWE v1.1 | 48.2% | 42.2% |
| Terminal Bench v2.1 | 71.6% | 73.0% — Qwen ahead |
| AutomationBench-AA v1.0.6 | 41.8% | 48.2% — Qwen ahead |
| MMMU Pro (vision) | 74.9% | 76.3% — Qwen ahead |
The honest summary of the lab's own framing: a 3.4B-active MoE that ties a dense 27B on the lab's headline index and beats it on the lab's coding benches, while trailing it on several agentic, knowledge-work and vision rows of the same table. The card is also unusually plain about the weak spots — multi-turn conversation, factual recall bounded by size, and agents that “may take destructive actions… without confirmation” in the lab's own words. Budget your own evaluation before believing any row, in either direction.
Licence — Apache 2.0, and the contrast is real
| The point | The detail |
|---|
| Apache 2.0, declared on the card | The card declares Apache 2.0, and the repository metadata agrees. Apache 2.0 is the whole licence: use it commercially, serve it, fine-tune it, with no revenue trigger, no user-count trigger and no separate licence to negotiate. Set that beside the Tencent Hy licences examined elsewhere on this site — where reselling inference above a $50M revenue line or 100M monthly users needs a separate licence — and the contrast is exactly as clean as it looks: nothing in the Apache 2.0 text reaches into your business that way. |
| Footnote one: no standalone LICENSE file | As of Nov 21, 2026 the repo's file listing contains no standalone LICENSE file — the licence is declared on the card and in the repo metadata; keep a copy of the card with your deployment records. |
| Footnote two: an Acceptable Use Policy link | The card also links the lab's Model Acceptable Use Policy alongside the licence, as it did for Inkling 2 Small — a policy link to read, not a licence condition we can quote you chapter and verse on here. Neither footnote changes the headline. |
Watch-outs before you rent
| Watch-out | The detail |
|---|
| Looped-model support is days old | The card itself warns that “support for looped models in local runtimes is new, and speed varies by runtime and hardware.” vLLM support lives in a pinned image, not a release; Transformers support was contributed the day this guide was drafted. Pin your images and expect movement. |
| Context on the box is not context on your card | 1M tokens is the model's maximum. The verified caps today: 65,536 on a 32GB 5090 (SGLang), 262,144 on H100 (vLLM), the full 1,048,576 only on a GB200 in vLLM's verified set. Size your card to the context you will actually run. |
| The agent warnings are the lab's, and they are blunt | The card reports repetitive reasoning loops in debugging, tool use that is less reliable than larger models, and possible destructive file actions without confirmation. Its own advice: sandbox agentic workloads, require confirmation before irreversible actions, and cap output tokens and agent turns. |
| Reasoning effort never fully turns off | Setting it to minimum does not disable reasoning (card, limitations) — and with 48 layer visits per token, compute per token is higher than the 3.4B active figure suggests. Do not budget throughput off the active-parameter count alone. |
| Speculative decoding gains are prompt-dependent | vLLM's recipe notes DSpark acceptance depends on prompts and temperature — random-token benchmarks accept almost nothing. Benchmark on text that looks like your traffic. |
| The cheap-card trap | The two cheapest ≥32GB rows on the board (A6000 $0.35/hr, L40S $0.79/hr) are not in any framework's verified matrix for this model today, and both are too small for the dequantized load. Verify before you commit a month to them. |
Who should run this, and how
| Your workload | The route |
|---|
| Coding-agent and tool-use workloads on a budget | The native route on an RTX 5090 at $0.69/hr. That is the deployment the release was designed for — one card, the shipped precision, SGLang's verified recipe, sandboxed per the lab's own advice. |
| Long-document and long-session serving | The RTX PRO 6000 at $1.69/hr. The 96GB is not about the weights; it is about KV cache headroom the 32GB card does not have. |
| Fine-tuning, evaluation, research | The BF16 route on an 80GB card — Transformers dequantizes on load, and the varloop research checkpoint (released alongside, loop passes adjustable from the research repo) exists for exactly this audience. Pay the $1.09–$1.99/hr knowing the A100 rows carry a framework check and the H100 row is the verified one. |
| Production serving at volume | vLLM or SGLang on Hopper or Blackwell, where both frameworks have verified the model — and run your own benchmark first, because every performance claim available today is the maker's. |
How we check
Model facts — 26.4B total / 3.4B active / 5.4B effective parameters, 24 layers with the middle 12 looped three times, 8 of 128 experts, NVFP4 experts in layers 0–19 and MXFP8 in layers 20–23, 1M-token context, the 32GB+ hardware recommendation and the Apache 2.0 declaration — are from Thinking Machines Lab's Hugging Face model card, read first-hand Nov 21, 2026. Checkpoint sizes (19.53GB main file, 0.64GB DSpark draft, 20.19GB total) are the repository's own file sizes via the Hugging Face API, read the same day. Architecture figures used in the KV maths — 8 KV heads, 128 head dimension, 1,024-token sliding window, 48 caches at one per layer visit — are from the repo's config.json and the Transformers model documentation, read Nov 21, 2026; the 20 sliding-window / 4 full-attention layer split is from the vLLM recipe. Serving verification is from the frameworks' own pages read the same day: the vLLM recipe (pinned image, serve command, verified hardware, context caps, ~1.8 concurrent 1M-token requests on GB200, DSpark prompt-dependence), the SGLang cookbook (pinned image, verified matrix including RTX 5090 and RTX PRO 6000 at TP1, parser flags, 5090 context caps), and the Transformers documentation (dequantize-on-load, eager attention, pipeline example, contributed Nov 21, 2026). Prices are GPUJournal's Nov 21, 2026 board snapshot: RTX 5090 RunPod Community $0.69/hr and Secure $1.39/hr; RTX PRO 6000 RunPod Community $1.69/hr and Nebius $1.80/hr; RTX A6000 Thunder Compute $0.35/hr; L40S RunPod Community $0.79/hr; A100 80GB Thunder Compute $1.09/hr and RunPod Community PCIe $1.19/hr; H100 PCIe RunPod Community $1.99/hr; H200 RunPod Community $3.59/hr. Monthly figures are hourly × 730. The BF16 weight figure (52.8GB) is arithmetic — 26.4B × 2 bytes — corroborated by a community BF16 GGUF conversion's measured 52.2GB on disk. The KV range (17–35GB at 1M tokens) is arithmetic under a stated assumption about the unpublished full-attention layer split, presented as a range for that reason. What is deliberately not claimed. No throughput, latency or tokens-per-dollar figure appears here as our finding — we ran no benchmarks, and every benchmark number printed is the lab's own table, labelled as such. No serving route is asserted beyond what the framework's own page documents: TokenSpeed and NVIDIA Dynamo are named as lab-listed only, llama.cpp GGUF is labelled community-plus-unmerged-PR, and Ampere/Ada cards are labelled unverified for this model because neither vLLM's recipe nor SGLang's matrix lists them as of Nov 21, 2026. The proposal-stage estimate of “~80GB” for the BF16 route did not survive the arithmetic and is not used: the derived figure is 52.8GB.
Model Deployment · 2026-11-12 · 15 min
Thinking Machines put the Inkling 2 Small weights on Hugging Face on Nov 12, 2026, under Apache 2.0 — and the first question every builder asks is the rental question. The honest answer starts with size: a 594-billion-parameter model whose own card asks for a GPU cluster with at least 500GB of memory. There is no single-card route. What there is: a cheapest verified way in at $14.36 an hour, a more comfortable one for 44 cents an hour more, and a clear set of people who should not rent at all and should use the APIs the maker lists instead. The model facts come from the maker's own card and repos, read on release day; the prices from our verified board, checked Nov 12, 2026.
Model Deployment · 2026-10-30 · 12 min
The weights are out. Moonshot published Kimi K3.1 Pro on Hugging Face on Nov 4, 2026 — 96 safetensors shards summing to 1,561,293,160,568 bytes (about 1.56TB) in native MXFP4 — under its own Kimi K3.1-Pro License, with vLLM, SGLang and TokenSpeed named on the model card as the serving engines. This guide, first published Oct 30, 2026 as pre-release maths, was refreshed on Nov 6, 2026 with the real file sizes, the real licence and the official recipes: the smallest honest setup is one 8-GPU B300 node, the cheapest node-shaped setup on the current board is 16× H200 at $57.44/hr — and Moonshot’s own API, at $2.00 in / $10.00 out per 1M tokens, still beats self-hosting on price at ordinary volumes.
GPU Pricing · 2026-11-14 · 13 min
On the board we checked on Nov 14, 2026, the cheapest verified on-demand offers in the table below start at $0.34 an hour for an RTX 4090 and top out at $14.27 an hour for a GB300. That range is real, but it is not the whole answer. The lowest H100 SXM rate, TensorDock's $2.25 per GPU per hour, comes with an eight-GPU minimum — an $18.00 hourly commitment, not a $2.25 rental. This guide separates the sticker price from the amount you actually commit to, then adds the costs the hourly rate leaves out.