Deploy Inkling 2 Small — what it actually costs

What Inkling 2 Small is

From the maker's own card, read Nov 12, 2026: a multimodal mixture-of-experts model — text and image in, text out — with 594 billion parameters in total and 17 billion active per token. A 48-layer decoder-only transformer; each token routed to 8 of 384 experts; local and global attention layers; a vision transformer for image input; a context window of 1 million tokens. It launched closed on Thinking Machines' Tinker platform on Nov 5, 2026; the weights landed on Hugging Face on Nov 12, 2026, under Apache 2.0, with day-one integration in SGLang, vLLM, TokenSpeed and Unsloth. Two checkpoints ship: the native NVFP4 build — the precision the model was trained in, and the one the card recommends — and an MXFP8 build. Full record: the model's catalogue entry on this site.

The memory maths

An MoE model carries a trap in its headline numbers. The 17B active figure is not the size of the download, the memory, or the rental. Every one of the 594 billion parameters sits in GPU memory whether a given token uses it or not — the active count shapes the compute per token; the total decides what you rent. Anyone who prices this as a “17B model” is about to be surprised by a cluster bill.

Why the NVFP4 file is bigger than the naive maths (594B × 0.5 bytes ≈ 297GB): it is a mixed build, not a uniform one. The repo's own metadata shows about 248GB of FP4-packed weights alongside about 128GB in FP8 and about 14GB in BF16 — and the vLLM recipe linked from the card explains the split: experts are NVFP4 in layers 0–39 and MXFP8 in layers 40–47. Four-bit is the bulk, not the whole. The context window deserves the same honesty. “1M tokens” is the model's ceiling, not a promise about what a 500GB cluster serves at full length to a crowd. The working, from the model's published config: 8 of its 48 layers attend globally (the other 40 use a 1,024-token sliding window, per the vLLM recipe), and a global layer's KV cache at BF16 costs 2 × 8 KV heads × 128 head-dim × 2 bytes = 4,096 bytes per token — about 32KB per token across the 8 global layers, so one sequence at the full 1M context carries roughly 33GB of KV cache alone (labelled assumptions: BF16 KV cache, per sequence; sliding-window layers excluded — their cache stops growing at the window). Against about 106GB of headroom, that is a handful of full-length sequences, not a crowd — the recipes set the context and batch you actually get. vLLM's verified configurations agree: on GB200 trays it verifies up to 262,144 tokens at TP4 and reserves the full 1,048,576 for TP8.

The configs, priced

The card names exactly two example configurations for NVFP4. Everything below that goes beyond them is labelled as what it is — memory arithmetic against the card's floor, not a maker's promise — and the serving recipes are the final arbiter of what runs well.

Two lessons in that table. First, the card's two examples land within 44 cents an hour of each other by different shapes — four mid cards against two big ones — so the choice is about headroom, not price: the B300 pair leaves about 91GB per card after the checkpoint loads, the H200 four about 43GB, and that difference is your room for KV cache and batch. Second, the cheap B200 rescues nobody: at $5.98 it is the board's best B200 rate, but clearing the floor takes three, and three cost more than either card example. A cheaper card you need more of is not a cheaper cluster. The minimums that bite. Some per-GPU rates on the board are divisions of whole machines. Oracle OCI's H200 ($10.00/GPU/hr) is an 8-GPU node at $80.00 an hour, take all eight or none; Azure's H200 ($10.60/GPU/hr) is an 8-GPU VM at $84.80 an hour; CoreWeave's ($6.31/GPU/hr) is an 8-GPU node at $50.44 an hour — a real route (1,128GB clears either checkpoint's floor), but priced as a node, because that is what you rent. Compare these node totals against the $14–18/hr shapes above, not the divided rates. The rest of the board, honestly. GB300 is listed per GPU-hour (Verda, $14.00), so two would cost $28.00/hr — double the B300 route, for a rack-scale part the card does not name; Oracle sells it only as a 4-GPU node at $72.00 an hour. The GH200 goes the other way: CoreWeave lists it at $6.50 as a 1-GPU instance with 96GB, so the floor takes six separate instances ($39.00/hr) before interconnect questions start. And no H100 configuration appears in this guide: the card's hardware story starts at the H200 and the B300, and we do not invent routes the maker has not documented. MXFP8's table is shorter still — 8× H200 at RunPod Community ($28.72/hr) or 4× B300 at Hyperstack ($29.60/hr) — because its honest use in a cost guide is as the priced warning in the watch-outs below.

The serving route, as documented

The software story is unusually well documented for a day-one release, because the maker shipped the integrations with the weights. The card names four local inference routes, each with its own recipe — SGLang, vLLM, TokenSpeed and Unsloth — plus Hugging Face's own stack, and three post-training routes: Miles, SkyRL and Snowflake Cortex Training. What the documentation also says, before your first invoice:

What it costs over real stretches of time

Assumptions, stated plainly: on-demand rates checked Nov 12, 2026; GPU-hours only — storage, downloads, egress and your own serving work excluded, because they vary by provider and are not part of the card rate. We publish no throughput figures for this model, so nothing here is priced per token; these are the costs of holding the cluster, which is the decision a renter actually signs.

Read the shape, not just the numbers. A serious evaluation — a day, a weekend — costs a few hundred dollars and is a perfectly sane rental. The commitment gets heavy only when the cluster never sleeps: five figures a month, every month, whether or not anyone is asking it questions. That asymmetry decides the next section.

Who should not self-host

Anyone whose usage is occasional, bursty, or still being figured out. The rental floor is about $345 a day before your first token, and it runs while you sleep. The same weights are served by API at NVIDIA, Fireworks, Together, Baseten, Nebius, Modal and Cloudflare, and the maker runs them on its own Tinker platform and playground — all named on the card itself. We hold no verified per-token API price for this model as of Nov 12, 2026, so we print no break-even and invent none: price your expected token volume at those APIs against roughly $10,500 a month for an always-on cluster, and notice how much round-the-clock traffic it takes before renting wins.

Watch-outs