MI355X vs B300: inference cost per token, priced from the board

The assumption set — read this before quoting any cost per token

Cost per token is an hourly price divided by a speed. The prices on this page are verified; the speeds are assumptions, chosen once, printed here, and applied identically to both cards so the comparison is fair. They are not benchmark results, and no cost figure on this page should be quoted as one. The workload: a 70B-class open model at FP8 (about 70GB of weights), continuous batching at roughly 32 concurrent sequences, output tokens only — a job both 288GB cards hold easily, with most of their memory left for KV cache. The formula, the same one used across this site: cost per 1M output tokens = hourly price ÷ (assumed tokens/sec × 3,600) × 1,000,000. The assumed speed is set the way our other inference guides set it — roughly in line with memory bandwidth, the thing that limits batched decode. Both makers publish 8 TB/s of peak bandwidth for these cards, the same figure as NVIDIA's B200, which our inference guides anchor at 4,000 tok/s; so both cards are assumed at 4,000 tok/s here. Worked check: $2.95 ÷ (4,000 × 3,600) × 1,000,000 = $0.205, printed as $0.20. Real published benchmarks do exist, and they are reported in this guide attributed to their sources — MLPerf Inference v6.0 (MLCommons, published Apr 1, 2026) put an 8-GPU MI355X node at 100,282 tok/s against 115,730 tok/s for an 8-GPU B300 node on the Llama 2 70B Server scenario, as reported in Spheron's Nov 3, 2026 post from AMD's MLPerf submission — but they were measured on the submitters' own stacks, not on rented machines, so they are context, not inputs to the price maths. The sensitivity is the same as everywhere on this site: if your stack delivers half the assumed speed on both cards, every cost on this page doubles and the order does not change. What changes the order is one card beating its assumption while the other misses — which is why the first rented hour on either card should be a measurement, not a commitment.

The spec sheet — a tie at the top

Sources: AMD's Instinct MI355X product page (amd.com) and NVIDIA's Blackwell Ultra material (nvidia.com and NVIDIA's developer blog), checked Nov 5, 2026. Peak figures are the makers' published spec-sheet numbers, not measured throughput on any provider's machines.

On the two numbers that decide most inference bills — how much a card holds and how fast it streams weights — these chips are identical on paper. The differences sit one layer down: NVIDIA publishes the higher peak FP4 figure (in its own NVFP4 format; AMD's is MXFP4, and cross-vendor FLOPS in different 4-bit formats are not a like-for-like speed test), AMD's card is direct-liquid-cooled only at 1,400W, and the two run entirely different software ecosystems, ROCm against CUDA. None of that is in the hourly rate — which is where this comparison is actually decided.

The B300 rental market, verified

The B300 is rented the way mainstream NVIDIA cards are rented: per card, self-serve, from a spread of providers. Board rows are from our daily check of Nov 5, 2026 (banked snapshot 2026-11-05); Together AI's row was re-read on its own pricing page on Nov 5, 2026. Cost per 1M tokens uses the printed assumed speed of 4,000 tok/s throughout (hourly price ÷ 14.4). Monthly figures elsewhere in this guide use 730 hours.

Same GPU, a 2× spread between the cheapest verified self-serve offer and hyperscaler list — the first thing to get right when you rent a B300. One stated figure for contrast, labelled as stated and never used in our maths: Spheron's own post of Nov 3, 2026 lists B300 on Spheron at $11.52/hr as of Nov 5, 2026 — a rate stated on the seller's own pages, which we report the way we report all Spheron prices: as their claim, not as a verified board price.

The MI355X side: the prices that actually exist

AMD's flagship rents through far fewer doors, and both verified doors sell the same shape: a whole 8-GPU node. Both pages were re-read first-hand on Nov 5, 2026. Cost per 1M tokens uses the same printed assumed speed of 4,000 tok/s (hourly price ÷ 14.4).

Two honest observations. First, there is no single-card MI355X rental on any price list we have verified — if you want this chip, you are buying nodes, and the honest comparison is node against node. Second, the roughly $2.50/GPU/hr MI355X rate behind the year's biggest AMD cost-per-token claims is not a rate any provider publishes: the closest real published figure is TensorWave's $2.95, and it comes with the node and the sales call attached. Spheron, for its part, does not carry the MI355X at all — its Nov 3 post says so — so its comparison is priced from other people's listings and modelled rates, a point it is open about and we simply underline.

The worked maths — and what survives of the 45% claim

Start with the claim this guide answers. The widely shared MI355X cost-per-token advantages — up to about 40–45% better tokens per dollar — are built on modelled hourly rates: SemiAnalysis's InferenceX, the benchmark behind most of them, turns its measured throughput into dollars using its own TCO model rates (about $1.50/hr for MI355X against $2.26/hr for B300, per its chip pages), not rates anyone rents at. Priced instead at verified rental rates, with both cards at the same bandwidth-anchored assumed speed, the picture splits by which door you walk through. At TensorWave's published rate, MI355X tokens cost $0.20 per 1M against $0.51 at the B300's best verified rate — about 60% cheaper per token, a bigger edge than the headlines, but one you can only collect at node scale, through a sales process, on ROCm. Like-for-like on one provider's own list, Oracle prices the MI355X at $8.60 against its own B300 at $15.00 — $0.60 against $1.04 per 1M tokens, about 42% cheaper per GPU-hour for the same 288GB and the same published bandwidth. And where real throughput measurements are paired with real rates, the edge narrows hard: Spheron's Nov 3 post, re-running Kimi K3 at verified on-demand rates, found 13.8 tokens/sec per dollar for MI355X against 13.1 for B300 — a 6% AMD edge, not 45% — because on that workload the B300's measured per-chip throughput is higher. All three statements are true at once; they answer different questions. The node maths changes nothing per token and everything per invoice: an 8-GPU node at the assumed speed produces 32,000 tok/s on either chip, so the per-token figures hold — what changes is the smallest cheque, $23.60/hr at TensorWave against $7.40/hr for one B300 card at Hyperstack. If your workload is one card's worth of serving, the B300 is the only side of this comparison you can buy in that size at all.