Local AI Deals

Cheapest box to run Kimi-K2-Instruct at Q6_K

Moonshot’s Kimi-K2-Instruct is 1000B parameters, 32B of them read per token. The number below is computed from that and each vendor’s published specs, not asserted. Switch the quant or the context and it recomputes in place.

Fit finderspecs, not prices
Quant
Context

Kimi-K2-Instruct at Q6_K needs 822GB.

Nothing on the Grid runs Kimi-K2-Instruct at Q6_K today. The largest Config, Mac Studio, M5 Ultra, 512GB, is 318GB short. At it fits with 13GB headroom.

A trillion parameters. No Config we track holds it at any quant, and we list it so the arithmetic says so plainly.

GeForce RTX 5090 is a card, not a computer: every price shown for it is the cheapest listing we found plus $1,000 for the minimum host machine.

memory = 1000B × 0.820 B/param (6.56 bits) + 0.070 GB/1K × 8K K/V + 1.2 GB runtime
speed = bandwidth × 0.5 ÷ (32B active × 0.820 B/param) = 26.2 GB read per token
usable = memory − operating system reserve (8% on a Mac, 6% on a DGX Spark, 4% on a card)

Bandwidth is the vendor’s published figure. Every speed is an estimate and says so. Methodology.