Open Weights Are Free. Serving Them Costs $35 an Hour: Run the Break-Even First

Before you rent a GPU to run an open-weight model, work out one number: the monthly token volume at which the rental is cheaper than the API. For an eight-GPU H200 node on RunPod's published on-demand pricing, that number is roughly 4.2 billion tokens a month if you are replacing Kimi K3's own API, and roughly 46 billion if you are replacing DeepSeek V4 Pro. Below those volumes, self-hosting costs more per token no matter how good your inference stack is, because you are paying for wall-clock hours and billing yourself for idle time. The arithmetic is short and it settles most of these arguments before anyone installs vLLM.

The occasion is Moonshot's Kimi-K3 page on Hugging Face, which as of 27 July 2026 describes an upcoming open release scheduled for that day, billed as "the world's first open 3T-class model". At the time we checked, the page carried no parameter counts, no licence text and no weight files, so everything circulating about file sizes and licence terms is unverified. The pricing, though, is published, and that is the half of the question that decides most projects.

The two prices you are comparing

One side is a per-token price. The other is a per-hour price. Converting between them is the whole exercise.

Methodology for what follows: all prices are the published list rates on the pages above, opened 27 July 2026. We use RunPod's Secure Cloud tier rather than its cheaper Community Cloud because the community tier does not offer the availability guarantees a production endpoint needs. We assume a 3:1 ratio of input to output tokens, which is typical for retrieval-augmented and agentic workloads and unusually generous to the self-hosting case, since output tokens are the expensive ones on every API here.

The arithmetic, ours

Start with the rental. Eight H200s at $4.39 each is $35.12 an hour. That is $842.88 a day and $25,286.40 over a 30-day month, assuming you keep the node running continuously. You need the node running continuously, because a model that takes several minutes to load is not something you spin up per request.

Now convert the API to the same unit. At a 3:1 input to output mix, DeepSeek V4 Pro costs (3 x $0.435 + $0.87) / 4, which is $0.544 per million tokens blended. Divide the monthly rental by that price: $25,286.40 / $0.544 gives about 46,480 million tokens, or roughly 46.5 billion tokens a month. Spread across 720 hours, that is 64.6 million tokens an hour, or about 17,900 tokens per second, sustained, every second of the month.

Almost nobody reading this serves 18,000 tokens per second. That is the point. Against a cheap, well-optimised API, an eight-GPU node loses on cost by an enormous margin.

Run the same calculation against Kimi K3's own API and the picture changes shape. At $3 and $15 per million, the same 3:1 mix blends to $6.00 per million tokens. $25,286.40 / $6.00 is about 4,214 million tokens, roughly 4.2 billion a month, or about 1,625 tokens per second sustained. That is still a serious production load, but it is a load a real product can reach. The gap between the two break-evens is not about hardware at all. It is about which API you are actually replacing.

Two adjustments that move the number a lot

Duty cycle. The rental is billed on wall-clock hours; the API is billed on tokens. If your traffic runs at 30% of peak on average, your effective cost per served token roughly triples, and the break-even volume triples with it. In the DeepSeek comparison that pushes the crossover past 130 billion tokens a month. Any honest self-hosting business case has to state the assumed utilisation, and most do not.

Prompt caching. Kimi K3 charges $0.30 per million for cached input against $3.00 for uncached, a ten-fold difference. If your workload has a large stable system prompt and you achieve an 80% cache hit rate on input, the blended API price falls to about $4.38 per million, and the break-even rises from 4.2 billion tokens to roughly 5.8 billion. Caching is the single cheapest optimisation available and it works against you every time you price a self-hosted alternative. Measure your cache hit rate before you build the spreadsheet, not after.

When self-hosting is right anyway

Cost is not the only variable, and for a minority of teams it is not the deciding one. Four questions separate the real cases from the enthusiasm:

1. Does data leave your boundary? If a contract, a regulator or a client security review prohibits sending payloads to a third-party inference provider, the price comparison is irrelevant. You are buying compliance, and you should say so in the budget line rather than pretending it is cheaper.
2. Do you need the weights pinned? Hosted models change under you. If your evaluations were built against a specific checkpoint and a silent update would break them, holding the weights is worth real money. This is the strongest technical argument for open weights and it has nothing to do with dollars per token.
3. Is latency the product? A dedicated node with no queue can beat a shared endpoint at the tail. If your p99 matters more than your median, measure both before committing, because a badly configured self-hosted node is usually worse on both.
4. Can you keep it running? Somebody has to handle driver updates, OOM crashes at 3am, and the failover path when the host has a bad night. Price that person's time into the monthly figure. Our piece on building a second lane for AI provider outages applies here in reverse: self-hosting does not remove the need for a fallback, it just makes you the provider who might go down.

If none of those four apply, the answer is to keep using an API and spend the effort on the part of the stack that is actually yours. That is the same conclusion we reached about infrastructure generally in choosing technology for a one-person product, and about tooling spend in the AI subscription audit. The cheap alternative to a rented GPU cluster is usually a smaller model on somebody else's cluster: DeepSeek V4 Flash at $0.14 and $0.28 per million is 40 times cheaper than Kimi K3 per token, and for classification, extraction and routing it is very often good enough.

Run your own version

The formula is three lines. Take your monthly token volume, split into input and output. Multiply each by the published API price to get your current monthly spend. Then divide that spend by the hourly rental of the node you would need, and you have the number of hours per month the rental can run before it costs more.

Worked, with numbers a small product might actually have: 400 million input tokens and 130 million output tokens a month against Kimi K3's API is (400 x $3.00) + (130 x $15.00) per million, which is $1,200 plus $1,950, so $3,150 a month. A single H200 at $4.39 an hour costs $3,160 over 720 hours. On paper it looks like a tie. In practice a 3T-class model does not fit on one H200, you would need eight of them, and the honest comparison is $3,150 against $25,286. That is the shape of almost every version of this calculation at small scale, and it is why the open-weights release you are excited about today is best consumed through somebody else's endpoint for a while longer. Keep the weights in your plan as an exit option, and check the arithmetic again when your volume has grown ten times.