🖥 What is the true cost of renting a GPU cluster?

What does an AI GPU cluster really cost? A single Blackwell GPU can cost more than a car, and many AI startups spend more on GPUs than their entire human team. For a long time, the industry focused on a simple metric of hourly cost, but when renting thousands of GPUs, the sticker price hides the reality. This calculator breaks down the real cost of GPU clusters to see how much useful work you actually get per dollar.

The sticker price is what everyone looks at first. It is simply the dollars per GPU hour multiplied by the number of GPUs and your target training time. But this assumes a perfect world where the cluster is 100% productive from day one.

As SemiAnalysis notes, two clusters can have the exact same GPU hour price, and yet one of them can still be 10, 15, even 50% more expensive in reality.

Goodput is the amount of useful work the cluster actually performs. Throughput loss from failures, slow checkpoints, and restarts means you have to rent the GPUs for longer than your target hours.

As SemiAnalysis puts it, bad goodput means you are buying expensive GPUs and then throwing away some of the work they produce. Add the initial setup time where engineers are tuning the cluster, but you are already paying for idle GPUs.

A cluster isn't just GPUs. Fast storage, networking, the control plane, and engineering support are not free. Plus, expensive engineering time spent debugging adds significant financial cost.

As SemiAnalysis explains, a GPU sitting idle during debugging is not cheap. You must account for the entire system's reliability because the cheapest GPU is the one that helps you finish the work fastest.