Spot vs on-demand GPUs: $4.00/h instead of $7.99/h, if your job survives interruption

Spot capacity is often treated as a small coupon that procurement might negotiate someday. At Verda, it is a separate price list. On 2026-10-18, a B200 SXM cost $4.00/h on spot and $7.99/h on-demand: same GPU model, same provider, same day.

The lower rate comes with a condition: the provider can take the GPU back. If your job can absorb that cheaply, spot is a real saving. If it cannot, the spot price is a failure mode that happens to look cheap.

The price list on 2026-10-18

Verda lists spot and on-demand rates for the same GPU models. Prices as of 2026-10-18, per GPU-hour:

The gap shows up in every row, from the newest Blackwell part down to a V100. Our data records the price, not the terms. Before relying on spot, check Verda's own documentation for how and when interruptions are signalled, what happens to attached or local storage when an instance is reclaimed, how quickly capacity tends to come back, and what hardware configuration each lane gets.

These are Verda's prices only. The same chip can cost very different amounts elsewhere, as we showed in our provider spread analysis. Compare providers before you compare lanes.

Both lanes rose between our two readings

You might expect spot to react differently from on-demand when list prices change. Comparing our 2026-09-16 reading with our 2026-10-18 reading, the percentage increases were the same or nearly the same in both lanes:

B200 SXM rose +26% in both lanes. H100 SXM rose +30% in both.

This compares two snapshots. It says nothing about the path prices took between those dates, it does not establish a pricing formula, and it does not predict the next move. The narrow conclusion: between these two readings, being on spot did not shield anyone from the increase. Choose spot because your workload tolerates interruption, not as a hedge against repricing.

The practical consequence: a B200 SXM spot job budgeted at $3.18/h on 2026-09-16 was priced at $4.00/h on 2026-10-18. Check the per-GPU page when you size a run, not when the invoice arrives.

What spot actually sells you

On spot, you rent the same GPU model at a lower rate and accept that the provider may reclaim it. When that happens, the job on it stops. Depending on the provider's terms, you may also wait before spot capacity is available again.

The discount pays for your tolerance to interruption, and building that tolerance has costs of its own. So the useful comparison is not hourly rate against hourly rate. It is the cost of a finished job in each lane, measured with the same scope on both sides.

Note that a resumable job also benefits on-demand: crashes and node failures happen there too, so some of the checkpointing work is not a spot-only cost. Count only the additional effort spot requires.

Spot wins when the second line comes out lower. A wide hourly gap like $4.00/h against $7.99/h for a B200 SXM leaves room to absorb overhead, but it does not guarantee a saving: a job that is interrupted often and loses a lot each time can end up costing more than on-demand.

One question gets you most of the way:

If this process were killed right now, what would I lose, and how long would it take to get back to where I was?

Jobs that fit spot

These jobs survive interruption in different ways. You need the mechanism that fits your job, not all of them at once.

Fine-tuning with frequent checkpoints. If weights, optimizer state, data-loader position and random seeds are written to durable storage on a regular cadence, and restoring them actually reproduces the run's state, an interruption costs roughly the steps since the last checkpoint plus restart time. That restore path is the part to test, not assume. The interval sets your worst-case loss, and shorter intervals add write overhead, so tune it against how often you are actually interrupted.

Batch inference. Scoring a dataset, generating embeddings or labelling a backlog splits naturally into independent shards. If each shard writes its output and marks itself done, a killed worker only redoes its current shard.

Evaluation sweeps. A benchmark suite across checkpoints or hyperparameter settings is many small, independent jobs. A scheduler that requeues failed items turns an interruption into a delay instead of a lost result.

What they share: progress is persisted somewhere that outlives the instance, and the deadline is measured in hours, not milliseconds.

Jobs that need more care

Single-instance real-time serving. An inference endpoint running on one spot instance goes down when that instance is reclaimed, and users see errors, not a cheaper GPU. Spot can still carry part of a serving fleet if you run enough redundant replicas, have failover that routes traffic away from reclaimed nodes, and your availability target tolerates the reduced capacity while replacements start and load weights. Whether that is acceptable depends on your requirements, not on the price.

Latency-bound APIs with strict targets. Starting a replacement instance and loading weights takes time a synchronous request does not have. If you have an availability or latency commitment, check what each lane actually promises in the provider's terms before deciding which one backs it. On-demand is not reclaimed the way spot is, but it is not automatically an SLA either.

Long runs with no resume path. A training job that only saves at the end, or a pipeline that keeps everything in memory, gets the worst of both lanes. One late interruption throws away every hour before it, and the total bill can exceed what on-demand would have cost.

A common split: keep a baseline of serving capacity on on-demand and send the offline work around it (re-embedding, evals, nightly fine-tunes) to spot.

The LLM parallel: Batch API

If you use LLM APIs, you have seen a similar trade. In The Batch API: half price for anything that can wait, we covered the asynchronous lane that OpenAI, Anthropic, Google, Mistral and several aggregators run; the discount is 50% at the big three. You give up low latency and get a ~50% discount and far higher rate limits. The provider runs your file when it has spare capacity, with a completion deadline that is usually 24 hours.

The economics rhyme: a provider prices capacity lower for customers who accept weaker guarantees. The engineering does not fully carry over.

A workload that suits the Batch API has passed the first test: it can wait. Its self-hosted version only suits spot if it can also resume after being stopped, and that part you build yourself. For more API-side savings, see Six levers to cut your LLM API bill.

Before you switch

• Read the spot terms. Confirm with Verda (or whichever provider you use) how interruptions are signalled, what happens to storage, and how capacity returns.
• Kill it on purpose. Stop a test run manually and confirm it resumes from saved progress, with the state you expect, without intervention.
• Move state off the instance. Checkpoints, outputs and progress markers belong on storage you have confirmed survives a reclaim.
• Price the finished job. Use the formula above with the same scope on both sides.
• Size serving by availability needs. Put serving on spot only with redundancy and failover that meet your target.
• Compare providers, not just lanes. A spot price at one provider can still lose to an on-demand price elsewhere. Check live prices for the B200, H200 and H100.

If you are still deciding whether to rent GPUs at all, start with Is it cheaper to self-host, or just use an API?. Idle hours can matter more than the lane you pick.

Takeaway

At Verda on 2026-10-18, a B200 SXM was $4.00/h on spot against $7.99/h on-demand, and between our 2026-09-16 and 2026-10-18 readings both lanes rose +26%. The spot rate discounts hours, not finished work: decide on the cost of the completed job, and move only the work you have already seen survive a kill.