The Batch API: half price for anything that can wait
Every major LLM provider has an asynchronous batch lane at roughly 50% of the normal per-token price. If a job does not need an answer in seconds, it belongs there.
There is a lane on almost every LLM API that costs half as much and most teams never use it: the batch API. You give up low latency; you get a ~50% discount and far higher rate limits.
How it works
Instead of one request at a time over HTTP, you upload a file - one JSON line per request, thousands of them. The provider runs the file whenever it has spare capacity and returns a results file, with a completion deadline that is usually 24 hours (often much faster in practice).
Same models, same parameters, same output quality. The only difference is you cannot stream the response and you do not control exactly when it runs.
What belongs on it
- Nightly or scheduled summarisation, tagging, extraction
- Bulk classification or moderation of a backlog
- Embedding a whole corpus for a new rag index
- Generating a synthetic dataset for fine tuning
- Running an eval suite across hundreds or thousands of prompts
- Any back-fill or one-off reprocessing job
Roughly: if a human is not waiting on the response right now, it can be a batch job.
The economics
2M classification calls/mo, 600 in + 20 out tokens each
realtime API .......... ~$95 / mo
batch API (−50%) ...... ~$48 / mo
The discount stacks with everything else - a smaller model, prompt caching, a shorter prompt. Batch is the cheapest per-token lane a provider offers.
Availability
OpenAI, Anthropic, Google, Mistral and several aggregators run a batch API; the discount is 50% at the big three. Obolith's pricing table flags batch support per model where the provider publishes it.
Check batch support by model →Figures are illustrative. Confirm current prices on the provider’s own site before deciding.