API vs self-hosting
The core infrastructure decision for anyone running LLMs at scale: pay per token to a provider, or pay per hour for GPUs you operate. The answer turns on volume, utilisation and how much control you need.
What it is
| API | Self-hosting | |
| Billing | per token | per GPU-hour |
| Idle cost | zero | you pay for it |
| Ops effort | near zero | significant |
| Best when | variable / low / mixed volume | high, steady, predictable volume |
Why it matters
Getting this wrong is expensive in both directions: a dedicated GPU for a bursty app burns money on idle time; a per-token bill for a steady high-volume service can be 3-5x what self-hosting would cost. Find your break even point before committing.