Self-hosting
Running an open-weight model on GPUs you rent or own, instead of calling a provider API. You trade a per-token bill for a per-hour bill plus engineering effort.
What it is
You pick an open-weight model (Llama, Qwen, Mistral, DeepSeek, gpt-oss), a serving stack (vLLM, TGI, SGLang), and a GPU with enough VRAM - possibly after quantization. You run it on rented cloud GPUs, a dedicated endpoint, or your own machine.
Why it matters
Self-hosting can be much cheaper at high, steady volume, and it gives control over data, latency and fine-tuned weights. It can also be much more expensive at low utilisation, and it adds a real operational burden the per-token math ignores.
Cost & infrastructure impact
Costs the token math leaves out, all favouring the API: engineering time to run inference reliably, redundancy, and the capacity you pay for during traffic troughs. Reserved GPUs cut the hourly rate but raise the utilisation you need.