Time to first token (TTFT)
How long a user waits between sending a request and seeing the first word appear. Dominated by the prefill phase, so it grows with prompt length.
What it is
TTFT covers queueing, network, and the prefill pass that reads the entire prompt and builds the kv cache. Prefill is parallel and compute-bound, so TTFT scales roughly with the number of input tokens.
Why it matters
For chat and streaming UIs, TTFT is the perceived speed - once tokens start flowing, tokens per second takes over. A long RAG prompt or a huge context can push TTFT from ~0.3s to several seconds.
Cost & infrastructure impact
TTFT is not billed directly, but it is a hidden cost of large context: the compute that makes prefill slow is the same compute you pay for. Trimming the prompt with RAG improves TTFT and the bill together.