Input tokens
The tokens you send to a model: system prompt, conversation history, retrieved context and the user message. Usually the cheaper half of the bill.
What it is
Everything the model reads before it answers, counted in tokens: the system prompt, the full chat history you resend each turn, any documents you paste in for RAG, and the latest user message. In long conversations the history dominates.
Why it matters
Input is priced separately from output tokens and is normally 3-5x cheaper. It also has a ceiling: the model refuses once input plus output would exceed its context window.
Cost & infrastructure impact
Input volume grows with every turn of a conversation and with every document in a RAG prompt, so it is the part that quietly scales. Prompt caching can cut the cost of the repeated prefix (system prompt, few-shot examples) by up to ~90%.