RAG (Retrieval-Augmented Generation)
Answering with an LLM by first retrieving relevant snippets from your own data and putting them in the prompt. Cheaper and more current than stuffing everything into a huge context window.
What it is
A RAG pipeline: split documents into chunks, embed them, store them in a vector database. At query time, embed the question, retrieve the top-k similar chunks, and paste them into the LLM prompt as context.
Why it matters
RAG keeps the LLM prompt small - a few thousand input tokens instead of an entire corpus - which is usually far cheaper than a giant context window, and lets you update knowledge without retraining.
Cost & infrastructure impact
A RAG stack has four cost lines: LLM API calls, an embedding model, a vector database, and object storage for source docs. The LLM is usually the largest; the vector DB is the one that surprises people.