Free explainer

What Is RAG? Retrieval-Augmented Generation in Plain English

RAG (retrieval-augmented generation) means the AI looks up your documents before it answers. A search step finds the most relevant passages, adds them to the prompt, and the model answers from them with citations, so it makes things up less and stays current without retraining.

Below, watch a working RAG pipeline answer questions in your browser, then see when to use RAG instead of fine-tuning or long context, and what usually goes wrong. No sign-up. (Looking for RAG status, red-amber-green, in project reports? That is a different RAG.)

Watch RAG answer a question, step by step

The AI below has a small help center for an imaginary outdoor store, Trailhead Gear: 6 short documents it was never trained on. Pick a question or type your own, then step through the pipeline.

Step 1 of 5

Chunk: split documents into small pieces

Done once, ahead of time. The 6 documents are split on paragraphs into 12 chunks, so the AI can later pull in just the pieces it needs instead of everything.

  • #1 Returns and refunds

    You can return unused items within 30 days of delivery for a full refund. Refunds go back to your original payment method within 5 business days after we receive the item. Worn or used items can be returned within 30 days for store credit only.

  • #2 Returns and refunds

    Items marked Final Sale cannot be returned. To start a return, open the order in your account and choose Start a return, and we email you a prepaid label.

  • #3 Shipping

    Standard shipping is free on orders over $75 and takes 3 to 5 business days. Express shipping costs $15 and arrives in 1 to 2 business days.

  • #4 Shipping

    We ship to the United States and Canada. Orders placed before 1pm Eastern ship the same day.

  • #5 Warranty

    Every tent and backpack has a lifetime warranty against defects in materials and workmanship. The warranty does not cover normal wear, accidents or damage from misuse, such as a pole that snaps in a fall.

  • #6 Warranty

    To make a warranty claim, email photos of the problem and your order number to support, and we repair or replace the item.

  • #7 Gear rentals

    You can rent tents, sleeping bags and stoves for the weekend. A weekend rental runs from Friday to Monday and costs 20 percent of the retail price.

  • #8 Gear rentals

    Late rentals are charged one extra day for each day late. Cleaning after the trip is included in the rental price.

  • #9 Trailhead Club membership

    Trailhead Club membership costs $30 a year. Members earn 10 percent back on purchases as rewards and get free express shipping.

  • #10 Trailhead Club membership

    You can cancel your membership any time in your account. Unused months of membership are not refunded.

  • #11 Denver store

    Our Denver store is open 9am to 8pm Monday to Saturday and 11am to 5pm on Sunday.

  • #12 Denver store

    Free gear repair clinics run on the first Saturday of every month.

How RAG works: two phases

1. Indexing (once, and on every update)

Documents → split into chunks → each chunk embedded into a vector → stored in a vector database or search index.

2. Answering (every question)

Question → embedded → closest chunks retrieved → added to the prompt → the model writes a cited answer.

Real-world RAG examples

  • Customer support bot

    Answers customers from your help center and policy pages, and links the article it used.

    Retrieves from: Help docs, FAQs, past tickets

  • Internal policy Q&A

    Lets staff ask about HR, travel or security policy without searching a wiki.

    Retrieves from: Handbook, policy PDFs, intranet

  • Codebase assistant

    Finds the files and functions related to a question before suggesting a change.

    Retrieves from: Source code, READMEs, design docs

  • Sales and RFP helper

    Drafts answers to security questionnaires and RFPs from approved past answers.

    Retrieves from: Past RFPs, security docs, product sheets

  • Legal and contract search

    Finds the clause that answers a question across many contracts and cites it.

    Retrieves from: Contracts, case notes, regulations

  • Research and analyst copilot

    Summarizes what reports and transcripts say about a topic, with sources.

    Retrieves from: Reports, earnings calls, papers

RAG vs fine-tuning vs long context vs agents

RAGFine-tuningLong contextAgents with tools
Best forAnswering from your own or fast-changing documentsTeaching a style, format or narrow taskOne-off questions over a few documentsLive data and actions (search, APIs, databases)
Keeps facts currentYes: update the index, no retrainingNo: retrain to change factsYes: paste the latest versionYes: calls live sources
Cites sourcesYes, naturallyNoPossible, if you askYes, if tools return links
Cost per questionLow: only the top chunks go in the promptLow after a training costHigh: every question resends all the textVaries: several model and tool calls
Scales toMillions of documentsWhatever fits in training dataWhat fits in the context windowWhatever the tools can reach
Main riskRetrieval misses the right chunkStill makes up facts, costly to updateSlow, costly, details lost in the middleMore steps that can fail or loop

They combine well: an agent can use RAG as one of its tools, and a long context window lets it read the retrieved documents in full.

Common RAG failure modes and fixes

  • Bad chunking

    Chunks split a table or an answer in half, so no single chunk holds the full answer.

    Fix: Split on headings and paragraphs, overlap chunks slightly, and keep tables whole.

  • Retrieval misses

    The question uses different words than the document, so the right chunk never ranks.

    Fix: Use hybrid search (keywords plus embeddings), add a reranker, or rewrite the query.

  • Stale index

    The policy changed but the old version is still in the index, so the answer is outdated.

    Fix: Re-index on every change and delete old versions; store a last-updated date.

  • Too much or too little context

    One chunk lacks the answer; twenty chunks bury it and raise cost.

    Fix: Tune how many chunks you retrieve and test with real questions.

  • Answer ignores the sources

    The model mixes in its own guesses even though the sources say otherwise.

    Fix: Tell it to answer only from sources, require citations, and allow "I don't know".

  • Leaking private data

    A user retrieves chunks from documents they should not see.

    Fix: Filter retrieval by the user's permissions before anything reaches the prompt.

How to build a basic RAG system

  1. 1. Collect your documents

    Gather the help articles, policies, PDFs or code the AI should answer from, and decide who may see what.

  2. 2. Split them into chunks

    Split on headings and paragraphs so each chunk holds one idea, usually a few hundred words.

  3. 3. Embed and store the chunks

    Run each chunk through an embedding model and store the vectors in a vector database or a search index.

  4. 4. Retrieve at question time

    Embed the user's question, find the closest chunks (often with keyword search and a reranker too) and keep the top few.

  5. 5. Augment the prompt and generate

    Add the chunks to the prompt with instructions to answer only from them and cite sources, then let the model answer.

  6. 6. Test and keep the index fresh

    Check answers against real questions, and re-index whenever documents change.

RAG glossary

Chunk
A small piece of a document, often a paragraph or a few hundred words, that gets indexed and retrieved on its own.
Embedding
A list of numbers that represents the meaning of text, so texts with similar meaning get similar numbers.
Vector database
A database that stores embeddings and quickly finds the ones closest to a question's embedding.
Similarity score
How close two embeddings are, often measured with cosine similarity from 0 (unrelated) to 1 (same).
Top-k
How many of the best-matching chunks are retrieved and added to the prompt.
Reranking
A second, more careful model that reorders retrieved chunks so the most useful come first.
Hybrid search
Combining keyword search (exact words) with embedding search (meaning) for better recall.
Grounding
Making the model's answer depend on supplied sources instead of only on what it learned in training.
Agentic RAG
An AI agent decides when to search, rewrites queries and searches again until it has enough to answer.
Context window
The maximum amount of text a model can read at once, including the prompt and the retrieved chunks.

RAG FAQ

What is RAG in AI?

RAG (retrieval-augmented generation) is a way to make an AI model answer from your own documents. Before the model answers, a search step finds the most relevant passages and adds them to the prompt, so the answer is grounded in those sources and can cite them.

How does RAG work?

Ahead of time, documents are split into chunks, turned into embeddings and stored in a vector database. When a question comes in, it is embedded too, the closest chunks are retrieved, they are added to the prompt with instructions to answer only from them, and the model generates a cited answer.

Is RAG still needed with long context windows?

Often, yes. Long context works well for a few documents, but resending everything with every question is slow and costly, recall can drop for details buried in the middle, and no window holds a whole company's documents. Many systems now combine the two: retrieve the best material, then use a long window to read it in full.

Does ChatGPT use RAG?

In some features. When ChatGPT searches the web or answers from files you upload to a project or a custom GPT, it retrieves relevant passages and adds them to its context, which is RAG. A plain chat with no search and no files answers from training data alone.

What is agentic RAG?

Agentic RAG lets an AI agent decide when and what to search. Instead of one fixed lookup, the agent can rewrite the query, search several sources, check what it found and search again before answering.

What is the difference between RAG and fine-tuning?

RAG gives the model facts at question time by looking them up; fine-tuning changes the model itself by training it on examples. Use RAG for knowledge that changes or must be cited, and fine-tuning for a consistent style, format or narrow skill. They can be combined.

Does RAG stop hallucinations?

It reduces them but does not remove them. If retrieval misses the right passage or the model ignores the sources, it can still make things up. Citations, an "I don't know" instruction and testing with real questions help.

Who invented RAG?

The term comes from a 2020 paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", by Patrick Lewis and colleagues at Facebook AI Research (now Meta AI), University College London and New York University.

Checked 2026-11-15. Source for the term: Lewis et al., 2020. Other meanings of RAG include red-amber-green status ratings and a cloth rag.

Run retrieval-backed agents as repeatable workflows

Tornic turns the Claude Code, Codex or Cursor CLI you already pay for into deterministic multi-step workflows, so an agent that searches your docs and answers runs the same way every time.

Explore Tornic

Related free tools