• October 25, 2026
• 8 min read

LlamaIndex — RAG and Agents over Your Data¶

LlamaIndex (llama-index on PyPI, GitHub run-llama/llama_index) is an MIT-licensed Python framework for applications that answer questions over your own data and act on it. It covers the whole path: readers load files and services into Document objects, node parsers cut them into chunks (TextNode), an index stores the embeddings, a retriever finds chunks, a response synthesizer asks the LLM to answer from them, and agents and Workflows wrap this in tool use and multi-step logic.

This guide covers llama-index-core 0.14.25 (released 21 September 2026) with llama-index-workflows 2.26, on Python 3.10 or newer. The hosted products LlamaParse and LlamaCloud are commercial services and are not covered.

flowchart LR
  F["files / APIs / DBs"] -->|"reader"| D["Document<br/>text + metadata"]
  D -->|"node parser<br/>(IngestionPipeline)"| N["TextNode chunks"]
  N -->|"embed model"| E["embeddings"]
  E --> V[("vector store<br/>VectorStoreIndex")]
  Q["query"] --> R["retriever<br/>top_k + filters"]
  V --> R
  R --> P["node postprocessors<br/>cutoff / rerank"]
  P --> S["response synthesizer<br/>compact / refine / ..."]
  S --> A["Response<br/>text + source_nodes"]

flowchart LR
  U["user message"] --> AG["FunctionAgent<br/>(LLM + tools)"]
  AG -->|"tool call"| T["FunctionTool /<br/>QueryEngineTool / MCP tool"]
  T -->|"tool result"| AG
  AG -->|"can_handoff_to"| AG2["next agent<br/>(AgentWorkflow)"]
  AG2 --> AG
  AG -->|"final text"| O["AgentOutput"]
  T -.->|"InputRequiredEvent"| H["human"]
  H -.->|"HumanResponseEvent"| T

Installation¶

uv add llama-index-core                                 # Document, nodes, indices, engines, agents, Workflows, evaluators, MockLLM
uv add llama-index-llms-openai llama-index-embeddings-openai   # OpenAI LLM and embeddings (the defaults)
uv add llama-index                                      # starter bundle: core + OpenAI LLM + OpenAI embeddings + nltk
uv add llama-index-readers-file                         # PDF, DOCX, PPTX, CSV, XLSX, EPUB, ... for SimpleDirectoryReader
uv add llama-index-vector-stores-chroma                 # or -qdrant, -postgres, ...
uv add llama-index-retrievers-bm25                      # BM25Retriever
uv add llama-index-tools-mcp                            # MCP client tools
uv add llama-index-llms-openai-like llama-index-embeddings-openai-like   # any OpenAI-compatible server
uv add llama-index-llms-ollama llama-index-embeddings-ollama             # local Ollama models
uv add llama-index-embeddings-huggingface               # local sentence-transformers embeddings
uv add llama-index-observability-otel                   # OpenTelemetry export
uv add openinference-instrumentation-llama-index        # OpenInference spans (Phoenix, Langfuse, ...)
uv add llama-index-utils-workflow                       # draw_all_possible_flows
uv add --dev pytest pytest-asyncio

LlamaIndex is split into many packages. Install the smallest set you use.

Section Map¶

When to Use LlamaIndex vs the Alternatives¶

Rule of thumb: LlamaIndex is strongest when the hard part is your data (parsing, chunking, metadata, retrieval quality). When the hard part is agent control flow, state and persistence, LangGraph or Pydantic AI fit better, and LlamaIndex still works as the retrieval layer inside them. Architectural background: Agentic AI Architecture.

Core Ideas¶

Minimal Example¶

from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)               # reads OPENAI_API_KEY
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

documents = SimpleDirectoryReader("data").load_data()                   # one Document per file
index = VectorStoreIndex.from_documents(documents)                      # chunk, embed, store in memory
query_engine = index.as_query_engine(similarity_top_k=3)

response = query_engine.query("How long does express shipping take?")
print(response)                                                         # the answer
for hit in response.source_nodes:                                       # the evidence
    print(hit.node.metadata["file_name"], round(hit.score, 2))

Names from Older Tutorials¶

LlamaIndex has been reorganised several times (package split, then agents rebuilt on Workflows), and many tutorials predate that. In core 0.14.25 these older patterns fail or behave differently:

Quick Rules¶

1. Install per-provider packages — llama-index-core plus the exact LLM, embedding and vector-store packages; pin them in uv.lock.
2. Set Settings.llm and Settings.embed_model once at startup, and pass explicit models in libraries and tests — the lazy default is OpenAI and fails without a key.
3. Give every Document a stable doc_id (filename_as_id=True or your own) — updates, deletes and dedup depend on it.
4. Control what the model sees — metadata is embedded and sent to the LLM unless you exclude it; file_path is included by default.
5. Set chunk_overlap below chunk_size and size chunks for your embedding model, not for the defaults (1024 / 200).
6. Use the same embedding model for indexing and querying, and record its name and dimension next to the index.
7. Persist indexes (storage_context.persist, or a real vector store) — an in-memory index is rebuilt, and re-embedded, on every start.
8. Return source_nodes to callers and test retrieval separately from generation.
9. Use FunctionAgent / AgentWorkflow and Workflow for new code — the old agent classes are gone.
10. Run offline tests with MockLLM, a keyword embedding and a fake server; run evaluators with a real judge on a small golden set (05).

See also¶

• Digital Garden: Knowledge Base
• Python Libraries
• LangChain — LLM Application Framework
• LangChain — RAG & Retrieval
• LangGraph — Stateful Agent Orchestration
• Pydantic AI — Type-Safe Agents
• Ragas — Evaluation for LLM, RAG and Agent Applications
• LiteLLM — One API for 100+ LLM Providers
• DeepEval — LLM Testing Guide
• Agentic AI Architecture