Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

What is llm-mem?

llm-mem is a persistent semantic knowledge index for AI agents. It gives an MCP-compatible AI assistant a real, queryable memory that:

  1. Persists across sessions — memories survive restarts, crashes, and reboots.
  2. Is searchable by meaning — semantic vector search finds relevant memories even when exact keywords differ.
  3. Organizes itself into layers — raw content is automatically distilled into summaries, connections, concepts, and insights.
  4. Stays isolated per bank — separate knowledge bases for separate projects or topics.
  5. Runs locally by default — everything happens on your machine; no network calls required.

The core idea

When you ask an AI assistant to remember something, it normally can’t — once the conversation ends, the context window is gone. The next time you talk to it, you start from zero.

llm-mem solves this by sitting between the AI and a local vector database. The AI calls MCP tools (add_content_memory, query_memory, etc.) to read and write to the database. The database remembers.

┌──────────────┐    stdio/MCP    ┌──────────────┐    SQLite+Arrow    ┌──────────────┐
│  AI client   │ ──────────────▶ │  llm-mem-mcp │ ─────────────────▶ │   LanceDB    │
│ (opencode,   │                 │   (server)   │                    │ (banks/*.lance)
│  Copilot,    │ ◀────────────── │              │ ◀───────────────── │
│  any MCP)    │     results     │              │     vectors       │
└──────────────┘                 └──────────────┘                    └──────────────┘
                                          │
                                          ▼
                                   llama.cpp + fastembed
                                   (local LLM + embeddings)

What’s stored in a memory?

Each memory has:

  • content — the actual text (a fact, a conversation snippet, a document chunk, etc.)
  • context — broad topic tags (e.g. ["project-x", "auth", "jwt"])
  • topics — specific subjects within the content
  • relations — links to other memories ({"relation": "depends_on", "target": "memory_id"})
  • metadata — arbitrary key/value (source URL, file path, line range, etc.)
  • event_at — when the event actually happened (ISO 8601, optional; falls back to created_at)
  • created_at / updated_at — when the memory was stored/last modified (set by the system)
  • An embedding vector — generated by the embedding model for semantic search

The system separates event time (when something happened) from storage time (when you asked the AI to remember it), so “what happened in the last 2 days” doesn’t get confused with “what was stored recently”.

When to use it

Use llm-mem whenever you’d otherwise repeat yourself to an AI:

  • Codebases — store module responsibilities, API contracts, architectural decisions, gotchas. Per-repo banks.
  • Research — store web findings, paper summaries, citations. Per-topic banks.
  • Conversations & decisions — capture meeting outcomes, action items, team decisions.
  • Documents — drop a PDF or markdown file in, it gets chunked, summarized, and indexed automatically.
  • Personal knowledge — preferences, contacts, projects, todos. The more you store, the more useful semantic search becomes.

When not to use it

  • Production-critical data — it’s beta software. Don’t store anything you can’t lose.
  • Real-time low-latency queries — semantic search is fast (sub-second for small banks) but not instant. Don’t expect to handle thousands of queries per second.
  • Massive datasets — the embedded vector store is fine up to ~hundreds of thousands of memories. Beyond that, consider a dedicated vector database.

Next