Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

llm-mem is a memory server for AI agents. It gives your AI assistant persistent, searchable memory that works across sessions — conversations, codebases, documents, and anything else you want it to remember.

It’s a single self-contained binary written in Rust. No databases to install, no cloud services required, no API keys needed by default. Everything runs locally on your machine, and data never leaves your computer.

Warning

Beta software —

This is beta software. Expect rough edges and breaking changes. Use it to try things out, not for anything important.

How it works

When connected to an AI assistant via the Model Context Protocol (MCP), llm-mem exposes a set of tools the assistant can call to:

  • Store raw text, documents, or AI-extracted facts as memories
  • Search by meaning (semantic search) or by keyword
  • Browse chronologically with timeline tools
  • Navigate a layered pyramid from raw content up to abstract insights
  • Manage isolated memory banks, backups, and document uploads

The assistant calls these tools just like any other MCP tool — no special prompting required.

What makes it different

Featurellm-mem
Local-firstRuns entirely on your machine by default. No cloud, no accounts, no data leaves your box.
Embedded vector storeUses LanceDB embedded. No separate database server.
Layered memoryL0 raw → L1 summaries → L2 connections → L3 concepts → L4 insights, with auto-abstraction.
AI in-processBundled llama.cpp for local LLM, fastembed for local embeddings. Zero-config.
MCP-nativeSpeaks MCP stdio. Drop into opencode, VS Code Copilot, or any MCP client.
Standalone CLIIncludes an interactive REPL and single-command mode for shell scripting.

Next steps

What is llm-mem?

llm-mem is a persistent semantic knowledge index for AI agents. It gives an MCP-compatible AI assistant a real, queryable memory that:

  1. Persists across sessions — memories survive restarts, crashes, and reboots.
  2. Is searchable by meaning — semantic vector search finds relevant memories even when exact keywords differ.
  3. Organizes itself into layers — raw content is automatically distilled into summaries, connections, concepts, and insights.
  4. Stays isolated per bank — separate knowledge bases for separate projects or topics.
  5. Runs locally by default — everything happens on your machine; no network calls required.

The core idea

When you ask an AI assistant to remember something, it normally can’t — once the conversation ends, the context window is gone. The next time you talk to it, you start from zero.

llm-mem solves this by sitting between the AI and a local vector database. The AI calls MCP tools (add_content_memory, query_memory, etc.) to read and write to the database. The database remembers.

┌──────────────┐    stdio/MCP    ┌──────────────┐    SQLite+Arrow    ┌──────────────┐
│  AI client   │ ──────────────▶ │  llm-mem-mcp │ ─────────────────▶ │   LanceDB    │
│ (opencode,   │                 │   (server)   │                    │ (banks/*.lance)
│  Copilot,    │ ◀────────────── │              │ ◀───────────────── │
│  any MCP)    │     results     │              │     vectors       │
└──────────────┘                 └──────────────┘                    └──────────────┘
                                          │
                                          ▼
                                   llama.cpp + fastembed
                                   (local LLM + embeddings)

What’s stored in a memory?

Each memory has:

  • content — the actual text (a fact, a conversation snippet, a document chunk, etc.)
  • context — broad topic tags (e.g. ["project-x", "auth", "jwt"])
  • topics — specific subjects within the content
  • relations — links to other memories ({"relation": "depends_on", "target": "memory_id"})
  • metadata — arbitrary key/value (source URL, file path, line range, etc.)
  • event_at — when the event actually happened (ISO 8601, optional; falls back to created_at)
  • created_at / updated_at — when the memory was stored/last modified (set by the system)
  • An embedding vector — generated by the embedding model for semantic search

The system separates event time (when something happened) from storage time (when you asked the AI to remember it), so “what happened in the last 2 days” doesn’t get confused with “what was stored recently”.

When to use it

Use llm-mem whenever you’d otherwise repeat yourself to an AI:

  • Codebases — store module responsibilities, API contracts, architectural decisions, gotchas. Per-repo banks.
  • Research — store web findings, paper summaries, citations. Per-topic banks.
  • Conversations & decisions — capture meeting outcomes, action items, team decisions.
  • Documents — drop a PDF or markdown file in, it gets chunked, summarized, and indexed automatically.
  • Personal knowledge — preferences, contacts, projects, todos. The more you store, the more useful semantic search becomes.

When not to use it

  • Production-critical data — it’s beta software. Don’t store anything you can’t lose.
  • Real-time low-latency queries — semantic search is fast (sub-second for small banks) but not instant. Don’t expect to handle thousands of queries per second.
  • Massive datasets — the embedded vector store is fine up to ~hundreds of thousands of memories. Beyond that, consider a dedicated vector database.

Next

The memory pyramid

llm-mem doesn’t just dump everything into a flat pile. It organizes memories in layers, similar to how human memory works — from concrete facts up to abstract understanding.

LayerNameWhat it is
L0Raw contentYour conversations, notes, and documents as-is
L1SummariesAI-generated summaries and structural outlines
L2ConnectionsCross-document links and thematic relationships
L3ConceptsDomain principles and theories
L4InsightsMental models, paradigms, and high-level patterns

Background workers automatically create higher layers over time. You can search at any layer, zoom in to see sources of a concept, or zoom out to see what abstractions were built from something you stored.

Why layers?

Flat search works for keyword matching but fails on questions like “What pattern connects these accidental discoveries?”. Pyramid search across layers can synthesize the answer.

The pyramid is also how human memory works:

Human cognitionllm-mem layerExample
Sensory inputL0 raw“Meeting at 3pm, Alice said PostgreSQL, see #arch-2026-02-15”
EpisodicL1 summary“Feb 15 architecture meeting”
SemanticL2 connection“PostgreSQL chosen over MongoDB for analytics service”
ConceptualL3 concept“Relational vs. document databases for join-heavy workloads”
WisdomL4 insight“Schema-first design reduces migration cost at scale”

The L0 content is verbatim — exact original phrasing, file paths, line numbers, timestamps. Higher layers compress and connect, but you can always zoom back to the source.

How layers are created

L0 — Raw content: created by you (or your AI assistant) when you call add_content_memory, store_memories, or upload_document. These are immutable entries.

L1 — Summaries: created automatically by a background worker that watches L0 content. When enough L0 content accumulates (controlled by auto_summary_threshold), the worker picks related memories and asks the LLM to produce a summary.

L2 — Connections: created by another background worker that looks at L1 summaries, finds semantic overlap, and emits related_to, references, extends, or contradicts relations.

L3 — Concepts: created by an L2→L3 worker that clusters L2 connections into domain concepts.

L4 — Insights: created by an L3→L4 worker that finds cross-domain patterns and abstracts them into principles.

You don’t manage this pipeline. It runs in the background and adds higher layers as material accumulates. You can query at any layer, force a specific layer to be built (create_abstraction), or pause the pipeline (stop_abstraction_pipeline / start_abstraction_pipeline).

The navigate_memory tool moves through the pyramid:

  • zoom out from a concrete L0 memory → see what L1+ insights it contributed to
  • zoom in from an abstract L3 concept → see the L0 source evidence
  • search at layer → restrict results to a specific abstraction level

This is the killer feature for research workflows. Ask “what did we decide about auth in February?” and you get raw meeting notes. Ask “what’s our auth architecture?” and you get a synthesized concept with source links.

The two timestamps

Every memory carries two distinct timestamps:

  • event_at — when the event actually happened (caller-supplied for L0; derived as a range from source memories for L1+)
  • created_at — when the memory was stored (set automatically)

This separation lets you answer “what happened in the last two days” without confusing it with “what was stored recently”. Old memories without an explicit event_at fall back to created_at so the timeline stays complete.

Next

Ready to run it? See What you need and then Installation.

What you need

The requirements depend on which build you choose and which AI backend you use.

For a default (fully-local) build

  • Rust 1.85+ (2024 edition) — only required if building from source via cargo
  • A C/C++ compiler and CMake — needed to compile the bundled llama.cpp and fastembed native code
  • ~3 GB of free disk space — for the language model (~2.5 GB) and embedding model (~90 MB), downloaded on first run
  • ~4 GB of RAM — the LLM lives in memory once loaded

If you don’t want to compile a local AI engine, see the API-only build below.

For an API-only build

  • Rust 1.85+ — only required if building from source
  • An API key for an OpenAI-compatible endpoint (OpenAI, OpenRouter, Anthropic, Ollama, llama-server, etc.)
  • No C/C++ toolchain — native inference is disabled in this build

For the AppImage

  • Linux with glibc ≥ 2.28 — Ubuntu 20.04+, Debian 11+, RHEL 9+, Fedora, Arch, openSUSE
  • libfuse2 (Debian/Ubuntu) or fuse (Fedora) — usually pre-installed. Only needed if you don’t set APPIMAGE_EXTRACT_AND_RUN=1
  • No compiler toolchain — fully self-contained
  • ~3 GB disk for models on first run
PlatformGPU supportBuild flag
macOS (Apple Silicon)M1/M2/M3--features metal
Linux/Windows (AMD, Intel, NVIDIA)Vulkan--features vulkan
Linux/Windows (NVIDIA only)CUDA--features cuda

GPU acceleration makes the LLM substantially faster (often 5-20x) but is not required.

For document parsing

Document parsing uses pure-Rust libraries by default. No system dependencies required for:

  • Markdown, plain text, CSV, TSV, JSON, JSONL, YAML, TOML, XML
  • PDF (via lopdf)
  • DOCX, XLSX, XLS (via calamine)
  • ZIP archives
  • PNG, JPEG, GIF, WebP images (descriptions via local LLM if vision_enabled = true)

MCP client requirements

The MCP server (llm-mem-mcp) speaks stdio and is compatible with any MCP client. Confirmed working with:

  • opencode (recommended for local development)
  • VS Code Copilot (with the MCP extension)
  • Any other MCP-compatible client (Claude Code, Cursor, etc.)

Next

Ready to install? See Installation.

Installation

There are five ways to install llm-mem, listed from least to most effort:

  1. Pre-built AppImage (Linux) — single file, no install, no compiler
  2. Pre-built native binary (Linux/macOS/Windows) — download from Releases
  3. cargo install — Rust toolchain required
  4. Build from source — for contributors / custom builds
  5. Docker build — for the AppImage itself (advanced)

For most users, option 1 or 2 is the right choice.

A single-file AppImage is available that works on every modern Linux distribution without glibc version issues. It bundles both binaries (llm-mem-mcp and llm-mem) in one file, dispatching by argv[0] (multicall pattern). Includes LanceDB, the Vulkan loader, and a bundled OpenSSL.

# 1. Download from the GitHub Releases page
#    https://github.com/vsndev3/llm-mem/releases
#    pick: llm-mem-mcp-x86_64.AppImage   (or -aarch64)

# 2. Make it executable
chmod +x llm-mem-mcp-x86_64.AppImage

# 3. Run the MCP server (default when invoked as llm-mem-mcp)
./llm-mem-mcp-x86_64.AppImage --help

# 4. (Optional) Use the CLI via a symlink
ln -s "$(pwd)/llm-mem-mcp-x86_64.AppImage" ~/bin/llm-mem
llm-mem --help
llm-mem --single search --query "vegan recipes"

The first run downloads the AI models it needs:

  • A language model: Gemma 4 E2B (gemma-4-E2B-it-Q8_0.gguf, ~2.5 GB)
  • An embedding model: all-MiniLM-L6-v2 (~90 MB)

Downloads resume if interrupted. HTTP/HTTPS/SOCKS proxies are respected via HTTPS_PROXY / HTTP_PROXY / ALL_PROXY environment variables.

If FUSE isn’t available on your system (e.g. some container environments), set:

export APPIMAGE_EXTRACT_AND_RUN=1

Note

The AppImage downloads a separate copy of the models per invocation. To share models across runs, set LLM_MEM_MODELS_DIR to a stable path.

Option 2: Pre-built native binary (Linux, macOS, Windows)

Native binaries are built for the host platform’s typical glibc/libc version. They don’t bundle a C++ runtime or OpenSSL, so they rely on system libraries.

ArchivePlatformArchitectureGPU
llm-mem-mcp-linux-x86_64.tar.gzLinux (Ubuntu 24.04+)x86_64Vulkan
llm-mem-mcp-linux-aarch64.tar.gzLinux ARM64aarch64Vulkan
llm-mem-mcp-macos-x86_64.tar.gzmacOS 15+ (Intel)x86_64Metal
llm-mem-mcp-macos-aarch64.tar.gzmacOS 15+ (Apple Silicon)arm64Metal
llm-mem-mcp-windows-x86_64.zipWindows 10/11x86_64Vulkan

Install:

# Linux / macOS
tar xzf llm-mem-mcp-linux-x86_64.tar.gz
sudo mv llm-mem-mcp /usr/local/bin/

# Windows (PowerShell)
Expand-Archive llm-mem-mcp-windows-x86_64.zip
move llm-mem-mcp.exe C:\Windows\System32\

On first run, the same auto-download kicks in. On Windows, you may need to install Microsoft Visual C++ Redistributable for the local LLM engine.

Option 3: cargo install

If you have a Rust toolchain installed and want the latest from crates.io (when published):

cargo install llm-mem --features local

This requires a C/C++ compiler and CMake on the system. Build time is ~10-20 minutes the first time (compile llama.cpp + fastembed native code).

Option 4: Build from source

For contributors, custom builds, or to get unreleased features:

git clone https://github.com/vsndev3/llm-mem.git
cd llm-mem

# Default build: local LLM (llama.cpp) + local embeddings (fastembed) + LanceDB
cargo build --release

# Artifacts land in target/release/:
#   llm-mem-mcp  — MCP server
#   llm-mem     — standalone CLI

Build feature combinations

CommandWhat you get
cargo build --releaseDefault: local LLM + local embeddings + LanceDB (fully offline)
cargo build --release --no-default-features --features lancedbAPI-only, no local AI
cargo build --release --no-default-features --features lancedb,local-embedAPI LLM + local embeddings
cargo build --release --features local,vulkanLocal + Vulkan GPU
cargo build --release --features local,metalLocal + Metal GPU (macOS)
cargo build --release --features local,cudaLocal + CUDA GPU (NVIDIA)
cargo build --release --no-default-features --features lancedb,vector-liteAPI + VectorLite (legacy)

See GPU acceleration for details on the GPU backends.

Option 5: Build the AppImage yourself

If you want to produce an AppImage for a custom feature combination or to test changes:

git clone https://github.com/vsndev3/llm-mem.git
cd llm-mem

# Local Docker build (auto-detects docker or podman)
scripts/build_appimage.sh

# Native build (no Docker) — requires gcc 10+, cmake, appimagetool
scripts/build_appimage.sh --no-docker

# Build for ARM64 from an x86_64 host
scripts/build_appimage.sh --arch aarch64

# Build with zsync auto-update metadata
scripts/build_appimage.sh --update-info 'gh-releases-zsync|vsndev3|llm-mem|latest|llm-mem-mcp-x86_64.AppImage.zsync'

Output: dist/llm-mem-mcp-x86_64.AppImage (or -aarch64) plus a .zsync delta-update file.

The CI workflow .github/workflows/appimage.yml is manual-trigger only — it produces x86_64 and aarch64 artifacts in the Actions UI without auto-publishing a release.

Verifying the install

# MCP server
llm-mem-mcp --version
llm-mem-mcp --help

# CLI
llm-mem --help

Both binaries should print their version and exit. If the binary is missing or won’t run, see Troubleshooting.

Next

  • First run — what happens on the very first invocation
  • Quickstart — get from install to working memory in 10 minutes

First run

What happens when you start llm-mem-mcp for the very first time.

Step 1: Config discovery

The server searches for a config file in this order:

  1. --config <path> CLI flag (explicit)
  2. ./config.toml (current working directory)
  3. ~/.config/llm-mem/config.toml (XDG config dir)
  4. /etc/llm-mem/config.toml (system-wide)

If none of these exist, the server uses all defaults and applies any environment variable overrides (see Environment variables).

Tip

Generate a template —

To see every available setting with comments, run:

bash llm-mem-mcp –generate-config > config.toml


Edit the file, then start the server with `llm-mem-mcp --config config.toml`.

Step 2: Model download (local backend)

If you configured provider = "local" (the default), the server downloads:

  • Language model: gemma-4-E2B-it-Q8_0.gguf (~2.5 GB) from HuggingFace
  • Embedding model: all-MiniLM-L6-v2 (~90 MB) from fastembed’s model registry

Models are stored under the directory specified by llm.models_dir (default: llm-mem-data/models/) and cached separately under ~/.cache/llm-mem/models/ (configurable via llm.cache_dir).

Downloads are resumable — if interrupted, just restart the server. A smaller alternative model (smollm2-1.7b-instruct-q4_k_m.gguf, ~1 GB) is also available but requires a larger context size.

Note

Manual model download —

If you’re behind a corporate proxy or on an air-gapped machine, see Logging & debugging for instructions on downloading the models manually.

Step 3: Bank creation

The default memory bank is created automatically on first run. The bank lives at:

<llm.banks_dir>/default.lance/
<llm.banks_dir>/default.sessions.db

Default llm.banks_dir: ./llm-mem-data/banks/. See Data directory layout for the full schema.

Step 4: Service startup

Once models are loaded and the bank is ready, the service:

  1. Starts the abstraction pipeline (background L0→L1, L1→L2, … workers)
  2. Resumes any document upload sessions that were interrupted
  3. Listens on stdio for MCP requests

You’ll see log output on stderr (or in the log file if logging.enabled = true):

INFO  Starting LLM Memory MCP Server
INFO  Configuration loaded from: None
INFO  Logging to directory: ./llm-mem-data/logs
INFO  Initialized LLM client (backend: Local)
INFO  Initialized memory bank manager
INFO  Default memory bank loaded
INFO  Abstraction pipeline initialization requested
INFO  MCP server initialized successfully

Step 5: Connect a client

The server is now waiting on stdio for MCP traffic. Connect it to your AI client:

Stopping the server

The server handles SIGINT (Ctrl+C) and SIGTERM gracefully:

  • Stops the MCP listener
  • Stops the abstraction pipeline
  • Flushes any in-flight writes
  • Releases the llama.cpp backend

No special shutdown command is required.

Next

Quickstart — run your first query in 5 minutes.

Quickstart

A 10-minute tour: install, connect to an AI assistant, and ask it to remember something.

0. Prerequisites

  • llm-mem installed (see Installation)
  • An MCP client. We’ll use opencode here, but VS Code Copilot works too.

1. Start the server in MCP mode

# Either run the MCP server directly, or use the AppImage multicall
llm-mem-mcp

The server will:

  • Look for a config file (or use defaults)
  • Download models on first run (~3 GB total, resumable)
  • Start listening on stdio

Leave this running, or background it:

# It runs as a foreground process — your MCP client launches it on demand.
# For manual testing:
llm-mem-mcp

Note

You don’t usually run the MCP server by hand — the MCP client launches it. The above is for sanity-checking that it starts and prints its banner.

2. Connect opencode

In your project directory (or globally), create opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/path/to/llm-mem-mcp-x86_64.AppImage"],
      "enabled": true
    }
  }
}

Or if you built from source with cargo build --release:

{
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/absolute/path/to/llm-mem/target/release/llm-mem-mcp"],
      "enabled": true
    }
  }
}

Start (or restart) opencode. It discovers the MCP server and lists the 32 llm-mem tools in its tool palette.

3. Tell the AI to remember something

Open opencode and chat. Try:

Use the llm-mem tools to remember that the AuthService in this project handles JWT token validation and refresh — entry point is src/auth/service.rs:45-80. The refresh tokens expire after 7 days.

The AI will call add_content_memory (or add_intuitive_memory if it wants the LLM to extract structured facts).

4. Ask it back later

In a fresh conversation (or after restarting the AI):

Where is JWT validation handled, and what’s the refresh token expiry?

The AI will call query_memory or search_memory and return the stored fact with the source attribution.

The search works by meaning, not just keywords. Try:

Find anything I stored about user authentication

Even if you stored “AuthService handles JWT token validation”, the search will find it because the embedding captures the meaning of “user authentication”.

6. Add a document

Upload the file docs/architecture.md to memory.

The AI calls upload_document, which:

  1. Chunks the file
  2. Embeds each chunk
  3. Stores them as L0 memories
  4. Triggers the L0→L1 worker to create summaries in the background

Later queries about the document return the relevant chunks.

7. Look at the timeline

Show me what I stored in the last 2 days, grouped by day.

The AI calls get_timeline with the --since 2d equivalent. You get a chronological list grouped by day.

8. Navigate the pyramid

Zoom out from the JWT memory — what higher-level insights were built from it?

The AI calls navigate_memory with zoom_out. You get the L1/L2/L3 memories that were synthesized from the original L0 entry.

What just happened

You went from zero to a working AI memory in 10 minutes. The same patterns work for:

  • Storing entire codebases (one bank per repo, upload_document for each file)
  • Research projects (one bank per topic, semantic search to find related work)
  • Personal notes (one bank, multiple event_at-tagged entries, timeline views)
  • Meeting logs (one bank, intuitive memories for each decision)

Next

MCP clients overview

llm-mem is an MCP server. It speaks the Model Context Protocol over stdio (standard input/output) and registers 32 tools that the AI assistant calls.

Any MCP client can connect to it. The configuration is a JSON object that tells the client how to launch the server process.

General shape

Every MCP client config has the same essential fields:

{
  "<mcp-key>": {
    "type": "local",
    "command": "/absolute/path/to/llm-mem-mcp",
    "args": [],
    "env": {}
  }
}
FieldMeaning
type: "local"The server runs as a local subprocess (the only kind llm-mem supports)
commandAbsolute path to the llm-mem-mcp binary (or AppImage)
argsCommand-line arguments (optional — see CLI flags)
envEnvironment variables to set when launching (optional)

The command must be absolute. The client launches it as a child process and pipes JSON-RPC over its stdin/stdout.

Confirmed clients

ClientStatusConfig keyDocs
opencodeRecommendedmcp.<name> in opencode.jsonopencode docs
VS Code CopilotSupportedservers.<name> in .vscode/mcp.jsonVS Code MCP docs
Claude CodeCompatiblemcpServers in .mcp.jsonAnthropic docs
CursorCompatiblemcpServers in ~/.cursor/mcp.jsonCursor docs
ZedCompatiblecontext_servers in ~/.config/zed/settings.jsonZed docs

The configuration format varies by client. The command and behavior are the same.

Choosing between AppImage and direct binary

AppImage (llm-mem-mcp-x86_64.AppImage):

  • One file, works on any glibc ≥ 2.28 distro
  • Bundled OpenSSL + C++ runtime
  • Larger download (~150 MB) but zero install
  • Best for: portable, no-system-deps deployment

Native binary (llm-mem-mcp):

  • Smaller, depends on system libc/libstdc++
  • Released per platform (Linux x86_64, macOS aarch64, etc.)
  • Best for: when you control the runtime environment

From-source build (target/release/llm-mem-mcp):

  • After cargo build --release
  • Best for: development, custom feature combinations

Pick the binary that matches how you installed llm-mem, and use the absolute path in your MCP config.

Environment variables to set

The env map in the MCP config can set any of the environment variables llm-mem reads. Common ones:

{
  "env": {
    "LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
    "LLM_MEM_LLM_API_KEY": "sk-...",
    "HTTPS_PROXY": "http://proxy.corp:8080",
    "RUST_LOG": "info"
  }
}

Tip

Don’t put secrets in version control —

MCP config files are often committed to source repos. Use environment variables or a secret manager for API keys, not literal values in the config.

Verifying the connection

After updating your MCP config:

  1. Restart the client (opencode, VS Code, etc.) — it spawns the server as a subprocess
  2. Look in the client’s MCP/tool panel — you should see 32 llm-mem-* tools
  3. Ask the AI to call system_status — the response confirms the server is reachable and the models are loaded

If the tools don’t appear, see Troubleshooting MCP.

Next

opencode

opencode is the recommended MCP client for llm-mem. It’s an open-source terminal UI for AI coding assistants with first-class MCP support.

1. Install opencode

Follow the opencode install instructions. The quick version:

# macOS / Linux
curl -fsSL https://opencode.ai/install | bash

# Or via Homebrew
brew install opencode

# Or via npm
npm install -g @opencode/cli

2. Locate your opencode.json

opencode reads a config from one of two places:

  • Project-local: <your-project>/opencode.json — only affects this project
  • Global: ~/.config/opencode/config.json — affects every project

For a project-local config (most common for code work):

cd /path/to/your/project
touch opencode.json

For a global config:

mkdir -p ~/.config/opencode
touch ~/.config/opencode/config.json

3. Add llm-mem

Edit the file you created. The minimum configuration:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/absolute/path/to/llm-mem-mcp"],
      "enabled": true
    }
  }
}

If you installed the AppImage

{
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/home/you/apps/llm-mem-mcp-x86_64.AppImage"],
      "enabled": true
    }
  }
}

Make the AppImage executable first:

chmod +x /home/you/apps/llm-mem-mcp-x86_64.AppImage

If you built from source

{
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/home/you/projects/llm-mem/target/release/llm-mem-mcp"],
      "enabled": true
    }
  }
}

If you have multiple memory banks

You can declare multiple llm-mem instances, one per bank:

{
  "mcp": {
    "llm-mem-default": {
      "type": "local",
      "command": ["/path/to/llm-mem-mcp"],
      "enabled": true
    },
    "llm-mem-research": {
      "type": "local",
      "command": ["/path/to/llm-mem-mcp"],
      "args": ["--config", "/path/to/research-config.toml"],
      "enabled": true
    }
  }
}

The first one is the default; the second uses a separate config that points to a different banks_dir.

4. Add env vars (optional)

Common additions:

{
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/path/to/llm-mem-mcp"],
      "enabled": true,
      "env": {
        "LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
        "RUST_LOG": "info"
      }
    }
  }
}

See Environment variables for the full list.

5. Restart opencode and verify

opencode

The first MCP handshake can take 10-30 seconds (model download on first run). Watch the status indicator — when it turns green, the server is ready.

In opencode, run:

/tools

You should see 32 llm-mem tools listed. Try:

> Call system_status with llm-mem

If the response includes "backend": "local" and "llm_available": true, the connection is working.

6. Use it

From here on, the AI automatically has access to the llm-mem tools. You can just talk normally:

Remember that the AuthService refresh tokens expire after 7 days. What did I store about authentication? Upload docs/architecture.md to memory. Show me what I stored this week.

The AI decides which tool to call based on your request.

Troubleshooting

If the server doesn’t start or opencode can’t reach it, see Troubleshooting MCP.

Common issues specific to opencode:

  • Config not picked up: confirm the path with opencode --print-config or check the opencode logs
  • Permission denied on AppImage: chmod +x it
  • Server starts but no tools appear: check the opencode version supports MCP (>= 0.5.x)
  • Stale config: restart opencode after every config change — it doesn’t hot-reload MCP servers

Next

VS Code Copilot

VS Code’s Copilot Chat supports MCP servers via the MCP extension. Once connected, Copilot gains the 32 llm-mem tools as part of its tool palette.

1. Install prerequisites

  • VS Code 1.85+
  • GitHub Copilot Chat extension
  • MCP support is built into VS Code 1.85+ (no extra extension needed)

2. Choose config location

VS Code reads MCP servers from two places:

LocationScopePath
WorkspaceThis project only<workspace>/.vscode/mcp.json
User profileAll your workspacesUse MCP: Open User Configuration from the command palette

For a single project, use workspace scope (.vscode/mcp.json).

For a global setup that follows you across projects, use user scope.

3. Add llm-mem

Create .vscode/mcp.json in your workspace root:

{
  "servers": {
    "llm-mem": {
      "type": "stdio",
      "command": "/absolute/path/to/llm-mem-mcp",
      "args": []
    }
  }
}

AppImage variant

{
  "servers": {
    "llm-mem": {
      "type": "stdio",
      "command": "/home/you/apps/llm-mem-mcp-x86_64.AppImage",
      "args": []
    }
  }
}

From-source variant

{
  "servers": {
    "llm-mem": {
      "type": "stdio",
      "command": "/home/you/projects/llm-mem/target/release/llm-mem-mcp",
      "args": []
    }
  }
}

With environment variables

{
  "servers": {
    "llm-mem": {
      "type": "stdio",
      "command": "/path/to/llm-mem-mcp",
      "args": [],
      "env": {
        "LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
        "RUST_LOG": "info"
      }
    }
  }
}

4. Reload VS Code

After saving the config, run MCP: List Servers from the command palette (Ctrl+Shift+P / Cmd+Shift+P). You should see llm-mem listed with a green dot indicating the server started successfully.

If the dot is red or yellow, see Troubleshooting MCP.

5. Use it

Open Copilot Chat (the chat panel in the sidebar, or Ctrl+Shift+I / Cmd+Shift+I).

In agent mode, Copilot will automatically call the MCP tools when relevant. Try:

What llm-mem tools are available?

Copilot will list the 32 tools. Then:

Remember that the AuthService uses bcrypt for password hashing, see src/auth/hasher.rs:23. Search llm-mem for any auth-related facts. Upload README.md to the default memory bank.

Server lifecycle

VS Code spawns the MCP server on demand when you first use a tool, and shuts it down when VS Code quits. If you edit .vscode/mcp.json, you need to reload the server:

  1. Open command palette
  2. Run MCP: Restart Server (or MCP: List Servers → click the server)

Multiple servers

Add multiple entries under servers:

{
  "servers": {
    "llm-mem-default": {
      "type": "stdio",
      "command": "/path/to/llm-mem-mcp",
      "args": []
    },
    "llm-mem-research": {
      "type": "stdio",
      "command": "/path/to/llm-mem-mcp",
      "args": ["--config", "/path/to/research.toml"]
    }
  }
}

Each shows up as a separate tool group in Copilot.

Troubleshooting

  • Server not found: confirm the absolute path with ls or file <path> from a terminal
  • Permission denied on AppImage: chmod +x it
  • AppImage fails to launch: VS Code may not have FUSE access. Try APPIMAGE_EXTRACT_AND_RUN=1 in the env block
  • Models not downloading: check ~/.cache/llm-mem/models and your network/proxy settings
  • Tools don’t appear: reload the server from the command palette

See Troubleshooting MCP for more.

Next

Troubleshooting MCP

Common issues when connecting an MCP client to llm-mem-mcp.

Server doesn’t start

Symptom: Client shows the server as “failed” or “not connected”. Log file (if logging.enabled = true) shows an error.

Check:

  1. Absolute path: command must be an absolute path, not a relative one or a ~-expanded one.
  2. Executable bit: AppImage needs chmod +x. Native binaries usually don’t.
  3. Run it manually: try ./llm-mem-mcp --version from a terminal. If that fails, the binary itself is broken.
  4. Missing libraries: on Linux, run ldd /path/to/llm-mem-mcp to check for missing .so files. On macOS, use otool -L. On Windows, use Dependencies (https://lucasg.github.io/Dependencies/).
  5. AppImage + no FUSE: add "env": {"APPIMAGE_EXTRACT_AND_RUN": "1"} to the MCP config.

Server starts but no tools appear

Check:

  1. MCP version mismatch: confirm your client supports MCP. opencode 0.5+, VS Code 1.85+, Claude Code 1.0+.
  2. Restart the client: MCP server lists are loaded at startup, not hot-reloaded.
  3. Check stderr: the server logs to stderr. With logging.enabled = true, also check the log file. Look for JSON-RPC errors.
  4. Try a different transport: llm-mem only supports stdio. If your client is trying HTTP/SSE, it won’t work.

Server starts, tools appear, but calls fail

Check:

  1. system_status first: ask the AI to call it. The response tells you if models are loaded, what backend is active, and whether the LLM/embedding are reachable.
  2. Wait for model download: on first run, the server downloads ~3 GB of models. Until that’s done, most tools return errors. Watch the log file for “model loaded” messages.
  3. API key issues: if using the API backend, verify the key is set via LLM_MEM_LLM_API_KEY (or in the config). The server validates at startup and will refuse to start without it.
  4. Bank creation error: if the banks_dir isn’t writable, the server fails to create the default bank. Check filesystem permissions and the banks_dir setting.

Performance is slow

Check:

  1. GPU not used: see GPU acceleration. CPU-only inference is 5-20x slower.
  2. First query is slow: model loading + warmup takes 5-30 seconds. Subsequent queries are fast.
  3. Concurrent requests: default max_concurrent_requests = 1 for stability. Bump it if you have a beefy machine.
  4. max_concurrent_requests for API backend: set to 0 (unlimited) if you have API rate-limit headroom.
  5. Large bank: queries get slower as the bank grows past ~100k memories. Consider per-project banks.

Logs

The server logs to stderr. With logging.enabled = true, it also writes to a rotating log file under log_directory (default llm-mem-data/logs/).

To get more detail, set the RUST_LOG env var in the MCP config’s env block:

"env": {
  "RUST_LOG": "info,llama_cpp_2=warn"
}

Useful levels:

ValueWhat you get
errorOnly errors
warnErrors + warnings
info (default)Lifecycle events, request summaries
debugPer-request detail
traceEverything, including the LLM’s prompts

See Logging & debugging for more.

Common error messages

Failed to resolve memory bank: ...

The bank name passed to the tool doesn’t exist and can’t be created. Check spelling. Banks are created on first write, not on first read.

LLM timeout after 120 seconds

The LLM is taking too long. Either:

  • Increase llm.llm_timeout_secs in the config
  • Switch to a smaller/faster model
  • Enable GPU acceleration
  • If using an API, the upstream is slow — try a different model or provider

No API key configured

You set provider = "api" but didn’t provide a key. Set LLM_MEM_LLM_API_KEY (or in the config), or change provider = "local".

Configuration Error: llm.context_size is too small

llm.context_size must be larger than memory.document_chunk_size / 2 + llm.max_tokens + 512. Increase context_size or decrease document_chunk_size.

Bank deletion requires user confirmation

You called cleanup_resources with target: "banks" but didn’t pass the exact confirmation phrase. This is a safety check — pass confirm: "I confirm this data will be permanently lost".

Still stuck?

  1. Run llm-mem-mcp --generate-config to confirm the binary works
  2. Run llm-mem health-check --live (the CLI binary) to validate the config end-to-end
  3. Check the GitHub issue tracker: https://github.com/vsndev3/llm-mem/issues
  4. Include the log output and your config (with secrets redacted) when asking for help

Next

Configuration

llm-mem has a single TOML config file (or no file at all — defaults work). The config has six top-level sections:

SectionPurpose
[llm]Language model (local llama.cpp or OpenAI-compatible API)
[embedding]Embedding model (local fastembed or OpenAI-compatible API)
[vector_store]Storage backend (LanceDB) and bank directory
[memory]Memory behavior: thresholds, dedup, chunking, abstraction
[server]HTTP server config (host/port)
[logging]File logging with rotation

Generating a template

The fastest way to see every option is to generate a commented template:

llm-mem-mcp --generate-config > config.toml

The result is a complete config.toml with every field commented out and explained.

Loading order

On startup, the server:

  1. Loads the config file (search order: --config flag → ./config.toml → ~/.config/llm-mem/config.toml → /etc/llm-mem/config.toml)
  2. Applies environment variable overrides (see Environment variables)
  3. Applies CLI flag overrides (see CLI flags)
  4. Validates the resulting config (refuses to start if invalid)

If no config file is found, all defaults are used and the server runs in fully-local mode with a Gemma 4 language model and an all-MiniLM embedding model.

Minimal local config

This is the default. You don’t actually need a config file, but if you want to customize banks_dir:

[vector_store]
banks_dir = "/var/lib/llm-mem/banks"

Minimal API config

[llm]
provider = "api"
api_key = "sk-..."              # or set LLM_MEM_LLM_API_KEY
api_url = "https://api.openai.com/v1"
model = "gpt-4o-mini"

[embedding]
provider = "api"
api_key = "sk-..."              # or set LLM_MEM_EMBEDDING_API_KEY
api_url = "https://api.openai.com/v1"
model = "text-embedding-3-small"

Mixed config: API LLM, local embeddings

[llm]
provider = "api"
api_key = "sk-..."
model = "gpt-4o-mini"

[embedding]
provider = "local"              # keep embeddings local
model = "all-MiniLM-L6-v2"

Validation

The server validates the config at startup and refuses to start on errors. Common validations:

FieldConstraint
memory.similarity_threshold0.0 ≤ x ≤ 1.0
memory.merge_threshold0.0 ≤ x ≤ 1.0
memory.search_similarity_threshold0.0 ≤ x ≤ 1.0 (if set)
llm.temperature0.0 ≤ x ≤ 2.0
llm.context_size> 0
llm.max_tokens> 0
llm.batch_max_tokens> 0 and ≤ llm.max_tokens
embedding.batch_size> 0
memory.max_memories> 0
memory.max_content_length> 0
llm.context_size≥ memory.document_chunk_size / 2 + llm.max_tokens + 512 (local backend)
llm.provider = "local"requires build feature local-llm or local
embedding.provider = "local"requires build feature local-embed or local
API providerrequires api_key (or env var)

If validation fails, the server prints a clear error message and exits.

Section reference

Environment variables

Any config field can be set via environment variable. See Environment variables for the full list.

CLI flags

The llm-mem-mcp binary accepts flags that override the config at runtime. See CLI flags.

Next

Read through the section references, or jump to the one you need.

[llm] — language model

Settings for the language model used for extraction, summarization, classification, and query answering.

[llm]
provider = "local"               # "local" (llama.cpp) or "api" (OpenAI-compatible)

Provider selection

ValueWhat it does
"local"Embedded llama.cpp inference. No network, no API key. Downloads a GGUF model on first run.
"api"OpenAI-compatible HTTP API. Requires api_key (or env var).

You can mix: see Embedding section for the combinations.

Local provider settings

[llm]
provider = "local"
model_file = "gemma-4-E2B-it-Q8_0.gguf"   # GGUF filename (auto-downloaded if missing)
models_dir = "llm-mem-data/models"         # where to put downloaded models
gpu_layers = 0                            # 0 = CPU only; set to 20+ for GPU
context_size = 16644                      # context window in tokens
cpu_threads = 0                           # 0 = auto-detect
auto_download = true                      # fetch from HuggingFace on first run
cache_model = true                        # cache in ~/.cache/llm-mem/models
cache_dir = ""                            # custom cache dir (default: ~/.cache/llm-mem/models)
llm_timeout_secs = 120                    # completion timeout
use_grammar = false                       # grammar-constrained sampling (faster, less flexible)
proxy_url = ""                            # proxy for downloads (overrides HTTPS_PROXY)

Vision (local)

If you have a vision-capable GGUF model + mmproj file:

vision_enabled = true
mmproj_file = ""                          # empty = auto-detect mmproj-F16.gguf for Gemma 4 E2B
vision_prompt_template = "Describe this image in detail, focusing on what would make it searchable in a personal memory system. What objects, people, text, colors, scene elements, actions, and contextual cues are visible? What domain or topic does this image relate to? Be specific and concrete."

When vision_enabled = true, ingested images (PNG/JPEG/GIF/WebP) get an AI-generated description stored alongside the original.

API provider settings

[llm]
provider = "api"
api_url = "https://api.openai.com/v1"
api_key = ""                              # or set LLM_MEM_LLM_API_KEY / OPENAI_API_KEY
model = "gpt-4o-mini"

Common api_url values:

ServiceURL
OpenAIhttps://api.openai.com/v1
OpenRouterhttps://openrouter.ai/api/v1
Anthropicuse api_dialect = "anthropic" with the Anthropic endpoint
Ollamahttp://localhost:11434/v1
llama-serverhttp://localhost:8080/v1
LM Studiohttp://localhost:1234/v1

Dialects

api_dialect = "openai-chat"   # one of: openai-chat, openai-completion, anthropic, ollama-chat, ollama-completion, custom

openai-chat (default) works for any OpenAI-compatible chat completions endpoint. Use anthropic for Anthropic, ollama-* for Ollama native APIs, custom for fully custom request/response shapes.

Custom dialect

api_dialect = "custom"

[llm.custom_dialect]
endpoint_path = "/v1/generate"
request_body_template = '{"model": "{{model}}", "prompt": "{{prompt}}", "max_tokens": {{max_tokens}}, "temperature": {{temperature}}}'
response_content_pointer = "/choices/0/text"

Placeholders: {{prompt}}, {{model}}, {{temperature}}, {{max_tokens}}. response_content_pointer is a JSON pointer to the text in the response.

Request format

request_format = "auto"        # "auto" (default), "rig", or "raw"
  • auto — try rig-core first, fall back to raw HTTP on 422 errors (most compatible)
  • rig — always use rig-core (may 422 on strict backends)
  • raw — always use raw HTTP with plain strings (bypasses rig-core)

Structured output

use_structured_output = true   # JSON schema validation (API only)
structured_output_retries = 2  # retries on validation failure

The server asks the API for JSON schema-validated output, retrying if the LLM returns malformed JSON. Disable for backends that don’t support it.

Generation settings (both providers)

temperature = 0.7              # 0.0 = deterministic, 2.0 = very creative
max_tokens = 4096              # max tokens per completion
max_concurrent_requests = 1    # 0 = unlimited, 1 = serial (safest)
strip_tags = ["think"]         # XML tags to strip from LLM output (e.g. "think", "reason")

Advanced / batching

batch_size = 10                # items per batch request
batch_max_tokens = 3000        # tokens per batch request (must be ≤ max_tokens)
batch_timeout_secs = 120       # base timeout for batch calls
batch_timeout_multiplier = 1.0 # timeout multiplier

The server batches independent LLM calls (e.g. classifying multiple memories at once) to reduce API overhead.

Next

[embedding] — embedding model

Settings for the embedding model that converts text to vectors for semantic search.

[embedding]
provider = "local"             # "local" (fastembed) or "api" (OpenAI-compatible)
model = "all-MiniLM-L6-v2"     # model name

Provider selection

ValueWhat it does
"local"Embedded fastembed inference. No network, no API key. Downloads ONNX model on first run.
"api"OpenAI-compatible HTTP API. Requires api_key (or env var).

Local provider settings

[embedding]
provider = "local"
model = "all-MiniLM-L6-v2"     # any fastembed-supported model

Supported fastembed models include:

  • all-MiniLM-L6-v2 (default, 384 dimensions, ~90 MB)
  • all-mpnet-base-v2 (768 dimensions, higher quality)
  • BAAI/bge-small-en-v1.5 (384 dimensions)
  • BAAI/bge-large-en-v1.5 (1024 dimensions)
  • intfloat/multilingual-e5-large (1024 dimensions, multilingual)

The dimension is auto-detected from the model. The local backend requires the local-embed (or aggregate local) build feature.

API provider settings

[embedding]
provider = "api"
api_url = "https://api.openai.com/v1"
api_key = ""                   # or set LLM_MEM_EMBEDDING_API_KEY / OPENAI_API_KEY
model = "text-embedding-3-small"

Common model choices:

ModelDimensions
text-embedding-3-small1536
text-embedding-3-large3072
text-embedding-ada-0021536

Warning

Dimension must match storage —

If you change the embedding model, the dimension changes. New memories use the new dimension; old memories keep their old vectors. Configure the corresponding [vector_store.lancedb].embedding_dimension (or VectorLite equivalent) to match.

Tuning

batch_size = 64                # texts per API call
timeout_secs = 30              # per-call timeout

batch_size is how many texts are bundled into a single embedding request. Larger batches are more efficient; smaller batches are more responsive.

Combinations

You can mix LLM and embedding providers:

LLMEmbeddingUse case
locallocalDefault. Fully offline.
apiapiCloud LLM + cloud embeddings.
apilocalCloud LLM with offline embeddings (faster, cheaper).
localapiLocal LLM with cloud embeddings.

The four combinations are auto-detected from the two provider settings.

Next

[vector_store] — storage backend

Settings for the embedded vector database (LanceDB by default) and the memory bank directory.

[vector_store]
banks_dir = "llm-mem-data/banks"   # where bank databases live
store_type = "lancedb"             # "lancedb" (default) or "vectorlite" (legacy)
collection_name = "llm-memories"   # default collection name

Banks directory

banks_dir is the parent directory for all memory banks. Each bank gets its own subdirectory:

<banks_dir>/
├── default.lance/             # the default bank
├── default.sessions.db        # document upload sessions for default
├── research.lance/            # another bank
└── research.sessions.db

To use a different location:

[vector_store]
banks_dir = "/var/lib/llm-mem/banks"

The directory is created automatically on first run if it doesn’t exist.

LanceDB settings (default)

[vector_store]
store_type = "lancedb"

[vector_store.lancedb]
table_name = "memories"
database_path = "./lancedb"
embedding_dimension = 384       # MUST match the embedding model

embedding_dimension is critical: it must equal the dimension of the embedding model. Mismatched dimensions cause a runtime error.

Embedding modelDimension
all-MiniLM-L6-v2384
all-mpnet-base-v2768
BAAI/bge-small-en-v1.5384
BAAI/bge-large-en-v1.51024
text-embedding-3-small1536
text-embedding-3-large3072

VectorLite settings (legacy)

[vector_store]
store_type = "vectorlite"

[vector_store.vectorlite]
index_type = "hnsw"              # "hnsw" or "flat"
metric = "cosine"                # "cosine", "euclidean", or "dot"
persistence_path = ""            # defaults to banks_dir
embedding_dimension = 384        # optional; inferred from data if absent

VectorLite is the legacy backend. LanceDB is recommended for new deployments.

Note

VectorLite requires the vector-lite build feature, which is off by default. Build with --features vector-lite to use it.

Multiple banks

Banks are not configured in the TOML — they are created on demand by the AI calling create_memory_bank (or implicitly by writing to a bank name). Each bank gets its own .lance directory.

To pre-create banks on disk, use the CLI:

llm-mem --single list-banks

To restrict which banks exist, you can manage the directory directly:

mkdir -p /var/lib/llm-mem/banks
mkdir /var/lib/llm-mem/banks/research.lance
# Pre-creating an empty bank directory is allowed — the server will populate it on first write.

Storage size

LanceDB stores vectors + metadata. Rough estimates:

  • 384-dim f32 vector: 1.5 KB
  • 100k memories × 1.5 KB vectors + 1 KB metadata ≈ 250 MB on disk
  • 1M memories ≈ 2.5 GB on disk

The .sessions.db file is a small SQLite file tracking document upload sessions — usually a few MB.

Next

[memory] — memory behavior

Settings for how memories are stored, deduplicated, chunked, and abstracted.

[memory]
max_memories = 10000
similarity_threshold = 0.65
search_similarity_threshold = 0.2
max_search_results = 50

Capacity

FieldDefaultWhat it controls
max_memories10000Soft cap on memories per bank. Used for log warnings and admission control.
max_search_results50Default upper bound on results per query. Tools can override per-call.
max_list_limit10000Hard ceiling for list_memories with no explicit limit. Prevents accidental full-table scans.
memory_ttl_hoursNoneTime-to-live in hours. None = no expiry. Some(0) = no expiry (same as None).
max_content_length32768Reject content longer than this (bytes).
max_cascade_fanout5000Max abstraction dependents to load at once during cascade deletion. Prevents OOM.
raw_content_scan_limit5000Max raw content candidates during keyword/text search fallback.
max_total_candidates10000Max in-memory candidates during pyramid assembly before final ranking.

Similarity thresholds

FieldDefaultWhat it controls
similarity_threshold0.65Deduplication threshold. When a new memory’s embedding is closer than this to an existing one, dedup logic kicks in.
merge_threshold0.75Higher threshold to merge two memories into one (vs. keep both).
search_similarity_thresholdSome(0.2)Reject search results below this. Lower = more results.
near_duplicate_threshold0.92When a new memory’s embedding similarity to an existing memory exceeds this, a warning is logged. 0.0 disables. (Quality-control)
auto_link_threshold0.75Similarity threshold for auto-creating references relations between new and existing memories. 0.0 disables auto-linking.
auto_link_max_relations10Max auto-link relations per stored memory.

Enhancement

FieldDefaultWhat it controls
auto_enhancetrueAutomatically enrich new memories with LLM-generated metadata (entities, keywords, importance). Costs LLM tokens.
deduplicatetrueSkip storing near-duplicates.
auto_summary_threshold32768When L0 content accumulates past this size, trigger the L0→L1 summarization worker.

Document processing

FieldDefaultWhat it controls
document_chunk_size2000Chunk size in characters for document ingestion. 0 = no chunking.
chunk_threshold_chars1000When a single L0 memory exceeds this, split into sub-chunks for embedding. 0 = disable.
chunk_size_chars1000Target sub-chunk size in characters.
chunk_overlap_chars100Overlap between consecutive sub-chunks.

Warning

chunk_threshold_chars > chunk_size_chars will log a warning at startup — it means some content past the threshold can still slip through unsplit if it’s under chunk_threshold_chars.

LLM-assisted parsing (off by default)

These cost LLM tokens but help with malformed or unknown content.

FieldDefaultWhat it controls
use_llm_query_classificationfalseUse LLM for query intent classification (vs. keyword heuristic).
llm_format_detectionfalseWhen content-based format detection returns “unknown”, ask the LLM to identify. Sends first 4 KB.
llm_fallback_parsingfalseWhen structured parsers fail, ask the LLM to extract summary, entities, and content type.

Cost & quality

FieldDefaultWhat it controls
session_token_budget0Per-session LLM token budget. 0 = unlimited. When exceeded, expensive ops (enhance, classify, abstract) are skipped.
dry_runfalseLog what would be done without executing LLM calls. Useful for cost estimation.
contradiction_detectionfalseWhen storing new memories, ask the LLM to check the top-3 semantically similar memories for factual contradictions.
access_decay_hours168Time decay half-life (hours) for access-frequency boosting. Recent accesses boost score; older decay. 0 disables. (168 = 1 week)

Next

[server] — HTTP server

The MCP transport is always stdio. The [server] section is reserved for an optional HTTP/management interface that isn’t currently used by the MCP server.

[server]
host = "0.0.0.0"
port = 8000
FieldDefaultWhat it controls
host"0.0.0.0"Bind address (currently informational).
port8000Bind port (currently informational).

Note

This section is reserved for future use. The MCP server only speaks stdio. Standalone use of the [server] section is not yet supported.

For practical server management, see Operations and Logging & debugging.

[logging] — log output

File-based logging with size-based rotation. Disabled by default; logs go to stderr.

[logging]
enabled = false
log_directory = "llm-mem-data/logs"
level = "info"
max_size_mb = 1
max_files = 5
FieldDefaultWhat it controls
enabledfalseIf true, write to llm-mem-mcp.log in log_directory in addition to stderr.
log_directory"llm-mem-data/logs"Directory for the rotating log file. Created if missing.
level"info"Default level: error, warn, info, debug, trace.
max_size_mb1Rotate the log when it exceeds this size.
max_files5Keep this many rotated log files (e.g. llm-mem-mcp.log.1, .2, etc.).

Per-request verbosity

The level setting is the default, but the RUST_LOG env var (set in the MCP config’s env block) overrides it per-process. It accepts the tracing_subscriber filter syntax:

"env": {
  "RUST_LOG": "info,llama_cpp_2=warn,llm_mem::search=debug"
}

This sets the global default to info, silences llama_cpp_2 to warn, and turns on debug for the search module.

Stderr vs file

When logging.enabled = true:

  • The MCP JSON-RPC traffic goes to stdout (untouched — required by the protocol)
  • All tracing events go to stderr (always) and to the rotating log file (when enabled)

This means the MCP client can talk to the server without log lines contaminating the protocol stream.

Examples

Quiet (errors only):

[logging]
enabled = true
level = "error"

Verbose debugging:

[logging]
enabled = true
level = "debug"

Larger log files, more history:

[logging]
enabled = true
max_size_mb = 50
max_files = 10

Next

Environment variables

Any config field can be set via an environment variable. The variables are applied after the config file loads and before the server validates.

LLM

VariableMaps toNotes
LLM_MEM_LLM_API_KEYllm.api_key
LLM_MEM_LLM_API_BASE_URLllm.api_url
LLM_MEM_LLM_MODELllm.model
LLM_MEM_MODELS_DIRllm.models_dir
LLM_MEM_GPU_LAYERSllm.gpu_layersu32
LLM_MEM_CONTEXT_SIZEllm.context_sizeu32
LLM_MEM_TEMPERATUREllm.temperaturef32
LLM_MEM_MAX_TOKENSllm.max_tokensu32
LLM_MEM_CPU_THREADSllm.cpu_threadsi32
LLM_MEM_MAX_CONCURRENT_REQUESTSllm.max_concurrent_requestsusize

Embedding

VariableMaps toNotes
LLM_MEM_EMBEDDING_API_KEYembedding.api_key
LLM_MEM_EMBEDDING_API_BASE_URLembedding.api_url
LLM_MEM_EMBEDDING_MODELembedding.model

Fallback

VariableMaps toNotes
OPENAI_API_KEYllm.api_key and embedding.api_keyOnly fills if both are still empty after the specific env vars are applied. Standard OpenAI convention.

Network

VariableMaps toNotes
HTTPS_PROXYoutbound HTTPS proxyUsed for model downloads. Format: http://host:port or http://user:pass@host:port
HTTP_PROXYoutbound HTTP proxy
ALL_PROXY / all_proxySOCKS proxy

The llm.proxy_url config (or --proxy CLI flag) overrides the env-var proxy.

Logging

VariableNotes
RUST_LOGtracing-subscriber filter syntax. Overrides the logging.level config.
APPIMAGE_EXTRACT_AND_RUNAppImage-only. Run without FUSE by extracting to a tmpfs.

Abstraction pipeline

Unlike the tables above, these variables are not config-field overrides — they are read directly at startup. Invalid values are logged and fall back to the default.

VariableDefaultNotes
LLM_MEM_ABSTRACTION_DELAY_SECS30Delay between abstraction pipeline cycles.
LLM_MEM_ABSTRACTION_CONCURRENCY3Concurrent abstraction tasks per cycle (max_concurrent_tasks). Raise for batch/benchmark workloads.
LLM_MEM_BACKGROUND_LLM_CONCURRENCY3Background LLM permits available to the abstraction pipeline. Raise alongside LLM_MEM_ABSTRACTION_CONCURRENCY so concurrency isn’t throttled by the interactive-friendly default.

Usage in MCP config

Set environment variables in the MCP config’s env block:

{
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/path/to/llm-mem-mcp"],
      "env": {
        "LLM_MEM_LLM_API_KEY": "sk-...",
        "LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
        "RUST_LOG": "info,llm_mem::search=debug",
        "HTTPS_PROXY": "http://proxy.corp:8080"
      }
    }
  }
}

Tip

For per-machine settings (proxy, models dir, API key), prefer the env block in the MCP config. For per-deployment settings (URL, model), prefer the TOML config file. Don’t put secrets in the TOML if the file is committed.

Override order

When the same field is set in multiple places, the priority is:

  1. CLI flag (highest)
  2. Specific env var (e.g. LLM_MEM_LLM_API_KEY)
  3. Generic env var (e.g. OPENAI_API_KEY)
  4. Config file value
  5. Default (lowest)

So --api-url on the CLI beats LLM_MEM_LLM_API_BASE_URL which beats [llm].api_url in config.toml which beats the hardcoded default.

Next

CLI flags

The llm-mem-mcp binary accepts these flags. They override both the config file and environment variables.

llm-mem-mcp [OPTIONS]

General

FlagShortTypeDefaultWhat it does
--config <PATH>-cpathauto-discoverPath to the config file. If not given, searches ./config.toml, ~/.config/llm-mem/config.toml, /etc/llm-mem/config.toml.
--agent <ID>stringnoneDefault agent_id for all tool calls. Overrides the per-tool agent_id parameter when not set.
--proxy <URL>stringnoneProxy URL for model downloads. Format: http://host:port or http://user:pass@host:port. Overrides HTTPS_PROXY.
--banks-dir <PATH>pathfrom configOverride vector_store.banks_dir.
--generate-configboolfalsePrint a commented config template to stdout and exit.

LLM behavior

FlagTypeDefaultWhat it does
--no-grammarboolfalseDisable grammar-constrained sampling for local LLM structured output. Grammar is enabled by default.
--no-structured-outputboolfalseDisable structured output mode for API LLM. Structured output is enabled by default.
--request-format <FMT>stringautoauto, rig, or raw. See [llm] request_format. Invalid values cause the server to exit.

Model caching

FlagTypeDefaultWhat it does
--no-cache-modelboolfalseDon’t cache models in ~/.cache/llm-mem/models. Caching is enabled by default.
--cache-dir <PATH>path~/.cache/llm-mem/modelsCustom cache directory.

Examples

# Use a specific config file
llm-mem-mcp --config /etc/llm-mem/prod.toml

# Run with raw HTTP (skip rig-core) against a self-hosted llama.cpp
llm-mem-mcp --request-format raw

# Force re-download the model (skip cache)
llm-mem-mcp --no-cache-model

# Pre-populate the cache, then run normally
llm-mem-mcp --cache-dir /mnt/fast/models

# Use a different banks directory
llm-mem-mcp --banks-dir /mnt/data/banks

# Print the config template
llm-mem-mcp --generate-config > config.toml

Standalone CLI

The llm-mem (no -mcp suffix) binary has a different flag set. It has subcommands like search, list, stats, db, etc. See CLI overview.

Next

MCP tools reference

llm-mem-mcp exposes 32 tools to MCP clients. They’re grouped by purpose here.

The 32 tools at a glance

Storing

Finding

Chronology

Abstraction

Banks

Documents

Introspection

How parameters work

Every tool takes a JSON object of parameters. Optional parameters can be omitted. Parameters marked as accepting a bank field default to the "default" bank if not provided.

The common bank parameter

Most tools accept an optional bank string. If omitted, the tool operates on the "default" bank.

{ "query": "JWT auth", "bank": "research" }

Agent and user IDs

FieldUsed byPurpose
agent_idstore/find toolsThe agent storing or querying. Set once via --agent CLI flag and inherited by all calls.
user_idstore toolsOptional. Only needed when multiple users share a bank.

Common response shape

Tools return a JSON object with at least:

{
  "success": true,
  "message": "human-readable status",
  "data": { /* tool-specific */ },
  "error": null
}

On failure, success is false and error contains a human-readable description.

Reading the per-tool pages

Each category page has a one-paragraph description of the tool, the input schema (as a table), and the output schema. For the canonical schema, see src/operations/tools.rs in the source.

Next

Storing memories

Tools for writing new memories to a bank. There are four “store” variants plus document ingestion (covered under Document upload).

add_content_memory

Add raw content to memory without any AI transformation. The content is stored and embedded exactly as-is, preserving all original phrases, keywords, and structure.

Use this when:

  • You need exact-phrase searchability (e.g. finding “vegan chili” later)
  • Storing conversation logs, documents, or code snippets where original text matters
  • You want predictable semantic search based on the actual content, not AI interpretations

For AI-processed structured facts, use add_intuitive_memory instead.

Input

FieldTypeRequiredDescription
contentstring✓The fact or piece of information. Should be concise and atomic.
metadataobjectKey-value pairs (source file, page, timestamp, author).
user_idstringOnly if multiple users share a bank.
agent_idstringAgent ID. Inherited from CLI if unset.
topicsarray of stringSubjects within the content.
contextarray of stringBroad topic tags.
relationsarray of object{relation, target} pairs to other memories or entities.
bankstringDefaults to "default".
auto_linkbooleanWhether to auto-create references relations to similar memories. Server default if unset.
event_atstringISO 8601 datetime — when the event actually happened.
sourcestringFree-form provenance (file name, URL, book title).
forcebooleanSkip near-duplicate / contradiction checks.

Output

{
  "success": true,
  "message": "Memory stored",
  "data": {
    "memory_id": "uuid",
    "user_id": "...",
    "agent_id": "..."
  }
}

add_intuitive_memory

Add memories with AI-powered extraction and structuring. The LLM analyzes your content, extracts key facts, organizes them into atomic insights, and generates searchable keywords.

Use this when:

  • You want structured, reasoning-ready memories
  • You need condensed insights from long conversations or documents
  • You want automatic keyword extraction for hybrid search

The original text is transformed by the LLM. For verbatim storage, use add_content_memory.

Input

FieldTypeRequiredDescription
messagesarray of object✓Conversation messages: {role, content, name?}.
metadataobjectAttached to all extracted memories.
user_idstring
agent_idstring
bankstring
source_memory_idstringLink extracted memories to a source. Auto-creates derived_from relation.
event_atstringISO 8601 — applied to all extracted memories.

Output

{
  "success": true,
  "message": "...",
  "data": {
    "results": [
      { "id": "uuid", "memory": "...", "event": "...", "actor_id": "...", "role": "..." }
    ],
    "user_id": "...",
    "agent_id": "..."
  }
}

data.results holds one entry per extracted memory.

store_memories

Store multiple content memories in a single call. Much faster than calling add_content_memory repeatedly for bulk ingestion.

Rules:

  • items cannot be empty
  • Each item must have non-empty content
  • Invalid items cause the entire batch to be rejected before any storage

Input

FieldTypeRequiredDescription
itemsarray of object✓Each: {content, topics?, context?, relations?, metadata?, event_at?, source?}.
bankstring
forcebooleanSkip near-duplicate / contradiction checks for the whole batch.
waitbooleanfalse (default) returns immediately with a batch_id and processes in the background (poll get_batch_status); true blocks until all items are stored.

Output

{
  "success": true,
  "message": "Stored 5 memories",
  "data": {
    "results": [ /* per-item results */ ],
    "total": 5,
    "batch_id": "uuid",
    "status": "completed"
  }
}

Which to use?

ScenarioTool
“Remember this exact sentence”add_content_memory
“Extract the key facts from this conversation”add_intuitive_memory
“Ingest these 100 notes at once”store_memories
“Store this PDF / markdown file”upload_document
“Ingest this file, please chunk it and extract structure”ingest

Quality-control flags

add_content_memory accepts a force boolean:

  • force: false (default) — the server checks for near-duplicates and contradictions. On detection, the call is blocked and the error describes the conflict.
  • force: true — bypass the checks. Use only when you’re certain the content should be stored despite the warning.

store_memories has a top-level force flag applying to the whole batch. add_intuitive_memory has no force flag — it always runs the near-duplicate / contradiction checks.

Warning

Setting force: true defeats the contradiction-detection safety net. Use it sparingly and only after reviewing what the conflict actually is.

Next

Finding memories

Tools for retrieving stored memories. Four tools cover different access patterns.

query_memory

The most powerful search tool. Supports hybrid keyword/semantic search, pyramid allocation, graph traversal, and context filtering.

Important: score interpretation

Scores reflect semantic similarity, NOT whether the answer is present. A high score means the memory is topically related — it does not guarantee the answer exists in that memory. Always verify retrieved content against what was asked:

  • Use k >= 5 to ensure adequate recall.
  • L0 = concrete source content (facts, verbatim text). The answer lives here.
  • L1+ = increasingly abstract (summaries → concepts → wisdom). These help you navigate to relevant L0 facts but are not themselves the answer.

Pyramid allocation (pyramid_config)

Pyramid search distributes result slots across abstraction layers, so a single query returns both concrete facts and abstract insights. Set pyramid_config.mode to one of:

ModeBehavior
"bottom_heavy" (default)More L0 facts, fewer abstract concepts
"balanced"Equal distribution across layers
"top_heavy"More abstract concepts, fewer concrete facts
"dynamic"LLM classifies query intent automatically (requires use_llm_query_classification config)
"none"Skip pyramid assembly, return flat results by raw score

pyramid_config also accepts layer_weights (per-layer weight overrides) and per_layer_multiplier (default 2.0).

Graph traversal (graph_traversal)

Enable graph_traversal.enabled to replace semantic search with a pure BFS multi-hop traversal, following relations (derived_from, mentions, knows, …) from the top-scoring entry points:

FieldDescription
enabledTurn on deep traversal (default false).
max_depthMax hops (1-5, default 2).
direction"outgoing", "incoming", or "both" (default).
relation_typesRestrict to specific relation types.
entry_point_limitMax seed memories (default 5, max 10).
include_pathsInclude per-result graph_info (distance, path, boosts).

Without deep traversal, standard search still applies automatic 1-hop graph refinement on the top results.

Input

FieldTypeRequiredDescription
querystring✓The search query.
kintegerMax results (default 10).
bankstringMemory bank name (default "default").
topicsarray of stringFilter by topics.
contextarray of stringContext tags for semantic scoping.
keyword_onlybooleanKeyword-only search, no embeddings (default false).
keyword_split_ratiofloatFraction of results from raw keyword matching vs. semantic (0.0-1.0, default 0.2).
similarity_thresholdfloatOverride similarity threshold (0.0-1.0).
min_saliencefloatMinimum salience/importance score (0-1).
granularitystring"full" (default) resolves chunk hits to the complete parent memory; "excerpt" returns a compact window around matched regions (session header preserved), under excerpt_max_chars.
excerpt_max_charsintegerTotal char budget for excerpt-mode content (default 12000, min 1000).
pyramid_configobjectmode, layer_weights, per_layer_multiplier (see above).
graph_traversalobjectDeep traversal config (see above).
created_after / created_beforestringISO 8601 — filter by storage time.
event_after / event_beforestringISO 8601 — filter by event_at (or created_at).
user_id / agent_idstringFilter by author.

Output

{
  "success": true,
  "count": 12,
  "memories": [
    {
      "memory_id": "uuid",
      "content": "...",
      "score": 0.87,
      "layer": 0,
      "layer_name": "raw_content",
      "context": ["project-x", "auth"],
      "topics": ["jwt"],
      "metadata": { /* ... */ },
      "event_at": "2026-02-15T10:00:00Z",
      "created_at": "2026-02-15T11:23:45Z"
    }
  ]
}

search_memory

A simplified search with sensible defaults. Use this when you don’t need the full pyramid/graph power.

Same score interpretation rules apply: high similarity does not guarantee the answer is present, use k >= 5, and L0 is where concrete facts live.

Input

FieldTypeRequiredDescription
querystring✓
kintegerMax results (default 10).
bankstring

Output

Same { success, count, memories } shape as query_memory.

list_memories

Browse by structured filter — no semantic similarity. Useful for “show me everything I stored yesterday”.

Input

FieldTypeRequiredDescription
limitintegerDefault 100, max 1000.
bankstring
created_after / created_beforestringISO 8601.
event_after / event_beforestringISO 8601.
user_id / agent_idstring

Output

{
  "success": true,
  "count": 1234,
  "memories": [ /* memory objects */ ]
}

get_memory

Look up a specific memory by ID.

Input

FieldTypeRequiredDescription
memory_idstring✓
bankstring

Output

{
  "success": true,
  "memory": {
    "memory_id": "uuid",
    "content": "...",
    "layer": 0,
    "context": ["..."],
    "topics": ["..."],
    "metadata": { /* ... */ },
    "event_at": "...",
    "created_at": "...",
    "updated_at": "..."
  }
}

Choosing the right tool

I want to…Use
Find memories that match a question / topicquery_memory or search_memory
Browse by date / type / taglist_memories
Get the full record for a specific memory IDget_memory
Traverse relations from a seed memoryquery_memory with graph_traversal: { enabled: true }
Favor abstract concepts over raw factsquery_memory with pyramid_config: { mode: "top_heavy" }
Keep retrieved content compactquery_memory with granularity: "excerpt"

Next

Chronological queries

Tools for time-based views of memories. Three tools cover different temporal access patterns.

All three bucket by event_at (the date the content refers to); if event_at is missing, they fall back to created_at.

get_timeline

Returns memories bucketed by time interval. Answers “what happened in the last 2 days” or “give me a monthly summary of 2025”.

Input

FieldTypeRequiredDescription
startstringISO 8601 lower bound. Default: end - 7d.
endstringISO 8601 upper bound. Default: now.
granularitystringhour, day (default), week, month, none (single bucket).
bankstringMemory bank (default "default").
topicsarray of stringFilter to memories tagged with any of these topics.
max_results_per_bucketintegerCap per bucket. Default 50.
include_derivedbooleanInclude L1+ abstractions (default false, L0 only).
orderstringasc (default) or desc within each bucket.
user_id / agent_idstringFilter by author.

Output

{
  "success": true,
  "start": "2026-02-14T00:00:00Z",
  "end": "2026-02-16T00:00:00Z",
  "granularity": "day",
  "total_count": 23,
  "bucket_count": 2,
  "buckets": [
    {
      "start": "2026-02-14T00:00:00Z",
      "end": "2026-02-15T00:00:00Z",
      "label": "2026-02-14",
      "count": 8,
      "memories": [ /* truncated to max_results_per_bucket */ ]
    }
  ]
}

Examples

// "What happened in the last 2 days?"
{ "end": "2026-02-15T00:00:00Z", "granularity": "day" }

// "Monthly summary of 2025"
{ "start": "2025-01-01T00:00:00Z", "end": "2026-01-01T00:00:00Z", "granularity": "month" }

get_timeline_graph

Returns a graph of memories: nodes (memories sorted by event_at) plus edges. Edges are either:

  • Auto-derived temporal: happened_after — connecting memories whose events were close in time (configurable window)
  • Auto-derived co-occurrence: happens_within — near-simultaneous events (include_simultaneous)
  • Optional semantic: existing relation edges (derived_from, mentions, …) traversed from the relation graph (include_semantic_edges)

Use this tool to render a timeline as a network with D3, Graphviz, etc.

Input

FieldTypeRequiredDescription
start / endstringTime window.
granularitystringhour, day, week, month, none.
bankstring
topicsarray of stringFilter by topics.
max_results_per_bucketinteger
include_derivedbooleanInclude L1+ abstractions.
orderstringasc / desc.
max_depthintegerSemantic-relation hops from each node (default 1, max 3).
relation_typesarray of stringWhitelist of semantic relation types to follow.
temporal_edge_window_secsintegerWindow for happened_after edges (default 86400 = 1 day).
include_simultaneousbooleanAlso derive happens_within edges for near-simultaneous events.
simultaneous_window_secsintegerWindow for happens_within (default 60).
include_semantic_edgesbooleanInclude semantic-relation edges (default true).
user_id / agent_idstring

Output

{
  "success": true,
  "start": "...",
  "end": "...",
  "granularity": "day",
  "stats": {
    "node_count": 47,
    "edge_count": 92,
    "temporal_edge_count": 80,
    "semantic_edge_count": 12
  },
  "nodes": [
    { "id": "uuid", "event_at": "...", "layer": 0, "bucket": "2026-02-14", "memory": {} }
  ],
  "edges": [
    { "source": "uuid-a", "target": "uuid-b", "type": "happened_after", "delta_secs": 3600 }
  ]
}

get_context_resume

Progressive precision: the most recent window returns L0 memories at full detail; progressively older windows return higher-layer abstractions (L1 summaries, L2 links, L3 concepts). Produces an exponential decay curve where precision peaks at the current time. Use this as the first call when resuming a session.

Input

FieldTypeRequiredDescription
endstringISO 8601 end of window. Default: now.
lookbackstringTotal lookback from end, e.g. 30d (default), 12h, 1w.
decay_factorfloatEach segment is ~N× the previous one’s duration. Default 2.0.
segmentsintegerNumber of precision tiers (1-10). Default 5.
max_per_segmentintegerCap memories per segment. Default 20.
bankstring
topicsarray of stringFilter by topics.
user_id / agent_idstring

Output

{
  "success": true,
  "start": "...",
  "end": "...",
  "total_lookback_secs": 2592000,
  "decay_factor": 2.0,
  "segment_count": 5,
  "total_memories": 87,
  "segments": [
    {
      "label": "1d 06-03 — 06-04",
      "layer": 0,
      "start": "...",
      "end": "...",
      "duration_secs": 86400,
      "count": 12,
      "memories": [ /* L0 full detail */ ]
    }
  ]
}

Segment 0 (most recent) queries L0, segment 1 queries L1, and so on — newer segments have full detail, older ones are progressively compressed.

When to use which

I want to…Use
Quick “what happened recently” listget_timeline
Visualize memories as a temporal graphget_timeline_graph
Get a “context resume” of a long time period, in bounded sizeget_context_resume
Group by hour for fine-grained analysisget_timeline with granularity: "hour"
See causal/temporal relations between memoriesget_timeline_graph with include_semantic_edges: true

Next

Navigation & relations

Tools for moving through the memory pyramid and managing relations between memories.

Move up and down the abstraction hierarchy from a seed memory. zoom_out returns higher-layer (more abstract) memories derived from it; zoom_in returns lower-layer (more detailed) source memories it was abstracted from; both returns both directions.

Input

FieldTypeRequiredDescription
memory_idstring✓The seed memory.
directionstringzoom_in, zoom_out, or both (default).
levelsintegerLevels to traverse for zoom_in (1-5, default 1).
bankstring

Output

{
  "success": true,
  "source_memory_id": "uuid",
  "source_layer": 1,
  "zoom_in": [ /* lower-layer source memories */ ],
  "zoom_out": [ /* higher-layer abstractions */ ]
}

Examples

// From a JWT note, find the auth-architecture concept built on top of it
{ "memory_id": "uuid-of-jwt-note", "direction": "zoom_out" }

// From a concept memory, find all source evidence
{ "memory_id": "uuid-of-concept", "direction": "zoom_in", "levels": 3 }

Note

To restrict a semantic search to a specific layer, use query_memory with pyramid_config (e.g. { "mode": "top_heavy" } or layer_weights), not navigate_memory.

update_memory

Update an existing memory’s content and/or append relations by ID.

Input

FieldTypeRequiredDescription
memory_idstring✓
contentstringNew content. Re-embeds the memory.
relationsarrayRelations to append, as [{ "relation": "...", "target": "..." }].
bankstring

Output

{
  "success": true,
  "message": "Memory updated"
}

Warning

Updating content re-embeds the memory. This is an LLM-free operation (uses the same embedding model), but it does cost time and changes the semantic identity. Use it for typo fixes, not wholesale rewrites.

Manually create a relation between two existing memories. The reverse relation is created automatically (e.g. references → referenced_by).

Input

FieldTypeRequiredDescription
source_idstring✓Source memory ID (the “from” side).
relationstring✓Relation type: references, contradicts, supports, depends_on, part_of, extends, similar_to, summary_of, synthesizes.
target_idstring✓Target memory ID (must be an existing memory).
strengthnumberRelation strength 0.0-1.0 (default 1.0).
bankstring

Output

{
  "success": true,
  "message": "Relation created",
  "data": {
    "source_id": "uuid",
    "relation": "depends_on",
    "target_id": "uuid",
    "reverse_relation": "depended_on_by",
    "reverse_created": true
  }
}

Note

source_id and target_id must both be valid UUIDs of existing memories. Hierarchical relations (summary_of, part_of, synthesizes) require the source to be at a higher layer than the target. Duplicate links are rejected.

Suggested relation predicates

CategoryPredicates
Structuralchunk_of, summary_of, part_of, next_chunk, previous_chunk
Semanticreferences, extends, supersedes, depends_on, implements, configures
Layeredderived_from, emerges_from, instance_of, broader_than
Provenanceauthored_by, cited_by
Contradictioncontradicts

remove_relation

Remove a specific relation from a memory. The reverse relation on the target is removed automatically.

Input

FieldTypeRequiredDescription
memory_idstring✓Memory to remove the relation from.
relation_typestring✓The relation type to remove (e.g. references).
target_idstring✓The target memory ID in the relation.
bankstring

Output

{
  "success": true,
  "message": "Relation removed",
  "data": {
    "memory_id": "uuid",
    "removed_relation": "references",
    "removed_target": "uuid",
    "reverse_relation": "referenced_by",
    "reverse_cleaned": true
  }
}

Relations vs auto-linking

When auto-linking is enabled (via auto_link_threshold), the server automatically creates references relations to semantically similar existing memories. force_link is for cases where:

  • You know of a relation the auto-linker missed
  • You want a non-semantic relation (e.g. next_chunk, part_of)
  • You want to control the relation strength explicitly

Next

Abstraction pipeline

Tools for controlling the background workers that build higher layers of the pyramid from lower-layer content.

start_abstraction_pipeline

Start (or resume) the background workers. The pipeline auto-starts on server boot, so this is mainly useful after stop_abstraction_pipeline.

Input

None.

Output

{
  "success": true,
  "message": "Abstraction pipeline started"
}

stop_abstraction_pipeline

Pause the background workers. Useful before bulk operations or for debugging.

Input

None.

Output

{
  "success": true,
  "message": "Abstraction pipeline stopped"
}

Note

stop_abstraction_pipeline does not cancel an in-flight LLM call — it just stops scheduling new work. Requests already in progress will complete.

trigger_abstraction

Run one-shot abstraction processing now (unlike the pipeline, this runs once and does not start background workers). The pipeline must already be running.

Input

FieldTypeRequiredDescription
target_layerinteger1 = L0→L1 (summaries), 2 = L1→L2, 3 = L2→L3, 0 = all. Default 1.

Output

{
  "success": true,
  "l0_to_l1_created": 3,
  "l1_to_l2_created": 1,
  "l2_to_l3_created": 0,
  "errors": []
}

create_abstraction

Create a manual abstraction (L1/L2/L3) from specific source memory IDs, with your own content.

Input

FieldTypeRequiredDescription
contentstring✓The abstraction content (summary, synthesis, or concept).
source_idsarray of string✓Source memories this abstraction derives from.
target_layerinteger✓Target layer (1=structural, 2=semantic, 3=concept). Must be higher than all source layers.
relation_typestringRelation to sources. Defaults by layer: summary_of (L1), synthesizes (L2), abstracts_to_concept (L3+).
user_id / agent_idstring
bankstring

Output

{
  "success": true,
  "message": "Abstraction created",
  "data": {
    "memory_id": "uuid",
    "target_layer": 2,
    "relation_type": "synthesizes",
    "source_count": 3,
    "reverse_relation": "synthesized_by",
    "reverse_created": true
  }
}

What the pipeline actually does

The pipeline is a set of background workers, one per layer transition:

WorkerTriggerWhat it produces
L0 → L1When total L0 content exceeds auto_summary_threshold bytesL1 summaries grouped by topic. relation: "summary_of" from summary to sources.
L1 → L2When multiple L1 summaries share themesL2 connections with relation: "related_to", references, etc.
L2 → L3When L2 clusters formL3 concepts with relation: "emerges_from".
L3 → L4+When L3 concepts span domainsL4+ insights / mental models.

Each worker:

  1. Selects candidate source memories
  2. Asks the LLM to synthesize
  3. Stores the new abstraction
  4. Creates the derived_from / emerges_from / etc. relations
  5. Notifies the next-layer worker (which may decide to act)

Workers respect session_token_budget and circuit-breaker around the LLM. Failed abstractions get retried with exponential backoff; persistent failures get logged and skipped (use clear-backoff from the CLI to force retry).

How to use it

In normal operation, you don’t. The pipeline runs and gradually builds higher layers as content accumulates.

You intervene when:

  • You just stored something important and want insights now → trigger_abstraction
  • You want to see the architecture before doing bulk ingestion → stop_abstraction_pipeline, do the ingest, start_abstraction_pipeline
  • You want a specific abstraction on demand → create_abstraction with explicit source IDs and content
  • The pipeline is failing repeatedly → clear-backoff (CLI), then trigger_abstraction to debug

Next

Banks & backups

Tools for managing multiple isolated memory banks, backing them up, and cleaning up resources.

list_memory_banks

List all memory banks in the configured banks_dir.

Input

None.

Output

{
  "success": true,
  "data": {
    "banks": [
      {
        "name": "default",
        "memory_count": 1234,
        "size_bytes": 45678901,
        "created_at": "..."
      },
      {
        "name": "research",
        "memory_count": 567,
        "size_bytes": 12345678,
        "created_at": "..."
      }
    ],
    "count": 2,
    "banks_dir": "/var/lib/llm-mem/banks"
  }
}

create_memory_bank

Create a new bank. The bank is also created implicitly on the first write to a new name, so this is mostly for documentation or to pre-create a known structure.

Input

FieldTypeRequiredDescription
namestring✓Bank name. Allowed chars: letters, digits, -, _.
descriptionstringHuman-readable description.

Output

{
  "success": true,
  "message": "Memory bank 'research' ready",
  "data": {
    "bank": {
      "name": "research",
      "memory_count": 0,
      "size_bytes": 0
    }
  }
}

rename_memory_bank

Atomically rename a bank. Moves both the memory database and the session database.

Input

FieldTypeRequiredDescription
old_namestring✓
new_namestring✓

Output

{
  "success": true,
  "message": "Bank renamed from 'foo' to 'bar'",
  "data": { "old_name": "foo", "new_name": "bar" }
}

Warning

If any AI client is currently using the old name, it’ll see a “bank not found” error after the rename. Update the client config (or recreate the bank) before continuing.

backup_bank

Create a versioned backup of a bank. Backups are SHA-256 verified and timestamped.

Input

FieldTypeRequiredDescription
namestringBank to back up. Default: default.
destinationstringDirectory for the backup. Default: ~/llm-mem-backups/.

Output

{
  "success": true,
  "message": "Bank 'default' backed up successfully (v3)",
  "data": {
    "backup_path": "/home/you/llm-mem-backups/default-2026-02-15-v3.db",
    "manifest": {
      "version": 3,
      "created_at": "2026-02-15T...",
      "memory_count": 1234,
      "sha256": "...",
      "size_bytes": 45678901
    }
  }
}

restore_bank

Restore a bank from a backup file. Two modes:

  • replace (default) — overwrite the current bank. Requires confirm: true as a safety check.
  • merge — import the backup into the existing bank. Duplicates are skipped by content hash.

Input

FieldTypeRequiredDescription
namestringBank to restore into. Default: default.
sourcestring✓Path to the backup .db file.
modestringreplace (default) or merge.
confirmbooleanRequired: true for replace mode.

Output

{
  "success": true,
  "message": "Bank 'default' restored from backup (replace mode)",
  "data": {
    "restored_path": "/var/lib/llm-mem/banks/default.lance",
    "source": "/path/to/backup.db"
  }
}

For merge mode, the response also includes imported, skipped_duplicates, and total_after_merge.

cleanup_resources

Delete models or memory banks. Two target values:

  • "models" — delete the entire models_dir (then re-create it empty). The next run will re-download.
  • "banks" — delete a specific bank by name. Requires the literal confirmation phrase.

Input

FieldTypeRequiredDescription
targetstring"models" (default) or "banks".
namestring(for target: "banks")Bank to delete.
confirmboolean or stringFor target: "models": confirm: true. For target: "banks": confirm: "I confirm this data will be permanently lost".

Output

{
  "success": true,
  "message": "Models directory cleaned up successfully. ..."
}

[!DANGER] Destructive —

cleanup_resources with target: "banks" permanently deletes the bank’s database. There is no undo. The confirmation phrase is a safety net — never bypass it.

Backup strategy recommendations

Use caseRecommendation
Personal use, low churnWeekly backup, keep last 4.
Multi-project, activeDaily backup, keep last 7.
Production / sharedContinuous backup, off-site storage, restore tested.

Backups are stored as .db files plus a manifest. They include the bank database but not the documents in the upload session state. To capture document sessions too, use db export --include-sessions (CLI).

Next

Document upload

Tools for ingesting whole files (markdown, code, PDFs, spreadsheets, images, etc.) into memory with automatic chunking and abstraction.

upload_document

Upload a file from the server’s local filesystem. The server reads the file, splits it into chunks, embeds each chunk, and stores them as L0 memories. The L0→L1 worker then creates summaries in the background.

Input

FieldTypeRequiredDescription
file_pathstring✓Absolute path to the file. Must be readable by the server process.
file_namestringOptional name (defaults to the basename of file_path).
mime_typestringOptional MIME type (default text/plain).
chunk_sizeintegerOverride document_chunk_size.
process_immediatelybooleanStart processing after upload (default true).
topicsarrayTopics applied to all chunks.
contextarrayContext tags applied to all chunks.
event_atstringISO 8601 for when the document’s events occurred. Defaults to upload time.
bankstring
user_id / agent_idstring

Output

The response is not schema-constrained. A representative shape:

{
  "success": true,
  "data": {
    "session_id": "uuid",
    "file_name": "architecture.md",
    "chunks_stored": 23,
    "memory_ids": ["uuid-a", "uuid-b", "…"]
  }
}

Limits

upload_document is best for small to medium files (up to a few MB). For very large files (10s of MB+) or files you already have in memory, use the ingest tool, or the CLI’s multi-part upload flow (begin-upload / upload-part / process-document, see CLI commands).

ingest

Universal decomposition entry point. Takes raw content (a string, or base64 for binary) and automatically detects the format, decomposes it into semantic L0 chunks, creates structural relations between chunks, and optionally auto-links to existing memories.

Unlike upload_document, this does not read a file from disk — you pass the content directly. For binary formats (PDF, DOCX, images), base64-encode the content and set content_encoding: "base64".

Input

FieldTypeRequiredDescription
contentstring✓Raw content. For binary formats, base64-encoded.
content_encodingstring"base64" for binary formats.
format_hintstringOptional hint: markdown, json, yaml, toml, text.
file_namestringFor extension-based detection and provenance.
bankstring
auto_linkbooleanAuto-link chunks to existing memories (default true).
generate_abstractionsbooleanGenerate L1+ interpretations (default true).
max_chunk_sizeintegerMax characters per L0 chunk (default 2000).
metadataobjectMetadata attached to all chunks.
sourcestringOverride the auto-derived source (e.g. a URL or DOI).
describe_imagesbooleanGenerate AI descriptions for images (default true; requires a vision-capable LLM).

Output

{
  "success": true,
  "message": "…",
  "data": {
    "status": "success",
    "session_id": "uuid",
    "format": "markdown",
    "detected_mime": "text/markdown",
    "byte_size": 45678,
    "l0_chunks": [
      { "id": "chunk-0", "memory_id": "uuid", "node_type": "text", "content_preview": "…", "char_count": 1980, "order": 0 }
    ],
    "relations": [
      { "source_chunk_id": "chunk-0", "target_chunk_id": "chunk-1", "relation": "next_chunk", "strength": 1.0 }
    ],
    "issues": [],
    "warnings": [],
    "format_hints_available": ["markdown", "text"]
  }
}

document_status

Check the status of a document upload session.

Input

FieldTypeRequiredDescription
session_idstringSpecific session. If omitted, lists all sessions.
bankstring

Output (specific session)

{
  "success": true,
  "data": {
    "session_id": "uuid",
    "file_name": "large.pdf",
    "status": "processing",
    "chunks_processed": 42,
    "chunks_total": 100,
    "started_at": "...",
    "completed_at": null,
    "error": null
  }
}

cancel_document

Cancel an in-flight upload or processing session.

Input

FieldTypeRequiredDescription
session_idstring✓
bankstring

Output

{
  "success": true,
  "message": "Session 'uuid' cancelled",
  "data": { "session_id": "uuid" }
}

Document storage

Documents themselves are not stored in the bank. Only the chunks (after parsing) and the session state are kept. To keep the original file, store it elsewhere or use the db export flow to capture both the bank and the session database.

Next

Introspection

Tools for inspecting the server’s health, configuration, and database integrity.

system_status

Tip

Call this first —

The recommended best practice is to call system_status at the start of every session. It returns backend info, model availability, and token usage stats so you know what you’re working with.

Input

None.

Output

{
  "success": true,
  "data": {
    "backend": "local",                  // "local", "api", or mixed
    "state": "ready",                    // "ready" / "loading" / "error"
    "llm_model": "gemma-4-E2B-it-Q8_0.gguf",
    "embedding_model": "all-MiniLM-L6-v2",
    "llm_available": true,
    "embedding_available": true,
    "total_llm_calls": 42,
    "total_embedding_calls": 128,
    "total_prompt_tokens": 56789,
    "total_completion_tokens": 12345,
    "details": { /* backend-specific extra info */ }
  }
}

details is opaque per backend. Local backend includes things like GPU layer count, model size, llama.cpp backend info. API backend includes things like last API call duration, error rate.

health_check

Run a full end-to-end check. Static checks (config validity, build features, API key presence, banks dir writability) are always free. Live checks (live: true) also issue a tiny embed("ping") and complete("...pong") against the backend to confirm it actually responds. Live checks consume a few API tokens per call.

Input

FieldTypeRequiredDescription
livebooleanDefault false. If true, run live backend probes.
embed_onlybooleanLive mode only: only probe the embedding endpoint.
llm_onlybooleanLive mode only: only probe the LLM endpoint.
embed_timeout_secsintegerDefault 15.
llm_timeout_secsintegerDefault 30.

Output

{
  "success": true,
  "data": {
    "healthy": true,
    "backend": "local",
    "llm_model": "...",
    "embedding_model": "...",
    "live_run": true,
    "checks": [
      {
        "name": "config_valid",
        "category": "config",
        "status": "pass",
        "detail": "All required fields present",
        "duration_ms": 1
      },
      {
        "name": "build_features_match",
        "category": "build",
        "status": "pass",
        "detail": "local-llm and local-embed features enabled",
        "duration_ms": 0
      },
      {
        "name": "live_embed",
        "category": "live",
        "status": "pass",
        "detail": "Embedded 'ping' to dim 384 in 234ms",
        "duration_ms": 234
      },
      {
        "name": "live_llm",
        "category": "live",
        "status": "pass",
        "detail": "Completed '...pong' in 1.2s",
        "duration_ms": 1203
      }
    ]
  }
}

healthy: false means at least one check failed. Inspect the checks array for which one and the detail field for the reason.

This tool is also available as a CLI subcommand: llm-mem health-check [--live].

check_consistency

Scan one or all banks for integrity issues:

  • Orphaned abstractions (L1+ with no L0 source)
  • Stale state (sessions stuck in processing past their timeout)
  • Missing embeddings
  • Hash mismatches (content changed since embedding was generated)
  • Unreferenced forgotten memories
  • Duplicate content
  • Invalid layer structure

The companion CLI command is llm-mem db check [--all] [--verbose].

Input

FieldTypeRequiredDescription
bankstringSpecific bank. If omitted, checks all.
verbosebooleanInclude detailed issue info. Default false.

Output

{
  "success": true,
  "data": {
    "issues": [
      {
        "type": "orphaned-abstraction",
        "memory_id": "uuid",
        "severity": "warn",
        "detail": "L2 memory references missing L0 source"
      }
    ],
    "issue_count": 1,
    "bank": "default"
  }
}

To fix issues, use the CLI: llm-mem db fix [--dry-run] [--purge].

help

Returns the embedded usage guide that the MCP server exposes to AI clients. Includes best practices, the layered memory architecture explanation, memory type vocabulary, and tips.

Input

None.

Output

{
  "success": true,
  "data": {
    "guide": {
      "overview": "...",
      "layered_memory_architecture": { /* L0 - L4+ */ },
      "domain_patterns": { /* codebase, documents, web, conversations */ },
      "banks_and_user_id": { /* when to use banks vs user_id */ },
      "critical_guidelines": [ /* ... */ ],
      "tips": [ /* ... */ ]
    }
  }
}

This is the same content an AI client sees when it calls the MCP prompts/list endpoint. The AI is expected to read this and adapt its tool-use strategy accordingly.

Debugging flow

  1. system_status — is the server even ready?
  2. health_check (with live: true) — is the backend actually responsive?
  3. check_consistency — is the database intact?
  4. Check the log file for errors
  5. Run the same tool manually from the CLI to isolate MCP from application issues

Next

Standalone CLI

The llm-mem binary (no -mcp suffix) is a standalone command-line interface for working with memory banks directly, without an MCP client. It reads the same config file, talks to the same backend, and operates on the same banks as the MCP server.

llm-mem [OPTIONS] [COMMAND]

When to use it

  • Local testing and debugging — try a query without spinning up an MCP client
  • Scripts and automation — pipe search results into other tools
  • Database administration — exports, merges, consistency checks, renames
  • Configuration management — generate a template config, run health checks
  • Interactive exploration — drop into the REPL for ad-hoc work

Modes

FlagBehavior
(no args)Start the interactive REPL
--singleRun a single command and exit. Useful for scripts.
--batch <file>Execute commands from a file (one per line). See Batch mode.
--replForce REPL mode even if subcommand is present.
--config <path>Config file path (auto-discovered if not given).
--banks-dir <path>Override vector_store.banks_dir.

All commands at a glance

Document handling

  • upload — ingest a file
  • begin-upload / upload-part / process-document — multi-part upload for large files
  • doc-status / list-sessions — check upload progress

Browsing

  • list — list memories with filters
  • show — full details of a memory by ID
  • search — text or semantic search
  • stats / layer-stats / layer-tree — bank and pyramid statistics
  • timeline / timeline-graph / context-resume — chronological views
  • dag-export — interactive DAG visualization (HTML)

Configuration

  • generate-config — write a config template
  • health-check — run static + optional live config check
  • system-status — backend health

Database admin

  • db export / db import — backup and restore
  • db merge — merge banks
  • db check / db fix — integrity check and repair
  • db rename — atomic rename
  • db export-jsonl / db import (from JSONL) — backend-independent format
  • db compact — merge fragments (also prunes old versions)
  • db prune — reclaim disk space from old LanceDB versions

Misc

  • list-banks — list all banks
  • list-devices — show available embedding/LLM devices
  • export — bank export to JSON
  • metrics — query and cache metrics
  • clear-backoff — reset abstraction retry timers
  • viz — real-time TUI for processing

REPL-only

Inside the REPL, you also get use <bank> (switch active bank) and savelog (dump log buffer). Output can be formatted as table, detail, json, jsonl, or csv.

Examples

# Interactive REPL
llm-mem

# Single-command scripting
llm-mem --single search --query "JWT auth" --mode semantic --limit 5
llm-mem --single list --bank research --limit 20 --format json
llm-mem --single stats --bank default

# Database admin
llm-mem db check --all
llm-mem db fix --bank research --purge
llm-mem db export --bank default --output backups/default.db --include-sessions
llm-mem db prune --all --older-than-days 0    # reclaim disk from old versions

# Visualize
llm-mem dag-export --bank default --output graph.html --max-nodes 300

# Health & config
llm-mem health-check --live
llm-mem generate-config --output /etc/llm-mem/config.toml

Output formats

Most commands accept --format with one of:

ValueBest for
tableHuman reading (default for many commands)
detailSingle-memory views (show)
jsonProgrammatic consumption
jsonlStreaming, line-by-line
csvSpreadsheets / awk

Next

Interactive REPL

Drop into a read-eval-print loop for ad-hoc work. The REPL has command history, tab completion, and the same commands as single-command mode.

Starting

llm-mem
# or
llm-mem --repl

The first run shows:

llm-mem interactive CLI
Type 'help' for available commands, 'exit' to quit
>

Available commands

Inside the REPL, the same commands as single-command mode work, without the -- prefix. A few extras:

CommandWhat it does
helpList all commands with brief descriptions.
exit / quit / Ctrl+DLeave the REPL.
use <bank>Switch the active bank. Subsequent commands operate on it until you switch again.
savelog <path>Dump the captured log buffer to a file. Useful for debugging.
clearClear the screen.

Example session

> use research
Switched to bank 'research'

> search "JWT auth" --mode semantic --limit 3
┌────┬──────────────────────────────────┬────────┬──────┐
│ id │ content                          │ layer  │ score │
├────┼──────────────────────────────────┼────────┼──────┤
│ a1 │ AuthService handles JWT validation│ L0     │ 0.89  │
│ b2 │ Refresh tokens expire after 7 days│ L0     │ 0.82  │
└────┴──────────────────────────────────┴────────┴──────┘

> show a1
ID:        a1
Layer:     0
Type:      factual
Created:   2026-02-15 11:23:45
Event at:  2026-02-10
Content:   AuthService handles JWT token validation and refresh — entry point is src/auth/service.rs:45-80
Context:   project-x, auth
Topics:    jwt, validation
Metadata:  source=src/auth/service.rs:45-80
Relations:
  - depends_on: jwt-secret-rotation
  - derived_from: arch-decision-2026-02-10

> use default
Switched to bank 'default'

> stats
Bank:      default
Memories:  1234
Size:      45.6 MB
Layers:    L0: 1100, L1: 100, L2: 30, L3: 4
Banks:     2 (default, research)

> exit
Goodbye.

Tab completion

Type the start of a command and press Tab to complete. Tab also completes bank names and other enums.

History

The REPL uses rustyline, so arrow keys (up/down) navigate history and Ctrl+R does reverse search. History is persisted to ~/.llm-mem_history (a plain text file you can grep).

Log capture

While the REPL runs, the server emits logs to a buffer. You can dump them with:

> savelog debug.log

The buffer captures everything down to TRACE level, even if the visible stderr log is at INFO. Useful for diagnosing issues without restarting with RUST_LOG=trace.

Custom prompts

The default prompt is > . To customize, set the LLM_MEM_PROMPT env var before launching the REPL.

Next

Single commands

Reference for every CLI subcommand. The same commands work inside the REPL (without the -- prefix).

The convention is llm-mem --single <command> [flags] or just llm-mem <command> [flags] from the shell.

upload

Ingest a file with auto-chunking and processing.

llm-mem upload --file-path /path/to/doc.md --bank project-x
llm-mem upload --file-path /path/to/code.rs --context src
FlagTypeDefaultDescription
--file-path <PATH>pathrequiredFile to ingest.
--bank <NAME>stringdefaultTarget bank.
--process-immediatelybooltrueStart background processing now.
--chunk-size <N>integerconfigOverride document_chunk_size.
--context <TAG>string[]noneRepeatable.
--format <FMT>enumtabletable, json, jsonl, csv.

begin-upload / upload-part / process-document

Three-step upload for very large files.

# Step 1: start the session
llm-mem begin-upload --file-name huge.pdf --total-size 104857600 --bank research

# Step 2: send parts (repeat for each part)
llm-mem upload-part --session-id <id> --part-index 0 --file-path part0.bin
llm-mem upload-part --session-id <id> --part-index 1 --file-path part1.bin
# ...

# Step 3: finalize
llm-mem process-document --session-id <id>

Use --partial-closure on process-document if you can’t send all expected parts (e.g. upload was interrupted and you want to proceed with what you have).

doc-status / list-sessions

Check upload progress.

llm-mem doc-status --session-id <id>
llm-mem list-sessions --bank research

list

Browse memories with filters.

llm-mem list --bank default --limit 20
llm-mem list --bank research --limit 50 --format jsonl > research.jsonl
FlagTypeDefaultDescription
--bank <NAME>stringdefault
--limit <N>integer50
--format <FMT>enumtable

show

Full details of a memory.

llm-mem show --bank default --memory-id <uuid>
llm-mem show --memory-id <uuid> --format detail

Text or semantic search.

llm-mem search --query "JWT auth" --mode semantic --limit 5
llm-mem search --query "TODO" --mode text --case-insensitive --show-scores
FlagTypeDefaultDescription
--query <TEXT>stringrequired
--mode <MODE>enumtexttext (fast, no LLM) or semantic (embedding-based).
--bank <NAME>stringdefault
--limit <N>integer10
--case-insensitiveboolfalseText mode only.
--show-scoresboolfalseText mode only.
--threshold <F>floatconfigOverride search_similarity_threshold.
--format <FMT>enumtable

stats / layer-stats / layer-tree

Bank and pyramid statistics.

llm-mem stats --bank default
llm-mem layer-stats --bank default
llm-mem layer-tree --bank default --max-depth 5 --show-ids

layer-tree prints an ASCII tree of the pyramid.

timeline / timeline-graph / context-resume

Chronological views.

llm-mem timeline --bank default --since 2d --granularity day
llm-mem timeline --bank default --start 2026-01-01T00:00:00Z --end 2026-02-01T00:00:00Z --granularity week
llm-mem timeline-graph --bank default --since 1w --include-semantic-edges
llm-mem context-resume --bank default --since 30d --decay-factor 4.0 --segments 5

See Tools chronology for the conceptual reference; the CLI exposes the same parameters.

generate-config

Write a commented config template.

llm-mem generate-config --output /etc/llm-mem/config.toml

health-check

Validate the config and (optionally) probe the live backend.

llm-mem health-check                   # static checks only
llm-mem health-check --live            # + tiny embed + completion
llm-mem health-check --live --embed-only
llm-mem health-check --live --llm-timeout-secs 60

This is the CLI equivalent of the health_check MCP tool. Use it standalone when debugging config issues before connecting an MCP client.

system-status

Backend health and token usage.

llm-mem system-status
llm-mem system-status --format json

list-banks

Show all memory banks.

llm-mem list-banks
llm-mem list-banks --format json

list-devices

Show available embedding/LLM devices (useful for GPU config).

llm-mem list-devices

export

Export a bank to JSON (different from db export — this is for application-level use).

llm-mem export --bank default --output default.json --pretty

metrics

Show accumulated query and cache metrics. Pass --reset to zero them after display.

llm-mem metrics
llm-mem metrics --reset

clear-backoff

Reset abstraction retry timers. Use after fixing a transient backend issue.

llm-mem clear-backoff --bank default
llm-mem clear-backoff --bank default --layer 1

viz

Launch a real-time TUI for visualizing document processing and abstraction pipeline activity.

llm-mem viz

dag-export

Export the memory bank as an interactive DAG visualization — a self-contained HTML file you can open in any browser. Nodes are memories colored by abstraction layer; edges represent abstraction, semantic, and temporal relations.

# Default: up to 200 most important nodes
llm-mem dag-export --bank default --output graph.html

# Customize node count and filtering
llm-mem dag-export --bank research --output research-graph.html --max-nodes 300 --min-importance 0.5
llm-mem dag-export --bank default --output l3-above.html --min-layer 3

# Disable certain edge types for a cleaner view
llm-mem dag-export --bank default --output no-temp.html --no-temporal
FlagTypeDefaultDescription
--bank <NAME>stringdefaultMemory bank to export.
--output <PATH>pathrequiredOutput HTML file path.
--max-nodes <N>integer200Maximum nodes in the graph.
--min-importance <F>float0.0Minimum importance score (0.0–1.0).
--semantic / --no-semanticbooltrueInclude semantic relation edges.
--temporal / --no-temporalbooltrueInclude temporal (happened_after) edges.
--abstraction / --no-abstractionbooltrueInclude abstraction pyramid edges.
--min-layer <N>integer-1Minimum layer level to include.
--max-layer <N>integer99Maximum layer level to include.
--min-relation-strength <F>float0.0Minimum strength for semantic edges (0.0–1.0).

Layered scaling

The graph automatically adjusts to the node count:

Total bank sizeStrategy
≤100 memoriesFull graph: all layers, all edges
100–500L1+ pyramid + high-importance L0 + strong edges
500–2,000L2+ concepts + abstraction hierarchy only
2,000–10,000L3+L4 wisdom/concepts only
>10,000Top-N by importance, pyramid only

Interactivity

Open the HTML file in a browser and use:

  • Scroll/pinch to zoom, drag to pan
  • Drag nodes to reposition them
  • Hover over a node for a content preview tooltip
  • Click a node to open the detail panel with full metadata and edge list
  • Toggle layers and edge types in the sidebar
  • Search bar to filter nodes by label, ID, or content
  • Sliders to adjust link distance and charge strength
  • Keyboard shortcuts: r = reset view, f = fit to screen, / = focus search, Esc = close detail panel

Next

Database management

The llm-mem db subcommand handles bank-level operations: export, import, merge, check, fix, rename, compact, prune.

db export

Export a bank to a portable .db file. The exported bank is ready for continued use — it’s a self-contained copy.

llm-mem db export --bank default --output backups/default.db
llm-mem db export --bank research --output research-portable.db --include-sessions
FlagTypeDescription
--bank <NAME>stringBank to export. Default: default.
--output <PATH>pathOutput file or directory.
--include-sessionsboolAlso copy the .sessions.db alongside.

db export-jsonl

Export a bank to a streaming JSONL text file. This format is backend-independent — a JSONL export from VectorLite can be imported into LanceDB and vice versa.

llm-mem db export-jsonl --bank default --output default.jsonl
llm-mem db export-jsonl --bank research --output research.jsonl --include-sessions

The JSONL file has:

  • An envelope header with format version, app version, embedding dimension, and config snapshot
  • One memory object per line in the body
  • A checksum footer

This makes it the safest format for archival and cross-backend migration.

db import

Import memories from a JSONL file into a bank.

llm-mem db import --bank default --input default.jsonl
llm-mem db import --bank default --input default.jsonl --dry-run    # preview only
llm-mem db import --bank default --input default.jsonl --strip-embeddings
FlagTypeDescription
--bank <NAME>stringTarget bank.
--input <PATH>pathThe JSONL file.
--strip-embeddingsboolDon’t import the embedding vectors; re-embed on import. Use when the new model has a different dimension.
--dry-runboolShow what would be imported without modifying anything.

The importer auto-handles dimension mismatches: if the JSONL has dimension 384 but your current model is 768, it’ll strip the old vectors and mark memories for re-embedding.

db merge

Merge one or more source banks (or .db files) into a target bank.

llm-mem db merge --source research --source notes --into combined --on-duplicate keep-newest
llm-mem db merge --source old-backup.db --into default --on-duplicate keep-first
llm-mem db merge --source a.lance --source b.lance --into merged --dry-run
FlagTypeDefaultDescription
--source <NAME_OR_PATH>string[]requiredRepeatable. Bank names or .db file paths.
--into <NAME>stringrequiredTarget bank name (created if missing).
--on-duplicate <STRATEGY>stringkeep-newestkeep-newest, keep-first, or keep-all.
--dry-runboolfalseShow what would be merged.

Duplicate detection uses content hash.

db check

Scan for consistency issues.

llm-mem db check --bank default
llm-mem db check --all
llm-mem db check --bank research --verbose
llm-mem db check --file external.db
FlagTypeDescription
--bank <NAME>stringSpecific bank.
--file <PATH>pathCheck an external .db file.
--allboolCheck every bank.
--verboseboolDetailed issue info.

Detects:

  • Orphaned abstractions (L1+ with no L0 source)
  • Stale state (sessions stuck in processing past their timeout)
  • Missing embeddings
  • Hash mismatches
  • Unreferenced forgotten memories
  • Duplicate content
  • Invalid layer structure

db fix

Repair detected issues.

llm-mem db fix --bank default --dry-run                    # preview
llm-mem db fix --bank default                              # fix all
llm-mem db fix --bank default --fix orphaned-abstractions  # fix specific types
llm-mem db fix --bank default --no-backup                  # skip auto-backup
llm-mem db fix --bank default --purge                      # hard-delete unreferenced Forgotten
FlagTypeDefaultDescription
--bank <NAME>stringdefault
--fix <TYPE>string[](all)Specific issue types. Valid: orphaned-abstractions, stale-states, missing-embeddings, hash-mismatches, unreferenced-forgotten, duplicate-content, invalid-layer-structure.
--dry-runboolfalseShow what would be fixed.
--no-backupboolfalseSkip automatic backup before fixing.
--purgeboolfalseHard-delete unreferenced Forgotten memories (vs. just marking them).

Warning

db fix is destructive for unreferenced-forgotten memories (with --purge) and for invalid-layer-structure. Always run with --dry-run first. The default behavior (without --no-backup) is to take a backup automatically.

db rename

Atomic rename of a bank. Moves both the memory database and the session database.

llm-mem db rename --old-name foo --new-name bar

db compact

Merge a bank’s small LanceDB fragments into larger ones for faster reads, and prune superseded versions. Compaction runs automatically every few writes, but this command forces it on demand.

llm-mem db compact --bank default
llm-mem db compact --all
FlagTypeDefaultDescription
--bank <NAME>stringdefaultSpecific bank.
--allboolfalseCompact every bank.

Compaction also triggers a safe prune (versions older than 7 days), so it reclaims disk space from accumulated old snapshots.

db prune

Delete old LanceDB dataset versions from disk to reclaim space.

LanceDB is copy-on-write: every add, update, delete, and compaction creates a new version snapshot and leaves the old version’s files in place. Without pruning, these superseded versions accumulate forever, causing unbounded disk bloat. This command deletes versions older than the cutoff.

# Reclaim all reclaimable space from a single bank (safe default)
llm-mem db prune --bank old-data

# Prune every bank
llm-mem db prune --all

# Keep the last 7 days of versions
llm-mem db prune --bank default --older-than-days 7

# Aggressive: also remove unverified files newer than 7 days (see warning)
llm-mem db prune --all --delete-unverified
FlagTypeDefaultDescription
--bank <NAME>stringdefaultSpecific bank.
--allboolfalsePrune every bank.
--older-than-days <N>integer0Keep versions newer than this many days. 0 removes everything eligible.
--delete-unverifiedboolfalseAlso delete files newer than 7 days that can’t be verified as committed.

Warning

--delete-unverified is only safe when no other process is reading or writing the dataset (e.g. the MCP server is stopped). Otherwise the dataset may be corrupted. Without this flag, LanceDB never deletes files newer than 7 days regardless of --older-than-days, so the default is always safe.

Tip

A safe one-off cleanup of bloated old databases:

llm-mem db prune --all --older-than-days 0

Common workflows

Migrate a bank to a new machine

# Source
llm-mem db export --bank default --output default.db --include-sessions
# scp default.db new-host:

# Destination
llm-mem db export --bank default --output /tmp/backup.db  # safety backup
# (stop the server, place default.db in the banks_dir, restart)

Cross-backend migration (e.g. VectorLite → LanceDB)

# Export as backend-independent JSONL
llm-mem db export-jsonl --bank old --output migration.jsonl

# On the new system with LanceDB configured:
llm-mem db import --bank new --input migration.jsonl

Recover from corruption

llm-mem db check --all --verbose                # identify
llm-mem db fix --bank default --dry-run         # preview
llm-mem db fix --bank default                   # repair (auto-backup)

If db fix can’t repair, restore from a backup created via backup_bank (MCP) or db export.

Next

Data directory layout

Everything llm-mem writes to disk lives under a few well-known directories. Understanding the layout helps with backups, migrations, and debugging disk space issues.

Top-level layout

After first run, you’ll see something like this in the working directory (or wherever you pointed LLM_MEM_MODELS_DIR / vector_store.banks_dir):

./llm-mem-data/
├── banks/                      # one .lance/ per memory bank
│   ├── default.lance/          # the default bank
│   ├── default.sessions.db     # document upload sessions for default
│   ├── research.lance/
│   └── research.sessions.db
├── models/                     # GGUF language model + ONNX embedding model
│   ├── gemma-4-E2B-it-Q8_0.gguf
│   └── (other model files)
└── logs/                       # when logging.enabled = true
    └── llm-mem-mcp.log

Plus the model cache (separate from llm-mem-data):

~/.cache/llm-mem/models/         # downloaded HF/fastembed files (when cache_model = true)

What’s in a .lance/ directory?

Each bank is a single LanceDB table. The directory is opaque to the user (it’s managed by the LanceDB library) but contains:

  • A memories.lance/ subdirectory with the actual vector data
  • _versions/ for transaction history
  • _latest.manifest and similar bookkeeping files

Don’t edit files inside a .lance/ directory manually. Use the db export / db import / db check / db fix tools to manipulate bank data safely.

What’s in .sessions.db?

A small SQLite file tracking document upload sessions:

  • session_id
  • file_name, file_size, mime_type
  • status (uploading, processing, completed, failed, cancelled)
  • Per-part metadata: part_index, md5sum, bytes_received
  • MD5 of the assembled file
  • Bank name

The sessions database is what enables resumable uploads: if the server crashes mid-upload, the next startup re-attaches to the incomplete session and resumes from the last received part.

The default bank

The default bank is created automatically on first run, even if you never explicitly call create_memory_bank. To pre-create it manually:

mkdir -p /var/lib/llm-mem/banks
# (the server populates the directory on first write)

Moving the data directory

To relocate the entire data tree:

  1. Stop the server
  2. Move the directory: mv ./llm-mem-data /new/path/llm-mem-data
  3. Update the config or set env vars:
    • LLM_MEM_MODELS_DIR=/new/path/llm-mem-data/models
    • vector_store.banks_dir=/new/path/llm-mem-data/banks
  4. Restart

Or just symlink:

ln -s /mnt/storage/llm-mem-data ./llm-mem-data

Disk space

Rough size estimates:

ComponentPer memoryNotes
Vector (384-dim f32)~1.5 KB+ small overhead
Metadata~1 KBdepends on usage
Sessions DBsmalla few MB total

100k memories ≈ 250 MB. 1M memories ≈ 2.5 GB.

If you need to free space:

  • cleanup_resources (target: models) — reclaims the model cache
  • cleanup_resources (target: banks) — deletes a specific bank
  • db export + delete — manual cleanup with a backup
  • db prune --all --older-than-days 0 — reclaim disk space from accumulated old LanceDB version snapshots
  • db compact --bank default — merge fragments (also prunes); helps if there’s significant Forgotten memory accumulation

Permissions

The server process needs read/write access to:

  • models_dir (default llm-mem-data/models/)
  • banks_dir (default llm-mem-data/banks/)
  • log_directory (default llm-mem-data/logs/)
  • The cache directory (~/.cache/llm-mem/models/, or whatever cache_dir is set to)

When running as a subprocess of an MCP client, it inherits the client’s user. Make sure the working directory is writable.

Next

Memory banks

Banks are isolated memory stores. Each bank is a separate database file with its own memories, sessions, and statistics. Memories in one bank are invisible to another.

Why use multiple banks?

Use caseStrategy
Per-projectOne bank per codebase, switch via the bank parameter or --bank flag
Per-topicOne bank for cooking, another for finance, another for travel
Per-environmentOne bank for dev, one for production data
Per-userIf you share a machine, separate banks per user
Per-purposeShort-term context (auto-cleaned) vs. long-term knowledge (persistent)

The default bank is always available — it’s created on first run. Use additional banks when you want hard isolation.

Naming

Bank names are free-form strings but follow these rules:

  • Allowed: letters, digits, -, _
  • Avoid spaces (use - or _)
  • The name default is reserved
  • Case-sensitive

Good names: project-x, research-2026, meeting-notes, personal.

Creating banks

Banks are created automatically on first write to a new name. The server handles this transparently:

// MCP call to add_content_memory
{
  "content": "Postgres over MongoDB for analytics",
  "bank": "project-x"     // <-- creates project-x if it doesn't exist
}

You can also pre-create with create_memory_bank (MCP) or mkdir + populate on disk.

Listing banks

# CLI
llm-mem list-banks
// MCP: list_memory_banks
{}

Output:

{
  "banks": [
    { "name": "default", "memory_count": 1234, "size_bytes": 45678901 },
    { "name": "research", "memory_count": 567, "size_bytes": 12345678 }
  ],
  "count": 2
}

Switching the active bank

MCP: pass bank: "<name>" on every call. There’s no global “active bank” concept at the protocol level.

CLI REPL: use <bank> switches the active bank for the session.

CLI single-command: --bank <name> per command.

Isolation guarantees

  • Cross-bank searches are impossible (a tool can only see one bank at a time)
  • Cross-bank relations don’t exist
  • The abstraction pipeline operates per-bank (L0→L1 in bank A doesn’t see content from bank B)
  • Backups and exports are per-bank

Soft grouping within a bank

For softer grouping without full isolation, use context tags and topics:

  • context — broad topic tags (e.g. ["project-x", "auth", "jwt"])
  • topics — specific subjects within the content

You can then filter by context/topic in queries:

// MCP: query_memory
{
  "query": "JWT auth",
  "context": ["project-x"]
}

Bank vs user_id

user_id is a per-memory filter within a bank. It’s optional and only useful when multiple users share a single bank.

In most cases:

  • Single user per bank: omit user_id
  • Multiple users per bank: set user_id to differentiate

If two users share a bank and you don’t filter by user_id, each will see the other’s memories. Use a separate bank if you want hard isolation per user.

Lifecycle

OperationToolCLI
Createcreate_memory_bank(auto, or mkdir)
Listlist_memory_bankslist-banks
Renamerename_memory_bankdb rename
Backupbackup_bankdb export
Restorerestore_bankdb import
Deletecleanup_resources (target: “banks”)rm -rf the bank directory

Next

Backups & restore

Banks are self-contained on disk. Backing up is just copying files; restoring is putting them back. The higher-level tools add versioning, checksums, and metadata.

Levels of backup

LevelWhat it capturesWhen to use
Raw file copyThe .lance/ directory as-isFast snapshots, machine-level backups
db exportA portable .db file with format metadataCross-version, cross-platform
db export-jsonlA streaming text file with envelope + checksumLong-term archival, cross-backend migration
backup_bank (MCP)A versioned backup with SHA-256 + manifestProgrammatic, scripted

For most users, db export-jsonl is the safest long-term choice because it’s backend-independent and human-readable.

Manual backup (file copy)

Stop the server, then:

# Backup everything
cp -a ./llm-mem-data /backup/llm-mem-data-$(date +%F)

# Or per-bank
cp -a ./llm-mem-data/banks/default.lance /backup/default-$(date +%F).lance
cp -a ./llm-mem-data/banks/default.sessions.db /backup/default-sessions-$(date +%F).db

Stop the server first — copying a LanceDB table mid-write can produce a corrupt backup. The .sessions.db SQLite file is durable, but a mid-transaction copy might lose the last few changes.

db export (CLI)

Creates a portable, versioned .db file:

llm-mem db export --bank default --output /backup/default.db
llm-mem db export --bank research --output /backup/research.db --include-sessions

The .db file includes:

  • A format version
  • A snapshot of the bank’s tables
  • An optional .sessions.db (with --include-sessions)

To restore:

# Stop the server
# Replace the bank directory with the contents of the .db
llm-mem db import --bank default --input /backup/default.db
# Restart

db import is the inverse — it can read .db files exported with db export and load them into a target bank.

Backend-independent streaming text format. The file looks like:

{"_format":"llm-mem-jsonl","_version":1,"_app_version":"0.1.0","_embedding_dim":384}
{"memory_id":"a1","content":"...","embedding":[...],"metadata":{...}}
{"memory_id":"b2","content":"...","embedding":[...],"metadata":{...}}
...
{"_checksum":"sha256:abc123...","_memory_count":1234}

To export:

llm-mem db export-jsonl --bank default --output /backup/default.jsonl

To import:

llm-mem db import --bank new --input /backup/default.jsonl
llm-mem db import --bank new --input /backup/default.jsonl --strip-embeddings   # if dim changed
llm-mem db import --bank new --input /backup/default.jsonl --dry-run            # preview

If the embedding dimension in the JSONL doesn’t match the current model, the importer strips the old vectors and re-embeds on import. This makes it safe to migrate to a new embedding model.

backup_bank (MCP tool)

For programmatic / AI-driven backups:

{
  "name": "default",
  "destination": "/home/you/llm-mem-backups"
}

Output:

{
  "success": true,
  "data": {
    "backup_path": "/home/you/llm-mem-backups/default-2026-02-15-v3.db",
    "manifest": {
      "version": 3,
      "created_at": "2026-02-15T...",
      "memory_count": 1234,
      "sha256": "...",
      "size_bytes": 45678901
    }
  }
}

Versioning is automatic — the tool keeps incrementing v1, v2, v3, … in the destination directory.

To restore: use restore_bank with mode: "replace" (requires confirm: true) or mode: "merge".

Backup retention

The backup_bank tool doesn’t have a built-in retention policy. For long-running setups, prune old backups with a cron job:

# Keep last 30 days
find /home/you/llm-mem-backups -name "*.db" -mtime +30 -delete

For db export / db export-jsonl outputs, do the same — they’re regular files.

Testing your backups

A backup you haven’t tested is a backup you don’t have. Periodically:

  1. Restore a backup to a temp directory
  2. Start the server with --banks-dir <temp dir>
  3. Run llm-mem health-check --live
  4. Run a few queries to confirm memories are intact
  5. Delete the temp directory

Cross-backend migration

To move from VectorLite to LanceDB (or vice versa):

# Old system
llm-mem db export-jsonl --bank old-bank --output migration.jsonl

# New system (with new backend configured)
llm-mem db import --bank new-bank --input migration.jsonl

The JSONL format is the only one that works across backends. .db files are backend-specific.

Next

GPU acceleration

By default, llm-mem runs on CPU. This works everywhere but is slow for the language model. GPU acceleration is optional but strongly recommended if you have a compatible GPU.

Supported platforms

PlatformGPUBuild feature
macOS (Apple Silicon)M1, M2, M3, M4metal
macOS (Intel)Intel iGPUmetal (limited)
Linux (AMD, Intel, NVIDIA)Any Vulkan-capablevulkan
Linux (NVIDIA only)NVIDIAcuda
Windows (AMD, Intel, NVIDIA)Any Vulkan-capablevulkan
Windows (NVIDIA only)NVIDIAcuda

Vulkan is the most portable GPU backend. CUDA is fastest for NVIDIA hardware. Metal is the only option on macOS.

Building with GPU support

Default (CPU only)

cargo build --release

Vulkan

# Linux
cargo build --release --features local,vulkan

# Windows
cargo build --release --features local,vulkan

Requires the Vulkan SDK and a Vulkan-capable driver installed on the system at build time.

CUDA (NVIDIA only)

cargo build --release --features local,cuda

Requires CUDA toolkit + a compatible NVIDIA driver.

Metal (macOS)

cargo build --release --features local,metal

Enabling GPU at runtime

After building with the right feature, tell the runtime how many model layers to offload:

[llm]
gpu_layers = 20     # 0 = CPU only; set to 20+ for GPU

gpu_layers means how many transformer layers run on the GPU. The full model has a fixed number of layers (e.g. Gemma 4 E2B has ~26 layers). Setting gpu_layers = 20 offloads 20 of 26 layers; the rest stay on CPU.

ValueBehavior
0CPU only
Small (e.g. 5)Partial offload — useful if your VRAM is limited
Number of model layers (e.g. 26 for Gemma 4 E2B)Full offload — all on GPU
Larger than model layersCapped at model layer count

If you have less VRAM than the model needs, set gpu_layers to fit. The server will run the offloaded layers on GPU and the rest on CPU. There’s no all-or-nothing requirement.

Verifying GPU usage

In the log file (with logging.level = "debug"), look for:

INFO llama.cpp: loading model with 20 GPU layers

The system_status tool also reports backend details:

{
  "backend": "local",
  "details": {
    "n_gpu_layers": 20,
    /* ... */
  }
}

For the CLI:

llm-mem list-devices    # shows available devices
llm-mem system-status   # shows current backend and layer count

Performance expectations

Speedups depend on hardware. Rough numbers for Gemma 4 E2B at 16K context:

HardwareApprox tokens/sec
Apple M1 CPU5-8
Apple M1 GPU (Metal)20-35
Apple M3 Pro GPU40-60
NVIDIA RTX 3060 (CUDA)30-50
NVIDIA RTX 4090 (CUDA)100-150
AMD RX 7900 XT (Vulkan)50-80
Intel i7-12700 CPU4-7

These are approximate — actual numbers depend on prompt length, model size, and driver version. The point is: GPU is 5-20x faster than CPU.

AppImage and GPU

The pre-built AppImage includes both Vulkan and CUDA backends, dynamically loaded. To use GPU:

  1. Install the Vulkan loader (libvulkan1 on Debian/Ubuntu, vulkan-tools on Fedora) OR the CUDA runtime
  2. Set gpu_layers in your config
  3. Make sure /dev/dri/ is accessible (for Vulkan) or nvidia-smi works (for CUDA)

If the AppImage fails to find the GPU at runtime, it’ll fall back to CPU automatically. Check the log file for “GPU not available” warnings.

Building the AppImage with GPU

scripts/build_appimage.sh --update-info 'gh-releases-zsync|vsndev3|llm-mem|latest|llm-mem-mcp-x86_64.AppImage.zsync'

The AppImage build uses the local feature by default. For GPU support in the AppImage, you need a build environment with Vulkan SDK or CUDA toolkit installed. See packaging/appimage/README.md for details.

Multiple GPUs

llm-mem doesn’t currently support splitting a model across multiple GPUs. It also doesn’t let you pin to a specific GPU — it uses the first one llama.cpp finds. For most setups this is fine.

Next

Logging & debugging

llm-mem uses the tracing crate for structured logging. By default, events go to stderr (so they don’t pollute the MCP JSON-RPC stream on stdout). With logging.enabled = true, they also go to a rotating log file.

Log levels

LevelWhat you getWhen to use
errorOnly errorsQuiet production
warnErrors + warningsDefault verbosity
infoLifecycle events, request summariesNormal operation
debugPer-request detailInvestigating an issue
traceEverything, including LLM promptsDeep debugging

Setting the level

In config

[logging]
enabled = true
level = "debug"

At runtime via env

The RUST_LOG env var overrides the config. Set it in the MCP config’s env block:

{
  "mcp": {
    "llm-mem": {
      "type": "local",
      "command": ["/path/to/llm-mem-mcp"],
      "env": {
        "RUST_LOG": "info,llama_cpp_2=warn,llm_mem::search=debug"
      }
    }
  }
}

RUST_LOG accepts the tracing_subscriber filter syntax:

SyntaxMeaning
infoGlobal default = info
llama_cpp_2=warnSpecific crate at a level
llm_mem::search=debugSpecific module
trace,hyper=info,h2=infoGlobal trace except quiet a few noisy crates

Reading the log file

With logging.enabled = true, the log file is at <log_directory>/llm-mem-mcp.log. Rotation produces .1, .2, … up to max_files.

# Tail the live log
tail -f llm-mem-data/logs/llm-mem-mcp.log

# Look for errors
grep ERROR llm-mem-data/logs/llm-mem-mcp.log

# Inspect a specific request lifecycle
grep "memory_id=abc-123" llm-mem-data/logs/llm-mem-mcp.log

The format is:

2026-02-15T10:23:45.123Z INFO llm_mem::mcp: store_memory called memory_id=
2026-02-15T10:23:46.456Z INFO llm_mem::memory: stored memory memory_id=abc-123

What to look for

Server didn’t start

ERROR config: llm.context_size (8192) is too small for memory.document_chunk_size (2000) and llm.max_tokens (4096)

→ Increase llm.context_size in your config (or decrease the chunk size / max tokens).

API key missing

ERROR config: LLM API key is not configured. Set it in config.toml under [llm].api_key, or via env var LLM_MEM_LLM_API_KEY.

→ Set LLM_MEM_LLM_API_KEY (or OPENAI_API_KEY as a fallback).

Model download failed

ERROR model_downloader: failed to download gemma-4-E2B-it-Q8_0.gguf: HTTPS error: 407 Proxy Authentication Required

→ Set HTTPS_PROXY with auth, or download manually (see below).

LLM timeout

ERROR llm: completion timed out after 120s

→ Increase llm.llm_timeout_secs, switch to a smaller model, or enable GPU.

Manual model download

If you’re behind a corporate proxy, on an air-gapped machine, or just want to pre-fetch the models, download them manually:

mkdir -p llm-mem-data/models
cd llm-mem-data/models

# Language model (~2.5 GB)
curl -L -o gemma-4-E2B-it-Q8_0.gguf \
  https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF/resolve/main/gemma-4-E2B-it-Q8_0.gguf

# Embedding model — let fastembed fetch this; if blocked, download from
# https://github.com/Anush008/fastembed-rs and place in the cache dir.

Then disable auto-download:

[llm]
auto_download = false

Note

Smaller alternative —

A smaller model smollm2-1.7b-instruct-q4_k_m.gguf (~1 GB) is also available but requires a larger context size (set llm.context_size = 8192 or more).

Performance debugging

To find slow operations, set RUST_LOG=info,llm_mem::metrics=trace and look for:

INFO llm_mem::memory: search returned 50 results in 234ms
INFO llm_mem::memory: store took 1.2s (embedding=200ms, llm_extract=900ms, write=100ms)

If search is slow on a large bank:

  • Try search_memory instead of query_memory (less work)
  • Increase search_similarity_threshold to filter earlier
  • Consider per-project banks to keep each one small

If the LLM is the bottleneck:

  • Enable GPU
  • Reduce max_concurrent_requests to 1 to avoid contention
  • Use a smaller/faster model

Inspecting banks

# CLI
llm-mem stats --bank default
llm-mem layer-stats --bank default
llm-mem db check --all --verbose

# MCP
{"tool": "system_status"}
{"tool": "check_consistency", "arguments": {"verbose": true}}

Capturing logs from the REPL

The CLI REPL captures logs in a separate buffer down to TRACE level. Dump them with the savelog command:

> savelog debug.log
Wrote 12345 lines to debug.log

This is useful for diagnosing issues without restarting the CLI with RUST_LOG=trace.

Next

Troubleshooting

Common issues, organized by symptom.

“Server failed to start”

Failed to load config from <path>

The config file exists but has invalid TOML or fails validation. Check the message — it tells you which field is wrong.

Common culprits:

  • Typo in a key name (TOML doesn’t error on unknown keys by default; but serde does for known keys with deny_unknown_fields)
  • A numeric field is a string (max_memories = "10000" should be 10000)
  • An enum value is wrong (provider = "OPENAI" should be provider = "api")

No API key configured

You’re using provider = "api" but didn’t set the key. Fix:

# In the MCP config's env block
"env": {
  "LLM_MEM_LLM_API_KEY": "sk-..."
}

# Or in the TOML config
[llm]
api_key = "sk-..."

# Or as a generic fallback
"env": { "OPENAI_API_KEY": "sk-..." }

llm.context_size is too small

context_size must be larger than memory.document_chunk_size / 2 + llm.max_tokens + 512. The default 16644 works for the defaults, but if you’ve raised document_chunk_size or max_tokens, raise context_size too.

llm.provider = "local" but this build does not have the 'local-llm' feature enabled

You built with --no-default-features and then didn’t include local-llm (or the aggregate local). Rebuild:

cargo build --release --features local

“Models not downloading”

Failed to download: 407 Proxy Authentication Required

The proxy needs auth. Set HTTPS_PROXY=http://user:pass@proxy.host:port in the MCP env block, or in the [llm].proxy_url config (which overrides the env var).

Failed to download: 404 Not Found

The HuggingFace URL has changed or the model was renamed. Check the llm-mem releases for the latest model filenames, or download manually (see Logging & debugging).

Failed to download: connection timeout

Firewall or DNS. Verify you can reach huggingface.co from the server’s environment. If you can’t, download the model on a different machine and copy it in.

“MCP client can’t connect”

See Troubleshooting MCP for the full flow. The short version:

  • Absolute path in the command field (no ~, no relative paths)
  • AppImage needs chmod +x
  • Try APPIMAGE_EXTRACT_AND_RUN=1 if FUSE is unavailable
  • Restart the client after every config change
  • Check the client’s MCP panel for the server status

“Tool calls fail with errors”

LLM timeout after 120 seconds

The LLM is taking too long. Mitigations:

[llm]
llm_timeout_secs = 300     # give it more time
gpu_layers = 20            # use the GPU

Or switch to a smaller model.

Bank not found: 'foo'

You passed bank: "foo" but no such bank exists. Either:

  • Create it first with create_memory_bank (MCP) or just write to it (auto-create)
  • Check the spelling — bank names are case-sensitive

Near-duplicate detected

The new memory’s embedding is very close to an existing one. By default, the server blocks the store and returns this error. Either:

  • Pass force: true to store anyway
  • Edit the new content to be different from the existing memory
  • Re-read what the existing memory says and decide if the new content is actually redundant

Contradiction detected

memory.contradiction_detection = true is enabled and the new memory contradicts an existing one. Same options as above.

Bank deletion requires user confirmation

You called cleanup_resources with target: "banks" but didn’t pass the exact confirmation phrase. This is intentional — pass:

{ "target": "banks", "name": "foo", "confirm": "I confirm this data will be permanently lost" }

“Performance is bad”

SymptomLikely causeFix
First query takes 30+ secondsModel warmupWait. Subsequent queries are fast.
All queries take 5+ secondsCPU-only inference on large modelEnable GPU (see GPU acceleration)
Bank has 500k+ memories, queries are slowVector search scales with bank sizeUse smaller banks per project, increase search_similarity_threshold
API backend, every query is slowRound-trip latencyUse a faster provider or local backend
High memory usageLarge model + large contextReduce context_size or use a smaller model
High disk usageMany banks, large embeddingsRun db prune --all --older-than-days 0 to reclaim old LanceDB versions; run db check to find Forgotten memories; clean up unused banks

“Database corruption”

llm-mem db check --all --verbose
llm-mem db fix --bank default --dry-run    # preview
llm-mem db fix --bank default              # repair (auto-backup)

If db fix can’t repair, restore from backup:

llm-mem db export --bank default --output /tmp/before.db   # safety
llm-mem db import --bank default --input /backup/latest.db

If even that fails, the bank is genuinely corrupted — delete the .lance/ directory and re-create (data loss; restore from your latest JSONL export if you have one).

“I get a different error message than the docs”

llm-mem is beta. Error messages are not stable. Open an issue at https://github.com/vsndev3/llm-mem/issues with:

  1. The exact error message
  2. The log file excerpt (logging.level = "debug")
  3. Your config.toml (with secrets redacted)
  4. The version (llm-mem-mcp --version)

Next

Performance

A reference for tuning llm-mem for your workload.

First, measure

Don’t tune blind. Run:

llm-mem health-check --live
llm-mem system-status
llm-mem metrics

These give you the backend details and per-call timings. With RUST_LOG=debug,llm_mem::metrics=trace, you get per-request durations in the log.

Identify the bottleneck

Performance bottlenecks fall into a few categories:

1. LLM inference is slow

Symptoms: store operations take seconds; abstraction pipeline crawls; first query takes 30s.

Mitigations (try in order):

  1. Enable GPU — biggest single win, often 5-20x. See GPU acceleration.
  2. Use a smaller model — gemma-4-E2B is ~2.5 GB; if you don’t need that quality, try smollm2-1.7b-instruct-q4_k_m (~1 GB).
  3. Reduce max_concurrent_requests — default is 1 (serial), which is safest. If you have GPU headroom, raise to 2-4 for parallelism.
  4. Use API backend — if you have a fast API provider, an API call can be faster than local CPU inference.

2. Embedding is slow

Symptoms: searches and stores take a long time on the embedding step.

Mitigations:

  1. Local fastembed is fast by default — should be 1000+ texts/sec on CPU. If it’s slow, your CPU is loaded by other things.
  2. API embeddings — if you have API budget, use provider = "api" for embeddings.
  3. Increase batch_size — embedding.batch_size = 256 (default 64) is more efficient for large batch operations.

3. Vector search is slow

Symptoms: query_memory and search_memory are slow on a large bank.

Mitigations:

  1. Use smaller banks — split into per-project banks. Vector search is roughly linear in the bank’s size.
  2. Use search_memory instead of query_memory — the former has fewer features and is faster.
  3. Increase search_similarity_threshold — filters out low-similarity candidates early.
  4. Favor lower layers — pyramid_config: { "mode": "bottom_heavy" } concentrates results on L0 and skips most higher-layer assembly.
  5. Lower max_search_results — fewer results to rank and return.

4. Abstraction pipeline is slow

Symptoms: L1+ memories take a long time to appear; background worker is the bottleneck.

Mitigations:

  1. Tune auto_summary_threshold — the L0→L1 worker only fires past this. Higher = less frequent but cheaper per run.
  2. Pause the pipeline during bulk operations — stop_abstraction_pipeline → bulk ingest → start_abstraction_pipeline. Avoids contention with the LLM.
  3. Set session_token_budget — caps the LLM tokens spent per session, preventing runaway costs.
  4. Disable auto_enhance — if you don’t need LLM-generated metadata on every store, set auto_enhance = false to skip the LLM call.

5. Database operations are slow

Symptoms: db check, db fix, large exports take a long time.

Mitigations:

  1. Use db export-jsonl for archival — streaming write, doesn’t materialize the whole bank in memory.
  2. For very large banks, use per-bank operations — don’t db check --all on 50 banks at once.
  3. Disk I/O — make sure the bank directory is on a fast SSD, not a network mount.

Configuration tuning guide

For a fast local setup

[llm]
provider = "local"
gpu_layers = 30               # all layers on GPU
max_concurrent_requests = 2   # parallel inference

[memory]
auto_enhance = true
auto_summary_threshold = 16384
session_token_budget = 100000

For a low-resource setup

[llm]
provider = "local"
gpu_layers = 0
cpu_threads = 4
max_concurrent_requests = 1
llm_timeout_secs = 180

[memory]
auto_enhance = false
deduplicate = true
auto_summary_threshold = 65536

For an API-only setup

[llm]
provider = "api"
model = "gpt-4o-mini"
max_concurrent_requests = 4
use_structured_output = true
llm_timeout_secs = 60

[embedding]
provider = "api"
model = "text-embedding-3-small"
batch_size = 100

[memory]
auto_enhance = true

For very large banks (>1M memories)

[memory]
raw_content_scan_limit = 1000
max_total_candidates = 5000
max_list_limit = 1000

And consider per-project banks instead of one giant bank.

When to scale

Bank sizeRecommendation
< 10k memoriesDefault config is fine
10k-100kTune max_total_candidates and raw_content_scan_limit
100k-1MConsider per-project banks, LanceDB only (not VectorLite)
> 1MSpecialized tuning + faster hardware; consider a dedicated vector DB instead

Next