Introduction
llm-mem is a memory server for AI agents. It gives your AI assistant persistent, searchable memory that works across sessions — conversations, codebases, documents, and anything else you want it to remember.
It’s a single self-contained binary written in Rust. No databases to install, no cloud services required, no API keys needed by default. Everything runs locally on your machine, and data never leaves your computer.
Warning
Beta software —
This is beta software. Expect rough edges and breaking changes. Use it to try things out, not for anything important.
How it works
When connected to an AI assistant via the Model Context Protocol (MCP), llm-mem exposes a set of tools the assistant can call to:
- Store raw text, documents, or AI-extracted facts as memories
- Search by meaning (semantic search) or by keyword
- Browse chronologically with timeline tools
- Navigate a layered pyramid from raw content up to abstract insights
- Manage isolated memory banks, backups, and document uploads
The assistant calls these tools just like any other MCP tool — no special prompting required.
What makes it different
| Feature | llm-mem |
|---|---|
| Local-first | Runs entirely on your machine by default. No cloud, no accounts, no data leaves your box. |
| Embedded vector store | Uses LanceDB embedded. No separate database server. |
| Layered memory | L0 raw → L1 summaries → L2 connections → L3 concepts → L4 insights, with auto-abstraction. |
| AI in-process | Bundled llama.cpp for local LLM, fastembed for local embeddings. Zero-config. |
| MCP-native | Speaks MCP stdio. Drop into opencode, VS Code Copilot, or any MCP client. |
| Standalone CLI | Includes an interactive REPL and single-command mode for shell scripting. |
Next steps
- New here? Start with What is llm-mem? for the conceptual overview.
- Ready to install? Jump to Installation.
- Want to skim the tool surface? See Tools overview.
What is llm-mem?
llm-mem is a persistent semantic knowledge index for AI agents. It gives an MCP-compatible AI assistant a real, queryable memory that:
- Persists across sessions — memories survive restarts, crashes, and reboots.
- Is searchable by meaning — semantic vector search finds relevant memories even when exact keywords differ.
- Organizes itself into layers — raw content is automatically distilled into summaries, connections, concepts, and insights.
- Stays isolated per bank — separate knowledge bases for separate projects or topics.
- Runs locally by default — everything happens on your machine; no network calls required.
The core idea
When you ask an AI assistant to remember something, it normally can’t — once the conversation ends, the context window is gone. The next time you talk to it, you start from zero.
llm-mem solves this by sitting between the AI and a local vector database. The AI calls MCP tools (add_content_memory, query_memory, etc.) to read and write to the database. The database remembers.
┌──────────────┐ stdio/MCP ┌──────────────┐ SQLite+Arrow ┌──────────────┐
│ AI client │ ──────────────▶ │ llm-mem-mcp │ ─────────────────▶ │ LanceDB │
│ (opencode, │ │ (server) │ │ (banks/*.lance)
│ Copilot, │ ◀────────────── │ │ ◀───────────────── │
│ any MCP) │ results │ │ vectors │
└──────────────┘ └──────────────┘ └──────────────┘
│
▼
llama.cpp + fastembed
(local LLM + embeddings)
What’s stored in a memory?
Each memory has:
content— the actual text (a fact, a conversation snippet, a document chunk, etc.)context— broad topic tags (e.g.["project-x", "auth", "jwt"])topics— specific subjects within the contentrelations— links to other memories ({"relation": "depends_on", "target": "memory_id"})metadata— arbitrary key/value (source URL, file path, line range, etc.)event_at— when the event actually happened (ISO 8601, optional; falls back tocreated_at)created_at/updated_at— when the memory was stored/last modified (set by the system)- An embedding vector — generated by the embedding model for semantic search
The system separates event time (when something happened) from storage time (when you asked the AI to remember it), so “what happened in the last 2 days” doesn’t get confused with “what was stored recently”.
When to use it
Use llm-mem whenever you’d otherwise repeat yourself to an AI:
- Codebases — store module responsibilities, API contracts, architectural decisions, gotchas. Per-repo banks.
- Research — store web findings, paper summaries, citations. Per-topic banks.
- Conversations & decisions — capture meeting outcomes, action items, team decisions.
- Documents — drop a PDF or markdown file in, it gets chunked, summarized, and indexed automatically.
- Personal knowledge — preferences, contacts, projects, todos. The more you store, the more useful semantic search becomes.
When not to use it
- Production-critical data — it’s beta software. Don’t store anything you can’t lose.
- Real-time low-latency queries — semantic search is fast (sub-second for small banks) but not instant. Don’t expect to handle thousands of queries per second.
- Massive datasets — the embedded vector store is fine up to ~hundreds of thousands of memories. Beyond that, consider a dedicated vector database.
Next
- Understand the memory pyramid — how raw content becomes abstract insights.
- Check What you need before installing.
The memory pyramid
llm-mem doesn’t just dump everything into a flat pile. It organizes memories in layers, similar to how human memory works — from concrete facts up to abstract understanding.
| Layer | Name | What it is |
|---|---|---|
| L0 | Raw content | Your conversations, notes, and documents as-is |
| L1 | Summaries | AI-generated summaries and structural outlines |
| L2 | Connections | Cross-document links and thematic relationships |
| L3 | Concepts | Domain principles and theories |
| L4 | Insights | Mental models, paradigms, and high-level patterns |
Background workers automatically create higher layers over time. You can search at any layer, zoom in to see sources of a concept, or zoom out to see what abstractions were built from something you stored.
Why layers?
Flat search works for keyword matching but fails on questions like “What pattern connects these accidental discoveries?”. Pyramid search across layers can synthesize the answer.
The pyramid is also how human memory works:
| Human cognition | llm-mem layer | Example |
|---|---|---|
| Sensory input | L0 raw | “Meeting at 3pm, Alice said PostgreSQL, see #arch-2026-02-15” |
| Episodic | L1 summary | “Feb 15 architecture meeting” |
| Semantic | L2 connection | “PostgreSQL chosen over MongoDB for analytics service” |
| Conceptual | L3 concept | “Relational vs. document databases for join-heavy workloads” |
| Wisdom | L4 insight | “Schema-first design reduces migration cost at scale” |
The L0 content is verbatim — exact original phrasing, file paths, line numbers, timestamps. Higher layers compress and connect, but you can always zoom back to the source.
How layers are created
L0 — Raw content: created by you (or your AI assistant) when you call add_content_memory, store_memories, or upload_document. These are immutable entries.
L1 — Summaries: created automatically by a background worker that watches L0 content. When enough L0 content accumulates (controlled by auto_summary_threshold), the worker picks related memories and asks the LLM to produce a summary.
L2 — Connections: created by another background worker that looks at L1 summaries, finds semantic overlap, and emits related_to, references, extends, or contradicts relations.
L3 — Concepts: created by an L2→L3 worker that clusters L2 connections into domain concepts.
L4 — Insights: created by an L3→L4 worker that finds cross-domain patterns and abstracts them into principles.
You don’t manage this pipeline. It runs in the background and adds higher layers as material accumulates. You can query at any layer, force a specific layer to be built (create_abstraction), or pause the pipeline (stop_abstraction_pipeline / start_abstraction_pipeline).
Navigation
The navigate_memory tool moves through the pyramid:
- zoom out from a concrete L0 memory → see what L1+ insights it contributed to
- zoom in from an abstract L3 concept → see the L0 source evidence
- search at layer → restrict results to a specific abstraction level
This is the killer feature for research workflows. Ask “what did we decide about auth in February?” and you get raw meeting notes. Ask “what’s our auth architecture?” and you get a synthesized concept with source links.
The two timestamps
Every memory carries two distinct timestamps:
event_at— when the event actually happened (caller-supplied for L0; derived as a range from source memories for L1+)created_at— when the memory was stored (set automatically)
This separation lets you answer “what happened in the last two days” without confusing it with “what was stored recently”. Old memories without an explicit event_at fall back to created_at so the timeline stays complete.
Next
Ready to run it? See What you need and then Installation.
What you need
The requirements depend on which build you choose and which AI backend you use.
For a default (fully-local) build
- Rust 1.85+ (2024 edition) — only required if building from source via
cargo - A C/C++ compiler and CMake — needed to compile the bundled
llama.cppandfastembednative code - ~3 GB of free disk space — for the language model (~2.5 GB) and embedding model (~90 MB), downloaded on first run
- ~4 GB of RAM — the LLM lives in memory once loaded
If you don’t want to compile a local AI engine, see the API-only build below.
For an API-only build
- Rust 1.85+ — only required if building from source
- An API key for an OpenAI-compatible endpoint (OpenAI, OpenRouter, Anthropic, Ollama, llama-server, etc.)
- No C/C++ toolchain — native inference is disabled in this build
For the AppImage
- Linux with glibc ≥ 2.28 — Ubuntu 20.04+, Debian 11+, RHEL 9+, Fedora, Arch, openSUSE
- libfuse2 (Debian/Ubuntu) or
fuse(Fedora) — usually pre-installed. Only needed if you don’t setAPPIMAGE_EXTRACT_AND_RUN=1 - No compiler toolchain — fully self-contained
- ~3 GB disk for models on first run
For GPU acceleration (optional but recommended)
| Platform | GPU support | Build flag |
|---|---|---|
| macOS (Apple Silicon) | M1/M2/M3 | --features metal |
| Linux/Windows (AMD, Intel, NVIDIA) | Vulkan | --features vulkan |
| Linux/Windows (NVIDIA only) | CUDA | --features cuda |
GPU acceleration makes the LLM substantially faster (often 5-20x) but is not required.
For document parsing
Document parsing uses pure-Rust libraries by default. No system dependencies required for:
- Markdown, plain text, CSV, TSV, JSON, JSONL, YAML, TOML, XML
- PDF (via
lopdf) - DOCX, XLSX, XLS (via
calamine) - ZIP archives
- PNG, JPEG, GIF, WebP images (descriptions via local LLM if
vision_enabled = true)
MCP client requirements
The MCP server (llm-mem-mcp) speaks stdio and is compatible with any MCP client. Confirmed working with:
- opencode (recommended for local development)
- VS Code Copilot (with the MCP extension)
- Any other MCP-compatible client (Claude Code, Cursor, etc.)
Next
Ready to install? See Installation.
Installation
There are five ways to install llm-mem, listed from least to most effort:
- Pre-built AppImage (Linux) — single file, no install, no compiler
- Pre-built native binary (Linux/macOS/Windows) — download from Releases
cargo install— Rust toolchain required- Build from source — for contributors / custom builds
- Docker build — for the AppImage itself (advanced)
For most users, option 1 or 2 is the right choice.
Option 1: Pre-built AppImage (Linux, recommended)
A single-file AppImage is available that works on every modern Linux distribution without glibc version issues. It bundles both binaries (llm-mem-mcp and llm-mem) in one file, dispatching by argv[0] (multicall pattern). Includes LanceDB, the Vulkan loader, and a bundled OpenSSL.
# 1. Download from the GitHub Releases page
# https://github.com/vsndev3/llm-mem/releases
# pick: llm-mem-mcp-x86_64.AppImage (or -aarch64)
# 2. Make it executable
chmod +x llm-mem-mcp-x86_64.AppImage
# 3. Run the MCP server (default when invoked as llm-mem-mcp)
./llm-mem-mcp-x86_64.AppImage --help
# 4. (Optional) Use the CLI via a symlink
ln -s "$(pwd)/llm-mem-mcp-x86_64.AppImage" ~/bin/llm-mem
llm-mem --help
llm-mem --single search --query "vegan recipes"
The first run downloads the AI models it needs:
- A language model: Gemma 4 E2B (
gemma-4-E2B-it-Q8_0.gguf, ~2.5 GB) - An embedding model: all-MiniLM-L6-v2 (~90 MB)
Downloads resume if interrupted. HTTP/HTTPS/SOCKS proxies are respected via HTTPS_PROXY / HTTP_PROXY / ALL_PROXY environment variables.
If FUSE isn’t available on your system (e.g. some container environments), set:
export APPIMAGE_EXTRACT_AND_RUN=1
Note
The AppImage downloads a separate copy of the models per invocation. To share models across runs, set
LLM_MEM_MODELS_DIRto a stable path.
Option 2: Pre-built native binary (Linux, macOS, Windows)
Native binaries are built for the host platform’s typical glibc/libc version. They don’t bundle a C++ runtime or OpenSSL, so they rely on system libraries.
| Archive | Platform | Architecture | GPU |
|---|---|---|---|
llm-mem-mcp-linux-x86_64.tar.gz | Linux (Ubuntu 24.04+) | x86_64 | Vulkan |
llm-mem-mcp-linux-aarch64.tar.gz | Linux ARM64 | aarch64 | Vulkan |
llm-mem-mcp-macos-x86_64.tar.gz | macOS 15+ (Intel) | x86_64 | Metal |
llm-mem-mcp-macos-aarch64.tar.gz | macOS 15+ (Apple Silicon) | arm64 | Metal |
llm-mem-mcp-windows-x86_64.zip | Windows 10/11 | x86_64 | Vulkan |
Install:
# Linux / macOS
tar xzf llm-mem-mcp-linux-x86_64.tar.gz
sudo mv llm-mem-mcp /usr/local/bin/
# Windows (PowerShell)
Expand-Archive llm-mem-mcp-windows-x86_64.zip
move llm-mem-mcp.exe C:\Windows\System32\
On first run, the same auto-download kicks in. On Windows, you may need to install Microsoft Visual C++ Redistributable for the local LLM engine.
Option 3: cargo install
If you have a Rust toolchain installed and want the latest from crates.io (when published):
cargo install llm-mem --features local
This requires a C/C++ compiler and CMake on the system. Build time is ~10-20 minutes the first time (compile llama.cpp + fastembed native code).
Option 4: Build from source
For contributors, custom builds, or to get unreleased features:
git clone https://github.com/vsndev3/llm-mem.git
cd llm-mem
# Default build: local LLM (llama.cpp) + local embeddings (fastembed) + LanceDB
cargo build --release
# Artifacts land in target/release/:
# llm-mem-mcp — MCP server
# llm-mem — standalone CLI
Build feature combinations
| Command | What you get |
|---|---|
cargo build --release | Default: local LLM + local embeddings + LanceDB (fully offline) |
cargo build --release --no-default-features --features lancedb | API-only, no local AI |
cargo build --release --no-default-features --features lancedb,local-embed | API LLM + local embeddings |
cargo build --release --features local,vulkan | Local + Vulkan GPU |
cargo build --release --features local,metal | Local + Metal GPU (macOS) |
cargo build --release --features local,cuda | Local + CUDA GPU (NVIDIA) |
cargo build --release --no-default-features --features lancedb,vector-lite | API + VectorLite (legacy) |
See GPU acceleration for details on the GPU backends.
Option 5: Build the AppImage yourself
If you want to produce an AppImage for a custom feature combination or to test changes:
git clone https://github.com/vsndev3/llm-mem.git
cd llm-mem
# Local Docker build (auto-detects docker or podman)
scripts/build_appimage.sh
# Native build (no Docker) — requires gcc 10+, cmake, appimagetool
scripts/build_appimage.sh --no-docker
# Build for ARM64 from an x86_64 host
scripts/build_appimage.sh --arch aarch64
# Build with zsync auto-update metadata
scripts/build_appimage.sh --update-info 'gh-releases-zsync|vsndev3|llm-mem|latest|llm-mem-mcp-x86_64.AppImage.zsync'
Output: dist/llm-mem-mcp-x86_64.AppImage (or -aarch64) plus a .zsync delta-update file.
The CI workflow .github/workflows/appimage.yml is manual-trigger only — it produces x86_64 and aarch64 artifacts in the Actions UI without auto-publishing a release.
Verifying the install
# MCP server
llm-mem-mcp --version
llm-mem-mcp --help
# CLI
llm-mem --help
Both binaries should print their version and exit. If the binary is missing or won’t run, see Troubleshooting.
Next
- First run — what happens on the very first invocation
- Quickstart — get from install to working memory in 10 minutes
First run
What happens when you start llm-mem-mcp for the very first time.
Step 1: Config discovery
The server searches for a config file in this order:
--config <path>CLI flag (explicit)./config.toml(current working directory)~/.config/llm-mem/config.toml(XDG config dir)/etc/llm-mem/config.toml(system-wide)
If none of these exist, the server uses all defaults and applies any environment variable overrides (see Environment variables).
Tip
Generate a template —
To see every available setting with comments, run:
bash llm-mem-mcp –generate-config > config.toml
Edit the file, then start the server with `llm-mem-mcp --config config.toml`.
Step 2: Model download (local backend)
If you configured provider = "local" (the default), the server downloads:
- Language model:
gemma-4-E2B-it-Q8_0.gguf(~2.5 GB) from HuggingFace - Embedding model:
all-MiniLM-L6-v2(~90 MB) from fastembed’s model registry
Models are stored under the directory specified by llm.models_dir (default: llm-mem-data/models/) and cached separately under ~/.cache/llm-mem/models/ (configurable via llm.cache_dir).
Downloads are resumable — if interrupted, just restart the server. A smaller alternative model (smollm2-1.7b-instruct-q4_k_m.gguf, ~1 GB) is also available but requires a larger context size.
Note
Manual model download —
If you’re behind a corporate proxy or on an air-gapped machine, see Logging & debugging for instructions on downloading the models manually.
Step 3: Bank creation
The default memory bank is created automatically on first run. The bank lives at:
<llm.banks_dir>/default.lance/
<llm.banks_dir>/default.sessions.db
Default llm.banks_dir: ./llm-mem-data/banks/. See Data directory layout for the full schema.
Step 4: Service startup
Once models are loaded and the bank is ready, the service:
- Starts the abstraction pipeline (background L0→L1, L1→L2, … workers)
- Resumes any document upload sessions that were interrupted
- Listens on stdio for MCP requests
You’ll see log output on stderr (or in the log file if logging.enabled = true):
INFO Starting LLM Memory MCP Server
INFO Configuration loaded from: None
INFO Logging to directory: ./llm-mem-data/logs
INFO Initialized LLM client (backend: Local)
INFO Initialized memory bank manager
INFO Default memory bank loaded
INFO Abstraction pipeline initialization requested
INFO MCP server initialized successfully
Step 5: Connect a client
The server is now waiting on stdio for MCP traffic. Connect it to your AI client:
- opencode
- VS Code Copilot
- Troubleshooting MCP — if connection fails
Stopping the server
The server handles SIGINT (Ctrl+C) and SIGTERM gracefully:
- Stops the MCP listener
- Stops the abstraction pipeline
- Flushes any in-flight writes
- Releases the llama.cpp backend
No special shutdown command is required.
Next
Quickstart — run your first query in 5 minutes.
Quickstart
A 10-minute tour: install, connect to an AI assistant, and ask it to remember something.
0. Prerequisites
- llm-mem installed (see Installation)
- An MCP client. We’ll use opencode here, but VS Code Copilot works too.
1. Start the server in MCP mode
# Either run the MCP server directly, or use the AppImage multicall
llm-mem-mcp
The server will:
- Look for a config file (or use defaults)
- Download models on first run (~3 GB total, resumable)
- Start listening on stdio
Leave this running, or background it:
# It runs as a foreground process — your MCP client launches it on demand.
# For manual testing:
llm-mem-mcp
Note
You don’t usually run the MCP server by hand — the MCP client launches it. The above is for sanity-checking that it starts and prints its banner.
2. Connect opencode
In your project directory (or globally), create opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/path/to/llm-mem-mcp-x86_64.AppImage"],
"enabled": true
}
}
}
Or if you built from source with cargo build --release:
{
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/absolute/path/to/llm-mem/target/release/llm-mem-mcp"],
"enabled": true
}
}
}
Start (or restart) opencode. It discovers the MCP server and lists the 32 llm-mem tools in its tool palette.
3. Tell the AI to remember something
Open opencode and chat. Try:
Use the llm-mem tools to remember that the AuthService in this project handles JWT token validation and refresh — entry point is
src/auth/service.rs:45-80. The refresh tokens expire after 7 days.
The AI will call add_content_memory (or add_intuitive_memory if it wants the LLM to extract structured facts).
4. Ask it back later
In a fresh conversation (or after restarting the AI):
Where is JWT validation handled, and what’s the refresh token expiry?
The AI will call query_memory or search_memory and return the stored fact with the source attribution.
5. Try semantic search
The search works by meaning, not just keywords. Try:
Find anything I stored about user authentication
Even if you stored “AuthService handles JWT token validation”, the search will find it because the embedding captures the meaning of “user authentication”.
6. Add a document
Upload the file
docs/architecture.mdto memory.
The AI calls upload_document, which:
- Chunks the file
- Embeds each chunk
- Stores them as L0 memories
- Triggers the L0→L1 worker to create summaries in the background
Later queries about the document return the relevant chunks.
7. Look at the timeline
Show me what I stored in the last 2 days, grouped by day.
The AI calls get_timeline with the --since 2d equivalent. You get a chronological list grouped by day.
8. Navigate the pyramid
Zoom out from the JWT memory — what higher-level insights were built from it?
The AI calls navigate_memory with zoom_out. You get the L1/L2/L3 memories that were synthesized from the original L0 entry.
What just happened
You went from zero to a working AI memory in 10 minutes. The same patterns work for:
- Storing entire codebases (one bank per repo,
upload_documentfor each file) - Research projects (one bank per topic, semantic search to find related work)
- Personal notes (one bank, multiple
event_at-tagged entries, timeline views) - Meeting logs (one bank, intuitive memories for each decision)
Next
- MCP clients overview — connect other clients beyond opencode
- Tools overview — the full 32-tool reference
- Configuration — customize the server
MCP clients overview
llm-mem is an MCP server. It speaks the Model Context Protocol over stdio (standard input/output) and registers 32 tools that the AI assistant calls.
Any MCP client can connect to it. The configuration is a JSON object that tells the client how to launch the server process.
General shape
Every MCP client config has the same essential fields:
{
"<mcp-key>": {
"type": "local",
"command": "/absolute/path/to/llm-mem-mcp",
"args": [],
"env": {}
}
}
| Field | Meaning |
|---|---|
type: "local" | The server runs as a local subprocess (the only kind llm-mem supports) |
command | Absolute path to the llm-mem-mcp binary (or AppImage) |
args | Command-line arguments (optional — see CLI flags) |
env | Environment variables to set when launching (optional) |
The command must be absolute. The client launches it as a child process and pipes JSON-RPC over its stdin/stdout.
Confirmed clients
| Client | Status | Config key | Docs |
|---|---|---|---|
| opencode | Recommended | mcp.<name> in opencode.json | opencode docs |
| VS Code Copilot | Supported | servers.<name> in .vscode/mcp.json | VS Code MCP docs |
| Claude Code | Compatible | mcpServers in .mcp.json | Anthropic docs |
| Cursor | Compatible | mcpServers in ~/.cursor/mcp.json | Cursor docs |
| Zed | Compatible | context_servers in ~/.config/zed/settings.json | Zed docs |
The configuration format varies by client. The command and behavior are the same.
Choosing between AppImage and direct binary
AppImage (llm-mem-mcp-x86_64.AppImage):
- One file, works on any glibc ≥ 2.28 distro
- Bundled OpenSSL + C++ runtime
- Larger download (~150 MB) but zero install
- Best for: portable, no-system-deps deployment
Native binary (llm-mem-mcp):
- Smaller, depends on system libc/libstdc++
- Released per platform (Linux x86_64, macOS aarch64, etc.)
- Best for: when you control the runtime environment
From-source build (target/release/llm-mem-mcp):
- After
cargo build --release - Best for: development, custom feature combinations
Pick the binary that matches how you installed llm-mem, and use the absolute path in your MCP config.
Environment variables to set
The env map in the MCP config can set any of the environment variables llm-mem reads. Common ones:
{
"env": {
"LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
"LLM_MEM_LLM_API_KEY": "sk-...",
"HTTPS_PROXY": "http://proxy.corp:8080",
"RUST_LOG": "info"
}
}
Tip
Don’t put secrets in version control —
MCP config files are often committed to source repos. Use environment variables or a secret manager for API keys, not literal values in the config.
Verifying the connection
After updating your MCP config:
- Restart the client (opencode, VS Code, etc.) — it spawns the server as a subprocess
- Look in the client’s MCP/tool panel — you should see 32
llm-mem-*tools - Ask the AI to call
system_status— the response confirms the server is reachable and the models are loaded
If the tools don’t appear, see Troubleshooting MCP.
Next
- opencode — the recommended client
- VS Code Copilot — for VS Code users
- Troubleshooting MCP — common connection issues
opencode
opencode is the recommended MCP client for llm-mem. It’s an open-source terminal UI for AI coding assistants with first-class MCP support.
1. Install opencode
Follow the opencode install instructions. The quick version:
# macOS / Linux
curl -fsSL https://opencode.ai/install | bash
# Or via Homebrew
brew install opencode
# Or via npm
npm install -g @opencode/cli
2. Locate your opencode.json
opencode reads a config from one of two places:
- Project-local:
<your-project>/opencode.json— only affects this project - Global:
~/.config/opencode/config.json— affects every project
For a project-local config (most common for code work):
cd /path/to/your/project
touch opencode.json
For a global config:
mkdir -p ~/.config/opencode
touch ~/.config/opencode/config.json
3. Add llm-mem
Edit the file you created. The minimum configuration:
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/absolute/path/to/llm-mem-mcp"],
"enabled": true
}
}
}
If you installed the AppImage
{
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/home/you/apps/llm-mem-mcp-x86_64.AppImage"],
"enabled": true
}
}
}
Make the AppImage executable first:
chmod +x /home/you/apps/llm-mem-mcp-x86_64.AppImage
If you built from source
{
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/home/you/projects/llm-mem/target/release/llm-mem-mcp"],
"enabled": true
}
}
}
If you have multiple memory banks
You can declare multiple llm-mem instances, one per bank:
{
"mcp": {
"llm-mem-default": {
"type": "local",
"command": ["/path/to/llm-mem-mcp"],
"enabled": true
},
"llm-mem-research": {
"type": "local",
"command": ["/path/to/llm-mem-mcp"],
"args": ["--config", "/path/to/research-config.toml"],
"enabled": true
}
}
}
The first one is the default; the second uses a separate config that points to a different banks_dir.
4. Add env vars (optional)
Common additions:
{
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/path/to/llm-mem-mcp"],
"enabled": true,
"env": {
"LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
"RUST_LOG": "info"
}
}
}
}
See Environment variables for the full list.
5. Restart opencode and verify
opencode
The first MCP handshake can take 10-30 seconds (model download on first run). Watch the status indicator — when it turns green, the server is ready.
In opencode, run:
/tools
You should see 32 llm-mem tools listed. Try:
> Call system_status with llm-mem
If the response includes "backend": "local" and "llm_available": true, the connection is working.
6. Use it
From here on, the AI automatically has access to the llm-mem tools. You can just talk normally:
Remember that the AuthService refresh tokens expire after 7 days. What did I store about authentication? Upload docs/architecture.md to memory. Show me what I stored this week.
The AI decides which tool to call based on your request.
Troubleshooting
If the server doesn’t start or opencode can’t reach it, see Troubleshooting MCP.
Common issues specific to opencode:
- Config not picked up: confirm the path with
opencode --print-configor check the opencode logs - Permission denied on AppImage:
chmod +xit - Server starts but no tools appear: check the opencode version supports MCP (>= 0.5.x)
- Stale config: restart opencode after every config change — it doesn’t hot-reload MCP servers
Next
- VS Code Copilot — alternative client
- Tools overview — what the AI can do with the connected server
VS Code Copilot
VS Code’s Copilot Chat supports MCP servers via the MCP extension. Once connected, Copilot gains the 32 llm-mem tools as part of its tool palette.
1. Install prerequisites
- VS Code 1.85+
- GitHub Copilot Chat extension
- MCP support is built into VS Code 1.85+ (no extra extension needed)
2. Choose config location
VS Code reads MCP servers from two places:
| Location | Scope | Path |
|---|---|---|
| Workspace | This project only | <workspace>/.vscode/mcp.json |
| User profile | All your workspaces | Use MCP: Open User Configuration from the command palette |
For a single project, use workspace scope (.vscode/mcp.json).
For a global setup that follows you across projects, use user scope.
3. Add llm-mem
Create .vscode/mcp.json in your workspace root:
{
"servers": {
"llm-mem": {
"type": "stdio",
"command": "/absolute/path/to/llm-mem-mcp",
"args": []
}
}
}
AppImage variant
{
"servers": {
"llm-mem": {
"type": "stdio",
"command": "/home/you/apps/llm-mem-mcp-x86_64.AppImage",
"args": []
}
}
}
From-source variant
{
"servers": {
"llm-mem": {
"type": "stdio",
"command": "/home/you/projects/llm-mem/target/release/llm-mem-mcp",
"args": []
}
}
}
With environment variables
{
"servers": {
"llm-mem": {
"type": "stdio",
"command": "/path/to/llm-mem-mcp",
"args": [],
"env": {
"LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
"RUST_LOG": "info"
}
}
}
}
4. Reload VS Code
After saving the config, run MCP: List Servers from the command palette (Ctrl+Shift+P / Cmd+Shift+P). You should see llm-mem listed with a green dot indicating the server started successfully.
If the dot is red or yellow, see Troubleshooting MCP.
5. Use it
Open Copilot Chat (the chat panel in the sidebar, or Ctrl+Shift+I / Cmd+Shift+I).
In agent mode, Copilot will automatically call the MCP tools when relevant. Try:
What llm-mem tools are available?
Copilot will list the 32 tools. Then:
Remember that the AuthService uses bcrypt for password hashing, see src/auth/hasher.rs:23. Search llm-mem for any auth-related facts. Upload README.md to the default memory bank.
Server lifecycle
VS Code spawns the MCP server on demand when you first use a tool, and shuts it down when VS Code quits. If you edit .vscode/mcp.json, you need to reload the server:
- Open command palette
- Run MCP: Restart Server (or MCP: List Servers → click the server)
Multiple servers
Add multiple entries under servers:
{
"servers": {
"llm-mem-default": {
"type": "stdio",
"command": "/path/to/llm-mem-mcp",
"args": []
},
"llm-mem-research": {
"type": "stdio",
"command": "/path/to/llm-mem-mcp",
"args": ["--config", "/path/to/research.toml"]
}
}
}
Each shows up as a separate tool group in Copilot.
Troubleshooting
- Server not found: confirm the absolute path with
lsorfile <path>from a terminal - Permission denied on AppImage:
chmod +xit - AppImage fails to launch: VS Code may not have FUSE access. Try
APPIMAGE_EXTRACT_AND_RUN=1in theenvblock - Models not downloading: check
~/.cache/llm-mem/modelsand your network/proxy settings - Tools don’t appear: reload the server from the command palette
See Troubleshooting MCP for more.
Next
- Troubleshooting MCP — common connection issues
- Tools overview — the full tool surface
Troubleshooting MCP
Common issues when connecting an MCP client to llm-mem-mcp.
Server doesn’t start
Symptom: Client shows the server as “failed” or “not connected”. Log file (if logging.enabled = true) shows an error.
Check:
- Absolute path:
commandmust be an absolute path, not a relative one or a~-expanded one. - Executable bit: AppImage needs
chmod +x. Native binaries usually don’t. - Run it manually: try
./llm-mem-mcp --versionfrom a terminal. If that fails, the binary itself is broken. - Missing libraries: on Linux, run
ldd /path/to/llm-mem-mcpto check for missing.sofiles. On macOS, useotool -L. On Windows, use Dependencies (https://lucasg.github.io/Dependencies/). - AppImage + no FUSE: add
"env": {"APPIMAGE_EXTRACT_AND_RUN": "1"}to the MCP config.
Server starts but no tools appear
Check:
- MCP version mismatch: confirm your client supports MCP. opencode 0.5+, VS Code 1.85+, Claude Code 1.0+.
- Restart the client: MCP server lists are loaded at startup, not hot-reloaded.
- Check stderr: the server logs to stderr. With
logging.enabled = true, also check the log file. Look for JSON-RPC errors. - Try a different transport: llm-mem only supports stdio. If your client is trying HTTP/SSE, it won’t work.
Server starts, tools appear, but calls fail
Check:
system_statusfirst: ask the AI to call it. The response tells you if models are loaded, what backend is active, and whether the LLM/embedding are reachable.- Wait for model download: on first run, the server downloads ~3 GB of models. Until that’s done, most tools return errors. Watch the log file for “model loaded” messages.
- API key issues: if using the API backend, verify the key is set via
LLM_MEM_LLM_API_KEY(or in the config). The server validates at startup and will refuse to start without it. - Bank creation error: if the
banks_dirisn’t writable, the server fails to create the default bank. Check filesystem permissions and thebanks_dirsetting.
Performance is slow
Check:
- GPU not used: see GPU acceleration. CPU-only inference is 5-20x slower.
- First query is slow: model loading + warmup takes 5-30 seconds. Subsequent queries are fast.
- Concurrent requests: default
max_concurrent_requests = 1for stability. Bump it if you have a beefy machine. max_concurrent_requestsfor API backend: set to0(unlimited) if you have API rate-limit headroom.- Large bank: queries get slower as the bank grows past ~100k memories. Consider per-project banks.
Logs
The server logs to stderr. With logging.enabled = true, it also writes to a rotating log file under log_directory (default llm-mem-data/logs/).
To get more detail, set the RUST_LOG env var in the MCP config’s env block:
"env": {
"RUST_LOG": "info,llama_cpp_2=warn"
}
Useful levels:
| Value | What you get |
|---|---|
error | Only errors |
warn | Errors + warnings |
info (default) | Lifecycle events, request summaries |
debug | Per-request detail |
trace | Everything, including the LLM’s prompts |
See Logging & debugging for more.
Common error messages
Failed to resolve memory bank: ...
The bank name passed to the tool doesn’t exist and can’t be created. Check spelling. Banks are created on first write, not on first read.
LLM timeout after 120 seconds
The LLM is taking too long. Either:
- Increase
llm.llm_timeout_secsin the config - Switch to a smaller/faster model
- Enable GPU acceleration
- If using an API, the upstream is slow — try a different model or provider
No API key configured
You set provider = "api" but didn’t provide a key. Set LLM_MEM_LLM_API_KEY (or in the config), or change provider = "local".
Configuration Error: llm.context_size is too small
llm.context_size must be larger than memory.document_chunk_size / 2 + llm.max_tokens + 512. Increase context_size or decrease document_chunk_size.
Bank deletion requires user confirmation
You called cleanup_resources with target: "banks" but didn’t pass the exact confirmation phrase. This is a safety check — pass confirm: "I confirm this data will be permanently lost".
Still stuck?
- Run
llm-mem-mcp --generate-configto confirm the binary works - Run
llm-mem health-check --live(the CLI binary) to validate the config end-to-end - Check the GitHub issue tracker: https://github.com/vsndev3/llm-mem/issues
- Include the log output and your config (with secrets redacted) when asking for help
Next
Configuration
llm-mem has a single TOML config file (or no file at all — defaults work). The config has six top-level sections:
| Section | Purpose |
|---|---|
[llm] | Language model (local llama.cpp or OpenAI-compatible API) |
[embedding] | Embedding model (local fastembed or OpenAI-compatible API) |
[vector_store] | Storage backend (LanceDB) and bank directory |
[memory] | Memory behavior: thresholds, dedup, chunking, abstraction |
[server] | HTTP server config (host/port) |
[logging] | File logging with rotation |
Generating a template
The fastest way to see every option is to generate a commented template:
llm-mem-mcp --generate-config > config.toml
The result is a complete config.toml with every field commented out and explained.
Loading order
On startup, the server:
- Loads the config file (search order:
--configflag →./config.toml→~/.config/llm-mem/config.toml→/etc/llm-mem/config.toml) - Applies environment variable overrides (see Environment variables)
- Applies CLI flag overrides (see CLI flags)
- Validates the resulting config (refuses to start if invalid)
If no config file is found, all defaults are used and the server runs in fully-local mode with a Gemma 4 language model and an all-MiniLM embedding model.
Minimal local config
This is the default. You don’t actually need a config file, but if you want to customize banks_dir:
[vector_store]
banks_dir = "/var/lib/llm-mem/banks"
Minimal API config
[llm]
provider = "api"
api_key = "sk-..." # or set LLM_MEM_LLM_API_KEY
api_url = "https://api.openai.com/v1"
model = "gpt-4o-mini"
[embedding]
provider = "api"
api_key = "sk-..." # or set LLM_MEM_EMBEDDING_API_KEY
api_url = "https://api.openai.com/v1"
model = "text-embedding-3-small"
Mixed config: API LLM, local embeddings
[llm]
provider = "api"
api_key = "sk-..."
model = "gpt-4o-mini"
[embedding]
provider = "local" # keep embeddings local
model = "all-MiniLM-L6-v2"
Validation
The server validates the config at startup and refuses to start on errors. Common validations:
| Field | Constraint |
|---|---|
memory.similarity_threshold | 0.0 ≤ x ≤ 1.0 |
memory.merge_threshold | 0.0 ≤ x ≤ 1.0 |
memory.search_similarity_threshold | 0.0 ≤ x ≤ 1.0 (if set) |
llm.temperature | 0.0 ≤ x ≤ 2.0 |
llm.context_size | > 0 |
llm.max_tokens | > 0 |
llm.batch_max_tokens | > 0 and ≤ llm.max_tokens |
embedding.batch_size | > 0 |
memory.max_memories | > 0 |
memory.max_content_length | > 0 |
llm.context_size | ≥ memory.document_chunk_size / 2 + llm.max_tokens + 512 (local backend) |
llm.provider = "local" | requires build feature local-llm or local |
embedding.provider = "local" | requires build feature local-embed or local |
| API provider | requires api_key (or env var) |
If validation fails, the server prints a clear error message and exits.
Section reference
[llm]— language model settings[embedding]— embedding model settings[vector_store]— storage backend[memory]— memory behavior[server]— HTTP server[logging]— log output
Environment variables
Any config field can be set via environment variable. See Environment variables for the full list.
CLI flags
The llm-mem-mcp binary accepts flags that override the config at runtime. See CLI flags.
Next
Read through the section references, or jump to the one you need.
[llm] — language model
Settings for the language model used for extraction, summarization, classification, and query answering.
[llm]
provider = "local" # "local" (llama.cpp) or "api" (OpenAI-compatible)
Provider selection
| Value | What it does |
|---|---|
"local" | Embedded llama.cpp inference. No network, no API key. Downloads a GGUF model on first run. |
"api" | OpenAI-compatible HTTP API. Requires api_key (or env var). |
You can mix: see Embedding section for the combinations.
Local provider settings
[llm]
provider = "local"
model_file = "gemma-4-E2B-it-Q8_0.gguf" # GGUF filename (auto-downloaded if missing)
models_dir = "llm-mem-data/models" # where to put downloaded models
gpu_layers = 0 # 0 = CPU only; set to 20+ for GPU
context_size = 16644 # context window in tokens
cpu_threads = 0 # 0 = auto-detect
auto_download = true # fetch from HuggingFace on first run
cache_model = true # cache in ~/.cache/llm-mem/models
cache_dir = "" # custom cache dir (default: ~/.cache/llm-mem/models)
llm_timeout_secs = 120 # completion timeout
use_grammar = false # grammar-constrained sampling (faster, less flexible)
proxy_url = "" # proxy for downloads (overrides HTTPS_PROXY)
Vision (local)
If you have a vision-capable GGUF model + mmproj file:
vision_enabled = true
mmproj_file = "" # empty = auto-detect mmproj-F16.gguf for Gemma 4 E2B
vision_prompt_template = "Describe this image in detail, focusing on what would make it searchable in a personal memory system. What objects, people, text, colors, scene elements, actions, and contextual cues are visible? What domain or topic does this image relate to? Be specific and concrete."
When vision_enabled = true, ingested images (PNG/JPEG/GIF/WebP) get an AI-generated description stored alongside the original.
API provider settings
[llm]
provider = "api"
api_url = "https://api.openai.com/v1"
api_key = "" # or set LLM_MEM_LLM_API_KEY / OPENAI_API_KEY
model = "gpt-4o-mini"
Common api_url values:
| Service | URL |
|---|---|
| OpenAI | https://api.openai.com/v1 |
| OpenRouter | https://openrouter.ai/api/v1 |
| Anthropic | use api_dialect = "anthropic" with the Anthropic endpoint |
| Ollama | http://localhost:11434/v1 |
| llama-server | http://localhost:8080/v1 |
| LM Studio | http://localhost:1234/v1 |
Dialects
api_dialect = "openai-chat" # one of: openai-chat, openai-completion, anthropic, ollama-chat, ollama-completion, custom
openai-chat (default) works for any OpenAI-compatible chat completions endpoint. Use anthropic for Anthropic, ollama-* for Ollama native APIs, custom for fully custom request/response shapes.
Custom dialect
api_dialect = "custom"
[llm.custom_dialect]
endpoint_path = "/v1/generate"
request_body_template = '{"model": "{{model}}", "prompt": "{{prompt}}", "max_tokens": {{max_tokens}}, "temperature": {{temperature}}}'
response_content_pointer = "/choices/0/text"
Placeholders: {{prompt}}, {{model}}, {{temperature}}, {{max_tokens}}. response_content_pointer is a JSON pointer to the text in the response.
Request format
request_format = "auto" # "auto" (default), "rig", or "raw"
auto— try rig-core first, fall back to raw HTTP on 422 errors (most compatible)rig— always use rig-core (may 422 on strict backends)raw— always use raw HTTP with plain strings (bypasses rig-core)
Structured output
use_structured_output = true # JSON schema validation (API only)
structured_output_retries = 2 # retries on validation failure
The server asks the API for JSON schema-validated output, retrying if the LLM returns malformed JSON. Disable for backends that don’t support it.
Generation settings (both providers)
temperature = 0.7 # 0.0 = deterministic, 2.0 = very creative
max_tokens = 4096 # max tokens per completion
max_concurrent_requests = 1 # 0 = unlimited, 1 = serial (safest)
strip_tags = ["think"] # XML tags to strip from LLM output (e.g. "think", "reason")
Advanced / batching
batch_size = 10 # items per batch request
batch_max_tokens = 3000 # tokens per batch request (must be ≤ max_tokens)
batch_timeout_secs = 120 # base timeout for batch calls
batch_timeout_multiplier = 1.0 # timeout multiplier
The server batches independent LLM calls (e.g. classifying multiple memories at once) to reduce API overhead.
Next
[embedding] — embedding model
Settings for the embedding model that converts text to vectors for semantic search.
[embedding]
provider = "local" # "local" (fastembed) or "api" (OpenAI-compatible)
model = "all-MiniLM-L6-v2" # model name
Provider selection
| Value | What it does |
|---|---|
"local" | Embedded fastembed inference. No network, no API key. Downloads ONNX model on first run. |
"api" | OpenAI-compatible HTTP API. Requires api_key (or env var). |
Local provider settings
[embedding]
provider = "local"
model = "all-MiniLM-L6-v2" # any fastembed-supported model
Supported fastembed models include:
all-MiniLM-L6-v2(default, 384 dimensions, ~90 MB)all-mpnet-base-v2(768 dimensions, higher quality)BAAI/bge-small-en-v1.5(384 dimensions)BAAI/bge-large-en-v1.5(1024 dimensions)intfloat/multilingual-e5-large(1024 dimensions, multilingual)
The dimension is auto-detected from the model. The local backend requires the local-embed (or aggregate local) build feature.
API provider settings
[embedding]
provider = "api"
api_url = "https://api.openai.com/v1"
api_key = "" # or set LLM_MEM_EMBEDDING_API_KEY / OPENAI_API_KEY
model = "text-embedding-3-small"
Common model choices:
| Model | Dimensions |
|---|---|
text-embedding-3-small | 1536 |
text-embedding-3-large | 3072 |
text-embedding-ada-002 | 1536 |
Warning
Dimension must match storage —
If you change the embedding model, the dimension changes. New memories use the new dimension; old memories keep their old vectors. Configure the corresponding
[vector_store.lancedb].embedding_dimension(or VectorLite equivalent) to match.
Tuning
batch_size = 64 # texts per API call
timeout_secs = 30 # per-call timeout
batch_size is how many texts are bundled into a single embedding request. Larger batches are more efficient; smaller batches are more responsive.
Combinations
You can mix LLM and embedding providers:
| LLM | Embedding | Use case |
|---|---|---|
| local | local | Default. Fully offline. |
| api | api | Cloud LLM + cloud embeddings. |
| api | local | Cloud LLM with offline embeddings (faster, cheaper). |
| local | api | Local LLM with cloud embeddings. |
The four combinations are auto-detected from the two provider settings.
Next
[vector_store]— where the vectors are stored[llm]— the LLM settings
[vector_store] — storage backend
Settings for the embedded vector database (LanceDB by default) and the memory bank directory.
[vector_store]
banks_dir = "llm-mem-data/banks" # where bank databases live
store_type = "lancedb" # "lancedb" (default) or "vectorlite" (legacy)
collection_name = "llm-memories" # default collection name
Banks directory
banks_dir is the parent directory for all memory banks. Each bank gets its own subdirectory:
<banks_dir>/
├── default.lance/ # the default bank
├── default.sessions.db # document upload sessions for default
├── research.lance/ # another bank
└── research.sessions.db
To use a different location:
[vector_store]
banks_dir = "/var/lib/llm-mem/banks"
The directory is created automatically on first run if it doesn’t exist.
LanceDB settings (default)
[vector_store]
store_type = "lancedb"
[vector_store.lancedb]
table_name = "memories"
database_path = "./lancedb"
embedding_dimension = 384 # MUST match the embedding model
embedding_dimension is critical: it must equal the dimension of the embedding model. Mismatched dimensions cause a runtime error.
| Embedding model | Dimension |
|---|---|
all-MiniLM-L6-v2 | 384 |
all-mpnet-base-v2 | 768 |
BAAI/bge-small-en-v1.5 | 384 |
BAAI/bge-large-en-v1.5 | 1024 |
text-embedding-3-small | 1536 |
text-embedding-3-large | 3072 |
VectorLite settings (legacy)
[vector_store]
store_type = "vectorlite"
[vector_store.vectorlite]
index_type = "hnsw" # "hnsw" or "flat"
metric = "cosine" # "cosine", "euclidean", or "dot"
persistence_path = "" # defaults to banks_dir
embedding_dimension = 384 # optional; inferred from data if absent
VectorLite is the legacy backend. LanceDB is recommended for new deployments.
Note
VectorLite requires the
vector-litebuild feature, which is off by default. Build with--features vector-liteto use it.
Multiple banks
Banks are not configured in the TOML — they are created on demand by the AI calling create_memory_bank (or implicitly by writing to a bank name). Each bank gets its own .lance directory.
To pre-create banks on disk, use the CLI:
llm-mem --single list-banks
To restrict which banks exist, you can manage the directory directly:
mkdir -p /var/lib/llm-mem/banks
mkdir /var/lib/llm-mem/banks/research.lance
# Pre-creating an empty bank directory is allowed — the server will populate it on first write.
Storage size
LanceDB stores vectors + metadata. Rough estimates:
- 384-dim f32 vector: 1.5 KB
- 100k memories × 1.5 KB vectors + 1 KB metadata ≈ 250 MB on disk
- 1M memories ≈ 2.5 GB on disk
The .sessions.db file is a small SQLite file tracking document upload sessions — usually a few MB.
Next
[memory]— what happens when you store/search- Data directory layout — the full disk schema
[memory] — memory behavior
Settings for how memories are stored, deduplicated, chunked, and abstracted.
[memory]
max_memories = 10000
similarity_threshold = 0.65
search_similarity_threshold = 0.2
max_search_results = 50
Capacity
| Field | Default | What it controls |
|---|---|---|
max_memories | 10000 | Soft cap on memories per bank. Used for log warnings and admission control. |
max_search_results | 50 | Default upper bound on results per query. Tools can override per-call. |
max_list_limit | 10000 | Hard ceiling for list_memories with no explicit limit. Prevents accidental full-table scans. |
memory_ttl_hours | None | Time-to-live in hours. None = no expiry. Some(0) = no expiry (same as None). |
max_content_length | 32768 | Reject content longer than this (bytes). |
max_cascade_fanout | 5000 | Max abstraction dependents to load at once during cascade deletion. Prevents OOM. |
raw_content_scan_limit | 5000 | Max raw content candidates during keyword/text search fallback. |
max_total_candidates | 10000 | Max in-memory candidates during pyramid assembly before final ranking. |
Similarity thresholds
| Field | Default | What it controls |
|---|---|---|
similarity_threshold | 0.65 | Deduplication threshold. When a new memory’s embedding is closer than this to an existing one, dedup logic kicks in. |
merge_threshold | 0.75 | Higher threshold to merge two memories into one (vs. keep both). |
search_similarity_threshold | Some(0.2) | Reject search results below this. Lower = more results. |
near_duplicate_threshold | 0.92 | When a new memory’s embedding similarity to an existing memory exceeds this, a warning is logged. 0.0 disables. (Quality-control) |
auto_link_threshold | 0.75 | Similarity threshold for auto-creating references relations between new and existing memories. 0.0 disables auto-linking. |
auto_link_max_relations | 10 | Max auto-link relations per stored memory. |
Enhancement
| Field | Default | What it controls |
|---|---|---|
auto_enhance | true | Automatically enrich new memories with LLM-generated metadata (entities, keywords, importance). Costs LLM tokens. |
deduplicate | true | Skip storing near-duplicates. |
auto_summary_threshold | 32768 | When L0 content accumulates past this size, trigger the L0→L1 summarization worker. |
Document processing
| Field | Default | What it controls |
|---|---|---|
document_chunk_size | 2000 | Chunk size in characters for document ingestion. 0 = no chunking. |
chunk_threshold_chars | 1000 | When a single L0 memory exceeds this, split into sub-chunks for embedding. 0 = disable. |
chunk_size_chars | 1000 | Target sub-chunk size in characters. |
chunk_overlap_chars | 100 | Overlap between consecutive sub-chunks. |
Warning
chunk_threshold_chars > chunk_size_charswill log a warning at startup — it means some content past the threshold can still slip through unsplit if it’s underchunk_threshold_chars.
LLM-assisted parsing (off by default)
These cost LLM tokens but help with malformed or unknown content.
| Field | Default | What it controls |
|---|---|---|
use_llm_query_classification | false | Use LLM for query intent classification (vs. keyword heuristic). |
llm_format_detection | false | When content-based format detection returns “unknown”, ask the LLM to identify. Sends first 4 KB. |
llm_fallback_parsing | false | When structured parsers fail, ask the LLM to extract summary, entities, and content type. |
Cost & quality
| Field | Default | What it controls |
|---|---|---|
session_token_budget | 0 | Per-session LLM token budget. 0 = unlimited. When exceeded, expensive ops (enhance, classify, abstract) are skipped. |
dry_run | false | Log what would be done without executing LLM calls. Useful for cost estimation. |
contradiction_detection | false | When storing new memories, ask the LLM to check the top-3 semantically similar memories for factual contradictions. |
access_decay_hours | 168 | Time decay half-life (hours) for access-frequency boosting. Recent accesses boost score; older decay. 0 disables. (168 = 1 week) |
Next
[server] — HTTP server
The MCP transport is always stdio. The [server] section is reserved for an optional HTTP/management interface that isn’t currently used by the MCP server.
[server]
host = "0.0.0.0"
port = 8000
| Field | Default | What it controls |
|---|---|---|
host | "0.0.0.0" | Bind address (currently informational). |
port | 8000 | Bind port (currently informational). |
Note
This section is reserved for future use. The MCP server only speaks stdio. Standalone use of the
[server]section is not yet supported.
For practical server management, see Operations and Logging & debugging.
[logging] — log output
File-based logging with size-based rotation. Disabled by default; logs go to stderr.
[logging]
enabled = false
log_directory = "llm-mem-data/logs"
level = "info"
max_size_mb = 1
max_files = 5
| Field | Default | What it controls |
|---|---|---|
enabled | false | If true, write to llm-mem-mcp.log in log_directory in addition to stderr. |
log_directory | "llm-mem-data/logs" | Directory for the rotating log file. Created if missing. |
level | "info" | Default level: error, warn, info, debug, trace. |
max_size_mb | 1 | Rotate the log when it exceeds this size. |
max_files | 5 | Keep this many rotated log files (e.g. llm-mem-mcp.log.1, .2, etc.). |
Per-request verbosity
The level setting is the default, but the RUST_LOG env var (set in the MCP config’s env block) overrides it per-process. It accepts the tracing_subscriber filter syntax:
"env": {
"RUST_LOG": "info,llama_cpp_2=warn,llm_mem::search=debug"
}
This sets the global default to info, silences llama_cpp_2 to warn, and turns on debug for the search module.
Stderr vs file
When logging.enabled = true:
- The MCP JSON-RPC traffic goes to stdout (untouched — required by the protocol)
- All
tracingevents go to stderr (always) and to the rotating log file (when enabled)
This means the MCP client can talk to the server without log lines contaminating the protocol stream.
Examples
Quiet (errors only):
[logging]
enabled = true
level = "error"
Verbose debugging:
[logging]
enabled = true
level = "debug"
Larger log files, more history:
[logging]
enabled = true
max_size_mb = 50
max_files = 10
Next
- Logging & debugging — runtime tips, manual model download
- Environment variables
Environment variables
Any config field can be set via an environment variable. The variables are applied after the config file loads and before the server validates.
LLM
| Variable | Maps to | Notes |
|---|---|---|
LLM_MEM_LLM_API_KEY | llm.api_key | |
LLM_MEM_LLM_API_BASE_URL | llm.api_url | |
LLM_MEM_LLM_MODEL | llm.model | |
LLM_MEM_MODELS_DIR | llm.models_dir | |
LLM_MEM_GPU_LAYERS | llm.gpu_layers | u32 |
LLM_MEM_CONTEXT_SIZE | llm.context_size | u32 |
LLM_MEM_TEMPERATURE | llm.temperature | f32 |
LLM_MEM_MAX_TOKENS | llm.max_tokens | u32 |
LLM_MEM_CPU_THREADS | llm.cpu_threads | i32 |
LLM_MEM_MAX_CONCURRENT_REQUESTS | llm.max_concurrent_requests | usize |
Embedding
| Variable | Maps to | Notes |
|---|---|---|
LLM_MEM_EMBEDDING_API_KEY | embedding.api_key | |
LLM_MEM_EMBEDDING_API_BASE_URL | embedding.api_url | |
LLM_MEM_EMBEDDING_MODEL | embedding.model |
Fallback
| Variable | Maps to | Notes |
|---|---|---|
OPENAI_API_KEY | llm.api_key and embedding.api_key | Only fills if both are still empty after the specific env vars are applied. Standard OpenAI convention. |
Network
| Variable | Maps to | Notes |
|---|---|---|
HTTPS_PROXY | outbound HTTPS proxy | Used for model downloads. Format: http://host:port or http://user:pass@host:port |
HTTP_PROXY | outbound HTTP proxy | |
ALL_PROXY / all_proxy | SOCKS proxy |
The llm.proxy_url config (or --proxy CLI flag) overrides the env-var proxy.
Logging
| Variable | Notes |
|---|---|
RUST_LOG | tracing-subscriber filter syntax. Overrides the logging.level config. |
APPIMAGE_EXTRACT_AND_RUN | AppImage-only. Run without FUSE by extracting to a tmpfs. |
Abstraction pipeline
Unlike the tables above, these variables are not config-field overrides — they are read directly at startup. Invalid values are logged and fall back to the default.
| Variable | Default | Notes |
|---|---|---|
LLM_MEM_ABSTRACTION_DELAY_SECS | 30 | Delay between abstraction pipeline cycles. |
LLM_MEM_ABSTRACTION_CONCURRENCY | 3 | Concurrent abstraction tasks per cycle (max_concurrent_tasks). Raise for batch/benchmark workloads. |
LLM_MEM_BACKGROUND_LLM_CONCURRENCY | 3 | Background LLM permits available to the abstraction pipeline. Raise alongside LLM_MEM_ABSTRACTION_CONCURRENCY so concurrency isn’t throttled by the interactive-friendly default. |
Usage in MCP config
Set environment variables in the MCP config’s env block:
{
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/path/to/llm-mem-mcp"],
"env": {
"LLM_MEM_LLM_API_KEY": "sk-...",
"LLM_MEM_MODELS_DIR": "/home/you/.cache/llm-mem/models",
"RUST_LOG": "info,llm_mem::search=debug",
"HTTPS_PROXY": "http://proxy.corp:8080"
}
}
}
}
Tip
For per-machine settings (proxy, models dir, API key), prefer the
envblock in the MCP config. For per-deployment settings (URL, model), prefer the TOML config file. Don’t put secrets in the TOML if the file is committed.
Override order
When the same field is set in multiple places, the priority is:
- CLI flag (highest)
- Specific env var (e.g.
LLM_MEM_LLM_API_KEY) - Generic env var (e.g.
OPENAI_API_KEY) - Config file value
- Default (lowest)
So --api-url on the CLI beats LLM_MEM_LLM_API_BASE_URL which beats [llm].api_url in config.toml which beats the hardcoded default.
Next
- CLI flags — runtime overrides
[llm],[embedding]
CLI flags
The llm-mem-mcp binary accepts these flags. They override both the config file and environment variables.
llm-mem-mcp [OPTIONS]
General
| Flag | Short | Type | Default | What it does |
|---|---|---|---|---|
--config <PATH> | -c | path | auto-discover | Path to the config file. If not given, searches ./config.toml, ~/.config/llm-mem/config.toml, /etc/llm-mem/config.toml. |
--agent <ID> | string | none | Default agent_id for all tool calls. Overrides the per-tool agent_id parameter when not set. | |
--proxy <URL> | string | none | Proxy URL for model downloads. Format: http://host:port or http://user:pass@host:port. Overrides HTTPS_PROXY. | |
--banks-dir <PATH> | path | from config | Override vector_store.banks_dir. | |
--generate-config | bool | false | Print a commented config template to stdout and exit. |
LLM behavior
| Flag | Type | Default | What it does |
|---|---|---|---|
--no-grammar | bool | false | Disable grammar-constrained sampling for local LLM structured output. Grammar is enabled by default. |
--no-structured-output | bool | false | Disable structured output mode for API LLM. Structured output is enabled by default. |
--request-format <FMT> | string | auto | auto, rig, or raw. See [llm] request_format. Invalid values cause the server to exit. |
Model caching
| Flag | Type | Default | What it does |
|---|---|---|---|
--no-cache-model | bool | false | Don’t cache models in ~/.cache/llm-mem/models. Caching is enabled by default. |
--cache-dir <PATH> | path | ~/.cache/llm-mem/models | Custom cache directory. |
Examples
# Use a specific config file
llm-mem-mcp --config /etc/llm-mem/prod.toml
# Run with raw HTTP (skip rig-core) against a self-hosted llama.cpp
llm-mem-mcp --request-format raw
# Force re-download the model (skip cache)
llm-mem-mcp --no-cache-model
# Pre-populate the cache, then run normally
llm-mem-mcp --cache-dir /mnt/fast/models
# Use a different banks directory
llm-mem-mcp --banks-dir /mnt/data/banks
# Print the config template
llm-mem-mcp --generate-config > config.toml
Standalone CLI
The llm-mem (no -mcp suffix) binary has a different flag set. It has subcommands like search, list, stats, db, etc. See CLI overview.
Next
MCP tools reference
llm-mem-mcp exposes 32 tools to MCP clients. They’re grouped by purpose here.
The 32 tools at a glance
Storing
add_content_memory— raw verbatim textadd_intuitive_memory— AI-extracted structured factsstore_memories— batch storeupload_document— small documents in one shotingest— alternate ingest entry point
Finding
query_memory— full pyramid + graph searchsearch_memory— simple semantic searchlist_memories— filter-based browseget_memory— by ID
Chronology
get_timeline— bucketed listget_timeline_graph— nodes + edgesget_context_resume— progressive precision
Navigation & relations
navigate_memory— zoom in/out across pyramidupdate_memory— edit content or relationsforce_link— create a relationremove_relation— delete a relation
Abstraction
start_abstraction_pipeline— start background workersstop_abstraction_pipeline— pause workerstrigger_abstraction— force one specific abstractioncreate_abstraction— manual layer creation
Banks
list_memory_bankscreate_memory_bankrename_memory_bankbackup_bankrestore_bankcleanup_resources— delete models or banks
Documents
Introspection
system_status— health + token statshealth_check— config + live backend probecheck_consistency— DB integrityhelp— built-in tool guidance
How parameters work
Every tool takes a JSON object of parameters. Optional parameters can be omitted. Parameters marked as accepting a bank field default to the "default" bank if not provided.
The common bank parameter
Most tools accept an optional bank string. If omitted, the tool operates on the "default" bank.
{ "query": "JWT auth", "bank": "research" }
Agent and user IDs
| Field | Used by | Purpose |
|---|---|---|
agent_id | store/find tools | The agent storing or querying. Set once via --agent CLI flag and inherited by all calls. |
user_id | store tools | Optional. Only needed when multiple users share a bank. |
Common response shape
Tools return a JSON object with at least:
{
"success": true,
"message": "human-readable status",
"data": { /* tool-specific */ },
"error": null
}
On failure, success is false and error contains a human-readable description.
Reading the per-tool pages
Each category page has a one-paragraph description of the tool, the input schema (as a table), and the output schema. For the canonical schema, see src/operations/tools.rs in the source.
Next
Storing memories
Tools for writing new memories to a bank. There are four “store” variants plus document ingestion (covered under Document upload).
add_content_memory
Add raw content to memory without any AI transformation. The content is stored and embedded exactly as-is, preserving all original phrases, keywords, and structure.
Use this when:
- You need exact-phrase searchability (e.g. finding “vegan chili” later)
- Storing conversation logs, documents, or code snippets where original text matters
- You want predictable semantic search based on the actual content, not AI interpretations
For AI-processed structured facts, use add_intuitive_memory instead.
Input
| Field | Type | Required | Description |
|---|---|---|---|
content | string | ✓ | The fact or piece of information. Should be concise and atomic. |
metadata | object | Key-value pairs (source file, page, timestamp, author). | |
user_id | string | Only if multiple users share a bank. | |
agent_id | string | Agent ID. Inherited from CLI if unset. | |
topics | array of string | Subjects within the content. | |
context | array of string | Broad topic tags. | |
relations | array of object | {relation, target} pairs to other memories or entities. | |
bank | string | Defaults to "default". | |
auto_link | boolean | Whether to auto-create references relations to similar memories. Server default if unset. | |
event_at | string | ISO 8601 datetime — when the event actually happened. | |
source | string | Free-form provenance (file name, URL, book title). | |
force | boolean | Skip near-duplicate / contradiction checks. |
Output
{
"success": true,
"message": "Memory stored",
"data": {
"memory_id": "uuid",
"user_id": "...",
"agent_id": "..."
}
}
add_intuitive_memory
Add memories with AI-powered extraction and structuring. The LLM analyzes your content, extracts key facts, organizes them into atomic insights, and generates searchable keywords.
Use this when:
- You want structured, reasoning-ready memories
- You need condensed insights from long conversations or documents
- You want automatic keyword extraction for hybrid search
The original text is transformed by the LLM. For verbatim storage, use add_content_memory.
Input
| Field | Type | Required | Description |
|---|---|---|---|
messages | array of object | ✓ | Conversation messages: {role, content, name?}. |
metadata | object | Attached to all extracted memories. | |
user_id | string | ||
agent_id | string | ||
bank | string | ||
source_memory_id | string | Link extracted memories to a source. Auto-creates derived_from relation. | |
event_at | string | ISO 8601 — applied to all extracted memories. |
Output
{
"success": true,
"message": "...",
"data": {
"results": [
{ "id": "uuid", "memory": "...", "event": "...", "actor_id": "...", "role": "..." }
],
"user_id": "...",
"agent_id": "..."
}
}
data.results holds one entry per extracted memory.
store_memories
Store multiple content memories in a single call. Much faster than calling add_content_memory repeatedly for bulk ingestion.
Rules:
itemscannot be empty- Each item must have non-empty
content - Invalid items cause the entire batch to be rejected before any storage
Input
| Field | Type | Required | Description |
|---|---|---|---|
items | array of object | ✓ | Each: {content, topics?, context?, relations?, metadata?, event_at?, source?}. |
bank | string | ||
force | boolean | Skip near-duplicate / contradiction checks for the whole batch. | |
wait | boolean | false (default) returns immediately with a batch_id and processes in the background (poll get_batch_status); true blocks until all items are stored. |
Output
{
"success": true,
"message": "Stored 5 memories",
"data": {
"results": [ /* per-item results */ ],
"total": 5,
"batch_id": "uuid",
"status": "completed"
}
}
Which to use?
| Scenario | Tool |
|---|---|
| “Remember this exact sentence” | add_content_memory |
| “Extract the key facts from this conversation” | add_intuitive_memory |
| “Ingest these 100 notes at once” | store_memories |
| “Store this PDF / markdown file” | upload_document |
| “Ingest this file, please chunk it and extract structure” | ingest |
Quality-control flags
add_content_memory accepts a force boolean:
force: false(default) — the server checks for near-duplicates and contradictions. On detection, the call is blocked and the error describes the conflict.force: true— bypass the checks. Use only when you’re certain the content should be stored despite the warning.
store_memories has a top-level force flag applying to the whole batch. add_intuitive_memory has no force flag — it always runs the near-duplicate / contradiction checks.
Warning
Setting
force: truedefeats the contradiction-detection safety net. Use it sparingly and only after reviewing what the conflict actually is.
Next
Finding memories
Tools for retrieving stored memories. Four tools cover different access patterns.
query_memory
The most powerful search tool. Supports hybrid keyword/semantic search, pyramid allocation, graph traversal, and context filtering.
Important: score interpretation
Scores reflect semantic similarity, NOT whether the answer is present. A high score means the memory is topically related — it does not guarantee the answer exists in that memory. Always verify retrieved content against what was asked:
- Use
k >= 5to ensure adequate recall. - L0 = concrete source content (facts, verbatim text). The answer lives here.
- L1+ = increasingly abstract (summaries → concepts → wisdom). These help you navigate to relevant L0 facts but are not themselves the answer.
Pyramid allocation (pyramid_config)
Pyramid search distributes result slots across abstraction layers, so a single query returns both concrete facts and abstract insights. Set pyramid_config.mode to one of:
| Mode | Behavior |
|---|---|
"bottom_heavy" (default) | More L0 facts, fewer abstract concepts |
"balanced" | Equal distribution across layers |
"top_heavy" | More abstract concepts, fewer concrete facts |
"dynamic" | LLM classifies query intent automatically (requires use_llm_query_classification config) |
"none" | Skip pyramid assembly, return flat results by raw score |
pyramid_config also accepts layer_weights (per-layer weight overrides) and per_layer_multiplier (default 2.0).
Graph traversal (graph_traversal)
Enable graph_traversal.enabled to replace semantic search with a pure BFS multi-hop traversal, following relations (derived_from, mentions, knows, …) from the top-scoring entry points:
| Field | Description |
|---|---|
enabled | Turn on deep traversal (default false). |
max_depth | Max hops (1-5, default 2). |
direction | "outgoing", "incoming", or "both" (default). |
relation_types | Restrict to specific relation types. |
entry_point_limit | Max seed memories (default 5, max 10). |
include_paths | Include per-result graph_info (distance, path, boosts). |
Without deep traversal, standard search still applies automatic 1-hop graph refinement on the top results.
Input
| Field | Type | Required | Description |
|---|---|---|---|
query | string | ✓ | The search query. |
k | integer | Max results (default 10). | |
bank | string | Memory bank name (default "default"). | |
topics | array of string | Filter by topics. | |
context | array of string | Context tags for semantic scoping. | |
keyword_only | boolean | Keyword-only search, no embeddings (default false). | |
keyword_split_ratio | float | Fraction of results from raw keyword matching vs. semantic (0.0-1.0, default 0.2). | |
similarity_threshold | float | Override similarity threshold (0.0-1.0). | |
min_salience | float | Minimum salience/importance score (0-1). | |
granularity | string | "full" (default) resolves chunk hits to the complete parent memory; "excerpt" returns a compact window around matched regions (session header preserved), under excerpt_max_chars. | |
excerpt_max_chars | integer | Total char budget for excerpt-mode content (default 12000, min 1000). | |
pyramid_config | object | mode, layer_weights, per_layer_multiplier (see above). | |
graph_traversal | object | Deep traversal config (see above). | |
created_after / created_before | string | ISO 8601 — filter by storage time. | |
event_after / event_before | string | ISO 8601 — filter by event_at (or created_at). | |
user_id / agent_id | string | Filter by author. |
Output
{
"success": true,
"count": 12,
"memories": [
{
"memory_id": "uuid",
"content": "...",
"score": 0.87,
"layer": 0,
"layer_name": "raw_content",
"context": ["project-x", "auth"],
"topics": ["jwt"],
"metadata": { /* ... */ },
"event_at": "2026-02-15T10:00:00Z",
"created_at": "2026-02-15T11:23:45Z"
}
]
}
search_memory
A simplified search with sensible defaults. Use this when you don’t need the full pyramid/graph power.
Same score interpretation rules apply: high similarity does not guarantee the answer is present, use k >= 5, and L0 is where concrete facts live.
Input
| Field | Type | Required | Description |
|---|---|---|---|
query | string | ✓ | |
k | integer | Max results (default 10). | |
bank | string |
Output
Same { success, count, memories } shape as query_memory.
list_memories
Browse by structured filter — no semantic similarity. Useful for “show me everything I stored yesterday”.
Input
| Field | Type | Required | Description |
|---|---|---|---|
limit | integer | Default 100, max 1000. | |
bank | string | ||
created_after / created_before | string | ISO 8601. | |
event_after / event_before | string | ISO 8601. | |
user_id / agent_id | string |
Output
{
"success": true,
"count": 1234,
"memories": [ /* memory objects */ ]
}
get_memory
Look up a specific memory by ID.
Input
| Field | Type | Required | Description |
|---|---|---|---|
memory_id | string | ✓ | |
bank | string |
Output
{
"success": true,
"memory": {
"memory_id": "uuid",
"content": "...",
"layer": 0,
"context": ["..."],
"topics": ["..."],
"metadata": { /* ... */ },
"event_at": "...",
"created_at": "...",
"updated_at": "..."
}
}
Choosing the right tool
| I want to… | Use |
|---|---|
| Find memories that match a question / topic | query_memory or search_memory |
| Browse by date / type / tag | list_memories |
| Get the full record for a specific memory ID | get_memory |
| Traverse relations from a seed memory | query_memory with graph_traversal: { enabled: true } |
| Favor abstract concepts over raw facts | query_memory with pyramid_config: { mode: "top_heavy" } |
| Keep retrieved content compact | query_memory with granularity: "excerpt" |
Next
- Chronological queries — timeline views
- Navigation & relations — pyramid zoom
Chronological queries
Tools for time-based views of memories. Three tools cover different temporal access patterns.
All three bucket by event_at (the date the content refers to); if event_at is missing, they fall back to created_at.
get_timeline
Returns memories bucketed by time interval. Answers “what happened in the last 2 days” or “give me a monthly summary of 2025”.
Input
| Field | Type | Required | Description |
|---|---|---|---|
start | string | ISO 8601 lower bound. Default: end - 7d. | |
end | string | ISO 8601 upper bound. Default: now. | |
granularity | string | hour, day (default), week, month, none (single bucket). | |
bank | string | Memory bank (default "default"). | |
topics | array of string | Filter to memories tagged with any of these topics. | |
max_results_per_bucket | integer | Cap per bucket. Default 50. | |
include_derived | boolean | Include L1+ abstractions (default false, L0 only). | |
order | string | asc (default) or desc within each bucket. | |
user_id / agent_id | string | Filter by author. |
Output
{
"success": true,
"start": "2026-02-14T00:00:00Z",
"end": "2026-02-16T00:00:00Z",
"granularity": "day",
"total_count": 23,
"bucket_count": 2,
"buckets": [
{
"start": "2026-02-14T00:00:00Z",
"end": "2026-02-15T00:00:00Z",
"label": "2026-02-14",
"count": 8,
"memories": [ /* truncated to max_results_per_bucket */ ]
}
]
}
Examples
// "What happened in the last 2 days?"
{ "end": "2026-02-15T00:00:00Z", "granularity": "day" }
// "Monthly summary of 2025"
{ "start": "2025-01-01T00:00:00Z", "end": "2026-01-01T00:00:00Z", "granularity": "month" }
get_timeline_graph
Returns a graph of memories: nodes (memories sorted by event_at) plus edges. Edges are either:
- Auto-derived temporal:
happened_after— connecting memories whose events were close in time (configurable window) - Auto-derived co-occurrence:
happens_within— near-simultaneous events (include_simultaneous) - Optional semantic: existing relation edges (
derived_from,mentions, …) traversed from the relation graph (include_semantic_edges)
Use this tool to render a timeline as a network with D3, Graphviz, etc.
Input
| Field | Type | Required | Description |
|---|---|---|---|
start / end | string | Time window. | |
granularity | string | hour, day, week, month, none. | |
bank | string | ||
topics | array of string | Filter by topics. | |
max_results_per_bucket | integer | ||
include_derived | boolean | Include L1+ abstractions. | |
order | string | asc / desc. | |
max_depth | integer | Semantic-relation hops from each node (default 1, max 3). | |
relation_types | array of string | Whitelist of semantic relation types to follow. | |
temporal_edge_window_secs | integer | Window for happened_after edges (default 86400 = 1 day). | |
include_simultaneous | boolean | Also derive happens_within edges for near-simultaneous events. | |
simultaneous_window_secs | integer | Window for happens_within (default 60). | |
include_semantic_edges | boolean | Include semantic-relation edges (default true). | |
user_id / agent_id | string |
Output
{
"success": true,
"start": "...",
"end": "...",
"granularity": "day",
"stats": {
"node_count": 47,
"edge_count": 92,
"temporal_edge_count": 80,
"semantic_edge_count": 12
},
"nodes": [
{ "id": "uuid", "event_at": "...", "layer": 0, "bucket": "2026-02-14", "memory": {} }
],
"edges": [
{ "source": "uuid-a", "target": "uuid-b", "type": "happened_after", "delta_secs": 3600 }
]
}
get_context_resume
Progressive precision: the most recent window returns L0 memories at full detail; progressively older windows return higher-layer abstractions (L1 summaries, L2 links, L3 concepts). Produces an exponential decay curve where precision peaks at the current time. Use this as the first call when resuming a session.
Input
| Field | Type | Required | Description |
|---|---|---|---|
end | string | ISO 8601 end of window. Default: now. | |
lookback | string | Total lookback from end, e.g. 30d (default), 12h, 1w. | |
decay_factor | float | Each segment is ~N× the previous one’s duration. Default 2.0. | |
segments | integer | Number of precision tiers (1-10). Default 5. | |
max_per_segment | integer | Cap memories per segment. Default 20. | |
bank | string | ||
topics | array of string | Filter by topics. | |
user_id / agent_id | string |
Output
{
"success": true,
"start": "...",
"end": "...",
"total_lookback_secs": 2592000,
"decay_factor": 2.0,
"segment_count": 5,
"total_memories": 87,
"segments": [
{
"label": "1d 06-03 — 06-04",
"layer": 0,
"start": "...",
"end": "...",
"duration_secs": 86400,
"count": 12,
"memories": [ /* L0 full detail */ ]
}
]
}
Segment 0 (most recent) queries L0, segment 1 queries L1, and so on — newer segments have full detail, older ones are progressively compressed.
When to use which
| I want to… | Use |
|---|---|
| Quick “what happened recently” list | get_timeline |
| Visualize memories as a temporal graph | get_timeline_graph |
| Get a “context resume” of a long time period, in bounded size | get_context_resume |
| Group by hour for fine-grained analysis | get_timeline with granularity: "hour" |
| See causal/temporal relations between memories | get_timeline_graph with include_semantic_edges: true |
Next
Navigation & relations
Tools for moving through the memory pyramid and managing relations between memories.
navigate_memory
Move up and down the abstraction hierarchy from a seed memory. zoom_out returns higher-layer (more abstract) memories derived from it; zoom_in returns lower-layer (more detailed) source memories it was abstracted from; both returns both directions.
Input
| Field | Type | Required | Description |
|---|---|---|---|
memory_id | string | ✓ | The seed memory. |
direction | string | zoom_in, zoom_out, or both (default). | |
levels | integer | Levels to traverse for zoom_in (1-5, default 1). | |
bank | string |
Output
{
"success": true,
"source_memory_id": "uuid",
"source_layer": 1,
"zoom_in": [ /* lower-layer source memories */ ],
"zoom_out": [ /* higher-layer abstractions */ ]
}
Examples
// From a JWT note, find the auth-architecture concept built on top of it
{ "memory_id": "uuid-of-jwt-note", "direction": "zoom_out" }
// From a concept memory, find all source evidence
{ "memory_id": "uuid-of-concept", "direction": "zoom_in", "levels": 3 }
Note
To restrict a semantic search to a specific layer, use
query_memorywithpyramid_config(e.g.{ "mode": "top_heavy" }orlayer_weights), notnavigate_memory.
update_memory
Update an existing memory’s content and/or append relations by ID.
Input
| Field | Type | Required | Description |
|---|---|---|---|
memory_id | string | ✓ | |
content | string | New content. Re-embeds the memory. | |
relations | array | Relations to append, as [{ "relation": "...", "target": "..." }]. | |
bank | string |
Output
{
"success": true,
"message": "Memory updated"
}
Warning
Updating
contentre-embeds the memory. This is an LLM-free operation (uses the same embedding model), but it does cost time and changes the semantic identity. Use it for typo fixes, not wholesale rewrites.
force_link
Manually create a relation between two existing memories. The reverse relation is created automatically (e.g. references → referenced_by).
Input
| Field | Type | Required | Description |
|---|---|---|---|
source_id | string | ✓ | Source memory ID (the “from” side). |
relation | string | ✓ | Relation type: references, contradicts, supports, depends_on, part_of, extends, similar_to, summary_of, synthesizes. |
target_id | string | ✓ | Target memory ID (must be an existing memory). |
strength | number | Relation strength 0.0-1.0 (default 1.0). | |
bank | string |
Output
{
"success": true,
"message": "Relation created",
"data": {
"source_id": "uuid",
"relation": "depends_on",
"target_id": "uuid",
"reverse_relation": "depended_on_by",
"reverse_created": true
}
}
Note
source_idandtarget_idmust both be valid UUIDs of existing memories. Hierarchical relations (summary_of,part_of,synthesizes) require the source to be at a higher layer than the target. Duplicate links are rejected.
Suggested relation predicates
| Category | Predicates |
|---|---|
| Structural | chunk_of, summary_of, part_of, next_chunk, previous_chunk |
| Semantic | references, extends, supersedes, depends_on, implements, configures |
| Layered | derived_from, emerges_from, instance_of, broader_than |
| Provenance | authored_by, cited_by |
| Contradiction | contradicts |
remove_relation
Remove a specific relation from a memory. The reverse relation on the target is removed automatically.
Input
| Field | Type | Required | Description |
|---|---|---|---|
memory_id | string | ✓ | Memory to remove the relation from. |
relation_type | string | ✓ | The relation type to remove (e.g. references). |
target_id | string | ✓ | The target memory ID in the relation. |
bank | string |
Output
{
"success": true,
"message": "Relation removed",
"data": {
"memory_id": "uuid",
"removed_relation": "references",
"removed_target": "uuid",
"reverse_relation": "referenced_by",
"reverse_cleaned": true
}
}
Relations vs auto-linking
When auto-linking is enabled (via auto_link_threshold), the server automatically creates references relations to semantically similar existing memories. force_link is for cases where:
- You know of a relation the auto-linker missed
- You want a non-semantic relation (e.g.
next_chunk,part_of) - You want to control the relation strength explicitly
Next
Abstraction pipeline
Tools for controlling the background workers that build higher layers of the pyramid from lower-layer content.
start_abstraction_pipeline
Start (or resume) the background workers. The pipeline auto-starts on server boot, so this is mainly useful after stop_abstraction_pipeline.
Input
None.
Output
{
"success": true,
"message": "Abstraction pipeline started"
}
stop_abstraction_pipeline
Pause the background workers. Useful before bulk operations or for debugging.
Input
None.
Output
{
"success": true,
"message": "Abstraction pipeline stopped"
}
Note
stop_abstraction_pipelinedoes not cancel an in-flight LLM call — it just stops scheduling new work. Requests already in progress will complete.
trigger_abstraction
Run one-shot abstraction processing now (unlike the pipeline, this runs once and does not start background workers). The pipeline must already be running.
Input
| Field | Type | Required | Description |
|---|---|---|---|
target_layer | integer | 1 = L0→L1 (summaries), 2 = L1→L2, 3 = L2→L3, 0 = all. Default 1. |
Output
{
"success": true,
"l0_to_l1_created": 3,
"l1_to_l2_created": 1,
"l2_to_l3_created": 0,
"errors": []
}
create_abstraction
Create a manual abstraction (L1/L2/L3) from specific source memory IDs, with your own content.
Input
| Field | Type | Required | Description |
|---|---|---|---|
content | string | ✓ | The abstraction content (summary, synthesis, or concept). |
source_ids | array of string | ✓ | Source memories this abstraction derives from. |
target_layer | integer | ✓ | Target layer (1=structural, 2=semantic, 3=concept). Must be higher than all source layers. |
relation_type | string | Relation to sources. Defaults by layer: summary_of (L1), synthesizes (L2), abstracts_to_concept (L3+). | |
user_id / agent_id | string | ||
bank | string |
Output
{
"success": true,
"message": "Abstraction created",
"data": {
"memory_id": "uuid",
"target_layer": 2,
"relation_type": "synthesizes",
"source_count": 3,
"reverse_relation": "synthesized_by",
"reverse_created": true
}
}
What the pipeline actually does
The pipeline is a set of background workers, one per layer transition:
| Worker | Trigger | What it produces |
|---|---|---|
| L0 → L1 | When total L0 content exceeds auto_summary_threshold bytes | L1 summaries grouped by topic. relation: "summary_of" from summary to sources. |
| L1 → L2 | When multiple L1 summaries share themes | L2 connections with relation: "related_to", references, etc. |
| L2 → L3 | When L2 clusters form | L3 concepts with relation: "emerges_from". |
| L3 → L4+ | When L3 concepts span domains | L4+ insights / mental models. |
Each worker:
- Selects candidate source memories
- Asks the LLM to synthesize
- Stores the new abstraction
- Creates the
derived_from/emerges_from/ etc. relations - Notifies the next-layer worker (which may decide to act)
Workers respect session_token_budget and circuit-breaker around the LLM. Failed abstractions get retried with exponential backoff; persistent failures get logged and skipped (use clear-backoff from the CLI to force retry).
How to use it
In normal operation, you don’t. The pipeline runs and gradually builds higher layers as content accumulates.
You intervene when:
- You just stored something important and want insights now →
trigger_abstraction - You want to see the architecture before doing bulk ingestion →
stop_abstraction_pipeline, do the ingest,start_abstraction_pipeline - You want a specific abstraction on demand →
create_abstractionwith explicit source IDs and content - The pipeline is failing repeatedly →
clear-backoff(CLI), thentrigger_abstractionto debug
Next
Banks & backups
Tools for managing multiple isolated memory banks, backing them up, and cleaning up resources.
list_memory_banks
List all memory banks in the configured banks_dir.
Input
None.
Output
{
"success": true,
"data": {
"banks": [
{
"name": "default",
"memory_count": 1234,
"size_bytes": 45678901,
"created_at": "..."
},
{
"name": "research",
"memory_count": 567,
"size_bytes": 12345678,
"created_at": "..."
}
],
"count": 2,
"banks_dir": "/var/lib/llm-mem/banks"
}
}
create_memory_bank
Create a new bank. The bank is also created implicitly on the first write to a new name, so this is mostly for documentation or to pre-create a known structure.
Input
| Field | Type | Required | Description |
|---|---|---|---|
name | string | ✓ | Bank name. Allowed chars: letters, digits, -, _. |
description | string | Human-readable description. |
Output
{
"success": true,
"message": "Memory bank 'research' ready",
"data": {
"bank": {
"name": "research",
"memory_count": 0,
"size_bytes": 0
}
}
}
rename_memory_bank
Atomically rename a bank. Moves both the memory database and the session database.
Input
| Field | Type | Required | Description |
|---|---|---|---|
old_name | string | ✓ | |
new_name | string | ✓ |
Output
{
"success": true,
"message": "Bank renamed from 'foo' to 'bar'",
"data": { "old_name": "foo", "new_name": "bar" }
}
Warning
If any AI client is currently using the old name, it’ll see a “bank not found” error after the rename. Update the client config (or recreate the bank) before continuing.
backup_bank
Create a versioned backup of a bank. Backups are SHA-256 verified and timestamped.
Input
| Field | Type | Required | Description |
|---|---|---|---|
name | string | Bank to back up. Default: default. | |
destination | string | Directory for the backup. Default: ~/llm-mem-backups/. |
Output
{
"success": true,
"message": "Bank 'default' backed up successfully (v3)",
"data": {
"backup_path": "/home/you/llm-mem-backups/default-2026-02-15-v3.db",
"manifest": {
"version": 3,
"created_at": "2026-02-15T...",
"memory_count": 1234,
"sha256": "...",
"size_bytes": 45678901
}
}
}
restore_bank
Restore a bank from a backup file. Two modes:
replace(default) — overwrite the current bank. Requiresconfirm: trueas a safety check.merge— import the backup into the existing bank. Duplicates are skipped by content hash.
Input
| Field | Type | Required | Description |
|---|---|---|---|
name | string | Bank to restore into. Default: default. | |
source | string | ✓ | Path to the backup .db file. |
mode | string | replace (default) or merge. | |
confirm | boolean | Required: true for replace mode. |
Output
{
"success": true,
"message": "Bank 'default' restored from backup (replace mode)",
"data": {
"restored_path": "/var/lib/llm-mem/banks/default.lance",
"source": "/path/to/backup.db"
}
}
For merge mode, the response also includes imported, skipped_duplicates, and total_after_merge.
cleanup_resources
Delete models or memory banks. Two target values:
"models"— delete the entiremodels_dir(then re-create it empty). The next run will re-download."banks"— delete a specific bank by name. Requires the literal confirmation phrase.
Input
| Field | Type | Required | Description |
|---|---|---|---|
target | string | "models" (default) or "banks". | |
name | string | (for target: "banks") | Bank to delete. |
confirm | boolean or string | For target: "models": confirm: true. For target: "banks": confirm: "I confirm this data will be permanently lost". |
Output
{
"success": true,
"message": "Models directory cleaned up successfully. ..."
}
[!DANGER] Destructive —
cleanup_resourceswithtarget: "banks"permanently deletes the bank’s database. There is no undo. The confirmation phrase is a safety net — never bypass it.
Backup strategy recommendations
| Use case | Recommendation |
|---|---|
| Personal use, low churn | Weekly backup, keep last 4. |
| Multi-project, active | Daily backup, keep last 7. |
| Production / shared | Continuous backup, off-site storage, restore tested. |
Backups are stored as .db files plus a manifest. They include the bank database but not the documents in the upload session state. To capture document sessions too, use db export --include-sessions (CLI).
Next
Document upload
Tools for ingesting whole files (markdown, code, PDFs, spreadsheets, images, etc.) into memory with automatic chunking and abstraction.
upload_document
Upload a file from the server’s local filesystem. The server reads the file, splits it into chunks, embeds each chunk, and stores them as L0 memories. The L0→L1 worker then creates summaries in the background.
Input
| Field | Type | Required | Description |
|---|---|---|---|
file_path | string | ✓ | Absolute path to the file. Must be readable by the server process. |
file_name | string | Optional name (defaults to the basename of file_path). | |
mime_type | string | Optional MIME type (default text/plain). | |
chunk_size | integer | Override document_chunk_size. | |
process_immediately | boolean | Start processing after upload (default true). | |
topics | array | Topics applied to all chunks. | |
context | array | Context tags applied to all chunks. | |
event_at | string | ISO 8601 for when the document’s events occurred. Defaults to upload time. | |
bank | string | ||
user_id / agent_id | string |
Output
The response is not schema-constrained. A representative shape:
{
"success": true,
"data": {
"session_id": "uuid",
"file_name": "architecture.md",
"chunks_stored": 23,
"memory_ids": ["uuid-a", "uuid-b", "…"]
}
}
Limits
upload_document is best for small to medium files (up to a few MB). For very large files (10s of MB+) or files you already have in memory, use the ingest tool, or the CLI’s multi-part upload flow (begin-upload / upload-part / process-document, see CLI commands).
ingest
Universal decomposition entry point. Takes raw content (a string, or base64 for binary) and automatically detects the format, decomposes it into semantic L0 chunks, creates structural relations between chunks, and optionally auto-links to existing memories.
Unlike upload_document, this does not read a file from disk — you pass the content directly. For binary formats (PDF, DOCX, images), base64-encode the content and set content_encoding: "base64".
Input
| Field | Type | Required | Description |
|---|---|---|---|
content | string | ✓ | Raw content. For binary formats, base64-encoded. |
content_encoding | string | "base64" for binary formats. | |
format_hint | string | Optional hint: markdown, json, yaml, toml, text. | |
file_name | string | For extension-based detection and provenance. | |
bank | string | ||
auto_link | boolean | Auto-link chunks to existing memories (default true). | |
generate_abstractions | boolean | Generate L1+ interpretations (default true). | |
max_chunk_size | integer | Max characters per L0 chunk (default 2000). | |
metadata | object | Metadata attached to all chunks. | |
source | string | Override the auto-derived source (e.g. a URL or DOI). | |
describe_images | boolean | Generate AI descriptions for images (default true; requires a vision-capable LLM). |
Output
{
"success": true,
"message": "…",
"data": {
"status": "success",
"session_id": "uuid",
"format": "markdown",
"detected_mime": "text/markdown",
"byte_size": 45678,
"l0_chunks": [
{ "id": "chunk-0", "memory_id": "uuid", "node_type": "text", "content_preview": "…", "char_count": 1980, "order": 0 }
],
"relations": [
{ "source_chunk_id": "chunk-0", "target_chunk_id": "chunk-1", "relation": "next_chunk", "strength": 1.0 }
],
"issues": [],
"warnings": [],
"format_hints_available": ["markdown", "text"]
}
}
document_status
Check the status of a document upload session.
Input
| Field | Type | Required | Description |
|---|---|---|---|
session_id | string | Specific session. If omitted, lists all sessions. | |
bank | string |
Output (specific session)
{
"success": true,
"data": {
"session_id": "uuid",
"file_name": "large.pdf",
"status": "processing",
"chunks_processed": 42,
"chunks_total": 100,
"started_at": "...",
"completed_at": null,
"error": null
}
}
cancel_document
Cancel an in-flight upload or processing session.
Input
| Field | Type | Required | Description |
|---|---|---|---|
session_id | string | ✓ | |
bank | string |
Output
{
"success": true,
"message": "Session 'uuid' cancelled",
"data": { "session_id": "uuid" }
}
Document storage
Documents themselves are not stored in the bank. Only the chunks (after parsing) and the session state are kept. To keep the original file, store it elsewhere or use the db export flow to capture both the bank and the session database.
Next
Introspection
Tools for inspecting the server’s health, configuration, and database integrity.
system_status
Tip
Call this first —
The recommended best practice is to call
system_statusat the start of every session. It returns backend info, model availability, and token usage stats so you know what you’re working with.
Input
None.
Output
{
"success": true,
"data": {
"backend": "local", // "local", "api", or mixed
"state": "ready", // "ready" / "loading" / "error"
"llm_model": "gemma-4-E2B-it-Q8_0.gguf",
"embedding_model": "all-MiniLM-L6-v2",
"llm_available": true,
"embedding_available": true,
"total_llm_calls": 42,
"total_embedding_calls": 128,
"total_prompt_tokens": 56789,
"total_completion_tokens": 12345,
"details": { /* backend-specific extra info */ }
}
}
details is opaque per backend. Local backend includes things like GPU layer count, model size, llama.cpp backend info. API backend includes things like last API call duration, error rate.
health_check
Run a full end-to-end check. Static checks (config validity, build features, API key presence, banks dir writability) are always free. Live checks (live: true) also issue a tiny embed("ping") and complete("...pong") against the backend to confirm it actually responds. Live checks consume a few API tokens per call.
Input
| Field | Type | Required | Description |
|---|---|---|---|
live | boolean | Default false. If true, run live backend probes. | |
embed_only | boolean | Live mode only: only probe the embedding endpoint. | |
llm_only | boolean | Live mode only: only probe the LLM endpoint. | |
embed_timeout_secs | integer | Default 15. | |
llm_timeout_secs | integer | Default 30. |
Output
{
"success": true,
"data": {
"healthy": true,
"backend": "local",
"llm_model": "...",
"embedding_model": "...",
"live_run": true,
"checks": [
{
"name": "config_valid",
"category": "config",
"status": "pass",
"detail": "All required fields present",
"duration_ms": 1
},
{
"name": "build_features_match",
"category": "build",
"status": "pass",
"detail": "local-llm and local-embed features enabled",
"duration_ms": 0
},
{
"name": "live_embed",
"category": "live",
"status": "pass",
"detail": "Embedded 'ping' to dim 384 in 234ms",
"duration_ms": 234
},
{
"name": "live_llm",
"category": "live",
"status": "pass",
"detail": "Completed '...pong' in 1.2s",
"duration_ms": 1203
}
]
}
}
healthy: false means at least one check failed. Inspect the checks array for which one and the detail field for the reason.
This tool is also available as a CLI subcommand: llm-mem health-check [--live].
check_consistency
Scan one or all banks for integrity issues:
- Orphaned abstractions (L1+ with no L0 source)
- Stale state (sessions stuck in
processingpast their timeout) - Missing embeddings
- Hash mismatches (content changed since embedding was generated)
- Unreferenced forgotten memories
- Duplicate content
- Invalid layer structure
The companion CLI command is llm-mem db check [--all] [--verbose].
Input
| Field | Type | Required | Description |
|---|---|---|---|
bank | string | Specific bank. If omitted, checks all. | |
verbose | boolean | Include detailed issue info. Default false. |
Output
{
"success": true,
"data": {
"issues": [
{
"type": "orphaned-abstraction",
"memory_id": "uuid",
"severity": "warn",
"detail": "L2 memory references missing L0 source"
}
],
"issue_count": 1,
"bank": "default"
}
}
To fix issues, use the CLI: llm-mem db fix [--dry-run] [--purge].
help
Returns the embedded usage guide that the MCP server exposes to AI clients. Includes best practices, the layered memory architecture explanation, memory type vocabulary, and tips.
Input
None.
Output
{
"success": true,
"data": {
"guide": {
"overview": "...",
"layered_memory_architecture": { /* L0 - L4+ */ },
"domain_patterns": { /* codebase, documents, web, conversations */ },
"banks_and_user_id": { /* when to use banks vs user_id */ },
"critical_guidelines": [ /* ... */ ],
"tips": [ /* ... */ ]
}
}
}
This is the same content an AI client sees when it calls the MCP prompts/list endpoint. The AI is expected to read this and adapt its tool-use strategy accordingly.
Debugging flow
system_status— is the server even ready?health_check(withlive: true) — is the backend actually responsive?check_consistency— is the database intact?- Check the log file for errors
- Run the same tool manually from the CLI to isolate MCP from application issues
Next
Standalone CLI
The llm-mem binary (no -mcp suffix) is a standalone command-line interface for working with memory banks directly, without an MCP client. It reads the same config file, talks to the same backend, and operates on the same banks as the MCP server.
llm-mem [OPTIONS] [COMMAND]
When to use it
- Local testing and debugging — try a query without spinning up an MCP client
- Scripts and automation — pipe search results into other tools
- Database administration — exports, merges, consistency checks, renames
- Configuration management — generate a template config, run health checks
- Interactive exploration — drop into the REPL for ad-hoc work
Modes
| Flag | Behavior |
|---|---|
| (no args) | Start the interactive REPL |
--single | Run a single command and exit. Useful for scripts. |
--batch <file> | Execute commands from a file (one per line). See Batch mode. |
--repl | Force REPL mode even if subcommand is present. |
--config <path> | Config file path (auto-discovered if not given). |
--banks-dir <path> | Override vector_store.banks_dir. |
All commands at a glance
Document handling
upload— ingest a filebegin-upload/upload-part/process-document— multi-part upload for large filesdoc-status/list-sessions— check upload progress
Browsing
list— list memories with filtersshow— full details of a memory by IDsearch— text or semantic searchstats/layer-stats/layer-tree— bank and pyramid statisticstimeline/timeline-graph/context-resume— chronological viewsdag-export— interactive DAG visualization (HTML)
Configuration
generate-config— write a config templatehealth-check— run static + optional live config checksystem-status— backend health
Database admin
db export/db import— backup and restoredb merge— merge banksdb check/db fix— integrity check and repairdb rename— atomic renamedb export-jsonl/db import(from JSONL) — backend-independent formatdb compact— merge fragments (also prunes old versions)db prune— reclaim disk space from old LanceDB versions
Misc
list-banks— list all bankslist-devices— show available embedding/LLM devicesexport— bank export to JSONmetrics— query and cache metricsclear-backoff— reset abstraction retry timersviz— real-time TUI for processing
REPL-only
Inside the REPL, you also get use <bank> (switch active bank) and savelog (dump log buffer). Output can be formatted as table, detail, json, jsonl, or csv.
Examples
# Interactive REPL
llm-mem
# Single-command scripting
llm-mem --single search --query "JWT auth" --mode semantic --limit 5
llm-mem --single list --bank research --limit 20 --format json
llm-mem --single stats --bank default
# Database admin
llm-mem db check --all
llm-mem db fix --bank research --purge
llm-mem db export --bank default --output backups/default.db --include-sessions
llm-mem db prune --all --older-than-days 0 # reclaim disk from old versions
# Visualize
llm-mem dag-export --bank default --output graph.html --max-nodes 300
# Health & config
llm-mem health-check --live
llm-mem generate-config --output /etc/llm-mem/config.toml
Output formats
Most commands accept --format with one of:
| Value | Best for |
|---|---|
table | Human reading (default for many commands) |
detail | Single-memory views (show) |
json | Programmatic consumption |
jsonl | Streaming, line-by-line |
csv | Spreadsheets / awk |
Next
Interactive REPL
Drop into a read-eval-print loop for ad-hoc work. The REPL has command history, tab completion, and the same commands as single-command mode.
Starting
llm-mem
# or
llm-mem --repl
The first run shows:
llm-mem interactive CLI
Type 'help' for available commands, 'exit' to quit
>
Available commands
Inside the REPL, the same commands as single-command mode work, without the -- prefix. A few extras:
| Command | What it does |
|---|---|
help | List all commands with brief descriptions. |
exit / quit / Ctrl+D | Leave the REPL. |
use <bank> | Switch the active bank. Subsequent commands operate on it until you switch again. |
savelog <path> | Dump the captured log buffer to a file. Useful for debugging. |
clear | Clear the screen. |
Example session
> use research
Switched to bank 'research'
> search "JWT auth" --mode semantic --limit 3
┌────┬──────────────────────────────────┬────────┬──────┐
│ id │ content │ layer │ score │
├────┼──────────────────────────────────┼────────┼──────┤
│ a1 │ AuthService handles JWT validation│ L0 │ 0.89 │
│ b2 │ Refresh tokens expire after 7 days│ L0 │ 0.82 │
└────┴──────────────────────────────────┴────────┴──────┘
> show a1
ID: a1
Layer: 0
Type: factual
Created: 2026-02-15 11:23:45
Event at: 2026-02-10
Content: AuthService handles JWT token validation and refresh — entry point is src/auth/service.rs:45-80
Context: project-x, auth
Topics: jwt, validation
Metadata: source=src/auth/service.rs:45-80
Relations:
- depends_on: jwt-secret-rotation
- derived_from: arch-decision-2026-02-10
> use default
Switched to bank 'default'
> stats
Bank: default
Memories: 1234
Size: 45.6 MB
Layers: L0: 1100, L1: 100, L2: 30, L3: 4
Banks: 2 (default, research)
> exit
Goodbye.
Tab completion
Type the start of a command and press Tab to complete. Tab also completes bank names and other enums.
History
The REPL uses rustyline, so arrow keys (up/down) navigate history and Ctrl+R does reverse search. History is persisted to ~/.llm-mem_history (a plain text file you can grep).
Log capture
While the REPL runs, the server emits logs to a buffer. You can dump them with:
> savelog debug.log
The buffer captures everything down to TRACE level, even if the visible stderr log is at INFO. Useful for diagnosing issues without restarting with RUST_LOG=trace.
Custom prompts
The default prompt is > . To customize, set the LLM_MEM_PROMPT env var before launching the REPL.
Next
- Single commands — the full command reference for scripting
- Database management
Single commands
Reference for every CLI subcommand. The same commands work inside the REPL (without the -- prefix).
The convention is llm-mem --single <command> [flags] or just llm-mem <command> [flags] from the shell.
upload
Ingest a file with auto-chunking and processing.
llm-mem upload --file-path /path/to/doc.md --bank project-x
llm-mem upload --file-path /path/to/code.rs --context src
| Flag | Type | Default | Description |
|---|---|---|---|
--file-path <PATH> | path | required | File to ingest. |
--bank <NAME> | string | default | Target bank. |
--process-immediately | bool | true | Start background processing now. |
--chunk-size <N> | integer | config | Override document_chunk_size. |
--context <TAG> | string[] | none | Repeatable. |
--format <FMT> | enum | table | table, json, jsonl, csv. |
begin-upload / upload-part / process-document
Three-step upload for very large files.
# Step 1: start the session
llm-mem begin-upload --file-name huge.pdf --total-size 104857600 --bank research
# Step 2: send parts (repeat for each part)
llm-mem upload-part --session-id <id> --part-index 0 --file-path part0.bin
llm-mem upload-part --session-id <id> --part-index 1 --file-path part1.bin
# ...
# Step 3: finalize
llm-mem process-document --session-id <id>
Use --partial-closure on process-document if you can’t send all expected parts (e.g. upload was interrupted and you want to proceed with what you have).
doc-status / list-sessions
Check upload progress.
llm-mem doc-status --session-id <id>
llm-mem list-sessions --bank research
list
Browse memories with filters.
llm-mem list --bank default --limit 20
llm-mem list --bank research --limit 50 --format jsonl > research.jsonl
| Flag | Type | Default | Description |
|---|---|---|---|
--bank <NAME> | string | default | |
--limit <N> | integer | 50 | |
--format <FMT> | enum | table |
show
Full details of a memory.
llm-mem show --bank default --memory-id <uuid>
llm-mem show --memory-id <uuid> --format detail
search
Text or semantic search.
llm-mem search --query "JWT auth" --mode semantic --limit 5
llm-mem search --query "TODO" --mode text --case-insensitive --show-scores
| Flag | Type | Default | Description |
|---|---|---|---|
--query <TEXT> | string | required | |
--mode <MODE> | enum | text | text (fast, no LLM) or semantic (embedding-based). |
--bank <NAME> | string | default | |
--limit <N> | integer | 10 | |
--case-insensitive | bool | false | Text mode only. |
--show-scores | bool | false | Text mode only. |
--threshold <F> | float | config | Override search_similarity_threshold. |
--format <FMT> | enum | table |
stats / layer-stats / layer-tree
Bank and pyramid statistics.
llm-mem stats --bank default
llm-mem layer-stats --bank default
llm-mem layer-tree --bank default --max-depth 5 --show-ids
layer-tree prints an ASCII tree of the pyramid.
timeline / timeline-graph / context-resume
Chronological views.
llm-mem timeline --bank default --since 2d --granularity day
llm-mem timeline --bank default --start 2026-01-01T00:00:00Z --end 2026-02-01T00:00:00Z --granularity week
llm-mem timeline-graph --bank default --since 1w --include-semantic-edges
llm-mem context-resume --bank default --since 30d --decay-factor 4.0 --segments 5
See Tools chronology for the conceptual reference; the CLI exposes the same parameters.
generate-config
Write a commented config template.
llm-mem generate-config --output /etc/llm-mem/config.toml
health-check
Validate the config and (optionally) probe the live backend.
llm-mem health-check # static checks only
llm-mem health-check --live # + tiny embed + completion
llm-mem health-check --live --embed-only
llm-mem health-check --live --llm-timeout-secs 60
This is the CLI equivalent of the health_check MCP tool. Use it standalone when debugging config issues before connecting an MCP client.
system-status
Backend health and token usage.
llm-mem system-status
llm-mem system-status --format json
list-banks
Show all memory banks.
llm-mem list-banks
llm-mem list-banks --format json
list-devices
Show available embedding/LLM devices (useful for GPU config).
llm-mem list-devices
export
Export a bank to JSON (different from db export — this is for application-level use).
llm-mem export --bank default --output default.json --pretty
metrics
Show accumulated query and cache metrics. Pass --reset to zero them after display.
llm-mem metrics
llm-mem metrics --reset
clear-backoff
Reset abstraction retry timers. Use after fixing a transient backend issue.
llm-mem clear-backoff --bank default
llm-mem clear-backoff --bank default --layer 1
viz
Launch a real-time TUI for visualizing document processing and abstraction pipeline activity.
llm-mem viz
dag-export
Export the memory bank as an interactive DAG visualization — a self-contained HTML file you can open in any browser. Nodes are memories colored by abstraction layer; edges represent abstraction, semantic, and temporal relations.
# Default: up to 200 most important nodes
llm-mem dag-export --bank default --output graph.html
# Customize node count and filtering
llm-mem dag-export --bank research --output research-graph.html --max-nodes 300 --min-importance 0.5
llm-mem dag-export --bank default --output l3-above.html --min-layer 3
# Disable certain edge types for a cleaner view
llm-mem dag-export --bank default --output no-temp.html --no-temporal
| Flag | Type | Default | Description |
|---|---|---|---|
--bank <NAME> | string | default | Memory bank to export. |
--output <PATH> | path | required | Output HTML file path. |
--max-nodes <N> | integer | 200 | Maximum nodes in the graph. |
--min-importance <F> | float | 0.0 | Minimum importance score (0.0–1.0). |
--semantic / --no-semantic | bool | true | Include semantic relation edges. |
--temporal / --no-temporal | bool | true | Include temporal (happened_after) edges. |
--abstraction / --no-abstraction | bool | true | Include abstraction pyramid edges. |
--min-layer <N> | integer | -1 | Minimum layer level to include. |
--max-layer <N> | integer | 99 | Maximum layer level to include. |
--min-relation-strength <F> | float | 0.0 | Minimum strength for semantic edges (0.0–1.0). |
Layered scaling
The graph automatically adjusts to the node count:
| Total bank size | Strategy |
|---|---|
| ≤100 memories | Full graph: all layers, all edges |
| 100–500 | L1+ pyramid + high-importance L0 + strong edges |
| 500–2,000 | L2+ concepts + abstraction hierarchy only |
| 2,000–10,000 | L3+L4 wisdom/concepts only |
| >10,000 | Top-N by importance, pyramid only |
Interactivity
Open the HTML file in a browser and use:
- Scroll/pinch to zoom, drag to pan
- Drag nodes to reposition them
- Hover over a node for a content preview tooltip
- Click a node to open the detail panel with full metadata and edge list
- Toggle layers and edge types in the sidebar
- Search bar to filter nodes by label, ID, or content
- Sliders to adjust link distance and charge strength
- Keyboard shortcuts:
r= reset view,f= fit to screen,/= focus search,Esc= close detail panel
Next
- Database management
- Tools overview — for the MCP equivalents
Database management
The llm-mem db subcommand handles bank-level operations: export, import, merge, check, fix, rename, compact, prune.
db export
Export a bank to a portable .db file. The exported bank is ready for continued use — it’s a self-contained copy.
llm-mem db export --bank default --output backups/default.db
llm-mem db export --bank research --output research-portable.db --include-sessions
| Flag | Type | Description |
|---|---|---|
--bank <NAME> | string | Bank to export. Default: default. |
--output <PATH> | path | Output file or directory. |
--include-sessions | bool | Also copy the .sessions.db alongside. |
db export-jsonl
Export a bank to a streaming JSONL text file. This format is backend-independent — a JSONL export from VectorLite can be imported into LanceDB and vice versa.
llm-mem db export-jsonl --bank default --output default.jsonl
llm-mem db export-jsonl --bank research --output research.jsonl --include-sessions
The JSONL file has:
- An envelope header with format version, app version, embedding dimension, and config snapshot
- One memory object per line in the body
- A checksum footer
This makes it the safest format for archival and cross-backend migration.
db import
Import memories from a JSONL file into a bank.
llm-mem db import --bank default --input default.jsonl
llm-mem db import --bank default --input default.jsonl --dry-run # preview only
llm-mem db import --bank default --input default.jsonl --strip-embeddings
| Flag | Type | Description |
|---|---|---|
--bank <NAME> | string | Target bank. |
--input <PATH> | path | The JSONL file. |
--strip-embeddings | bool | Don’t import the embedding vectors; re-embed on import. Use when the new model has a different dimension. |
--dry-run | bool | Show what would be imported without modifying anything. |
The importer auto-handles dimension mismatches: if the JSONL has dimension 384 but your current model is 768, it’ll strip the old vectors and mark memories for re-embedding.
db merge
Merge one or more source banks (or .db files) into a target bank.
llm-mem db merge --source research --source notes --into combined --on-duplicate keep-newest
llm-mem db merge --source old-backup.db --into default --on-duplicate keep-first
llm-mem db merge --source a.lance --source b.lance --into merged --dry-run
| Flag | Type | Default | Description |
|---|---|---|---|
--source <NAME_OR_PATH> | string[] | required | Repeatable. Bank names or .db file paths. |
--into <NAME> | string | required | Target bank name (created if missing). |
--on-duplicate <STRATEGY> | string | keep-newest | keep-newest, keep-first, or keep-all. |
--dry-run | bool | false | Show what would be merged. |
Duplicate detection uses content hash.
db check
Scan for consistency issues.
llm-mem db check --bank default
llm-mem db check --all
llm-mem db check --bank research --verbose
llm-mem db check --file external.db
| Flag | Type | Description |
|---|---|---|
--bank <NAME> | string | Specific bank. |
--file <PATH> | path | Check an external .db file. |
--all | bool | Check every bank. |
--verbose | bool | Detailed issue info. |
Detects:
- Orphaned abstractions (L1+ with no L0 source)
- Stale state (sessions stuck in
processingpast their timeout) - Missing embeddings
- Hash mismatches
- Unreferenced forgotten memories
- Duplicate content
- Invalid layer structure
db fix
Repair detected issues.
llm-mem db fix --bank default --dry-run # preview
llm-mem db fix --bank default # fix all
llm-mem db fix --bank default --fix orphaned-abstractions # fix specific types
llm-mem db fix --bank default --no-backup # skip auto-backup
llm-mem db fix --bank default --purge # hard-delete unreferenced Forgotten
| Flag | Type | Default | Description |
|---|---|---|---|
--bank <NAME> | string | default | |
--fix <TYPE> | string[] | (all) | Specific issue types. Valid: orphaned-abstractions, stale-states, missing-embeddings, hash-mismatches, unreferenced-forgotten, duplicate-content, invalid-layer-structure. |
--dry-run | bool | false | Show what would be fixed. |
--no-backup | bool | false | Skip automatic backup before fixing. |
--purge | bool | false | Hard-delete unreferenced Forgotten memories (vs. just marking them). |
Warning
db fixis destructive for unreferenced-forgotten memories (with--purge) and for invalid-layer-structure. Always run with--dry-runfirst. The default behavior (without--no-backup) is to take a backup automatically.
db rename
Atomic rename of a bank. Moves both the memory database and the session database.
llm-mem db rename --old-name foo --new-name bar
db compact
Merge a bank’s small LanceDB fragments into larger ones for faster reads, and prune superseded versions. Compaction runs automatically every few writes, but this command forces it on demand.
llm-mem db compact --bank default
llm-mem db compact --all
| Flag | Type | Default | Description |
|---|---|---|---|
--bank <NAME> | string | default | Specific bank. |
--all | bool | false | Compact every bank. |
Compaction also triggers a safe prune (versions older than 7 days), so it reclaims disk space from accumulated old snapshots.
db prune
Delete old LanceDB dataset versions from disk to reclaim space.
LanceDB is copy-on-write: every add, update, delete, and compaction creates a new version snapshot and leaves the old version’s files in place. Without pruning, these superseded versions accumulate forever, causing unbounded disk bloat. This command deletes versions older than the cutoff.
# Reclaim all reclaimable space from a single bank (safe default)
llm-mem db prune --bank old-data
# Prune every bank
llm-mem db prune --all
# Keep the last 7 days of versions
llm-mem db prune --bank default --older-than-days 7
# Aggressive: also remove unverified files newer than 7 days (see warning)
llm-mem db prune --all --delete-unverified
| Flag | Type | Default | Description |
|---|---|---|---|
--bank <NAME> | string | default | Specific bank. |
--all | bool | false | Prune every bank. |
--older-than-days <N> | integer | 0 | Keep versions newer than this many days. 0 removes everything eligible. |
--delete-unverified | bool | false | Also delete files newer than 7 days that can’t be verified as committed. |
Warning
--delete-unverifiedis only safe when no other process is reading or writing the dataset (e.g. the MCP server is stopped). Otherwise the dataset may be corrupted. Without this flag, LanceDB never deletes files newer than 7 days regardless of--older-than-days, so the default is always safe.
Tip
A safe one-off cleanup of bloated old databases:
llm-mem db prune --all --older-than-days 0
Common workflows
Migrate a bank to a new machine
# Source
llm-mem db export --bank default --output default.db --include-sessions
# scp default.db new-host:
# Destination
llm-mem db export --bank default --output /tmp/backup.db # safety backup
# (stop the server, place default.db in the banks_dir, restart)
Cross-backend migration (e.g. VectorLite → LanceDB)
# Export as backend-independent JSONL
llm-mem db export-jsonl --bank old --output migration.jsonl
# On the new system with LanceDB configured:
llm-mem db import --bank new --input migration.jsonl
Recover from corruption
llm-mem db check --all --verbose # identify
llm-mem db fix --bank default --dry-run # preview
llm-mem db fix --bank default # repair (auto-backup)
If db fix can’t repair, restore from a backup created via backup_bank (MCP) or db export.
Next
Data directory layout
Everything llm-mem writes to disk lives under a few well-known directories. Understanding the layout helps with backups, migrations, and debugging disk space issues.
Top-level layout
After first run, you’ll see something like this in the working directory (or wherever you pointed LLM_MEM_MODELS_DIR / vector_store.banks_dir):
./llm-mem-data/
├── banks/ # one .lance/ per memory bank
│ ├── default.lance/ # the default bank
│ ├── default.sessions.db # document upload sessions for default
│ ├── research.lance/
│ └── research.sessions.db
├── models/ # GGUF language model + ONNX embedding model
│ ├── gemma-4-E2B-it-Q8_0.gguf
│ └── (other model files)
└── logs/ # when logging.enabled = true
└── llm-mem-mcp.log
Plus the model cache (separate from llm-mem-data):
~/.cache/llm-mem/models/ # downloaded HF/fastembed files (when cache_model = true)
What’s in a .lance/ directory?
Each bank is a single LanceDB table. The directory is opaque to the user (it’s managed by the LanceDB library) but contains:
- A
memories.lance/subdirectory with the actual vector data _versions/for transaction history_latest.manifestand similar bookkeeping files
Don’t edit files inside a .lance/ directory manually. Use the db export / db import / db check / db fix tools to manipulate bank data safely.
What’s in .sessions.db?
A small SQLite file tracking document upload sessions:
session_idfile_name,file_size,mime_typestatus(uploading,processing,completed,failed,cancelled)- Per-part metadata:
part_index,md5sum,bytes_received - MD5 of the assembled file
- Bank name
The sessions database is what enables resumable uploads: if the server crashes mid-upload, the next startup re-attaches to the incomplete session and resumes from the last received part.
The default bank
The default bank is created automatically on first run, even if you never explicitly call create_memory_bank. To pre-create it manually:
mkdir -p /var/lib/llm-mem/banks
# (the server populates the directory on first write)
Moving the data directory
To relocate the entire data tree:
- Stop the server
- Move the directory:
mv ./llm-mem-data /new/path/llm-mem-data - Update the config or set env vars:
LLM_MEM_MODELS_DIR=/new/path/llm-mem-data/modelsvector_store.banks_dir=/new/path/llm-mem-data/banks
- Restart
Or just symlink:
ln -s /mnt/storage/llm-mem-data ./llm-mem-data
Disk space
Rough size estimates:
| Component | Per memory | Notes |
|---|---|---|
| Vector (384-dim f32) | ~1.5 KB | + small overhead |
| Metadata | ~1 KB | depends on usage |
| Sessions DB | small | a few MB total |
100k memories ≈ 250 MB. 1M memories ≈ 2.5 GB.
If you need to free space:
cleanup_resources(target:models) — reclaims the model cachecleanup_resources(target:banks) — deletes a specific bankdb export+ delete — manual cleanup with a backupdb prune --all --older-than-days 0— reclaim disk space from accumulated old LanceDB version snapshotsdb compact --bank default— merge fragments (also prunes); helps if there’s significant Forgotten memory accumulation
Permissions
The server process needs read/write access to:
models_dir(defaultllm-mem-data/models/)banks_dir(defaultllm-mem-data/banks/)log_directory(defaultllm-mem-data/logs/)- The cache directory (
~/.cache/llm-mem/models/, or whatevercache_diris set to)
When running as a subprocess of an MCP client, it inherits the client’s user. Make sure the working directory is writable.
Next
- Memory banks — using banks in practice
- Backups & restore
Memory banks
Banks are isolated memory stores. Each bank is a separate database file with its own memories, sessions, and statistics. Memories in one bank are invisible to another.
Why use multiple banks?
| Use case | Strategy |
|---|---|
| Per-project | One bank per codebase, switch via the bank parameter or --bank flag |
| Per-topic | One bank for cooking, another for finance, another for travel |
| Per-environment | One bank for dev, one for production data |
| Per-user | If you share a machine, separate banks per user |
| Per-purpose | Short-term context (auto-cleaned) vs. long-term knowledge (persistent) |
The default bank is always available — it’s created on first run. Use additional banks when you want hard isolation.
Naming
Bank names are free-form strings but follow these rules:
- Allowed: letters, digits,
-,_ - Avoid spaces (use
-or_) - The name
defaultis reserved - Case-sensitive
Good names: project-x, research-2026, meeting-notes, personal.
Creating banks
Banks are created automatically on first write to a new name. The server handles this transparently:
// MCP call to add_content_memory
{
"content": "Postgres over MongoDB for analytics",
"bank": "project-x" // <-- creates project-x if it doesn't exist
}
You can also pre-create with create_memory_bank (MCP) or mkdir + populate on disk.
Listing banks
# CLI
llm-mem list-banks
// MCP: list_memory_banks
{}
Output:
{
"banks": [
{ "name": "default", "memory_count": 1234, "size_bytes": 45678901 },
{ "name": "research", "memory_count": 567, "size_bytes": 12345678 }
],
"count": 2
}
Switching the active bank
MCP: pass bank: "<name>" on every call. There’s no global “active bank” concept at the protocol level.
CLI REPL: use <bank> switches the active bank for the session.
CLI single-command: --bank <name> per command.
Isolation guarantees
- Cross-bank searches are impossible (a tool can only see one bank at a time)
- Cross-bank relations don’t exist
- The abstraction pipeline operates per-bank (L0→L1 in bank A doesn’t see content from bank B)
- Backups and exports are per-bank
Soft grouping within a bank
For softer grouping without full isolation, use context tags and topics:
context— broad topic tags (e.g.["project-x", "auth", "jwt"])topics— specific subjects within the content
You can then filter by context/topic in queries:
// MCP: query_memory
{
"query": "JWT auth",
"context": ["project-x"]
}
Bank vs user_id
user_id is a per-memory filter within a bank. It’s optional and only useful when multiple users share a single bank.
In most cases:
- Single user per bank: omit
user_id - Multiple users per bank: set
user_idto differentiate
If two users share a bank and you don’t filter by user_id, each will see the other’s memories. Use a separate bank if you want hard isolation per user.
Lifecycle
| Operation | Tool | CLI |
|---|---|---|
| Create | create_memory_bank | (auto, or mkdir) |
| List | list_memory_banks | list-banks |
| Rename | rename_memory_bank | db rename |
| Backup | backup_bank | db export |
| Restore | restore_bank | db import |
| Delete | cleanup_resources (target: “banks”) | rm -rf the bank directory |
Next
Backups & restore
Banks are self-contained on disk. Backing up is just copying files; restoring is putting them back. The higher-level tools add versioning, checksums, and metadata.
Levels of backup
| Level | What it captures | When to use |
|---|---|---|
| Raw file copy | The .lance/ directory as-is | Fast snapshots, machine-level backups |
db export | A portable .db file with format metadata | Cross-version, cross-platform |
db export-jsonl | A streaming text file with envelope + checksum | Long-term archival, cross-backend migration |
backup_bank (MCP) | A versioned backup with SHA-256 + manifest | Programmatic, scripted |
For most users, db export-jsonl is the safest long-term choice because it’s backend-independent and human-readable.
Manual backup (file copy)
Stop the server, then:
# Backup everything
cp -a ./llm-mem-data /backup/llm-mem-data-$(date +%F)
# Or per-bank
cp -a ./llm-mem-data/banks/default.lance /backup/default-$(date +%F).lance
cp -a ./llm-mem-data/banks/default.sessions.db /backup/default-sessions-$(date +%F).db
Stop the server first — copying a LanceDB table mid-write can produce a corrupt backup. The .sessions.db SQLite file is durable, but a mid-transaction copy might lose the last few changes.
db export (CLI)
Creates a portable, versioned .db file:
llm-mem db export --bank default --output /backup/default.db
llm-mem db export --bank research --output /backup/research.db --include-sessions
The .db file includes:
- A format version
- A snapshot of the bank’s tables
- An optional
.sessions.db(with--include-sessions)
To restore:
# Stop the server
# Replace the bank directory with the contents of the .db
llm-mem db import --bank default --input /backup/default.db
# Restart
db import is the inverse — it can read .db files exported with db export and load them into a target bank.
db export-jsonl (CLI, recommended for archival)
Backend-independent streaming text format. The file looks like:
{"_format":"llm-mem-jsonl","_version":1,"_app_version":"0.1.0","_embedding_dim":384}
{"memory_id":"a1","content":"...","embedding":[...],"metadata":{...}}
{"memory_id":"b2","content":"...","embedding":[...],"metadata":{...}}
...
{"_checksum":"sha256:abc123...","_memory_count":1234}
To export:
llm-mem db export-jsonl --bank default --output /backup/default.jsonl
To import:
llm-mem db import --bank new --input /backup/default.jsonl
llm-mem db import --bank new --input /backup/default.jsonl --strip-embeddings # if dim changed
llm-mem db import --bank new --input /backup/default.jsonl --dry-run # preview
If the embedding dimension in the JSONL doesn’t match the current model, the importer strips the old vectors and re-embeds on import. This makes it safe to migrate to a new embedding model.
backup_bank (MCP tool)
For programmatic / AI-driven backups:
{
"name": "default",
"destination": "/home/you/llm-mem-backups"
}
Output:
{
"success": true,
"data": {
"backup_path": "/home/you/llm-mem-backups/default-2026-02-15-v3.db",
"manifest": {
"version": 3,
"created_at": "2026-02-15T...",
"memory_count": 1234,
"sha256": "...",
"size_bytes": 45678901
}
}
}
Versioning is automatic — the tool keeps incrementing v1, v2, v3, … in the destination directory.
To restore: use restore_bank with mode: "replace" (requires confirm: true) or mode: "merge".
Backup retention
The backup_bank tool doesn’t have a built-in retention policy. For long-running setups, prune old backups with a cron job:
# Keep last 30 days
find /home/you/llm-mem-backups -name "*.db" -mtime +30 -delete
For db export / db export-jsonl outputs, do the same — they’re regular files.
Testing your backups
A backup you haven’t tested is a backup you don’t have. Periodically:
- Restore a backup to a temp directory
- Start the server with
--banks-dir <temp dir> - Run
llm-mem health-check --live - Run a few queries to confirm memories are intact
- Delete the temp directory
Cross-backend migration
To move from VectorLite to LanceDB (or vice versa):
# Old system
llm-mem db export-jsonl --bank old-bank --output migration.jsonl
# New system (with new backend configured)
llm-mem db import --bank new-bank --input migration.jsonl
The JSONL format is the only one that works across backends. .db files are backend-specific.
Next
GPU acceleration
By default, llm-mem runs on CPU. This works everywhere but is slow for the language model. GPU acceleration is optional but strongly recommended if you have a compatible GPU.
Supported platforms
| Platform | GPU | Build feature |
|---|---|---|
| macOS (Apple Silicon) | M1, M2, M3, M4 | metal |
| macOS (Intel) | Intel iGPU | metal (limited) |
| Linux (AMD, Intel, NVIDIA) | Any Vulkan-capable | vulkan |
| Linux (NVIDIA only) | NVIDIA | cuda |
| Windows (AMD, Intel, NVIDIA) | Any Vulkan-capable | vulkan |
| Windows (NVIDIA only) | NVIDIA | cuda |
Vulkan is the most portable GPU backend. CUDA is fastest for NVIDIA hardware. Metal is the only option on macOS.
Building with GPU support
Default (CPU only)
cargo build --release
Vulkan
# Linux
cargo build --release --features local,vulkan
# Windows
cargo build --release --features local,vulkan
Requires the Vulkan SDK and a Vulkan-capable driver installed on the system at build time.
CUDA (NVIDIA only)
cargo build --release --features local,cuda
Requires CUDA toolkit + a compatible NVIDIA driver.
Metal (macOS)
cargo build --release --features local,metal
Enabling GPU at runtime
After building with the right feature, tell the runtime how many model layers to offload:
[llm]
gpu_layers = 20 # 0 = CPU only; set to 20+ for GPU
gpu_layers means how many transformer layers run on the GPU. The full model has a fixed number of layers (e.g. Gemma 4 E2B has ~26 layers). Setting gpu_layers = 20 offloads 20 of 26 layers; the rest stay on CPU.
| Value | Behavior |
|---|---|
0 | CPU only |
Small (e.g. 5) | Partial offload — useful if your VRAM is limited |
Number of model layers (e.g. 26 for Gemma 4 E2B) | Full offload — all on GPU |
| Larger than model layers | Capped at model layer count |
If you have less VRAM than the model needs, set gpu_layers to fit. The server will run the offloaded layers on GPU and the rest on CPU. There’s no all-or-nothing requirement.
Verifying GPU usage
In the log file (with logging.level = "debug"), look for:
INFO llama.cpp: loading model with 20 GPU layers
The system_status tool also reports backend details:
{
"backend": "local",
"details": {
"n_gpu_layers": 20,
/* ... */
}
}
For the CLI:
llm-mem list-devices # shows available devices
llm-mem system-status # shows current backend and layer count
Performance expectations
Speedups depend on hardware. Rough numbers for Gemma 4 E2B at 16K context:
| Hardware | Approx tokens/sec |
|---|---|
| Apple M1 CPU | 5-8 |
| Apple M1 GPU (Metal) | 20-35 |
| Apple M3 Pro GPU | 40-60 |
| NVIDIA RTX 3060 (CUDA) | 30-50 |
| NVIDIA RTX 4090 (CUDA) | 100-150 |
| AMD RX 7900 XT (Vulkan) | 50-80 |
| Intel i7-12700 CPU | 4-7 |
These are approximate — actual numbers depend on prompt length, model size, and driver version. The point is: GPU is 5-20x faster than CPU.
AppImage and GPU
The pre-built AppImage includes both Vulkan and CUDA backends, dynamically loaded. To use GPU:
- Install the Vulkan loader (
libvulkan1on Debian/Ubuntu,vulkan-toolson Fedora) OR the CUDA runtime - Set
gpu_layersin your config - Make sure
/dev/dri/is accessible (for Vulkan) ornvidia-smiworks (for CUDA)
If the AppImage fails to find the GPU at runtime, it’ll fall back to CPU automatically. Check the log file for “GPU not available” warnings.
Building the AppImage with GPU
scripts/build_appimage.sh --update-info 'gh-releases-zsync|vsndev3|llm-mem|latest|llm-mem-mcp-x86_64.AppImage.zsync'
The AppImage build uses the local feature by default. For GPU support in the AppImage, you need a build environment with Vulkan SDK or CUDA toolkit installed. See packaging/appimage/README.md for details.
Multiple GPUs
llm-mem doesn’t currently support splitting a model across multiple GPUs. It also doesn’t let you pin to a specific GPU — it uses the first one llama.cpp finds. For most setups this is fine.
Next
- Configuration —
gpu_layersand friends - Logging & debugging
Logging & debugging
llm-mem uses the tracing crate for structured logging. By default, events go to stderr (so they don’t pollute the MCP JSON-RPC stream on stdout). With logging.enabled = true, they also go to a rotating log file.
Log levels
| Level | What you get | When to use |
|---|---|---|
error | Only errors | Quiet production |
warn | Errors + warnings | Default verbosity |
info | Lifecycle events, request summaries | Normal operation |
debug | Per-request detail | Investigating an issue |
trace | Everything, including LLM prompts | Deep debugging |
Setting the level
In config
[logging]
enabled = true
level = "debug"
At runtime via env
The RUST_LOG env var overrides the config. Set it in the MCP config’s env block:
{
"mcp": {
"llm-mem": {
"type": "local",
"command": ["/path/to/llm-mem-mcp"],
"env": {
"RUST_LOG": "info,llama_cpp_2=warn,llm_mem::search=debug"
}
}
}
}
RUST_LOG accepts the tracing_subscriber filter syntax:
| Syntax | Meaning |
|---|---|
info | Global default = info |
llama_cpp_2=warn | Specific crate at a level |
llm_mem::search=debug | Specific module |
trace,hyper=info,h2=info | Global trace except quiet a few noisy crates |
Reading the log file
With logging.enabled = true, the log file is at <log_directory>/llm-mem-mcp.log. Rotation produces .1, .2, … up to max_files.
# Tail the live log
tail -f llm-mem-data/logs/llm-mem-mcp.log
# Look for errors
grep ERROR llm-mem-data/logs/llm-mem-mcp.log
# Inspect a specific request lifecycle
grep "memory_id=abc-123" llm-mem-data/logs/llm-mem-mcp.log
The format is:
2026-02-15T10:23:45.123Z INFO llm_mem::mcp: store_memory called memory_id=
2026-02-15T10:23:46.456Z INFO llm_mem::memory: stored memory memory_id=abc-123
What to look for
Server didn’t start
ERROR config: llm.context_size (8192) is too small for memory.document_chunk_size (2000) and llm.max_tokens (4096)
→ Increase llm.context_size in your config (or decrease the chunk size / max tokens).
API key missing
ERROR config: LLM API key is not configured. Set it in config.toml under [llm].api_key, or via env var LLM_MEM_LLM_API_KEY.
→ Set LLM_MEM_LLM_API_KEY (or OPENAI_API_KEY as a fallback).
Model download failed
ERROR model_downloader: failed to download gemma-4-E2B-it-Q8_0.gguf: HTTPS error: 407 Proxy Authentication Required
→ Set HTTPS_PROXY with auth, or download manually (see below).
LLM timeout
ERROR llm: completion timed out after 120s
→ Increase llm.llm_timeout_secs, switch to a smaller model, or enable GPU.
Manual model download
If you’re behind a corporate proxy, on an air-gapped machine, or just want to pre-fetch the models, download them manually:
mkdir -p llm-mem-data/models
cd llm-mem-data/models
# Language model (~2.5 GB)
curl -L -o gemma-4-E2B-it-Q8_0.gguf \
https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF/resolve/main/gemma-4-E2B-it-Q8_0.gguf
# Embedding model — let fastembed fetch this; if blocked, download from
# https://github.com/Anush008/fastembed-rs and place in the cache dir.
Then disable auto-download:
[llm]
auto_download = false
Note
Smaller alternative —
A smaller model
smollm2-1.7b-instruct-q4_k_m.gguf(~1 GB) is also available but requires a larger context size (setllm.context_size = 8192or more).
Performance debugging
To find slow operations, set RUST_LOG=info,llm_mem::metrics=trace and look for:
INFO llm_mem::memory: search returned 50 results in 234ms
INFO llm_mem::memory: store took 1.2s (embedding=200ms, llm_extract=900ms, write=100ms)
If search is slow on a large bank:
- Try
search_memoryinstead ofquery_memory(less work) - Increase
search_similarity_thresholdto filter earlier - Consider per-project banks to keep each one small
If the LLM is the bottleneck:
- Enable GPU
- Reduce
max_concurrent_requeststo 1 to avoid contention - Use a smaller/faster model
Inspecting banks
# CLI
llm-mem stats --bank default
llm-mem layer-stats --bank default
llm-mem db check --all --verbose
# MCP
{"tool": "system_status"}
{"tool": "check_consistency", "arguments": {"verbose": true}}
Capturing logs from the REPL
The CLI REPL captures logs in a separate buffer down to TRACE level. Dump them with the savelog command:
> savelog debug.log
Wrote 12345 lines to debug.log
This is useful for diagnosing issues without restarting the CLI with RUST_LOG=trace.
Next
- Configuration —
[logging]section - Environment variables —
RUST_LOGsyntax - Troubleshooting
Troubleshooting
Common issues, organized by symptom.
“Server failed to start”
Failed to load config from <path>
The config file exists but has invalid TOML or fails validation. Check the message — it tells you which field is wrong.
Common culprits:
- Typo in a key name (TOML doesn’t error on unknown keys by default; but serde does for known keys with
deny_unknown_fields) - A numeric field is a string (
max_memories = "10000"should be10000) - An enum value is wrong (
provider = "OPENAI"should beprovider = "api")
No API key configured
You’re using provider = "api" but didn’t set the key. Fix:
# In the MCP config's env block
"env": {
"LLM_MEM_LLM_API_KEY": "sk-..."
}
# Or in the TOML config
[llm]
api_key = "sk-..."
# Or as a generic fallback
"env": { "OPENAI_API_KEY": "sk-..." }
llm.context_size is too small
context_size must be larger than memory.document_chunk_size / 2 + llm.max_tokens + 512. The default 16644 works for the defaults, but if you’ve raised document_chunk_size or max_tokens, raise context_size too.
llm.provider = "local" but this build does not have the 'local-llm' feature enabled
You built with --no-default-features and then didn’t include local-llm (or the aggregate local). Rebuild:
cargo build --release --features local
“Models not downloading”
Failed to download: 407 Proxy Authentication Required
The proxy needs auth. Set HTTPS_PROXY=http://user:pass@proxy.host:port in the MCP env block, or in the [llm].proxy_url config (which overrides the env var).
Failed to download: 404 Not Found
The HuggingFace URL has changed or the model was renamed. Check the llm-mem releases for the latest model filenames, or download manually (see Logging & debugging).
Failed to download: connection timeout
Firewall or DNS. Verify you can reach huggingface.co from the server’s environment. If you can’t, download the model on a different machine and copy it in.
“MCP client can’t connect”
See Troubleshooting MCP for the full flow. The short version:
- Absolute path in the
commandfield (no~, no relative paths) - AppImage needs
chmod +x - Try
APPIMAGE_EXTRACT_AND_RUN=1if FUSE is unavailable - Restart the client after every config change
- Check the client’s MCP panel for the server status
“Tool calls fail with errors”
LLM timeout after 120 seconds
The LLM is taking too long. Mitigations:
[llm]
llm_timeout_secs = 300 # give it more time
gpu_layers = 20 # use the GPU
Or switch to a smaller model.
Bank not found: 'foo'
You passed bank: "foo" but no such bank exists. Either:
- Create it first with
create_memory_bank(MCP) or just write to it (auto-create) - Check the spelling — bank names are case-sensitive
Near-duplicate detected
The new memory’s embedding is very close to an existing one. By default, the server blocks the store and returns this error. Either:
- Pass
force: trueto store anyway - Edit the new content to be different from the existing memory
- Re-read what the existing memory says and decide if the new content is actually redundant
Contradiction detected
memory.contradiction_detection = true is enabled and the new memory contradicts an existing one. Same options as above.
Bank deletion requires user confirmation
You called cleanup_resources with target: "banks" but didn’t pass the exact confirmation phrase. This is intentional — pass:
{ "target": "banks", "name": "foo", "confirm": "I confirm this data will be permanently lost" }
“Performance is bad”
| Symptom | Likely cause | Fix |
|---|---|---|
| First query takes 30+ seconds | Model warmup | Wait. Subsequent queries are fast. |
| All queries take 5+ seconds | CPU-only inference on large model | Enable GPU (see GPU acceleration) |
| Bank has 500k+ memories, queries are slow | Vector search scales with bank size | Use smaller banks per project, increase search_similarity_threshold |
| API backend, every query is slow | Round-trip latency | Use a faster provider or local backend |
| High memory usage | Large model + large context | Reduce context_size or use a smaller model |
| High disk usage | Many banks, large embeddings | Run db prune --all --older-than-days 0 to reclaim old LanceDB versions; run db check to find Forgotten memories; clean up unused banks |
“Database corruption”
llm-mem db check --all --verbose
llm-mem db fix --bank default --dry-run # preview
llm-mem db fix --bank default # repair (auto-backup)
If db fix can’t repair, restore from backup:
llm-mem db export --bank default --output /tmp/before.db # safety
llm-mem db import --bank default --input /backup/latest.db
If even that fails, the bank is genuinely corrupted — delete the .lance/ directory and re-create (data loss; restore from your latest JSONL export if you have one).
“I get a different error message than the docs”
llm-mem is beta. Error messages are not stable. Open an issue at https://github.com/vsndev3/llm-mem/issues with:
- The exact error message
- The log file excerpt (
logging.level = "debug") - Your
config.toml(with secrets redacted) - The version (
llm-mem-mcp --version)
Next
Performance
A reference for tuning llm-mem for your workload.
First, measure
Don’t tune blind. Run:
llm-mem health-check --live
llm-mem system-status
llm-mem metrics
These give you the backend details and per-call timings. With RUST_LOG=debug,llm_mem::metrics=trace, you get per-request durations in the log.
Identify the bottleneck
Performance bottlenecks fall into a few categories:
1. LLM inference is slow
Symptoms: store operations take seconds; abstraction pipeline crawls; first query takes 30s.
Mitigations (try in order):
- Enable GPU — biggest single win, often 5-20x. See GPU acceleration.
- Use a smaller model —
gemma-4-E2Bis ~2.5 GB; if you don’t need that quality, trysmollm2-1.7b-instruct-q4_k_m(~1 GB). - Reduce
max_concurrent_requests— default is 1 (serial), which is safest. If you have GPU headroom, raise to 2-4 for parallelism. - Use API backend — if you have a fast API provider, an API call can be faster than local CPU inference.
2. Embedding is slow
Symptoms: searches and stores take a long time on the embedding step.
Mitigations:
- Local fastembed is fast by default — should be 1000+ texts/sec on CPU. If it’s slow, your CPU is loaded by other things.
- API embeddings — if you have API budget, use
provider = "api"for embeddings. - Increase
batch_size—embedding.batch_size = 256(default 64) is more efficient for large batch operations.
3. Vector search is slow
Symptoms: query_memory and search_memory are slow on a large bank.
Mitigations:
- Use smaller banks — split into per-project banks. Vector search is roughly linear in the bank’s size.
- Use
search_memoryinstead ofquery_memory— the former has fewer features and is faster. - Increase
search_similarity_threshold— filters out low-similarity candidates early. - Favor lower layers —
pyramid_config: { "mode": "bottom_heavy" }concentrates results on L0 and skips most higher-layer assembly. - Lower
max_search_results— fewer results to rank and return.
4. Abstraction pipeline is slow
Symptoms: L1+ memories take a long time to appear; background worker is the bottleneck.
Mitigations:
- Tune
auto_summary_threshold— the L0→L1 worker only fires past this. Higher = less frequent but cheaper per run. - Pause the pipeline during bulk operations —
stop_abstraction_pipeline→ bulk ingest →start_abstraction_pipeline. Avoids contention with the LLM. - Set
session_token_budget— caps the LLM tokens spent per session, preventing runaway costs. - Disable
auto_enhance— if you don’t need LLM-generated metadata on every store, setauto_enhance = falseto skip the LLM call.
5. Database operations are slow
Symptoms: db check, db fix, large exports take a long time.
Mitigations:
- Use
db export-jsonlfor archival — streaming write, doesn’t materialize the whole bank in memory. - For very large banks, use per-bank operations — don’t
db check --allon 50 banks at once. - Disk I/O — make sure the bank directory is on a fast SSD, not a network mount.
Configuration tuning guide
For a fast local setup
[llm]
provider = "local"
gpu_layers = 30 # all layers on GPU
max_concurrent_requests = 2 # parallel inference
[memory]
auto_enhance = true
auto_summary_threshold = 16384
session_token_budget = 100000
For a low-resource setup
[llm]
provider = "local"
gpu_layers = 0
cpu_threads = 4
max_concurrent_requests = 1
llm_timeout_secs = 180
[memory]
auto_enhance = false
deduplicate = true
auto_summary_threshold = 65536
For an API-only setup
[llm]
provider = "api"
model = "gpt-4o-mini"
max_concurrent_requests = 4
use_structured_output = true
llm_timeout_secs = 60
[embedding]
provider = "api"
model = "text-embedding-3-small"
batch_size = 100
[memory]
auto_enhance = true
For very large banks (>1M memories)
[memory]
raw_content_scan_limit = 1000
max_total_candidates = 5000
max_list_limit = 1000
And consider per-project banks instead of one giant bank.
When to scale
| Bank size | Recommendation |
|---|---|
| < 10k memories | Default config is fine |
| 10k-100k | Tune max_total_candidates and raw_content_scan_limit |
| 100k-1M | Consider per-project banks, LanceDB only (not VectorLite) |
| > 1M | Specialized tuning + faster hardware; consider a dedicated vector DB instead |