First run
What happens when you start llm-mem-mcp for the very first time.
Step 1: Config discovery
The server searches for a config file in this order:
--config <path>CLI flag (explicit)./config.toml(current working directory)~/.config/llm-mem/config.toml(XDG config dir)/etc/llm-mem/config.toml(system-wide)
If none of these exist, the server uses all defaults and applies any environment variable overrides (see Environment variables).
Tip
Generate a template —
To see every available setting with comments, run:
bash llm-mem-mcp –generate-config > config.toml
Edit the file, then start the server with `llm-mem-mcp --config config.toml`.
Step 2: Model download (local backend)
If you configured provider = "local" (the default), the server downloads:
- Language model:
gemma-4-E2B-it-Q8_0.gguf(~2.5 GB) from HuggingFace - Embedding model:
all-MiniLM-L6-v2(~90 MB) from fastembed’s model registry
Models are stored under the directory specified by llm.models_dir (default: llm-mem-data/models/) and cached separately under ~/.cache/llm-mem/models/ (configurable via llm.cache_dir).
Downloads are resumable — if interrupted, just restart the server. A smaller alternative model (smollm2-1.7b-instruct-q4_k_m.gguf, ~1 GB) is also available but requires a larger context size.
Note
Manual model download —
If you’re behind a corporate proxy or on an air-gapped machine, see Logging & debugging for instructions on downloading the models manually.
Step 3: Bank creation
The default memory bank is created automatically on first run. The bank lives at:
<llm.banks_dir>/default.lance/
<llm.banks_dir>/default.sessions.db
Default llm.banks_dir: ./llm-mem-data/banks/. See Data directory layout for the full schema.
Step 4: Service startup
Once models are loaded and the bank is ready, the service:
- Starts the abstraction pipeline (background L0→L1, L1→L2, … workers)
- Resumes any document upload sessions that were interrupted
- Listens on stdio for MCP requests
You’ll see log output on stderr (or in the log file if logging.enabled = true):
INFO Starting LLM Memory MCP Server
INFO Configuration loaded from: None
INFO Logging to directory: ./llm-mem-data/logs
INFO Initialized LLM client (backend: Local)
INFO Initialized memory bank manager
INFO Default memory bank loaded
INFO Abstraction pipeline initialization requested
INFO MCP server initialized successfully
Step 5: Connect a client
The server is now waiting on stdio for MCP traffic. Connect it to your AI client:
- opencode
- VS Code Copilot
- Troubleshooting MCP — if connection fails
Stopping the server
The server handles SIGINT (Ctrl+C) and SIGTERM gracefully:
- Stops the MCP listener
- Stops the abstraction pipeline
- Flushes any in-flight writes
- Releases the llama.cpp backend
No special shutdown command is required.
Next
Quickstart — run your first query in 5 minutes.