Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Installation

There are five ways to install llm-mem, listed from least to most effort:

  1. Pre-built AppImage (Linux) — single file, no install, no compiler
  2. Pre-built native binary (Linux/macOS/Windows) — download from Releases
  3. cargo install — Rust toolchain required
  4. Build from source — for contributors / custom builds
  5. Docker build — for the AppImage itself (advanced)

For most users, option 1 or 2 is the right choice.

A single-file AppImage is available that works on every modern Linux distribution without glibc version issues. It bundles both binaries (llm-mem-mcp and llm-mem) in one file, dispatching by argv[0] (multicall pattern). Includes LanceDB, the Vulkan loader, and a bundled OpenSSL.

# 1. Download from the GitHub Releases page
#    https://github.com/vsndev3/llm-mem/releases
#    pick: llm-mem-mcp-x86_64.AppImage   (or -aarch64)

# 2. Make it executable
chmod +x llm-mem-mcp-x86_64.AppImage

# 3. Run the MCP server (default when invoked as llm-mem-mcp)
./llm-mem-mcp-x86_64.AppImage --help

# 4. (Optional) Use the CLI via a symlink
ln -s "$(pwd)/llm-mem-mcp-x86_64.AppImage" ~/bin/llm-mem
llm-mem --help
llm-mem --single search --query "vegan recipes"

The first run downloads the AI models it needs:

  • A language model: Gemma 4 E2B (gemma-4-E2B-it-Q8_0.gguf, ~2.5 GB)
  • An embedding model: all-MiniLM-L6-v2 (~90 MB)

Downloads resume if interrupted. HTTP/HTTPS/SOCKS proxies are respected via HTTPS_PROXY / HTTP_PROXY / ALL_PROXY environment variables.

If FUSE isn’t available on your system (e.g. some container environments), set:

export APPIMAGE_EXTRACT_AND_RUN=1

Note

The AppImage downloads a separate copy of the models per invocation. To share models across runs, set LLM_MEM_MODELS_DIR to a stable path.

Option 2: Pre-built native binary (Linux, macOS, Windows)

Native binaries are built for the host platform’s typical glibc/libc version. They don’t bundle a C++ runtime or OpenSSL, so they rely on system libraries.

ArchivePlatformArchitectureGPU
llm-mem-mcp-linux-x86_64.tar.gzLinux (Ubuntu 24.04+)x86_64Vulkan
llm-mem-mcp-linux-aarch64.tar.gzLinux ARM64aarch64Vulkan
llm-mem-mcp-macos-x86_64.tar.gzmacOS 15+ (Intel)x86_64Metal
llm-mem-mcp-macos-aarch64.tar.gzmacOS 15+ (Apple Silicon)arm64Metal
llm-mem-mcp-windows-x86_64.zipWindows 10/11x86_64Vulkan

Install:

# Linux / macOS
tar xzf llm-mem-mcp-linux-x86_64.tar.gz
sudo mv llm-mem-mcp /usr/local/bin/

# Windows (PowerShell)
Expand-Archive llm-mem-mcp-windows-x86_64.zip
move llm-mem-mcp.exe C:\Windows\System32\

On first run, the same auto-download kicks in. On Windows, you may need to install Microsoft Visual C++ Redistributable for the local LLM engine.

Option 3: cargo install

If you have a Rust toolchain installed and want the latest from crates.io (when published):

cargo install llm-mem --features local

This requires a C/C++ compiler and CMake on the system. Build time is ~10-20 minutes the first time (compile llama.cpp + fastembed native code).

Option 4: Build from source

For contributors, custom builds, or to get unreleased features:

git clone https://github.com/vsndev3/llm-mem.git
cd llm-mem

# Default build: local LLM (llama.cpp) + local embeddings (fastembed) + LanceDB
cargo build --release

# Artifacts land in target/release/:
#   llm-mem-mcp  — MCP server
#   llm-mem     — standalone CLI

Build feature combinations

CommandWhat you get
cargo build --releaseDefault: local LLM + local embeddings + LanceDB (fully offline)
cargo build --release --no-default-features --features lancedbAPI-only, no local AI
cargo build --release --no-default-features --features lancedb,local-embedAPI LLM + local embeddings
cargo build --release --features local,vulkanLocal + Vulkan GPU
cargo build --release --features local,metalLocal + Metal GPU (macOS)
cargo build --release --features local,cudaLocal + CUDA GPU (NVIDIA)
cargo build --release --no-default-features --features lancedb,vector-liteAPI + VectorLite (legacy)

See GPU acceleration for details on the GPU backends.

Option 5: Build the AppImage yourself

If you want to produce an AppImage for a custom feature combination or to test changes:

git clone https://github.com/vsndev3/llm-mem.git
cd llm-mem

# Local Docker build (auto-detects docker or podman)
scripts/build_appimage.sh

# Native build (no Docker) — requires gcc 10+, cmake, appimagetool
scripts/build_appimage.sh --no-docker

# Build for ARM64 from an x86_64 host
scripts/build_appimage.sh --arch aarch64

# Build with zsync auto-update metadata
scripts/build_appimage.sh --update-info 'gh-releases-zsync|vsndev3|llm-mem|latest|llm-mem-mcp-x86_64.AppImage.zsync'

Output: dist/llm-mem-mcp-x86_64.AppImage (or -aarch64) plus a .zsync delta-update file.

The CI workflow .github/workflows/appimage.yml is manual-trigger only — it produces x86_64 and aarch64 artifacts in the Actions UI without auto-publishing a release.

Verifying the install

# MCP server
llm-mem-mcp --version
llm-mem-mcp --help

# CLI
llm-mem --help

Both binaries should print their version and exit. If the binary is missing or won’t run, see Troubleshooting.

Next

  • First run — what happens on the very first invocation
  • Quickstart — get from install to working memory in 10 minutes