If you find memU useful or interesting, a GitHub Star ⭐️ would be greatly appreciated.
---
## ✨ Core Features
| Capability | Description |
|------------|-------------|
| 🗂️ **Multimodal Ingestion** | Write conversations, documents, images, video, audio, URLs, logs, and local files into memory |
| 📁 **Compiled Memory Workspace** | Persist the Index, Skill, and Memory layers — folders (categories), files (items), source artifacts, links, summaries, and embeddings |
| 🧠 **Typed Memory Extraction** | Extract profile, event, knowledge, behavior, skill, and tool memories from raw sources |
| 🛠️ **Self-Evolving Skills** | Auto-extract reusable tool patterns and workflows from agent traces, then merge and refine them on every workspace sync instead of relearning |
| 🧭 **Self-Organizing Folders** | Auto-build categories, links, summaries, and embeddings without manual tagging |
| 🤖 **Agent-Ready Retrieval** | LLM-free `retrieve_workspace()` ranks memory segments, files, and source resources directly |
| 🔄 **Incremental Workspace Sync** | `memorize_workspace()` diffs a folder against a manifest — only changed files are (re)processed, deletions cascade |
| 🧱 **Pluggable Storage** | Use in-memory, SQLite, or Postgres backends with the same repository contracts |
| 🔀 **Profile-Based LLM Routing** | Route chat, embedding, vision, and transcription work through configurable LLM profiles |
| ⌨️ **CLI** | `memu` command (pip) and `npx memu-cli` (npm) — memorize and retrieve from the terminal or CI |
---
## 🎯 Use Cases
Every use case is the same loop: drop sources into a folder, sync it with `memorize_workspace()`, then ask with `retrieve_workspace()`. The sync is incremental (only changed files are reprocessed), and the top-level directory decides the treatment — `chat/` → memory topics, `agent/` → skills, everything else → indexed context.
### 1. **Personal Memory**
*Turn chat logs into user preferences, goals, events, decisions, and relationship context.*
```python
# workspace/chat/*.json — conversation logs become memory topic files
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("What should I remember about this user?")
```
### 2. **Workspace Context for Coding Agents**
*Convert docs, PR notes, logs, and design decisions into reusable project memory.*
```python
# docs, notes, and logs anywhere in the folder are captioned and indexed
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("How should I structure this module?")
```
### 3. **Multimodal Knowledge Layer**
*Extract searchable facts from documents, screenshots, images, videos, and audio notes.*
```python
# modality is inferred per file: .pdf/.docx/.pptx/.xlsx/.html (via MarkItDown —
# pip install 'memu-py[document]'), .png/.jpg, .mp3/.wav, .mp4/.mov, ...
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("What matters for the next research plan?")
```
### 4. **Tool and Agent Learning**
*Turn execution traces into skills that tell future agents what worked and what to avoid.*
```python
# workspace/agent/*.txt — execution traces are distilled into skill files
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("Which tools worked for config editing?")
```
---
## 🗂️ Architecture
The compiled workspace is easiest to read as two directions:
- `memorize_workspace()` writes a folder into durable memory files, skill files, resource records, segments, links, and embeddings.
- `retrieve_workspace()` reads those layers directly, ranking segments first and rolling results up to the files and resources an agent should load.
Memory is stored in three representation layers:
| Layer | What it holds | Retrieval Role |
|-------|---------------|----------------|
| **File** (`RecallFile`) | A synthesized memory topic or skill document | The unit returned to the agent — hit segments roll up to their file |
| **Segment** | Fine slices of a file (paragraph lines, skill descriptions) | The embedded search unit — queries rank segments first |
| **Resource** | The raw source artifact with its caption | Recall original context when synthesized summaries are not enough |
`retrieve_workspace()` embeds the query once, ranks segments and resources by similarity, and returns compact context with zero chat-LLM calls.
See [docs/architecture.md](docs/architecture.md) for the runtime view of `MemoryService`, workflow pipelines, storage backends, and LLM routing, and [docs/adr/](docs/adr/README.md) for the decision records behind the layered design.
---
## 🧰 Agent Skills
The repo ships one [Agent Skill](https://docs.claude.com/en/docs/agents-and-tools/agent-skills) — [`.claude/skills/memu/SKILL.md`](.claude/skills/memu/SKILL.md) — that gives Claude Code (and any skills-compatible agent) the workspace pair. The agent decides when to use each direction:
- **memorize** (`memu memorize-workspace`) — "remember this", "sync this folder into memory", finishing work worth persisting
- **retrieve** (`memu retrieve-workspace`) — "what do we know about…", starting a task with likely prior context
It works out of the box inside this repo. To use it in your own project, copy the skill folder into that project's `.claude/skills/` (or `~/.claude/skills/` to enable it everywhere):
```bash
cp -r .claude/skills/memu /path/to/your-project/.claude/skills/
```
The skill locates the CLI automatically (`memu`, `uvx --from memu-py memu`, or `npx memu-cli`) and keeps state in the project-local `./data/memu.sqlite3`, so what one session memorizes the next can retrieve. For LangGraph agents, see the [LangGraph integration](docs/langgraph_integration.md) instead.
---
## 🚀 Quick Start
### Option 1: Cloud Version
👉 **[memu.so](https://memu.so)** — Hosted API for managed ingestion, structured memory, and retrieval
For enterprise deployment: **info@nevamind.ai**
#### Cloud API (v3)
| Base URL | `https://api.memu.so` |
|----------|----------------------|
| Auth | `Authorization: Bearer 
```python
result = await service.memorize_workspace(
folder="./workspace", # scanned recursively; modality inferred per file
user={"user_id": "123"}, # optional scope
)
# Returns the diff plus what changed:
# { "added": [...], "modified": [...], "deleted": [...],
# "resources": [...], "entries": [...], "files": [...] }
```
- Diffs the folder against a sidecar `.memu_manifest.json` — only added/modified files are processed, memory from deleted files is cascade-removed
- Routes by top-level directory: `chat/` → memory files, `agent/` → skill files, everything else → indexed workspace context
- Rebuilds the markdown memory tree (`INDEX.md` / `MEMORY.md` / `SKILL.md`) when `memory_files_config.enabled=True`
---
### `retrieve_workspace()` — Fast, LLM-Free Retrieval
```python
result = await service.retrieve_workspace(
"deploy checklist",
where={"user_id": "123"},
)
# Returns:
# { "segments": [...], # embedded slices ranked by similarity
# "files": [...], # the memory/skill files those segments roll up to
# "resources": [...] } # workspace resources ranked by similarity
```
The query is embedded once and ranked by vector similarity — no intention routing, no query rewriting, no sufficiency checks, zero LLM calls. Use it for high-frequency lookups where latency and cost matter more than deep reasoning.
---
## 💡 Example Workflows
### Always-Learning Assistant
```bash
export OPENAI_API_KEY=your_key
uv run python examples/example_1_conversation_memory.py
```
Automatically extracts preferences, builds relationship models, and surfaces relevant context in future conversations.
### Self-Improving Agent
```bash
uv run python examples/example_2_skill_extraction.py
```
Monitors agent actions, identifies patterns in successes and failures, auto-generates skill guides from experience.
### Multimodal Context Builder
```bash
uv run python examples/example_3_multimodal_memory.py
```
Cross-references text, images, and documents automatically into a unified memory layer.
---
## 📊 Performance
memU achieves **92.09% average accuracy** on the Locomo benchmark across all reasoning tasks.
