wehub/nevamind-ai--memu
1
0
Fork 0
Code Issues Pull Requests Actions 2 Deployments Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
build / build (3.13) (push) Waiting to run
Details
release-please / release-please (push) Waiting to run
Details
release-please / build wheels (macos-aarch64) (push) Blocked by required conditions
Details
release-please / build wheels (macos-x86_64) (push) Blocked by required conditions
Details
release-please / build wheels (windows-x86_64) (push) Blocked by required conditions
Details
release-please / build wheels (linux-aarch64) (push) Blocked by required conditions
Details
release-please / build wheels (linux-x86_64) (push) Blocked by required conditions
Details
release-please / build sdist (push) Blocked by required conditions
Details
release-please / publish release artifacts (push) Blocked by required conditions
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:36:10 +08:00
commit 75c67150d0
321 changed files with 42034 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/memu/SKILL.md
+46
View File
@@ -0,0 +1,46 @@
---
name: memu
description: Give this agent persistent memory via memU. Use to memorize (save files/folders so context survives this session) when the user asks to remember or sync something, or after completing work worth keeping; and to retrieve (search memory from earlier sessions) when the user asks what is known about a person/project/topic or references context not in this conversation.
---
# memU: memorize and retrieve persistent memory
memU compiles sources into a persistent local store (`./data/memu.sqlite3` + a browsable `./data/memory/` markdown tree, relative to CWD). What one session memorizes, the next can retrieve — always run from the project root so every command hits the same store.
Use only the two commands below. Do not use the legacy `memu memorize` / `memu retrieve` commands.
Both directions need an API key: make sure `OPENAI_API_KEY` (or `MEMU_LLM_PROVIDER` + its matching key) is set, and tell the user if it is missing.
## Locate the CLI
Use the first available runner:
1. `memu` (installed via `pip install memu-py`)
2. `uvx --from memu-py memu`
3. `npx memu-cli`
## Memorize
```bash
memu memorize-workspace <folder>
```
- Incremental and safe to re-run: diffs against `<folder>/.memu_manifest.json`, so only added/modified files are processed and memory from deleted files is removed.
- Top-level directory decides the treatment: `chat/` → memory topics, `agent/` → skills, everything else → indexed workspace context (modality inferred per file from its extension).
- To memorize a single file, place (or copy) it into the workspace folder and sync — there is no separate single-file path.
Report the printed diff (added/modified/deleted) to the user; pass `--json` if you need to parse the result.
## Retrieve
```bash
memu retrieve-workspace "<query>"
```
Single-shot embedding search, no LLM calls. Returns JSON in three layers:
- `segments` — the matched slices, ranked by similarity (check `score`)
- `files` — the memory/skill documents they belong to (usually what you want)
- `resources` — matching raw sources, when summaries are not enough
If a query misses, retry with different phrasing or more specific terms. Low scores across the board usually mean nothing relevant is stored — say so rather than stretching weak matches. You can also read `./data/memory/MEMORY.md` / `SKILL.md` / `INDEX.md` directly for a browsable overview.
.github/ISSUE_TEMPLATE/bug_report.yml
+75
View File
@@ -0,0 +1,75 @@
name: Bug Report
description: Report a bug or issue with memU
title: "[BUG] "
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to report a bug in memU! Please fill in the following details.
- type: textarea
id: description
attributes:
label: Description
description: A clear and detailed description of the bug.
placeholder: "Enter a clear and concise description of what the bug is."
validations:
required: true
- type: input
id: environment
attributes:
label: Environment
description: The environment where this bug occurred (e.g., operating system, CPU arch, etc.).
placeholder: "Enter details about the environment."
validations:
required: true
- type: textarea
id: steps
attributes:
label: Steps to reproduce
description: What are the steps to reproduce this issue?
placeholder: "Enter step-by-step instructions to reproduce the issue."
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
description: What should have happened instead?
placeholder: "Describe what you expected to happen."
validations:
required: true
- type: input
id: version
attributes:
label: Version
description: What version of memU are you using?
placeholder: e.g., v1.0.0
validations:
required: true
- type: dropdown
id: severity
attributes:
label: Severity
description: How severe is the bug?
options:
- Critical
- Major
- Minor
validations:
required: true
- type: textarea
id: additional_info
attributes:
label: Additional Information
description: Any other context or screenshots related to the bug.
placeholder: "Enter additional context or information."
validations:
required: false
.github/ISSUE_TEMPLATE/config.yml
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: GitHub Discussions
url: https://github.com/NevaMind-AI/memU/discussions
about: For questions and general discussions about memU
- name: Documentation
url: https://github.com/NevaMind-AI/memU/blob/main/README.md
about: Check out the memU documentation
.github/ISSUE_TEMPLATE/feature_request.yml
+61
View File
@@ -0,0 +1,61 @@
name: Feature Request
description: Request a feature for memU
title: "[FEATURE] "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to request a feature for memU! Please fill in the following details.
- type: textarea
id: description
attributes:
label: Description
description: A clear and detailed description of the feature request.
placeholder: "Enter a clear and concise description of what the feature request is."
validations:
required: true
- type: textarea
id: motivation
attributes:
label: Motivation
description: Why do we need this feature? What problem does it solve?
placeholder: "Explain the use case and why this feature would be valuable."
validations:
required: true
- type: dropdown
id: platform
attributes:
label: Platform
description: Which platform(s) does this feature apply to?
options:
- Platform Independent
- Linux
- macOS
- Windows
validations:
required: true
- type: dropdown
id: priority
attributes:
label: Priority
description: How important is this feature?
options:
- Critical
- Major
- Minor
validations:
required: true
- type: textarea
id: additional_info
attributes:
label: Additional Information
description: Any other context or screenshots related to the feature request.
placeholder: "Enter additional context or information."
validations:
required: false
.github/ISSUE_TEMPLATE/hackathon_task.yml
+65
View File
@@ -0,0 +1,65 @@
name: Hackathon Task
description: The official designated task for the memU track in the 2026 New Year Challenge
title: "[2026NewYearChallenge] "
labels: ["hackathon-task"]
assignees: []
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to join the challenge, and best of luck!
- type: dropdown
id: label_hint
attributes:
label: Label
description: Pick the most appropriate label for this task.
options:
- hackathon-task
validations:
required: true
- type: input
id: assignees_hint
attributes:
label: Assignees
description: GitHub usernames to assign (comma-separated). If empty, maintainers will triage.
placeholder: "e.g., alice,bob"
validations:
required: false
- type: textarea
id: description
attributes:
label: Description*
description: What will this task implement?
placeholder: "Enter a clear and concise description of the task you want to implement."
validations:
required: true
- type: textarea
id: requirements
attributes:
label: Requirements (optional)
description: What will your planned pull request include? (APIs, modules, tests, docs, etc.)
placeholder: |
- [ ] Implementation details
- [ ] Tests
- [ ] Docs / examples
- [ ] Compatibility notes
validations:
required: false
- type: textarea
id: review_criteria
attributes:
label: Review Criteria (optional)
description: How will this task be evaluated?
placeholder: |
Example:
- Correctness (passes tests, no regressions)
- Quality (readability, modularity)
- DX (good docs / examples)
- Impact (useful for memU users)
validations:
required: false
.github/ISSUE_TEMPLATE/improvement_suggestion.yml
+63
View File
@@ -0,0 +1,63 @@
name: Improvement Suggestion
description: Suggest an improvement for memU
title: "[IMPROVEMENT] "
labels: ["enhancement", "improvement"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to suggest an improvement for memU! Please fill in the following details.
- type: textarea
id: description
attributes:
label: Description
description: A clear and detailed description of the improvement suggestion.
placeholder: "Enter a clear and concise description of what you'd like to improve."
validations:
required: true
- type: textarea
id: current_behavior
attributes:
label: Current Behavior
description: How does it work currently?
placeholder: "Describe the current behavior or implementation."
validations:
required: false
- type: textarea
id: proposed_improvement
attributes:
label: Proposed Improvement
description: How would you like it to work?
placeholder: "Describe your proposed improvement."
validations:
required: true
- type: textarea
id: benefits
attributes:
label: Benefits
description: What are the benefits of this improvement?
placeholder: "Explain why this improvement would be valuable."
validations:
required: false
- type: input
id: version
attributes:
label: Version
description: What version of memU are you using?
placeholder: e.g., v1.0.0
validations:
required: false
- type: textarea
id: additional_info
attributes:
label: Additional Information
description: Any other context or screenshots related to the improvement.
placeholder: "Enter additional context or information."
validations:
required: false
.github/PULL_REQUEST_TEMPLATE.md
+44
View File
@@ -0,0 +1,44 @@
## 📝 Pull Request Summary
Please provide a short summary explaining the purpose of this PR.
---
## ✅ What does this PR do?
- Clearly describe the change introduced.
- Mention the motivation or problem it solves.
---
## 🤔 Why is this change needed?
- Explain the context or user impact.
- Link any relevant issue or discussion.
---
## 🔍 Type of Change
Please check what applies:
- [ ] Bug fix
- [ ] New feature
- [ ] Documentation update
- [ ] Refactor / cleanup
- [ ] Other (please explain)
---
## ✅ PR Quality Checklist
- [ ] PR title follows an allowed format (for example `feat:`, `fix:`, `docs:`, `memory base:`)
- [ ] Changes are limited in scope and easy to review
- [ ] Documentation updated where applicable
- [ ] No breaking changes (or clearly documented)
- [ ] Related issues or discussions linked
---
## 📌 Optional
- [ ] Screenshots or examples added (if applicable)
- [ ] Edge cases considered
- [ ] Follow-up tasks mentioned
.github/workflows/build.yml
+35
View File
@@ -0,0 +1,35 @@
name: build
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
strategy:
matrix:
python-version: ["3.13"]
steps:
- uses: actions/checkout@v6
- name: Install uv & set Python
uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-python: true
- name: Ensure Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}
- name: Sync dependencies
run: |
uv sync --frozen --extra document
uv run pre-commit install
- name: Run style checks
run: uv run make check
- name: Run tests
run: uv run make test
.github/workflows/pr-title.yml
+47
View File
@@ -0,0 +1,47 @@
name: pr-title
on:
pull_request_target:
types:
- opened
- edited
- reopened
- synchronize
permissions:
contents: read
statuses: write
jobs:
validate:
name: Validate PR title
runs-on: ubuntu-latest
steps:
- uses: amannn/action-semantic-pull-request@v6
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
headerPattern: '^([\w ]*)(?:\(([\w$.\-*/ ]*)\))?: (.*)$'
headerPatternCorrespondence: type, scope, subject
types: |
build
chore
ci
docs
feat
fix
memory base
perf
refactor
revert
style
test
- name: Disallow trailing ellipsis
shell: bash
env:
PR_TITLE: ${{ github.event.pull_request.title }}
run: |
if [[ "$PR_TITLE" == *... ]]; then
echo "::error title=Invalid PR title::PR title must not end with an ellipsis (...)"
exit 1
fi
.github/workflows/release-please.yml
+160
View File
@@ -0,0 +1,160 @@
on:
push:
branches:
- main
permissions:
contents: write
issues: write
pull-requests: write
name: release-please
jobs:
release-please:
runs-on: ubuntu-latest
outputs:
releases_created: ${{ steps.release.outputs.releases_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
steps:
- uses: googleapis/release-please-action@v4
id: release
with:
release-type: python
target-branch: main
build-wheels:
name: build wheels (${{ matrix.label }})
runs-on: ${{ matrix.os }}
needs: release-please
if: ${{ needs.release-please.outputs.releases_created == 'true' && needs.release-please.outputs.tag_name != '' }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
label: linux-x86_64
target: x86_64-unknown-linux-gnu
extra-args: "--compatibility manylinux_2_39"
python-version: "3.13"
- os: ubuntu-latest
label: linux-aarch64
target: aarch64-unknown-linux-gnu
extra-args: "--compatibility manylinux_2_39"
python-version: "3.13"
- os: macos-15-intel
label: macos-x86_64
target: ""
extra-args: ""
python-version: "3.13"
- os: macos-latest
label: macos-aarch64
target: ""
extra-args: ""
python-version: "3.13"
- os: windows-latest
label: windows-x86_64
target: ""
extra-args: ""
python-version: "3.13"
steps:
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python-version }}
- name: Install maturin
run: uv tool install maturin
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- name: Setup cross-compilation for Linux ARM64
if: matrix.label == 'linux-aarch64'
run: |
sudo apt-get update
sudo apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
- name: Build wheel
run: uvx maturin build --release --out dist ${{ matrix.extra-args }} ${{ matrix.target && format('--target {0}', matrix.target) || '' }}
env:
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
- name: Upload wheel artifact
uses: actions/upload-artifact@v6
with:
name: wheels-${{ matrix.label }}
path: dist/*.whl
if-no-files-found: error
build-sdist:
name: build sdist
runs-on: ubuntu-latest
needs: release-please
if: ${{ needs.release-please.outputs.releases_created == 'true' && needs.release-please.outputs.tag_name != '' }}
steps:
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
python-version: "3.13"
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
- name: Install maturin
run: uv tool install maturin
- name: Build sdist
run: uvx maturin sdist --out dist
- name: Upload sdist artifact
uses: actions/upload-artifact@v6
with:
name: sdist
path: dist/*.tar.gz
if-no-files-found: error
publish:
name: publish release artifacts
runs-on: ubuntu-latest
needs:
- release-please
- build-wheels
- build-sdist
if: ${{ needs.release-please.outputs.releases_created == 'true' && needs.release-please.outputs.tag_name != '' }}
environment: pypi
permissions:
id-token: write
contents: write
steps:
- name: Download wheel artifacts
uses: actions/download-artifact@v7
with:
pattern: wheels-*
merge-multiple: true
path: dist
- name: Download sdist artifact
uses: actions/download-artifact@v7
with:
name: sdist
path: dist
- name: Upload binaries to release
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: dist/*
tag: ${{ needs.release-please.outputs.tag_name }}
file_glob: true
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/
.gitignore
+218
View File
@@ -0,0 +1,218 @@
data/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.env.*
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be added to the global gitignore or merged into this project gitignore. For a PyCharm
# project, it is common to ignore sensitive information such as project configurations.
.idea/
# Database files
*.db
*.sqlite
*.sqlite3
persona_memory.db
persona_conversations.db
basic_memory_demo.db
demo_memory.db
demo_conversations.db
chatbot_memory.db
chatbot_conversations.db
integration_memory.db
integration_conversations.db
conversation_demo.db
# Log files
*.log
# Temporary files
tmp/
temp/
.tmp/
# OS specific
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
# IDE
.vscode/
*.swp
*.swo
*~
# Model files (if using local models)
models/
*.bin
*.pt
*.pth
*.onnx
# API keys and secrets
.env.local
.env.*.local
secrets.txt
api_keys.txt
.cursor
.local/
# Locally built Rust extension (offline dev convenience; normally built by maturin)
src/memu/_core.abi3.so
.pre-commit-config.yaml
+20
View File
@@ -0,0 +1,20 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: "v6.0.0"
hooks:
- id: check-case-conflict
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: check-json
- id: pretty-format-json
args: [--autofix, --no-sort-keys]
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.14.3"
hooks:
- id: ruff
args: [--exit-non-zero-on-fix]
- id: ruff-format
.python-version
+1
View File
@@ -0,0 +1 @@
3.13
AGENTS.md
+165
View File
@@ -0,0 +1,165 @@
# AGENTS.md
Operational guide for AI coding agents working in this repository.
## Mission
Ship small, verified feature and bugfix changes while preserving memU's current architecture.
Treat `docs/architecture.md` as the runtime source of truth and keep this file aligned with it.
Core invariants:
- `MemoryService` is the composition root for config, storage, LLM/VLM/embedding clients, workflows, interceptors, and memory file export.
- Public APIs should remain stable unless the requested change explicitly requires an API change.
- Major behavior runs through workflows: `memorize`, `retrieve_rag`, `retrieve_llm`, and CRUD/patch operations.
- Workflow steps must keep explicit `requires`/`produces` contracts and declared capability tags.
- Storage is pluggable across `inmemory`, `sqlite`, and `postgres`; repository contract changes require backend parity.
- Scope data comes from `UserConfig.model`; validate `where` filters and do not bypass scope filtering.
- LLM routing is profile-based across chat, embedding, VLM, and custom profiles.
- The exported memory workspace should stay human-readable and agent-traversable: `INDEX.md`, `MEMORY.md`, `memory/`, `resource/`, and `skill/`.
## First Move
Before editing, classify the request by the layer it touches:
- Service/runtime wiring: `src/memu/app/service.py`
- Memorize pipeline: `src/memu/app/memorize.py`
- Retrieve pipelines: `src/memu/app/retrieve.py`
- CRUD/patch behavior: `src/memu/app/crud.py`
- Config models/defaults: `src/memu/app/settings.py`
- Workflow engine and pipeline mutation: `src/memu/workflow/*`
- Memory file export/synthesis: `src/memu/app/memory_files.py`, `src/memu/memory_fs/*`, `src/memu/prompts/memory_fs*`
- Storage protocols/factory: `src/memu/database/interfaces.py`, `src/memu/database/factory.py`
- In-memory backend: `src/memu/database/inmemory/*`
- SQLite backend: `src/memu/database/sqlite/*`
- Postgres backend: `src/memu/database/postgres/*`
- Vector math/ranking: `src/memu/vector.py`
- LLM clients, wrappers, interceptors: `src/memu/llm/*`
- Embedding clients: `src/memu/embedding/*`
- Vision clients: `src/memu/vlm/*`
- Integrations/client surfaces: `src/memu/integrations/*`, `src/memu/client/*`
- Tests: `tests/*`
- User-facing docs/examples: `README.md`, `readme/*`, `docs/*`, `examples/*`
Read the relevant implementation and nearby tests first. Prefer local patterns over new abstractions.
## Implementation Rules
- Keep changes narrow and localized to the affected layer.
- Preserve async behavior and avoid sync work in async paths unless existing code already does it.
- Prefer adding or modifying a workflow step over embedding new flow logic directly in `MemoryService`.
- When adding configurable behavior, update typed settings and wire it through step config or service config.
- Preserve result shapes unless the task explicitly asks for a breaking change.
- Maintain type hints and mypy compatibility.
- Keep provider-specific logic inside provider/backend modules.
- Keep storage-neutral logic outside concrete backends; vector ranking helpers belong in shared modules such as `memu.vector`.
- Do not duplicate client caching. Use the existing client pool/profile routing patterns.
- Do not silently swallow errors that should be visible to callers or tests.
- Avoid broad refactors unless they are required to make the requested change safe.
## Workflow Guidance
For memorize work:
- Keep modality routing in the preprocessing pipeline.
- Use VLM clients for image/video paths and chat clients for conversation/document/audio paths.
- Preserve the flow: ingest resource, preprocess, extract items, dedupe/merge, categorize, persist index, build response.
- If rich document ingestion changes, consider the optional `document` extra and MarkItDown behavior.
For retrieve work:
- Check both `retrieve_rag` and `retrieve_llm` when behavior is shared.
- Validate `where` filters against the configured user model before querying.
- Keep category, item, and resource recall responsibilities separated.
- Preserve early-stop/sufficiency behavior when modifying ranking or response build steps.
For CRUD/patch work:
- Respect repository contracts and scope fields.
- Preserve existing output shapes for list/create/update/delete operations.
- Add backend coverage when contracts change.
For memory file export:
- Keep raw resources copied verbatim into `resource/`.
- Keep category memory split under `memory/<slug>.md`.
- Keep synthesized skills under `skill/<name>/SKILL.md`.
- Keep root files as indexes/overviews, not hidden sources of untracked state.
- Treat synthesized skill output as derived from source descriptions, not from unrelated runtime state.
## Backend Parity
If a repository method, model field, filter behavior, or vector search contract changes:
1. Update the protocol/interface.
2. Update `inmemory`, `sqlite`, and `postgres` implementations where applicable.
3. Add or extend conformance tests.
4. Check migrations/bootstrap behavior for SQL backends.
5. Verify fallback behavior when pgvector or optional extras are unavailable.
## LLM, VLM, And Embedding Routing
- Chat-like work uses LLM profiles.
- Embedding work uses embedding profiles and dedicated embedding clients.
- Image/video understanding uses VLM clients derived from LLM profiles unless explicitly configured otherwise.
- Step config should select profiles with existing keys such as `llm_profile`, `chat_llm_profile`, and `embed_llm_profile`.
- Keep interceptors and usage metadata behavior consistent across capabilities.
- Do not add embeddings back into chat clients; embedding is intentionally decoupled.
## Feature Checklist
1. Locate the affected flow, backend, integration, or export path.
2. Update settings/defaults only if behavior is configurable.
3. Wire the change through `MemoryService`, pipeline registration, or step config as appropriate.
4. Implement repository/backend changes for all impacted providers.
5. Add focused tests for the happy path and edge cases.
6. Update docs/examples for user-visible behavior.
7. Add or update ADRs under `docs/adr/` for architectural changes.
## Bug Fix Checklist
1. Reproduce the bug with an existing or new failing test.
2. Fix the smallest correct layer.
3. Add a regression test that fails before and passes after.
4. Check cross-backend effects when storage behavior changes.
5. Check both retrieval modes when recall/ranking/response behavior changes.
6. Confirm no unintended public API or output-shape changes.
## Testing And Validation
Use `uv` for local runs.
- Setup: `make install`
- Run all tests: `make test`
- Run focused tests: `uv run python -m pytest tests/<target_test>.py`
- Full quality checks: `make check`
At minimum, run targeted tests for touched code. Run `make check` for broad, cross-cutting, or release-critical changes.
If a required check cannot be run, say why in the final summary.
Useful focused areas:
- Storage parity: `tests/test_backend_conformance.py`, `tests/test_inmemory.py`, `tests/test_sqlite.py`, `tests/test_postgres.py`
- Retrieval/vector behavior: `tests/test_vector.py`, `tests/test_salience.py`, retrieve-related tests
- Multimodal preprocessing: `tests/test_conversation_preprocess.py`, `tests/test_document_text.py`, `tests/test_audio_preprocess.py`, `tests/test_vlm_preprocess.py`
- Memory file export/synthesis: `tests/test_memory_files.py`, `tests/test_memory_fs_synthesis.py`
- Provider routing: `tests/test_embedding.py`, `tests/test_openrouter.py`, `tests/test_openai_max_tokens.py`, `tests/test_lazyllm.py`
- Integrations/tool memory: `tests/test_tool_memory.py`, LangGraph-related tests
## Documentation Rules
- Update `README.md`, localized `readme/*`, `docs/*`, or examples when behavior visible to users changes.
- Keep architecture docs current when composition, workflow contracts, backends, or profile routing changes.
- Keep examples runnable and aligned with the public API.
- Do not document speculative features as implemented.
## Done Criteria
Before finishing:
- Code compiles for touched paths.
- Tests for changed behavior pass.
- New behavior has test coverage.
- User-visible or architectural behavior is documented.
- Backend parity has been considered and implemented where needed.
- No unrelated files were modified.
CHANGELOG.md
+290
View File
@@ -0,0 +1,290 @@
# Changelog
## [1.5.1](https://github.com/NevaMind-AI/memU/compare/v1.5.0...v1.5.1) (2026-03-23)
### Bug Fixes
* use correct interpolation in alembic url ([#390](https://github.com/NevaMind-AI/memU/issues/390)) ([fd87ceb](https://github.com/NevaMind-AI/memU/commit/fd87ceb558eaa800aeb694b045847971958f23a3))
## [1.5.0](https://github.com/NevaMind-AI/memU/compare/v1.4.0...v1.5.0) (2026-03-17)
### Features
* add non-propagate option for memory patch ([#386](https://github.com/NevaMind-AI/memU/issues/386)) ([3b67458](https://github.com/NevaMind-AI/memU/commit/3b67458c6325b4fcdaccbc34f6e0fea491b7ba96))
* **http:** add HTTP proxy support for LLM & embedding clients ([#310](https://github.com/NevaMind-AI/memU/issues/310)) ([a3d45e2](https://github.com/NevaMind-AI/memU/commit/a3d45e2f4174702fd8fd83cd13862285cd47cad9))
### Bug Fixes
* httpx base_url path discarded when endpoint starts with / ([#328](https://github.com/NevaMind-AI/memU/issues/328), [#336](https://github.com/NevaMind-AI/memU/issues/336), [#341](https://github.com/NevaMind-AI/memU/issues/341), [#329](https://github.com/NevaMind-AI/memU/issues/329)) ([#344](https://github.com/NevaMind-AI/memU/issues/344)) ([9e31ef2](https://github.com/NevaMind-AI/memU/commit/9e31ef235d46317f383613825090c52943cb55a2))
### Documentation
* add architecture explanation and adrs ([#353](https://github.com/NevaMind-AI/memU/issues/353)) ([8676a27](https://github.com/NevaMind-AI/memU/commit/8676a2721917f4e0baee6d01a9d6b95bd642be97))
* add pull request template to improve contribution quality ([#261](https://github.com/NevaMind-AI/memU/issues/261)) ([2256119](https://github.com/NevaMind-AI/memU/commit/2256119fb2d177890c7f738c926f437a355e7359))
* Update Discord link in README.md ([#377](https://github.com/NevaMind-AI/memU/issues/377)) ([163d050](https://github.com/NevaMind-AI/memU/commit/163d050299b77143226e9727f67d4826c9a69f92))
* update memU Bot section with GitHub reference and "Now open source" ([#366](https://github.com/NevaMind-AI/memU/issues/366)) ([dc6880a](https://github.com/NevaMind-AI/memU/commit/dc6880a793d40b04cd599298a110af5f0f0b9cb5))
## [1.4.0](https://github.com/NevaMind-AI/memU/compare/v1.3.0...v1.4.0) (2026-02-06)
### Features
* Add inline memory item references in category summaries ([#202](https://github.com/NevaMind-AI/memU/issues/202)) ([#205](https://github.com/NevaMind-AI/memU/issues/205)) ([5213571](https://github.com/NevaMind-AI/memU/commit/5213571b218d85784e0771f7a721eafd7da1c1ff))
* Add salience-aware memory with reinforcement tracking ([#186](https://github.com/NevaMind-AI/memU/issues/186)) ([#206](https://github.com/NevaMind-AI/memU/issues/206)) ([2bdbcce](https://github.com/NevaMind-AI/memU/commit/2bdbcce1a87ae1017d5930901fb0ae8d2924dcee))
* Add Tool Memory type with specialized metadata and statistics ([#247](https://github.com/NevaMind-AI/memU/issues/247)) ([4e8a035](https://github.com/NevaMind-AI/memU/commit/4e8a03578641afd0e07f9700629dff8d91d2b3fb))
### Bug Fixes
* add chat() API and stop misusing summarize() ([#208](https://github.com/NevaMind-AI/memU/issues/208)) ([be0a5c7](https://github.com/NevaMind-AI/memU/commit/be0a5c73250f0a21ad5e0d39b0e66e3018809e0f))
* remove unused type: ignore comment and add lazyllm mypy override ([#275](https://github.com/NevaMind-AI/memU/issues/275)) ([0e490f7](https://github.com/NevaMind-AI/memU/commit/0e490f7333feecffaef0901cccb1c9a5dbb7bafb))
* **video:** cleanup temp files on extraction failure ([#295](https://github.com/NevaMind-AI/memU/issues/295)) ([16b65e5](https://github.com/NevaMind-AI/memU/commit/16b65e54aef69dfe617835ca534c12752e06eda8))
### Documentation
* file system in readme ([#301](https://github.com/NevaMind-AI/memU/issues/301)) ([ee27d8a](https://github.com/NevaMind-AI/memU/commit/ee27d8aa5e42d249963b08d5775338080b0255f8))
* fix name ([#291](https://github.com/NevaMind-AI/memU/issues/291)) ([fd0d179](https://github.com/NevaMind-AI/memU/commit/fd0d17998f6c0c9ba5c7e91964c39ee909f2b366))
* fix readme lint ([#290](https://github.com/NevaMind-AI/memU/issues/290)) ([a7a5516](https://github.com/NevaMind-AI/memU/commit/a7a55169ecbc1579ae87dc68c5e09b1d62f07b56))
* lint error ([#287](https://github.com/NevaMind-AI/memU/issues/287)) ([7f61dc7](https://github.com/NevaMind-AI/memU/commit/7f61dc72ffbadf849f227cf314bba7ce7964761b))
* readme memubot ([#289](https://github.com/NevaMind-AI/memU/issues/289)) ([2f30798](https://github.com/NevaMind-AI/memU/commit/2f30798367e3591107e1e66a184d0dd4ad6c2f93))
* Update README.md ([#300](https://github.com/NevaMind-AI/memU/issues/300)) ([1075d7c](https://github.com/NevaMind-AI/memU/commit/1075d7c9ec3a272846c9fdb8c22ec58ff73cf6e7))
## [1.3.0](https://github.com/NevaMind-AI/memU/compare/v1.2.0...v1.3.0) (2026-01-29)
### Features
* add happened at and extra fields to memory item ([#262](https://github.com/NevaMind-AI/memU/issues/262)) ([77938e9](https://github.com/NevaMind-AI/memU/commit/77938e9c282e1c0eda11088675c35975d85c4ff0))
* add proactive example ([#268](https://github.com/NevaMind-AI/memU/issues/268)) ([d3d1de1](https://github.com/NevaMind-AI/memU/commit/d3d1de1d9b0f45d9b14479cbaa4462458b172005))
* add Sealos support agent use case (Track G) ([#255](https://github.com/NevaMind-AI/memU/issues/255)) ([8fbdf3c](https://github.com/NevaMind-AI/memU/commit/8fbdf3c301f74f2aa85604837e00bb305b8801ec))
* integrate LazyLLM to provide more llm services ([#265](https://github.com/NevaMind-AI/memU/issues/265)) ([c03f639](https://github.com/NevaMind-AI/memU/commit/c03f639677d6c897b75dfe28d0cd92d5b5270957))
* **integrations:** Add LangGraph Adapter for MemU (Track A) ([#258](https://github.com/NevaMind-AI/memU/issues/258)) ([50b5502](https://github.com/NevaMind-AI/memU/commit/50b5502ebcacd86401f98b1bb7e5a6577fab7126))
* **llm:** add Grok (xAI) integration ([#179](https://github.com/NevaMind-AI/memU/issues/179)) ([#236](https://github.com/NevaMind-AI/memU/issues/236)) ([1e16309](https://github.com/NevaMind-AI/memU/commit/1e1630948af535f8ed9d608e6c4f9d2748d4ce8e))
* Openrouter integration as backend provider ([#182](https://github.com/NevaMind-AI/memU/issues/182)) ([cba667a](https://github.com/NevaMind-AI/memU/commit/cba667a56daca5093c9b79a598c7d2ffda813756))
### Bug Fixes
* memory type & proactive example ([#272](https://github.com/NevaMind-AI/memU/issues/272)) ([710f14d](https://github.com/NevaMind-AI/memU/commit/710f14d4b171c5cde609a9dc2caf454681ab94b3))
* proactive examples ([#273](https://github.com/NevaMind-AI/memU/issues/273)) ([603ae12](https://github.com/NevaMind-AI/memU/commit/603ae122b94741bb350656086960c4e2bb868c2a))
### Documentation
* Add link to memU bot ([#276](https://github.com/NevaMind-AI/memU/issues/276)) ([2f84231](https://github.com/NevaMind-AI/memU/commit/2f842311ed592c1a45cdb9e24f7a128da97a0a39))
* multilingual readme ([#271](https://github.com/NevaMind-AI/memU/issues/271)) ([200f47a](https://github.com/NevaMind-AI/memU/commit/200f47a15ff7d05fc435abe1d2cefbb3774548fe))
* Update memU bot link to include full URL ([#277](https://github.com/NevaMind-AI/memU/issues/277)) ([6874eaf](https://github.com/NevaMind-AI/memU/commit/6874eaf278e910d07a4427dd4c06f9c4c21283ce))
* update readme ([#266](https://github.com/NevaMind-AI/memU/issues/266)) ([16ae534](https://github.com/NevaMind-AI/memU/commit/16ae534675a5e0711256a5e2147b9190fd8b2524))
* update README ([#270](https://github.com/NevaMind-AI/memU/issues/270)) ([b531d39](https://github.com/NevaMind-AI/memU/commit/b531d39e5538449b31cb212aea1deea24e12180f))
## [1.2.0](https://github.com/NevaMind-AI/memU/compare/v1.1.2...v1.2.0) (2026-01-14)
### Features
* clear memory ([#239](https://github.com/NevaMind-AI/memU/issues/239)) ([7da36da](https://github.com/NevaMind-AI/memU/commit/7da36da57d7013df213e537b1f238f5a526e69d9))
* **database:** add SQLite storage backend ([#201](https://github.com/NevaMind-AI/memU/issues/201)) ([4b899d7](https://github.com/NevaMind-AI/memU/commit/4b899d7b768fcd48b6456234800033e3b2e3173c))
* improve issue template ([#199](https://github.com/NevaMind-AI/memU/issues/199)) ([abe0f1b](https://github.com/NevaMind-AI/memU/commit/abe0f1b77d5f8294ff45c9f57f5aeb6de33977d6))
* optimize topk pick function ([#196](https://github.com/NevaMind-AI/memU/issues/196)) ([b474c54](https://github.com/NevaMind-AI/memU/commit/b474c54a7eea06ba56844d6482d5e78b3cb3f020))
* workflow step interceptor ([#240](https://github.com/NevaMind-AI/memU/issues/240)) ([bf2ac96](https://github.com/NevaMind-AI/memU/commit/bf2ac96f1cb26b196f27b543d4c3e157f408f343))
### Bug Fixes
* allow nullable resource_id in Postgres items ([#232](https://github.com/NevaMind-AI/memU/issues/232)) ([ae2ffbb](https://github.com/NevaMind-AI/memU/commit/ae2ffbb0bcd69170875a47cebf322b71a15f4bd3))
* correct coverage source to track memu package instead of tests ([#220](https://github.com/NevaMind-AI/memU/issues/220)) ([2460bbd](https://github.com/NevaMind-AI/memU/commit/2460bbd2a2c4df5b3664db81d89886b926b51031))
* correct typo in PromptBlock class label attribute ([#231](https://github.com/NevaMind-AI/memU/issues/231)) ([d69053c](https://github.com/NevaMind-AI/memU/commit/d69053cece3467dd7c8cbf0634e72447649095f7))
* default embed size ([#192](https://github.com/NevaMind-AI/memU/issues/192)) ([144fd32](https://github.com/NevaMind-AI/memU/commit/144fd32cfd6917b08f29cf56f251f239c360d2ae))
* readme partners link & github issue link ([#198](https://github.com/NevaMind-AI/memU/issues/198)) ([849f881](https://github.com/NevaMind-AI/memU/commit/849f881f004b0772f372069e40f937731c114c89))
### Documentation
* add Chinese translation of README ([#212](https://github.com/NevaMind-AI/memU/issues/212)) ([d15ecf9](https://github.com/NevaMind-AI/memU/commit/d15ecf97647d8fd9f1d2ae95008a08bbba93a44a))
* add How to Contribute section to README ([#215](https://github.com/NevaMind-AI/memU/issues/215)) ([bf44de7](https://github.com/NevaMind-AI/memU/commit/bf44de722965fd611c0dc585712eac587caa9e6c))
* **readme:** update Chinese README with fixes and improvements ([#221](https://github.com/NevaMind-AI/memU/issues/221)) ([75a8f65](https://github.com/NevaMind-AI/memU/commit/75a8f651bb4b491ab5b0363de789c0dcd0291287))
## [1.1.2](https://github.com/NevaMind-AI/memU/compare/v1.1.1...v1.1.2) (2026-01-08)
### Bug Fixes
* custom memory type default prompt ([#169](https://github.com/NevaMind-AI/memU/issues/169)) ([5a0032f](https://github.com/NevaMind-AI/memU/commit/5a0032fc0f29229524d0356d454a3f5991e04f7b))
* prompt & item fallback ([#183](https://github.com/NevaMind-AI/memU/issues/183)) ([bc95ade](https://github.com/NevaMind-AI/memU/commit/bc95adeb26789104c0bbe199e126cf05def27941))
### Documentation
* add docs folder ([#181](https://github.com/NevaMind-AI/memU/issues/181)) ([919d2ca](https://github.com/NevaMind-AI/memU/commit/919d2caef23107d539c36f30c44d4fb5aa38b324))
* fix issue template dropdown ([#167](https://github.com/NevaMind-AI/memU/issues/167)) ([14d0333](https://github.com/NevaMind-AI/memU/commit/14d03331ed14f1e593fdebbf561189a3775651be))
* issue template fix ([#165](https://github.com/NevaMind-AI/memU/issues/165)) ([5d91237](https://github.com/NevaMind-AI/memU/commit/5d912372e1bf70d7ff573cd5b92e917118048e25))
## [1.1.1](https://github.com/NevaMind-AI/memU/compare/v1.1.0...v1.1.1) (2026-01-07)
### Bug Fixes
* ensure both Linux x86_64 and ARM64 wheels are built ([#162](https://github.com/NevaMind-AI/memU/issues/162)) ([51c9ea4](https://github.com/NevaMind-AI/memU/commit/51c9ea4335a1e9eac227c79783d7aa8cca2b883b))
### Documentation
* add custom LLM and embedding configuration guide ([#160](https://github.com/NevaMind-AI/memU/issues/160)) ([29c414a](https://github.com/NevaMind-AI/memU/commit/29c414a84d1c9c2f989e2165d0b6508c3fe62862))
* add template ([#158](https://github.com/NevaMind-AI/memU/issues/158)) ([b79a78d](https://github.com/NevaMind-AI/memU/commit/b79a78d25b96e670888c18b8f09277db33803865))
## [1.1.0](https://github.com/NevaMind-AI/memU/compare/v1.0.1...v1.1.0) (2026-01-07)
### Features
* add Linux ARM64 (aarch64) build target ([#156](https://github.com/NevaMind-AI/memU/issues/156)) ([0c90fcf](https://github.com/NevaMind-AI/memU/commit/0c90fcfb19fc3e91b951f4ba7454798e4b83e42c))
### Bug Fixes
* llm client profile & wrapper ([#157](https://github.com/NevaMind-AI/memU/issues/157)) ([e55c668](https://github.com/NevaMind-AI/memU/commit/e55c66847eac3ceaf276587d58b76d953d7be9f8))
### Documentation
* remove legacy docs ([#154](https://github.com/NevaMind-AI/memU/issues/154)) ([11fda41](https://github.com/NevaMind-AI/memU/commit/11fda41dda8a5c38b790d3c2fb7053b57b16606e))
## [1.0.1](https://github.com/NevaMind-AI/memU/compare/v1.0.0...v1.0.1) (2026-01-06)
### Bug Fixes
* get embedding client ([#152](https://github.com/NevaMind-AI/memU/issues/152)) ([76716a4](https://github.com/NevaMind-AI/memU/commit/76716a4f00127a3cc21444996f32a9cf810c9282))
* Readme incomplete ([#148](https://github.com/NevaMind-AI/memU/issues/148)) ([f8bc748](https://github.com/NevaMind-AI/memU/commit/f8bc748b9ebfb652ca8a5e06edbbe7eb648ed4f6))
## [1.0.0](https://github.com/NevaMind-AI/memU/compare/v0.9.0...v1.0.0) (2026-01-05)
### ⚠ BREAKING CHANGES
* v1.0.0 ([#147](https://github.com/NevaMind-AI/memU/issues/147))
### Bug Fixes
* v1.0.0 ([#147](https://github.com/NevaMind-AI/memU/issues/147)) ([23f37ee](https://github.com/NevaMind-AI/memU/commit/23f37ee403ecaf88b48af0144e3d701c22ccfddd))
### Documentation
* add readme cloud api ([#144](https://github.com/NevaMind-AI/memU/issues/144)) ([42fd5ba](https://github.com/NevaMind-AI/memU/commit/42fd5babe6748ef54254cf114a54bdefea304f07))
* fix api doc link ([#146](https://github.com/NevaMind-AI/memU/issues/146)) ([23ce7d1](https://github.com/NevaMind-AI/memU/commit/23ce7d19b1d68f4fd0a5a5ac6b756f69fede8d09))
## [0.9.0](https://github.com/NevaMind-AI/memU/compare/v0.8.0...v0.9.0) (2026-01-04)
### Features
* add GitHub issue templates (bug report, designer feedback, feat… ([#132](https://github.com/NevaMind-AI/memU/issues/132)) ([aee22ee](https://github.com/NevaMind-AI/memU/commit/aee22ee94f275749f69367b83a02a2e819cfd001))
* add LLM wrapper and interceptors for LLM calls ([#131](https://github.com/NevaMind-AI/memU/issues/131)) ([416e102](https://github.com/NevaMind-AI/memU/commit/416e1029e7752f173c133ebc83bc42801a313059))
* patch & crud workflows ([#127](https://github.com/NevaMind-AI/memU/issues/127)) ([3cd3dc6](https://github.com/NevaMind-AI/memU/commit/3cd3dc65ae9488207ff8fb0c81e4adb0d22c0f91))
### Bug Fixes
* category summary & category user scopes ([#125](https://github.com/NevaMind-AI/memU/issues/125)) ([a24efd5](https://github.com/NevaMind-AI/memU/commit/a24efd57df2e305fa3eae8622ea889318979c2f7))
* config & resource caption ([#142](https://github.com/NevaMind-AI/memU/issues/142)) ([ea4be13](https://github.com/NevaMind-AI/memU/commit/ea4be1396c0f55b02d706819f6c2b4d5c6e68be8))
* remove duplicate and unnecessary issue templates ([#133](https://github.com/NevaMind-AI/memU/issues/133)) ([559ec14](https://github.com/NevaMind-AI/memU/commit/559ec14d2561ac09162bbb93f178e0f74cc58b23))
### Documentation
* fix README and CONTRIBUTING to match actual codebase ([#130](https://github.com/NevaMind-AI/memU/issues/130)) ([65d7ef4](https://github.com/NevaMind-AI/memU/commit/65d7ef414563395c6cfa0a75343864671c11b62d))
* modify readme.md 0102 ([#136](https://github.com/NevaMind-AI/memU/issues/136)) ([f114ee4](https://github.com/NevaMind-AI/memU/commit/f114ee46c8639b256472a8e989695b2f2215f4d4))
## [0.8.0](https://github.com/NevaMind-AI/memU/compare/v0.7.0...v0.8.0) (2025-12-24)
### Features
* add configurable batch_size for embedding API calls ([#114](https://github.com/NevaMind-AI/memU/issues/114)) ([060067a](https://github.com/NevaMind-AI/memU/commit/060067a5ebe8ad36c4ca4ad2cc6033303c3f1a36))
* add conversation created at ([#120](https://github.com/NevaMind-AI/memU/issues/120)) ([825df56](https://github.com/NevaMind-AI/memU/commit/825df5640037fa2345c3153c3df89174059551b3))
* add workflow implementation and postgres store ([#122](https://github.com/NevaMind-AI/memU/issues/122)) ([a175811](https://github.com/NevaMind-AI/memU/commit/a1758110f859f3725960d308fa0344a1056be52c))
### Bug Fixes
* example 3 output ([#117](https://github.com/NevaMind-AI/memU/issues/117)) ([65ef7c6](https://github.com/NevaMind-AI/memU/commit/65ef7c6a497047f0f86938cf77ee4602a630216b))
* postgres model definitions & database initialization ([#124](https://github.com/NevaMind-AI/memU/issues/124)) ([7d5e0cb](https://github.com/NevaMind-AI/memU/commit/7d5e0cb02837a6e2071d3c3344ef6dad613f9a43))
### Documentation
* Add memU-experiment link to README ([#119](https://github.com/NevaMind-AI/memU/issues/119)) ([2d908e1](https://github.com/NevaMind-AI/memU/commit/2d908e17ee2e15b95d4eb2f35dd09f924d57f8cf))
* clearer introduction and new agent examples ([#115](https://github.com/NevaMind-AI/memU/issues/115)) ([da27875](https://github.com/NevaMind-AI/memU/commit/da2787536d91490f7a96a0c7379abd1ad1c6d9ac))
## [0.7.0](https://github.com/NevaMind-AI/memU/compare/v0.6.0...v0.7.0) (2025-12-05)
### Features
* add user model and user context store ([#113](https://github.com/NevaMind-AI/memU/issues/113)) ([7c37fb1](https://github.com/NevaMind-AI/memU/commit/7c37fb166b0a85bf7c89d0b109cbaa882fa80064))
* add valcano model support ([#110](https://github.com/NevaMind-AI/memU/issues/110)) ([704c302](https://github.com/NevaMind-AI/memU/commit/704c3024946e8d4a1d1b13ffef33126bb68bd9c4))
### Bug Fixes
* resource caption miss problem ([#111](https://github.com/NevaMind-AI/memU/issues/111)) ([85586f5](https://github.com/NevaMind-AI/memU/commit/85586f52d6bd6355ae440ad06d8cc6779c689ce8))
### Documentation
* fix example file path ([#105](https://github.com/NevaMind-AI/memU/issues/105)) ([21aad6a](https://github.com/NevaMind-AI/memU/commit/21aad6a7f070e7666ac6a41c91980a4fa9696918))
* fix readme test case ([#107](https://github.com/NevaMind-AI/memU/issues/107)) ([228306c](https://github.com/NevaMind-AI/memU/commit/228306c897402c633f57c7aa31e8c6cd4e995d5d))
* highlight OpenAI key ([#106](https://github.com/NevaMind-AI/memU/issues/106)) ([5b6ce54](https://github.com/NevaMind-AI/memU/commit/5b6ce54def5aa7743a85bec629da65bfa9d8333b))
* highlight readme openai key ([#103](https://github.com/NevaMind-AI/memU/issues/103)) ([fc4154a](https://github.com/NevaMind-AI/memU/commit/fc4154a707220ced4359e7296218451c43cf0681))
## [0.6.0](https://github.com/NevaMind-AI/memU/compare/v0.5.0...v0.6.0) (2025-11-20)
### Features
* retrieve args change conversation_history to queries ([#98](https://github.com/NevaMind-AI/memU/issues/98)) ([6370c6e](https://github.com/NevaMind-AI/memU/commit/6370c6e968c9d0922120bf2a41e8b4206bab87cb))
## [0.5.0](https://github.com/NevaMind-AI/memU/compare/v0.4.0...v0.5.0) (2025-11-18)
### Features
* add usecase examples ([#94](https://github.com/NevaMind-AI/memU/issues/94)) ([47b5b39](https://github.com/NevaMind-AI/memU/commit/47b5b390e065ccac1cd2173fa2d6c41549e01063))
## [0.4.0](https://github.com/NevaMind-AI/memU/compare/v0.3.0...v0.4.0) (2025-11-18)
### Features
* add non-RAG retrieve solution ([#84](https://github.com/NevaMind-AI/memU/issues/84)) ([fb96e54](https://github.com/NevaMind-AI/memU/commit/fb96e5405f1c0e3477929b7d9874e624dd0453cb))
### Bug Fixes
* correct binary name after making a release ([#91](https://github.com/NevaMind-AI/memU/issues/91)) ([0fa721a](https://github.com/NevaMind-AI/memU/commit/0fa721aaae294c6f230221af11084a0a6c1f478d))
### Documentation
* fix several words in README ([#89](https://github.com/NevaMind-AI/memU/issues/89)) ([1e3baf9](https://github.com/NevaMind-AI/memU/commit/1e3baf92d5a08ec51a7eda3249bbd4b40530f56c))
* revise README for clarity and roadmap inclusion ([#86](https://github.com/NevaMind-AI/memU/issues/86)) ([2235b09](https://github.com/NevaMind-AI/memU/commit/2235b099acc7e92ac52b2613fa731f85259d58fd))
* update images and refine punctuation in README ([#88](https://github.com/NevaMind-AI/memU/issues/88)) ([cc375e8](https://github.com/NevaMind-AI/memU/commit/cc375e89a9b49bda0b7057eee696d74c4c1d9cee))
## [0.3.0](https://github.com/NevaMind-AI/memU/compare/v0.2.2...v0.3.0) (2025-11-17)
### Features
* initialize the memorize and retrieve workflows with the new 3-layer architecture ([#81](https://github.com/NevaMind-AI/memU/issues/81)) ([4a2e86c](https://github.com/NevaMind-AI/memU/commit/4a2e86c0e8bc3e50c82c4fde33d07ad741c8a65e))
### Documentation
* add German translation for README ([f6d6ab1](https://github.com/NevaMind-AI/memU/commit/f6d6ab1a51b76bd4312542422aebac89410b8da9))
* add German translation for README ([76c3fc4](https://github.com/NevaMind-AI/memU/commit/76c3fc4227a8e349fbcf1e5bfec29d68e984ff7a))
CONTRIBUTING.md
+239
View File
@@ -0,0 +1,239 @@
# Contributing to MemU
Thank you for your interest in contributing to MemU! This document provides guidelines and information for contributors.
## 🌟 Ways to Contribute
We welcome all types of contributions:
- 🐛 **Bug Reports** - Help us identify and fix issues
- 💡 **Feature Requests** - Suggest new capabilities and improvements
- 📝 **Documentation** - Improve guides, examples, and API docs
- 🔧 **Code Contributions** - Add features, fix bugs, optimize performance
- 🧪 **Testing** - Write tests, improve coverage, test edge cases
- 🎨 **UI/UX** - Enhance user experience and interface design
- 🌐 **Translations** - Help make MemU accessible globally
- 📢 **Community** - Help others in discussions and support channels
## 🚀 Quick Start for Contributors
### Prerequisites
- Python 3.13+
- Git
- [uv](https://github.com/astral-sh/uv) (Python package manager)
- A code editor (VS Code recommended)
### Development Setup
```bash
# 1. Fork the repository on GitHub
# 2. Clone your fork locally
git clone https://github.com/YOUR_USERNAME/MemU.git
cd MemU
# 3. Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# 4. Install development dependencies and pre-commit hooks
make install
# 5. Verify setup by running tests
make test
```
### Available Commands
```bash
make install # Create virtual environment and install dependencies with uv
make test # Run tests with pytest and coverage
make check # Run all checks (lock file, pre-commit, mypy, deptry)
```
## 🔧 Development Guidelines
### Code Style
- Follow **PEP 8** Python style guidelines
- Use **Ruff** for code formatting and linting (line length: 120)
- Use **type hints** for all functions and methods
- Write **docstrings** for public APIs
### Code Quality Standards
- Maintain **test coverage > 80%**
- All code must pass **linting** (ruff, mypy)
- Use **meaningful variable and function names**
- Keep functions **focused and small**
- Follow **SOLID principles**
### Testing
```bash
# Run all tests with coverage
make test
# Run tests directly with pytest
uv run python -m pytest
# Run with coverage report
uv run python -m pytest --cov --cov-config=pyproject.toml --cov-report=html
# Run specific test file
uv run python -m pytest tests/rust_entry_test.py
# Run tests with specific marker
uv run python -m pytest -m "not slow"
```
## 📝 Submitting Changes
### Before You Start
1. **Search existing issues** to avoid duplicates
2. **Create an issue** for new features or major changes
3. **Discuss your approach** in the issue before implementing
### Creating Issues
When reporting bugs, please include:
- **Environment details** (Python version, OS, MemU version)
- **Reproduction steps** with minimal code example
- **Expected vs actual behavior**
- **Error messages** or stack traces
For feature requests, please describe:
- **The problem** you're trying to solve
- **Proposed solution** or approach
- **Alternative solutions** you've considered
- **Use cases** and examples
### Pull Request Process
1. **Create a feature branch**
```bash
git checkout -b feature/amazing-feature
# or for bug fixes
git checkout -b bugfix/fix-memory-leak
```
2. **Make your changes**
- Write clear, descriptive commit messages
- Keep commits focused and atomic
- Add tests for new functionality
- Update documentation as needed
3. **Test your changes**
```bash
make test
make lint
make coverage
```
4. **Submit pull request**
- Use descriptive title and description
- Reference related issues (e.g., "Fixes #123")
- Include testing instructions
- Add screenshots for UI changes
### Commit Message Format
Use conventional commit format:
```
type(scope): description
Examples:
feat(memory): add semantic search functionality
fix(llm): resolve OpenAI timeout issues
docs(readme): update installation instructions
test(agent): add unit tests for memory retrieval
refactor(core): restructure memory storage logic
```
**Types:**
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation changes
- `test`: Adding or fixing tests
- `refactor`: Code restructuring without feature changes
- `perf`: Performance improvements
- `chore`: Maintenance tasks
## 🎯 Current Priorities
We're currently focusing on:
| Priority | Area | Description |
|----------|------|-------------|
| 🔥 **High** | Multi-modal Support | Images, audio, video memory processing |
| 🔥 **High** | Performance | Memory retrieval optimization, caching |
| 🔥 **High** | LLM Providers | Additional provider integrations |
| 🟡 **Medium** | Enterprise Features | SSO, RBAC, audit logging |
| 🟡 **Medium** | Mobile SDKs | React Native, Flutter support |
| 🟢 **Low** | UI Improvements | Dashboard, memory visualization |
## 🏷️ Issue Labels
| Label | Description |
|-------|-------------|
| `good first issue` | Perfect for newcomers |
| `help wanted` | Extra attention needed |
| `bug` | Something isn't working |
| `enhancement` | New feature request |
| `documentation` | Improvements to docs |
| `performance` | Performance optimization |
| `breaking change` | Requires version bump |
| `priority:high` | Urgent issues |
| `priority:medium` | Important issues |
| `priority:low` | Nice to have |
## 📋 Code Review Process
### For Contributors
- Be open to feedback and constructive criticism
- Respond promptly to review comments
- Make requested changes in new commits (don't force push)
- Ask questions if feedback is unclear
### For Reviewers
- Be constructive and respectful in feedback
- Focus on code quality, maintainability, and project goals
- Suggest improvements with explanations
- Approve when ready, request changes when needed
## 🔒 Security
**Reporting Security Issues:**
- **DO NOT** create public issues for security vulnerabilities
- Email security issues privately to [contact@nevamind.ai](mailto:contact@nevamind.ai)
- Include detailed reproduction steps and impact assessment
- We'll acknowledge receipt within 24 hours
## 📄 License and Attribution
By contributing to MemU, you agree that:
- Your contributions will be licensed under the **Apache License 2.0**
- You have the right to contribute the code/content
- Your contribution doesn't violate any third-party rights
## 🌍 Community Guidelines
- Be respectful and inclusive
- Follow our [Code of Conduct](CODE_OF_CONDUCT.md)
- Help others learn and grow
- Share knowledge and best practices
- Celebrate diverse perspectives and experiences
## 📞 Getting Help
| Channel | Best For |
|---------|----------|
| 💬 [Discord](https://discord.gg/memu) | Real-time chat, quick questions |
| 🗣️ [GitHub Discussions](https://github.com/NevaMind-AI/MemU/discussions) | Feature discussions, Q&A |
| 🐛 [GitHub Issues](https://github.com/NevaMind-AI/MemU/issues) | Bug reports, feature requests |
| 📧 [Email](mailto:contact@nevamind.ai) | Private inquiries |
## 🎉 Recognition
Contributors are recognized in:
- README.md contributors section
- Release notes for significant contributions
- Our [Contributors](https://github.com/NevaMind-AI/MemU/graphs/contributors) page
Thank you for helping make MemU better! 🚀
Cargo.lock
Generated
+172
View File
@@ -0,0 +1,172 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "autocfg"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "indoc"
version = "2.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706"
dependencies = [
"rustversion",
]
[[package]]
name = "libc"
version = "0.2.177"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2874a2af47a2325c2001a6e6fad9b16a53b802102b528163885171cf92b15976"
[[package]]
name = "memoffset"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
dependencies = [
"autocfg",
]
[[package]]
name = "memu"
version = "0.1.0"
dependencies = [
"pyo3",
]
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "portable-atomic"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483"
[[package]]
name = "proc-macro2"
version = "1.0.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ee95bc4ef87b8d5ba32e8b7714ccc834865276eab0aed5c9958d00ec45f49e8"
dependencies = [
"unicode-ident",
]
[[package]]
name = "pyo3"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37a6df7eab65fc7bee654a421404947e10a0f7085b6951bf2ea395f4659fb0cf"
dependencies = [
"indoc",
"libc",
"memoffset",
"once_cell",
"portable-atomic",
"pyo3-build-config",
"pyo3-ffi",
"pyo3-macros",
"unindent",
]
[[package]]
name = "pyo3-build-config"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f77d387774f6f6eec64a004eac0ed525aab7fa1966d94b42f743797b3e395afb"
dependencies = [
"target-lexicon",
]
[[package]]
name = "pyo3-ffi"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dd13844a4242793e02df3e2ec093f540d948299a6a77ea9ce7afd8623f542be"
dependencies = [
"libc",
"pyo3-build-config",
]
[[package]]
name = "pyo3-macros"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eaf8f9f1108270b90d3676b8679586385430e5c0bb78bb5f043f95499c821a71"
dependencies = [
"proc-macro2",
"pyo3-macros-backend",
"quote",
"syn",
]
[[package]]
name = "pyo3-macros-backend"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70a3b2274450ba5288bc9b8c1b69ff569d1d61189d4bff38f8d22e03d17f932b"
dependencies = [
"heck",
"proc-macro2",
"pyo3-build-config",
"quote",
"syn",
]
[[package]]
name = "quote"
version = "1.0.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a338cc41d27e6cc6dce6cefc13a0729dfbb81c262b1f519331575dd80ef3067f"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "syn"
version = "2.0.109"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f17c7e013e88258aa9543dcbe81aca68a667a9ac37cd69c9fbc07858bfe0e2f"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "target-lexicon"
version = "0.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df7f62577c25e07834649fc3b39fafdc597c0a3527dc1c60129201ccfcbaa50c"
[[package]]
name = "unicode-ident"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9312f7c4f6ff9069b165498234ce8be658059c6728633667c526e27dc2cf1df5"
[[package]]
name = "unindent"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
Cargo.toml
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "memu"
version = "0.1.0"
edition = "2024"
[lib]
name = "_core"
# "cdylib" is necessary to produce a shared library for Python to import from.
crate-type = ["cdylib"]
[dependencies]
# "extension-module" tells pyo3 we want to build an extension module (skips linking against libpython.so)
# "abi3-py313" tells pyo3 (and maturin) to build using the stable ABI with minimum Python version 3.13
pyo3 = { version = "0.27.1", features = ["extension-module", "abi3-py313"] }
LICENSE.txt
+194
View File
@@ -0,0 +1,194 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(which shall not include communication that is conspicuously
marked or otherwise designated in writing by the copyright owner
as "Not a Contribution").
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based upon (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and derivative works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control
systems, and issue tracking systems that are managed by, or on behalf
of, the Licensor for the purpose of discussing and improving the Work,
but excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution".
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to use, reproduce, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Work, and to
permit persons to whom the Work is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Work.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, trademark, patent,
attribution and other notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright notice to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Support. You may choose to offer, and to
charge a fee for, warranty, support, indemnity or other liability
obligations and/or rights consistent with this License. However, in
accepting such obligations, You may act only on Your own behalf and
on Your sole responsibility, not on behalf of any other Contributor,
and only if You agree to indemnify, defend, and hold each Contributor
harmless for any liability incurred by, or claims asserted against,
such Contributor by reason of your accepting any such warranty or
support.
END OF TERMS AND CONDITIONS
Copyright 2024 MemU Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
MANIFEST.in
+21
View File
@@ -0,0 +1,21 @@
include README.md
recursive-include memu *.py
prune example
include setup_postgres_env.sh
exclude .env
exclude .env.example
exclude setup.py.backup
exclude */__pycache__/*
exclude *.pyc
exclude *.pyo
exclude .git/*
exclude .github/*
exclude server/*
exclude docs/*
exclude scripts/*
exclude docker-compose.yml
exclude Dockerfile
exclude .dockerignore
exclude PROJECT_RELEASE_SUMMARY.md
exclude Makefile
exclude .pre-commit-config.yaml
Makefile
+22
View File
@@ -0,0 +1,22 @@
.PHONY: install
install:
@echo "🚀 Creating virtual environment using uv"
@uv sync
@uv run pre-commit install
.PHONY: check
check:
@echo "🚀 Checking lock file consistency with 'pyproject.toml'"
@uv lock --locked
@echo "🚀 Linting code: Running pre-commit"
@uv run pre-commit run -a
@echo "🚀 Static type checking: Running mypy"
@uv run mypy
@echo "🚀 Checking for obsolete dependencies: Running deptry"
@uv run deptry src
.PHONY: test
test:
@echo "🚀 Testing code: Running pytest"
@uv run python -m pytest --cov --cov-config=pyproject.toml --cov-report=xml
README.md
+466
View File
@@ -0,0 +1,466 @@
![MemU Banner](assets/banner.png)
<div align="center">
# memU
### Personal memory, stored as files
**Fast retrieval. Higher accuracy. Lower cost.**
[![PyPI version](https://badge.fury.io/py/memu-py.svg)](https://badge.fury.io/py/memu-py)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)
[![Discord](https://img.shields.io/badge/Discord-Join%20Chat-5865F2?logo=discord&logoColor=white)](https://discord.com/invite/hQZntfGsbJ)
[![Twitter](https://img.shields.io/badge/Twitter-Follow-1DA1F2?logo=x&logoColor=white)](https://x.com/memU_ai)
<a href="https://trendshift.io/repositories/17374" target="_blank"><img src="https://trendshift.io/api/badge/repositories/17374" alt="NevaMind-AI%2FmemU | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
**[English](readme/README_en.md) | [中文](readme/README_zh.md) | [日本語](readme/README_ja.md) | [한국어](readme/README_ko.md) | [Español](readme/README_es.md) | [Français](readme/README_fr.md)**
</div>
---
> [!WARNING]
> 🚧 **Under heavy construction** — memU is undergoing a major rework. APIs, CLI commands, and docs may change without notice. Things are expected to stabilize around **July 15, 2026**.
> 🤖 **Agents**: read [`.claude/skills/memu/SKILL.md`](.claude/skills/memu/SKILL.md) and you can `memorize-workspace` and `retrieve-workspace` right away.
memU compiles conversations, documents, code, images, audio, video, URLs, and tool traces into human-readable Markdown files (`INDEX.md`, `MEMORY.md`, `SKILL.md`). Agents traverse the tree and load only what the moment needs — instead of rescanning everything or stuffing long histories into every prompt.
```python
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("What should I know about this user's launch preferences?")
```
Or straight from the terminal — no code:
```bash
npx memu-cli memorize-workspace ./workspace
npx memu-cli retrieve-workspace "What should I know about this user's launch preferences?"
```
That's it. Instead of one giant prompt about a person or their workspace, your agent gets three durable layers it can traverse:
```txt
workspace/
├── INDEX.md ← Index: a map of everything — raw sources and summaries
├── MEMORY.md ← Memory: an overview that links into memory/
├── SKILL.md ← Skill: an overview that links into skill/
├── resource/ ← the raw source files, copied verbatim
├── memory/
│ └── <topic>.md ← one memory file per topic: facts, preferences, goals, events
└── skill/
└── <name>.md ← one skill file per learned pattern, workflow, or mistake to avoid
```
- **Index (`INDEX.md`)** — a map of your memories: what exists, where it came from, and where to look first
- **Memory (`MEMORY.md`)** — personal facts, preferences, goals, events, and decisions extracted from source data
- **Skill (`SKILL.md`)** — **auto-extracted from agent traces and refined on every workspace sync** so the agent improves at recurring tasks
When you sync a folder with `memorize_workspace`, the top-level directory decides the treatment: files under `chat/` become memory, files under `agent/` become skills, and everything else is indexed as workspace context.
Three things make it different from stuffing everything into the prompt:
- **Fast retrieval** — walk to the right folder and rank the right files instead of scanning everything every time.
- **Higher accuracy** — scope by user, task, or session, and trace every item back to the exact conversation, document, image, or log it came from.
- **Lower cost** — retrieve compact, scoped context instead of reinjecting long histories, documents, logs, and media-derived text into every prompt.
- **Yours to inspect** — a human-readable file tree you can audit, edit, scope, and route through your own storage (`inmemory`, `sqlite`, `postgres`) and LLM providers.
---
## ⭐️ Star the repository
<img width="100%" src="https://github.com/NevaMind-AI/memU/blob/main/assets/star.gif" />
If you find memU useful or interesting, a GitHub Star ⭐️ would be greatly appreciated.
---
## ✨ Core Features
| Capability | Description |
|------------|-------------|
| 🗂️ **Multimodal Ingestion** | Write conversations, documents, images, video, audio, URLs, logs, and local files into memory |
| 📁 **Compiled Memory Workspace** | Persist the Index, Skill, and Memory layers — folders (categories), files (items), source artifacts, links, summaries, and embeddings |
| 🧠 **Typed Memory Extraction** | Extract profile, event, knowledge, behavior, skill, and tool memories from raw sources |
| 🛠️ **Self-Evolving Skills** | Auto-extract reusable tool patterns and workflows from agent traces, then merge and refine them on every workspace sync instead of relearning |
| 🧭 **Self-Organizing Folders** | Auto-build categories, links, summaries, and embeddings without manual tagging |
| 🤖 **Agent-Ready Retrieval** | LLM-free `retrieve_workspace()` ranks memory segments, files, and source resources directly |
| 🔄 **Incremental Workspace Sync** | `memorize_workspace()` diffs a folder against a manifest — only changed files are (re)processed, deletions cascade |
| 🧱 **Pluggable Storage** | Use in-memory, SQLite, or Postgres backends with the same repository contracts |
| 🔀 **Profile-Based LLM Routing** | Route chat, embedding, vision, and transcription work through configurable LLM profiles |
| ⌨️ **CLI** | `memu` command (pip) and `npx memu-cli` (npm) — memorize and retrieve from the terminal or CI |
---
## 🎯 Use Cases
Every use case is the same loop: drop sources into a folder, sync it with `memorize_workspace()`, then ask with `retrieve_workspace()`. The sync is incremental (only changed files are reprocessed), and the top-level directory decides the treatment — `chat/` → memory topics, `agent/` → skills, everything else → indexed context.
### 1. **Personal Memory**
*Turn chat logs into user preferences, goals, events, decisions, and relationship context.*
```python
# workspace/chat/*.json — conversation logs become memory topic files
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("What should I remember about this user?")
```
### 2. **Workspace Context for Coding Agents**
*Convert docs, PR notes, logs, and design decisions into reusable project memory.*
```python
# docs, notes, and logs anywhere in the folder are captioned and indexed
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("How should I structure this module?")
```
### 3. **Multimodal Knowledge Layer**
*Extract searchable facts from documents, screenshots, images, videos, and audio notes.*
```python
# modality is inferred per file: .pdf/.docx/.pptx/.xlsx/.html (via MarkItDown —
# pip install 'memu-py[document]'), .png/.jpg, .mp3/.wav, .mp4/.mov, ...
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("What matters for the next research plan?")
```
### 4. **Tool and Agent Learning**
*Turn execution traces into skills that tell future agents what worked and what to avoid.*
```python
# workspace/agent/*.txt — execution traces are distilled into skill files
await service.memorize_workspace(folder="./workspace")
context = await service.retrieve_workspace("Which tools worked for config editing?")
```
---
## 🗂️ Architecture
The compiled workspace is easiest to read as two directions:
- `memorize_workspace()` writes a folder into durable memory files, skill files, resource records, segments, links, and embeddings.
- `retrieve_workspace()` reads those layers directly, ranking segments first and rolling results up to the files and resources an agent should load.
Memory is stored in three representation layers:
| Layer | What it holds | Retrieval Role |
|-------|---------------|----------------|
| **File** (`RecallFile`) | A synthesized memory topic or skill document | The unit returned to the agent — hit segments roll up to their file |
| **Segment** | Fine slices of a file (paragraph lines, skill descriptions) | The embedded search unit — queries rank segments first |
| **Resource** | The raw source artifact with its caption | Recall original context when synthesized summaries are not enough |
`retrieve_workspace()` embeds the query once, ranks segments and resources by similarity, and returns compact context with zero chat-LLM calls.
See [docs/architecture.md](docs/architecture.md) for the runtime view of `MemoryService`, workflow pipelines, storage backends, and LLM routing, and [docs/adr/](docs/adr/README.md) for the decision records behind the layered design.
---
## 🧰 Agent Skills
The repo ships one [Agent Skill](https://docs.claude.com/en/docs/agents-and-tools/agent-skills) — [`.claude/skills/memu/SKILL.md`](.claude/skills/memu/SKILL.md) — that gives Claude Code (and any skills-compatible agent) the workspace pair. The agent decides when to use each direction:
- **memorize** (`memu memorize-workspace`) — "remember this", "sync this folder into memory", finishing work worth persisting
- **retrieve** (`memu retrieve-workspace`) — "what do we know about…", starting a task with likely prior context
It works out of the box inside this repo. To use it in your own project, copy the skill folder into that project's `.claude/skills/` (or `~/.claude/skills/` to enable it everywhere):
```bash
cp -r .claude/skills/memu /path/to/your-project/.claude/skills/
```
The skill locates the CLI automatically (`memu`, `uvx --from memu-py memu`, or `npx memu-cli`) and keeps state in the project-local `./data/memu.sqlite3`, so what one session memorizes the next can retrieve. For LangGraph agents, see the [LangGraph integration](docs/langgraph_integration.md) instead.
---
## 🚀 Quick Start
### Option 1: Cloud Version
👉 **[memu.so](https://memu.so)** — Hosted API for managed ingestion, structured memory, and retrieval
For enterprise deployment: **info@nevamind.ai**
#### Cloud API (v3)
| Base URL | `https://api.memu.so` |
|----------|----------------------|
| Auth | `Authorization: Bearer <token>` |
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v3/memory/memorize` | Ingest raw data and build structured memory |
| `GET` | `/api/v3/memory/memorize/status/{task_id}` | Check processing status |
| `POST` | `/api/v3/memory/categories` | List auto-generated categories |
| `POST` | `/api/v3/memory/retrieve` | Query memory for agent context |
📚 **[Full API Documentation](https://memu.pro/docs#cloud-version)**
---
### Option 2: Self-Hosted
#### Installation
From a clone of this repository:
```bash
uv sync
# or, for the full development setup:
make install
```
To install the published package instead:
```bash
pip install memu-py # library + `memu` CLI
# or from the JS ecosystem (thin launcher over memu-py, uses uvx/pipx automatically):
npx memu-cli --help
```
> **Requirements**: Python 3.13+. The default examples use OpenAI, so set `OPENAI_API_KEY` or pass another provider through `llm_profiles`.
#### Command line
The `memu` command wraps the same service the library exposes. State persists in a local SQLite database (`./data/memu.sqlite3` by default), so memorize in one invocation and retrieve in the next:
```bash
export OPENAI_API_KEY=your_key
memu memorize-workspace ./workspace # diff-sync a folder (alias: memu sync)
memu retrieve-workspace "deploy checklist" # LLM-free embedding retrieval (alias: memu search)
memu export # rebuild the INDEX.md/MEMORY.md/SKILL.md tree
```
Every flag has a `MEMU_*` environment variable (`--provider`/`MEMU_LLM_PROVIDER`, `--model`/`MEMU_CHAT_MODEL`, `--db`/`MEMU_DB`, ...) — run `memu <command> --help` for the full list. `--db` accepts a SQLite path, a `postgres://` DSN, or `:memory:`.
**Run an in-memory smoke script:**
```bash
export OPENAI_API_KEY=your_key
cd tests
uv run python test_inmemory.py
```
**Run with PostgreSQL + pgvector:**
```bash
uv sync --extra postgres
docker run -d --name memu-postgres \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=postgres \
-e POSTGRES_DB=memu \
-p 5432:5432 \
pgvector/pgvector:pg16
export OPENAI_API_KEY=your_key
export POSTGRES_DSN=postgresql+psycopg://postgres:postgres@127.0.0.1:5432/memu
cd tests
uv run python test_postgres.py
```
---
### Custom LLM and Embedding Providers
```python
from memu import MemoryService
service = MemoryService(
llm_profiles={
"default": {
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"api_key": "your_key",
"chat_model": "qwen3-max",
"client_backend": "sdk"
},
"embedding": {
"base_url": "https://api.voyageai.com/v1",
"api_key": "your_key",
"embed_model": "voyage-3.5-lite"
}
},
)
```
---
### OpenRouter Integration
```python
from memu import MemoryService
service = MemoryService(
llm_profiles={
"default": {
"provider": "openrouter",
"client_backend": "httpx",
"base_url": "https://openrouter.ai",
"api_key": "your_key",
"chat_model": "anthropic/claude-3.5-sonnet",
"embed_model": "openai/text-embedding-3-small",
},
},
database_config={"metadata_store": {"provider": "inmemory"}},
)
```
---
## 📖 Core APIs
The primary API pair is `memorize_workspace()` / `retrieve_workspace()` — folder in, ranked context out.
### `memorize_workspace()` — Sync a Folder
<img width="100%" alt="memorize_workspace" src="assets/memorize.png" />
```python
result = await service.memorize_workspace(
folder="./workspace", # scanned recursively; modality inferred per file
user={"user_id": "123"}, # optional scope
)
# Returns the diff plus what changed:
# { "added": [...], "modified": [...], "deleted": [...],
# "resources": [...], "entries": [...], "files": [...] }
```
- Diffs the folder against a sidecar `.memu_manifest.json` — only added/modified files are processed, memory from deleted files is cascade-removed
- Routes by top-level directory: `chat/` → memory files, `agent/` → skill files, everything else → indexed workspace context
- Rebuilds the markdown memory tree (`INDEX.md` / `MEMORY.md` / `SKILL.md`) when `memory_files_config.enabled=True`
---
### `retrieve_workspace()` — Fast, LLM-Free Retrieval
<img width="100%" alt="retrieve_workspace" src="assets/retrieve.png" />
```python
result = await service.retrieve_workspace(
"deploy checklist",
where={"user_id": "123"},
)
# Returns:
# { "segments": [...], # embedded slices ranked by similarity
# "files": [...], # the memory/skill files those segments roll up to
# "resources": [...] } # workspace resources ranked by similarity
```
The query is embedded once and ranked by vector similarity — no intention routing, no query rewriting, no sufficiency checks, zero LLM calls. Use it for high-frequency lookups where latency and cost matter more than deep reasoning.
---
## 💡 Example Workflows
### Always-Learning Assistant
```bash
export OPENAI_API_KEY=your_key
uv run python examples/example_1_conversation_memory.py
```
Automatically extracts preferences, builds relationship models, and surfaces relevant context in future conversations.
### Self-Improving Agent
```bash
uv run python examples/example_2_skill_extraction.py
```
Monitors agent actions, identifies patterns in successes and failures, auto-generates skill guides from experience.
### Multimodal Context Builder
```bash
uv run python examples/example_3_multimodal_memory.py
```
Cross-references text, images, and documents automatically into a unified memory layer.
---
## 📊 Performance
memU achieves **92.09% average accuracy** on the Locomo benchmark across all reasoning tasks.
<img width="100%" alt="benchmark" src="https://github.com/user-attachments/assets/6fec4884-94e5-4058-ad5c-baac3d7e76d9" />
View detailed results: [memU-experiment](https://github.com/NevaMind-AI/memU-experiment)
---
## 🧩 Ecosystem
| Repository | Description |
|------------|-------------|
| **[memU](https://github.com/NevaMind-AI/memU)** | Personal memory as files — fast retrieval, higher accuracy, lower cost |
| **[memU-server](https://github.com/NevaMind-AI/memU-server)** | Backend with real-time sync and webhook triggers |
| **[memU-ui](https://github.com/NevaMind-AI/memU-ui)** | Visual dashboard for browsing and monitoring memory |
**Quick Links:**
- 🚀 [Try MemU Cloud](https://app.memu.so/quick-start)
- 📚 [API Documentation](https://memu.pro/docs)
- 💬 [Discord Community](https://discord.com/invite/hQZntfGsbJ)
---
## 🤝 Partners
<div align="center">
<a href="https://github.com/TEN-framework/ten-framework"><img src="https://avatars.githubusercontent.com/u/113095513?s=200&v=4" alt="Ten" height="40" style="margin: 10px;"></a>
<a href="https://openagents.org"><img src="assets/partners/openagents.png" alt="OpenAgents" height="40" style="margin: 10px;"></a>
<a href="https://github.com/milvus-io/milvus"><img src="https://miro.medium.com/v2/resize:fit:2400/1*-VEGyAgcIBD62XtZWavy8w.png" alt="Milvus" height="40" style="margin: 10px;"></a>
<a href="https://xroute.ai/"><img src="assets/partners/xroute.png" alt="xRoute" height="40" style="margin: 10px;"></a>
<a href="https://jaaz.app/"><img src="assets/partners/jazz.png" alt="Jazz" height="40" style="margin: 10px;"></a>
<a href="https://github.com/Buddie-AI/Buddie"><img src="assets/partners/buddie.png" alt="Buddie" height="40" style="margin: 10px;"></a>
<a href="https://github.com/bytebase/bytebase"><img src="assets/partners/bytebase.png" alt="Bytebase" height="40" style="margin: 10px;"></a>
<a href="https://github.com/LazyAGI/LazyLLM"><img src="assets/partners/LazyLLM.png" alt="LazyLLM" height="40" style="margin: 10px;"></a>
<a href="https://clawdchat.ai/"><img src="assets/partners/Clawdchat.png" alt="Clawdchat" height="40" style="margin: 10px;"></a>
</div>
---
## 🤝 Contributing
```bash
# Fork and clone
git clone https://github.com/YOUR_USERNAME/memU.git
cd memU
# Install dev dependencies
make install
# Run quality checks before submitting
make check
```
See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.
**Prerequisites:** Python 3.13+, [uv](https://github.com/astral-sh/uv), Git
---
## 📄 License
[Apache License 2.0](LICENSE.txt)
---
## 🌍 Community
- **GitHub Issues**: [Report bugs & request features](https://github.com/NevaMind-AI/memU/issues)
- **Discord**: [Join the community](https://discord.com/invite/hQZntfGsbJ)
- **X (Twitter)**: [Follow @memU_ai](https://x.com/memU_ai)
- **Contact**: info@nevamind.ai
---
<div align="center">
⭐ **Star us on GitHub** to get notified about new releases!
</div>
README.wehub.md
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目：`NevaMind-AI/memU`
- 原始仓库：https://github.com/NevaMind-AI/memU
- 导入方式：上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
assets/banner.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 258 KiB

assets/benchmark.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 77 KiB

assets/memUbot.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 285 KiB

assets/memorize.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 889 KiB

assets/partners/Clawdchat.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 470 KiB

assets/partners/LazyLLM.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 248 KiB

assets/partners/buddie.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

assets/partners/bytebase.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

assets/partners/jazz.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 31 KiB

assets/partners/openagents.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

assets/partners/xroute.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

assets/qrcode.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 161 KiB

assets/retrieve.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 840 KiB

assets/star.gif
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.9 MiB

assets/structure.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.6 MiB

assets/usecase/ai_companion-0000.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 19 KiB

assets/usecase/ai_creation-0000.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

assets/usecase/ai_edu-0000.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 22 KiB

assets/usecase/ai_ip-0000.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 556 KiB

assets/usecase/ai_robot-0000.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

assets/usecase/ai_role_play-0000.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 25 KiB

assets/usecase/ai_therapy-0000.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

docs/HACKATHON_ISSUE_DRAFT.md
+68
View File
@@ -0,0 +1,68 @@
# GitHub Issue Draft: Memory Types + Tool Memory
## Title
`[2026NewYearChallenge] Specialized Memory Types with Tool Learning`
---
## Description
### What will this task implement?
This PR enhances MemU's memory type system to support specialized memory structures with type-specific metadata and introduces Tool Memory for agent self-improvement.
**Current State:** MemU has a `memory_type` field with 5 types (profile, event, knowledge, behavior, skill) and uses different LLM prompts to extract each type. However, after extraction, all memories share the same storage schema - just `summary` and `embedding`. There's no type-specific metadata, no type-aware retrieval, and no way for agents to learn from their tool usage.
**Enhancement:** Extend the memory system to support:
- Type-specific metadata fields (e.g., `when_to_use` for smarter retrieval)
- Tool Memory type for tracking tool execution history
- Tool usage statistics for agent self-improvement
- Type-aware retrieval filtering
**Key Benefits:**
- Agents can learn from their own tool usage patterns
- Smarter retrieval based on memory context
- Foundation for agents that improve over time
- Better alignment with agentic application needs
---
## Requirements
- [x] Type-specific metadata schema with `when_to_use` field
- [x] Tool Memory implementation with execution tracking
- [x] Tool statistics (success_rate, avg_time_cost, avg_score)
- [ ] Type-aware retrieval filtering
- [x] Tests for Tool Memory CRUD and statistics
- [ ] Documentation and usage examples
---
## Review Criteria
- Correctness: All tests pass, no regressions
- Quality: Clean code, follows existing patterns
- DX: Clear documentation and examples
- Impact: Enables agent self-improvement use cases
---
## Implementation Summary
### Files Modified:
1. `src/memu/database/models.py` - Added `ToolCallResult` model, extended `MemoryItem` with `when_to_use`, `metadata`, `tool_calls` fields
2. `src/memu/database/repositories/memory_item.py` - Updated interface with new fields
3. `src/memu/database/inmemory/repositories/memory_item_repo.py` - Updated implementation
4. `src/memu/database/postgres/repositories/memory_item_repo.py` - Updated implementation
5. `src/memu/database/postgres/models.py` - Added JSON columns for new fields
6. `src/memu/prompts/memory_type/__init__.py` - Added tool type
7. `src/memu/prompts/memory_type/tool.py` - New prompt for tool memory extraction
### Files Added:
1. `tests/test_tool_memory.py` - 14 unit tests for Tool Memory feature
---
## Notes
This builds on MemU's existing memory type foundation while adding the specialized structures needed for agentic applications. The Tool Memory feature is particularly valuable for agents that need to learn which tools work best in different situations.
docs/HACKATHON_MAD_COMBOS.md
+659
View File
@@ -0,0 +1,659 @@
# 🔥🔥🔥 MAD COMBO OPTIONS FOR MEMU HACKATHON 🔥🔥🔥
> **Goal:** Implement high-impact features that MemU is missing, sourced from competitor analysis of 7 memory repos (memoripy, memlayer, ReMe, memX, memphora-sdk, MemOS, memor).
---
## 📋 COMPLETE FEATURE GAP ANALYSIS
### FEATURES MEMU IS MISSING (Deep Scan Results)
#### FROM MEMORIPY:
- ❌ `access_counts[]` - Track how often each memory is accessed
- ❌ `timestamps[]` - Track when memory was created/last accessed
- ❌ `decay_factor` - Exponential time-based decay: `np.exp(-decay_rate * time_diff)`
- ❌ `reinforcement_factor` - Log-scaled access boost: `np.log1p(access_count)`
- ❌ `adjusted_similarity` - `similarity * decay_factor * reinforcement_factor`
- ❌ Short-term → Long-term memory promotion (when `access_count > 10`)
- ❌ `nx.Graph()` concept associations (NetworkX graph)
- ❌ `spreading_activation()` - Spread activation through concept graph
- ❌ `cluster_interactions()` - KMeans clustering for hierarchical memory
- ❌ `semantic_memory` clusters - Retrieve from semantic memory clusters
#### FROM MEMLAYER:
- ❌ `SalienceGate` - Filter what's worth saving vs noise
- ❌ `SalienceMode.LOCAL` - Local ML model for salience
- ❌ `SalienceMode.ONLINE` - OpenAI API for salience
- ❌ `SalienceMode.LIGHTWEIGHT` - Keyword-based salience (no embeddings)
- ❌ `SALIENT_PROTOTYPES` / `NON_SALIENT_PROTOTYPES` - Prototype sentences
- ❌ `is_worth_saving()` - Determine if text should be saved
- ❌ `CurationService` - Background memory decay/expiration
- ❌ `_calculate_relevance()` - Score based on age, recency, attention
- ❌ Auto-archive low-relevance memories
- ❌ Auto-delete expired memories (`expiration_timestamp`)
- ❌ `SchedulerService` - Background task scheduler
- ❌ `get_due_tasks_for_user()` - Check for pending scheduled tasks
- ❌ `ConsolidationService` - Background knowledge extraction
- ❌ `analyze_and_extract_knowledge()` - Extract facts, entities, relationships
- ❌ `NetworkXStorage` - Graph storage for entities/relationships
- ❌ `add_entity()` / `add_relationship()` - Knowledge graph operations
- ❌ `get_subgraph_context()` - Graph traversal for context
- ❌ `find_matching_nodes()` - Fuzzy entity matching
- ❌ `_find_canonical_entity()` - Entity deduplication
- ❌ `_merge_entity_nodes()` - Merge duplicate entities
- ❌ `importance_score` / `expiration_timestamp` metadata
- ❌ `track_memory_access()` - Track when memories are accessed
- ❌ Task reminders system (`add_task`, `get_pending_tasks`, `update_task_status`)
#### FROM REME:
- ❌ `UpdateMemoryFreqOp` - Increment frequency counter on recall
- ❌ `metadata["freq"]` - Frequency counter in metadata
- ❌ `UpdateMemoryUtilityOp` - Increment utility score when useful
- ❌ `metadata["utility"]` - Utility score in metadata
- ❌ `DeleteMemoryOp` - Delete based on freq/utility thresholds
- ❌ `utility/freq < threshold` pruning - Prune low-value memories
- ❌ **MEMORY TYPES:**
- ❌ `TaskMemory` - Task-related information
- ❌ `PersonalMemory` - Personal info with `target` and `reflection_subject`
- ❌ `ToolMemory` - Tool call execution history
- ❌ `ToolCallResult` - Record tool execution results with hash deduplication
- ❌ `MemoryDeduplicationOp` - Remove duplicate memories using embedding similarity
- ❌ `WorkingMemory` operations:
- ❌ `MessageCompressOp` - LLM-based compression for long conversations
- ❌ `MessageCompactOp` - Compact verbose tool messages
- ❌ `MessageOffloadOp` - Orchestrate compaction + compression
- ❌ `WorkingSummaryMode.COMPACT/COMPRESS/AUTO`
- ❌ `UpdateMemory` tool - Update/edit existing memories
- ❌ `session_memory_id` tracking - Track memories per session
- ❌ Tool memory statistics (`avg_token_cost`, `success_rate`, `avg_time_cost`, `avg_score`)
#### FROM MEMX:
- ❌ `pubsub.py` - Real-time pub/sub system
- ❌ `subscribe(key, websocket)` - WebSocket subscriptions
- ❌ `publish(key, payload)` - Broadcast updates to subscribers
- ❌ `set_value()` with timestamps - Last-write-wins with timestamps
- ❌ Redis-backed shared memory - Multi-agent shared state
- ❌ `register_schema()` / `validate_schema()` - JSON schema validation
#### FROM MEMOS (MemOS):
- ❌ **Memory Scheduler System:**
- ❌ `BaseScheduler` - Full task scheduling infrastructure
- ❌ `SchedulerDispatcher` - Parallel task dispatch
- ❌ `ScheduleTaskQueue` - Priority task queue
- ❌ `TaskStatusTracker` - Track task status in Redis
- ❌ `TaskPriorityLevel` - Priority levels for tasks
- ❌ `MemoryMonitorItem` - Monitor memory with importance scores
- ❌ `replace_working_memory()` - Replace working memory after reranking
- ❌ `update_activation_memory()` - Update activation memory periodically
- ❌ `transform_working_memories_to_monitors()` - Convert memories to monitors
- ❌ `visibility` field - Public/private memory visibility
- ❌ `confidence` score - Confidence level for memories
- ❌ `status` field (activated/archived) - Memory activation status
- ❌ `tags` field - Memory tagging system
- ❌ `entities` extraction - Extract entities from memories
#### FROM MEMPHORA-SDK:
- ❌ `store_shared()` - Store shared memory for groups
- ❌ Multi-agent crew memory - Shared memory for agent crews
- ❌ Per-agent namespaces - Isolated memory per agent
- ❌ Framework integrations (AutoGen, CrewAI, LangChain, LlamaIndex)
---
---
## 🏆 COMBO 1: "INTELLIGENT MEMORY LIFECYCLE"
**Theme:** Memory that learns, ages, and self-curates like human memory
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Decay & Reinforcement | memoripy | 3 pts | LOW |
| Frequency & Utility Tracking | ReMe | 3 pts | LOW |
| Auto-Pruning Low-Value Memories | ReMe | 3 pts | LOW |
**Total: 9 pts | LOW-MEDIUM effort**
### Why it's MAD:
```
Memory accessed often → gets STRONGER (reinforcement)
Memory ignored → gets WEAKER (decay)
Memory with low utility/freq ratio → gets DELETED automatically
Result: Self-healing, self-optimizing memory that mimics human forgetting!
```
### The Pitch:
> "MemU now has HUMAN-LIKE memory - it remembers what matters and forgets what doesn't!"
### Technical Implementation:
```python
# Decay formula (from memoripy)
decay_factor = np.exp(-decay_rate * time_diff)
reinforcement_factor = np.log1p(access_count)
adjusted_similarity = similarity * decay_factor * reinforcement_factor
# Pruning logic (from ReMe)
if freq >= freq_threshold:
if utility / freq < utility_threshold:
delete_memory(memory_id)
```
---
## 🏆 COMBO 2: "SMART MEMORY GATE"
**Theme:** Don't save garbage, only save gold
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Salience Filtering | memlayer | 3-5 pts | MEDIUM |
| Decay & Reinforcement | memoripy | 3 pts | LOW |
| Background Curation Service | memlayer | 3 pts | MEDIUM |
**Total: 9-11 pts | MEDIUM effort**
### Why it's MAD:
```
INPUT: "Hello!" → BLOCKED (not salient)
INPUT: "My name is John, I work at Google" → SAVED (salient)
BACKGROUND: Old unused memories → AUTO-ARCHIVED
RETRIEVAL: Frequently accessed → BOOSTED
Result: Clean, high-quality memory that doesn't bloat!
```
### The Pitch:
> "MemU now has a BOUNCER - only important memories get in, garbage stays out!"
### Technical Implementation:
```python
# Salience Gate (from memlayer)
class SalienceGate:
SALIENT_PROTOTYPES = ["My name is...", "I work at...", "The deadline is..."]
NON_SALIENT_PROTOTYPES = ["Hello", "Thanks", "Okay", "Got it"]
def is_worth_saving(self, text: str) -> bool:
# Compare similarity to salient vs non-salient prototypes
salient_score = max_similarity(text, SALIENT_PROTOTYPES)
non_salient_score = max_similarity(text, NON_SALIENT_PROTOTYPES)
return salient_score > (non_salient_score + threshold)
```
---
## 🏆 COMBO 3: "KNOWLEDGE BRAIN"
**Theme:** Memory that understands relationships
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Knowledge Graph | memlayer | 5 pts | HIGH |
| Entity Extraction | memlayer | 3 pts | MEDIUM |
| Graph Traversal Retrieval | memlayer | 3 pts | MEDIUM |
**Total: 11 pts | HIGH effort**
### Why it's MAD:
```
INPUT: "John works at Google. Sarah is John's wife."
GRAPH:
John --[works_at]--> Google
John --[married_to]--> Sarah
QUERY: "Who is related to Google?"
RESULT: John (works there), Sarah (married to John who works there)
Result: Memory that REASONS about relationships!
```
### The Pitch:
> "MemU now has a BRAIN - it understands how things connect!"
### Technical Implementation:
```python
# Knowledge Graph (from memlayer)
import networkx as nx
class KnowledgeGraph:
def __init__(self):
self.graph = nx.Graph()
def add_entity(self, name: str, node_type: str):
self.graph.add_node(name, type=node_type)
def add_relationship(self, subject: str, predicate: str, obj: str):
self.graph.add_edge(subject, obj, relation=predicate)
def get_subgraph_context(self, entity: str, depth: int = 2):
# Traverse graph for related entities
return nx.ego_graph(self.graph, entity, radius=depth)
```
---
## 🏆 COMBO 4: "MEMORY EVOLUTION" ⭐ TOP PICK
**Theme:** Memory that evolves and improves itself
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Salience Gate (LIGHTWEIGHT mode) | memlayer | 3 pts | LOW |
| Decay & Reinforcement | memoripy | 3 pts | LOW |
| Frequency & Utility | ReMe | 3 pts | LOW |
| Auto-Pruning | ReMe | 3 pts | LOW |
**Total: 12 pts | LOW-MEDIUM effort**
### Why it's MAD:
```
STAGE 1: Salience Gate filters noise at INPUT
STAGE 2: Decay/Reinforcement adjusts scores at RETRIEVAL
STAGE 3: Frequency/Utility tracks VALUE over time
STAGE 4: Auto-Pruning DELETES low-value memories
Result: FULL LIFECYCLE MANAGEMENT - from birth to death!
```
### The Pitch:
> "MemU memories now have a LIFECYCLE - they're born, they grow, they age, they die!"
### Memory Lifecycle Diagram:
```
┌─────────────────────────────────────────────────────────────────┐
│ MEMORY LIFECYCLE │
├─────────────────────────────────────────────────────────────────┤
│ │
│ INPUT ──► [SALIENCE GATE] ──► SAVE or REJECT │
│ │ │
│ ▼ │
│ ┌─────────┐ │
│ │ MEMORY │ ◄── access_count, last_accessed │
│ │ ITEM │ ◄── freq, utility, salience_score │
│ └────┬────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ ▼ ▼ │
│ [RETRIEVAL] [BACKGROUND] │
│ │ │ │
│ decay_factor auto_prune() │
│ reinforcement if utility/freq < threshold │
│ │ │ │
│ ▼ ▼ │
│ BOOSTED SCORE DELETE MEMORY │
│ │
└─────────────────────────────────────────────────────────────────┘
```
---
## 📊 COMPARISON TABLE
| Combo | Points | Effort | WOW Factor | Complexity | Recommendation |
|-------|--------|--------|------------|------------|----------------|
| 1. Intelligent Lifecycle | 9 | LOW | ⭐⭐⭐⭐ | LOW | ✅ SAFE BET |
| 2. Smart Gate | 9-11 | MEDIUM | ⭐⭐⭐⭐ | MEDIUM | ✅ GOOD |
| 3. Knowledge Brain | 11 | HIGH | ⭐⭐⭐⭐⭐ | HIGH | ⚠️ RISKY |
| 4. Memory Evolution | 12 | LOW-MED | ⭐⭐⭐⭐⭐ | MEDIUM | 🏆 **BEST COMBO** |
---
## 🎯 RECOMMENDATION: COMBO 4 "MEMORY EVOLUTION"
### Why This Combo Wins:
1. **12 points** - highest point potential
2. **LOW-MEDIUM effort** - achievable in hackathon timeframe
3. **4 features that SYNERGIZE** - each builds on the other
4. **UNIQUE story** - "memory lifecycle" is a compelling narrative
5. **Easy to demo** - show memory being filtered, decaying, getting pruned
### Implementation Order:
```
Step 1: Add fields to MemoryItem model
- access_count: int = 0
- last_accessed: datetime
- freq: int = 0
- utility: int = 0
- salience_score: float = 0.0
Step 2: Implement lightweight salience gate (keyword-based, no ML)
- SALIENT_KEYWORDS list
- NON_SALIENT_KEYWORDS list
- is_worth_saving() function
Step 3: Implement decay-aware retrieval
- Modify cosine_topk() to apply decay formula
- Update access_count and last_accessed on retrieval
Step 4: Implement frequency/utility tracking
- Increment freq on every retrieval
- Increment utility when memory contributes to response
Step 5: Implement auto-pruning
- Background check for low utility/freq ratio
- Delete memories below threshold
```
### Files to Modify:
```
memU/src/memu/database/models.py # Add new fields
memU/src/memu/database/inmemory/vector.py # Decay-aware retrieval
memU/src/memu/app/memorize.py # Salience gate
memU/src/memu/app/retrieve.py # Frequency/utility tracking
memU/src/memu/app/service.py # Auto-pruning service
```
---
## 📚 Reference Implementations
### From memoripy (Decay & Reinforcement):
- File: `prospects/memoripy/memoripy/memory_store.py`
- Key functions: `retrieve()`, `classify_memory()`
### From memlayer (Salience Gate):
- File: `prospects/memlayer/memlayer/ml_gate.py`
- Key class: `SalienceGate`
### From ReMe (Frequency & Utility):
- Files:
- `prospects/ReMe/reme_ai/vector_store/update_memory_freq_op.py`
- `prospects/ReMe/reme_ai/vector_store/update_memory_utility_op.py`
- `prospects/ReMe/reme_ai/vector_store/delete_memory_op.py`
---
## 🚀 Ready to Implement?
Choose your combo and let's build! 🔥
## 🏆 UPDATED MAD COMBOS (After Deep Scan)
---
## 🏆 COMBO 1: "INTELLIGENT MEMORY LIFECYCLE"
**Theme:** Memory that learns, ages, and self-curates like human memory
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Decay & Reinforcement | memoripy | 3 pts | LOW |
| Frequency & Utility Tracking | ReMe | 3 pts | LOW |
| Auto-Pruning Low-Value Memories | ReMe | 3 pts | LOW |
**Total: 9 pts | LOW-MEDIUM effort**
### Why it's MAD:
```
Memory accessed often → gets STRONGER (reinforcement)
Memory ignored → gets WEAKER (decay)
Memory with low utility/freq ratio → gets DELETED automatically
Result: Self-healing, self-optimizing memory that mimics human forgetting!
```
### The Pitch:
> "MemU now has HUMAN-LIKE memory - it remembers what matters and forgets what doesn't!"
---
## 🏆 COMBO 2: "SMART MEMORY GATE"
**Theme:** Don't save garbage, only save gold
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Salience Filtering | memlayer | 3-5 pts | MEDIUM |
| Decay & Reinforcement | memoripy | 3 pts | LOW |
| Background Curation Service | memlayer | 3 pts | MEDIUM |
**Total: 9-11 pts | MEDIUM effort**
### Why it's MAD:
```
INPUT: "Hello!" → BLOCKED (not salient)
INPUT: "My name is John, I work at Google" → SAVED (salient)
BACKGROUND: Old unused memories → AUTO-ARCHIVED
RETRIEVAL: Frequently accessed → BOOSTED
Result: Clean, high-quality memory that doesn't bloat!
```
### The Pitch:
> "MemU now has a BOUNCER - only important memories get in, garbage stays out!"
---
## 🏆 COMBO 3: "KNOWLEDGE BRAIN"
**Theme:** Memory that understands relationships
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Knowledge Graph (NetworkX) | memlayer | 5 pts | HIGH |
| Entity Extraction & Deduplication | memlayer | 3 pts | MEDIUM |
| Graph Traversal Retrieval | memlayer | 3 pts | MEDIUM |
**Total: 11 pts | HIGH effort**
### Why it's MAD:
```
INPUT: "John works at Google. Sarah is John's wife."
GRAPH:
John --[works_at]--> Google
John --[married_to]--> Sarah
QUERY: "Who is related to Google?"
RESULT: John (works there), Sarah (married to John who works there)
Result: Memory that REASONS about relationships!
```
### The Pitch:
> "MemU now has a BRAIN - it understands how things connect!"
---
## 🏆 COMBO 4: "MEMORY EVOLUTION" ⭐ TOP PICK
**Theme:** Memory that evolves and improves itself
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| Salience Gate (LIGHTWEIGHT mode) | memlayer | 3 pts | LOW |
| Decay & Reinforcement | memoripy | 3 pts | LOW |
| Frequency & Utility | ReMe | 3 pts | LOW |
| Auto-Pruning | ReMe | 3 pts | LOW |
**Total: 12 pts | LOW-MEDIUM effort**
### Why it's MAD:
```
STAGE 1: Salience Gate filters noise at INPUT
STAGE 2: Decay/Reinforcement adjusts scores at RETRIEVAL
STAGE 3: Frequency/Utility tracks VALUE over time
STAGE 4: Auto-Pruning DELETES low-value memories
Result: FULL LIFECYCLE MANAGEMENT - from birth to death!
```
### The Pitch:
> "MemU memories now have a LIFECYCLE - they're born, they grow, they age, they die!"
---
## 🏆 COMBO 5: "MEMORY TYPES" (NEW!)
**Theme:** Different memory types for different purposes
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| TaskMemory type | ReMe | 3 pts | MEDIUM |
| PersonalMemory type | ReMe | 3 pts | MEDIUM |
| ToolMemory type | ReMe | 5 pts | HIGH |
| Memory Deduplication | ReMe | 3 pts | MEDIUM |
**Total: 14 pts | MEDIUM-HIGH effort**
### Why it's MAD:
```
TaskMemory: "Complete the report by Friday"
- when_to_use: "When user asks about deadlines"
- content: "Report due Friday"
PersonalMemory: "User prefers dark mode"
- target: "user_preferences"
- reflection_subject: "ui_settings"
ToolMemory: "file_reader tool usage history"
- tool_call_results: [...]
- statistics: {avg_token_cost, success_rate, avg_score}
Result: Specialized memory for specialized tasks!
```
### The Pitch:
> "MemU now has SPECIALIZED MEMORY - task memory, personal memory, tool memory!"
---
## 🏆 COMBO 6: "WORKING MEMORY COMPRESSION" (NEW!)
**Theme:** Handle long conversations without losing context
| Component | Source | Points | Effort |
|-----------|--------|--------|--------|
| MessageCompressOp | ReMe | 3 pts | MEDIUM |
| MessageCompactOp | ReMe | 3 pts | MEDIUM |
| MessageOffloadOp | ReMe | 3 pts | MEDIUM |
**Total: 9 pts | MEDIUM effort**
### Why it's MAD:
```
LONG CONVERSATION (50k tokens) → COMPRESS → STATE SNAPSHOT (5k tokens)
Modes:
- COMPACT: Store full content externally, keep short previews
- COMPRESS: LLM-based compression to generate dense summaries
- AUTO: Compact first, then compress if needed
Result: Handle infinite conversations without context overflow!
```
### The Pitch:
> "MemU now handles INFINITE conversations - compress, compact, never forget!"
---
## 📊 UPDATED COMPARISON TABLE
| Combo | Points | Effort | WOW Factor | Complexity | Recommendation |
|-------|--------|--------|------------|------------|----------------|
| 1. Intelligent Lifecycle | 9 | LOW | ⭐⭐⭐⭐ | LOW | ✅ SAFE BET |
| 2. Smart Gate | 9-11 | MEDIUM | ⭐⭐⭐⭐ | MEDIUM | ✅ GOOD |
| 3. Knowledge Brain | 11 | HIGH | ⭐⭐⭐⭐⭐ | HIGH | ⚠️ RISKY |
| 4. Memory Evolution | 12 | LOW-MED | ⭐⭐⭐⭐⭐ | MEDIUM | 🏆 **BEST COMBO** |
| 5. Memory Types | 14 | MED-HIGH | ⭐⭐⭐⭐⭐ | HIGH | 🔥 HIGH POINTS |
| 6. Working Memory | 9 | MEDIUM | ⭐⭐⭐⭐ | MEDIUM | ✅ GOOD |
---
## 🎯 FINAL RECOMMENDATION
### For MAX POINTS with REASONABLE EFFORT: **COMBO 4 "MEMORY EVOLUTION"**
**Why?**
1. **12 points** - highest point potential for effort
2. **LOW-MEDIUM effort** - achievable in hackathon timeframe
3. **4 features that SYNERGIZE** - each builds on the other
4. **UNIQUE story** - "memory lifecycle" is a compelling narrative
5. **Easy to demo** - show memory being filtered, decaying, getting pruned
### Implementation Order:
```
Step 1: Add fields to MemoryItem model
- access_count: int = 0
- last_accessed: datetime
- freq: int = 0
- utility: int = 0
- salience_score: float = 0.0
Step 2: Implement lightweight salience gate (keyword-based, no ML)
- SALIENT_KEYWORDS list
- NON_SALIENT_KEYWORDS list
- is_worth_saving() function
Step 3: Implement decay-aware retrieval
- Modify cosine_topk() to apply decay formula
- Update access_count and last_accessed on retrieval
Step 4: Implement frequency/utility tracking
- Increment freq on every retrieval
- Increment utility when memory contributes to response
Step 5: Implement auto-pruning
- Background check for low utility/freq ratio
- Delete memories below threshold
```
### Files to Modify:
```
memU/src/memu/database/models.py # Add new fields
memU/src/memu/database/inmemory/vector.py # Decay-aware retrieval
memU/src/memu/app/memorize.py # Salience gate
memU/src/memu/app/retrieve.py # Frequency/utility tracking
memU/src/memu/app/service.py # Auto-pruning service
```
---
## 📚 Reference Implementations
### From memoripy (Decay & Reinforcement):
- File: `prospects/memoripy/memoripy/memory_store.py`
- Key functions: `retrieve()`, `classify_memory()`
### From memlayer (Salience Gate + Knowledge Graph):
- File: `prospects/memlayer/memlayer/ml_gate.py` - SalienceGate
- File: `prospects/memlayer/memlayer/storage/networkx.py` - Knowledge Graph
- File: `prospects/memlayer/memlayer/services.py` - CurationService
### From ReMe (Frequency & Utility + Memory Types):
- File: `prospects/ReMe/reme_ai/vector_store/update_memory_freq_op.py`
- File: `prospects/ReMe/reme_ai/vector_store/update_memory_utility_op.py`
- File: `prospects/ReMe/reme_ai/vector_store/delete_memory_op.py`
- File: `prospects/ReMe/reme_ai/schema/memory.py` - Memory types
- File: `prospects/ReMe/reme_ai/summary/task/memory_deduplication_op.py`
- File: `prospects/ReMe/reme_ai/summary/working/` - Working memory ops
### From MemOS (Scheduler):
- File: `prospects/MemOS/src/memos/mem_scheduler/base_scheduler.py`
---
## 🚀 Ready to Implement?
Choose your combo and let's build! 🔥
docs/adr/0001-workflow-pipeline-architecture.md
+35
View File
@@ -0,0 +1,35 @@
# ADR 0001: Use Workflow Pipelines for Core Operations
- Status: Accepted
- Date: 2026-02-24
## Context
memU has multiple high-level operations (`memorize`, `retrieve`, and CRUD/patch operations) that each require multi-stage execution, LLM calls, storage writes, and optional short-circuit behavior.
A single monolithic function per operation would make these flows hard to extend, observe, and customize.
## Decision
Model each core operation as a named workflow pipeline composed of ordered `WorkflowStep` units.
- Register pipelines centrally in `MemoryService` via `PipelineManager`
- Validate required/produced state keys at pipeline registration/mutation time
- Execute through a `WorkflowRunner` abstraction (`local` by default)
- Support runtime customization by step-level config and structural mutation (insert/replace/remove)
- Provide before/after/on_error step interceptors for instrumentation and control
## Consequences
Positive:
- uniform execution model across memorize/retrieve/CRUD
- explicit, inspectable stage boundaries
- extension points for custom runners and step customization
- easier interception and observability around stage execution
Negative:
- dict-based workflow state relies on key naming discipline
- pipeline mutation can increase behavioral variance between deployments
- more framework code compared to direct function calls
docs/adr/0002-pluggable-storage-and-vector-strategy.md
+42
View File
@@ -0,0 +1,42 @@
# ADR 0002: Use Pluggable Storage with Backend-Specific Vector Search
- Status: Accepted
- Date: 2026-02-24
## Context
memU must support:
- zero-setup local development
- lightweight persisted deployments
- production deployments that need scalable vector similarity
No single storage engine fits all three cases.
## Decision
Adopt repository-based storage abstraction behind a `Database` protocol, with selectable providers:
- `inmemory`: in-process state, brute-force similarity
- `sqlite`: file-based persistence, embeddings stored as JSON text, brute-force similarity
- `postgres`: SQL persistence, pgvector-enabled similarity when configured
Vector behavior is backend-aware:
- brute-force cosine search remains available for portability
- Postgres can use pgvector distance queries when vector support is enabled
- salience ranking (reinforcement/recency-aware) uses local scoring logic
## Consequences
Positive:
- one service API works across local and production footprints
- clear backend contracts through repository interfaces
- predictable fallback behavior when native vector index is unavailable
Negative:
- duplicate repository logic across backends
- behavior/performance differences between providers
- SQLite and in-memory vector search does not scale as well as indexed pgvector
docs/adr/0003-user-scope-in-data-model.md
+32
View File
@@ -0,0 +1,32 @@
# ADR 0003: Model User Scope as First-Class Fields on Memory Records
- Status: Accepted
- Date: 2026-02-24
## Context
memU retrieval and writes need scoped operation (for example per `user_id`, `agent_id`, or session) for multi-user and multi-agent scenarios.
Keeping scope outside stored records would force ad-hoc filtering logic and weaken data isolation.
## Decision
Embed scope directly into all persisted entities by merging a configurable `UserConfig.model` with core record models.
- Scope fields are part of resource/category/item/relation models
- Repositories accept `user_data` on writes and `where` filters on reads
- API-level `where` filters are validated against configured scope fields before execution
## Consequences
Positive:
- consistent filtering model across memorize/retrieve/CRUD APIs
- backend-independent scoping semantics
- supports multi-tenant and multi-agent patterns without separate storage stacks
Negative:
- schema/model generation complexity increases
- schema and index shape can vary by chosen scope model
- callers must keep `where` and `user` payloads aligned with configured scope fields
docs/adr/0004-workspace-memorize-and-memory-file-system.md
+79
View File
@@ -0,0 +1,79 @@
# ADR 0004: Add Workspace Memorize Without Touching Single-File `memorize`
- Status: Accepted
- Date: 2026-06-22
## Context
A feature branch (`feat/memory-fs-synthesis`) added a markdown "memory file system"
export, LLM synthesis of `MEMORY.md`/`SKILL.md`, an incremental update path, an
embedding-decoupling refactor, and a folder/workspace memorize entry point.
In its original arrangement that work mutates the existing single-file
`memorize(resource_url, modality)` entry point — and ultimately rewrites it to
take a `folder` argument. We want the existing `memorize` contract to stay
untouched and to introduce the directory behavior as a new, additive entry point
instead.
## Decision
Land the work as seven new, re-scoped commits:
1. **Folder backbone** — add `blob/folder.py` (scan / manifest / diff) and a new
`memorize_workspace(folder)` entry that diff-syncs a directory
(add/modify/delete with cascade-delete of stale memory) by looping over the
**unchanged** `memorize`. No export yet.
2. **Export** — add the memory-file-system export and wire a best-effort full
re-export into `memorize_workspace`.
3. **Synthesis + update path** — add LLM synthesis of `MEMORY.md`/`SKILL.md` and
the stateful initialize-vs-incremental-update path. The update-on-memorize hook
attaches to `memorize_workspace`, not `memorize`.
4. **Finish `skill/` decoupling** in `memory_fs`: the `skill/` tree is always
synthesized from descriptions, never derived from extracted skill-type memory
items. (Unblocks the step-5 restructure.)
5. **Export-tree restructure** — restructure the export tree into `resource/` /
`memory/` / `skill/` with root indexes (`INDEX.md`/`MEMORY.md`/`SKILL.md`) and
bytes-aware diffing. Only the directory restructure is taken; `skill/` stays
synthesized from descriptions (the step-4 design), so the
synthesizer/`_build_memory_files` stay as they were and only `exporter.py` is
reworked.
6. **Flow + protocol cleanup** — split the oversized flows
(`retrieve_llm.py` / `memorize_parse.py`), narrow the `Database` protocol to
repositories only, and remove the Rust scaffolding.
7. **Decouple embedding** — route vectorization through the dedicated
`memu.embedding` clients and drop `embed()` from the chat clients. Kept strictly
last and isolated so it is independently revertable.
`memorize(resource_url, modality)` remains byte-for-byte unchanged throughout.
Embedding can be last because the decouple only swaps the implementation behind
already-stable call sites (`_get_step_embedding_client` / `.embed()`); nothing in
steps 1–6 depends on the new embedding clients.
## Notes
- The export, synthesizer, and update-path work do not touch `memorize`, so those
steps are clean. Only the initialize-vs-update hook is relocated from `memorize`
to `memorize_workspace`.
- The source branch oscillated on where `skill/` content comes from — first
decoupling it to LLM synthesis, then re-coupling it to extracted skill-type
memory items. We keep the synthesis design (step 4) and take only the directory
restructure (step 5), so `exporter.py` is reworked by hand rather than applied
wholesale.
- Cleanup concerns that were originally bundled together are split so the embedding
decouple lands alone in step 7, isolated and easy to revert.
## Consequences
Positive:
- existing `memorize` callers are unaffected; directory ingestion is purely additive
- embedding decoupling is isolated in its own commit (step 7) and easy to revert
- each new commit is independently reviewable
Negative:
- the work is split and reordered relative to the source branch, so the new history
is a re-scoped reconstruction rather than a replay of it
- `memorize_workspace` and `memorize` share the single-file core but are separate
entry points to maintain
docs/adr/0005-dedicated-embedding-package.md
+88
View File
@@ -0,0 +1,88 @@
# ADR 0005: Extract Embedding into a Dedicated Package, Fully Decoupled from Chat Clients
- Status: Accepted
- Date: 2026-06-24
## Context
Embedding (vectorization) was coupled into the text/chat LLM clients: `embed()`
lived on `OpenAIClient`, `HTTPLLMClient` (which carried its own inline
`_EmbeddingBackend` classes), `AnthropicClient` (raising), and `LazyLLMClient`.
The service routed vectorization through an LLM profile (`_get_step_embedding_client`
returned a chat client), and some retrieval paths reused a single client for both
chat and `embed()`.
This had several problems:
- embedding providers were limited to whatever the chat client happened to
support, so embedding-only providers (Jina, Voyage) had no clean home;
- embedding payload/parse logic was duplicated (an orphaned `memu.embedding`
module plus the inline backends inside `HTTPLLMClient`);
- the `vision` capability already had a clean sibling package (`memu.vlm`, see the
LLM/VLM gateway refactor), so embedding was the odd one out;
- mixing chat and embedding on one client blurred capability boundaries and made
per-capability provider/transport selection impossible.
ADR 0004 deliberately deferred this as an isolated, independently revertable step
("Decouple embedding").
## Decision
Make embedding a first-class capability in its own package, `memu.embedding`,
structured identically to `memu.llm` and `memu.vlm`, and remove embedding from the
chat clients entirely.
Package layout (`memu.embedding`):
- `backends/`: per-provider request/response shapes — `openai`, `jina`, `voyage`,
`doubao` (incl. multimodal), `openrouter`; the HTTP client falls back to an
OpenAI-compatible backend for any other provider.
- `http_client` / `openai_sdk`: transport clients; `embed()` returns
`(vectors, raw_response)` so usage metadata is preserved.
- `gateway.build_embedding_client(cfg)`: dispatch on `client_backend`
(`sdk` / `httpx` / `lazyllm_backend`; `anthropic` raises — Claude has no
embeddings API).
- `defaults`: per-provider default models and embedding-only endpoints.
Configuration and wiring:
- New `EmbeddingConfig` / `EmbeddingProfilesConfig`, sibling to `LLMConfig` /
`VLMConfig`. `MemoryService` accepts an optional `embedding_profiles`; when
omitted, profiles are derived from the LLM profiles via
`embedding_config_from_llm`, so existing configs keep working.
- `MemoryService` builds/caches embedding clients per profile and wraps them with
the same `LLMClientWrapper`, so interceptors and usage tracking are identical to
chat/vision.
- All vectorization call sites (query embedding, category/item embedding, RAG
ranking) go through `_get_step_embedding_client` / `_get_embedding_client`.
Decoupling the chat clients:
- `OpenAIClient`, `HTTPLLMClient`, and `AnthropicClient` no longer expose
`embed()`; `HTTPLLMClient` no longer carries inline embedding backends.
- `LazyLLMClient` keeps `embed()` because it is a multi-capability framework
adapter and serves as the embedding transport for `lazyllm_backend`.
- `LLMConfig.embed_model` / `embed_batch_size` are retained **only** as a
backward-compat bridge consumed by `embedding_config_from_llm`; they no longer
affect chat clients. New configs should set `embedding_profiles` directly.
## Consequences
Positive:
- clear capability boundaries: `memu.llm` (text), `memu.vlm` (vision),
`memu.embedding` (vectors) are isomorphic and independent;
- embedding-only providers (Jina, Voyage) are first-class; new providers are a
single backend module plus a registry entry;
- no duplicated embedding logic; one source of truth;
- per-capability provider/transport selection (e.g. chat via OpenAI, embed via
Voyage) with zero-config derivation as the default.
Negative:
- `LLMConfig` still carries `embed_model` / `embed_batch_size` as a bridge, so the
decoupling is not yet "pure" at the config layer (kept for backward
compatibility);
- an explicit `embedding_profiles` is required to use an embedding provider that
differs from the chat provider;
- one more package/gateway to maintain in parity with `llm`/`vlm`.
docs/adr/0006-from-memory-item-category-to-tracked-workspace-memorization.md
+162
View File
@@ -0,0 +1,162 @@
ADR 0006: From Memory Item/Category to a Tracked RecallFile/RecallEntry Store for Workspace Memorize/Retrieve
- Status: Accepted
- Date: 2026-06-30
## Context
ADR 0004 landed `memorize_workspace` and the markdown "memory file system", and
deliberately kept the `skill/` tree **synthesized at export time** from the shared
per-source description trunk (never persisted, never derived from extracted items).
Skills therefore live only on disk: `MemoryFilesBuilder.build` synthesizes a
`slug -> body` map via `MemorySynthesizer`, the exporter writes `skill/<slug>/SKILL.md`,
and an incremental update reads those files back off disk (`read_skills`) to feed the
"existing skills" half of the next synthesis.
This has grown awkward:
- skills are not first-class data — they cannot be scoped (`user_data`), queried, or
reasoned about alongside the structured store; the markdown tree *is* the database;
- the "existing skills" context for incremental synthesis is recovered by parsing
markdown back off disk, rather than read from a table;
- the memory track and the skill track are asymmetric: memory categories are
`RecallFile` rows rendered deterministically into `memory/`, while skills are an
ephemeral synthesis artifact with a parallel-but-different code path in the exporter.
Meanwhile the memory track already does, per source file in the `memorize_workspace`
loop, exactly the shape skills want: take preprocessed content, fold it into a durable
`RecallFile.summary` via an LLM patch. Skills should reuse that machinery.
This work lands on top of a domain-wide rename done as the first commit on the branch:
`MemoryItem` → `RecallEntry`, `MemoryCategory` → `RecallFile`, `CategoryItem` →
`RecallFileEntry` (and their repositories), a pure no-behavior-change rename that
establishes the `RecallFile`/`RecallEntry` vocabulary this ADR uses throughout.
## Decision
Make a **skill** a first-class `RecallFile`, distinguished from a memory category by a
new `track` column, and generate skills **inside the memorize workflow** (the
workspace path) rather than freshly at export time.
### Data model
- Add `RecallFile.track: str` with values `"memory" | "skill"`, defaulting to
`"memory"`. Existing rows backfill to `"memory"`.
- Rename `RecallFile.summary` → `RecallFile.content`. The `summary` name predates the
`MemoryCategory` → `RecallFile` rename; the column holds the file's body, so
`content` is the correct name. (`RecallEntry.summary` is **unchanged** — only the
file-level column is renamed.) Both changes ship in one migration since the `track`
add already forces one.
- `get_or_create_category` keys on `(name, track, *scope)`, and the Postgres
unique-with-scope index becomes `(name, track, *scope)`, so a skill and a memory
category may share a name without colliding.
### Track isolation via `where`
Every `RecallFile` reader scopes by track through the existing `where` filter rather
than ad-hoc post-filtering:
- the memory/categorize/retrieve/CRUD paths and the exporter's `memory/` rendering
read `track="memory"`;
- the skill generation step and (future) skill rendering read `track="skill"`.
This keeps skills out of memory retrieval/RAG scoring and out of `memory/`.
### Skill generation in the workflow
`_memorize_one` (the `memorize_workspace` per-file path) runs a **new** workflow
(`memorize_workspace`) — the existing `memorize` steps plus a skill-generation step —
rather than overloading the shared `memorize` pipeline (single-file `memorize()` is
unchanged, mirroring ADR 0004's "don't touch `memorize`" stance). The new step:
- runs per file, after the memory persist step, taking `preprocessed_resources` as
input (it has no data dependency on the memory categorize/persist outputs);
- reads existing `track="skill"` `RecallFile`s as the "existing skills" context — read
from the DB, not from disk — so file *N* sees skills created by files *1..N-1*;
- reuses the existing skill-synthesis prompt/algorithm (processed content + existing
skills → skills to add/replace), now emitting `(name, description, body)`;
- persists each skill **directly** as a `RecallFile(track="skill", content=body)`,
embedding `name + description`, bypassing the `RecallEntry` plane entirely;
- is gated behind `memory_files_config.synthesize`.
### Export (mirrors the memory track)
The exporter mirrors the memory track exactly: it renders each `track="skill"`
`RecallFile` deterministically into `skill/<slug>.md` (just as `memory/<slug>.md` is
rendered from memory-track files — one shared, track-agnostic renderer), and the
synthesizer's skill role is reduced to producing a single overview `SKILL.md` body (the
mirror of `MEMORY.md`) synthesized from the skill files' bodies, rather than the
per-skill bodies. The old description-based skill synthesis, the `RecallEntry` skill
bypass, and the `skill/<slug>/SKILL.md` folder layout are gone; `read_skills` is
replaced by `read_skill_body` (the SKILL.md-overview analog of `read_memory_body`).
`SKILL.md` falls back to a deterministic link index when `synthesize=False`, mirroring
`MEMORY.md`.
### Retrieve (consuming the track)
The `track` column also opens the read side. A new `retrieve_workspace` entry point — the
read-side analog of `memorize_workspace`, and the simple counterpart to the routing-heavy
`retrieve` — does single-shot, LLM-free retrieval: it embeds the query once and ranks the
file/entry/resource layers by vector similarity, with none of the intention routing,
sufficiency checks, or summarization of the RAG path. The file layer scopes by `track`
through `RetrieveFileConfig.tracks`, so skill-track `RecallFile`s can be retrieved (or
excluded) independently of the memory track — the read-side mirror of the track isolation
above. Driven by a new `RetrieveWorkspaceConfig`.
## Scope of these commits
This ADR spans four commits on the branch (a fifth — gating new-category creation behind
`allow_new_categories` — is an unrelated fix to the existing taxonomy path):
1. **Domain rename** (foundation). `MemoryItem` → `RecallEntry`, `MemoryCategory` →
`RecallFile`, `CategoryItem` → `RecallFileEntry`, and the matching repositories. Pure
rename, no behavior change; establishes the vocabulary the rest of this work assumes.
2. **Skill track + workflow.** Schema foundation (`track` column, `summary` → `content`,
`(name, track)` keying, migration), track isolation (memory-track readers filter
`track="memory"`), and the new `memorize_workspace` workflow with its per-file skill
generation step that persists `track="skill"` `RecallFile`s.
3. **Export mirror.** `skill/<slug>.md` rendered deterministically from the DB + a
synthesized/indexed `SKILL.md` overview, symmetric with `memory/` + `MEMORY.md`.
4. **Workspace retrieve.** The LLM-free `retrieve_workspace` path that reads the track
back out — see "Retrieve (consuming the track)" above.
## Open issues (deferred)
- **Skill staleness / provenance on deletion.** By bypassing the `RecallEntry` plane,
skill `RecallFile`s have no link back to the resources that produced them. The memory
track invalidates correctly on file change/delete via
`RecallEntry.resource_id → RecallFileEntry → category` cascade
(`_cascade_delete_by_urls`); skills have no equivalent, so a skill derived from a
now-deleted source file lingers. We do not yet have a chosen solution (candidates: a
provenance link from skill files to contributing resources, or a full skill-track
rebuild on `diff.has_removals`). Until then the skill track is **append/merge-only**
and does not shrink on deletion. *A solution is expected in a future commit.*
- **Cost / latency.** Skill synthesis moves from ~1 merged LLM call per workspace sync
to one call per added/modified file (consistent with how per-file memory summaries
already work, and gated behind `synthesize`). Plus one `SKILL.md`-overview synthesis
per export, the mirror of the existing `MEMORY.md` synthesis. Acceptable, but noted.
## Consequences
Positive:
- skills are first-class, scoped, queryable rows; the markdown tree stops being the
only home for skills;
- the "existing skills" context for incremental generation is a table read, not a
markdown re-parse;
- memory and skill tracks become symmetric (`RecallFile` rows distinguished by
`track`), collapsing two divergent code paths;
- single-file `memorize()` stays untouched; the skill step lives only on the workspace
path;
- making skills first-class rows lets the new `retrieve_workspace` path read them back by
`track`, so the workspace gets a symmetric memorize/retrieve pair without skills needing
a bespoke read path.
Negative:
- a schema migration is required (`track` add + `summary`→`content` rename); existing
Postgres/SQLite deployments need it applied (fresh DBs get it from `create_all`);
- every `RecallFile` reader must now be track-aware, a cross-cutting concern threaded
through `where`;
- deletion/provenance for skills is a known, deferred gap (see open issues).
docs/adr/0007-three-independent-memory-lines-wiki-graph.md
+242
View File
@@ -0,0 +1,242 @@
ADR 0007: Split Memorize/Retrieve into Three Independent Lines on a Layered Kernel
- Status: Proposed
- Date: 2026-07-01
- Supersedes: ADR 0004 (workspace memorize + memory file system), ADR 0006 (skill as a `RecallFile` track)
## Context
ADR 0004 added `memorize_workspace`/the markdown memory file system; ADR 0006 promoted
skills to a `track="skill"` `RecallFile` generated inside the workspace memorize workflow.
Living with both has surfaced a consistent coupling problem.
1. **One pipeline does everything, per file.** `memorize_workspace(folder)` is a fan-out:
for each changed file it runs the full `memorize` workflow *plus* a `generate_skills`
step, then rebuilds the entire `INDEX.md`/`MEMORY.md`/`SKILL.md` tree. Memory
extraction, skill synthesis, and indexing all fire for every file regardless of what
that file is for.
2. **`generate_skills` is a parallel mechanism that bypasses the entry plane.** It
re-synthesizes skills from the raw preprocessed text (not from the extracted
`RecallEntry`s), and persists them directly as `RecallFile(track="skill")`, bypassing
the `RecallEntry → RecallFileEntry → RecallFile` cascade. That bypass *is* the cause of
ADR 0006's own deferred open issue: skills have no provenance link, so they never
invalidate on source change/delete.
3. **The `track` column exists only to jam two different things into one table.** Memory
categories and skills share `RecallFile` and are split by `track`. That forced sharing
is the root of the asymmetry and the `generate_skills` bypass.
4. **The read side is forked, not composed.** `retrieve_rag`/`retrieve_llm` (heavy
intention-routing + sufficiency RAG over the structured store) and `retrieve_workspace`
(flat embedding recall) share **no** handlers — `_rag_*`/`_llm_*` vs `_ws_*` are
independent reimplementations of the same recall. The write side composes
(`memorize_workspace` = `memorize` + one step); the read side does not.
The deeper observation: `INDEX` / `MEMORY` / `SKILL` are not three outputs of one process —
they are three different **treatments** of source material, with three different **read
models**. Routing them through one pipeline and one store is the coupling.
A second observation refines the design: **embedding search and a graph are separable.**
Embedding search (embed → vector rank) is a flat capability over a set of pages; a graph
adds edges + traversal. Of the three layers, memory and the workspace index are
*relational* (entities interlink; documents reference each other and have hierarchy —
traversal matters), while skills are a *retrieval* problem ("which skill fits my current
task" = flat top-k), where a graph adds cost for little value.
## Decision
Refactor `memorize`/`retrieve` into **three independent lines**, each fed by its own
source, owning its own store and output, and sharing a **layered wiki-graph kernel**.
### The three lines
| Line | Source | Output index | L0 → L1 → L2 (raw → doc → item slices) | Read |
| --- | --- | --- | --- | --- |
| **chat** | conversation logs | `MEMORY.md` | raw chat → memory category → paragraph slices | hybrid: embedding + BM25 |
| **workspace** | workspace files | `INDEX.md` | raw media → caption paragraph → caption slices | hybrid: embedding + BM25 |
| **skill** | execution / tool traces | `SKILL.md` | raw logs → skill md → per-skill descriptions | hybrid: embedding + BM25 |
Each line keeps the established output-file name as its L1 index/map, over its own L0/L1/L2 stores.
> **Note.** This section's original "graph wiki / graph traversal" framing is **superseded**
> by "Refinement: hybrid retrieval (embedding + BM25), no graph" below; there is no graph.
### Shared kernel (one implementation, three instances)
All three lines share one **in-process** kernel: record stores (resource / item / category),
embedding, **hybrid search (cosine embedding + BM25 keyword)**, rank, markdown render, and
manifest diff. There is **no graph layer** and no external database — see "Refinement: hybrid
retrieval" below.
### Per-line data model (independent stores, no `track`)
```
Node (a wiki page / file): id · slug · content(md) · embedding · source_ref · meta
Edge (a [[wikilink]]): from_node · to_node · type? ← graph lines only
```
`INDEX.md` / `MEMORY.md` / `SKILL.md` are rendered deterministically from each line's Node
set. The three lines have **separate stores** — there is no shared table and no `track`
discriminator.
### Layer model: L0 / L1 / L2 (refines the data model above)
The generic "Node/Edge" sketch above is refined into three **representation layers**. **Every
line has all three.** The layers share a uniform *role* but hold line-specific artifacts, and
each is **derived from the one below** (L0 → L1 → L2):
| Layer | Role (uniform) | chat | workspace | skill |
| --- | --- | --- | --- | --- |
| **L0** | raw source (`resource`) | raw chat corpus | raw multimodal data | raw agent-run logs |
| **L1** | coarse document derived from L0 | the classified **memory category** file | the preprocessed **caption paragraph** | the **skill markdown** |
| **L2** (**item**) | fine slices/extracts of L1 — the **embed/search unit** | slices of the category's paragraphs | slices of the caption paragraph | the **description** extracted per skill |
- **L1 ⊇ L2 in every line**: an *item* (L2) is a slice/extract of its L1 document. This
**inverts the earlier wording** ("L1 = atomic item, L2 = category"): now **L1 is the coarse
document and L2 is the fine item**, consistently across all three lines.
- **Derivation** runs L0 → L1 → L2: preprocess the raw source into the L1 document, then
slice/extract it into L2 items.
- Because the meanings are now per-line (and inverted from the old model), they **no longer map
1:1 onto `Resource`/`RecallEntry`/`RecallFile`**; each line's store holds a resource (L0), a
document (L1), and item slices (L2).
**Retrieval (hybrid, no graph, single pass).** The **L2 items are embedded**; a query runs
one hybrid pass — cosine embedding + BM25 keyword, each min-max normalized and fused — over
the L2 items, and the top items **roll up to their L1 document** (and its L0 resource) for the
result. No `item ↔ item` edges, no entity index, no traversal, no multi-hop.
**Markdown output.** `MEMORY.md` / `SKILL.md` render the **L1** documents (memory categories /
skills); `INDEX.md` indexes the **L0** resources. L2 items live only in the store as the
embed/search units, not as separate files.
**Seam with preprocessing.** Preprocessing is the **injected first step of every line's
memorize pipeline** (`preprocess → …`): it takes a source *reference* and produces the **L1
document** (e.g. caption paragraph / classified memory / skill md) from the raw L0 source;
slicing/extracting that document into **L2 items** is the next step. Like the other treatments
(embed, and the per-line slice/extract), `preprocess` is injected so lines stay testable; a
real implementation wraps `memu.preprocess` (fetch via blob + modality decode + describe).
> **Divergence note.** This redefinition inverts L1/L2 and gives every line all three layers,
> so it is **ahead of the current `memu.lines` code** (which still has L1 = atomic item, L2 =
> grouping, workspace with no L2, skill with no L1). The code will be re-aligned to this model.
### Refinement: hybrid retrieval (embedding + BM25), no graph
This **supersedes every "graph / edges / backlinks / traversal / entity linking / GraphRAG /
multi-hop" description earlier in this ADR.** L1 has **no connective structure**:
- **No graph.** No `item ↔ item` edges, no `neighbors` / `backlinks` / `traverse`.
- **No entity layer / GraphRAG / multi-hop.** No parallel entity store, no item→entity index,
no entity-overlap ranking, no iterative expansion.
Retrieval is a **single pass, hybrid**: a cosine embedding score and a BM25 keyword score are
each min-max normalized across candidates and fused into one rank. No iteration, no graph
walk. All three lines retrieve this way, differing only in *what* they search and *what* they
return:
All three lines retrieve the same way — embed the **L2 items**, one hybrid pass, roll up to
the **L1 document**:
| Line | embeds / searches (L2 items) | rolls up to / returns (L1) |
| --- | --- | --- |
| **chat** | slices of the memory category's paragraphs | the memory category (file) |
| **workspace** | slices of the caption paragraph | the caption paragraph (and its L0 resource) |
| **skill** | descriptions extracted per skill | the skill markdown |
Rationale: without multi-hop traversal, an entity index is not a graph — it is only a ranking
feature, and it did not justify graph machinery here. Hybrid (embedding + keyword) is a
standard, cheap, single-pass retrieval covering both semantic and exact-term matches; a graph
/ entity / multi-hop signal can be added later as an explicit, measured enhancement if a
benchmark shows it helps.
Concretely, `Edge` / `neighbors` / `backlinks` / `traverse`, any entity store / item→entity
index, and any multi-hop loop are **not part of the design**. The stores hold records +
embeddings and expose a single `search(query_vec, query_text, k)` over the L2 items that fuses
cosine + BM25; the hits then roll up to their L1 document before returning.
### Per-line ingest treatment (the only code that differs)
Each line's ingest is `preprocess (raw L0 → L1 document)` → `slice/extract (L1 → L2 items)` → `embed`:
- **chat:** classify the conversation into a memory category (L1); slice its paragraphs into items (L2).
- **workspace:** preprocess media into a caption paragraph (L1); slice it into items (L2).
- **skill:** synthesize a skill markdown (L1) from the logs; extract a description per skill as the item (L2).
### Independent triggers
Each line has its own source manifest and change detection. A change under one line's
source rebuilds **only that line**; lines never trigger each other. (A change in `/chat`
no longer re-synthesizes skills or rebuilds the workspace index.)
### Shared, not duplicated
The kernel is one implementation; the three lines are instances of it. Existing low-level
utilities stay shared *calls*, not copies: `LocalFS` fetch, the preprocess registry
(modality handlers), the embedding clients / `ClientPool`, and the workflow engine. A
single **scope convention** (`user_id` / `where`) is preserved across all three lines so
cross-line scoping ("everything about user X") still works even though the stores are
separate.
## What this removes
- `RecallEntry` / `RecallFile` / `RecallFileEntry` and the `track` column → per-line
`Node`/`Edge`.
- The monolithic per-file `memorize_workspace` pipeline and the `generate_skills` step →
three ingest treatments.
- `retrieve_rag` / `retrieve_llm` / `retrieve_workspace` → one shared read mechanism
(embedding search, plus graph traversal on the two graph lines).
- The skill provenance/deletion gap (ADR 0006 open issue): the skill line owns its store
and bypasses nothing; deletion is driven by the line's own manifest.
## Consequences
Positive:
- Three high-cohesion modules replace the two ~1500-line `memorize.py`/`retrieve.py`
mixins; each line is end-to-end self-contained (source → ingest → store → read →
output).
- The `track` column and the `generate_skills` bypass are gone; the skill provenance gap
is resolved structurally rather than deferred.
- The read side becomes coherent — one search mechanism (+ optional traversal) instead of
three duplicated recall paths.
- Each line's structure matches its real need (graph for relational lines, flat for
retrieval), instead of an over-imposed uniform shape.
- Independent triggers cut wasted work: a chat change no longer re-synthesizes skills or
rebuilds the index.
Negative / costs:
- A large refactor with a data migration (track-based rows → per-line stores).
- **No cross-line edges:** an entity relevant to more than one line is duplicated, with no
cross-domain traversal. Accepted as the separation tax; a read-only weak cross-line
reference (e.g. a skill page citing a memory entity by URI) is a future option, not built
now.
- The workspace graph relies on **structural** links to avoid LLM link-inference cost; richer
semantic linking there is deliberately out of scope.
- Three stores instead of one: a cross-line query must hit three lines (mitigated by the
shared scope convention).
## Open issues (deferred)
- **Skill source.** Assumed to be a `/log` (trace) folder. Whether skill is a primary
source line or is *distilled* from the chat/workspace lines is unresolved; it changes
whether skill is a true third line or a second-order derivation.
- **Migration mechanics** from the current `RecallFile(track=…)` schema to three per-line
stores (sketch: `track=memory` → chat nodes; `track=skill` → skill nodes; resources/INDEX
→ workspace nodes).
- **Workspace linking** — keep it purely structural, or add light semantic linking later.
## Implementation plan
PoC the **workspace line first**: it has the least LLM (structural links, kept-whole pages)
and the clearest graph, so it best validates the Base + Graph kernel end-to-end before the
LLM-heavy chat and skill lines are built on the same kernel.
## Related ADRs
- Supersedes `docs/adr/0004-workspace-memorize-and-memory-file-system.md`
- Supersedes `docs/adr/0006-from-memory-item-category-to-tracked-workspace-memorization.md`
- Builds on `docs/adr/0001-workflow-pipeline-architecture.md` (the kernel still composes as
workflow steps), `docs/adr/0002-pluggable-storage-and-vector-strategy.md`, and
`docs/adr/0005-dedicated-embedding-package.md`.
docs/adr/0008-two-integration-surfaces-hooks-and-api.md
+274
View File
@@ -0,0 +1,274 @@
ADR 0008: Trajectory as the Source — Zero-Code Hooks over a Programmatic API
- Status: Proposed
- Date: 2026-07-07
- Builds on: ADR 0007 (three independent memory lines on a layered kernel), ADR 0001 (workflow pipelines)
- Refines: ADR 0007's source model (resolves its "skill source" open issue) and output model
(no rendered index files: per-line folders of L1 documents, searched via a pure per-line
embedding index of L2 items carrying L1-pointing metadata)
## Context
ADR 0007 settled the *engine*: three independent lines (chat / workspace / skill) over a
shared layered kernel, driven by `memorize-workspace` (write) and `retrieve-workspace`
(read), against a per-project store. What it did **not** settle is how an *agent* — Claude
Code, Codex, OpenClaw, Hermes, or an in-house one — actually feeds sources in and gets
memory back out during a live session.
Today there is effectively one path: a human (or an agent following `SKILL.md`) runs the
CLI by hand against a **curated folder** — sources are whatever someone placed under `chat/`
or `agent/`, and the directory decides the treatment. That is fine for "sync this folder,"
but it does not give an agent **continuous** memory: the user turns and the agent's own tool
trajectory are never captured unless someone remembers to save them, and retrieved memory is
never injected unless the agent explicitly asks.
The deeper problem is the **source model itself**. The richest record of a working session is
the agent's **trajectory** — the interleaved user messages, agent reasoning, and tool actions.
Asking users (or agents) to pre-sort that record into per-line source directories throws away
the fact that a single stretch of work simultaneously contains *who the user is*, *how the
agent got something done*, and *where the project stands*. The structural, directory-based
routing of ADR 0007's sources cannot see any of that.
Beyond the source model, two different consumers want two different things:
1. **A drop-in user** wants memory to "just work" with **no code changes** to their agent —
install something, and the agent starts remembering across sessions.
2. **A builder** embedding memU inside their own agent wants **explicit control** — call
retrieve at a chosen point, memorize a chosen artifact, with their own batching and
scoping.
Trying to serve both with one mechanism has been the tension. A pure API serves the builder
but leaves the drop-in user writing glue. A pure "magic" integration serves the drop-in user
but is a black box the builder cannot steer. And every host agent exposes a **different** hook
system (Claude Code `settings.json` events, Codex/OpenClaw/Hermes each their own), so "just
use hooks" is not one integration but N.
## Decision
Three decisions: one philosophical (the source), one structural (the surfaces), one of
naming (the CLI).
### 1. Trajectory is the primary source; the three lines are three extractions from it
The agent's **trajectory** (user turns + agent actions/tool traces) becomes the primary raw
source. Instead of routing pre-sorted directories to lines, memorize runs a **semantic
extraction** over the trajectory that distills three kinds of memory — each landing on the
ADR 0007 line built for it. Concretely, each line's **L0 is a different projection of the
same trajectory** (see the layer mapping below), rather than a separate source folder.
The lines are **renamed after what they extract** — `memory` / `skill` / `project` are the
canonical terms from here on (ADR 0007's `chat` and `workspace` are the old names of the
`memory` and `project` lines respectively):
| Line (canonical term) | What it captures | ADR 0007 name / source dir | Output |
| --- | --- | --- | --- |
| **memory** | user facts, preferences, conversational context | chat (`/chat`) | `memory/` folder (L1 docs) + embedding index |
| **skill** | reusable procedures — how the agent accomplished something | skill (`/skill`) | `skill/` folder (L1 docs) + embedding index |
| **project** | the files the work touched — project state as reflected in their content | workspace (`/workspace`) | `project/` folder (L1 docs) + embedding index |
**Output model: per-line L1 folders + a pure per-line embedding index. No rendered index
files.** ADR 0007's single rendered files (`MEMORY.md` / `SKILL.md` / `INDEX.md`) are
**removed**, replaced by two artifacts per line:
- **Browsable output** — a folder per line (`memory/`, `skill/`, `project/`) holding **one
markdown file per L1 document** (per memory category, per skill, per project document).
The folder *is* the human-facing output; nothing is rendered on top of it.
- **Searchable output** — a **pure embedding index per line**: one embedding per **L2 item**,
whose embedded content is the L2 slice itself and whose **metadata points to its L1
document** (and through it, the L0 trajectory span). A retrieval hit rolls up L2 → L1 via
that metadata alone; L2 items never materialize as files.
Rationale: a change to one document touches one file instead of rewriting a monolith; the
search path needs no rendered artifact at all — embeddings + metadata carry the full
L2 → L1 → L0 chain; and the on-disk names match the line terms exactly.
**Layer mapping.** Each line keeps ADR 0007's L0 → L1 → L2 derivation, but the L0s are now
three projections of the **same trajectory** instead of three separate source folders:
| Layer (role) | memory | skill | project |
| --- | --- | --- | --- |
| **L0** — the line's projection of the trajectory | the **user queries and agent responses only** (no tool/action traces) | the **entire user–agent trajectory** (turns + actions/tool traces) | the **contents of files** touched in the trajectory |
| **L1** — summarized document (one md file in the line's folder) | the summarized **memory file** | the synthesized **skill** | the **summary** of the file's content |
| **L2** — the embedded unit (metadata → L1) | **slices** of the memory file | the skill's **description** | **slices** of the summary |
Two properties distinguish this from ADR 0007's directory routing:
- **Extraction is one-to-many, not routing.** A single turn can yield a user fact *and* a
reusable skill *and* a project-state update; each extractor takes what it needs from the
same trajectory. Directory routing was one-to-one by construction and could never do this.
- **The intelligence moves from the caller into the pipeline.** Nobody pre-sorts sources; the
classify/extract step (the L0 → L1 `preprocess` seam of each line, per ADR 0007) decides
what a trajectory contains. Callers — human, hook, or code — just hand over raw trajectory.
This **resolves ADR 0007's open "skill source" question**: skill is *distilled from the
trajectory*, not a primary source line fed by a separate `/log` folder. It also shifts the
**project** line's meaning from "indexed multimodal files" (the old workspace framing) toward
**project memory**; direct file ingestion (pointing `memu memorize` at a curated folder)
remains supported as a secondary path on the same line.
### 2. Two integration surfaces, layered — not two parallel products
- **The API/CLI is the base plane.** `memu memorize` / `memu retrieve` (Decision 3; and the
in-process service behind them) are the single source of truth. Everything writes to and
reads from the same per-project store defined in ADR 0007.
- **Hooks are a zero-code wrapper built *on* that base plane.** The hook integration is not a
second engine; it is an adapter layer that calls the same CLI/service on the agent's
behalf. Because both surfaces share one store, a project can **mix** them freely — record
via hooks, retrieve explicitly in code, or vice versa — with no divergence.
### Surface A — Hook integration (zero-code, default distribution)
An agent loads memU's `SKILL.md` for capability description, and a **host adapter** wires memU
into the agent's lifecycle. The adapter does exactly three things, and only these three:
1. **Load** — make `SKILL.md` (and the store location) visible to the agent as a capability.
2. **Record trajectory** — on each turn, append the user input and the agent's action/tool
trace as raw trajectory, then trigger memorize. The adapter does **no sorting**: the
pipeline's extraction step (Decision 1) distills memory / skill / project from it.
3. **Inject memory** — before the agent generates, retrieve against the current prompt and
splice the results into the agent's context.
### The hook contract (one abstraction, N thin adapters)
Host hook systems are heterogeneous, so memU defines **one contract** with two seams, and each
host ships a **thin adapter** that maps its native events onto them:
| Seam | Fires | Does | memU call | Execution |
| --- | --- | --- | --- | --- |
| **`on_turn`** | after a turn completes (user msg + agent trace available) | append the turn's raw trajectory, trigger memorize (extraction distills memory/skill/project) | `memu memorize` | **async / background, non-blocking** |
| **`on_prompt`** | before the agent generates | retrieve for the current prompt, splice hits into context | `memu retrieve` | **synchronous, in the critical path** |
The adapter's whole job is to bind a host's native hook events to `on_turn` / `on_prompt`.
Concretely:
- **Claude Code** — `settings.json` hooks: a `Stop`/post-turn hook → `on_turn`; a
`UserPromptSubmit` hook → `on_prompt`.
- **Codex / OpenClaw / Hermes** — each maps its own post-turn and pre-generation events onto
the same two seams.
New hosts are added by writing one adapter, not by touching the kernel or the CLI.
### Execution model (per the decision above)
- **Recording is asynchronous.** `on_turn` enqueues; memorize runs in the background and must
never block the conversation. It reuses ADR 0007's incremental manifest diff, so repeated
turns are cheap and only changed sources are processed.
- **Injection is synchronous.** `on_prompt` does a single-shot `memu retrieve` (embedding +
BM25 hybrid search, no LLM per ADR 0007) in the critical path, under a **token budget**, and
**fails open**: a miss, a timeout, or an empty store injects nothing and the turn proceeds
normally.
### Surface B — Programmatic API (code-level control)
Builders embedding memU call retrieve / memorize directly from their agent code (in-process
service or CLI/JSON), choosing their own timing, batching, and scope (`user_id` / `where`).
This is the base plane exposed directly — the same calls the hook adapters make, with no hook
layer in between.
### Shared invariants (hard constraints)
- **One store, one semantics.** Both surfaces target the same per-project store and the same
three lines. Hook mode must not invent a private store or a divergent record format.
- **Scope convention preserved.** The ADR 0007 `user_id` / `where` scope convention holds
across both surfaces, so cross-surface queries ("everything about user X") work regardless
of how it was written.
- **Hooks add no engine.** Any behavior available via hooks must be expressible as base-plane
calls; the adapter is orchestration only.
### 3. CLI: reclaim `memu memorize` / `memu retrieve` as the canonical pair
With the lines renamed (Decision 1), "workspace" no longer names a line — leaving the
`memorize-workspace` / `retrieve-workspace` commands carrying a retired term that now reads
as "one line" when they actually operate on the whole store. Rename the pair:
- **`memu memorize`** and **`memu retrieve`** become the **only** commands, carrying the new
semantics (trajectory ingestion + extraction on the write side; hybrid search over all
three lines on the read side).
- The **legacy single-file `memorize` / `retrieve`** (already deprecated, documented as
never-fall-back) are **removed first**; the names are reclaimed only after removal, so no
release ships two behaviors under one name.
- **`memorize-workspace` / `retrieve-workspace` are removed outright** — no alias, no
deprecation window. This is a breaking major-update cut; adapters and `SKILL.md` ship the
canonical names in the same release.
The accepted risk: both prior command generations break at once — old legacy users find the
reclaimed names doing something new, and `-workspace` users get command-not-found. Accepted
deliberately: this ADR is the major update, and a clean cut beats carrying a retired term in
an alias.
## What this changes
- **The terms unify around the extractions**: the three lines are canonically named
**memory / skill / project** (renaming ADR 0007's `chat` and `workspace`); docs, CLI
output, and code should converge on these three terms.
- **The output changes shape**: `MEMORY.md` / `SKILL.md` / `INDEX.md` are gone; each line
yields a folder of L1 documents (`memory/` / `skill/` / `project/`, one file per document)
plus a pure embedding index of L2 items whose metadata points back to L1.
- **The source model inverts**: from "users curate per-line source folders" to "the trajectory
is the source; extraction curates." Directory-based routing (`chat/` → memory, `agent/` →
skill) becomes a legacy/secondary ingestion path.
- The project line's center of gravity shifts from multimodal file indexing to **project
memory**; ADR 0007's "skill source" open issue closes as *distilled from trajectory*.
- **The CLI collapses to one pair**: `memu memorize` / `memu retrieve` (new semantics); both
the legacy single-file pair and `memorize-workspace` / `retrieve-workspace` are removed —
no aliases.
- `SKILL.md` graduates from "instructions a human/agent follows to run the CLI" to "the
capability surface a host adapter loads," while remaining valid for manual use.
- memU gains a small **host-adapter layer** (one thin adapter per host) as the unit of new
integrations, with the two-seam contract as its only coupling to the engine.
- The API surface is stated as a **first-class, supported** integration path, not just an
internal detail behind the CLI.
## Consequences
Positive:
- Drop-in users get continuous cross-session memory with no code changes; builders keep full
programmatic control — from one engine, not two.
- Host heterogeneity is contained: adding Codex/OpenClaw/Hermes/etc. is one adapter against a
fixed two-seam contract, not a bespoke integration each time.
- Async record + sync inject keeps the conversation responsive (no per-turn LLM blocking)
while still injecting fresh memory before each generation.
- Mixing surfaces is safe by construction because both hit the same store with the same scope
convention.
Negative / costs:
- **Extraction is not free.** Directory routing was structural (zero-cost); semantic
extraction over trajectory is LLM work on every memorize, and a misclassifying extractor
pollutes lines silently — quality now depends on prompts, not conventions.
- N host adapters to build and maintain; each host's hook system evolves independently and can
break its adapter.
- Async recording means a just-finished turn may not yet be retrievable on the immediately
following prompt (eventual, not read-your-writes, consistency) — accepted for
responsiveness.
- Synchronous injection adds retrieve latency to every prompt; bounded by the token/time
budget and fail-open behavior, but non-zero.
- Auto-recording every turn raises **privacy/scope** questions (what gets persisted, and under
whose `user_id`) that manual CLI use did not.
## Open issues (deferred)
- **Injection shape & budget.** Where retrieved memory lands (system-prompt append vs. a
dedicated context block), and the exact token budget / ranking cutoff, are unspecified here.
- **Record batching policy.** Per-turn vs. per-session flushing, and debounce/coalescing of
rapid turns, are left to the adapter; a shared default is TBD.
- **Extraction quality bar.** What counts as a *skill-worthy* procedure or a *project-worthy*
state change (vs. noise) is a prompt/heuristic question for the extraction step; the
one-to-many contract is decided here, the classifier itself is not.
- **Trajectory format.** The wire format a host adapter emits (full transcript vs. structured
turn events, tool-output truncation) is per-adapter for now; a shared schema is TBD.
- **Privacy controls.** Opt-out, redaction, and per-turn scoping of auto-recorded content are
not designed yet.
## Related ADRs
- Builds on `docs/adr/0007-three-independent-memory-lines-wiki-graph.md` (engine, three lines,
store, `memorize-workspace` / `retrieve-workspace`); **refines its source model** —
trajectory replaces per-line source folders as the primary source, its "skill source"
open issue is resolved as *distilled from trajectory*, its `chat` / `workspace` lines
are renamed **memory** / **project**, and its rendered `MEMORY.md` / `SKILL.md` /
`INDEX.md` outputs are replaced by per-line L1 folders plus a pure per-line embedding
index of L2 items.
- Builds on `docs/adr/0001-workflow-pipeline-architecture.md` (both surfaces drive the same
workflow pipelines).
docs/adr/README.md
+10
View File
@@ -0,0 +1,10 @@
# Architecture Decision Records
- [0001: Use Workflow Pipelines for Core Operations](0001-workflow-pipeline-architecture.md)
- [0002: Use Pluggable Storage with Backend-Specific Vector Search](0002-pluggable-storage-and-vector-strategy.md)
- [0003: Model User Scope as First-Class Fields on Memory Records](0003-user-scope-in-data-model.md)
- [0004: Add Workspace Memorize Without Touching Single-File `memorize`](0004-workspace-memorize-and-memory-file-system.md)
- [0005: Extract Embedding into a Dedicated Package, Fully Decoupled from Chat Clients](0005-dedicated-embedding-package.md)
- [0006: Promote Skills to a First-Class `RecallFile` Track, Generated Inside Memorize](0006-from-memory-item-category-to-tracked-workspace-memorization.md)
- [0007: Split Memorize/Retrieve into Three Independent Lines on a Layered Wiki-Graph Kernel](0007-three-independent-memory-lines-wiki-graph.md)
- [0008: Trajectory as the Source — Zero-Code Hooks over a Programmatic API](0008-two-integration-surfaces-hooks-and-api.md)
docs/architecture.md
+392
View File
@@ -0,0 +1,392 @@
# memU Architecture
## Purpose and scope
This document describes the self-hosted `memu` Python package architecture as
implemented in this repository, reflecting the `RecallFile`/`RecallEntry` data
model and the workspace memorize/retrieve paths introduced by ADR 0006.
The repository also describes a hosted Cloud product in `README.md`, but this
document focuses on the local `MemoryService` runtime and its code paths.
## System overview
memU implements structured agent memory with four persistent record types:
- `Resource`: raw source artifacts (conversation/document/image/video/audio)
- `RecallEntry`: extracted atomic memories with embeddings (`memory_type` + `summary`)
- `RecallFile`: grouped topic files — a memory category **or** a skill, distinguished
by a `track` column (`"memory" | "skill"`)
- `RecallFileEntry`: entry-to-file relation edges
At runtime, `MemoryService` orchestrates ingestion, retrieval, manual CRUD, and an
optional markdown export over these records.
> **Vocabulary note.** Earlier revisions named these `MemoryItem` /
> `MemoryCategory` / `CategoryItem`. ADR 0006 renamed them to `RecallEntry` /
> `RecallFile` / `RecallFileEntry` (and renamed `RecallFile.summary` → `content`),
> a pure-rename foundation on top of which the skill track was added. A few
> back-compat dict views on the `Database` protocol (`items`, `categories`,
> `relations`) retain the older names.
```mermaid
flowchart TD
A["Resource or Query"] --> B["MemoryService (composition root)"]
B --> C["Workflow Pipelines"]
C --> D["Capability Clients (LLM / VLM / Embedding)"]
C --> E["Database Repositories"]
E --> F["Resources"]
E --> G["RecallEntries"]
E --> H["RecallFiles (track: memory | skill)"]
E --> I["RecallFileEntries"]
B --> J["Memory File-System Export (INDEX/MEMORY/SKILL .md)"]
```
## Core runtime components
### `MemoryService` as composition root
`src/memu/app/service.py` constructs and owns:
- typed configs (`LLMProfilesConfig`, `DatabaseConfig`, `MemorizeConfig`,
`RetrieveConfig`, `RetrieveWorkspaceConfig`, `MemoryFilesConfig`, `UserConfig`,
`EmbeddingProfilesConfig`)
- storage backend (`build_database(...)`)
- resource filesystem fetcher (`LocalFS`)
- per-capability client pools (one `ClientPool` each for chat/LLM, VLM and
embedding; see `src/memu/app/client_pool.py`)
- workflow and LLM interceptor registries
- workflow runner (`local` by default, pluggable)
- named workflow pipelines via `PipelineManager`
- the markdown memory file-system builder (`MemoryFilesBuilder`,
`src/memu/app/memory_files.py`), which owns the exporter/synthesizer/lock
The three client pools (`ClientPool[Config, Client]`) replace hand-rolled
per-capability caches: each lazily builds and caches a client per profile name
through the matching gateway (`build_llm_client` / `build_vlm_client` /
`build_embedding_client`), so adding a new capability is one more pool rather
than another copy of the get/cache/build dance.
Public APIs are assembled by mixins:
- `MemorizeMixin`: `memorize(...)`, `memorize_workspace(...)`
- `RetrieveMixin`: `retrieve(...)`, `retrieve_workspace(...)`
- `CRUDMixin`: list/clear/create/update/delete memory operations
### Workflow engine
All major operations execute as workflows (`WorkflowStep`) with:
- explicit required/produced state keys (`requires` / `produces`)
- declared capability tags (`llm`, `vector`, `db`, `io`, `vision`)
- per-step config (for profile selection)
`PipelineManager` validates step dependencies at registration/mutation time and
supports runtime pipeline revisioning (`config_step`, `insert_before/after`,
`replace_step`, `remove_step`).
`WorkflowRunner` is a protocol; the default `LocalWorkflowRunner` executes
sequentially with `run_steps(...)`. Workflow state is a plain `dict[str, Any]`,
so step contracts are validated by key names rather than static types.
Registered pipelines (see `MemoryService._register_pipelines`):
| Pipeline | Entry point |
| ------------------------ | ---------------------------- |
| `memorize` | `memorize(...)` |
| `memorize_workspace` | `memorize_workspace(...)` |
| `retrieve_rag` | `retrieve(...)` (method=rag) |
| `retrieve_llm` | `retrieve(...)` (method=llm) |
| `retrieve_workspace` | `retrieve_workspace(...)` |
| `patch_create` / `patch_update` / `patch_delete` | `CRUDMixin` create/update/delete |
| `crud_list_recall_entries` / `crud_list_recall_files` / `crud_clear_memory` | `CRUDMixin` list/clear |
### Interception and observability hooks
Two interceptor systems exist:
- workflow step interceptors: before/after/on_error around each step
- LLM call interceptors: before/after/on_error around `chat/summarize/vision/embed/transcribe`
LLM wrappers also extract best-effort usage metadata from raw provider responses.
## Ingestion architecture (`memorize`)
`memorize(...)` executes the `memorize` pipeline:
1. `ingest_resource`: fetch local/remote resource into `blob_config.resources_dir`
via `LocalFS`. For the `document` modality, rich formats (PDF, Word,
PowerPoint, Excel, HTML, EPub) are converted to Markdown text via
[MarkItDown](https://github.com/microsoft/markitdown) (`memu.blob.document_text`),
while plain text (`.txt`/`.md`/...) is read directly. MarkItDown is an optional
extra: `pip install 'memu-py[document]'`.
2. `preprocess_multimodal`: modality-specific preprocessing for
conversation/document/audio (text-oriented path) and image/video
(vision-oriented path)
3. `extract_entries`: per-memory-type LLM extraction into structured `RecallEntry`
records (`memory_type` ∈ profile/event/knowledge/behavior/skill/tool by default)
4. `dedupe_merge`: placeholder stage (currently pass-through)
5. `categorize_entries`: persist resource + recall entries + entry-file relations
and embeddings (memory track)
6. `persist_index`: update recall-file summaries; optionally persist entry references
7. `build_response`: return resource(s), entries, files, relations
Recall-file (category) bootstrap is lazy and scoped: files are initialized when
needed with embeddings, and mapped by normalized name. New-category creation can
be gated behind `allow_new_categories`.
### Workspace ingestion (`memorize_workspace`)
`memorize_workspace(...)` syncs a folder and, per changed file, runs the
**`memorize_workspace`** pipeline: the standard `memorize` steps plus a final
`generate_skills` step (ADR 0006). That step:
- runs per file, after the memory persist step, taking `preprocessed_resources`
as input (no data dependency on the memory categorize/persist outputs);
- reads existing `track="skill"` `RecallFile`s from the DB as the "existing skills"
context, so file *N* sees skills created by files *1..N-1*;
- emits `(name, description, body)` skills and persists each **directly** as a
`RecallFile(track="skill", content=body)`, embedding `name + description` and
bypassing the `RecallEntry` plane entirely;
- is gated behind `memory_files_config.synthesize`.
Single-file `memorize(...)` is intentionally left untouched and does **not** run
skill generation or drive the exporter.
## Retrieval architecture (`retrieve`)
`retrieve(...)` chooses one of two pipelines from `retrieve_config.method`:
- `retrieve_rag` (embedding-driven ranking)
- `retrieve_llm` (LLM-driven ranking)
Both share the same staged pattern (step ids):
1. `route_intention` — route intention + optional query rewrite
2. `route_file` — recall-file (category) recall
3. `sufficiency_after_file` — optional early stop
4. `recall_entries` — entry recall
5. `sufficiency_after_entries` — optional early stop
6. `recall_resources` — raw resource recall
7. `build_context` — response build
Key behavior:
- `where` filters are validated against `user_model` fields before querying
- file readers scope to `track="memory"` so skills stay out of memory RAG scoring
- RAG path uses vector similarity (and optional salience ranking for entries)
- LLM path ranks IDs from formatted file/entry/resource context
- each stage can stop early if sufficiency check decides context is enough
### Workspace retrieval (`retrieve_workspace`)
`retrieve_workspace(...)` is the LLM-free, embedding-only counterpart of the
routing-heavy `retrieve` (ADR 0006). Driven by `RetrieveWorkspaceConfig`
(`method="rag"` only), it embeds the query once and ranks the file/entry/resource
layers by vector similarity, with **no** intention routing, query rewrite,
sufficiency checks, or summarization. Steps: `recall_files` → `recall_entries`
→ `recall_resources` → `build_response`. The file layer scopes by track via
`RetrieveFileConfig.tracks`, so skill-track files can be retrieved (or excluded)
independently of the memory track — the read-side mirror of the ingestion track
isolation.
## Data and storage architecture
### Repository contracts
Storage is abstracted through a `Database` protocol (`src/memu/database/interfaces.py`)
with four repositories:
- `ResourceRepo` (incl. `vector_search_resources`)
- `RecallEntryRepo` (incl. `vector_search_items`, similarity/salience)
- `RecallFileRepo`
- `RecallFileEntryRepo`
Vector ranking over **stored** embeddings is a repository responsibility, which
keeps the retrieval layer from reaching into any concrete backend. The pure
cosine/salience math lives in the storage-neutral `memu.vector` module (not under
any backend), so the app layer and every backend depend on it instead of on each
other. (File recall still ranks freshly re-embedded summaries at query time in
the retrieval layer, since that is query-time policy rather than search over
stored vectors.)
### Backends
`build_database(...)` selects backend by `database_config.metadata_store.provider`:
- `inmemory`: in-process dict/list state
- `sqlite`: SQLModel persistence, embeddings stored as JSON text, brute-force cosine search
- `postgres`: SQLModel persistence with pgvector support (when enabled), local fallback ranking when needed
`DatabaseConfig` auto-derives `vector_index`: `pgvector` for postgres, `bruteforce`
otherwise. For Postgres, startup runs migration bootstrap and attempts
`CREATE EXTENSION IF NOT EXISTS vector` in `ddl_mode="create"`.
### Track column and scope model
`RecallFile.track` (`"memory" | "skill"`, default `"memory"`) splits the two
tracks within one table. `get_or_create_category` keys on `(name, track, *scope)`,
and the Postgres unique-with-scope index is `(name, track, *scope)`, so a skill
and a memory category may share a name without colliding. Every `RecallFile`
reader scopes by track through the `where` filter rather than ad-hoc
post-filtering.
`UserConfig.model` is merged into record/table models so scope fields (for example
`user_id`) become first-class columns/attributes across resources, entries, files,
and relations. This is why `where` filters and `user_data` writes are consistently
available across APIs.
## LLM/provider architecture
LLM access is profile-based (`llm_profiles`):
- `default` profile for chat-like tasks
- `embedding` profile for embedding tasks (auto-derived from default if not set)
Per-step profile routing happens through step config (`chat_llm_profile`,
`embed_llm_profile`, or `llm_profile`).
Client backends (selected per profile via `client_backend`):
- `sdk`: official OpenAI SDK wrapper
- `anthropic`: official Anthropic/Claude SDK wrapper
- `httpx`: provider-adapted HTTP backend (OpenAI, Claude, Grok, DeepSeek, Kimi,
MiniMax, Doubao, OpenRouter — see `memu.llm.backends`)
- `lazyllm_backend`: LazyLLM adapter
### VLM (vision-language) architecture
Image/video understanding uses a dedicated `memu.vlm` package, a sibling of
`memu.llm` mirroring its layout (`backends/`, `*_client.py`, `gateway.py`),
scoped to the multimodal `vision` capability. Only providers whose first-party
API offers native image understanding are included (`memu.vlm.backends`:
OpenAI, Claude, Grok, Kimi, MiniMax, Doubao, OpenRouter); text-only providers
such as DeepSeek are intentionally excluded.
VLM profiles are **derived** from the LLM profiles (`vlm_config_from_llm`): each
VLM client reuses the matching LLM profile's provider, credentials, and
`client_backend`, swapping only the model for that provider's latest VLM
(`memu.vlm.VLM_PROVIDER_DEFAULTS`), falling back to the LLM chat model when the
provider has no known VLM. This makes vision work with zero extra configuration.
During `preprocess_multimodal`, `MemoryService` routes by modality: `image` and
`video` use the VLM client (`_get_vlm_client`, profile from
`memorize_config.vlm_profile`), while `conversation`/`document`/`audio` use the
chat LLM client. VLM clients are cached per profile and wrapped by the same
`LLMClientWrapper`, so interceptors and usage metadata behave identically.
### Embedding (vectorization) architecture
Vectorization uses a dedicated `memu.embedding` package, a sibling of `memu.llm`
and `memu.vlm` mirroring their layout (`backends/`, `*_client.py`/`openai_sdk.py`,
`gateway.py`, `defaults.py`), scoped to the `embed` capability used by vector
search. `memu.embedding.backends` covers OpenAI, Jina, Voyage, Doubao and
OpenRouter; the HTTP client falls back to an OpenAI-compatible backend for any
other provider.
Embedding is **fully decoupled** from the text/chat clients: `OpenAIClient`,
`HTTPLLMClient` and `AnthropicClient` no longer expose `embed()`. All
vectorization — query embedding, file/entry embedding, RAG ranking — flows
through `_get_step_embedding_client` / `_get_embedding_client`, which build
dedicated `memu.embedding` clients. (`LazyLLMClient` remains multi-capability, as
it is the embedding transport for the `lazyllm_backend`.)
Embedding profiles (`EmbeddingConfig`) are **derived** from the LLM profiles
(`embedding_config_from_llm`) by default, reusing each profile's provider,
credentials and transport. Prefer passing an explicit `embedding_profiles` to
`MemoryService` to point vectorization at a dedicated embedding provider (e.g.
`jina`/`voyage`) independently of the chat provider. Embedding clients are built
via `build_embedding_client` (`client_backend`: `sdk`/`httpx`/`lazyllm_backend`;
`anthropic` raises, as Claude has no embeddings API), cached per profile, and
wrapped by the same `LLMClientWrapper`. `_get_step_embedding_client` resolves the
profile from step config (`embed_llm_profile`, default `embedding`).
## Integration surfaces
- `memu.integrations.langgraph`: LangChain/LangGraph tool adapter
(`save_memory`, `search_memory`)
There is no built-in HTTP server or CLI in this repository; usage is programmatic
via `MemoryService` (the hosted Cloud API is a separate product).
## Memory file system export (`memu.memory_fs`)
`MemoryService.export_memory_files(...)` renders the structured store into a
browsable markdown tree. Each root index sits beside a sibling payload directory:
```txt
<output_dir>/
├── INDEX.md ← index of the raw files under resource/
├── MEMORY.md ← overview + index of memory/
├── SKILL.md ← overview + index of skill/
├── resource/
│ └── <file_name> ← one copied raw source file (verbatim bytes)
├── memory/
│ └── <slug>.md ← one memory-track RecallFile (description + content)
├── skill/
│ └── <slug>.md ← one skill-track RecallFile (description + content)
└── .memufs_manifest.json ← per-artifact content hashes (diff detection)
```
- `resource/` holds the raw source files copied verbatim out of the blob store
(`Resource.local_path`); `INDEX.md` indexes them (name, modality, description,
link).
- `memory/<slug>.md` is one file per `track="memory"` `RecallFile`; `MEMORY.md`
links to each one.
- `skill/<slug>.md` is one file per `track="skill"` `RecallFile`, rendered by the
**same** track-agnostic renderer as `memory/` (ADR 0006). Skills are now
first-class rows, not an export-time synthesis artifact — the old
`skill/<name>/SKILL.md` folder layout is gone.
### Deterministic vs. synthesized
`INDEX.md`, the `resource/` copies, and the per-file `memory/<slug>.md` /
`skill/<slug>.md` documents are **always deterministic** — recomputed from the
current store, no LLM. Rendered content avoids volatile values so an unchanged
store re-exports as a no-op.
The two root overviews are deterministic link indexes by default. When
`memory_files_config.synthesize=True`, `MemoryFilesBuilder` synthesizes their
bodies with the `synthesis_llm_profile` LLM:
- `MEMORY.md` is synthesized from the per-source multimodal descriptions
- `SKILL.md` is synthesized from the skill-track files' bodies (the mirror of
`MEMORY.md`)
### Initialize vs. incremental update
`MemoryFilesBuilder.build(database, where, changed=...)` decides between two paths:
- **Initialization** (no prior tree on disk, or `changed is None`): rebuild from
the full scoped store. `export_memory_files(user=...)` always takes this path.
- **Incremental update** (a tree already exists and a changed set is supplied):
read existing overview bodies back off disk and merge only the changed sources'
descriptions into them.
`memorize_workspace(...)` drives this builder after each folder sync — passing the
just-created resources as the changed set, or forcing a full rebuild when files
were modified/deleted. That refresh is best-effort: an export failure is logged
and never fails the sync, since the structured memory is already persisted. The
exporter is read-only against the database, disabled by default
(`memory_files_config.enabled`), and serialized through a per-service lock.
## Current constraints and tradeoffs
- workflow state is dict-based, so step contracts are validated by key names
rather than static types
- SQLite/inmemory vector search is brute-force (portable but less scalable)
- file/entry summary and extraction quality are prompt/LLM dependent
- the `dedupe_merge` stage is a placeholder (currently pass-through)
- **skill provenance/deletion is a known, deferred gap** (ADR 0006): skill
`RecallFile`s bypass the `RecallEntry` plane and have no link back to their
source resources, so they do not invalidate on source change/delete. The skill
track is currently append/merge-only.
## Related ADRs
- `docs/adr/0001-workflow-pipeline-architecture.md`
- `docs/adr/0002-pluggable-storage-and-vector-strategy.md`
- `docs/adr/0003-user-scope-in-data-model.md`
- `docs/adr/0004-workspace-memorize-and-memory-file-system.md`
- `docs/adr/0005-dedicated-embedding-package.md`
- `docs/adr/0006-from-memory-item-category-to-tracked-workspace-memorization.md`
docs/integrations/grok.md
+53
View File
@@ -0,0 +1,53 @@
# Grok (xAI) Integration
MemU supports **Grok**, the AI model from xAI, as a first-class LLM provider.
## Prerequisites
1. **xAI Account:** You need an active account with [xAI](https://x.ai/).
2. **API Key:** Obtain an API key from the [xAI Console](https://console.x.ai/).
## Configuration
To enable Grok, you need to set the `XAI_API_KEY` environment variable.
### Environment Variable
```bash
export XAI_API_KEY="your-xai-api-key-here"
```
PowerShell:
```powershell
$env:XAI_API_KEY="your-xai-api-key-here"
```
## Usage
To use Grok as your LLM provider, switch the `provider` setting to `grok`. This can be done in your configuration file or when initializing the application.
### Python Example
```python
from memu.app.settings import LLMConfig
# Configure MemU to use Grok
config = LLMConfig(
provider="grok",
# The default API key env var is XAI_API_KEY
# The default model is grok-2-latest
)
print(f"Using provider: {config.provider}")
print(f"Base URL: {config.base_url}")
print(f"Chat Model: {config.chat_model}")
```
## Models Supported
We currently support the following Grok models:
* **grok-2-latest** (Default)
The integration automatically sets the base URL to `https://api.x.ai/v1`.
docs/langgraph_integration.md
+97
View File
@@ -0,0 +1,97 @@
# MemU LangGraph Integration
The MemU LangGraph Integration provides a seamless adapter to expose MemU's powerful memory capabilities (`memorize` and `retrieve`) as standard [LangChain](https://python.langchain.com/) / [LangGraph](https://langchain-ai.github.io/langgraph/) tools. This allows your agents to persist information and recall it across sessions using MemU as the long-term memory backend.
## Overview
This integration wraps the `MemoryService` and exposes two key tools:
- **`save_memory`**: Persists text, conversation snippets, or facts associated with a user.
- **`search_memory`**: Retrieves relevant memories based on semantic search queries.
These tools are fully typed and compatible with LangGraph's `prebuilt.ToolNode` and LangChain's agents.
## Installation
To use this integration, you need to install the optional dependencies:
```bash
uv add langgraph langchain-core
```
## Quick Start
Here is a complete example of how to initialize the MemU memory service and bind it to a LangGraph agent.
```python
import asyncio
import os
from memu.app.service import MemoryService
from memu.integrations.langgraph import MemULangGraphTools
# Ensure you have your configuration set (e.g., env vars for DB connection)
# os.environ["MEMU_DATABASE_URL"] = "..."
async def main():
# 1. Initialize MemoryService
memory_service = MemoryService()
# If your service requires async init (check your specific implementation):
# await memory_service.initialize()
# 2. Instantiate MemULangGraphTools
memu_tools = MemULangGraphTools(memory_service)
# Get the list of tools (BaseTool compatible)
tools = memu_tools.tools()
# 3. Example Usage: Manually invoking a tool
# In a real app, you would pass 'tools' to your LangGraph agent or StateGraph.
# Save a memory
save_tool = memu_tools.save_memory_tool()
print("Saving memory...")
result = await save_tool.ainvoke({
"content": "The user prefers dark mode.",
"user_id": "user_123",
"metadata": {"category": "preferences"}
})
print(f"Save Result: {result}")
# Search for a memory
search_tool = memu_tools.search_memory_tool()
print("\nSearching memory...")
search_result = await search_tool.ainvoke({
"query": "What are the user's preferences?",
"user_id": "user_123"
})
print(f"Search Result:\n{search_result}")
if __name__ == "__main__":
asyncio.run(main())
```
## API Reference
### `MemULangGraphTools`
The main adapter class.
```python
class MemULangGraphTools(memory_service: MemoryService)
```
#### `save_memory_tool() -> StructuredTool`
Returns a tool named `save_memory`.
- **Inputs**: `content` (str), `user_id` (str), `metadata` (dict, optional).
- **Description**: Save a piece of information, conversation snippet, or memory for a user.
#### `search_memory_tool() -> StructuredTool`
Returns a tool named `search_memory`.
- **Inputs**: `query` (str), `user_id` (str), `limit` (int, default=5), `metadata_filter` (dict, optional), `min_relevance_score` (float, default=0.0).
- **Description**: Search for relevant memories or information for a user based on a query.
## Troubleshooting
### Import Errors
If you see an `ImportError` regarding `langchain_core` or `langgraph`:
1. Ensure you have installed the extras: `uv add langgraph langchain-core` (or `pip install langgraph langchain-core`).
2. Verify your virtual environment is active.
docs/providers/grok.md
+70
View File
@@ -0,0 +1,70 @@
# Grok (xAI) Provider
memU includes first-class support for [Grok](https://grok.x.ai/), allowing you to leverage xAI's powerful language models directly within your application.
## Prerequisites
To use this provider, you must have an active xAI account.
1. Navigate to the [xAI Console](https://console.x.ai/).
2. Sign up or log in.
3. Create a new **API Key** in the API Keys section.
## Configuration
The integration is designed to work out-of-the-box with minimal configuration.
### Environment Variables
Set the following environment variable in your `.env` file or system environment:
```bash
GROK_API_KEY=xai-YOUR_API_KEY_HERE
```
### Defaults
When you select the `grok` provider, memU automatically configures the following defaults:
* **Base URL**: `https://api.x.ai/v1`
* **Model**: `grok-2-latest`
## Usage Example
You can enable the Grok provider by setting the `provider` field to `"grok"` in your application configuration.
### Using Python Configuration
```python
import os
from memu.app.settings import LLMConfig
from memu.app.service import MemoryService
# Configure the LLM provider to use Grok
llm_config = LLMConfig(
provider="grok",
api_key=os.environ.get("GROK_API_KEY", ""),
)
# Initialize the service — pass config via llm_profiles, not llm_config
service = MemoryService(llm_profiles={"default": llm_config})
print(f"Service initialized with model: {llm_config.chat_model}")
# Output: Service initialized with model: grok-2-latest
```
## Troubleshooting
### Connection Issues
If you are unable to connect to the xAI API:
1. Verify that your `GROK_API_KEY` is set correctly and has not expired.
2. Ensure that the `base_url` is resolving to `https://api.x.ai/v1`. If you have manual overrides in your settings, they might be conflicting with the default.
### Model Availability
If you receive a `404` or "Model not found" error, xAI may have updated their model names. You can override the model manually in the config if needed:
```python
config = LLMConfig(
provider="grok",
chat_model="grok-beta" # Example override
)
```
docs/sealos-devbox-guide.md
+410
View File
@@ -0,0 +1,410 @@
# Deploying MemU on Sealos DevBox
This guide demonstrates how to build and deploy a **Personal AI Assistant with Long-Term Memory** using MemU on [Sealos DevBox](https://sealos.io/products/devbox).
## Overview
MemU enables AI agents to maintain persistent, structured memory across conversations. Combined with Sealos DevBox's 1-click cloud development environment, you can quickly build and deploy memory-enabled AI applications.
**What we'll build:**
- A FastAPI-based AI assistant that remembers user preferences and past conversations
- Persistent memory storage using MemU's in-memory or PostgreSQL backend
- Simple REST API for chat interactions
- One-click deployment to production
**Time to complete:** ~15 minutes
## Prerequisites
- [Sealos account](https://sealos.io) (free tier available)
- OpenAI API key (or compatible provider like Nebius, Groq)
## Step 1: Create a DevBox Environment
1. Log in to [Sealos Dashboard](https://cloud.sealos.io)
2. Navigate to **DevBox** module
3. Click **Create New Project**
4. Select **Python 3.11+** template
5. Configure resources (recommended: 2 vCPU, 4GB RAM)
6. Click **Create** - your environment will be ready in ~60 seconds
## Step 2: Connect Your IDE
1. In the DevBox project list, click the **VS Code** or **Cursor** button
2. Your local IDE will open with a secure SSH connection to the cloud environment
3. All code runs in the cloud, keeping your local machine free
## Step 3: Set Up the Project
Open the terminal in your connected IDE and run:
```bash
# Clone or create project directory
mkdir memu-assistant && cd memu-assistant
# Create virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install memu fastapi uvicorn python-dotenv
```
## Step 4: Create the Application
Create the following files in your project:
### `.env`
```env
# LLM Provider Configuration
OPENAI_API_KEY=your_api_key_here
OPENAI_BASE_URL=https://api.openai.com/v1
# Or use Nebius (OpenAI-compatible)
# OPENAI_API_KEY=your_nebius_key
# OPENAI_BASE_URL=https://api.tokenfactory.nebius.com/v1/
# Model Configuration
CHAT_MODEL=gpt-4o-mini
EMBED_MODEL=text-embedding-3-small
# Server Configuration
HOST=0.0.0.0
PORT=8000
```
### `main.py`
```python
"""
Personal AI Assistant with Long-Term Memory
Powered by MemU + FastAPI on Sealos DevBox
"""
import os
from contextlib import asynccontextmanager
from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
load_dotenv()
# MemU imports
from memu.app import MemoryService
# Global memory service
memory_service: MemoryService | None = None
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Initialize MemU on startup."""
global memory_service
llm_profiles = {
"default": {
"provider": "openai",
"base_url": os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1"),
"api_key": os.getenv("OPENAI_API_KEY"),
"chat_model": os.getenv("CHAT_MODEL", "gpt-4o-mini"),
"client_backend": "sdk",
},
"embedding": {
"provider": "openai",
"base_url": os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1"),
"api_key": os.getenv("OPENAI_API_KEY"),
"embed_model": os.getenv("EMBED_MODEL", "text-embedding-3-small"),
"client_backend": "sdk",
},
}
memory_service = MemoryService(llm_profiles=llm_profiles)
print("✓ MemU Memory Service initialized")
yield
print("Shutting down...")
app = FastAPI(
title="MemU Assistant",
description="AI Assistant with Long-Term Memory",
lifespan=lifespan,
)
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
class ChatRequest(BaseModel):
message: str
user_id: str = "default"
class ChatResponse(BaseModel):
response: str
memories_used: int
memories_stored: int
class MemorizeRequest(BaseModel):
content: str
user_id: str = "default"
@app.get("/")
async def root():
return {
"service": "MemU Assistant",
"status": "running",
"endpoints": ["/chat", "/memorize", "/recall", "/health"],
}
@app.get("/health")
async def health():
return {"status": "healthy", "memory_service": memory_service is not None}
@app.post("/chat", response_model=ChatResponse)
async def chat(request: ChatRequest):
"""
Chat with the AI assistant. The assistant will:
1. Retrieve relevant memories from past conversations
2. Generate a response using those memories as context
3. Store new information from the conversation
"""
if not memory_service:
raise HTTPException(status_code=503, detail="Memory service not initialized")
# Step 1: Retrieve relevant memories
retrieve_result = await memory_service.retrieve(
queries=[{"role": "user", "content": request.message}]
)
memories = retrieve_result.get("items", [])
memories_context = ""
if memories:
memories_context = "\n\nRelevant memories from past conversations:\n"
for mem in memories[:5]: # Limit to top 5 memories
if isinstance(mem, dict):
memories_context += f"- {mem.get('summary', str(mem))}\n"
# Step 2: Generate response (simplified - in production, use full LLM call)
# For demo, we'll create a simple response acknowledging the memories
response_text = f"I received your message: '{request.message}'"
if memories:
response_text += f"\n\nI found {len(memories)} relevant memories that might help."
# Step 3: Store the conversation as a new memory
import tempfile
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f:
f.write(f"User ({request.user_id}): {request.message}")
temp_file = f.name
try:
memorize_result = await memory_service.memorize(
resource_url=temp_file,
modality="text",
)
memories_stored = len(memorize_result.get("items", []))
finally:
os.unlink(temp_file)
return ChatResponse(
response=response_text,
memories_used=len(memories),
memories_stored=memories_stored,
)
@app.post("/memorize")
async def memorize(request: MemorizeRequest):
"""Store information in long-term memory."""
if not memory_service:
raise HTTPException(status_code=503, detail="Memory service not initialized")
import tempfile
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f:
f.write(request.content)
temp_file = f.name
try:
result = await memory_service.memorize(
resource_url=temp_file,
modality="text",
)
return {
"status": "stored",
"items_created": len(result.get("items", [])),
"categories": len(result.get("categories", [])),
}
finally:
os.unlink(temp_file)
@app.get("/recall")
async def recall(query: str, limit: int = 5):
"""Recall memories related to a query."""
if not memory_service:
raise HTTPException(status_code=503, detail="Memory service not initialized")
result = await memory_service.retrieve(
queries=[{"role": "user", "content": query}]
)
items = result.get("items", [])[:limit]
return {
"query": query,
"memories_found": len(items),
"memories": [
{"summary": item.get("summary", str(item)) if isinstance(item, dict) else str(item)}
for item in items
],
}
if __name__ == "__main__":
import uvicorn
uvicorn.run(
"main:app",
host=os.getenv("HOST", "0.0.0.0"),
port=int(os.getenv("PORT", 8000)),
reload=True,
)
```
### `requirements.txt`
```
memu>=0.1.0
fastapi>=0.100.0
uvicorn[standard]>=0.23.0
python-dotenv>=1.0.0
```
### `entrypoint.sh`
```bash
#!/bin/bash
source venv/bin/activate
uvicorn main:app --host 0.0.0.0 --port 8000
```
## Step 5: Test Locally in DevBox
```bash
# Run the application
python main.py
```
Use the DevBox preview feature to access your running application, or test with curl:
```bash
# Health check
curl http://localhost:8000/health
# Store a memory
curl -X POST http://localhost:8000/memorize \
-H "Content-Type: application/json" \
-d '{"content": "User prefers dark mode and uses Python for AI development"}'
# Chat with memory
curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{"message": "What programming language do I use?"}'
# Recall memories
curl "http://localhost:8000/recall?query=programming%20preferences"
```
## Step 6: Deploy to Production
1. In the Sealos Dashboard, go to your DevBox project
2. Click **Create Release** to package your application
3. Click **Deploy** next to your release
4. Configure environment variables (OPENAI_API_KEY, etc.)
5. Click **Deploy** - your app will be live in minutes!
Your application will receive a public URL like: `https://your-app.cloud.sealos.io`
## Using with PostgreSQL (Optional)
For production deployments with persistent storage:
1. In Sealos Dashboard, go to **Database** module
2. Create a PostgreSQL instance
3. Update your `.env` with the connection string:
```env
DATABASE_URL=postgresql://user:password@host:5432/memu
```
4. Update `main.py` to use PostgreSQL backend (see MemU documentation)
## API Reference
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/` | GET | Service info |
| `/health` | GET | Health check |
| `/chat` | POST | Chat with memory-aware AI |
| `/memorize` | POST | Store information in memory |
| `/recall` | GET | Query stored memories |
## Architecture
```
┌─────────────────────────────────────────────────────────┐
│ Sealos DevBox │
│ ┌─────────────────────────────────────────────────┐ │
│ │ FastAPI Application │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────────────┐ │ │
│ │ │ /chat │ │/memorize│ │ /recall │ │ │
│ │ └────┬────┘ └────┬────┘ └────────┬────────┘ │ │
│ │ │ │ │ │ │
│ │ └────────────┼────────────────┘ │ │
│ │ │ │ │
│ │ ┌───────▼───────┐ │ │
│ │ │ MemU Service │ │ │
│ │ │ (Memory Mgmt)│ │ │
│ │ └───────┬───────┘ │ │
│ │ │ │ │
│ │ ┌────────────┼────────────┐ │ │
│ │ │ │ │ │ │
│ │ ┌────▼────┐ ┌────▼────┐ ┌───▼────┐ │ │
│ │ │ Vector │ │ LLM │ │Postgres│ │ │
│ │ │ Store │ │ API │ │(opt.) │ │ │
│ │ └─────────┘ └─────────┘ └────────┘ │ │
│ └─────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
```
## Benefits of This Setup
- **Zero Infrastructure Management**: Sealos handles Kubernetes complexity
- **Instant Environment**: Ready-to-code in 60 seconds
- **Persistent Memory**: MemU maintains context across sessions
- **Scalable**: Easily scale resources as needed
- **Cost-Effective**: Pay only for what you use
## Next Steps
- Add authentication for multi-user support
- Integrate with Slack, Discord, or other platforms
- Use PostgreSQL for production-grade persistence
- Add conversation history UI
## Resources
- [MemU Documentation](https://github.com/NevaMind-AI/memU)
- [Sealos DevBox Guide](https://sealos.io/blog/how-to-setup-devbox)
- [FastAPI Documentation](https://fastapi.tiangolo.com)
---
*This guide was created for the MemU PR Hackathon - 2026 New Year Challenge (Issue #228)*
docs/sealos_use_case.md
+54
View File
@@ -0,0 +1,54 @@
# 🛡️ Context-Aware Support Agent (Sealos Edition)
## Overview
This use case demonstrates how **MemU** enables a support agent to remember user history across sessions, deployed on a **Sealos Devbox** environment.
Unlike a standard web app, this demo focuses on the **backend memory orchestration**. It runs as a **CLI (Command Line Interface)** tool to transparently show the internal memory logs, retrieval process, and state persistence without the abstraction layer of a UI.
## 🚀 Quick Start
### Prerequisites
- Sealos Devbox Environment
- Python 3.13+
- MemU Library (installed via `make install`)
### How to Run the Demo
Since this is a backend demonstration, you will run the agent directly in the terminal to observe the memory cycle.
```bash
uv run python examples/sealos_support_agent.py
```
## 📸 Live Demo Output (Proof of Concept)
Below is the actual output captured from the Sealos terminal. This serves as verification of the "Demonstration Quality" requirement.
```plaintext
🚀 Starting Sealos Support Agent Demo (Offline Mode)
📝 --- Phase 1: Ingesting Conversation History ---
👤 Captain: "I'm getting a 502 Bad Gateway error on port 3000."
🤖 Agent: (Memorizing this interaction...)
✅ Memory stored! extracted 2 items.
- [profile] Captain reported a 502 Bad Gateway error on port 3000.
🔍 --- Phase 2: Retrieval on New Interaction ---
👤 Captain: "Hello"
🤖 Agent: (Searching memory for context...)
💡 Retrieved Context:
Found Memory: Captain reported a 502 Bad Gateway error on port 3000.
💬 --- Phase 3: Agent Response ---
🤖 Agent: "Welcome back, Captain. I see you had a 502 error on port 3000 recently. Is that resolved?"
✨ Demo Completed Successfully
```
## 💡 Code Highlights & Justification
- **CLI vs Web**: We chose a CLI implementation to provide clear visibility into the memory ingestion and retrieval logs, which are often hidden in web implementations.
- **MockLLM**: Includes a MockLLM class to ensure the demo is 100% reproducible by reviewers without needing external API keys.
- **Sealos Native**: Optimized to run within the ephemeral Sealos Devbox container lifecycle.
docs/sqlite.md
+230
View File
@@ -0,0 +1,230 @@
# SQLite Database Integration
MemU supports SQLite as a lightweight, file-based database backend for memory storage. This is ideal for:
- **Local development** and testing
- **Single-user applications** with persistent storage
- **Portable deployments** where you need a simple database solution
- **Offline-capable applications** that can't rely on external databases
## Quick Start
### Basic Configuration
```python
from memu.app import MemoryService
# Using default SQLite file (memu.db in current directory)
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
},
},
)
# Or specify a custom database path
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///path/to/your/memory.db",
},
},
)
```
### In-Memory SQLite (No Persistence)
For testing or temporary storage, you can use an in-memory SQLite database:
```python
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///:memory:",
},
},
)
```
## Configuration Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `provider` | `str` | `"inmemory"` | Set to `"sqlite"` to use SQLite backend |
| `dsn` | `str` | `"sqlite:///memu.db"` | SQLite connection string |
### DSN Format
SQLite DSN follows this format:
- **File-based**: `sqlite:///path/to/database.db`
- **In-memory**: `sqlite:///:memory:`
- **Relative path**: `sqlite:///./data/memu.db`
- **Absolute path**: `sqlite:////home/user/data/memu.db` (note the 4 slashes)
## Vector Search
SQLite doesn't have native vector support like PostgreSQL's pgvector. MemU uses **brute-force cosine similarity** for vector search when using SQLite:
```python
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///memu.db",
},
"vector_index": {
"provider": "bruteforce", # This is the default for SQLite
},
},
)
```
**Note**: Brute-force search loads all embeddings into memory and computes similarity for each. This works well for moderate dataset sizes (up to ~100k items) but may be slow for larger datasets.
## Database Schema
SQLite creates the following tables automatically:
- `sqlite_resources` - Multimodal resource records (images, documents, etc.)
- `sqlite_memory_items` - Extracted memory items with embeddings
- `sqlite_memory_categories` - Memory categories with summaries
- `sqlite_category_items` - Relationships between items and categories
Embeddings are stored as JSON-serialized text in SQLite since there's no native vector type.
## Data Import/Export
### Export Data
You can export your SQLite database for backup or migration:
```python
import shutil
# Simply copy the database file
shutil.copy("memu.db", "memu_backup.db")
```
### Import from SQLite to PostgreSQL
To migrate data from SQLite to PostgreSQL:
```python
import json
from memu.database.sqlite import build_sqlite_database
from memu.database.postgres import build_postgres_database
from memu.app.settings import DatabaseConfig
from pydantic import BaseModel
class UserScope(BaseModel):
user_id: str
# Load from SQLite
sqlite_config = DatabaseConfig(
metadata_store={"provider": "sqlite", "dsn": "sqlite:///memu.db"}
)
sqlite_db = build_sqlite_database(config=sqlite_config, user_model=UserScope)
sqlite_db.load_existing()
# Connect to PostgreSQL
postgres_config = DatabaseConfig(
metadata_store={"provider": "postgres", "dsn": "postgresql://..."}
)
postgres_db = build_postgres_database(config=postgres_config, user_model=UserScope)
# Migrate resources
for res_id, resource in sqlite_db.resources.items():
postgres_db.resource_repo.create_resource(
url=resource.url,
modality=resource.modality,
local_path=resource.local_path,
caption=resource.caption,
embedding=resource.embedding,
user_data={"user_id": getattr(resource, "user_id", None)},
)
# Similar for categories, items, and relations...
```
## Performance Considerations
| Aspect | SQLite | PostgreSQL |
|--------|--------|------------|
| Setup | Zero configuration | Requires server setup |
| Concurrency | Single writer, multiple readers | Full concurrent access |
| Vector Search | Brute-force (in-memory) | Native pgvector (indexed) |
| Scale | Up to ~100k items | Millions of items |
| Deployment | Single file, portable | External service |
## Example: Full Workflow
```python
import asyncio
from memu.app import MemoryService
async def main():
# Initialize with SQLite
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///my_memories.db",
},
},
)
# Memorize a conversation
result = await service.memorize(
resource_url="conversation.json",
modality="conversation",
user={"user_id": "alice"},
)
print(f"Created {len(result['categories'])} categories")
# Retrieve relevant memories
memories = await service.retrieve(
queries=[
{"role": "user", "content": {"text": "What are my preferences?"}}
],
where={"user_id": "alice"},
)
for item in memories.get("items", []):
print(f"- {item['summary']}")
asyncio.run(main())
```
## Troubleshooting
### Database Locked Error
SQLite only allows one writer at a time. If you see "database is locked" errors:
1. Ensure you're not running multiple processes writing to the same database
2. Consider using PostgreSQL for concurrent access needs
3. Use connection pooling with appropriate timeouts
### Permission Denied
Make sure the directory containing the SQLite file is writable:
```bash
chmod 755 /path/to/data/directory
```
### Slow Vector Search
If vector search is slow with large datasets:
1. Consider migrating to PostgreSQL with pgvector
2. Use more selective `where` filters to reduce the search space
3. Reduce `top_k` parameters in your retrieve configuration
docs/tutorials/getting_started.md
+181
View File
@@ -0,0 +1,181 @@
# Quickstart: Adding Long-Term Memory to Python Agents
Welcome to MemU! This guide will help you add robust long-term memory capabilities to your Python agents in just a few minutes. Without MemU, LLMs are limited by their context window. MemU solves this by providing an intelligent, persistent memory layer.
## Prerequisites
Before we begin, ensure you have the following:
- **Python 3.13+**: MemU takes advantage of modern Python features.
- **OpenAI API Key**: This quickstart uses OpenAI's models (`gpt-4o-mini`). You will need a valid API key.
## Step-by-Step Guide
### 1. Installation
Install MemU using `pip` or `uv`:
```bash
pip install memu
# OR
uv add memu
```
### 2. Configuration
MemU requires an LLM backend to function. By default, it looks for the `OPENAI_API_KEY` environment variable.
**Linux / macOS / Git Bash:**
```bash
export OPENAI_API_KEY=sk-proj-your-api-key
```
**Windows (PowerShell):**
```powershell
$env:OPENAI_API_KEY="sk-proj-your-api-key"
```
### 3. The Robust Starter Script
Below is a complete, production-ready script that demonstrates the full lifecycle of a memory-enabled agent: **Initialization**, **Injection** (adding memory), and **Retrieval** (searching memory).
Create a file named `getting_started.py` and paste the following code:
```python
"""
Getting Started with MemU: A Robust Example.
This script demonstrates the core lifecycle of MemU:
1. **Initialization**: Setting up the client with secure API key handling.
2. **Memory Injection**: Adding a specific memory with metadata.
3. **Retrieval**: Searching for that memory using natural language.
4. **Error Handling**: Catching common configuration issues.
Usage:
export OPENAI_API_KEY=your_api_key_here
python getting_started.py
"""
import asyncio
import logging
import os
import sys
from memu.app import MemoryService
# Configure logging to show info but suppress noisy libraries
logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
logging.getLogger("httpx").setLevel(logging.WARNING)
async def main() -> None:
"""Run the MemU lifecycle demonstration."""
print(">>> MemU Getting Started Example")
print("-" * 30)
# 1. API Key Handling
# MemU relies on an LLM backend (defaulting to OpenAI).
# We ensure the API key is present before proceeding.
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
print("[!] Error: OPENAI_API_KEY environment variable is not set.")
print("Please export it: export OPENAI_API_KEY=sk-...")
return
try:
# 2. Initialization
# We initialize the MemoryService with:
# - llm_profiles: Configuration for the LLM (model, api_key).
# - memorize_config: Pre-defining a memory category ensures we can organize memories efficiently.
print(f"[*] Initializing MemoryService with model: gpt-4o-mini...")
service = MemoryService(
llm_profiles={
"default": {
"api_key": api_key,
"chat_model": "gpt-4o-mini",
},
},
memorize_config={
"memory_categories": [
{
"name": "User Facts",
"description": "General and specific facts known about the user preference and identity.",
}
]
},
)
print("[OK] Service initialized successfully.\n")
# 3. Memory Injection
# We manually inject a memory into the system.
# This is useful for bootstrapping a user profile or adding explicit knowledge.
print("[*] Injecting memory...")
memory_content = "The user is a senior Python architect who loves clean code and type hints."
# We use 'create_memory_item' to insert a single memory record.
# memory_type='profile' indicates this is an attribute of the user.
result = await service.create_memory_item(
memory_type="profile",
memory_content=memory_content,
memory_categories=["User Facts"],
)
print(f"[OK] Memory created! ID: {result.get('memory_item', {}).get('id')}\n")
# 4. Retrieval
# Now we query the system naturally to see if it recalls the information.
query_text = "What kind of code does the user like?"
print(f"[*] Querying: '{query_text}'")
search_results = await service.retrieve(
queries=[{"role": "user", "content": query_text}]
)
# 5. Display Results
items = search_results.get("items", [])
if items:
print(f"[OK] Found {len(items)} relevant memory item(s):")
for idx, item in enumerate(items, 1):
print(f" {idx}. {item.get('summary')} (Type: {item.get('memory_type')})")
else:
print("[!] No relevant memories found.")
except Exception as e:
print(f"\n[!] An error occurred during execution: {e}")
logging.exception("Detailed traceback:")
finally:
print("\n[=] Example execution finished.")
if __name__ == "__main__":
asyncio.run(main())
```
### Understanding the Code
1. **Initialization**: We configure `MemoryService` with specific `llm_profiles`. This tells MemU which model to use. We also define a `memorize_config` with a "User Facts" category. Categories help the LLM organize and retrieve information more effectively.
2. **Memory Injection**: `create_memory_item` is used to explicitly add a piece of knowledge. We tag it with `memory_type="profile"` to semantically indicate this is a user attribute.
3. **Retrieval**: We use `retrieve` with a natural language query. MemU's internal workflow ("RAG" or "LLM" based) will determine the best way to find relevant memories.
## Troubleshooting
### `[!] Error: OPENAI_API_KEY environment variable is not set.`
This is the most common issue. It means the script cannot find your API key which is required to communicate with OpenAI.
**Solution:**
Ensure you have exported the key in your **current terminal session**.
- **Windows PowerShell**: `$env:OPENAI_API_KEY="sk-..."`
- **Linux/Mac**: `export OPENAI_API_KEY=sk-...`
Also, verify that you didn't accidentally include spaces around the `=` sign in bash.
## Next Steps
Now that you have the basics running, consider exploring:
- **Core Concepts**: Learn about `MemoryService`, `MemoryItem`, and `MemoryCategory`.
- **Advanced Configuration**: Switch to local LLMs or use different vector stores.
- **Integrations**: Connect MemU to your existing agent framework.
## Community Resources
This tutorial was created as part of the MemU 2026 Challenge. For a summary of the architectural analysis, see the author's [LinkedIn Post](https://www.linkedin.com/posts/david-a-mamani-c_github-nevamind-aimemu-memory-infrastructure-activity-7418493617482207232-_MtG?utm_source=share&utm_medium=member_desktop&rcm=ACoAAFdc0CIB__DJovR2t1BOxxJ6tgEeOqVEgx4).
examples/example_1_conversation_memory.py
+117
View File
@@ -0,0 +1,117 @@
"""
Example 1: Multiple Conversations -> Memory Category File
This example demonstrates how to process multiple conversation files
and generate a memory category JSON file.
Usage:
export OPENAI_API_KEY=your_api_key
python examples/example_1_conversation_memory.py
"""
import asyncio
import os
import sys
from memu.app import MemoryService
# Add src to sys.path
src_path = os.path.abspath("src")
sys.path.insert(0, src_path)
async def generate_memory_md(categories, output_dir):
"""Generate concise markdown files for each memory category."""
os.makedirs(output_dir, exist_ok=True)
generated_files = []
for cat in categories:
name = cat.get("name", "unknown")
summary = cat.get("summary", "")
filename = f"{name}.md"
filepath = os.path.join(output_dir, filename)
with open(filepath, "w", encoding="utf-8") as f:
# Title
# Content - concise version
if summary:
cleaned_summary = summary.replace("<content>", "").replace("</content>", "").strip()
f.write(f"{cleaned_summary}\n")
else:
f.write("*No content available*\n")
generated_files.append(filename)
return generated_files
async def main():
"""
Process multiple conversation files and generate memory categories.
This example:
1. Initializes MemoryService with OpenAI API
2. Processes conversation JSON files
3. Extracts memory categories from conversations
4. Outputs the categories to files
"""
print("Example 1: Conversation Memory Processing")
print("-" * 50)
# Get OpenAI API key from environment
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
msg = "Please set OPENAI_API_KEY environment variable"
raise ValueError(msg)
# Initialize service with OpenAI using llm_profiles
# The "default" profile is required and used as the primary LLM configuration
service = MemoryService(
llm_profiles={
"default": {
"api_key": api_key,
"chat_model": "gpt-4o-mini",
},
},
)
# Conversation files to process
conversation_files = [
"examples/resources/conversations/conv1.json",
"examples/resources/conversations/conv2.json",
"examples/resources/conversations/conv3.json",
]
# Process each conversation
print("\nProcessing conversations...")
total_items = 0
categories = []
for conv_file in conversation_files:
if not os.path.exists(conv_file):
continue
try:
result = await service.memorize(resource_url=conv_file, modality="conversation")
total_items += len(result.get("items", []))
# Categories are returned in the result and updated after each memorize call
categories = result.get("categories", [])
except Exception as e:
print(f"Error: {e}")
# Write to output files
output_dir = "examples/output/conversation_example"
os.makedirs(output_dir, exist_ok=True)
# 1. Generate individual Markdown files for each category
await generate_memory_md(categories, output_dir)
print(f"\n✓ Processed {len(conversation_files)} files, extracted {total_items} items")
print(f"✓ Generated {len(categories)} categories")
print(f"✓ Output: {output_dir}/")
if __name__ == "__main__":
asyncio.run(main())
examples/example_2_skill_extraction.py
+274
View File
@@ -0,0 +1,274 @@
"""
Example 2: Workflow & Agent Logs -> Skill Extraction
This example demonstrates how to extract skills from workflow descriptions
and agent runtime logs, then output them to a Markdown file.
Usage:
export OPENAI_API_KEY=your_api_key
python examples/example_2_skill_extraction.py
"""
import asyncio
import os
import sys
from openai import AsyncOpenAI
from memu.app import MemoryService
# Add src to sys.path
src_path = os.path.abspath("src")
sys.path.insert(0, src_path)
async def generate_skill_md(
all_skills, service, output_file, attempt_number, total_attempts, categories=None, is_final=False
):
"""
Use LLM to generate a concise task execution guide (skill.md).
This creates a production-ready guide incorporating lessons learned from deployment attempts.
"""
os.makedirs(os.path.dirname(output_file), exist_ok=True)
# Prepare context for LLM
skills_text = "\n\n".join([f"### From {skill_data['source']}\n{skill_data['skill']}" for skill_data in all_skills])
# Get category summaries if available
categories_text = ""
if categories:
categories_with_content = [cat for cat in categories if cat.get("summary") and cat.get("summary").strip()]
if categories_with_content:
categories_text = "\n\n".join([
f"**{cat.get('name', 'unknown')}**:\n{cat.get('summary', '')}" for cat in categories_with_content
])
# Construct prompt for LLM
prompt = f"""Generate a concise production-ready task execution guide.
**Context**:
- Task: Production Microservice Deployment with Blue-Green Strategy
- Progress: {attempt_number}/{total_attempts} attempts
- Status: {"Complete" if is_final else f"v0.{attempt_number}"}
**Skills Learned**:
{skills_text}
{f"**Categories**:\n{categories_text}" if categories_text else ""}
**Required Structure**:
1. **Frontmatter** (YAML):
- name: production-microservice-deployment
- description: Brief description
- version: {"1.0.0" if is_final else f"0.{attempt_number}.0"}
- status: {"Production-Ready" if is_final else "Evolving"}
2. **Introduction**: What this guide does and when to use it
3. **Deployment Context**: Strategy, environment, goals
4. **Pre-Deployment Checklist**:
- Actionable checks from lessons learned
- Group by category (Database, Monitoring, etc.)
- Mark critical items
5. **Deployment Procedure**:
- Step-by-step instructions with commands
- Include monitoring points
6. **Rollback Procedure**:
- When to rollback (thresholds)
- Exact commands
- Expected recovery time
7. **Common Pitfalls & Solutions**:
- Failures/issues encountered
- Root cause, symptoms, solution
8. **Best Practices**:
- What works well
- Expected timelines
9. **Key Takeaways**: 3-5 most important lessons
**Style**:
- Use markdown with clear hierarchy
- Be specific and concise
- Technical and production-grade tone
- Focus on PRACTICAL steps
**CRITICAL**:
- ONLY use information from provided skills/lessons
- DO NOT make assumptions or add generic advice
- Extract ACTUAL experiences from the logs
Generate the complete markdown document now:"""
client = AsyncOpenAI(api_key=service.llm_config.api_key)
response = await client.chat.completions.create(
model=service.llm_config.chat_model,
messages=[
{
"role": "system",
"content": "You are an expert technical writer creating concise, production-grade deployment guides from real experiences.",
},
{"role": "user", "content": prompt},
],
temperature=0.7,
max_tokens=3000,
)
generated_content = response.choices[0].message.content
# Write to file
with open(output_file, "w", encoding="utf-8") as f:
f.write(generated_content)
return True
async def main():
"""
Extract skills from agent logs using incremental memory updates.
This example demonstrates INCREMENTAL LEARNING:
1. Process files ONE BY ONE
2. Each file UPDATES existing memory
3. Category summaries EVOLVE with each new file
4. Final output shows accumulated knowledge
"""
print("Example 2: Incremental Skill Extraction")
print("-" * 50)
# Get OpenAI API key from environment
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
msg = "Please set OPENAI_API_KEY environment variable"
raise ValueError(msg)
# Custom config for skill extraction
skill_prompt = """
You are analyzing an agent execution log. Extract the key actions taken, their outcomes, and lessons learned.
For each significant action or phase:
1. **Action/Phase**: What was being attempted?
2. **Status**: SUCCESS ✅ or FAILURE ❌
3. **What Happened**: What was executed
4. **Outcome**: What worked/failed, metrics
5. **Root Cause** (for failures): Why did it fail?
6. **Lesson**: What did we learn?
7. **Action Items**: Concrete steps for next time
**IMPORTANT**:
- Focus on ACTIONS and outcomes
- Be specific: include actual metrics, errors, timing
- ONLY extract information explicitly stated
- DO NOT infer or assume information
Extract ALL significant actions from the text:
Text: {resource}
"""
# Define custom categories
skill_categories = [
{"name": "deployment_execution", "description": "Deployment actions, traffic shifting, environment management"},
{
"name": "pre_deployment_validation",
"description": "Capacity validation, configuration checks, readiness verification",
},
{
"name": "incident_response_rollback",
"description": "Incident response, error detection, rollback procedures",
},
{
"name": "performance_monitoring",
"description": "Metrics monitoring, performance analysis, bottleneck detection",
},
{"name": "database_management", "description": "Database capacity planning, optimization, schema changes"},
{"name": "testing_verification", "description": "Testing, smoke tests, load tests, verification"},
{"name": "infrastructure_setup", "description": "Kubernetes, containers, networking configuration"},
{"name": "lessons_learned", "description": "Key reflections, root cause analyses, action items"},
]
memorize_config = {
"memory_types": ["skill"],
"memory_type_prompts": {"skill": skill_prompt},
"memory_categories": skill_categories,
}
# Initialize service with OpenAI using llm_profiles
# The "default" profile is required and used as the primary LLM configuration
service = MemoryService(
llm_profiles={
"default": {
"api_key": api_key,
"chat_model": "gpt-4o-mini",
},
},
memorize_config=memorize_config,
)
# Resources to process
resources = [
("examples/resources/logs/log1.txt", "document"),
("examples/resources/logs/log2.txt", "document"),
("examples/resources/logs/log3.txt", "document"),
]
# Process each resource sequentially
print("\nProcessing files...")
all_skills = []
categories = []
for idx, (resource_file, modality) in enumerate(resources, 1):
if not os.path.exists(resource_file):
continue
try:
result = await service.memorize(resource_url=resource_file, modality=modality)
# Extract skill items
for item in result.get("items", []):
if item.get("memory_type") == "skill":
all_skills.append({"skill": item.get("summary", ""), "source": os.path.basename(resource_file)})
# Categories are returned in the result and updated after each memorize call
categories = result.get("categories", [])
# Generate intermediate skill.md
await generate_skill_md(
all_skills=all_skills,
service=service,
output_file=f"examples/output/skill_example/log_{idx}.md",
attempt_number=idx,
total_attempts=len(resources),
categories=categories,
)
except Exception as e:
print(f"Error: {e}")
# Generate final comprehensive skill.md
await generate_skill_md(
all_skills=all_skills,
service=service,
output_file="examples/output/skill_example/skill.md",
attempt_number=len(resources),
total_attempts=len(resources),
categories=categories,
is_final=True,
)
print(f"\n✓ Processed {len(resources)} files, extracted {len(all_skills)} skills")
print(f"✓ Generated {len(categories)} categories")
print("✓ Output: examples/output/skill_example/")
if __name__ == "__main__":
asyncio.run(main())
examples/example_3_multimodal_memory.py
+137
View File
@@ -0,0 +1,137 @@
"""
Example 3: Multimodal Processing -> Memory Category File
This example demonstrates how to process multiple modalities (images, documents)
and generate a unified memory category JSON file.
Usage:
export OPENAI_API_KEY=your_api_key
python examples/example_3_multimodal_memory.py
"""
import asyncio
import os
import sys
from memu.app import MemoryService
# Add src to sys.path
src_path = os.path.abspath("src")
sys.path.insert(0, src_path)
async def generate_memory_md(categories, output_dir):
"""Generate concise markdown files for each memory category."""
os.makedirs(output_dir, exist_ok=True)
generated_files = []
for cat in categories:
name = cat.get("name", "unknown")
description = cat.get("description", "")
summary = cat.get("summary", "")
filename = f"{name}.md"
filepath = os.path.join(output_dir, filename)
with open(filepath, "w", encoding="utf-8") as f:
# Title
formatted_name = name.replace("_", " ").title()
f.write(f"# {formatted_name}\n\n")
if description:
f.write(f"*{description}*\n\n")
# Content - full version
if summary:
cleaned_summary = summary.replace("<content>", "").replace("</content>", "").strip()
f.write(f"{cleaned_summary}\n")
else:
f.write("*No content available*\n")
generated_files.append(filename)
return generated_files
async def main():
"""
Process multiple modalities (images and documents) to generate memory categories.
This example:
1. Initializes MemoryService with OpenAI API
2. Processes documents and images
3. Extracts unified memory categories across modalities
4. Outputs the categories to files
"""
print("Example 3: Multimodal Memory Processing")
print("-" * 50)
# Get OpenAI API key from environment
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
msg = "Please set OPENAI_API_KEY environment variable"
raise ValueError(msg)
# Define custom categories for multimodal content
multimodal_categories = [
{"name": "technical_documentation", "description": "Technical documentation, guides, and tutorials"},
{
"name": "architecture_concepts",
"description": "System architecture, design patterns, and structural concepts",
},
{"name": "best_practices", "description": "Best practices, recommendations, and guidelines"},
{"name": "code_examples", "description": "Code snippets, examples, and implementation details"},
{"name": "visual_diagrams", "description": "Visual concepts, diagrams, charts, and illustrations from images"},
]
# Initialize service with OpenAI using llm_profiles
# The "default" profile is required and used as the primary LLM configuration
service = MemoryService(
llm_profiles={
"default": {
"api_key": api_key,
"chat_model": "gpt-4o-mini",
},
},
memorize_config={"memory_categories": multimodal_categories},
)
# Resources to process (file_path, modality)
resources = [
("examples/resources/docs/doc1.txt", "document"),
("examples/resources/docs/doc2.txt", "document"),
("examples/resources/images/image1.png", "image"),
]
# Process each resource
print("\nProcessing resources...")
total_items = 0
categories = []
for resource_file, modality in resources:
if not os.path.exists(resource_file):
continue
try:
result = await service.memorize(resource_url=resource_file, modality=modality)
total_items += len(result.get("items", []))
# Categories are returned in the result and updated after each memorize call
categories = result.get("categories", [])
except Exception as e:
print(f"Error: {e}")
# Write to output files
output_dir = "examples/output/multimodal_example"
os.makedirs(output_dir, exist_ok=True)
# 1. Generate individual Markdown files for each category
await generate_memory_md(categories, output_dir)
print(f"\n✓ Processed {len(resources)} files, extracted {total_items} items")
print(f"✓ Generated {len(categories)} categories")
print(f"✓ Output: {output_dir}/")
if __name__ == "__main__":
asyncio.run(main())
examples/example_4_openrouter_memory.py
+112
View File
@@ -0,0 +1,112 @@
"""
Example 4: Multiple Conversations -> Memory Category File (Using OpenRouter)
This example demonstrates how to process multiple conversation files
and generate memory categories using OpenRouter as the LLM backend.
Usage:
export OPENROUTER_API_KEY=your_api_key
python examples/example_4_openrouter_memory.py
"""
import asyncio
import os
import sys
from memu.app import MemoryService
src_path = os.path.abspath("src")
sys.path.insert(0, src_path)
async def generate_memory_md(categories, output_dir):
"""Generate concise markdown files for each memory category."""
os.makedirs(output_dir, exist_ok=True)
generated_files = []
for cat in categories:
name = cat.get("name", "unknown")
summary = cat.get("summary", "")
filename = f"{name}.md"
filepath = os.path.join(output_dir, filename)
with open(filepath, "w", encoding="utf-8") as f:
if summary:
cleaned_summary = summary.replace("<content>", "").replace("</content>", "").strip()
f.write(f"{cleaned_summary}\n")
else:
f.write("*No content available*\n")
generated_files.append(filename)
return generated_files
async def main():
"""
Process multiple conversation files and generate memory categories using OpenRouter.
This example:
1. Initializes MemoryService with OpenRouter API
2. Processes conversation JSON files
3. Extracts memory categories from conversations
4. Outputs the categories to files
"""
print("Example 4: Conversation Memory Processing (OpenRouter)")
print("-" * 50)
api_key = os.getenv("OPENROUTER_API_KEY")
if not api_key:
msg = "Please set OPENROUTER_API_KEY environment variable"
raise ValueError(msg)
# Initialize service with OpenRouter
service = MemoryService(
llm_profiles={
"default": {
"provider": "openrouter",
"client_backend": "httpx",
"base_url": "https://openrouter.ai",
"api_key": api_key,
"chat_model": "anthropic/claude-3.5-sonnet", # you can use any model from openrouter.ai
"embed_model": "openai/text-embedding-3-small", # you can use any model from openrouter.ai
},
},
)
conversation_files = [
"examples/resources/conversations/conv1.json",
"examples/resources/conversations/conv2.json",
"examples/resources/conversations/conv3.json",
]
print("\nProcessing conversations...")
total_items = 0
categories = []
for conv_file in conversation_files:
if not os.path.exists(conv_file):
print(f"Skipped: {conv_file} not found")
continue
try:
print(f"Processing: {conv_file}")
result = await service.memorize(resource_url=conv_file, modality="conversation")
total_items += len(result.get("items", []))
categories = result.get("categories", [])
except Exception as e:
print(f"Error processing {conv_file}: {e}")
output_dir = "examples/output/openrouter_example"
os.makedirs(output_dir, exist_ok=True)
await generate_memory_md(categories, output_dir)
print(f"\nProcessed {len(conversation_files)} files, extracted {total_items} items")
print(f"Generated {len(categories)} categories")
print(f"Output: {output_dir}/")
if __name__ == "__main__":
asyncio.run(main())
examples/example_5_with_lazyllm_client.py
+250
View File
@@ -0,0 +1,250 @@
"""
Unified Example: LazyLLM Integration Demo
=========================================
This example merges functionalities from:
1. Example 1: Conversation Memory Processing
2. Example 2: Skill Extraction
3. Example 3: Multimodal Processing
It demonstrates how to use the LazyLLM backend for:
- Processing conversation history
- Extracting technical skills from logs
- Handling multimodal content (images + text)
- defaut source and model are from qwen
Usage:
export MEMU_QWEN_API_KEY=your_api_key
python examples/example_5_with_lazyllm_client.py
"""
import asyncio
import os
import sys
from pathlib import Path
# Add src to sys.path FIRST before importing memu
project_root = Path(__file__).parent.parent
src_path = str(project_root / "src")
if src_path not in sys.path:
sys.path.insert(0, src_path)
from memu.app import MemoryService
# ==========================================
# PART 1: Conversation Memory Processing
# ==========================================
async def run_conversation_memory_demo(service):
print("\n" + "=" * 60)
print("PART 1: Conversation Memory Processing")
print("=" * 60)
conversation_files = [
"examples/resources/conversations/conv1.json",
"examples/resources/conversations/conv2.json",
"examples/resources/conversations/conv3.json",
]
total_items = 0
categories = []
for conv_file in conversation_files:
if not os.path.exists(conv_file):
print(f"⚠ File not found: {conv_file}")
continue
try:
print(f" Processing: {conv_file}")
result = await service.memorize(resource_url=conv_file, modality="conversation")
total_items += len(result.get("items", []))
categories = result.get("categories", [])
print(f" ✓ Extracted {len(result.get('items', []))} items")
except Exception as e:
print(f" ✗ Error processing {conv_file}: {e}")
# Output generation
output_dir = "examples/output/lazyllm_example/conversation"
os.makedirs(output_dir, exist_ok=True)
await generate_markdown_output(categories, output_dir)
print(f"✓ Conversation processing complete. Output: {output_dir}")
# ==========================================
# PART 2: Skill Extraction
# ==========================================
async def run_skill_extraction_demo(service):
print("\n" + "=" * 60)
print("PART 2: Skill Extraction from Logs")
print("=" * 60)
# Configure prompt for skill extraction
skill_prompt = """
You are analyzing an agent execution log. Extract the key actions taken, their outcomes, and lessons learned.
Output MUST be valid XML wrapped in <skills> tags.
Format:
<skills>
<memory>
<content>
[Action] Description...
[Lesson] Key lesson...
</content>
<categories>
<category>Category Name</category>
</categories>
</memory>
</skills>
Text: {resource}
"""
# Update service config for skill extraction
service.memorize_config.memory_types = ["skill"]
service.memorize_config.memory_type_prompts = {"skill": skill_prompt}
logs = ["examples/resources/logs/log1.txt", "examples/resources/logs/log2.txt", "examples/resources/logs/log3.txt"]
all_skills = []
for log_file in logs:
if not os.path.exists(log_file):
continue
print(f" Processing log: {log_file}")
try:
result = await service.memorize(resource_url=log_file, modality="document")
for item in result.get("items", []):
if item.get("memory_type") == "skill":
all_skills.append(item.get("summary", ""))
print(f" ✓ Extracted {len(result.get('items', []))} skills")
except Exception as e:
print(f" ✗ Error: {e}")
# Generate summary guide
if all_skills:
output_file = "examples/output/lazyllm_example/skills/skill_guide.md"
await generate_skill_guide(all_skills, service, output_file)
print(f"✓ Skill guide generated: {output_file}")
# ==========================================
# PART 3: Multimodal Memory
# ==========================================
async def run_multimodal_demo(service):
print("\n" + "=" * 60)
print("PART 3: Multimodal Memory Processing")
print("=" * 60)
# Configure for knowledge extraction
xml_prompt = """
Analyze content and extract key information.
Output MUST be valid XML wrapped in <knowledge> tags.
Format:
<knowledge>
<memory>
<content>Extracted content...</content>
<categories><category>category_name</category></categories>
</memory>
</knowledge>
Content: {resource}
"""
service.memorize_config.memory_types = ["knowledge"]
service.memorize_config.memory_type_prompts = {"knowledge": xml_prompt}
resources = [
("examples/resources/docs/doc1.txt", "document"),
("examples/resources/images/image1.png", "image"),
]
categories = []
for res_file, modality in resources:
if not os.path.exists(res_file):
continue
print(f" Processing {modality}: {res_file}")
try:
result = await service.memorize(resource_url=res_file, modality=modality)
categories = result.get("categories", [])
print(f" ✓ Extracted {len(result.get('items', []))} items")
except Exception as e:
print(f" ✗ Error: {e}")
output_dir = "examples/output/lazyllm_example/multimodal"
os.makedirs(output_dir, exist_ok=True)
await generate_markdown_output(categories, output_dir)
print(f"✓ Multimodal processing complete. Output: {output_dir}")
# ==========================================
# Helpers
# ==========================================
async def generate_markdown_output(categories, output_dir):
for cat in categories:
name = cat.get("name", "unknown")
summary = cat.get("summary", "")
if not summary:
continue
with open(os.path.join(output_dir, f"{name}.md"), "w", encoding="utf-8") as f:
f.write(f"# {name.replace('_', ' ').title()}\n\n")
cleaned = summary.replace("<content>", "").replace("</content>", "").strip()
f.write(cleaned)
async def generate_skill_guide(skills, service, output_file):
os.makedirs(os.path.dirname(output_file), exist_ok=True)
skills_text = "\n\n".join(skills)
prompt = f"Summarize these skills into a guide:\n\n{skills_text}"
# Use LazyLLM via service
summary = await service.llm_client.chat(text=prompt)
with open(output_file, "w", encoding="utf-8") as f:
f.write(summary)
# ==========================================
# Main Entry
# ==========================================
async def main():
print("Unified LazyLLM Example")
print("=" * 60)
# 1. Initialize Shared Service
service = MemoryService(
llm_profiles={
"default": {
"client_backend": "lazyllm_backend",
"chat_model": "qwen3-max",
"embed_model": "text-embedding-v3",
"lazyllm_source": {
"source": "qwen",
"llm_source": "qwen",
"vlm_source": "qwen",
"embed_source": "qwen",
"stt_source": "qwen",
"vlm_model": "qwen-vl-plus",
"stt_model": "qwen-audio-turbo",
},
},
}
)
# 2. Run Demos
await run_conversation_memory_demo(service)
# await run_skill_extraction_demo(service)
# await run_multimodal_demo(service)
if __name__ == "__main__":
asyncio.run(main())
examples/example_6_preprocess_modalities.py
+140
View File
@@ -0,0 +1,140 @@
"""
Example 6: Multimodal Preprocessing -> Extracted Text + Caption
This example exercises memU's preprocessing stage for every supported modality
(conversation, document, image, audio, video) and prints what each preprocessor
extracts. It is a quick way to verify that multimodal extraction works end to end
against a real provider:
- conversation / document : chat LLM (text understanding)
- image / video : VLM vision (video uses a sampled mid-frame)
- audio : speech-to-text transcription + chat LLM cleanup
Requirements:
- An OpenAI API key with access to a chat model, a vision model, and the
transcription model (``gpt-4o-mini-transcribe``).
- ``ffmpeg``/``ffprobe`` on PATH for the video frame extraction.
- Optional document extras for PDF/Office input: ``pip install 'memu-py[document]'``.
Usage:
export OPENAI_API_KEY=your_api_key
# Optional model overrides (defaults to the library defaults):
# export MEMU_CHAT_MODEL=gpt-4o-mini
# export MEMU_VLM_MODEL=gpt-4o-mini
python examples/example_6_preprocess_modalities.py
"""
from __future__ import annotations
import asyncio
import os
import sys
from memu.app import MemoryService
# Allow running from a source checkout without installing the package.
sys.path.insert(0, os.path.abspath("src"))
RESOURCES = "examples/resources"
# (modality, file). Vision modalities are routed to the VLM client automatically.
CASES: list[tuple[str, str]] = [
("conversation", f"{RESOURCES}/conversations/conv1.json"),
("document", f"{RESOURCES}/docs/doc1.txt"),
("document", f"{RESOURCES}/docs/doc_sample.pdf"),
("image", f"{RESOURCES}/images/image1.png"),
("audio", f"{RESOURCES}/audio/audio_intro.mp3"),
("video", f"{RESOURCES}/video/video_test.mp4"),
]
VISION_MODALITIES = {"image", "video"}
def build_service() -> MemoryService:
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
msg = "Please set OPENAI_API_KEY environment variable"
raise ValueError(msg)
default_profile: dict[str, str] = {"api_key": api_key}
chat_model = os.getenv("MEMU_CHAT_MODEL")
if chat_model:
default_profile["chat_model"] = chat_model
service = MemoryService(llm_profiles={"default": default_profile})
# VLM profiles are derived from the LLM profile; allow an explicit override of
# the vision model (handy when a default vision model is unavailable).
vlm_model = os.getenv("MEMU_VLM_MODEL")
if vlm_model:
for cfg in service.vlm_profiles.values():
cfg.vlm_model = vlm_model
return service
def _preview(value: str | None, limit: int = 500) -> str:
if not value:
return "<empty>"
value = value.strip()
return value if len(value) <= limit else value[:limit] + f"... [+{len(value) - limit} chars]"
async def preprocess_one(service: MemoryService, modality: str, path: str) -> bool:
"""Run the preprocessing stage for a single resource and print the result."""
print("\n" + "=" * 72)
print(f"MODALITY: {modality} ({os.path.basename(path)})")
print("-" * 72)
if not os.path.exists(path):
print(f" SKIP: missing sample file {path}")
return False
# Ingest (copies the file into the resources dir and extracts inline text for
# text modalities), then run the modality-specific preprocessor. Vision
# modalities use the VLM client; everything else uses the chat LLM client.
local_path, raw_text = await service.fs.fetch(path, modality)
if modality in VISION_MODALITIES:
client = service._get_vlm_client(service.memorize_config.vlm_profile)
else:
client = service._get_llm_client("default")
segments = await service._preprocess_resource_url(
local_path=local_path,
text=raw_text,
modality=modality,
llm_client=client,
)
extracted = False
for i, seg in enumerate(segments):
print(f" [segment {i}] caption: {_preview(seg.get('caption'), 200)}")
print(f" [segment {i}] text : {_preview(seg.get('text'))}")
if seg.get("text"):
extracted = True
return extracted
async def main() -> None:
print("Example 6: Multimodal Preprocessing")
print("-" * 50)
service = build_service()
results: list[tuple[str, str, bool]] = []
for modality, path in CASES:
try:
ok = await preprocess_one(service, modality, path)
except Exception as e:
print(f" ERROR: {e}")
ok = False
results.append((modality, os.path.basename(path), ok))
print("\n" + "#" * 72)
print("SUMMARY")
print("#" * 72)
for modality, name, ok in results:
print(f" [{'OK ' if ok else 'FAIL'}] {modality:<12} {name}")
if __name__ == "__main__":
asyncio.run(main())
examples/getting_started_robust.py
+107
View File
@@ -0,0 +1,107 @@
"""
Getting Started with MemU: A Robust Example.
This script demonstrates the core lifecycle of MemU:
1. **Initialization**: Setting up the client with secure API key handling.
2. **Memory Injection**: Adding a specific memory with metadata.
3. **Retrieval**: Searching for that memory using natural language.
4. **Error Handling**: Catching common configuration issues.
Usage:
export OPENAI_API_KEY=your_api_key_here
python examples/getting_started_robust.py
"""
import asyncio
import logging
import os
import sys
# Ensure src is in the path for local usage if custom installing
sys.path.insert(0, os.path.abspath("src"))
from memu.app import MemoryService
# Configure logging to show info but suppress noisy libraries
logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
logging.getLogger("httpx").setLevel(logging.WARNING)
async def main() -> None:
"""Run the MemU lifecycle demonstration."""
print(">>> MemU Getting Started Example")
print("-" * 30)
# 1. API Key Handling
# MemU relies on an LLM backend (defaulting to OpenAI).
# We ensure the API key is present before proceeding.
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
print("[!] Error: OPENAI_API_KEY environment variable is not set.")
print("Please export it: export OPENAI_API_KEY=sk-...")
return
try:
# 2. Initialization
# We initialize the MemoryService with:
# - llm_profiles: Configuration for the LLM (model, api_key).
# - memorize_config: Pre-defining a memory category ensures we can organize memories efficiently.
print("[*] Initializing MemoryService with model: gpt-4o-mini...")
service = MemoryService(
llm_profiles={
"default": {
"api_key": api_key,
"chat_model": "gpt-4o-mini",
},
},
memorize_config={
"memory_categories": [
{
"name": "User Facts",
"description": "General and specific facts known about the user preference and identity.",
}
]
},
)
print("[OK] Service initialized successfully.\n")
# 3. Memory Injection
# We manually inject a memory into the system.
# This is useful for bootstrapping a user profile or adding explicit knowledge.
print("[*] Injecting memory...")
memory_content = "The user is a senior Python architect who loves clean code and type hints."
# We use 'create_memory_item' to insert a single memory record.
# memory_type='profile' indicates this is an attribute of the user.
result = await service.create_memory_item(
memory_type="profile",
memory_content=memory_content,
memory_categories=["User Facts"],
)
print(f"[OK] Memory created! ID: {result.get('memory_item', {}).get('id')}\n")
# 4. Retrieval
# Now we query the system naturally to see if it recalls the information.
query_text = "What kind of code does the user like?"
print(f"[*] Querying: '{query_text}'")
search_results = await service.retrieve(queries=[{"role": "user", "content": query_text}])
# 5. Display Results
items = search_results.get("items", [])
if items:
print(f"[OK] Found {len(items)} relevant memory item(s):")
for idx, item in enumerate(items, 1):
print(f" {idx}. {item.get('summary')} (Type: {item.get('memory_type')})")
else:
print("[!] No relevant memories found.")
except Exception as e:
print(f"\n[!] An error occurred during execution: {e}")
logging.exception("Detailed traceback:")
finally:
print("\n[=] Example execution finished.")
if __name__ == "__main__":
asyncio.run(main())
examples/langgraph_demo.py
+76
View File
@@ -0,0 +1,76 @@
"""Demo script for MemU LangGraph Integration."""
import asyncio
import logging
import os
import sys
# Try imports and fail proactively if missing
try:
import langgraph # noqa: F401
from langchain_core.tools import BaseTool
from memu.app.service import MemoryService
from memu.integrations.langgraph import MemULangGraphTools
except ImportError:
print("Missing dependencies. Please run: uv sync --extra langgraph")
sys.exit(1)
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("langgraph_demo")
async def initialize_infrastructure() -> MemULangGraphTools:
"""Initialize the MemoryService and the LangGraph adapter."""
# Ensure OPENAI_API_KEY is present
if not os.environ.get("OPENAI_API_KEY"):
logger.warning("OPENAI_API_KEY not found in environment variables.")
# In a real scenario, you might load config from file or env
service = MemoryService()
return MemULangGraphTools(service)
async def process_conversation(tools: list[BaseTool], user_id: str) -> None:
"""Simulate a conversation where memory is saved."""
save_tool = next(t for t in tools if t.name == "save_memory")
logger.info("--- Simulating Save Memory ---")
inputs = {
"content": "The user prefers dark mode and likes Python programming.",
"user_id": user_id,
"metadata": {"source": "demo_script"},
}
# Invoke the tool (async execution)
result = await save_tool.ainvoke(inputs)
logger.info("Save Result: %s", result)
async def process_retrieval(tools: list[BaseTool], user_id: str) -> None:
"""Simulate retrieving memory."""
search_tool = next(t for t in tools if t.name == "search_memory")
logger.info("--- Simulating Search Memory ---")
inputs = {"query": "What are the user's preferences?", "user_id": user_id, "limit": 3}
result = await search_tool.ainvoke(inputs)
logger.info("Search Result:\n%s", result)
async def main() -> None:
"""Main entry point."""
logger.info("Starting LangGraph Demo...")
adapter = await initialize_infrastructure()
tools = adapter.tools()
user_id = "demo_user_123"
await process_conversation(tools, user_id)
await process_retrieval(tools, user_id)
logger.info("Demo completed.")
if __name__ == "__main__":
asyncio.run(main())
examples/output/conversation_example/activities.md
+9
View File
@@ -0,0 +1,9 @@
# activities
## Open Source Contributions
- The user enjoys contributing to open source projects, specifically a Python CLI tool used for automating deployment tasks.
## Running
- The user usually goes for a run every morning and is interested in running routes near downtown San Francisco.
## Dining Plans
- The user plans to try several vegetarian restaurants in San Francisco for their partner, who is vegetarian.
## Gym
- The user goes to the gym 3-4 times a week, usually after work around 7 PM.
examples/output/conversation_example/experiences.md
+5
View File
@@ -0,0 +1,5 @@
# experiences
## User Experiences
- The user has been programming for about 5 years.
- The user is leading a big product launch next month, which is causing work-related stress that is affecting their sleep schedule.
- The user usually goes to bed around 11 PM but finds themselves awake thinking about work projects.
examples/output/conversation_example/goals.md
+17
View File
@@ -0,0 +1,17 @@
# goals
## Learning Objectives
- The user is interested in learning more about system design and scalability patterns.
- The user is learning about event-driven architecture and message queues, specifically using Apache Kafka for event streaming.
## Product Development Goals
- The user wants to build their own SaaS product related to developer tools or automation.
- The user is currently in the research phase for their product idea, reading about OpenAPI specifications and exploring tools like Swagger and Postman.
- The user plans to start prototyping their SaaS product idea in the next few months.
## Tool Development Ideas
- The user is considering building a tool for API testing and documentation that automatically generates test cases based on API specifications.
- The user is researching tools for developing the API tool.
## Work-Life Balance Goals
- The user plans to talk to their team about response time expectations for emails, as they believe they are putting pressure on themselves.
- The user expresses a desire to establish better work-life boundaries and has been considering ways to limit work-related activities during the evening.
examples/output/conversation_example/habits.md
+13
View File
@@ -0,0 +1,13 @@
# habits
## Eating Habits
- The user is trying to eat less meat.
## Exercise Routine
- The user usually goes for a run every morning.
- The user exercises regularly, going to the gym 3-4 times a week.
## Sleep Habits
- The user has been having trouble sleeping due to work stress.
- The user usually tries to go to bed around 11 PM but struggles to fall asleep due to stress from work projects.
- The user checks their phone before bed, which may impact their sleep quality.
- The user used to read before bed but stopped because they were always checking work emails.
## Caffeine Consumption
- The user usually drinks coffee throughout the day to stay alert, typically having their last coffee around 3-4 PM.
examples/output/conversation_example/knowledge.md
+29
View File
@@ -0,0 +1,29 @@
# knowledge
## Reading Interests
- The user has been reading about OpenAPI specifications and exploring tools like Swagger and Postman.
## Technology Stack
- Alex's technology stack includes Django and FastAPI for Python services, Kubernetes for orchestration, Redis for caching, and Apache Kafka for event streaming.
## Interests
- Alex has an interest in system design and scalability patterns and is learning about event-driven architecture and message queues.
- For food in San Francisco, check out the Ferry Building Marketplace for gourmet options and local produce.
- Golden Gate Park, Lands End, and Muir Woods are recommended for nature activities in San Francisco.
- A self-guided tour of tech headquarters in Silicon Valley and the Computer History Museum are good tech-related activities in San Francisco.
- The de Young Museum and SFMOMA are suitable for photography and museum interests in San Francisco.
## Monitoring and Observability
- For monitoring and observability, Alex works with Prometheus and Grafana.
## Sleep and Wellness
- Work stress can negatively affect sleep quality, especially when one is in a leadership role.
- Using a phone before bed can disrupt sleep due to blue light, which suppresses melatonin production.
- Regular exercise can improve sleep, but working out close to bedtime may be stimulating.
- Caffeine consumption, particularly in the afternoon, can adversely affect sleep quality.
- Establishing a wind-down routine, such as reading or gentle stretching before bed, can help improve sleep.
- Keeping a phone out of the bedroom can create a boundary between work and sleep, improving sleep quality.
- Setting specific 'no work' hours can help transition from work mode to rest mode, reducing stress.
## Activities and Recommendations
- The best time to visit Muir Woods is early in the morning around 8 AM to avoid crowds and enjoy peacefulness.
- Parking reservations at Muir Woods are required and should be booked in advance.
- Recommended vegetarian restaurants in San Francisco include Greens Restaurant, Nourish Cafe, Shizen, Gracias Madre, and Ike's Love & Sandwiches.
- The Embarcadero is a great flat waterfront path for morning runs in San Francisco.
- Crissy Field offers stunning views of the Golden Gate Bridge and is another good running spot.
- Running to Coit Tower from North Beach provides a more challenging route with hills.
- San Francisco weather can be changeable, particularly across different neighborhoods, so packing layers is advisable.
examples/output/conversation_example/opinions.md
+1
View File
@@ -0,0 +1 @@
*No content available*
examples/output/conversation_example/personal_info.md
+3
View File
@@ -0,0 +1,3 @@
# personal_info
## Basic Information
- The user is named Alex and works as a software engineer at TechCorp.
examples/output/conversation_example/preferences.md
+6
View File
@@ -0,0 +1,6 @@
# preferences
## Interests
- The user loves food and nature.
- The user enjoys exploring tech companies due to their background in software development.
- The user is interested in vegetarian options as their partner is vegetarian and the user is trying to eat less meat.
- The user likes reading and used to read before bed.
examples/output/conversation_example/relationships.md
+3
View File
@@ -0,0 +1,3 @@
# relationships
## partner
- The user's partner enjoys photography and museums.
examples/output/conversation_example/work_life.md
+12
View File
@@ -0,0 +1,12 @@
# work_life
## Professional Background
- The user is a software engineer at TechCorp.
- Alex is a software engineer with approximately 5 years of programming experience.
- The user works in software development.
## Current Responsibilities
- The user primarily works on backend systems using Python and Go for an e-commerce platform.
- At TechCorp, Alex's team is building a distributed microservices architecture for their e-commerce platform.
- The user uses Django and FastAPI for Python services and is migrating to Go for better performance.
- The user works with Kubernetes for orchestration and Redis for caching.
- The user is responsible for monitoring and observability, using Prometheus and Grafana.
- The user is leading a big product launch next month.
examples/proactive/memory/config.py
+66
View File
@@ -0,0 +1,66 @@
memorize_config = {
"memory_types": [
"record",
],
"memory_type_prompts": {
"record": {
"objective": {
"ordinal": 10,
"prompt": "# Task Objective\nYou will be given a conversation between a user and an coding agent. Your goal is to extract detailed records for what are planed to do, and what have been done.",
},
"workflow": {
"ordinal": 20,
"prompt": "# Workflow\nRead through the conversation and extract records. You should expecially focus on:\n- What the user ask the agent to do\n- What plan does the agent suggest\n- What the agent has done",
},
"rules": {
"ordinal": -1,
"prompt": None,
},
"examples": {
"ordinal": 60,
"prompt": "# Example\n## Output\n<item>\n <memory>\n <content>The user ask the agent to generate a code example for fastapi</content>\n <categories>\n <category>todo</category>\n </categories>\n </memory>\n <memory>\n <content>The agent suggest to use the code example from the document</content>\n <categories>\n <category>todo</category>\n </categories>\n </memory>\n <memory>\n <content>The agent ask the user to specify the response type</content>\n <categories>\n <category>todo</category>\n </categories>\n </memory>\n</item>",
},
}
},
"memory_categories": [
{
"name": "todo",
"description": "This file traces the latest status of the task. All records should be included in this file.",
"target_length": None,
"custom_prompt": {
"objective": {
"ordinal": 10,
"prompt": "# Task Objective\nYou are a specialist in task management. You should update the markdown file to reflect the latest status of the task.",
},
"workflow": {
"ordinal": 20,
"prompt": "# Workflow\nRead through the existing markdown file and the new records. Then update the markdown file to reflect:\n- What existing tasks are completed\n- What new tasks are added\n- What tasks are still in progress",
},
"rules": {
"ordinal": 30,
"prompt": "# Rules\nFor each action-like record, explictly mark it as [Done] or [Todo].",
},
"examples": {
"ordinal": 50,
"prompt": "# Example\n## Output\n```markdown\n# Task\n## Task Objective\nThe user ask the agent to generate a code example for fastapi\n## Breakdown\n- [Done] The agent suggest to use the code example from the document\n- [Todo] The agent ask the user to specify the response type\n```",
},
},
}
],
}
retrieve_config = {
"method": "rag",
"route_intention": False,
"sufficiency_check": False,
"category": {
"enabled": False,
},
"item": {
"enabled": True,
"top_k": 10,
},
"resource": {
"enabled": False,
},
}
examples/proactive/memory/local/__init__.py
View File
examples/proactive/memory/local/common.py
+31
View File
@@ -0,0 +1,31 @@
import os
from memu.app import MemoryService
from ..config import memorize_config, retrieve_config
USER_ID = "claude_user"
SHARED_MEMORY_SERVICE = None
def get_memory_service() -> MemoryService:
global SHARED_MEMORY_SERVICE
if SHARED_MEMORY_SERVICE is not None:
return SHARED_MEMORY_SERVICE
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
msg = "Please set OPENAI_API_KEY environment variable"
raise ValueError(msg)
SHARED_MEMORY_SERVICE = MemoryService(
llm_profiles={
"default": {
"api_key": api_key,
"chat_model": "gpt-4o-mini",
},
},
memorize_config=memorize_config,
retrieve_config=retrieve_config,
)
return SHARED_MEMORY_SERVICE
examples/proactive/memory/local/memorize.py
+38
View File
@@ -0,0 +1,38 @@
import json
from collections.abc import Awaitable
from pathlib import Path
from typing import Any
import pendulum
from .common import get_memory_service
USER_ID = "claude_user"
def dump_conversation_resource(
conversation_messages: list[dict[str, Any]],
) -> str:
resource_data = {
"content": [
{
"role": message.get("role", "system"),
"content": {"text": message.get("content", "")},
"created_at": message.get("timestamp", pendulum.now().isoformat()),
}
for message in conversation_messages
]
}
time_string = pendulum.now().format("YYYYMMDD_HHmmss")
resource_url = Path(__file__).parent / "data" / f"conv_{time_string}.json"
resource_url.parent.mkdir(parents=True, exist_ok=True)
with open(resource_url, "w") as f:
json.dump(resource_data, f, indent=4, ensure_ascii=False)
return resource_url.as_posix()
def memorize(conversation_messages: list[dict[str, Any]]) -> Awaitable[dict[str, Any]]:
memory_service = get_memory_service()
resource_url = dump_conversation_resource(conversation_messages)
return memory_service.memorize(resource_url=resource_url, modality="conversation", user={"user_id": USER_ID})
examples/proactive/memory/local/tools.py
+42
View File
@@ -0,0 +1,42 @@
from typing import Any
from claude_agent_sdk import create_sdk_mcp_server, tool
from .common import get_memory_service
USER_ID = "claude_user"
@tool("memu_memory", "Retrieve memory based on a query", {"query": str})
async def get_memory(args: dict[str, Any]) -> dict[str, Any]:
"""Retrieve memory from the memory API based on the provided query."""
query = {"role": "user", "content": args["query"]}
memory_service = get_memory_service()
result = await memory_service.retrieve(query, where={"user_id": USER_ID})
return {"content": [{"type": "text", "text": str(result)}]}
async def _get_todos() -> str:
memory_service = get_memory_service()
result = await memory_service.list_memory_categories(where={"user_id": USER_ID})
categories = result["categories"]
todos = ""
for category in categories:
if category["name"] == "todo":
todos = category["summary"]
return todos
@tool("memu_todos", "Retrieve todos for the user", {})
async def get_todos() -> dict[str, Any]:
"""Retrieve todos from the memory API."""
todos = await _get_todos()
return {"content": [{"type": "text", "text": str(todos)}]}
memu_server = create_sdk_mcp_server(name="memu", version="1.0.0", tools=[get_memory, get_todos])
examples/proactive/memory/platform/__init__.py
View File
examples/proactive/memory/platform/memorize.py
+31
View File
@@ -0,0 +1,31 @@
from typing import Any
import aiohttp
from ..config import memorize_config
BASE_URL = "https://api.memu.so"
API_KEY = "your memu api key"
USER_ID = "claude_user"
AGENT_ID = "claude_agent"
async def memorize(conversation_messages: list[dict[str, Any]]) -> str | None:
payload = {
"conversation": conversation_messages,
"user_id": USER_ID,
"agent_id": AGENT_ID,
"override_config": memorize_config,
}
async with (
aiohttp.ClientSession() as session,
session.post(
f"{BASE_URL}/api/v3/memory/memorize",
headers={"Authorization": f"Bearer {API_KEY}"},
json=payload,
) as response,
):
result = await response.json()
task_id = result["task_id"]
return task_id
examples/proactive/memory/platform/tools.py
+52
View File
@@ -0,0 +1,52 @@
from typing import Any
import aiohttp
from claude_agent_sdk import create_sdk_mcp_server, tool
BASE_URL = "https://api.memu.so"
API_KEY = "your memu api key"
USER_ID = "claude_user"
AGENT_ID = "claude_agent"
@tool("memu_memory", "Retrieve memory based on a query", {"query": str})
async def get_memory(args: dict[str, Any]) -> dict[str, Any]:
"""Retrieve memory from the memory API based on the provided query."""
query = args["query"]
url = f"{BASE_URL}/api/v3/memory/retrieve"
headers = {"Authorization": f"Bearer {API_KEY}"}
data = {"user_id": USER_ID, "agent_id": AGENT_ID, "query": query}
async with aiohttp.ClientSession() as session, session.post(url, headers=headers, json=data) as response:
result = await response.json()
return {"content": [{"type": "text", "text": str(result)}]}
async def _get_todos() -> str:
url = f"{BASE_URL}/api/v3/memory/categories"
headers = {"Authorization": f"Bearer {API_KEY}"}
data = {
"user_id": USER_ID,
"agent_id": AGENT_ID,
}
async with aiohttp.ClientSession() as session, session.post(url, headers=headers, json=data) as response:
result = await response.json()
categories = result["categories"]
todos = ""
for category in categories:
if category["name"] == "todo":
todos = category["summary"]
return todos
@tool("memu_todos", "Retrieve todos for the user", {})
async def get_todos() -> dict[str, Any]:
"""Retrieve todos from the memory API."""
todos = await _get_todos()
return {"content": [{"type": "text", "text": str(todos)}]}
# Create the MCP server with the tool
memu_server = create_sdk_mcp_server(name="memu", version="1.0.0", tools=[get_memory, get_todos])
examples/proactive/proactive.py
+198
View File
@@ -0,0 +1,198 @@
import asyncio
from claude_agent_sdk import (
AssistantMessage,
ClaudeAgentOptions,
ClaudeSDKClient,
ResultMessage,
TextBlock,
)
from memory.local.memorize import memorize
from memory.local.tools import _get_todos, memu_server
# Set your Anthropic API key here if it's not set in the environment variables
# os.environ["ANTHROPIC_API_KEY"] = ""
N_MESSAGES_MEMORIZE = 2
RUNNING_MEMORIZATION: asyncio.Task | None = None
async def trigger_memorize(messages: list[dict[str, any]]) -> bool:
"""Create a background task to memorize conversation messages.
Returns True if the task was successfully created and registered.
"""
global RUNNING_MEMORIZATION
try:
memorize_awaitable = memorize(messages)
RUNNING_MEMORIZATION = asyncio.create_task(memorize_awaitable)
except Exception as e:
print(f"\n[Memory] Memorization initialization failed: {e!r}")
return False
else:
print("\n[Memory] Memorization task submitted.")
return True
async def get_next_input(iteration: int) -> tuple[str | None, bool]:
"""
Get the next input for the conversation.
Returns:
tuple of (input_text, should_break)
- input_text: The user input or todo-based input, None if should continue
- should_break: True if the loop should break
"""
if iteration == 0:
return await get_user_input()
todos = await _get_todos()
print(f">>> Todos:\n{todos}\n")
print("-" * 40)
if todos and "[todo]" in todos.lower():
return f"Please continue with the following todos:\n{todos}", False
return await get_user_input()
async def get_user_input() -> tuple[str | None, bool]:
"""
Get input from the user.
Returns:
tuple of (input_text, should_break)
"""
try:
user_input = input("\nYou: ").strip()
except EOFError:
return None, True
if not user_input:
return None, False
if user_input.lower() in ("quit", "exit"):
return None, True
return user_input, False
async def process_response(client: ClaudeSDKClient) -> list[str]:
"""Process the assistant response and return collected text parts."""
assistant_text_parts: list[str] = []
async for message in client.receive_response():
if isinstance(message, AssistantMessage):
for block in message.content:
if isinstance(block, TextBlock):
print(f"Claude: {block.text}")
assistant_text_parts.append(block.text)
elif isinstance(message, ResultMessage):
print(f"Result: {message.result}")
return assistant_text_parts
async def check_and_memorize(conversation_messages: list[dict[str, any]]) -> None:
"""Check if memorization threshold is reached and trigger if needed.
Skips triggering if a previous memorization task is still running.
"""
global RUNNING_MEMORIZATION
if len(conversation_messages) < N_MESSAGES_MEMORIZE:
return
# Check if there's a running memorization task
if RUNNING_MEMORIZATION is not None:
if not RUNNING_MEMORIZATION.done():
print("\n[Info] Have running memorization, skipping...")
return
# Previous task completed, check for exceptions
try:
RUNNING_MEMORIZATION.result()
except Exception as e:
print(f"\n[Memory] Memorization failed: {e!r}")
RUNNING_MEMORIZATION = None
print(f"\n[Info] Reached {N_MESSAGES_MEMORIZE} messages, triggering memorization...")
success = await trigger_memorize(conversation_messages.copy())
if success:
conversation_messages.clear()
async def run_conversation_loop(client: ClaudeSDKClient) -> list[dict[str, any]]:
"""Run the main conversation loop."""
conversation_messages: list[dict[str, any]] = []
iteration = 0
while True:
user_input, should_break = await get_next_input(iteration)
if should_break:
break
if user_input is None:
continue
conversation_messages.append({"role": "user", "content": user_input})
await client.query(user_input)
assistant_text_parts = await process_response(client)
if assistant_text_parts:
conversation_messages.append({
"role": "assistant",
"content": "\n".join(assistant_text_parts),
})
await check_and_memorize(conversation_messages)
iteration += 1
return conversation_messages
async def main():
options = ClaudeAgentOptions(
mcp_servers={"memu": memu_server},
allowed_tools=[
# "mcp__memu__memu_memory",
"mcp__memu__memu_todos",
],
)
print("Claude Autorun")
print("Type 'quit' or 'exit' to end the session.")
print("-" * 40)
async with ClaudeSDKClient(options=options) as client:
remaining_messages = await run_conversation_loop(client)
# Wait for any running memorization task to complete
global RUNNING_MEMORIZATION
if RUNNING_MEMORIZATION is not None and not RUNNING_MEMORIZATION.done():
print("\n[Info] Waiting for running memorization task to complete...")
try:
await RUNNING_MEMORIZATION
print("\n[Memory] Running memorization completed successfully.")
except Exception as e:
print(f"\n[Memory] Running memorization failed: {e!r}")
RUNNING_MEMORIZATION = None
# Memorize remaining messages and wait for completion
if remaining_messages:
print("\n[Info] Session ended, memorizing remaining messages...")
success = await trigger_memorize(remaining_messages.copy())
if success and RUNNING_MEMORIZATION is not None:
print("\n[Info] Waiting for final memorization to complete...")
try:
await RUNNING_MEMORIZATION
print("\n[Memory] Final memorization completed successfully.")
except Exception as e:
print(f"\n[Memory] Final memorization failed: {e!r}")
print("\nDone")
if __name__ == "__main__":
asyncio.run(main())
examples/resources/audio/audio_intro.mp3
BIN
View File
Binary file not shown.
examples/resources/conversations/conv1.json
+54
View File
@@ -0,0 +1,54 @@
[
{
"role": "user",
"content": "Hi! I'm Alex. I'm a software engineer working at TechCorp. I've been programming for about 5 years now."
},
{
"role": "assistant",
"content": "Nice to meet you, Alex! It's great to have you here. What kind of software development do you primarily work on at TechCorp?"
},
{
"role": "user",
"content": "I mainly work on backend systems using Python and Go. We're building a distributed microservices architecture for our e-commerce platform. I'm particularly interested in learning more about system design and scalability patterns."
},
{
"role": "assistant",
"content": "That sounds like an exciting project! Distributed systems and microservices certainly present interesting challenges. Are you working with any specific frameworks or technologies for the microservices?"
},
{
"role": "user",
"content": "Yes, we use Django and FastAPI for Python services, and we're migrating some critical services to Go for better performance. We use Kubernetes for orchestration and Redis for caching. I've been learning about event-driven architecture and message queues lately."
},
{
"role": "assistant",
"content": "Excellent technology stack! Event-driven architecture is a great pattern for microservices. Are you using any specific message broker like RabbitMQ or Kafka?"
},
{
"role": "user",
"content": "We're using Apache Kafka for our event streaming. I'm also responsible for monitoring and observability, so I work with Prometheus and Grafana. Outside of work, I like to contribute to open source projects. I recently started contributing to a Python CLI tool."
},
{
"role": "assistant",
"content": "That's wonderful! Open source contributions are a great way to learn and give back to the community. What made you interested in contributing to that particular CLI tool?"
},
{
"role": "user",
"content": "I actually use it in my daily workflow for automating deployment tasks. I found a few bugs and thought I could help improve it. I also want to eventually build my own SaaS product, maybe something related to developer tools or automation."
},
{
"role": "assistant",
"content": "That's a great goal! Having a SaaS product idea is exciting. Do you have any specific problem you're looking to solve with your developer tools idea?"
},
{
"role": "user",
"content": "I'm thinking about building a tool for API testing and documentation. Something that can automatically generate test cases based on API specifications and maintain up-to-date documentation. I notice this is a pain point in my current team."
},
{
"role": "assistant",
"content": "That's a valuable problem to solve! Many teams struggle with keeping API documentation synchronized with their codebase. Have you done any market research or prototyping yet?"
},
{
"role": "user",
"content": "Not yet, I'm still in the research phase. I've been reading about OpenAPI specifications and exploring tools like Swagger and Postman. I think there's room for a more automated and AI-powered solution. I'm planning to start prototyping in the next few months."
}
]
examples/resources/conversations/conv2.json
+67
View File
@@ -0,0 +1,67 @@
[
{
"role": "user",
"created_at": "2025-02-01T09:00:00Z",
"content": "I need help planning a weekend trip to San Francisco. Can you suggest some activities?"
},
{
"role": "assistant",
"created_at": "2025-02-01T09:00:08Z",
"content": "I'd be happy to help you plan your San Francisco trip! What are your main interests? Are you into tech, food, nature, art, or something else?"
},
{
"role": "user",
"created_at": "2025-02-01T09:00:25Z",
"content": "I love food and nature! I'm also interested in visiting some tech companies since I work in software development. My partner enjoys photography and museums."
},
{
"role": "assistant",
"created_at": "2025-02-01T09:00:40Z",
"content": "Perfect! For food, you should definitely check out the Ferry Building Marketplace for gourmet options and local produce. For nature, I'd recommend Golden Gate Park, hiking at Lands End, or visiting Muir Woods for the redwoods. For tech, you could do a self-guided tour of tech headquarters in Silicon Valley or visit the Computer History Museum. For your partner, there's the de Young Museum in Golden Gate Park and SFMOMA for modern art."
},
{
"role": "user",
"created_at": "2025-02-01T09:01:20Z",
"content": "Those sound great! I'm particularly excited about Muir Woods. I've never seen redwoods before. What's the best time to visit?"
},
{
"role": "assistant",
"created_at": "2025-02-01T09:01:35Z",
"content": "Muir Woods is magical! I'd recommend going early in the morning, ideally right when it opens around 8 AM. This way you'll avoid the crowds and experience the peaceful atmosphere of the forest. The morning light filtering through the redwoods is also spectacular for photography. Make sure to book parking reservations in advance as they're required."
},
{
"role": "user",
"created_at": "2025-02-01T09:02:10Z",
"content": "Good to know about the parking! Are there any good vegetarian restaurants you'd recommend? My partner is vegetarian and I'm trying to eat less meat."
},
{
"role": "assistant",
"created_at": "2025-02-01T09:02:25Z",
"content": "San Francisco has amazing vegetarian options! I'd recommend Greens Restaurant for upscale vegetarian with a view, Nourish Cafe for healthy bowls, and Shizen for vegan sushi. If you're in the Mission District, try Gracias Madre for organic Mexican food. For something casual, Ike's Love & Sandwiches has great veggie options."
},
{
"role": "user",
"created_at": "2025-02-01T09:03:05Z",
"content": "Perfect! I'll add these to my list. One more thing - I usually go for a run every morning. Are there good running routes near downtown?"
},
{
"role": "assistant",
"created_at": "2025-02-01T09:03:20Z",
"content": "Absolutely! The Embarcadero is perfect for morning runs - it's a flat waterfront path with beautiful bay views. You can run from the Ferry Building towards Fisherman's Wharf, which is about 3 miles one way. Alternatively, Crissy Field offers stunning views of the Golden Gate Bridge. If you want something more challenging with hills, try running up to Coit Tower from North Beach."
},
{
"role": "user",
"created_at": "2025-02-01T09:03:55Z",
"content": "The Embarcadero sounds perfect for my morning routine. Thanks for all the suggestions! I'm getting really excited about this trip now."
},
{
"role": "assistant",
"created_at": "2025-02-01T09:04:05Z",
"content": "You're welcome! It sounds like you have a great mix of activities planned. Don't forget to bring layers - San Francisco weather can be quite changeable, especially if you're visiting different neighborhoods. Have a wonderful trip!"
},
{
"role": "user",
"created_at": "2025-02-01T09:04:30Z",
"content": "Will do! I always forget how different the weather can be in different parts of the city. I'll pack accordingly."
}
]
examples/resources/conversations/conv3.json
+67
View File
@@ -0,0 +1,67 @@
[
{
"role": "user",
"created_at": "2025-03-10T22:00:00Z",
"content": "I've been having trouble sleeping lately. I think work stress might be affecting my sleep schedule."
},
{
"role": "assistant",
"created_at": "2025-03-10T22:00:12Z",
"content": "I'm sorry to hear you're having trouble sleeping. Work stress can definitely impact sleep quality. Can you tell me more about your current sleep routine and what's been going on at work?"
},
{
"role": "user",
"created_at": "2025-03-10T22:00:45Z",
"content": "I usually try to go to bed around 11 PM, but I find myself lying awake thinking about work projects. I'm leading a big product launch next month and there's a lot of pressure. I also tend to check my phone before bed."
},
{
"role": "assistant",
"created_at": "2025-03-10T22:01:05Z",
"content": "It's common for work pressure to interfere with sleep, especially when you're in a leadership role. The phone usage before bed could also be contributing to the problem - the blue light can suppress melatonin production. Have you tried any relaxation techniques or sleep hygiene practices?"
},
{
"role": "user",
"created_at": "2025-03-10T22:01:40Z",
"content": "I tried meditation once but didn't stick with it. I do exercise regularly - I go to the gym 3-4 times a week, usually after work around 7 PM. I also drink coffee throughout the day to stay alert."
},
{
"role": "assistant",
"created_at": "2025-03-10T22:01:55Z",
"content": "Exercise is great for sleep, but working out too close to bedtime can actually be stimulating. The caffeine consumption throughout the day could also be affecting your sleep, especially if you're having coffee in the afternoon. Could you try moving your workouts to the morning and cutting off caffeine after 2 PM?"
},
{
"role": "user",
"created_at": "2025-03-10T22:02:25Z",
"content": "That makes sense. I usually have my last coffee around 3-4 PM. Maybe I should switch to decaf in the afternoon. Morning workouts might be tough though - I'm not really a morning person."
},
{
"role": "assistant",
"created_at": "2025-03-10T22:02:40Z",
"content": "Starting with the caffeine cutoff is a good first step! As for morning workouts, you don't have to make a drastic change immediately. Even a 15-minute walk in the morning can help regulate your circadian rhythm. For the evening, you might want to establish a wind-down routine - maybe some light reading, gentle stretching, or breathing exercises about an hour before bed."
},
{
"role": "user",
"created_at": "2025-03-10T22:03:20Z",
"content": "I like reading, actually. I used to read before bed but stopped because I was always checking work emails. Maybe I should keep my phone out of the bedroom entirely."
},
{
"role": "assistant",
"created_at": "2025-03-10T22:03:35Z",
"content": "That's an excellent idea! Keeping your phone out of the bedroom can make a big difference. You could get a regular alarm clock if you use your phone for that. Creating a clear boundary between work and sleep time is important, especially with the product launch stress."
},
{
"role": "user",
"created_at": "2025-03-10T22:04:10Z",
"content": "You're right. I think I need to be better about work-life boundaries in general. I often answer emails late at night, which probably doesn't help my stress levels."
},
{
"role": "assistant",
"created_at": "2025-03-10T22:04:25Z",
"content": "Those late-night emails can definitely keep your mind in 'work mode' when it should be winding down. Have you considered setting specific 'no work' hours, maybe from 8 PM onwards? That way you give your brain time to transition from work mode to rest mode."
},
{
"role": "user",
"created_at": "2025-03-10T22:04:55Z",
"content": "That sounds reasonable. I'll talk to my team about response time expectations. I don't think anyone expects instant replies at night anyway - I'm probably putting that pressure on myself."
}
]
examples/resources/docs/doc1.txt
+330
View File
@@ -0,0 +1,330 @@
MemU: Agentic Memory Framework - Technical Documentation
Introduction
============
MemU is a sophisticated agentic memory framework designed to provide AI agents and applications with human-like memory capabilities. Unlike traditional RAG (Retrieval-Augmented Generation) systems that simply store and retrieve information, MemU organizes, categorizes, and maintains memories in a structured, semantically meaningful way.
Core Concepts
=============
1. Memory Organization
MemU organizes information into several layers:
- Memory Items: Individual pieces of information extracted from inputs
- Memory Categories: Semantic groupings of related memories
- Memory Types: Classifications of memory content (profile, event, knowledge, behavior)
2. Multi-Modal Support
The framework supports various input modalities:
- Text documents (PDF, TXT, DOC)
- Conversations (JSON chat logs)
- Images (PNG, JPG, with vision model integration)
- Audio (transcription and processing)
- Video (frame extraction and analysis)
3. Intelligent Processing Pipeline
Each input goes through several processing stages:
a. Preprocessing: Content extraction and normalization
b. Summarization: Key information extraction
c. Embedding: Vector representation generation
d. Classification: Memory type identification
e. Categorization: Semantic category assignment
f. Storage: Persistent storage with metadata
Architecture Components
=======================
1. MemoryService (Core Service Layer)
The main entry point for all memory operations:
- memorize(): Process and store new information
- retrieve(): Query and fetch relevant memories
- update(): Modify existing memories
- delete(): Remove memories
Configuration options:
- LLM provider settings (OpenAI, Azure, custom)
- Embedding model selection
- Memory type definitions
- Category templates
- Retrieval methods (RAG, LLM-based)
2. Storage Layer
Multiple storage backends supported:
- SQLite (default, local development)
- PostgreSQL (production deployments)
- In-memory (testing and temporary storage)
Data persistence:
- Memory items with metadata
- Category definitions and summaries
- Vector embeddings for similarity search
- Resource references and URLs
3. Vector Search Engine
Semantic search capabilities powered by:
- Dense embeddings (OpenAI text-embedding-3-small/large)
- Similarity metrics (cosine similarity, dot product)
- Efficient indexing for fast retrieval
- Hybrid search combining semantic and keyword matching
4. LLM Integration Layer
Flexible LLM backend support:
- OpenAI SDK client (primary)
- HTTP-based client (custom endpoints)
- Configurable model selection
- Prompt template system
- Streaming response support
Memory Types
============
1. Profile Memory
Stores persistent information about entities:
- Personal attributes (name, age, occupation)
- Preferences and interests
- Relationships and connections
- Identity and characteristics
Example: "Alex is a software engineer at TechCorp, specializing in backend development"
2. Event Memory
Records discrete occurrences and activities:
- Temporal events with timestamps
- Actions and experiences
- Milestones and achievements
- Incidents and interactions
Example: "Completed the deployment pipeline implementation on November 15, 2024"
3. Knowledge Memory
Captures factual information and learnings:
- Facts and concepts
- Skills and capabilities
- Domain expertise
- Technical knowledge
Example: "Proficient in Python, Go, Kubernetes, and microservices architecture"
4. Behavior Memory
Tracks patterns and tendencies:
- Habits and routines
- Decision patterns
- Behavioral preferences
- Interaction styles
Example: "Prefers morning workouts, typically exercises 3-4 times per week"
Category Management
===================
Dynamic Categorization:
MemU automatically assigns memories to semantic categories based on content similarity. Categories are created and maintained dynamically as new memories are added.
Default Categories:
- personal_info: Personal details and identity
- preferences: Likes, dislikes, and choices
- relationships: Connections with others
- activities: Hobbies and interests
- goals: Aspirations and objectives
- experiences: Past events and learnings
- knowledge: Facts and information
- opinions: Views and perspectives
- habits: Routines and patterns
- work_life: Professional information
Custom Categories:
Users can define custom categories with:
- Name and description
- Embedding vector for semantic matching
- Assignment threshold for automatic categorization
- Summary generation for category overview
Category Summaries:
MemU maintains auto-generated summaries for each category that:
- Provide overview of category contents
- Get updated as new memories are added
- Help with high-level information retrieval
- Support category-level search
Retrieval Strategies
====================
1. RAG-Based Retrieval (Default)
Vector similarity search approach:
- Query embedding generation
- Similarity calculation with stored memories
- Top-K selection per category
- Ranking by relevance score
- Context window assembly
Advantages:
- Fast and efficient
- Deterministic results
- Lower LLM costs
- Good for factual recall
2. LLM-Based Retrieval
AI-powered search and ranking:
- Query understanding and expansion
- Semantic relevance judgment
- Context-aware ranking
- Multi-hop reasoning support
- Natural language result explanation
Advantages:
- Better semantic understanding
- Handles complex queries
- Context-aware results
- Flexible interpretation
Retrieval Pipeline:
1. Pre-retrieval decision (should we retrieve?)
2. Query rewriting (optimize for search)
3. Category ranking (which categories are relevant?)
4. Item retrieval (fetch top-K items)
5. Item ranking (rerank by relevance)
6. Resource retrieval (fetch original sources)
7. Result assembly (format for output)
Best Practices
==============
1. Memory Quality
- Provide detailed, contextual inputs
- Include timestamps for events
- Maintain consistent terminology
- Regular memory consolidation
- Remove outdated information
2. Configuration Optimization
- Tune embedding models for your domain
- Adjust category assignment thresholds
- Customize memory type prompts
- Set appropriate top-K values
- Configure LLM parameters
3. Performance Optimization
- Batch memory operations when possible
- Use appropriate storage backend
- Index frequently queried fields
- Cache embeddings when reusing
- Monitor memory growth
4. Privacy and Security
- Implement access controls
- Encrypt sensitive memories
- Regular data audits
- Compliance with data regulations
- User consent for memory storage
Use Cases
=========
1. Personal AI Assistants
- Remember user preferences and context
- Maintain conversation history
- Learn from interactions
- Personalize responses
2. Customer Support Systems
- Track customer history and issues
- Remember preferences and complaints
- Build customer profiles
- Improve service quality
3. Educational Applications
- Track learning progress
- Remember concepts learned
- Adapt to learning style
- Provide personalized content
4. Knowledge Management
- Organize organizational knowledge
- Track project information
- Build expertise databases
- Enable knowledge discovery
5. Agent Workflows
- Maintain task context
- Remember tool usage patterns
- Learn from execution history
- Optimize decision making
API Reference
=============
Basic Usage Example:
```python
from memu.app import MemoryService
# Initialize service
service = MemoryService(
llm_config={
"api_key": "your-api-key",
"chat_model": "gpt-4o-mini"
}
)
# Store a memory
result = await service.memorize(
resource_url="conversation.json",
modality="conversation"
)
# Retrieve memories
memories = await service.retrieve(
query="What programming languages does Alex know?",
top_k=5
)
# Access categories
categories = service.store.categories
```
Advanced Configuration:
```python
# Custom memory types
memorize_config = {
"memory_types": ["profile", "knowledge", "custom"],
"memory_type_prompts": {
"custom": "Extract specific information: {resource}"
},
"memory_categories": [
{"name": "technical_skills", "description": "Programming and technical abilities"},
{"name": "soft_skills", "description": "Communication and interpersonal skills"}
]
}
service = MemoryService(
llm_config=llm_config,
memorize_config=memorize_config
)
```
Roadmap
=======
Upcoming Features:
- Long-term memory consolidation
- Federated memory systems
- Memory importance scoring
- Automatic memory pruning
- Cross-user memory sharing
- Memory versioning and history
- Enhanced temporal reasoning
- Graph-based memory relationships
- Memory export and import
- Advanced privacy controls
Contributing
============
MemU is open source and welcomes contributions:
- Bug reports and feature requests
- Documentation improvements
- Code contributions
- Example applications
- Performance optimizations
For more information, visit: https://github.com/mem-labs/memU
examples/resources/docs/doc2.txt
+433
View File
@@ -0,0 +1,433 @@
Building Production-Ready AI Agents: A Comprehensive Guide
Executive Summary
=================
This document provides a comprehensive guide to building, deploying, and maintaining production-ready AI agents. It covers architecture patterns, best practices, common pitfalls, and real-world implementation strategies based on experiences from deploying agents at scale.
What is an AI Agent?
====================
An AI agent is an autonomous system that can:
- Perceive its environment through sensors or data inputs
- Make decisions based on goals and constraints
- Take actions using available tools and APIs
- Learn and adapt from experience
- Operate with minimal human intervention
Key characteristics that differentiate agents from simple LLM applications:
1. Goal-directed behavior (not just reactive)
2. Multi-step reasoning and planning
3. Tool use and external API integration
4. Memory and state management
5. Error handling and recovery
6. Continuous operation
Agent Architecture Patterns
============================
1. ReAct (Reasoning + Acting) Pattern
The most common agent architecture:
- Thought: Reason about the current situation
- Action: Choose and execute a tool/action
- Observation: Observe the results
- Repeat until goal is achieved
Advantages:
- Interpretable decision-making process
- Easy to debug with visible reasoning
- Works well with modern LLMs
Challenges:
- Can be verbose and slow
- May get stuck in reasoning loops
- Token costs accumulate quickly
2. Plan-and-Execute Pattern
Separates planning from execution:
- Generate high-level plan upfront
- Execute steps sequentially
- Replan if execution fails
Advantages:
- More efficient for complex tasks
- Better resource allocation
- Clearer progress tracking
Challenges:
- Less adaptable to changing conditions
- Planning failures cascade
- Requires good task decomposition
3. Hierarchical Agent Systems
Multiple agents working together:
- Coordinator agent manages workflow
- Specialist agents handle specific domains
- Memory shared across agents
Advantages:
- Scalable to complex domains
- Parallel execution possible
- Clear separation of concerns
Challenges:
- Coordination overhead
- Complex error propagation
- Harder to debug
4. Autonomous Agent Pattern
Continuous operation without explicit tasks:
- Monitor environment for triggers
- Self-generate tasks and goals
- Learn from outcomes
- Adjust behavior over time
Advantages:
- Truly autonomous operation
- Proactive rather than reactive
- Continuous improvement
Challenges:
- Harder to control and bound
- Safety and alignment concerns
- Resource management critical
Essential Components
====================
1. Language Model Integration
Choosing the right LLM:
- GPT-4 family: Best reasoning, highest cost
- Claude: Strong safety, good reasoning
- Open source models: Lower cost, self-hosted
Optimization strategies:
- Use smaller models for simple tasks
- Implement prompt caching
- Batch similar requests
- Fine-tune for specific domains
2. Tool Registry and Execution
Managing agent capabilities:
- Dynamic tool discovery
- Type-safe tool schemas
- Sandboxed execution environment
- Rate limiting and quotas
- Error handling and retries
Tool design principles:
- Single responsibility per tool
- Clear input/output contracts
- Idempotent when possible
- Detailed error messages
- Comprehensive documentation
3. Memory Systems
Types of memory agents need:
- Working memory: Current task context
- Short-term memory: Recent interactions
- Long-term memory: Persistent knowledge
- Episodic memory: Past experiences
- Semantic memory: General knowledge
Implementation approaches:
- Vector databases for semantic search
- Graph databases for relationships
- Key-value stores for fast lookup
- SQL databases for structured data
4. Planning and Decision Making
Strategies for complex tasks:
- Breadth-first vs depth-first search
- Heuristic-guided planning
- Monte Carlo tree search
- Reinforcement learning
- Symbolic reasoning integration
5. Monitoring and Observability
Critical metrics to track:
- Task completion rate
- Average execution time
- Token usage and costs
- Error rates by type
- Tool usage patterns
- User satisfaction scores
Logging and tracing:
- Structured logging (JSON format)
- Distributed tracing (trace IDs)
- Action replay capabilities
- Performance profiling
- Anomaly detection
Production Deployment Considerations
====================================
1. Safety and Control
Implementing guardrails:
- Input validation and sanitization
- Output filtering and moderation
- Action confirmation for critical operations
- Rollback mechanisms
- Circuit breakers for cascading failures
- Human-in-the-loop for high-risk decisions
2. Cost Management
Strategies to control expenses:
- Budget limits per agent/user
- Automatic degradation to cheaper models
- Request batching and caching
- Usage analytics and alerts
- Prompt optimization for token efficiency
3. Latency Optimization
Reducing response time:
- Parallel tool execution when possible
- Streaming responses to users
- Pre-warming model connections
- Edge deployment for low latency
- Async processing for non-critical tasks
4. Scalability
Handling growth:
- Stateless agent design
- Horizontal scaling with load balancers
- Queue-based task distribution
- Database sharding strategies
- Caching layers (Redis, Memcached)
5. Reliability
Building fault-tolerant systems:
- Graceful degradation
- Automatic retry with exponential backoff
- Dead letter queues for failed tasks
- Health checks and auto-recovery
- Multi-region deployment
Common Pitfalls and Solutions
=============================
1. Infinite Loops
Problem: Agent gets stuck repeating same actions
Solutions:
- Implement maximum iteration limits
- Track state to detect loops
- Add randomization to break patterns
- Use reflection to detect stuck states
2. Context Window Overflow
Problem: Conversation history exceeds model limits
Solutions:
- Implement context summarization
- Use sliding window approach
- Store full history, send summaries
- Priority-based context selection
3. Tool Hallucination
Problem: Agent tries to use non-existent tools
Solutions:
- Provide clear tool documentation in prompt
- Validate tool names before execution
- Use structured output formats (JSON)
- Fine-tune on correct tool usage
4. Inconsistent Behavior
Problem: Agent gives different results for same input
Solutions:
- Set temperature to 0 for determinism
- Implement result caching
- Add explicit reasoning chain requirements
- Use voting/consensus from multiple runs
5. Poor Error Recovery
Problem: Single failures cause complete task abandonment
Solutions:
- Implement retry logic with backoff
- Fallback to alternative approaches
- Graceful degradation to partial results
- Clear error messages and recovery suggestions
Testing Strategies
==================
1. Unit Testing
- Test individual tool functions
- Mock LLM responses
- Validate prompt templates
- Test parsing and formatting logic
2. Integration Testing
- End-to-end task completion
- Tool chain execution
- Memory persistence
- API integration points
3. Evaluation Benchmarks
- Task success rate
- Response quality (human eval)
- Reasoning coherence
- Tool usage efficiency
- Cost per task
4. Adversarial Testing
- Malicious input handling
- Edge case scenarios
- Resource exhaustion attacks
- Prompt injection attempts
5. A/B Testing
- Compare prompt variations
- Test different model versions
- Evaluate architecture changes
- Measure user satisfaction
Real-World Use Cases
====================
1. Customer Support Agent
Capabilities:
- Answer common questions using knowledge base
- Create support tickets for complex issues
- Schedule appointments and callbacks
- Escalate to human agents when needed
Key challenges:
- Maintaining empathy in responses
- Handling frustrated customers
- Accurate issue classification
- Privacy and data security
2. Research Assistant Agent
Capabilities:
- Search academic databases
- Summarize research papers
- Identify trends and gaps
- Generate literature reviews
Key challenges:
- Source credibility verification
- Citation accuracy
- Handling conflicting information
- Domain expertise requirements
3. DevOps Automation Agent
Capabilities:
- Monitor system metrics
- Diagnose performance issues
- Execute remediation actions
- Generate incident reports
Key challenges:
- High stakes decision-making
- Complex system dependencies
- Security and access control
- Audit trail requirements
4. Sales Prospecting Agent
Capabilities:
- Research potential customers
- Personalize outreach messages
- Schedule meetings
- Track engagement and follow-ups
Key challenges:
- Avoiding spam-like behavior
- Personalization at scale
- CRM integration complexity
- Compliance with regulations
Performance Optimization
========================
1. Prompt Engineering
- Use clear, structured instructions
- Provide relevant examples (few-shot)
- Include constraints and boundaries
- Optimize token usage
- Version control prompts
2. Model Selection
- Match model capability to task complexity
- Use smaller models for simple tasks
- Consider latency requirements
- Balance cost vs performance
- Evaluate fine-tuning benefits
3. Caching Strategies
- Cache LLM responses for common queries
- Cache tool results when appropriate
- Implement embeddings cache
- Use CDN for static resources
- Cache database queries
4. Parallel Execution
- Identify independent tool calls
- Execute in parallel where possible
- Use async/await patterns
- Implement concurrent request limits
- Handle partial failures gracefully
Future Trends
=============
1. Multi-Modal Agents
- Vision, audio, and text integration
- Video understanding capabilities
- Embodied AI for robotics
- Mixed reality interactions
2. Improved Planning
- Better long-term reasoning
- Hierarchical task decomposition
- Probabilistic planning under uncertainty
- Resource-constrained optimization
3. Enhanced Memory
- Better long-term retention
- Efficient memory consolidation
- Personalization and adaptation
- Cross-agent knowledge sharing
4. Tool Learning
- Automatic tool discovery
- Tool composition and chaining
- Learning from tool usage patterns
- Generating new tools dynamically
5. Human-Agent Collaboration
- Natural delegation interfaces
- Explainable decision-making
- Interactive planning and refinement
- Shared mental models
Conclusion
==========
Building production-ready AI agents requires careful attention to architecture, safety, performance, and user experience. Success comes from:
- Starting with clear, well-defined use cases
- Iterating based on real-world feedback
- Investing in monitoring and observability
- Prioritizing safety and controllability
- Continuously optimizing costs and performance
The agent paradigm represents a significant leap from traditional applications, but with thoughtful design and implementation, agents can deliver tremendous value while operating reliably at scale.
References and Resources
========================
Key papers:
- "ReAct: Synergizing Reasoning and Acting in Language Models" (Yao et al., 2022)
- "Toolformer: Language Models Can Teach Themselves to Use Tools" (Schick et al., 2023)
- "Reflexion: Language Agents with Verbal Reinforcement Learning" (Shinn et al., 2023)
Frameworks and tools:
- LangChain: Popular agent framework
- AutoGPT: Autonomous agent implementation
- BabyAGI: Task-driven autonomous agent
- AgentGPT: Web-based agent platform
Communities:
- LangChain Discord
- AI Agent subreddit
- AutoGPT GitHub discussions
- Agent research papers on ArXiv
examples/resources/docs/doc_sample.pdf
BIN
View File
Binary file not shown.
examples/resources/images/image1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 267 KiB

examples/resources/logs/log1.txt
+212
View File
@@ -0,0 +1,212 @@
Agent Runtime Log - Production Microservice Deployment (ATTEMPT 1 - FAILURE)
================================================================================
Session ID: deploy-agent-20241120-1430-attempt1
Task: "Deploy recommendation microservice to production with zero downtime"
Agent: DeploymentAgent v3.2
Start Time: 2024-11-20 14:30:00 UTC
Attempt: 1/3
[14:30:00] [INIT] Agent initialization started
[14:30:00] [INFO] Task: Deploy recommendation-service v2.5.0 to production
[14:30:00] [INFO] Target environment: production-cluster-us-east
[14:30:01] [INFO] Deployment strategy: Blue-green with gradual traffic shift
[14:30:01] [SUCCESS] Agent initialized
[14:30:01] [PLANNING] Creating deployment plan
[14:30:02] [THOUGHT] Deployment steps should include:
1. Validate service configuration and dependencies
2. Build and push Docker image
3. Deploy green environment alongside blue
4. Run health checks and smoke tests
5. Gradually shift traffic from blue to green
6. Monitor metrics and rollback if needed
[14:30:02] [ACTION] Step 1: Validating service configuration
[14:30:02] [TOOL] Executing: validate_config(service="recommendation-service", version="v2.5.0")
[14:30:05] [OBSERVATION] Configuration validation results:
- Service manifest: Valid
- Environment variables: 12 variables defined
- Resource limits: CPU 2 cores, Memory 4Gi
- Dependencies: PostgreSQL, Redis, Kafka
- Health check endpoint: /health
[14:30:05] [THOUGHT] Configuration looks good. Proceeding with Docker build.
[14:30:05] [ACTION] Step 2: Building Docker image
[14:30:05] [TOOL] Executing: build_docker_image(
dockerfile="services/recommendation/Dockerfile",
tag="recommendation-service:v2.5.0"
)
[14:30:55] [OBSERVATION] Docker build completed:
- Image: recommendation-service:v2.5.0
- Size: 856 MB
- Build time: 50 seconds
- Base image: python:3.11-slim
- Layers: 12
[14:30:55] [ACTION] Pushing image to registry
[14:30:55] [TOOL] Executing: push_image(image="recommendation-service:v2.5.0")
[14:31:20] [OBSERVATION] Image pushed successfully to gcr.io/company/recommendation-service:v2.5.0
[14:31:20] [ACTION] Step 3: Deploying green environment
[14:31:20] [TOOL] Executing: deploy_kubernetes(
service="recommendation-service",
version="v2.5.0",
environment="green",
replicas=3
)
[14:31:25] [OBSERVATION] Kubernetes deployment initiated:
- Deployment: recommendation-service-green
- Replicas: 3
- Namespace: production
- Pods starting...
[14:31:45] [ACTION] Monitoring pod startup
[14:31:46] [OBSERVATION] Pod status:
- recommendation-green-0: Running
- recommendation-green-1: Running
- recommendation-green-2: Running
- All pods ready and passing health checks
[14:31:46] [ACTION] Step 4: Running smoke tests on green environment
[14:31:46] [TOOL] Executing: run_smoke_tests(target="recommendation-green")
[14:32:00] [OBSERVATION] Smoke test results:
- Health check: PASSED (200 OK)
- Database connectivity: PASSED
- Redis connectivity: PASSED
- Kafka connectivity: PASSED
- Sample recommendation request: PASSED (response time: 120ms)
[14:32:00] [THOUGHT] Green environment healthy. Ready to shift traffic.
[14:32:00] [ACTION] Step 5: Starting gradual traffic shift (Blue → Green)
[14:32:00] [TOOL] Executing: traffic_shift(
from_env="blue",
to_env="green",
strategy="gradual",
initial_percentage=10
)
[14:32:05] [OBSERVATION] Traffic shift initiated:
- Blue environment: 90% traffic
- Green environment: 10% traffic
- Monitoring for 5 minutes before next increment
[14:32:05] [ACTION] Monitoring metrics during traffic shift
[14:33:00] [OBSERVATION] Metrics at 10% traffic (1 minute):
- Request rate: 150 req/s to green (1500 req/s total)
- Average latency: 125ms (baseline: 110ms)
- Error rate: 0.1%
- All metrics within acceptable range
[14:35:00] [ACTION] Increasing traffic to 25%
[14:35:01] [TOOL] Executing: traffic_shift(percentage=25)
[14:35:05] [OBSERVATION] Traffic updated:
- Blue: 75%, Green: 25%
[14:36:00] [OBSERVATION] Metrics at 25% traffic (1 minute):
- Request rate: 375 req/s to green
- Average latency: 130ms
- Error rate: 0.2%
- Metrics stable
[14:38:00] [ACTION] Increasing traffic to 50%
[14:38:01] [TOOL] Executing: traffic_shift(percentage=50)
[14:38:05] [OBSERVATION] Traffic updated:
- Blue: 50%, Green: 50%
[14:39:00] [OBSERVATION] Metrics at 50% traffic (1 minute):
- Request rate: 750 req/s to green
- Average latency: 135ms
- Error rate: 0.3%
[14:40:30] [ERROR] Alert triggered: Error rate spike detected!
[14:40:31] [TOOL] Executing: check_error_logs()
[14:40:32] [OBSERVATION] ERROR ANALYSIS:
- Error rate jumped from 0.3% to 4.2% (14x increase)
- Error type: "DatabaseConnectionError: too many connections"
- Affected pods: All 3 green pods
- Pattern: Started at 50% traffic mark
[14:40:32] [THOUGHT] CRITICAL: Connection pool exhausted at 50% traffic!
[14:40:32] [ACTION] Investigating database connection configuration
[14:40:32] [TOOL] Executing: check_db_config(service="recommendation-green")
[14:40:35] [OBSERVATION] Database configuration:
- Max connections per pod: 20
- Total pods: 3
- Total connection capacity: 60 connections
- Current active connections: 58 (approaching limit)
- Database max_connections: 100
- Blue environment using: 40 connections
- Green environment using: 58 connections
- TOTAL: 98/100 connections (98% utilization!)
[14:40:35] [THOUGHT] Root cause identified: Insufficient connection pool size!
- At 50% traffic, green needs ~60 connections
- At 100% traffic, would need ~120 connections
- Database max_connections=100 is insufficient
- We're already at 98% capacity with both environments running
[14:40:35] [ACTION] Initiating emergency rollback
[14:40:35] [TOOL] Executing: rollback_traffic(to_env="blue")
[14:40:40] [OBSERVATION] Traffic rollback initiated:
- Shifting all traffic back to blue environment
- Green: 50% → 25% → 10% → 0%
[14:41:00] [OBSERVATION] Rollback progress:
- Traffic: 100% on blue, 0% on green
- Error rate: Dropping (4.2% → 2.1% → 0.4%)
- Service stabilizing
[14:41:30] [OBSERVATION] Rollback complete:
- Blue environment: 100% traffic, stable
- Green environment: 0% traffic, shutting down
- Error rate: Back to 0.2% (normal baseline)
- Service restored
[14:41:30] [FAILURE] Deployment failed - Root cause analysis:
PRIMARY ISSUE: Database connection pool misconfiguration
- Database max_connections: 100 (too low for blue+green running simultaneously)
- Each environment needs ~60 connections at 50% traffic
- During blue-green deployment, total needed: 100+ connections
- Database cannot support both environments concurrently
CONTRIBUTING FACTORS:
1. Did not account for connection pool sizing during dual environment operation
2. No load testing performed with both blue and green running simultaneously
3. Connection pool monitoring alerts not in place
4. Gradual rollout exposed issue only at 50% mark
LESSONS LEARNED:
1. ⚠️ CRITICAL: Database connection pools must accommodate BOTH environments during deployment
2. ⚠️ Connection pool size = (peak connections per pod) × (total pods in blue + green)
3. ⚠️ Must load test with full capacity of both environments before production
4. ⚠️ Set up connection pool monitoring and alerts BEFORE deployment
5. ⚠️ Have emergency rollback procedure ready (which saved us here)
REMEDIATION PLAN:
1. Increase database max_connections from 100 to 250
2. Adjust connection pool per pod from 20 to 30
3. Add connection pool utilization monitoring
4. Set alert threshold at 70% connection utilization
5. Document connection pool requirements in deployment checklist
DEPLOYMENT STATUS: FAILED
ROLLBACK STATUS: SUCCESSFUL (no customer impact beyond brief error spike)
TIME TO DETECT: 8 minutes
TIME TO ROLLBACK: 1.5 minutes
CUSTOMER IMPACT: 1.5 minutes of elevated errors (4.2% error rate)
[14:41:30] [EXPORT] Saving failure report: reports/deployment-attempt1-failure.json
[14:41:31] [NOTIFICATION] Alerting team about deployment failure and lessons learned
[14:41:31] [SHUTDOWN] Agent session terminated
=================================================================================
KEY TAKEAWAYS FOR NEXT ATTEMPT:
=================================================================================
1. Fix database connection pool sizing BEFORE retry
2. Calculate total connections needed: (pods_blue + pods_green) × connections_per_pod
3. Add connection monitoring to detect issues early
4. Test with both environments running at full capacity
5. Verify database can handle combined load before production deployment

Some files were not shown because too many files have changed in this diff Show More