nevamind-ai--memu/AGENTS.md

# AGENTS.md

Operational guide for AI coding agents working in this repository.

## Mission

Ship small, verified feature and bugfix changes while preserving memU's current architecture.
Treat `docs/architecture.md` as the runtime source of truth and keep this file aligned with it.

Core invariants:

- `MemoryService` is the composition root for config, storage, LLM/VLM/embedding clients, workflows, interceptors, and memory file export.
- Public APIs should remain stable unless the requested change explicitly requires an API change.
- Major behavior runs through workflows: `memorize`, `retrieve_rag`, `retrieve_llm`, and CRUD/patch operations.
- Workflow steps must keep explicit `requires`/`produces` contracts and declared capability tags.
- Storage is pluggable across `inmemory`, `sqlite`, and `postgres`; repository contract changes require backend parity.
- Scope data comes from `UserConfig.model`; validate `where` filters and do not bypass scope filtering.
- LLM routing is profile-based across chat, embedding, VLM, and custom profiles.
- The exported memory workspace should stay human-readable and agent-traversable: `INDEX.md`, `MEMORY.md`, `memory/`, `resource/`, and `skill/`.

## First Move

Before editing, classify the request by the layer it touches:

- Service/runtime wiring: `src/memu/app/service.py`
- Memorize pipeline: `src/memu/app/memorize.py`
- Retrieve pipelines: `src/memu/app/retrieve.py`
- CRUD/patch behavior: `src/memu/app/crud.py`
- Config models/defaults: `src/memu/app/settings.py`
- Workflow engine and pipeline mutation: `src/memu/workflow/*`
- Memory file export/synthesis: `src/memu/app/memory_files.py`, `src/memu/memory_fs/*`, `src/memu/prompts/memory_fs*`
- Storage protocols/factory: `src/memu/database/interfaces.py`, `src/memu/database/factory.py`
- In-memory backend: `src/memu/database/inmemory/*`
- SQLite backend: `src/memu/database/sqlite/*`
- Postgres backend: `src/memu/database/postgres/*`
- Vector math/ranking: `src/memu/vector.py`
- LLM clients, wrappers, interceptors: `src/memu/llm/*`
- Embedding clients: `src/memu/embedding/*`
- Vision clients: `src/memu/vlm/*`
- Integrations/client surfaces: `src/memu/integrations/*`, `src/memu/client/*`
- Tests: `tests/*`
- User-facing docs/examples: `README.md`, `readme/*`, `docs/*`, `examples/*`

Read the relevant implementation and nearby tests first. Prefer local patterns over new abstractions.

## Implementation Rules

- Keep changes narrow and localized to the affected layer.
- Preserve async behavior and avoid sync work in async paths unless existing code already does it.
- Prefer adding or modifying a workflow step over embedding new flow logic directly in `MemoryService`.
- When adding configurable behavior, update typed settings and wire it through step config or service config.
- Preserve result shapes unless the task explicitly asks for a breaking change.
- Maintain type hints and mypy compatibility.
- Keep provider-specific logic inside provider/backend modules.
- Keep storage-neutral logic outside concrete backends; vector ranking helpers belong in shared modules such as `memu.vector`.
- Do not duplicate client caching. Use the existing client pool/profile routing patterns.
- Do not silently swallow errors that should be visible to callers or tests.
- Avoid broad refactors unless they are required to make the requested change safe.

## Workflow Guidance

For memorize work:

- Keep modality routing in the preprocessing pipeline.
- Use VLM clients for image/video paths and chat clients for conversation/document/audio paths.
- Preserve the flow: ingest resource, preprocess, extract items, dedupe/merge, categorize, persist index, build response.
- If rich document ingestion changes, consider the optional `document` extra and MarkItDown behavior.

For retrieve work:

- Check both `retrieve_rag` and `retrieve_llm` when behavior is shared.
- Validate `where` filters against the configured user model before querying.
- Keep category, item, and resource recall responsibilities separated.
- Preserve early-stop/sufficiency behavior when modifying ranking or response build steps.

For CRUD/patch work:

- Respect repository contracts and scope fields.
- Preserve existing output shapes for list/create/update/delete operations.
- Add backend coverage when contracts change.

For memory file export:

- Keep raw resources copied verbatim into `resource/`.
- Keep category memory split under `memory/<slug>.md`.
- Keep synthesized skills under `skill/<name>/SKILL.md`.
- Keep root files as indexes/overviews, not hidden sources of untracked state.
- Treat synthesized skill output as derived from source descriptions, not from unrelated runtime state.

## Backend Parity

If a repository method, model field, filter behavior, or vector search contract changes:

1. Update the protocol/interface.
2. Update `inmemory`, `sqlite`, and `postgres` implementations where applicable.
3. Add or extend conformance tests.
4. Check migrations/bootstrap behavior for SQL backends.
5. Verify fallback behavior when pgvector or optional extras are unavailable.

## LLM, VLM, And Embedding Routing

- Chat-like work uses LLM profiles.
- Embedding work uses embedding profiles and dedicated embedding clients.
- Image/video understanding uses VLM clients derived from LLM profiles unless explicitly configured otherwise.
- Step config should select profiles with existing keys such as `llm_profile`, `chat_llm_profile`, and `embed_llm_profile`.
- Keep interceptors and usage metadata behavior consistent across capabilities.
- Do not add embeddings back into chat clients; embedding is intentionally decoupled.

## Feature Checklist

1. Locate the affected flow, backend, integration, or export path.
2. Update settings/defaults only if behavior is configurable.
3. Wire the change through `MemoryService`, pipeline registration, or step config as appropriate.
4. Implement repository/backend changes for all impacted providers.
5. Add focused tests for the happy path and edge cases.
6. Update docs/examples for user-visible behavior.
7. Add or update ADRs under `docs/adr/` for architectural changes.

## Bug Fix Checklist

1. Reproduce the bug with an existing or new failing test.
2. Fix the smallest correct layer.
3. Add a regression test that fails before and passes after.
4. Check cross-backend effects when storage behavior changes.
5. Check both retrieval modes when recall/ranking/response behavior changes.
6. Confirm no unintended public API or output-shape changes.

## Testing And Validation

Use `uv` for local runs.

- Setup: `make install`
- Run all tests: `make test`
- Run focused tests: `uv run python -m pytest tests/<target_test>.py`
- Full quality checks: `make check`

At minimum, run targeted tests for touched code. Run `make check` for broad, cross-cutting, or release-critical changes.
If a required check cannot be run, say why in the final summary.

Useful focused areas:

- Storage parity: `tests/test_backend_conformance.py`, `tests/test_inmemory.py`, `tests/test_sqlite.py`, `tests/test_postgres.py`
- Retrieval/vector behavior: `tests/test_vector.py`, `tests/test_salience.py`, retrieve-related tests
- Multimodal preprocessing: `tests/test_conversation_preprocess.py`, `tests/test_document_text.py`, `tests/test_audio_preprocess.py`, `tests/test_vlm_preprocess.py`
- Memory file export/synthesis: `tests/test_memory_files.py`, `tests/test_memory_fs_synthesis.py`
- Provider routing: `tests/test_embedding.py`, `tests/test_openrouter.py`, `tests/test_openai_max_tokens.py`, `tests/test_lazyllm.py`
- Integrations/tool memory: `tests/test_tool_memory.py`, LangGraph-related tests

## Documentation Rules

- Update `README.md`, localized `readme/*`, `docs/*`, or examples when behavior visible to users changes.
- Keep architecture docs current when composition, workflow contracts, backends, or profile routing changes.
- Keep examples runnable and aligned with the public API.
- Do not document speculative features as implemented.

## Done Criteria

Before finishing:

- Code compiles for touched paths.
- Tests for changed behavior pass.
- New behavior has test coverage.
- User-visible or architectural behavior is documented.
- Backend parity has been considered and implemented where needed.
- No unrelated files were modified.