chore: import upstream snapshot with attribution
pi-agent-plugin checks / lint (push) Has been cancelled
pi-agent-plugin checks / test (20) (push) Has been cancelled
pi-agent-plugin checks / test (22) (push) Has been cancelled
pi-agent-plugin checks / build (push) Has been cancelled
TypeScript SDK CI / check_changes (push) Has been cancelled
TypeScript SDK CI / changelog_check (push) Has been cancelled
ci / changelog_check (push) Has been cancelled
ci / check_changes (push) Has been cancelled
ci / build_mem0 (3.10) (push) Has been cancelled
ci / build_mem0 (3.11) (push) Has been cancelled
ci / build_mem0 (3.12) (push) Has been cancelled
CLI Node CI / lint (push) Has been cancelled
CLI Node CI / test (20) (push) Has been cancelled
CLI Node CI / test (22) (push) Has been cancelled
CLI Node CI / build (push) Has been cancelled
CLI Python CI / lint (push) Has been cancelled
CLI Python CI / test (3.10) (push) Has been cancelled
CLI Python CI / test (3.11) (push) Has been cancelled
CLI Python CI / test (3.12) (push) Has been cancelled
CLI Python CI / build (push) Has been cancelled
openclaw checks / lint (push) Has been cancelled
openclaw checks / test (20) (push) Has been cancelled
openclaw checks / test (22) (push) Has been cancelled
openclaw checks / build (push) Has been cancelled
opencode-plugin checks / build (push) Has been cancelled
TypeScript SDK CI / build_ts_sdk (20) (push) Has been cancelled
TypeScript SDK CI / build_ts_sdk (22) (push) Has been cancelled
TypeScript SDK CI / integration_ts_sdk (20) (push) Has been cancelled
TypeScript SDK CI / integration_ts_sdk (22) (push) Has been cancelled
pi-agent-plugin checks / lint (push) Has been cancelled
pi-agent-plugin checks / test (20) (push) Has been cancelled
pi-agent-plugin checks / test (22) (push) Has been cancelled
pi-agent-plugin checks / build (push) Has been cancelled
TypeScript SDK CI / check_changes (push) Has been cancelled
TypeScript SDK CI / changelog_check (push) Has been cancelled
ci / changelog_check (push) Has been cancelled
ci / check_changes (push) Has been cancelled
ci / build_mem0 (3.10) (push) Has been cancelled
ci / build_mem0 (3.11) (push) Has been cancelled
ci / build_mem0 (3.12) (push) Has been cancelled
CLI Node CI / lint (push) Has been cancelled
CLI Node CI / test (20) (push) Has been cancelled
CLI Node CI / test (22) (push) Has been cancelled
CLI Node CI / build (push) Has been cancelled
CLI Python CI / lint (push) Has been cancelled
CLI Python CI / test (3.10) (push) Has been cancelled
CLI Python CI / test (3.11) (push) Has been cancelled
CLI Python CI / test (3.12) (push) Has been cancelled
CLI Python CI / build (push) Has been cancelled
openclaw checks / lint (push) Has been cancelled
openclaw checks / test (20) (push) Has been cancelled
openclaw checks / test (22) (push) Has been cancelled
openclaw checks / build (push) Has been cancelled
opencode-plugin checks / build (push) Has been cancelled
TypeScript SDK CI / build_ts_sdk (20) (push) Has been cancelled
TypeScript SDK CI / build_ts_sdk (22) (push) Has been cancelled
TypeScript SDK CI / integration_ts_sdk (20) (push) Has been cancelled
TypeScript SDK CI / integration_ts_sdk (22) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# Ignore all .env files
|
||||
**/.env
|
||||
**/.env.*
|
||||
|
||||
# Ignore all database files
|
||||
**/*.db
|
||||
**/*.sqlite
|
||||
**/*.sqlite3
|
||||
|
||||
# Ignore logs
|
||||
**/*.log
|
||||
|
||||
# Ignore runtime data
|
||||
**/node_modules
|
||||
**/__pycache__
|
||||
**/.pytest_cache
|
||||
**/.coverage
|
||||
**/coverage
|
||||
|
||||
# Ignore Docker runtime files
|
||||
**/.dockerignore
|
||||
**/Dockerfile
|
||||
**/docker-compose*.yml
|
||||
@@ -0,0 +1,15 @@
|
||||
OPENAI_API_KEY=sk-xxx
|
||||
USER=user
|
||||
|
||||
# LLM Configuration (optional - defaults to openai/gpt-4o-mini)
|
||||
# LLM_PROVIDER=ollama
|
||||
# LLM_MODEL=llama3.1:latest
|
||||
# LLM_API_KEY=
|
||||
# LLM_BASE_URL=
|
||||
# OLLAMA_BASE_URL=http://localhost:11434
|
||||
|
||||
# Embedder Configuration (optional - defaults to openai/text-embedding-3-small)
|
||||
# EMBEDDER_PROVIDER=ollama
|
||||
# EMBEDDER_MODEL=nomic-embed-text
|
||||
# EMBEDDER_API_KEY=
|
||||
# EMBEDDER_BASE_URL=
|
||||
@@ -0,0 +1 @@
|
||||
3.12
|
||||
@@ -0,0 +1,14 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
LABEL org.opencontainers.image.name="mem0/openmemory-mcp"
|
||||
|
||||
WORKDIR /usr/src/openmemory
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
|
||||
COPY config.json .
|
||||
COPY . .
|
||||
|
||||
EXPOSE 8765
|
||||
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8765"]
|
||||
@@ -0,0 +1,60 @@
|
||||
# OpenMemory API
|
||||
|
||||
This directory contains the backend API for OpenMemory, built with FastAPI and SQLAlchemy. This also runs the Mem0 MCP Server that you can use with MCP clients to remember things.
|
||||
|
||||
## Quick Start with Docker (Recommended)
|
||||
|
||||
The easiest way to get started is using Docker. Make sure you have Docker and Docker Compose installed.
|
||||
|
||||
1. Build the containers:
|
||||
```bash
|
||||
make build
|
||||
```
|
||||
|
||||
2. Create `.env` file:
|
||||
```bash
|
||||
make env
|
||||
```
|
||||
|
||||
Once you run this command, edit the file `api/.env` and enter the `OPENAI_API_KEY`.
|
||||
|
||||
3. Start the services:
|
||||
```bash
|
||||
make up
|
||||
```
|
||||
|
||||
The API will be available at `http://localhost:8765`
|
||||
|
||||
### Common Docker Commands
|
||||
|
||||
- View logs: `make logs`
|
||||
- Open shell in container: `make shell`
|
||||
- Run database migrations: `make migrate`
|
||||
- Run tests: `make test`
|
||||
- Run tests and clean up: `make test-clean`
|
||||
- Stop containers: `make down`
|
||||
|
||||
## API Documentation
|
||||
|
||||
Once the server is running, you can access the API documentation at:
|
||||
- Swagger UI: `http://localhost:8765/docs`
|
||||
- ReDoc: `http://localhost:8765/redoc`
|
||||
|
||||
## Project Structure
|
||||
|
||||
- `app/`: Main application code
|
||||
- `models.py`: Database models
|
||||
- `database.py`: Database configuration
|
||||
- `routers/`: API route handlers
|
||||
- `migrations/`: Database migration files
|
||||
- `tests/`: Test files
|
||||
- `alembic/`: Alembic migration configuration
|
||||
- `main.py`: Application entry point
|
||||
|
||||
## Development Guidelines
|
||||
|
||||
- Follow PEP 8 style guide
|
||||
- Use type hints
|
||||
- Write tests for new features
|
||||
- Update documentation when making changes
|
||||
- Run migrations for database changes
|
||||
@@ -0,0 +1,114 @@
|
||||
# A generic, single database configuration.
|
||||
|
||||
[alembic]
|
||||
# path to migration scripts
|
||||
# Use forward slashes (/) also on windows to provide an os agnostic path
|
||||
script_location = alembic
|
||||
|
||||
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
|
||||
# Uncomment the line below if you want the files to be prepended with date and time
|
||||
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
|
||||
# for all available tokens
|
||||
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
|
||||
|
||||
# sys.path path, will be prepended to sys.path if present.
|
||||
# defaults to the current working directory.
|
||||
prepend_sys_path = .
|
||||
|
||||
# timezone to use when rendering the date within the migration file
|
||||
# as well as the filename.
|
||||
# If specified, requires the python-dateutil library that can be
|
||||
# installed by adding `alembic[tz]` to the pip requirements
|
||||
# timezone =
|
||||
|
||||
# max length of characters to apply to the "slug" field
|
||||
# truncate_slug_length = 40
|
||||
|
||||
# set to 'true' to run the environment during
|
||||
# the 'revision' command, regardless of autogenerate
|
||||
# revision_environment = false
|
||||
|
||||
# set to 'true' to allow .pyc and .pyo files without
|
||||
# a source .py file to be detected as revisions in the
|
||||
# versions/ directory
|
||||
# sourceless = false
|
||||
|
||||
# version location specification; This defaults
|
||||
# to alembic/versions. When using multiple version
|
||||
# directories, initial revisions must be specified with --version-path.
|
||||
# The path separator used here should be the separator specified by "version_path_separator" below.
|
||||
# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions
|
||||
|
||||
# version path separator; As mentioned above, this is the character used to split
|
||||
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
|
||||
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or colons.
|
||||
# Valid values for version_path_separator are:
|
||||
#
|
||||
# version_path_separator = :
|
||||
# version_path_separator = ;
|
||||
# version_path_separator = space
|
||||
version_path_separator = os # Use os.pathsep. Default configuration used for new projects.
|
||||
|
||||
# set to 'true' to search source files recursively
|
||||
# in each "version_locations" directory
|
||||
# new in Alembic version 1.10
|
||||
# recursive_version_locations = false
|
||||
|
||||
# the output encoding used when revision files
|
||||
# are written from script.py.mako
|
||||
# output_encoding = utf-8
|
||||
|
||||
sqlalchemy.url = sqlite:///./openmemory.db
|
||||
|
||||
|
||||
[post_write_hooks]
|
||||
# post_write_hooks defines scripts or Python functions that are run
|
||||
# on newly generated revision scripts. See the documentation for further
|
||||
# detail and examples
|
||||
|
||||
# format using "black" - use the console_scripts runner, against the "black" entrypoint
|
||||
# hooks = black
|
||||
# black.type = console_scripts
|
||||
# black.entrypoint = black
|
||||
# black.options = -l 79 REVISION_SCRIPT_FILENAME
|
||||
|
||||
# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
|
||||
# hooks = ruff
|
||||
# ruff.type = exec
|
||||
# ruff.executable = %(here)s/.venv/bin/ruff
|
||||
# ruff.options = check --fix REVISION_SCRIPT_FILENAME
|
||||
|
||||
# Logging configuration
|
||||
[loggers]
|
||||
keys = root,sqlalchemy,alembic
|
||||
|
||||
[handlers]
|
||||
keys = console
|
||||
|
||||
[formatters]
|
||||
keys = generic
|
||||
|
||||
[logger_root]
|
||||
level = WARN
|
||||
handlers = console
|
||||
qualname =
|
||||
|
||||
[logger_sqlalchemy]
|
||||
level = WARN
|
||||
handlers =
|
||||
qualname = sqlalchemy.engine
|
||||
|
||||
[logger_alembic]
|
||||
level = INFO
|
||||
handlers =
|
||||
qualname = alembic
|
||||
|
||||
[handler_console]
|
||||
class = StreamHandler
|
||||
args = (sys.stderr,)
|
||||
level = NOTSET
|
||||
formatter = generic
|
||||
|
||||
[formatter_generic]
|
||||
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||
datefmt = %H:%M:%S
|
||||
@@ -0,0 +1 @@
|
||||
Generic single-database configuration.
|
||||
@@ -0,0 +1,88 @@
|
||||
import os
|
||||
import sys
|
||||
from logging.config import fileConfig
|
||||
|
||||
from alembic import context
|
||||
from dotenv import load_dotenv
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
|
||||
# Add the parent directory to the Python path
|
||||
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
# Import your models here - moved after path setup
|
||||
from app.database import Base # noqa: E402
|
||||
|
||||
# this is the Alembic Config object, which provides
|
||||
# access to the values within the .ini file in use.
|
||||
config = context.config
|
||||
|
||||
# Interpret the config file for Python logging.
|
||||
# This line sets up loggers basically.
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
|
||||
# add your model's MetaData object here
|
||||
# for 'autogenerate' support
|
||||
target_metadata = Base.metadata
|
||||
|
||||
# other values from the config, defined by the needs of env.py,
|
||||
# can be acquired:
|
||||
# my_important_option = config.get_main_option("my_important_option")
|
||||
# ... etc.
|
||||
|
||||
|
||||
def run_migrations_offline() -> None:
|
||||
"""Run migrations in 'offline' mode.
|
||||
|
||||
This configures the context with just a URL
|
||||
and not an Engine, though an Engine is acceptable
|
||||
here as well. By skipping the Engine creation
|
||||
we don't even need a DBAPI to be available.
|
||||
|
||||
Calls to context.execute() here emit the given string to the
|
||||
script output.
|
||||
|
||||
"""
|
||||
url = os.getenv("DATABASE_URL", "sqlite:///./openmemory.db")
|
||||
context.configure(
|
||||
url=url,
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
)
|
||||
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
def run_migrations_online() -> None:
|
||||
"""Run migrations in 'online' mode.
|
||||
|
||||
In this scenario we need to create an Engine
|
||||
and associate a connection with the context.
|
||||
|
||||
"""
|
||||
configuration = config.get_section(config.config_ini_section)
|
||||
configuration["sqlalchemy.url"] = os.getenv("DATABASE_URL", "sqlite:///./openmemory.db")
|
||||
connectable = engine_from_config(
|
||||
configuration,
|
||||
prefix="sqlalchemy.",
|
||||
poolclass=pool.NullPool,
|
||||
)
|
||||
|
||||
with connectable.connect() as connection:
|
||||
context.configure(
|
||||
connection=connection, target_metadata=target_metadata
|
||||
)
|
||||
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
run_migrations_offline()
|
||||
else:
|
||||
run_migrations_online()
|
||||
@@ -0,0 +1,28 @@
|
||||
"""${message}
|
||||
|
||||
Revision ID: ${up_revision}
|
||||
Revises: ${down_revision | comma,n}
|
||||
Create Date: ${create_date}
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
${imports if imports else ""}
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = ${repr(up_revision)}
|
||||
down_revision: Union[str, None] = ${repr(down_revision)}
|
||||
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
|
||||
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Upgrade schema."""
|
||||
${upgrades if upgrades else "pass"}
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Downgrade schema."""
|
||||
${downgrades if downgrades else "pass"}
|
||||
@@ -0,0 +1,225 @@
|
||||
"""Initial migration
|
||||
|
||||
Revision ID: 0b53c747049a
|
||||
Revises:
|
||||
Create Date: 2025-04-19 00:59:56.244203
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = '0b53c747049a'
|
||||
down_revision: Union[str, None] = None
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Upgrade schema."""
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.create_table('access_controls',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('subject_type', sa.String(), nullable=False),
|
||||
sa.Column('subject_id', sa.UUID(), nullable=True),
|
||||
sa.Column('object_type', sa.String(), nullable=False),
|
||||
sa.Column('object_id', sa.UUID(), nullable=True),
|
||||
sa.Column('effect', sa.String(), nullable=False),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index('idx_access_object', 'access_controls', ['object_type', 'object_id'], unique=False)
|
||||
op.create_index('idx_access_subject', 'access_controls', ['subject_type', 'subject_id'], unique=False)
|
||||
op.create_index(op.f('ix_access_controls_created_at'), 'access_controls', ['created_at'], unique=False)
|
||||
op.create_index(op.f('ix_access_controls_effect'), 'access_controls', ['effect'], unique=False)
|
||||
op.create_index(op.f('ix_access_controls_object_id'), 'access_controls', ['object_id'], unique=False)
|
||||
op.create_index(op.f('ix_access_controls_object_type'), 'access_controls', ['object_type'], unique=False)
|
||||
op.create_index(op.f('ix_access_controls_subject_id'), 'access_controls', ['subject_id'], unique=False)
|
||||
op.create_index(op.f('ix_access_controls_subject_type'), 'access_controls', ['subject_type'], unique=False)
|
||||
op.create_table('archive_policies',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('criteria_type', sa.String(), nullable=False),
|
||||
sa.Column('criteria_id', sa.UUID(), nullable=True),
|
||||
sa.Column('days_to_archive', sa.Integer(), nullable=False),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index('idx_policy_criteria', 'archive_policies', ['criteria_type', 'criteria_id'], unique=False)
|
||||
op.create_index(op.f('ix_archive_policies_created_at'), 'archive_policies', ['created_at'], unique=False)
|
||||
op.create_index(op.f('ix_archive_policies_criteria_id'), 'archive_policies', ['criteria_id'], unique=False)
|
||||
op.create_index(op.f('ix_archive_policies_criteria_type'), 'archive_policies', ['criteria_type'], unique=False)
|
||||
op.create_table('categories',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('name', sa.String(), nullable=False),
|
||||
sa.Column('description', sa.String(), nullable=True),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('updated_at', sa.DateTime(), nullable=True),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index(op.f('ix_categories_created_at'), 'categories', ['created_at'], unique=False)
|
||||
op.create_index(op.f('ix_categories_name'), 'categories', ['name'], unique=True)
|
||||
op.create_table('users',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('user_id', sa.String(), nullable=False),
|
||||
sa.Column('name', sa.String(), nullable=True),
|
||||
sa.Column('email', sa.String(), nullable=True),
|
||||
sa.Column('metadata', sa.JSON(), nullable=True),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('updated_at', sa.DateTime(), nullable=True),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index(op.f('ix_users_created_at'), 'users', ['created_at'], unique=False)
|
||||
op.create_index(op.f('ix_users_email'), 'users', ['email'], unique=True)
|
||||
op.create_index(op.f('ix_users_name'), 'users', ['name'], unique=False)
|
||||
op.create_index(op.f('ix_users_user_id'), 'users', ['user_id'], unique=True)
|
||||
op.create_table('apps',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('owner_id', sa.UUID(), nullable=False),
|
||||
sa.Column('name', sa.String(), nullable=False),
|
||||
sa.Column('description', sa.String(), nullable=True),
|
||||
sa.Column('metadata', sa.JSON(), nullable=True),
|
||||
sa.Column('is_active', sa.Boolean(), nullable=True),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('updated_at', sa.DateTime(), nullable=True),
|
||||
sa.ForeignKeyConstraint(['owner_id'], ['users.id'], ),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index(op.f('ix_apps_created_at'), 'apps', ['created_at'], unique=False)
|
||||
op.create_index(op.f('ix_apps_is_active'), 'apps', ['is_active'], unique=False)
|
||||
op.create_index(op.f('ix_apps_name'), 'apps', ['name'], unique=True)
|
||||
op.create_index(op.f('ix_apps_owner_id'), 'apps', ['owner_id'], unique=False)
|
||||
op.create_table('memories',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('user_id', sa.UUID(), nullable=False),
|
||||
sa.Column('app_id', sa.UUID(), nullable=False),
|
||||
sa.Column('content', sa.String(), nullable=False),
|
||||
sa.Column('vector', sa.String(), nullable=True),
|
||||
sa.Column('metadata', sa.JSON(), nullable=True),
|
||||
sa.Column('state', sa.Enum('active', 'paused', 'archived', 'deleted', name='memorystate'), nullable=True),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('updated_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('archived_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('deleted_at', sa.DateTime(), nullable=True),
|
||||
sa.ForeignKeyConstraint(['app_id'], ['apps.id'], ),
|
||||
sa.ForeignKeyConstraint(['user_id'], ['users.id'], ),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index('idx_memory_app_state', 'memories', ['app_id', 'state'], unique=False)
|
||||
op.create_index('idx_memory_user_app', 'memories', ['user_id', 'app_id'], unique=False)
|
||||
op.create_index('idx_memory_user_state', 'memories', ['user_id', 'state'], unique=False)
|
||||
op.create_index(op.f('ix_memories_app_id'), 'memories', ['app_id'], unique=False)
|
||||
op.create_index(op.f('ix_memories_archived_at'), 'memories', ['archived_at'], unique=False)
|
||||
op.create_index(op.f('ix_memories_created_at'), 'memories', ['created_at'], unique=False)
|
||||
op.create_index(op.f('ix_memories_deleted_at'), 'memories', ['deleted_at'], unique=False)
|
||||
op.create_index(op.f('ix_memories_state'), 'memories', ['state'], unique=False)
|
||||
op.create_index(op.f('ix_memories_user_id'), 'memories', ['user_id'], unique=False)
|
||||
op.create_table('memory_access_logs',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('memory_id', sa.UUID(), nullable=False),
|
||||
sa.Column('app_id', sa.UUID(), nullable=False),
|
||||
sa.Column('accessed_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('access_type', sa.String(), nullable=False),
|
||||
sa.Column('metadata', sa.JSON(), nullable=True),
|
||||
sa.ForeignKeyConstraint(['app_id'], ['apps.id'], ),
|
||||
sa.ForeignKeyConstraint(['memory_id'], ['memories.id'], ),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index('idx_access_app_time', 'memory_access_logs', ['app_id', 'accessed_at'], unique=False)
|
||||
op.create_index('idx_access_memory_time', 'memory_access_logs', ['memory_id', 'accessed_at'], unique=False)
|
||||
op.create_index(op.f('ix_memory_access_logs_access_type'), 'memory_access_logs', ['access_type'], unique=False)
|
||||
op.create_index(op.f('ix_memory_access_logs_accessed_at'), 'memory_access_logs', ['accessed_at'], unique=False)
|
||||
op.create_index(op.f('ix_memory_access_logs_app_id'), 'memory_access_logs', ['app_id'], unique=False)
|
||||
op.create_index(op.f('ix_memory_access_logs_memory_id'), 'memory_access_logs', ['memory_id'], unique=False)
|
||||
op.create_table('memory_categories',
|
||||
sa.Column('memory_id', sa.UUID(), nullable=False),
|
||||
sa.Column('category_id', sa.UUID(), nullable=False),
|
||||
sa.ForeignKeyConstraint(['category_id'], ['categories.id'], ),
|
||||
sa.ForeignKeyConstraint(['memory_id'], ['memories.id'], ),
|
||||
sa.PrimaryKeyConstraint('memory_id', 'category_id')
|
||||
)
|
||||
op.create_index('idx_memory_category', 'memory_categories', ['memory_id', 'category_id'], unique=False)
|
||||
op.create_index(op.f('ix_memory_categories_category_id'), 'memory_categories', ['category_id'], unique=False)
|
||||
op.create_index(op.f('ix_memory_categories_memory_id'), 'memory_categories', ['memory_id'], unique=False)
|
||||
op.create_table('memory_status_history',
|
||||
sa.Column('id', sa.UUID(), nullable=False),
|
||||
sa.Column('memory_id', sa.UUID(), nullable=False),
|
||||
sa.Column('changed_by', sa.UUID(), nullable=False),
|
||||
sa.Column('old_state', sa.Enum('active', 'paused', 'archived', 'deleted', name='memorystate'), nullable=False),
|
||||
sa.Column('new_state', sa.Enum('active', 'paused', 'archived', 'deleted', name='memorystate'), nullable=False),
|
||||
sa.Column('changed_at', sa.DateTime(), nullable=True),
|
||||
sa.ForeignKeyConstraint(['changed_by'], ['users.id'], ),
|
||||
sa.ForeignKeyConstraint(['memory_id'], ['memories.id'], ),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index('idx_history_memory_state', 'memory_status_history', ['memory_id', 'new_state'], unique=False)
|
||||
op.create_index('idx_history_user_time', 'memory_status_history', ['changed_by', 'changed_at'], unique=False)
|
||||
op.create_index(op.f('ix_memory_status_history_changed_at'), 'memory_status_history', ['changed_at'], unique=False)
|
||||
op.create_index(op.f('ix_memory_status_history_changed_by'), 'memory_status_history', ['changed_by'], unique=False)
|
||||
op.create_index(op.f('ix_memory_status_history_memory_id'), 'memory_status_history', ['memory_id'], unique=False)
|
||||
op.create_index(op.f('ix_memory_status_history_new_state'), 'memory_status_history', ['new_state'], unique=False)
|
||||
op.create_index(op.f('ix_memory_status_history_old_state'), 'memory_status_history', ['old_state'], unique=False)
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Downgrade schema."""
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.drop_index(op.f('ix_memory_status_history_old_state'), table_name='memory_status_history')
|
||||
op.drop_index(op.f('ix_memory_status_history_new_state'), table_name='memory_status_history')
|
||||
op.drop_index(op.f('ix_memory_status_history_memory_id'), table_name='memory_status_history')
|
||||
op.drop_index(op.f('ix_memory_status_history_changed_by'), table_name='memory_status_history')
|
||||
op.drop_index(op.f('ix_memory_status_history_changed_at'), table_name='memory_status_history')
|
||||
op.drop_index('idx_history_user_time', table_name='memory_status_history')
|
||||
op.drop_index('idx_history_memory_state', table_name='memory_status_history')
|
||||
op.drop_table('memory_status_history')
|
||||
op.drop_index(op.f('ix_memory_categories_memory_id'), table_name='memory_categories')
|
||||
op.drop_index(op.f('ix_memory_categories_category_id'), table_name='memory_categories')
|
||||
op.drop_index('idx_memory_category', table_name='memory_categories')
|
||||
op.drop_table('memory_categories')
|
||||
op.drop_index(op.f('ix_memory_access_logs_memory_id'), table_name='memory_access_logs')
|
||||
op.drop_index(op.f('ix_memory_access_logs_app_id'), table_name='memory_access_logs')
|
||||
op.drop_index(op.f('ix_memory_access_logs_accessed_at'), table_name='memory_access_logs')
|
||||
op.drop_index(op.f('ix_memory_access_logs_access_type'), table_name='memory_access_logs')
|
||||
op.drop_index('idx_access_memory_time', table_name='memory_access_logs')
|
||||
op.drop_index('idx_access_app_time', table_name='memory_access_logs')
|
||||
op.drop_table('memory_access_logs')
|
||||
op.drop_index(op.f('ix_memories_user_id'), table_name='memories')
|
||||
op.drop_index(op.f('ix_memories_state'), table_name='memories')
|
||||
op.drop_index(op.f('ix_memories_deleted_at'), table_name='memories')
|
||||
op.drop_index(op.f('ix_memories_created_at'), table_name='memories')
|
||||
op.drop_index(op.f('ix_memories_archived_at'), table_name='memories')
|
||||
op.drop_index(op.f('ix_memories_app_id'), table_name='memories')
|
||||
op.drop_index('idx_memory_user_state', table_name='memories')
|
||||
op.drop_index('idx_memory_user_app', table_name='memories')
|
||||
op.drop_index('idx_memory_app_state', table_name='memories')
|
||||
op.drop_table('memories')
|
||||
op.drop_index(op.f('ix_apps_owner_id'), table_name='apps')
|
||||
op.drop_index(op.f('ix_apps_name'), table_name='apps')
|
||||
op.drop_index(op.f('ix_apps_is_active'), table_name='apps')
|
||||
op.drop_index(op.f('ix_apps_created_at'), table_name='apps')
|
||||
op.drop_table('apps')
|
||||
op.drop_index(op.f('ix_users_user_id'), table_name='users')
|
||||
op.drop_index(op.f('ix_users_name'), table_name='users')
|
||||
op.drop_index(op.f('ix_users_email'), table_name='users')
|
||||
op.drop_index(op.f('ix_users_created_at'), table_name='users')
|
||||
op.drop_table('users')
|
||||
op.drop_index(op.f('ix_categories_name'), table_name='categories')
|
||||
op.drop_index(op.f('ix_categories_created_at'), table_name='categories')
|
||||
op.drop_table('categories')
|
||||
op.drop_index(op.f('ix_archive_policies_criteria_type'), table_name='archive_policies')
|
||||
op.drop_index(op.f('ix_archive_policies_criteria_id'), table_name='archive_policies')
|
||||
op.drop_index(op.f('ix_archive_policies_created_at'), table_name='archive_policies')
|
||||
op.drop_index('idx_policy_criteria', table_name='archive_policies')
|
||||
op.drop_table('archive_policies')
|
||||
op.drop_index(op.f('ix_access_controls_subject_type'), table_name='access_controls')
|
||||
op.drop_index(op.f('ix_access_controls_subject_id'), table_name='access_controls')
|
||||
op.drop_index(op.f('ix_access_controls_object_type'), table_name='access_controls')
|
||||
op.drop_index(op.f('ix_access_controls_object_id'), table_name='access_controls')
|
||||
op.drop_index(op.f('ix_access_controls_effect'), table_name='access_controls')
|
||||
op.drop_index(op.f('ix_access_controls_created_at'), table_name='access_controls')
|
||||
op.drop_index('idx_access_subject', table_name='access_controls')
|
||||
op.drop_index('idx_access_object', table_name='access_controls')
|
||||
op.drop_table('access_controls')
|
||||
# ### end Alembic commands ###
|
||||
@@ -0,0 +1,40 @@
|
||||
"""add_config_table
|
||||
|
||||
Revision ID: add_config_table
|
||||
Revises: 0b53c747049a
|
||||
Create Date: 2023-06-01 10:00:00.000000
|
||||
|
||||
"""
|
||||
import uuid
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = 'add_config_table'
|
||||
down_revision = '0b53c747049a'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade():
|
||||
# Create configs table if it doesn't exist
|
||||
op.create_table(
|
||||
'configs',
|
||||
sa.Column('id', sa.UUID(), nullable=False, default=lambda: uuid.uuid4()),
|
||||
sa.Column('key', sa.String(), nullable=False),
|
||||
sa.Column('value', sa.JSON(), nullable=False),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=True),
|
||||
sa.Column('updated_at', sa.DateTime(), nullable=True),
|
||||
sa.PrimaryKeyConstraint('id'),
|
||||
sa.UniqueConstraint('key')
|
||||
)
|
||||
|
||||
# Create index for key lookups
|
||||
op.create_index('idx_configs_key', 'configs', ['key'])
|
||||
|
||||
|
||||
def downgrade():
|
||||
# Drop the configs table
|
||||
op.drop_index('idx_configs_key', 'configs')
|
||||
op.drop_table('configs')
|
||||
@@ -0,0 +1,34 @@
|
||||
"""remove_global_unique_constraint_on_app_name_add_composite_unique
|
||||
|
||||
Revision ID: afd00efbd06b
|
||||
Revises: add_config_table
|
||||
Create Date: 2025-06-04 01:59:41.637440
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = 'afd00efbd06b'
|
||||
down_revision: Union[str, None] = 'add_config_table'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Upgrade schema."""
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.drop_index('ix_apps_name', table_name='apps')
|
||||
op.create_index(op.f('ix_apps_name'), 'apps', ['name'], unique=False)
|
||||
op.create_index('idx_app_owner_name', 'apps', ['owner_id', 'name'], unique=True)
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Downgrade schema."""
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.drop_index('idx_app_owner_name', table_name='apps')
|
||||
op.drop_index(op.f('ix_apps_name'), table_name='apps')
|
||||
op.create_index('ix_apps_name', 'apps', ['name'], unique=True)
|
||||
# ### end Alembic commands ###
|
||||
@@ -0,0 +1 @@
|
||||
# This file makes the app directory a Python package
|
||||
@@ -0,0 +1,4 @@
|
||||
import os
|
||||
|
||||
USER_ID = os.getenv("USER", "default_user")
|
||||
DEFAULT_APP_ID = "openmemory"
|
||||
@@ -0,0 +1,30 @@
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import declarative_base, sessionmaker
|
||||
|
||||
# load .env file (make sure you have DATABASE_URL set)
|
||||
load_dotenv()
|
||||
|
||||
DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./openmemory.db")
|
||||
if not DATABASE_URL:
|
||||
raise RuntimeError("DATABASE_URL is not set in environment")
|
||||
|
||||
# SQLAlchemy engine & session
|
||||
engine = create_engine(
|
||||
DATABASE_URL,
|
||||
connect_args={"check_same_thread": False} # Needed for SQLite
|
||||
)
|
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
|
||||
# Base class for models
|
||||
Base = declarative_base()
|
||||
|
||||
# Dependency for FastAPI
|
||||
def get_db():
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
@@ -0,0 +1,574 @@
|
||||
"""
|
||||
MCP Server for OpenMemory with resilient memory client handling.
|
||||
|
||||
This module implements an MCP (Model Context Protocol) server that provides
|
||||
memory operations for OpenMemory. The memory client is initialized lazily
|
||||
to prevent server crashes when external dependencies (like Ollama) are
|
||||
unavailable. If the memory client cannot be initialized, the server will
|
||||
continue running with limited functionality and appropriate error messages.
|
||||
|
||||
Key features:
|
||||
- Lazy memory client initialization
|
||||
- Graceful error handling for unavailable dependencies
|
||||
- Fallback to database-only mode when vector store is unavailable
|
||||
- Proper logging for debugging connection issues
|
||||
- Environment variable parsing for API keys
|
||||
"""
|
||||
|
||||
import contextvars
|
||||
import datetime
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
|
||||
import anyio
|
||||
|
||||
from app.database import SessionLocal
|
||||
from app.models import Memory, MemoryAccessLog, MemoryState, MemoryStatusHistory
|
||||
from app.utils.db import get_user_and_app
|
||||
from app.utils.memory import get_memory_client
|
||||
from app.utils.permissions import check_memory_access_permissions
|
||||
from dotenv import load_dotenv
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.routing import APIRouter
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
from mcp.server.sse import SseServerTransport
|
||||
from mcp.server.streamable_http import StreamableHTTPServerTransport
|
||||
from starlette.responses import Response
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
# Initialize MCP
|
||||
mcp = FastMCP("mem0-mcp-server")
|
||||
|
||||
# Don't initialize memory client at import time - do it lazily when needed
|
||||
def get_memory_client_safe():
|
||||
"""Get memory client with error handling. Returns None if client cannot be initialized."""
|
||||
try:
|
||||
return get_memory_client()
|
||||
except Exception as e:
|
||||
logging.warning(f"Failed to get memory client: {e}")
|
||||
return None
|
||||
|
||||
# Context variables for user_id and client_name
|
||||
user_id_var: contextvars.ContextVar[str] = contextvars.ContextVar("user_id")
|
||||
client_name_var: contextvars.ContextVar[str] = contextvars.ContextVar("client_name")
|
||||
|
||||
# Create a router for MCP endpoints
|
||||
mcp_router = APIRouter(prefix="/mcp")
|
||||
|
||||
# Initialize SSE transport
|
||||
sse = SseServerTransport("/mcp/messages/")
|
||||
|
||||
@mcp.tool(description="Add a new memory. This method is called everytime the user informs anything about themselves, their preferences, or anything that has any relevant information which can be useful in the future conversation. This can also be called when the user asks you to remember something. Set infer to False to store the memory verbatim without LLM fact extraction.")
|
||||
async def add_memories(text: str, infer: bool = True) -> str:
|
||||
uid = user_id_var.get(None)
|
||||
client_name = client_name_var.get(None)
|
||||
|
||||
if not uid:
|
||||
return "Error: user_id not provided"
|
||||
if not client_name:
|
||||
return "Error: client_name not provided"
|
||||
|
||||
# Get memory client safely
|
||||
memory_client = get_memory_client_safe()
|
||||
if not memory_client:
|
||||
return "Error: Memory system is currently unavailable. Please try again later."
|
||||
|
||||
try:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Get or create user and app
|
||||
user, app = get_user_and_app(db, user_id=uid, app_id=client_name)
|
||||
|
||||
# Check if app is active
|
||||
if not app.is_active:
|
||||
return f"Error: App {app.name} is currently paused on OpenMemory. Cannot create new memories."
|
||||
|
||||
response = memory_client.add(text,
|
||||
user_id=uid,
|
||||
metadata={
|
||||
"source_app": "openmemory",
|
||||
"mcp_client": client_name,
|
||||
},
|
||||
infer=infer)
|
||||
|
||||
# Process the response and update database
|
||||
if isinstance(response, dict) and 'results' in response:
|
||||
for result in response['results']:
|
||||
memory_id = uuid.UUID(result['id'])
|
||||
memory = db.query(Memory).filter(Memory.id == memory_id).first()
|
||||
|
||||
if result['event'] == 'ADD':
|
||||
if not memory:
|
||||
memory = Memory(
|
||||
id=memory_id,
|
||||
user_id=user.id,
|
||||
app_id=app.id,
|
||||
content=result['memory'],
|
||||
state=MemoryState.active
|
||||
)
|
||||
db.add(memory)
|
||||
else:
|
||||
memory.state = MemoryState.active
|
||||
memory.content = result['memory']
|
||||
|
||||
# Create history entry
|
||||
history = MemoryStatusHistory(
|
||||
memory_id=memory_id,
|
||||
changed_by=user.id,
|
||||
old_state=MemoryState.deleted if memory else None,
|
||||
new_state=MemoryState.active
|
||||
)
|
||||
db.add(history)
|
||||
|
||||
elif result['event'] == 'DELETE':
|
||||
if memory:
|
||||
memory.state = MemoryState.deleted
|
||||
memory.deleted_at = datetime.datetime.now(datetime.UTC)
|
||||
# Create history entry
|
||||
history = MemoryStatusHistory(
|
||||
memory_id=memory_id,
|
||||
changed_by=user.id,
|
||||
old_state=MemoryState.active,
|
||||
new_state=MemoryState.deleted
|
||||
)
|
||||
db.add(history)
|
||||
|
||||
db.commit()
|
||||
|
||||
return json.dumps(response)
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as e:
|
||||
logging.exception(f"Error adding to memory: {e}")
|
||||
return f"Error adding to memory: {e}"
|
||||
|
||||
|
||||
@mcp.tool(description="Search through stored memories. This method is called EVERYTIME the user asks anything.")
|
||||
async def search_memory(query: str) -> str:
|
||||
uid = user_id_var.get(None)
|
||||
client_name = client_name_var.get(None)
|
||||
if not uid:
|
||||
return "Error: user_id not provided"
|
||||
if not client_name:
|
||||
return "Error: client_name not provided"
|
||||
|
||||
# Get memory client safely
|
||||
memory_client = get_memory_client_safe()
|
||||
if not memory_client:
|
||||
return "Error: Memory system is currently unavailable. Please try again later."
|
||||
|
||||
try:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Get or create user and app
|
||||
user, app = get_user_and_app(db, user_id=uid, app_id=client_name)
|
||||
|
||||
# Get accessible memory IDs based on ACL
|
||||
user_memories = db.query(Memory).filter(Memory.user_id == user.id).all()
|
||||
accessible_memory_ids = [memory.id for memory in user_memories if check_memory_access_permissions(db, memory, app.id)]
|
||||
|
||||
filters = {
|
||||
"user_id": uid
|
||||
}
|
||||
|
||||
embeddings = memory_client.embedding_model.embed(query, "search")
|
||||
|
||||
hits = memory_client.vector_store.search(
|
||||
query=query,
|
||||
vectors=embeddings,
|
||||
limit=10,
|
||||
filters=filters,
|
||||
)
|
||||
|
||||
allowed = set(str(mid) for mid in accessible_memory_ids) if accessible_memory_ids else None
|
||||
|
||||
results = []
|
||||
for h in hits:
|
||||
# All vector db search functions return OutputData class
|
||||
id, score, payload = h.id, h.score, h.payload
|
||||
if allowed and (h.id is None or h.id not in allowed):
|
||||
continue
|
||||
|
||||
results.append({
|
||||
"id": id,
|
||||
"memory": payload.get("data"),
|
||||
"hash": payload.get("hash"),
|
||||
"created_at": payload.get("created_at"),
|
||||
"updated_at": payload.get("updated_at"),
|
||||
"score": score,
|
||||
})
|
||||
|
||||
for r in results:
|
||||
if r.get("id"):
|
||||
access_log = MemoryAccessLog(
|
||||
memory_id=uuid.UUID(r["id"]),
|
||||
app_id=app.id,
|
||||
access_type="search",
|
||||
metadata_={
|
||||
"query": query,
|
||||
"score": r.get("score"),
|
||||
"hash": r.get("hash"),
|
||||
},
|
||||
)
|
||||
db.add(access_log)
|
||||
db.commit()
|
||||
|
||||
return json.dumps({"results": results}, indent=2)
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as e:
|
||||
logging.exception(e)
|
||||
return f"Error searching memory: {e}"
|
||||
|
||||
|
||||
@mcp.tool(description="List all memories in the user's memory")
|
||||
async def list_memories() -> str:
|
||||
uid = user_id_var.get(None)
|
||||
client_name = client_name_var.get(None)
|
||||
if not uid:
|
||||
return "Error: user_id not provided"
|
||||
if not client_name:
|
||||
return "Error: client_name not provided"
|
||||
|
||||
# Get memory client safely
|
||||
memory_client = get_memory_client_safe()
|
||||
if not memory_client:
|
||||
return "Error: Memory system is currently unavailable. Please try again later."
|
||||
|
||||
try:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Get or create user and app
|
||||
user, app = get_user_and_app(db, user_id=uid, app_id=client_name)
|
||||
|
||||
# Get all memories
|
||||
memories = memory_client.get_all(user_id=uid)
|
||||
filtered_memories = []
|
||||
|
||||
# Filter memories based on permissions
|
||||
user_memories = db.query(Memory).filter(Memory.user_id == user.id).all()
|
||||
accessible_memory_ids = [memory.id for memory in user_memories if check_memory_access_permissions(db, memory, app.id)]
|
||||
if isinstance(memories, dict) and 'results' in memories:
|
||||
for memory_data in memories['results']:
|
||||
if 'id' in memory_data:
|
||||
memory_id = uuid.UUID(memory_data['id'])
|
||||
if memory_id in accessible_memory_ids:
|
||||
# Create access log entry
|
||||
access_log = MemoryAccessLog(
|
||||
memory_id=memory_id,
|
||||
app_id=app.id,
|
||||
access_type="list",
|
||||
metadata_={
|
||||
"hash": memory_data.get('hash')
|
||||
}
|
||||
)
|
||||
db.add(access_log)
|
||||
filtered_memories.append(memory_data)
|
||||
db.commit()
|
||||
else:
|
||||
for memory in memories:
|
||||
memory_id = uuid.UUID(memory['id'])
|
||||
memory_obj = db.query(Memory).filter(Memory.id == memory_id).first()
|
||||
if memory_obj and check_memory_access_permissions(db, memory_obj, app.id):
|
||||
# Create access log entry
|
||||
access_log = MemoryAccessLog(
|
||||
memory_id=memory_id,
|
||||
app_id=app.id,
|
||||
access_type="list",
|
||||
metadata_={
|
||||
"hash": memory.get('hash')
|
||||
}
|
||||
)
|
||||
db.add(access_log)
|
||||
filtered_memories.append(memory)
|
||||
db.commit()
|
||||
return json.dumps(filtered_memories, indent=2)
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as e:
|
||||
logging.exception(f"Error getting memories: {e}")
|
||||
return f"Error getting memories: {e}"
|
||||
|
||||
|
||||
@mcp.tool(description="Delete specific memories by their IDs")
|
||||
async def delete_memories(memory_ids: list[str]) -> str:
|
||||
uid = user_id_var.get(None)
|
||||
client_name = client_name_var.get(None)
|
||||
if not uid:
|
||||
return "Error: user_id not provided"
|
||||
if not client_name:
|
||||
return "Error: client_name not provided"
|
||||
|
||||
# Get memory client safely
|
||||
memory_client = get_memory_client_safe()
|
||||
if not memory_client:
|
||||
return "Error: Memory system is currently unavailable. Please try again later."
|
||||
|
||||
try:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Get or create user and app
|
||||
user, app = get_user_and_app(db, user_id=uid, app_id=client_name)
|
||||
|
||||
# Convert string IDs to UUIDs and filter accessible ones
|
||||
requested_ids = [uuid.UUID(mid) for mid in memory_ids]
|
||||
user_memories = db.query(Memory).filter(Memory.user_id == user.id).all()
|
||||
accessible_memory_ids = [memory.id for memory in user_memories if check_memory_access_permissions(db, memory, app.id)]
|
||||
|
||||
# Only delete memories that are both requested and accessible
|
||||
ids_to_delete = [mid for mid in requested_ids if mid in accessible_memory_ids]
|
||||
|
||||
if not ids_to_delete:
|
||||
return "Error: No accessible memories found with provided IDs"
|
||||
|
||||
# Delete from vector store
|
||||
for memory_id in ids_to_delete:
|
||||
try:
|
||||
memory_client.delete(str(memory_id))
|
||||
except Exception as delete_error:
|
||||
logging.warning(f"Failed to delete memory {memory_id} from vector store: {delete_error}")
|
||||
|
||||
# Update each memory's state and create history entries
|
||||
now = datetime.datetime.now(datetime.UTC)
|
||||
for memory_id in ids_to_delete:
|
||||
memory = db.query(Memory).filter(Memory.id == memory_id).first()
|
||||
if memory:
|
||||
# Update memory state
|
||||
memory.state = MemoryState.deleted
|
||||
memory.deleted_at = now
|
||||
|
||||
# Create history entry
|
||||
history = MemoryStatusHistory(
|
||||
memory_id=memory_id,
|
||||
changed_by=user.id,
|
||||
old_state=MemoryState.active,
|
||||
new_state=MemoryState.deleted
|
||||
)
|
||||
db.add(history)
|
||||
|
||||
# Create access log entry
|
||||
access_log = MemoryAccessLog(
|
||||
memory_id=memory_id,
|
||||
app_id=app.id,
|
||||
access_type="delete",
|
||||
metadata_={"operation": "delete_by_id"}
|
||||
)
|
||||
db.add(access_log)
|
||||
|
||||
db.commit()
|
||||
return f"Successfully deleted {len(ids_to_delete)} memories"
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as e:
|
||||
logging.exception(f"Error deleting memories: {e}")
|
||||
return f"Error deleting memories: {e}"
|
||||
|
||||
|
||||
@mcp.tool(description="Delete all memories in the user's memory")
|
||||
async def delete_all_memories() -> str:
|
||||
uid = user_id_var.get(None)
|
||||
client_name = client_name_var.get(None)
|
||||
if not uid:
|
||||
return "Error: user_id not provided"
|
||||
if not client_name:
|
||||
return "Error: client_name not provided"
|
||||
|
||||
# Get memory client safely
|
||||
memory_client = get_memory_client_safe()
|
||||
if not memory_client:
|
||||
return "Error: Memory system is currently unavailable. Please try again later."
|
||||
|
||||
try:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Get or create user and app
|
||||
user, app = get_user_and_app(db, user_id=uid, app_id=client_name)
|
||||
|
||||
user_memories = db.query(Memory).filter(Memory.user_id == user.id).all()
|
||||
accessible_memory_ids = [memory.id for memory in user_memories if check_memory_access_permissions(db, memory, app.id)]
|
||||
|
||||
# delete the accessible memories only
|
||||
for memory_id in accessible_memory_ids:
|
||||
try:
|
||||
memory_client.delete(str(memory_id))
|
||||
except Exception as delete_error:
|
||||
logging.warning(f"Failed to delete memory {memory_id} from vector store: {delete_error}")
|
||||
|
||||
# Update each memory's state and create history entries
|
||||
now = datetime.datetime.now(datetime.UTC)
|
||||
for memory_id in accessible_memory_ids:
|
||||
memory = db.query(Memory).filter(Memory.id == memory_id).first()
|
||||
# Update memory state
|
||||
memory.state = MemoryState.deleted
|
||||
memory.deleted_at = now
|
||||
|
||||
# Create history entry
|
||||
history = MemoryStatusHistory(
|
||||
memory_id=memory_id,
|
||||
changed_by=user.id,
|
||||
old_state=MemoryState.active,
|
||||
new_state=MemoryState.deleted
|
||||
)
|
||||
db.add(history)
|
||||
|
||||
# Create access log entry
|
||||
access_log = MemoryAccessLog(
|
||||
memory_id=memory_id,
|
||||
app_id=app.id,
|
||||
access_type="delete_all",
|
||||
metadata_={"operation": "bulk_delete"}
|
||||
)
|
||||
db.add(access_log)
|
||||
|
||||
db.commit()
|
||||
return "Successfully deleted all memories"
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as e:
|
||||
logging.exception(f"Error deleting memories: {e}")
|
||||
return f"Error deleting memories: {e}"
|
||||
|
||||
|
||||
@mcp_router.get("/{client_name}/sse/{user_id}")
|
||||
async def handle_sse(request: Request):
|
||||
"""Handle SSE connections for a specific user and client"""
|
||||
# Extract user_id and client_name from path parameters
|
||||
uid = request.path_params.get("user_id")
|
||||
user_token = user_id_var.set(uid or "")
|
||||
client_name = request.path_params.get("client_name")
|
||||
client_token = client_name_var.set(client_name or "")
|
||||
|
||||
try:
|
||||
# NOTE: request._send is the raw ASGI `send` callable. Starlette does not
|
||||
# expose it publicly, but the MCP SDK transports require the raw ASGI
|
||||
# interface (scope, receive, send). This is the standard pattern from the
|
||||
# MCP Python SDK examples.
|
||||
async with sse.connect_sse(
|
||||
request.scope,
|
||||
request.receive,
|
||||
request._send,
|
||||
) as (read_stream, write_stream):
|
||||
await mcp._mcp_server.run(
|
||||
read_stream,
|
||||
write_stream,
|
||||
mcp._mcp_server.create_initialization_options(),
|
||||
)
|
||||
finally:
|
||||
# Clean up context variables
|
||||
user_id_var.reset(user_token)
|
||||
client_name_var.reset(client_token)
|
||||
|
||||
|
||||
@mcp_router.post("/messages/")
|
||||
async def handle_get_message(request: Request):
|
||||
return await handle_post_message(request)
|
||||
|
||||
|
||||
@mcp_router.post("/{client_name}/sse/{user_id}/messages/")
|
||||
async def handle_post_message(request: Request):
|
||||
return await handle_post_message(request)
|
||||
|
||||
async def handle_post_message(request: Request):
|
||||
"""Handle POST messages for SSE"""
|
||||
try:
|
||||
body = await request.body()
|
||||
|
||||
# Create a simple receive function that returns the body
|
||||
async def receive():
|
||||
return {"type": "http.request", "body": body, "more_body": False}
|
||||
|
||||
# Create a simple send function that does nothing
|
||||
async def send(message):
|
||||
return {}
|
||||
|
||||
# Call handle_post_message with the correct arguments
|
||||
await sse.handle_post_message(request.scope, receive, send)
|
||||
|
||||
# Return a success response
|
||||
return {"status": "ok"}
|
||||
finally:
|
||||
pass
|
||||
|
||||
|
||||
@mcp_router.api_route("/{client_name}/http/{user_id}", methods=["POST", "GET", "DELETE"])
|
||||
async def handle_streamable_http(request: Request):
|
||||
"""Handle Streamable HTTP connections for a specific user and client.
|
||||
|
||||
Uses the Streamable HTTP transport (MCP spec 2025-03-26+) which replaces
|
||||
the deprecated SSE transport. Runs in stateless mode — each request is
|
||||
handled independently with no persistent session.
|
||||
|
||||
The transport writes its response directly to the ASGI ``send`` callable.
|
||||
We intercept it via ``capture_send`` so we can return a proper ``Response``
|
||||
to FastAPI — otherwise FastAPI would also try to send its own response,
|
||||
causing a "double-response" bug.
|
||||
"""
|
||||
uid = request.path_params.get("user_id")
|
||||
user_token = user_id_var.set(uid or "")
|
||||
client_name = request.path_params.get("client_name")
|
||||
client_token = client_name_var.set(client_name or "")
|
||||
|
||||
# Intercept the ASGI messages the transport sends so we can return them
|
||||
# as a single Response to FastAPI. Without this, FastAPI would attempt to
|
||||
# write its own response after the transport already wrote one.
|
||||
response_started = False
|
||||
response_status = 200
|
||||
response_headers: list[tuple[bytes, bytes]] = []
|
||||
response_body = bytearray()
|
||||
|
||||
async def capture_send(message):
|
||||
nonlocal response_started, response_status
|
||||
if message["type"] == "http.response.start":
|
||||
response_started = True
|
||||
response_status = message["status"]
|
||||
response_headers.extend(message.get("headers", []))
|
||||
elif message["type"] == "http.response.body":
|
||||
response_body.extend(message.get("body", b""))
|
||||
|
||||
try:
|
||||
transport = StreamableHTTPServerTransport(
|
||||
mcp_session_id=None,
|
||||
is_json_response_enabled=True,
|
||||
)
|
||||
|
||||
async with anyio.create_task_group() as tg:
|
||||
|
||||
async def run_server(*, task_status=anyio.TASK_STATUS_IGNORED):
|
||||
async with transport.connect() as (read_stream, write_stream):
|
||||
task_status.started()
|
||||
await mcp._mcp_server.run(
|
||||
read_stream,
|
||||
write_stream,
|
||||
mcp._mcp_server.create_initialization_options(),
|
||||
stateless=True,
|
||||
)
|
||||
|
||||
await tg.start(run_server)
|
||||
await transport.handle_request(request.scope, request.receive, capture_send)
|
||||
await transport.terminate()
|
||||
tg.cancel_scope.cancel()
|
||||
finally:
|
||||
user_id_var.reset(user_token)
|
||||
client_name_var.reset(client_token)
|
||||
|
||||
if not response_started:
|
||||
return Response(status_code=500, content=b"Transport did not produce a response")
|
||||
|
||||
# Header dict conversion is safe here: the MCP transport in stateless JSON
|
||||
# mode only emits single-valued headers (Content-Type, Content-Length).
|
||||
return Response(
|
||||
content=bytes(response_body),
|
||||
status_code=response_status,
|
||||
headers={k.decode(): v.decode() for k, v in response_headers},
|
||||
)
|
||||
|
||||
|
||||
def setup_mcp_server(app: FastAPI):
|
||||
"""Setup MCP server with the FastAPI application"""
|
||||
mcp._mcp_server.name = "mem0-mcp-server"
|
||||
|
||||
# Include MCP router in the FastAPI app
|
||||
app.include_router(mcp_router)
|
||||
@@ -0,0 +1,243 @@
|
||||
import datetime
|
||||
import enum
|
||||
import uuid
|
||||
|
||||
import sqlalchemy as sa
|
||||
from app.database import Base
|
||||
from app.utils.categorization import get_categories_for_memory
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
UUID,
|
||||
Boolean,
|
||||
Column,
|
||||
DateTime,
|
||||
Enum,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
String,
|
||||
Table,
|
||||
event,
|
||||
)
|
||||
from sqlalchemy.orm import Session, relationship
|
||||
|
||||
|
||||
def get_current_utc_time():
|
||||
"""Get current UTC time"""
|
||||
return datetime.datetime.now(datetime.UTC)
|
||||
|
||||
|
||||
class MemoryState(enum.Enum):
|
||||
active = "active"
|
||||
paused = "paused"
|
||||
archived = "archived"
|
||||
deleted = "deleted"
|
||||
|
||||
|
||||
class User(Base):
|
||||
__tablename__ = "users"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
user_id = Column(String, nullable=False, unique=True, index=True)
|
||||
name = Column(String, nullable=True, index=True)
|
||||
email = Column(String, unique=True, nullable=True, index=True)
|
||||
metadata_ = Column('metadata', JSON, default=dict)
|
||||
created_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
updated_at = Column(DateTime,
|
||||
default=get_current_utc_time,
|
||||
onupdate=get_current_utc_time)
|
||||
|
||||
apps = relationship("App", back_populates="owner")
|
||||
memories = relationship("Memory", back_populates="user")
|
||||
|
||||
|
||||
class App(Base):
|
||||
__tablename__ = "apps"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
owner_id = Column(UUID, ForeignKey("users.id"), nullable=False, index=True)
|
||||
name = Column(String, nullable=False, index=True)
|
||||
description = Column(String)
|
||||
metadata_ = Column('metadata', JSON, default=dict)
|
||||
is_active = Column(Boolean, default=True, index=True)
|
||||
created_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
updated_at = Column(DateTime,
|
||||
default=get_current_utc_time,
|
||||
onupdate=get_current_utc_time)
|
||||
|
||||
owner = relationship("User", back_populates="apps")
|
||||
memories = relationship("Memory", back_populates="app")
|
||||
|
||||
__table_args__ = (
|
||||
sa.UniqueConstraint('owner_id', 'name', name='idx_app_owner_name'),
|
||||
)
|
||||
|
||||
|
||||
class Config(Base):
|
||||
__tablename__ = "configs"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
key = Column(String, unique=True, nullable=False, index=True)
|
||||
value = Column(JSON, nullable=False)
|
||||
created_at = Column(DateTime, default=get_current_utc_time)
|
||||
updated_at = Column(DateTime,
|
||||
default=get_current_utc_time,
|
||||
onupdate=get_current_utc_time)
|
||||
|
||||
|
||||
class Memory(Base):
|
||||
__tablename__ = "memories"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
user_id = Column(UUID, ForeignKey("users.id"), nullable=False, index=True)
|
||||
app_id = Column(UUID, ForeignKey("apps.id"), nullable=False, index=True)
|
||||
content = Column(String, nullable=False)
|
||||
vector = Column(String)
|
||||
metadata_ = Column('metadata', JSON, default=dict)
|
||||
state = Column(Enum(MemoryState), default=MemoryState.active, index=True)
|
||||
created_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
updated_at = Column(DateTime,
|
||||
default=get_current_utc_time,
|
||||
onupdate=get_current_utc_time)
|
||||
archived_at = Column(DateTime, nullable=True, index=True)
|
||||
deleted_at = Column(DateTime, nullable=True, index=True)
|
||||
|
||||
user = relationship("User", back_populates="memories")
|
||||
app = relationship("App", back_populates="memories")
|
||||
categories = relationship("Category", secondary="memory_categories", back_populates="memories")
|
||||
|
||||
__table_args__ = (
|
||||
Index('idx_memory_user_state', 'user_id', 'state'),
|
||||
Index('idx_memory_app_state', 'app_id', 'state'),
|
||||
Index('idx_memory_user_app', 'user_id', 'app_id'),
|
||||
)
|
||||
|
||||
|
||||
class Category(Base):
|
||||
__tablename__ = "categories"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
name = Column(String, unique=True, nullable=False, index=True)
|
||||
description = Column(String)
|
||||
created_at = Column(DateTime, default=datetime.datetime.now(datetime.UTC), index=True)
|
||||
updated_at = Column(DateTime,
|
||||
default=get_current_utc_time,
|
||||
onupdate=get_current_utc_time)
|
||||
|
||||
memories = relationship("Memory", secondary="memory_categories", back_populates="categories")
|
||||
|
||||
memory_categories = Table(
|
||||
"memory_categories", Base.metadata,
|
||||
Column("memory_id", UUID, ForeignKey("memories.id"), primary_key=True, index=True),
|
||||
Column("category_id", UUID, ForeignKey("categories.id"), primary_key=True, index=True),
|
||||
Index('idx_memory_category', 'memory_id', 'category_id')
|
||||
)
|
||||
|
||||
|
||||
class AccessControl(Base):
|
||||
__tablename__ = "access_controls"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
subject_type = Column(String, nullable=False, index=True)
|
||||
subject_id = Column(UUID, nullable=True, index=True)
|
||||
object_type = Column(String, nullable=False, index=True)
|
||||
object_id = Column(UUID, nullable=True, index=True)
|
||||
effect = Column(String, nullable=False, index=True)
|
||||
created_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
|
||||
__table_args__ = (
|
||||
Index('idx_access_subject', 'subject_type', 'subject_id'),
|
||||
Index('idx_access_object', 'object_type', 'object_id'),
|
||||
)
|
||||
|
||||
|
||||
class ArchivePolicy(Base):
|
||||
__tablename__ = "archive_policies"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
criteria_type = Column(String, nullable=False, index=True)
|
||||
criteria_id = Column(UUID, nullable=True, index=True)
|
||||
days_to_archive = Column(Integer, nullable=False)
|
||||
created_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
|
||||
__table_args__ = (
|
||||
Index('idx_policy_criteria', 'criteria_type', 'criteria_id'),
|
||||
)
|
||||
|
||||
|
||||
class MemoryStatusHistory(Base):
|
||||
__tablename__ = "memory_status_history"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
memory_id = Column(UUID, ForeignKey("memories.id"), nullable=False, index=True)
|
||||
changed_by = Column(UUID, ForeignKey("users.id"), nullable=False, index=True)
|
||||
old_state = Column(Enum(MemoryState), nullable=False, index=True)
|
||||
new_state = Column(Enum(MemoryState), nullable=False, index=True)
|
||||
changed_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
|
||||
__table_args__ = (
|
||||
Index('idx_history_memory_state', 'memory_id', 'new_state'),
|
||||
Index('idx_history_user_time', 'changed_by', 'changed_at'),
|
||||
)
|
||||
|
||||
|
||||
class MemoryAccessLog(Base):
|
||||
__tablename__ = "memory_access_logs"
|
||||
id = Column(UUID, primary_key=True, default=lambda: uuid.uuid4())
|
||||
memory_id = Column(UUID, ForeignKey("memories.id"), nullable=False, index=True)
|
||||
app_id = Column(UUID, ForeignKey("apps.id"), nullable=False, index=True)
|
||||
accessed_at = Column(DateTime, default=get_current_utc_time, index=True)
|
||||
access_type = Column(String, nullable=False, index=True)
|
||||
metadata_ = Column('metadata', JSON, default=dict)
|
||||
|
||||
__table_args__ = (
|
||||
Index('idx_access_memory_time', 'memory_id', 'accessed_at'),
|
||||
Index('idx_access_app_time', 'app_id', 'accessed_at'),
|
||||
)
|
||||
|
||||
def categorize_memory(memory: Memory, db: Session) -> None:
|
||||
"""Categorize a memory using OpenAI and store the categories in the database."""
|
||||
try:
|
||||
# Get categories from OpenAI
|
||||
categories = get_categories_for_memory(memory.content)
|
||||
|
||||
# Get or create categories in the database
|
||||
for category_name in categories:
|
||||
category = db.query(Category).filter(Category.name == category_name).first()
|
||||
if not category:
|
||||
category = Category(
|
||||
name=category_name,
|
||||
description=f"Automatically created category for {category_name}"
|
||||
)
|
||||
db.add(category)
|
||||
db.flush() # Flush to get the category ID
|
||||
|
||||
# Check if the memory-category association already exists
|
||||
existing = db.execute(
|
||||
memory_categories.select().where(
|
||||
(memory_categories.c.memory_id == memory.id) &
|
||||
(memory_categories.c.category_id == category.id)
|
||||
)
|
||||
).first()
|
||||
|
||||
if not existing:
|
||||
# Create the association
|
||||
db.execute(
|
||||
memory_categories.insert().values(
|
||||
memory_id=memory.id,
|
||||
category_id=category.id
|
||||
)
|
||||
)
|
||||
|
||||
db.commit()
|
||||
except Exception as e:
|
||||
db.rollback()
|
||||
print(f"Error categorizing memory: {e}")
|
||||
|
||||
|
||||
@event.listens_for(Memory, 'after_insert')
|
||||
def after_memory_insert(mapper, connection, target):
|
||||
"""Trigger categorization after a memory is inserted."""
|
||||
db = Session(bind=connection)
|
||||
categorize_memory(target, db)
|
||||
db.close()
|
||||
|
||||
|
||||
@event.listens_for(Memory, 'after_update')
|
||||
def after_memory_update(mapper, connection, target):
|
||||
"""Trigger categorization after a memory is updated."""
|
||||
db = Session(bind=connection)
|
||||
categorize_memory(target, db)
|
||||
db.close()
|
||||
@@ -0,0 +1,7 @@
|
||||
from .apps import router as apps_router
|
||||
from .backup import router as backup_router
|
||||
from .config import router as config_router
|
||||
from .memories import router as memories_router
|
||||
from .stats import router as stats_router
|
||||
|
||||
__all__ = ["memories_router", "apps_router", "stats_router", "config_router", "backup_router"]
|
||||
@@ -0,0 +1,223 @@
|
||||
from typing import Optional
|
||||
from uuid import UUID
|
||||
|
||||
from app.database import get_db
|
||||
from app.models import App, Memory, MemoryAccessLog, MemoryState
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from sqlalchemy import desc, func
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
|
||||
router = APIRouter(prefix="/api/v1/apps", tags=["apps"])
|
||||
|
||||
# Helper functions
|
||||
def get_app_or_404(db: Session, app_id: UUID) -> App:
|
||||
app = db.query(App).filter(App.id == app_id).first()
|
||||
if not app:
|
||||
raise HTTPException(status_code=404, detail="App not found")
|
||||
return app
|
||||
|
||||
# List all apps with filtering
|
||||
@router.get("/")
|
||||
async def list_apps(
|
||||
name: Optional[str] = None,
|
||||
is_active: Optional[bool] = None,
|
||||
sort_by: str = 'name',
|
||||
sort_direction: str = 'asc',
|
||||
page: int = Query(1, ge=1),
|
||||
page_size: int = Query(10, ge=1, le=100),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
# Create a subquery for memory counts
|
||||
memory_counts = db.query(
|
||||
Memory.app_id,
|
||||
func.count(Memory.id).label('memory_count')
|
||||
).filter(
|
||||
Memory.state.in_([MemoryState.active, MemoryState.paused, MemoryState.archived])
|
||||
).group_by(Memory.app_id).subquery()
|
||||
|
||||
# Create a subquery for access counts
|
||||
access_counts = db.query(
|
||||
MemoryAccessLog.app_id,
|
||||
func.count(func.distinct(MemoryAccessLog.memory_id)).label('access_count')
|
||||
).group_by(MemoryAccessLog.app_id).subquery()
|
||||
|
||||
# Base query
|
||||
query = db.query(
|
||||
App,
|
||||
func.coalesce(memory_counts.c.memory_count, 0).label('total_memories_created'),
|
||||
func.coalesce(access_counts.c.access_count, 0).label('total_memories_accessed')
|
||||
)
|
||||
|
||||
# Join with subqueries
|
||||
query = query.outerjoin(
|
||||
memory_counts,
|
||||
App.id == memory_counts.c.app_id
|
||||
).outerjoin(
|
||||
access_counts,
|
||||
App.id == access_counts.c.app_id
|
||||
)
|
||||
|
||||
if name:
|
||||
query = query.filter(App.name.ilike(f"%{name}%"))
|
||||
|
||||
if is_active is not None:
|
||||
query = query.filter(App.is_active == is_active)
|
||||
|
||||
# Apply sorting
|
||||
if sort_by == 'name':
|
||||
sort_field = App.name
|
||||
elif sort_by == 'memories':
|
||||
sort_field = func.coalesce(memory_counts.c.memory_count, 0)
|
||||
elif sort_by == 'memories_accessed':
|
||||
sort_field = func.coalesce(access_counts.c.access_count, 0)
|
||||
else:
|
||||
sort_field = App.name # default sort
|
||||
|
||||
if sort_direction == 'desc':
|
||||
query = query.order_by(desc(sort_field))
|
||||
else:
|
||||
query = query.order_by(sort_field)
|
||||
|
||||
total = query.count()
|
||||
apps = query.offset((page - 1) * page_size).limit(page_size).all()
|
||||
|
||||
return {
|
||||
"total": total,
|
||||
"page": page,
|
||||
"page_size": page_size,
|
||||
"apps": [
|
||||
{
|
||||
"id": app[0].id,
|
||||
"name": app[0].name,
|
||||
"is_active": app[0].is_active,
|
||||
"total_memories_created": app[1],
|
||||
"total_memories_accessed": app[2]
|
||||
}
|
||||
for app in apps
|
||||
]
|
||||
}
|
||||
|
||||
# Get app details
|
||||
@router.get("/{app_id}")
|
||||
async def get_app_details(
|
||||
app_id: UUID,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
app = get_app_or_404(db, app_id)
|
||||
|
||||
# Get memory access statistics
|
||||
access_stats = db.query(
|
||||
func.count(MemoryAccessLog.id).label("total_memories_accessed"),
|
||||
func.min(MemoryAccessLog.accessed_at).label("first_accessed"),
|
||||
func.max(MemoryAccessLog.accessed_at).label("last_accessed")
|
||||
).filter(MemoryAccessLog.app_id == app_id).first()
|
||||
|
||||
return {
|
||||
"is_active": app.is_active,
|
||||
"total_memories_created": db.query(Memory)
|
||||
.filter(Memory.app_id == app_id)
|
||||
.count(),
|
||||
"total_memories_accessed": access_stats.total_memories_accessed or 0,
|
||||
"first_accessed": access_stats.first_accessed,
|
||||
"last_accessed": access_stats.last_accessed
|
||||
}
|
||||
|
||||
# List memories created by app
|
||||
@router.get("/{app_id}/memories")
|
||||
async def list_app_memories(
|
||||
app_id: UUID,
|
||||
page: int = Query(1, ge=1),
|
||||
page_size: int = Query(10, ge=1, le=100),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
get_app_or_404(db, app_id)
|
||||
query = db.query(Memory).filter(
|
||||
Memory.app_id == app_id,
|
||||
Memory.state.in_([MemoryState.active, MemoryState.paused, MemoryState.archived])
|
||||
)
|
||||
# Add eager loading for categories
|
||||
query = query.options(joinedload(Memory.categories))
|
||||
total = query.count()
|
||||
memories = query.order_by(Memory.created_at.desc()).offset((page - 1) * page_size).limit(page_size).all()
|
||||
|
||||
return {
|
||||
"total": total,
|
||||
"page": page,
|
||||
"page_size": page_size,
|
||||
"memories": [
|
||||
{
|
||||
"id": memory.id,
|
||||
"content": memory.content,
|
||||
"created_at": memory.created_at,
|
||||
"state": memory.state.value,
|
||||
"app_id": memory.app_id,
|
||||
"categories": [category.name for category in memory.categories],
|
||||
"metadata_": memory.metadata_
|
||||
}
|
||||
for memory in memories
|
||||
]
|
||||
}
|
||||
|
||||
# List memories accessed by app
|
||||
@router.get("/{app_id}/accessed")
|
||||
async def list_app_accessed_memories(
|
||||
app_id: UUID,
|
||||
page: int = Query(1, ge=1),
|
||||
page_size: int = Query(10, ge=1, le=100),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
|
||||
# Get memories with access counts
|
||||
query = db.query(
|
||||
Memory,
|
||||
func.count(MemoryAccessLog.id).label("access_count")
|
||||
).join(
|
||||
MemoryAccessLog,
|
||||
Memory.id == MemoryAccessLog.memory_id
|
||||
).filter(
|
||||
MemoryAccessLog.app_id == app_id
|
||||
).group_by(
|
||||
Memory.id
|
||||
).order_by(
|
||||
desc("access_count")
|
||||
)
|
||||
|
||||
# Add eager loading for categories
|
||||
query = query.options(joinedload(Memory.categories))
|
||||
|
||||
total = query.count()
|
||||
results = query.offset((page - 1) * page_size).limit(page_size).all()
|
||||
|
||||
return {
|
||||
"total": total,
|
||||
"page": page,
|
||||
"page_size": page_size,
|
||||
"memories": [
|
||||
{
|
||||
"memory": {
|
||||
"id": memory.id,
|
||||
"content": memory.content,
|
||||
"created_at": memory.created_at,
|
||||
"state": memory.state.value,
|
||||
"app_id": memory.app_id,
|
||||
"app_name": memory.app.name if memory.app else None,
|
||||
"categories": [category.name for category in memory.categories],
|
||||
"metadata_": memory.metadata_
|
||||
},
|
||||
"access_count": count
|
||||
}
|
||||
for memory, count in results
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
@router.put("/{app_id}")
|
||||
async def update_app_details(
|
||||
app_id: UUID,
|
||||
is_active: bool,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
app = get_app_or_404(db, app_id)
|
||||
app.is_active = is_active
|
||||
db.commit()
|
||||
return {"status": "success", "message": "Updated app details successfully"}
|
||||
@@ -0,0 +1,499 @@
|
||||
from datetime import UTC, datetime
|
||||
import io
|
||||
import json
|
||||
import gzip
|
||||
import zipfile
|
||||
from typing import Optional, List, Dict, Any
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File, Query, Form
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
from sqlalchemy import and_
|
||||
|
||||
from app.database import get_db
|
||||
from app.models import (
|
||||
User, App, Memory, MemoryState, Category, memory_categories,
|
||||
MemoryStatusHistory, AccessControl
|
||||
)
|
||||
from app.utils.memory import get_memory_client
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
router = APIRouter(prefix="/api/v1/backup", tags=["backup"])
|
||||
|
||||
class ExportRequest(BaseModel):
|
||||
user_id: str
|
||||
app_id: Optional[UUID] = None
|
||||
from_date: Optional[int] = None
|
||||
to_date: Optional[int] = None
|
||||
include_vectors: bool = True
|
||||
|
||||
def _iso(dt: Optional[datetime]) -> Optional[str]:
|
||||
if isinstance(dt, datetime):
|
||||
try:
|
||||
return dt.astimezone(UTC).isoformat()
|
||||
except:
|
||||
return dt.replace(tzinfo=UTC).isoformat()
|
||||
return None
|
||||
|
||||
def _parse_iso(dt: Optional[str]) -> Optional[datetime]:
|
||||
if not dt:
|
||||
return None
|
||||
try:
|
||||
return datetime.fromisoformat(dt)
|
||||
except Exception:
|
||||
try:
|
||||
return datetime.fromisoformat(dt.replace("Z", "+00:00"))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def _export_sqlite(db: Session, req: ExportRequest) -> Dict[str, Any]:
|
||||
user = db.query(User).filter(User.user_id == req.user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
time_filters = []
|
||||
if req.from_date:
|
||||
time_filters.append(Memory.created_at >= datetime.fromtimestamp(req.from_date, tz=UTC))
|
||||
if req.to_date:
|
||||
time_filters.append(Memory.created_at <= datetime.fromtimestamp(req.to_date, tz=UTC))
|
||||
|
||||
mem_q = (
|
||||
db.query(Memory)
|
||||
.options(joinedload(Memory.categories), joinedload(Memory.app))
|
||||
.filter(
|
||||
Memory.user_id == user.id,
|
||||
*(time_filters or []),
|
||||
* ( [Memory.app_id == req.app_id] if req.app_id else [] ),
|
||||
)
|
||||
)
|
||||
|
||||
memories = mem_q.all()
|
||||
memory_ids = [m.id for m in memories]
|
||||
|
||||
app_ids = sorted({m.app_id for m in memories if m.app_id})
|
||||
apps = db.query(App).filter(App.id.in_(app_ids)).all() if app_ids else []
|
||||
|
||||
cats = sorted({c for m in memories for c in m.categories}, key = lambda c: str(c.id))
|
||||
|
||||
mc_rows = db.execute(
|
||||
memory_categories.select().where(memory_categories.c.memory_id.in_(memory_ids))
|
||||
).fetchall() if memory_ids else []
|
||||
|
||||
history = db.query(MemoryStatusHistory).filter(MemoryStatusHistory.memory_id.in_(memory_ids)).all() if memory_ids else []
|
||||
|
||||
acls = db.query(AccessControl).filter(
|
||||
AccessControl.subject_type == "app",
|
||||
AccessControl.subject_id.in_(app_ids) if app_ids else False
|
||||
).all() if app_ids else []
|
||||
|
||||
return {
|
||||
"user": {
|
||||
"id": str(user.id),
|
||||
"user_id": user.user_id,
|
||||
"name": user.name,
|
||||
"email": user.email,
|
||||
"metadata": user.metadata_,
|
||||
"created_at": _iso(user.created_at),
|
||||
"updated_at": _iso(user.updated_at)
|
||||
},
|
||||
"apps": [
|
||||
{
|
||||
"id": str(a.id),
|
||||
"owner_id": str(a.owner_id),
|
||||
"name": a.name,
|
||||
"description": a.description,
|
||||
"metadata": a.metadata_,
|
||||
"is_active": a.is_active,
|
||||
"created_at": _iso(a.created_at),
|
||||
"updated_at": _iso(a.updated_at),
|
||||
}
|
||||
for a in apps
|
||||
],
|
||||
"categories": [
|
||||
{
|
||||
"id": str(c.id),
|
||||
"name": c.name,
|
||||
"description": c.description,
|
||||
"created_at": _iso(c.created_at),
|
||||
"updated_at": _iso(c.updated_at),
|
||||
}
|
||||
for c in cats
|
||||
],
|
||||
"memories": [
|
||||
{
|
||||
"id": str(m.id),
|
||||
"user_id": str(m.user_id),
|
||||
"app_id": str(m.app_id) if m.app_id else None,
|
||||
"content": m.content,
|
||||
"metadata": m.metadata_,
|
||||
"state": m.state.value,
|
||||
"created_at": _iso(m.created_at),
|
||||
"updated_at": _iso(m.updated_at),
|
||||
"archived_at": _iso(m.archived_at),
|
||||
"deleted_at": _iso(m.deleted_at),
|
||||
"category_ids": [str(c.id) for c in m.categories], #TODO: figure out a way to add category names simply to this
|
||||
}
|
||||
for m in memories
|
||||
],
|
||||
"memory_categories": [
|
||||
{"memory_id": str(r.memory_id), "category_id": str(r.category_id)}
|
||||
for r in mc_rows
|
||||
],
|
||||
"status_history": [
|
||||
{
|
||||
"id": str(h.id),
|
||||
"memory_id": str(h.memory_id),
|
||||
"changed_by": str(h.changed_by),
|
||||
"old_state": h.old_state.value,
|
||||
"new_state": h.new_state.value,
|
||||
"changed_at": _iso(h.changed_at),
|
||||
}
|
||||
for h in history
|
||||
],
|
||||
"access_controls": [
|
||||
{
|
||||
"id": str(ac.id),
|
||||
"subject_type": ac.subject_type,
|
||||
"subject_id": str(ac.subject_id) if ac.subject_id else None,
|
||||
"object_type": ac.object_type,
|
||||
"object_id": str(ac.object_id) if ac.object_id else None,
|
||||
"effect": ac.effect,
|
||||
"created_at": _iso(ac.created_at),
|
||||
}
|
||||
for ac in acls
|
||||
],
|
||||
"export_meta": {
|
||||
"app_id_filter": str(req.app_id) if req.app_id else None,
|
||||
"from_date": req.from_date,
|
||||
"to_date": req.to_date,
|
||||
"version": "1",
|
||||
"generated_at": datetime.now(UTC).isoformat(),
|
||||
},
|
||||
}
|
||||
|
||||
def _export_logical_memories_gz(
|
||||
db: Session,
|
||||
*,
|
||||
user_id: str,
|
||||
app_id: Optional[UUID] = None,
|
||||
from_date: Optional[int] = None,
|
||||
to_date: Optional[int] = None
|
||||
) -> bytes:
|
||||
"""
|
||||
Export a provider-agnostic backup of memories so they can be restored to any vector DB
|
||||
by re-embedding content. One JSON object per line, gzip-compressed.
|
||||
|
||||
Schema (per line):
|
||||
{
|
||||
"id": "<uuid>",
|
||||
"content": "<text>",
|
||||
"metadata": {...},
|
||||
"created_at": "<iso8601 or null>",
|
||||
"updated_at": "<iso8601 or null>",
|
||||
"state": "active|paused|archived|deleted",
|
||||
"app": "<app name or null>",
|
||||
"categories": ["catA", "catB", ...]
|
||||
}
|
||||
"""
|
||||
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
time_filters = []
|
||||
if from_date:
|
||||
time_filters.append(Memory.created_at >= datetime.fromtimestamp(from_date, tz=UTC))
|
||||
if to_date:
|
||||
time_filters.append(Memory.created_at <= datetime.fromtimestamp(to_date, tz=UTC))
|
||||
|
||||
q = (
|
||||
db.query(Memory)
|
||||
.options(joinedload(Memory.categories), joinedload(Memory.app))
|
||||
.filter(
|
||||
Memory.user_id == user.id,
|
||||
*(time_filters or []),
|
||||
)
|
||||
)
|
||||
if app_id:
|
||||
q = q.filter(Memory.app_id == app_id)
|
||||
|
||||
buf = io.BytesIO()
|
||||
with gzip.GzipFile(fileobj=buf, mode="wb") as gz:
|
||||
for m in q.all():
|
||||
record = {
|
||||
"id": str(m.id),
|
||||
"content": m.content,
|
||||
"metadata": m.metadata_ or {},
|
||||
"created_at": _iso(m.created_at),
|
||||
"updated_at": _iso(m.updated_at),
|
||||
"state": m.state.value,
|
||||
"app": m.app.name if m.app else None,
|
||||
"categories": [c.name for c in m.categories],
|
||||
}
|
||||
gz.write((json.dumps(record) + "\n").encode("utf-8"))
|
||||
return buf.getvalue()
|
||||
|
||||
@router.post("/export")
|
||||
async def export_backup(req: ExportRequest, db: Session = Depends(get_db)):
|
||||
sqlite_payload = _export_sqlite(db=db, req=req)
|
||||
memories_blob = _export_logical_memories_gz(
|
||||
db=db,
|
||||
user_id=req.user_id,
|
||||
app_id=req.app_id,
|
||||
from_date=req.from_date,
|
||||
to_date=req.to_date,
|
||||
|
||||
)
|
||||
|
||||
#TODO: add vector store specific exports in future for speed
|
||||
|
||||
zip_buf = io.BytesIO()
|
||||
with zipfile.ZipFile(zip_buf, "w", compression=zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.writestr("memories.json", json.dumps(sqlite_payload, indent=2))
|
||||
zf.writestr("memories.jsonl.gz", memories_blob)
|
||||
|
||||
zip_buf.seek(0)
|
||||
return StreamingResponse(
|
||||
zip_buf,
|
||||
media_type="application/zip",
|
||||
headers={"Content-Disposition": f'attachment; filename="memories_export_{req.user_id}.zip"'},
|
||||
)
|
||||
|
||||
@router.post("/import")
|
||||
async def import_backup(
|
||||
file: UploadFile = File(..., description="Zip with memories.json and memories.jsonl.gz"),
|
||||
user_id: str = Form(..., description="Import memories into this user_id"),
|
||||
mode: str = Query("overwrite"),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
if not file.filename.endswith(".zip"):
|
||||
raise HTTPException(status_code=400, detail="Expected a zip file.")
|
||||
|
||||
if mode not in {"skip", "overwrite"}:
|
||||
raise HTTPException(status_code=400, detail="Invalid mode. Must be 'skip' or 'overwrite'.")
|
||||
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
content = await file.read()
|
||||
try:
|
||||
with zipfile.ZipFile(io.BytesIO(content), "r") as zf:
|
||||
names = zf.namelist()
|
||||
|
||||
def find_member(filename: str) -> Optional[str]:
|
||||
for name in names:
|
||||
# Skip directory entries
|
||||
if name.endswith('/'):
|
||||
continue
|
||||
if name.rsplit('/', 1)[-1] == filename:
|
||||
return name
|
||||
return None
|
||||
|
||||
sqlite_member = find_member("memories.json")
|
||||
if not sqlite_member:
|
||||
raise HTTPException(status_code=400, detail="memories.json missing in zip")
|
||||
|
||||
memories_member = find_member("memories.jsonl.gz")
|
||||
|
||||
sqlite_data = json.loads(zf.read(sqlite_member))
|
||||
memories_blob = zf.read(memories_member) if memories_member else None
|
||||
except Exception:
|
||||
raise HTTPException(status_code=400, detail="Invalid zip file")
|
||||
|
||||
default_app = db.query(App).filter(App.owner_id == user.id, App.name == "openmemory").first()
|
||||
if not default_app:
|
||||
default_app = App(owner_id=user.id, name="openmemory", is_active=True, metadata_={})
|
||||
db.add(default_app)
|
||||
db.commit()
|
||||
db.refresh(default_app)
|
||||
|
||||
cat_id_map: Dict[str, UUID] = {}
|
||||
for c in sqlite_data.get("categories", []):
|
||||
cat = db.query(Category).filter(Category.name == c["name"]).first()
|
||||
if not cat:
|
||||
cat = Category(name=c["name"], description=c.get("description"))
|
||||
db.add(cat)
|
||||
db.commit()
|
||||
db.refresh(cat)
|
||||
cat_id_map[c["id"]] = cat.id
|
||||
|
||||
old_to_new_id: Dict[str, UUID] = {}
|
||||
for m in sqlite_data.get("memories", []):
|
||||
incoming_id = UUID(m["id"])
|
||||
existing = db.query(Memory).filter(Memory.id == incoming_id).first()
|
||||
|
||||
# Cross-user collision: always mint a new UUID and import as a new memory
|
||||
if existing and existing.user_id != user.id:
|
||||
target_id = uuid4()
|
||||
else:
|
||||
target_id = incoming_id
|
||||
|
||||
old_to_new_id[m["id"]] = target_id
|
||||
|
||||
# Same-user collision + skip mode: leave existing row untouched
|
||||
if existing and (existing.user_id == user.id) and mode == "skip":
|
||||
continue
|
||||
|
||||
# Same-user collision + overwrite mode: treat import as ground truth
|
||||
if existing and (existing.user_id == user.id) and mode == "overwrite":
|
||||
incoming_state = m.get("state", "active")
|
||||
existing.user_id = user.id
|
||||
existing.app_id = default_app.id
|
||||
existing.content = m.get("content") or ""
|
||||
existing.metadata_ = m.get("metadata") or {}
|
||||
try:
|
||||
existing.state = MemoryState(incoming_state)
|
||||
except Exception:
|
||||
existing.state = MemoryState.active
|
||||
# Update state-related timestamps from import (ground truth)
|
||||
existing.archived_at = _parse_iso(m.get("archived_at"))
|
||||
existing.deleted_at = _parse_iso(m.get("deleted_at"))
|
||||
existing.created_at = _parse_iso(m.get("created_at")) or existing.created_at
|
||||
existing.updated_at = _parse_iso(m.get("updated_at")) or existing.updated_at
|
||||
db.add(existing)
|
||||
db.commit()
|
||||
continue
|
||||
|
||||
new_mem = Memory(
|
||||
id=target_id,
|
||||
user_id=user.id,
|
||||
app_id=default_app.id,
|
||||
content=m.get("content") or "",
|
||||
metadata_=m.get("metadata") or {},
|
||||
state=MemoryState(m.get("state", "active")) if m.get("state") else MemoryState.active,
|
||||
created_at=_parse_iso(m.get("created_at")) or datetime.now(UTC),
|
||||
updated_at=_parse_iso(m.get("updated_at")) or datetime.now(UTC),
|
||||
archived_at=_parse_iso(m.get("archived_at")),
|
||||
deleted_at=_parse_iso(m.get("deleted_at")),
|
||||
)
|
||||
db.add(new_mem)
|
||||
db.commit()
|
||||
|
||||
for link in sqlite_data.get("memory_categories", []):
|
||||
mid = old_to_new_id.get(link["memory_id"])
|
||||
cid = cat_id_map.get(link["category_id"])
|
||||
if not (mid and cid):
|
||||
continue
|
||||
exists = db.execute(
|
||||
memory_categories.select().where(
|
||||
(memory_categories.c.memory_id == mid) & (memory_categories.c.category_id == cid)
|
||||
)
|
||||
).first()
|
||||
|
||||
if not exists:
|
||||
db.execute(memory_categories.insert().values(memory_id=mid, category_id=cid))
|
||||
db.commit()
|
||||
|
||||
for h in sqlite_data.get("status_history", []):
|
||||
hid = UUID(h["id"])
|
||||
mem_id = old_to_new_id.get(h["memory_id"], UUID(h["memory_id"]))
|
||||
exists = db.query(MemoryStatusHistory).filter(MemoryStatusHistory.id == hid).first()
|
||||
if exists and mode == "skip":
|
||||
continue
|
||||
rec = exists if exists else MemoryStatusHistory(id=hid)
|
||||
rec.memory_id = mem_id
|
||||
rec.changed_by = user.id
|
||||
try:
|
||||
rec.old_state = MemoryState(h.get("old_state", "active"))
|
||||
rec.new_state = MemoryState(h.get("new_state", "active"))
|
||||
except Exception:
|
||||
rec.old_state = MemoryState.active
|
||||
rec.new_state = MemoryState.active
|
||||
rec.changed_at = _parse_iso(h.get("changed_at")) or datetime.now(UTC)
|
||||
db.add(rec)
|
||||
db.commit()
|
||||
|
||||
memory_client = get_memory_client()
|
||||
vector_store = getattr(memory_client, "vector_store", None) if memory_client else None
|
||||
|
||||
if vector_store and memory_client and hasattr(memory_client, "embedding_model"):
|
||||
def iter_logical_records():
|
||||
if memories_blob:
|
||||
gz_buf = io.BytesIO(memories_blob)
|
||||
with gzip.GzipFile(fileobj=gz_buf, mode="rb") as gz:
|
||||
for raw in gz:
|
||||
yield json.loads(raw.decode("utf-8"))
|
||||
else:
|
||||
for m in sqlite_data.get("memories", []):
|
||||
yield {
|
||||
"id": m["id"],
|
||||
"content": m.get("content"),
|
||||
"metadata": m.get("metadata") or {},
|
||||
"created_at": m.get("created_at"),
|
||||
"updated_at": m.get("updated_at"),
|
||||
}
|
||||
|
||||
for rec in iter_logical_records():
|
||||
old_id = rec["id"]
|
||||
new_id = old_to_new_id.get(old_id, UUID(old_id))
|
||||
content = rec.get("content") or ""
|
||||
metadata = rec.get("metadata") or {}
|
||||
created_at = rec.get("created_at")
|
||||
updated_at = rec.get("updated_at")
|
||||
|
||||
if mode == "skip":
|
||||
try:
|
||||
get_fn = getattr(vector_store, "get", None)
|
||||
if callable(get_fn) and vector_store.get(str(new_id)):
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
payload = dict(metadata)
|
||||
payload["data"] = content
|
||||
if created_at:
|
||||
payload["created_at"] = created_at
|
||||
if updated_at:
|
||||
payload["updated_at"] = updated_at
|
||||
payload["user_id"] = user_id
|
||||
payload.setdefault("source_app", "openmemory")
|
||||
|
||||
try:
|
||||
vec = memory_client.embedding_model.embed(content, "add")
|
||||
vector_store.insert(vectors=[vec], payloads=[payload], ids=[str(new_id)])
|
||||
except Exception as e:
|
||||
print(f"Vector upsert failed for memory {new_id}: {e}")
|
||||
continue
|
||||
|
||||
return {"message": f'Import completed into user "{user_id}"'}
|
||||
|
||||
return {"message": f'Import completed into user "{user_id}"'}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from app.database import get_db
|
||||
from app.models import Config as ConfigModel
|
||||
from app.utils.memory import reset_memory_client
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
router = APIRouter(prefix="/api/v1/config", tags=["config"])
|
||||
|
||||
class LLMConfig(BaseModel):
|
||||
model: str = Field(..., description="LLM model name")
|
||||
temperature: float = Field(..., description="Temperature setting for the model")
|
||||
max_tokens: int = Field(..., description="Maximum tokens to generate")
|
||||
api_key: Optional[str] = Field(None, description="API key or 'env:API_KEY' to use environment variable")
|
||||
ollama_base_url: Optional[str] = Field(None, description="Base URL for Ollama server (e.g., http://host.docker.internal:11434)")
|
||||
|
||||
class LLMProvider(BaseModel):
|
||||
provider: str = Field(..., description="LLM provider name")
|
||||
config: LLMConfig
|
||||
|
||||
class EmbedderConfig(BaseModel):
|
||||
model: str = Field(..., description="Embedder model name")
|
||||
api_key: Optional[str] = Field(None, description="API key or 'env:API_KEY' to use environment variable")
|
||||
ollama_base_url: Optional[str] = Field(None, description="Base URL for Ollama server (e.g., http://host.docker.internal:11434)")
|
||||
|
||||
class EmbedderProvider(BaseModel):
|
||||
provider: str = Field(..., description="Embedder provider name")
|
||||
config: EmbedderConfig
|
||||
|
||||
class VectorStoreProvider(BaseModel):
|
||||
provider: str = Field(..., description="Vector store provider name")
|
||||
# Below config can vary widely based on the vector store used. Refer https://docs.mem0.ai/components/vectordbs/config
|
||||
config: Dict[str, Any] = Field(..., description="Vector store-specific configuration")
|
||||
|
||||
class OpenMemoryConfig(BaseModel):
|
||||
custom_instructions: Optional[str] = Field(None, description="Custom instructions for memory management and fact extraction")
|
||||
|
||||
class Mem0Config(BaseModel):
|
||||
llm: Optional[LLMProvider] = None
|
||||
embedder: Optional[EmbedderProvider] = None
|
||||
vector_store: Optional[VectorStoreProvider] = None
|
||||
|
||||
class ConfigSchema(BaseModel):
|
||||
openmemory: Optional[OpenMemoryConfig] = None
|
||||
mem0: Optional[Mem0Config] = None
|
||||
|
||||
def get_default_configuration():
|
||||
"""Get the default configuration with sensible defaults for LLM and embedder."""
|
||||
return {
|
||||
"openmemory": {
|
||||
"custom_instructions": None
|
||||
},
|
||||
"mem0": {
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0.1,
|
||||
"max_tokens": 2000,
|
||||
"api_key": "env:OPENAI_API_KEY"
|
||||
}
|
||||
},
|
||||
"embedder": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "text-embedding-3-small",
|
||||
"api_key": "env:OPENAI_API_KEY"
|
||||
}
|
||||
},
|
||||
"vector_store": None
|
||||
}
|
||||
}
|
||||
|
||||
def get_config_from_db(db: Session, key: str = "main"):
|
||||
"""Get configuration from database."""
|
||||
config = db.query(ConfigModel).filter(ConfigModel.key == key).first()
|
||||
|
||||
if not config:
|
||||
# Create default config with proper provider configurations
|
||||
default_config = get_default_configuration()
|
||||
db_config = ConfigModel(key=key, value=default_config)
|
||||
db.add(db_config)
|
||||
db.commit()
|
||||
db.refresh(db_config)
|
||||
return default_config
|
||||
|
||||
# Ensure the config has all required sections with defaults
|
||||
config_value = config.value
|
||||
default_config = get_default_configuration()
|
||||
|
||||
# Merge with defaults to ensure all required fields exist
|
||||
if "openmemory" not in config_value:
|
||||
config_value["openmemory"] = default_config["openmemory"]
|
||||
|
||||
if "mem0" not in config_value:
|
||||
config_value["mem0"] = default_config["mem0"]
|
||||
else:
|
||||
# Ensure LLM config exists with defaults
|
||||
if "llm" not in config_value["mem0"] or config_value["mem0"]["llm"] is None:
|
||||
config_value["mem0"]["llm"] = default_config["mem0"]["llm"]
|
||||
|
||||
# Ensure embedder config exists with defaults
|
||||
if "embedder" not in config_value["mem0"] or config_value["mem0"]["embedder"] is None:
|
||||
config_value["mem0"]["embedder"] = default_config["mem0"]["embedder"]
|
||||
|
||||
# Ensure vector_store config exists with defaults
|
||||
if "vector_store" not in config_value["mem0"]:
|
||||
config_value["mem0"]["vector_store"] = default_config["mem0"]["vector_store"]
|
||||
|
||||
# Save the updated config back to database if it was modified
|
||||
if config_value != config.value:
|
||||
config.value = config_value
|
||||
db.commit()
|
||||
db.refresh(config)
|
||||
|
||||
return config_value
|
||||
|
||||
def save_config_to_db(db: Session, config: Dict[str, Any], key: str = "main"):
|
||||
"""Save configuration to database."""
|
||||
db_config = db.query(ConfigModel).filter(ConfigModel.key == key).first()
|
||||
|
||||
if db_config:
|
||||
db_config.value = config
|
||||
db_config.updated_at = None # Will trigger the onupdate to set current time
|
||||
else:
|
||||
db_config = ConfigModel(key=key, value=config)
|
||||
db.add(db_config)
|
||||
|
||||
db.commit()
|
||||
db.refresh(db_config)
|
||||
return db_config.value
|
||||
|
||||
@router.get("/", response_model=ConfigSchema)
|
||||
async def get_configuration(db: Session = Depends(get_db)):
|
||||
"""Get the current configuration."""
|
||||
config = get_config_from_db(db)
|
||||
return config
|
||||
|
||||
@router.put("/", response_model=ConfigSchema)
|
||||
async def update_configuration(config: ConfigSchema, db: Session = Depends(get_db)):
|
||||
"""Update the configuration."""
|
||||
current_config = get_config_from_db(db)
|
||||
|
||||
# Convert to dict for processing
|
||||
updated_config = current_config.copy()
|
||||
|
||||
# Update openmemory settings if provided
|
||||
if config.openmemory is not None:
|
||||
if "openmemory" not in updated_config:
|
||||
updated_config["openmemory"] = {}
|
||||
updated_config["openmemory"].update(config.openmemory.dict(exclude_none=True))
|
||||
|
||||
# Update mem0 settings
|
||||
updated_config["mem0"] = config.mem0.dict(exclude_none=True)
|
||||
|
||||
|
||||
@router.patch("/", response_model=ConfigSchema)
|
||||
async def patch_configuration(config_update: ConfigSchema, db: Session = Depends(get_db)):
|
||||
"""Update parts of the configuration."""
|
||||
current_config = get_config_from_db(db)
|
||||
|
||||
def deep_update(source, overrides):
|
||||
for key, value in overrides.items():
|
||||
if isinstance(value, dict) and key in source and isinstance(source[key], dict):
|
||||
source[key] = deep_update(source[key], value)
|
||||
else:
|
||||
source[key] = value
|
||||
return source
|
||||
|
||||
update_data = config_update.dict(exclude_unset=True)
|
||||
updated_config = deep_update(current_config, update_data)
|
||||
|
||||
save_config_to_db(db, updated_config)
|
||||
reset_memory_client()
|
||||
return updated_config
|
||||
|
||||
|
||||
@router.post("/reset", response_model=ConfigSchema)
|
||||
async def reset_configuration(db: Session = Depends(get_db)):
|
||||
"""Reset the configuration to default values."""
|
||||
try:
|
||||
# Get the default configuration with proper provider setups
|
||||
default_config = get_default_configuration()
|
||||
|
||||
# Save it as the current configuration in the database
|
||||
save_config_to_db(db, default_config)
|
||||
reset_memory_client()
|
||||
return default_config
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to reset configuration: {str(e)}"
|
||||
)
|
||||
|
||||
@router.get("/mem0/llm", response_model=LLMProvider)
|
||||
async def get_llm_configuration(db: Session = Depends(get_db)):
|
||||
"""Get only the LLM configuration."""
|
||||
config = get_config_from_db(db)
|
||||
llm_config = config.get("mem0", {}).get("llm", {})
|
||||
return llm_config
|
||||
|
||||
@router.put("/mem0/llm", response_model=LLMProvider)
|
||||
async def update_llm_configuration(llm_config: LLMProvider, db: Session = Depends(get_db)):
|
||||
"""Update only the LLM configuration."""
|
||||
current_config = get_config_from_db(db)
|
||||
|
||||
# Ensure mem0 key exists
|
||||
if "mem0" not in current_config:
|
||||
current_config["mem0"] = {}
|
||||
|
||||
# Update the LLM configuration
|
||||
current_config["mem0"]["llm"] = llm_config.dict(exclude_none=True)
|
||||
|
||||
# Save the configuration to database
|
||||
save_config_to_db(db, current_config)
|
||||
reset_memory_client()
|
||||
return current_config["mem0"]["llm"]
|
||||
|
||||
@router.get("/mem0/embedder", response_model=EmbedderProvider)
|
||||
async def get_embedder_configuration(db: Session = Depends(get_db)):
|
||||
"""Get only the Embedder configuration."""
|
||||
config = get_config_from_db(db)
|
||||
embedder_config = config.get("mem0", {}).get("embedder", {})
|
||||
return embedder_config
|
||||
|
||||
@router.put("/mem0/embedder", response_model=EmbedderProvider)
|
||||
async def update_embedder_configuration(embedder_config: EmbedderProvider, db: Session = Depends(get_db)):
|
||||
"""Update only the Embedder configuration."""
|
||||
current_config = get_config_from_db(db)
|
||||
|
||||
# Ensure mem0 key exists
|
||||
if "mem0" not in current_config:
|
||||
current_config["mem0"] = {}
|
||||
|
||||
# Update the Embedder configuration
|
||||
current_config["mem0"]["embedder"] = embedder_config.dict(exclude_none=True)
|
||||
|
||||
# Save the configuration to database
|
||||
save_config_to_db(db, current_config)
|
||||
reset_memory_client()
|
||||
return current_config["mem0"]["embedder"]
|
||||
|
||||
@router.get("/mem0/vector_store", response_model=Optional[VectorStoreProvider])
|
||||
async def get_vector_store_configuration(db: Session = Depends(get_db)):
|
||||
"""Get only the Vector Store configuration."""
|
||||
config = get_config_from_db(db)
|
||||
vector_store_config = config.get("mem0", {}).get("vector_store", None)
|
||||
return vector_store_config
|
||||
|
||||
@router.put("/mem0/vector_store", response_model=VectorStoreProvider)
|
||||
async def update_vector_store_configuration(vector_store_config: VectorStoreProvider, db: Session = Depends(get_db)):
|
||||
"""Update only the Vector Store configuration."""
|
||||
current_config = get_config_from_db(db)
|
||||
|
||||
# Ensure mem0 key exists
|
||||
if "mem0" not in current_config:
|
||||
current_config["mem0"] = {}
|
||||
|
||||
# Update the Vector Store configuration
|
||||
current_config["mem0"]["vector_store"] = vector_store_config.dict(exclude_none=True)
|
||||
|
||||
# Save the configuration to database
|
||||
save_config_to_db(db, current_config)
|
||||
reset_memory_client()
|
||||
return current_config["mem0"]["vector_store"]
|
||||
|
||||
@router.get("/openmemory", response_model=OpenMemoryConfig)
|
||||
async def get_openmemory_configuration(db: Session = Depends(get_db)):
|
||||
"""Get only the OpenMemory configuration."""
|
||||
config = get_config_from_db(db)
|
||||
openmemory_config = config.get("openmemory", {})
|
||||
return openmemory_config
|
||||
|
||||
@router.put("/openmemory", response_model=OpenMemoryConfig)
|
||||
async def update_openmemory_configuration(openmemory_config: OpenMemoryConfig, db: Session = Depends(get_db)):
|
||||
"""Update only the OpenMemory configuration."""
|
||||
current_config = get_config_from_db(db)
|
||||
|
||||
# Ensure openmemory key exists
|
||||
if "openmemory" not in current_config:
|
||||
current_config["openmemory"] = {}
|
||||
|
||||
# Update the OpenMemory configuration
|
||||
current_config["openmemory"].update(openmemory_config.dict(exclude_none=True))
|
||||
|
||||
# Save the configuration to database
|
||||
save_config_to_db(db, current_config)
|
||||
reset_memory_client()
|
||||
return current_config["openmemory"]
|
||||
@@ -0,0 +1,694 @@
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import List, Optional, Set
|
||||
from uuid import UUID
|
||||
|
||||
from app.database import get_db
|
||||
from app.models import (
|
||||
AccessControl,
|
||||
App,
|
||||
Category,
|
||||
Memory,
|
||||
MemoryAccessLog,
|
||||
MemoryState,
|
||||
MemoryStatusHistory,
|
||||
User,
|
||||
)
|
||||
from app.schemas import MemoryResponse
|
||||
from app.utils.memory import get_memory_client
|
||||
from app.utils.permissions import check_memory_access_permissions
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from fastapi_pagination import Page, Params
|
||||
from fastapi_pagination.ext.sqlalchemy import paginate as sqlalchemy_paginate
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
|
||||
router = APIRouter(prefix="/api/v1/memories", tags=["memories"])
|
||||
|
||||
|
||||
def get_memory_or_404(db: Session, memory_id: UUID) -> Memory:
|
||||
memory = db.query(Memory).filter(Memory.id == memory_id).first()
|
||||
if not memory:
|
||||
raise HTTPException(status_code=404, detail="Memory not found")
|
||||
return memory
|
||||
|
||||
|
||||
def update_memory_state(db: Session, memory_id: UUID, new_state: MemoryState, user_id: UUID):
|
||||
memory = get_memory_or_404(db, memory_id)
|
||||
old_state = memory.state
|
||||
|
||||
# Update memory state
|
||||
memory.state = new_state
|
||||
if new_state == MemoryState.archived:
|
||||
memory.archived_at = datetime.now(UTC)
|
||||
elif new_state == MemoryState.deleted:
|
||||
memory.deleted_at = datetime.now(UTC)
|
||||
|
||||
# Record state change
|
||||
history = MemoryStatusHistory(
|
||||
memory_id=memory_id,
|
||||
changed_by=user_id,
|
||||
old_state=old_state,
|
||||
new_state=new_state
|
||||
)
|
||||
db.add(history)
|
||||
db.commit()
|
||||
return memory
|
||||
|
||||
|
||||
def get_accessible_memory_ids(db: Session, app_id: UUID) -> Set[UUID]:
|
||||
"""
|
||||
Get the set of memory IDs that the app has access to based on app-level ACL rules.
|
||||
Returns all memory IDs if no specific restrictions are found.
|
||||
"""
|
||||
# Get app-level access controls
|
||||
app_access = db.query(AccessControl).filter(
|
||||
AccessControl.subject_type == "app",
|
||||
AccessControl.subject_id == app_id,
|
||||
AccessControl.object_type == "memory"
|
||||
).all()
|
||||
|
||||
# If no app-level rules exist, return None to indicate all memories are accessible
|
||||
if not app_access:
|
||||
return None
|
||||
|
||||
# Initialize sets for allowed and denied memory IDs
|
||||
allowed_memory_ids = set()
|
||||
denied_memory_ids = set()
|
||||
|
||||
# Process app-level rules
|
||||
for rule in app_access:
|
||||
if rule.effect == "allow":
|
||||
if rule.object_id: # Specific memory access
|
||||
allowed_memory_ids.add(rule.object_id)
|
||||
else: # All memories access
|
||||
return None # All memories allowed
|
||||
elif rule.effect == "deny":
|
||||
if rule.object_id: # Specific memory denied
|
||||
denied_memory_ids.add(rule.object_id)
|
||||
else: # All memories denied
|
||||
return set() # No memories accessible
|
||||
|
||||
# Remove denied memories from allowed set
|
||||
if allowed_memory_ids:
|
||||
allowed_memory_ids -= denied_memory_ids
|
||||
|
||||
return allowed_memory_ids
|
||||
|
||||
|
||||
# List all memories with filtering
|
||||
@router.get("/", response_model=Page[MemoryResponse])
|
||||
async def list_memories(
|
||||
user_id: str,
|
||||
app_id: Optional[UUID] = None,
|
||||
from_date: Optional[int] = Query(
|
||||
None,
|
||||
description="Filter memories created after this date (timestamp)",
|
||||
examples=[1718505600]
|
||||
),
|
||||
to_date: Optional[int] = Query(
|
||||
None,
|
||||
description="Filter memories created before this date (timestamp)",
|
||||
examples=[1718505600]
|
||||
),
|
||||
categories: Optional[str] = None,
|
||||
params: Params = Depends(),
|
||||
search_query: Optional[str] = None,
|
||||
sort_column: Optional[str] = Query(None, description="Column to sort by (memory, categories, app_name, created_at)"),
|
||||
sort_direction: Optional[str] = Query(None, description="Sort direction (asc or desc)"),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Build base query
|
||||
query = db.query(Memory).filter(
|
||||
Memory.user_id == user.id,
|
||||
Memory.state != MemoryState.deleted,
|
||||
Memory.state != MemoryState.archived,
|
||||
Memory.content.ilike(f"%{search_query}%") if search_query else True
|
||||
)
|
||||
|
||||
# Apply filters
|
||||
if app_id:
|
||||
query = query.filter(Memory.app_id == app_id)
|
||||
|
||||
if from_date:
|
||||
from_datetime = datetime.fromtimestamp(from_date, tz=UTC)
|
||||
query = query.filter(Memory.created_at >= from_datetime)
|
||||
|
||||
if to_date:
|
||||
to_datetime = datetime.fromtimestamp(to_date, tz=UTC)
|
||||
query = query.filter(Memory.created_at <= to_datetime)
|
||||
|
||||
# Add joins for app and categories after filtering
|
||||
query = query.outerjoin(App, Memory.app_id == App.id)
|
||||
query = query.outerjoin(Memory.categories)
|
||||
|
||||
# Apply category filter if provided
|
||||
if categories:
|
||||
category_list = [c.strip() for c in categories.split(",")]
|
||||
query = query.filter(Category.name.in_(category_list))
|
||||
|
||||
# Apply sorting if specified
|
||||
if sort_column:
|
||||
sort_field = getattr(Memory, sort_column, None)
|
||||
if sort_field:
|
||||
query = query.order_by(sort_field.desc()) if sort_direction == "desc" else query.order_by(sort_field.asc())
|
||||
|
||||
# Add eager loading for app and categories
|
||||
query = query.options(
|
||||
joinedload(Memory.app),
|
||||
joinedload(Memory.categories)
|
||||
).distinct(Memory.id)
|
||||
|
||||
# Get paginated results with transformer
|
||||
return sqlalchemy_paginate(
|
||||
query,
|
||||
params,
|
||||
transformer=lambda items: [
|
||||
MemoryResponse(
|
||||
id=memory.id,
|
||||
content=memory.content,
|
||||
created_at=memory.created_at,
|
||||
state=memory.state.value,
|
||||
app_id=memory.app_id,
|
||||
app_name=memory.app.name if memory.app else None,
|
||||
categories=[category.name for category in memory.categories],
|
||||
metadata_=memory.metadata_
|
||||
)
|
||||
for memory in items
|
||||
if check_memory_access_permissions(db, memory, app_id)
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
# Get all categories
|
||||
@router.get("/categories")
|
||||
async def get_categories(
|
||||
user_id: str,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Get unique categories associated with the user's memories
|
||||
# Get all memories
|
||||
memories = db.query(Memory).filter(Memory.user_id == user.id, Memory.state != MemoryState.deleted, Memory.state != MemoryState.archived).all()
|
||||
# Get all categories from memories
|
||||
categories = [category for memory in memories for category in memory.categories]
|
||||
# Get unique categories
|
||||
unique_categories = list(set(categories))
|
||||
|
||||
return {
|
||||
"categories": unique_categories,
|
||||
"total": len(unique_categories)
|
||||
}
|
||||
|
||||
|
||||
class CreateMemoryRequest(BaseModel):
|
||||
user_id: str
|
||||
text: str
|
||||
metadata: dict = {}
|
||||
infer: bool = True
|
||||
app: str = "openmemory"
|
||||
|
||||
|
||||
# Create new memory
|
||||
@router.post("/")
|
||||
async def create_memory(
|
||||
request: CreateMemoryRequest,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == request.user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
# Get or create app
|
||||
app_obj = db.query(App).filter(App.name == request.app,
|
||||
App.owner_id == user.id).first()
|
||||
if not app_obj:
|
||||
app_obj = App(name=request.app, owner_id=user.id)
|
||||
db.add(app_obj)
|
||||
db.commit()
|
||||
db.refresh(app_obj)
|
||||
|
||||
# Check if app is active
|
||||
if not app_obj.is_active:
|
||||
raise HTTPException(status_code=403, detail=f"App {request.app} is currently paused on OpenMemory. Cannot create new memories.")
|
||||
|
||||
# Log what we're about to do
|
||||
logging.info(f"Creating memory for user_id: {request.user_id} with app: {request.app}")
|
||||
|
||||
# Try to get memory client safely
|
||||
try:
|
||||
memory_client = get_memory_client()
|
||||
if not memory_client:
|
||||
raise Exception("Memory client is not available")
|
||||
except Exception as client_error:
|
||||
logging.warning(f"Memory client unavailable: {client_error}. Creating memory in database only.")
|
||||
# Return a json response with the error
|
||||
return {
|
||||
"error": str(client_error)
|
||||
}
|
||||
|
||||
# Try to save to Qdrant via memory_client
|
||||
try:
|
||||
qdrant_response = memory_client.add(
|
||||
request.text,
|
||||
user_id=request.user_id, # Use string user_id to match search
|
||||
metadata={
|
||||
"source_app": "openmemory",
|
||||
"mcp_client": request.app,
|
||||
},
|
||||
infer=request.infer
|
||||
)
|
||||
|
||||
# Log the response for debugging
|
||||
logging.info(f"Qdrant response: {qdrant_response}")
|
||||
|
||||
# Process Qdrant response
|
||||
if isinstance(qdrant_response, dict) and 'results' in qdrant_response:
|
||||
created_memories = []
|
||||
|
||||
for result in qdrant_response['results']:
|
||||
if result['event'] == 'ADD':
|
||||
# Get the Qdrant-generated ID
|
||||
memory_id = UUID(result['id'])
|
||||
|
||||
# Check if memory already exists
|
||||
existing_memory = db.query(Memory).filter(Memory.id == memory_id).first()
|
||||
|
||||
if existing_memory:
|
||||
# Update existing memory
|
||||
existing_memory.state = MemoryState.active
|
||||
existing_memory.content = result['memory']
|
||||
memory = existing_memory
|
||||
else:
|
||||
# Create memory with the EXACT SAME ID from Qdrant
|
||||
memory = Memory(
|
||||
id=memory_id, # Use the same ID that Qdrant generated
|
||||
user_id=user.id,
|
||||
app_id=app_obj.id,
|
||||
content=result['memory'],
|
||||
metadata_=request.metadata,
|
||||
state=MemoryState.active
|
||||
)
|
||||
db.add(memory)
|
||||
|
||||
# Create history entry
|
||||
history = MemoryStatusHistory(
|
||||
memory_id=memory_id,
|
||||
changed_by=user.id,
|
||||
old_state=MemoryState.deleted if existing_memory else MemoryState.deleted,
|
||||
new_state=MemoryState.active
|
||||
)
|
||||
db.add(history)
|
||||
|
||||
created_memories.append(memory)
|
||||
|
||||
# Commit all changes at once
|
||||
if created_memories:
|
||||
db.commit()
|
||||
for memory in created_memories:
|
||||
db.refresh(memory)
|
||||
|
||||
# Return the first memory (for API compatibility)
|
||||
# but all memories are now saved to the database
|
||||
return created_memories[0]
|
||||
except Exception as qdrant_error:
|
||||
logging.warning(f"Qdrant operation failed: {qdrant_error}.")
|
||||
# Return a json response with the error
|
||||
return {
|
||||
"error": str(qdrant_error)
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
# Get memory by ID
|
||||
@router.get("/{memory_id}")
|
||||
async def get_memory(
|
||||
memory_id: UUID,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
memory = get_memory_or_404(db, memory_id)
|
||||
return {
|
||||
"id": memory.id,
|
||||
"text": memory.content,
|
||||
"created_at": int(memory.created_at.timestamp()),
|
||||
"state": memory.state.value,
|
||||
"app_id": memory.app_id,
|
||||
"app_name": memory.app.name if memory.app else None,
|
||||
"categories": [category.name for category in memory.categories],
|
||||
"metadata_": memory.metadata_
|
||||
}
|
||||
|
||||
|
||||
class DeleteMemoriesRequest(BaseModel):
|
||||
memory_ids: List[UUID]
|
||||
user_id: str
|
||||
|
||||
# Delete multiple memories
|
||||
@router.delete("/")
|
||||
async def delete_memories(
|
||||
request: DeleteMemoriesRequest,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == request.user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Get memory client to delete from vector store
|
||||
try:
|
||||
memory_client = get_memory_client()
|
||||
if not memory_client:
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Memory client is not available"
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as client_error:
|
||||
logging.error(f"Memory client initialization failed: {client_error}")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=f"Memory service unavailable: {str(client_error)}"
|
||||
)
|
||||
|
||||
# Delete from vector store then mark as deleted in database
|
||||
for memory_id in request.memory_ids:
|
||||
try:
|
||||
memory_client.delete(str(memory_id))
|
||||
except Exception as delete_error:
|
||||
logging.warning(f"Failed to delete memory {memory_id} from vector store: {delete_error}")
|
||||
|
||||
update_memory_state(db, memory_id, MemoryState.deleted, user.id)
|
||||
|
||||
return {"message": f"Successfully deleted {len(request.memory_ids)} memories"}
|
||||
|
||||
|
||||
# Archive memories
|
||||
@router.post("/actions/archive")
|
||||
async def archive_memories(
|
||||
memory_ids: List[UUID],
|
||||
user_id: UUID,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
for memory_id in memory_ids:
|
||||
update_memory_state(db, memory_id, MemoryState.archived, user_id)
|
||||
return {"message": f"Successfully archived {len(memory_ids)} memories"}
|
||||
|
||||
|
||||
class PauseMemoriesRequest(BaseModel):
|
||||
memory_ids: Optional[List[UUID]] = None
|
||||
category_ids: Optional[List[UUID]] = None
|
||||
app_id: Optional[UUID] = None
|
||||
all_for_app: bool = False
|
||||
global_pause: bool = False
|
||||
state: Optional[MemoryState] = None
|
||||
user_id: str
|
||||
|
||||
# Pause access to memories
|
||||
@router.post("/actions/pause")
|
||||
async def pause_memories(
|
||||
request: PauseMemoriesRequest,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
|
||||
global_pause = request.global_pause
|
||||
all_for_app = request.all_for_app
|
||||
app_id = request.app_id
|
||||
memory_ids = request.memory_ids
|
||||
category_ids = request.category_ids
|
||||
state = request.state or MemoryState.paused
|
||||
|
||||
user = db.query(User).filter(User.user_id == request.user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
user_id = user.id
|
||||
|
||||
if global_pause:
|
||||
# Pause all memories
|
||||
memories = db.query(Memory).filter(
|
||||
Memory.state != MemoryState.deleted,
|
||||
Memory.state != MemoryState.archived
|
||||
).all()
|
||||
for memory in memories:
|
||||
update_memory_state(db, memory.id, state, user_id)
|
||||
return {"message": "Successfully paused all memories"}
|
||||
|
||||
if app_id:
|
||||
# Pause all memories for an app
|
||||
memories = db.query(Memory).filter(
|
||||
Memory.app_id == app_id,
|
||||
Memory.user_id == user.id,
|
||||
Memory.state != MemoryState.deleted,
|
||||
Memory.state != MemoryState.archived
|
||||
).all()
|
||||
for memory in memories:
|
||||
update_memory_state(db, memory.id, state, user_id)
|
||||
return {"message": f"Successfully paused all memories for app {app_id}"}
|
||||
|
||||
if all_for_app and memory_ids:
|
||||
# Pause all memories for an app
|
||||
memories = db.query(Memory).filter(
|
||||
Memory.user_id == user.id,
|
||||
Memory.state != MemoryState.deleted,
|
||||
Memory.id.in_(memory_ids)
|
||||
).all()
|
||||
for memory in memories:
|
||||
update_memory_state(db, memory.id, state, user_id)
|
||||
return {"message": "Successfully paused all memories"}
|
||||
|
||||
if memory_ids:
|
||||
# Pause specific memories
|
||||
for memory_id in memory_ids:
|
||||
update_memory_state(db, memory_id, state, user_id)
|
||||
return {"message": f"Successfully paused {len(memory_ids)} memories"}
|
||||
|
||||
if category_ids:
|
||||
# Pause memories by category
|
||||
memories = db.query(Memory).join(Memory.categories).filter(
|
||||
Category.id.in_(category_ids),
|
||||
Memory.state != MemoryState.deleted,
|
||||
Memory.state != MemoryState.archived
|
||||
).all()
|
||||
for memory in memories:
|
||||
update_memory_state(db, memory.id, state, user_id)
|
||||
return {"message": f"Successfully paused memories in {len(category_ids)} categories"}
|
||||
|
||||
raise HTTPException(status_code=400, detail="Invalid pause request parameters")
|
||||
|
||||
|
||||
# Get memory access logs
|
||||
@router.get("/{memory_id}/access-log")
|
||||
async def get_memory_access_log(
|
||||
memory_id: UUID,
|
||||
page: int = Query(1, ge=1),
|
||||
page_size: int = Query(10, ge=1, le=100),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
query = db.query(MemoryAccessLog).filter(MemoryAccessLog.memory_id == memory_id)
|
||||
total = query.count()
|
||||
logs = query.order_by(MemoryAccessLog.accessed_at.desc()).offset((page - 1) * page_size).limit(page_size).all()
|
||||
|
||||
# Get app name
|
||||
for log in logs:
|
||||
app = db.query(App).filter(App.id == log.app_id).first()
|
||||
log.app_name = app.name if app else None
|
||||
|
||||
return {
|
||||
"total": total,
|
||||
"page": page,
|
||||
"page_size": page_size,
|
||||
"logs": logs
|
||||
}
|
||||
|
||||
|
||||
class UpdateMemoryRequest(BaseModel):
|
||||
memory_content: str
|
||||
user_id: str
|
||||
|
||||
# Update a memory
|
||||
@router.put("/{memory_id}")
|
||||
async def update_memory(
|
||||
memory_id: UUID,
|
||||
request: UpdateMemoryRequest,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == request.user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
memory = get_memory_or_404(db, memory_id)
|
||||
memory.content = request.memory_content
|
||||
db.commit()
|
||||
db.refresh(memory)
|
||||
return memory
|
||||
|
||||
class FilterMemoriesRequest(BaseModel):
|
||||
user_id: str
|
||||
page: int = 1
|
||||
size: int = 10
|
||||
search_query: Optional[str] = None
|
||||
app_ids: Optional[List[UUID]] = None
|
||||
category_ids: Optional[List[UUID]] = None
|
||||
sort_column: Optional[str] = None
|
||||
sort_direction: Optional[str] = None
|
||||
from_date: Optional[int] = None
|
||||
to_date: Optional[int] = None
|
||||
show_archived: Optional[bool] = False
|
||||
|
||||
@router.post("/filter", response_model=Page[MemoryResponse])
|
||||
async def filter_memories(
|
||||
request: FilterMemoriesRequest,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == request.user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Build base query
|
||||
query = db.query(Memory).filter(
|
||||
Memory.user_id == user.id,
|
||||
Memory.state != MemoryState.deleted,
|
||||
)
|
||||
|
||||
# Filter archived memories based on show_archived parameter
|
||||
if not request.show_archived:
|
||||
query = query.filter(Memory.state != MemoryState.archived)
|
||||
|
||||
# Apply search filter
|
||||
if request.search_query:
|
||||
query = query.filter(Memory.content.ilike(f"%{request.search_query}%"))
|
||||
|
||||
# Apply app filter
|
||||
if request.app_ids:
|
||||
query = query.filter(Memory.app_id.in_(request.app_ids))
|
||||
|
||||
# Add joins for app and categories
|
||||
query = query.outerjoin(App, Memory.app_id == App.id)
|
||||
|
||||
# Apply category filter
|
||||
if request.category_ids:
|
||||
query = query.join(Memory.categories).filter(Category.id.in_(request.category_ids))
|
||||
else:
|
||||
query = query.outerjoin(Memory.categories)
|
||||
|
||||
# Apply date filters
|
||||
if request.from_date:
|
||||
from_datetime = datetime.fromtimestamp(request.from_date, tz=UTC)
|
||||
query = query.filter(Memory.created_at >= from_datetime)
|
||||
|
||||
if request.to_date:
|
||||
to_datetime = datetime.fromtimestamp(request.to_date, tz=UTC)
|
||||
query = query.filter(Memory.created_at <= to_datetime)
|
||||
|
||||
# Apply sorting
|
||||
if request.sort_column and request.sort_direction:
|
||||
sort_direction = request.sort_direction.lower()
|
||||
if sort_direction not in ['asc', 'desc']:
|
||||
raise HTTPException(status_code=400, detail="Invalid sort direction")
|
||||
|
||||
sort_mapping = {
|
||||
'memory': Memory.content,
|
||||
'app_name': App.name,
|
||||
'created_at': Memory.created_at
|
||||
}
|
||||
|
||||
if request.sort_column not in sort_mapping:
|
||||
raise HTTPException(status_code=400, detail="Invalid sort column")
|
||||
|
||||
sort_field = sort_mapping[request.sort_column]
|
||||
if sort_direction == 'desc':
|
||||
query = query.order_by(sort_field.desc())
|
||||
else:
|
||||
query = query.order_by(sort_field.asc())
|
||||
else:
|
||||
# Default sorting
|
||||
query = query.order_by(Memory.created_at.desc())
|
||||
|
||||
# Add eager loading for categories and make the query distinct
|
||||
query = query.options(
|
||||
joinedload(Memory.categories)
|
||||
).distinct(Memory.id)
|
||||
|
||||
# Use fastapi-pagination's paginate function
|
||||
return sqlalchemy_paginate(
|
||||
query,
|
||||
Params(page=request.page, size=request.size),
|
||||
transformer=lambda items: [
|
||||
MemoryResponse(
|
||||
id=memory.id,
|
||||
content=memory.content,
|
||||
created_at=memory.created_at,
|
||||
state=memory.state.value,
|
||||
app_id=memory.app_id,
|
||||
app_name=memory.app.name if memory.app else None,
|
||||
categories=[category.name for category in memory.categories],
|
||||
metadata_=memory.metadata_
|
||||
)
|
||||
for memory in items
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{memory_id}/related", response_model=Page[MemoryResponse])
|
||||
async def get_related_memories(
|
||||
memory_id: UUID,
|
||||
user_id: str,
|
||||
params: Params = Depends(),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
# Validate user
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Get the source memory
|
||||
memory = get_memory_or_404(db, memory_id)
|
||||
|
||||
# Extract category IDs from the source memory
|
||||
category_ids = [category.id for category in memory.categories]
|
||||
|
||||
if not category_ids:
|
||||
return Page.create([], total=0, params=params)
|
||||
|
||||
# Build query for related memories
|
||||
query = db.query(Memory).distinct(Memory.id).filter(
|
||||
Memory.user_id == user.id,
|
||||
Memory.id != memory_id,
|
||||
Memory.state != MemoryState.deleted
|
||||
).join(Memory.categories).filter(
|
||||
Category.id.in_(category_ids)
|
||||
).options(
|
||||
joinedload(Memory.categories),
|
||||
joinedload(Memory.app)
|
||||
).order_by(
|
||||
func.count(Category.id).desc(),
|
||||
Memory.created_at.desc()
|
||||
).group_by(Memory.id)
|
||||
|
||||
# ⚡ Force page size to be 5
|
||||
params = Params(page=params.page, size=5)
|
||||
|
||||
return sqlalchemy_paginate(
|
||||
query,
|
||||
params,
|
||||
transformer=lambda items: [
|
||||
MemoryResponse(
|
||||
id=memory.id,
|
||||
content=memory.content,
|
||||
created_at=memory.created_at,
|
||||
state=memory.state.value,
|
||||
app_id=memory.app_id,
|
||||
app_name=memory.app.name if memory.app else None,
|
||||
categories=[category.name for category in memory.categories],
|
||||
metadata_=memory.metadata_
|
||||
)
|
||||
for memory in items
|
||||
]
|
||||
)
|
||||
@@ -0,0 +1,29 @@
|
||||
from app.database import get_db
|
||||
from app.models import App, Memory, MemoryState, User
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
router = APIRouter(prefix="/api/v1/stats", tags=["stats"])
|
||||
|
||||
@router.get("/")
|
||||
async def get_profile(
|
||||
user_id: str,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Get total number of memories
|
||||
total_memories = db.query(Memory).filter(Memory.user_id == user.id, Memory.state != MemoryState.deleted).count()
|
||||
|
||||
# Get total number of apps
|
||||
apps = db.query(App).filter(App.owner == user)
|
||||
total_apps = apps.count()
|
||||
|
||||
return {
|
||||
"total_memories": total_memories,
|
||||
"total_apps": total_apps,
|
||||
"apps": apps.all()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
from datetime import datetime
|
||||
from typing import List, Optional
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, validator
|
||||
|
||||
|
||||
class MemoryBase(BaseModel):
|
||||
content: str
|
||||
metadata_: Optional[dict] = Field(default_factory=dict)
|
||||
|
||||
class MemoryCreate(MemoryBase):
|
||||
user_id: UUID
|
||||
app_id: UUID
|
||||
|
||||
|
||||
class Category(BaseModel):
|
||||
name: str
|
||||
|
||||
|
||||
class App(BaseModel):
|
||||
id: UUID
|
||||
name: str
|
||||
|
||||
|
||||
class Memory(MemoryBase):
|
||||
id: UUID
|
||||
user_id: UUID
|
||||
app_id: UUID
|
||||
created_at: datetime
|
||||
updated_at: Optional[datetime] = None
|
||||
state: str
|
||||
categories: Optional[List[Category]] = None
|
||||
app: App
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
class MemoryUpdate(BaseModel):
|
||||
content: Optional[str] = None
|
||||
metadata_: Optional[dict] = None
|
||||
state: Optional[str] = None
|
||||
|
||||
|
||||
class MemoryResponse(BaseModel):
|
||||
id: UUID
|
||||
content: str
|
||||
created_at: int
|
||||
state: str
|
||||
app_id: UUID
|
||||
app_name: str
|
||||
categories: List[str]
|
||||
metadata_: Optional[dict] = None
|
||||
|
||||
@validator('created_at', pre=True)
|
||||
def convert_to_epoch(cls, v):
|
||||
if isinstance(v, datetime):
|
||||
return int(v.timestamp())
|
||||
return v
|
||||
|
||||
class PaginatedMemoryResponse(BaseModel):
|
||||
items: List[MemoryResponse]
|
||||
total: int
|
||||
page: int
|
||||
size: int
|
||||
pages: int
|
||||
@@ -0,0 +1,43 @@
|
||||
import logging
|
||||
from typing import List
|
||||
|
||||
from app.utils.prompts import MEMORY_CATEGORIZATION_PROMPT
|
||||
from dotenv import load_dotenv
|
||||
from openai import OpenAI
|
||||
from pydantic import BaseModel
|
||||
from tenacity import retry, stop_after_attempt, wait_exponential
|
||||
|
||||
load_dotenv()
|
||||
openai_client = OpenAI()
|
||||
|
||||
|
||||
class MemoryCategories(BaseModel):
|
||||
categories: List[str]
|
||||
|
||||
|
||||
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=15))
|
||||
def get_categories_for_memory(memory: str) -> List[str]:
|
||||
try:
|
||||
messages = [
|
||||
{"role": "system", "content": MEMORY_CATEGORIZATION_PROMPT},
|
||||
{"role": "user", "content": memory}
|
||||
]
|
||||
|
||||
# Let OpenAI handle the pydantic parsing directly
|
||||
completion = openai_client.beta.chat.completions.parse(
|
||||
model="gpt-4o-mini",
|
||||
messages=messages,
|
||||
response_format=MemoryCategories,
|
||||
temperature=0
|
||||
)
|
||||
|
||||
parsed: MemoryCategories = completion.choices[0].message.parsed
|
||||
return [cat.strip().lower() for cat in parsed.categories]
|
||||
|
||||
except Exception as e:
|
||||
logging.error(f"[ERROR] Failed to get categories: {e}")
|
||||
try:
|
||||
logging.debug(f"[DEBUG] Raw response: {completion.choices[0].message.content}")
|
||||
except Exception as debug_e:
|
||||
logging.debug(f"[DEBUG] Could not extract raw response: {debug_e}")
|
||||
raise
|
||||
@@ -0,0 +1,33 @@
|
||||
from typing import Tuple
|
||||
|
||||
from app.models import App, User
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
|
||||
def get_or_create_user(db: Session, user_id: str) -> User:
|
||||
"""Get or create a user with the given user_id"""
|
||||
user = db.query(User).filter(User.user_id == user_id).first()
|
||||
if not user:
|
||||
user = User(user_id=user_id)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
return user
|
||||
|
||||
|
||||
def get_or_create_app(db: Session, user: User, app_id: str) -> App:
|
||||
"""Get or create an app for the given user"""
|
||||
app = db.query(App).filter(App.owner_id == user.id, App.name == app_id).first()
|
||||
if not app:
|
||||
app = App(owner_id=user.id, name=app_id)
|
||||
db.add(app)
|
||||
db.commit()
|
||||
db.refresh(app)
|
||||
return app
|
||||
|
||||
|
||||
def get_user_and_app(db: Session, user_id: str, app_id: str) -> Tuple[User, App]:
|
||||
"""Get or create both user and their app"""
|
||||
user = get_or_create_user(db, user_id)
|
||||
app = get_or_create_app(db, user, app_id)
|
||||
return user, app
|
||||
@@ -0,0 +1,504 @@
|
||||
"""
|
||||
Memory client utilities for OpenMemory.
|
||||
|
||||
This module provides functionality to initialize and manage the Mem0 memory client
|
||||
with automatic configuration management and Docker environment support.
|
||||
|
||||
Docker Ollama Configuration:
|
||||
When running inside a Docker container and using Ollama as the LLM or embedder provider,
|
||||
the system automatically detects the Docker environment and adjusts localhost URLs
|
||||
to properly reach the host machine where Ollama is running.
|
||||
|
||||
Supported Docker host resolution (in order of preference):
|
||||
1. OLLAMA_HOST environment variable (if set)
|
||||
2. host.docker.internal (Docker Desktop for Mac/Windows)
|
||||
3. Docker bridge gateway IP (typically 172.17.0.1 on Linux)
|
||||
4. Fallback to 172.17.0.1
|
||||
|
||||
Example configuration that will be automatically adjusted:
|
||||
{
|
||||
"llm": {
|
||||
"provider": "ollama",
|
||||
"config": {
|
||||
"model": "llama3.1:latest",
|
||||
"ollama_base_url": "http://localhost:11434" # Auto-adjusted in Docker
|
||||
}
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
|
||||
from app.database import SessionLocal
|
||||
from app.models import Config as ConfigModel
|
||||
|
||||
from mem0 import Memory
|
||||
|
||||
_memory_client = None
|
||||
_config_hash = None
|
||||
|
||||
|
||||
def _get_config_hash(config_dict):
|
||||
"""Generate a hash of the config to detect changes."""
|
||||
config_str = json.dumps(config_dict, sort_keys=True)
|
||||
return hashlib.md5(config_str.encode()).hexdigest()
|
||||
|
||||
|
||||
def _get_docker_host_url():
|
||||
"""
|
||||
Determine the appropriate host URL to reach host machine from inside Docker container.
|
||||
Returns the best available option for reaching the host from inside a container.
|
||||
"""
|
||||
# Check for custom environment variable first
|
||||
custom_host = os.environ.get('OLLAMA_HOST')
|
||||
if custom_host:
|
||||
print(f"Using custom Ollama host from OLLAMA_HOST: {custom_host}")
|
||||
return custom_host.replace('http://', '').replace('https://', '').split(':')[0]
|
||||
|
||||
# Check if we're running inside Docker
|
||||
if not os.path.exists('/.dockerenv'):
|
||||
# Not in Docker, return localhost as-is
|
||||
return "localhost"
|
||||
|
||||
print("Detected Docker environment, adjusting host URL for Ollama...")
|
||||
|
||||
# Try different host resolution strategies
|
||||
host_candidates = []
|
||||
|
||||
# 1. host.docker.internal (works on Docker Desktop for Mac/Windows)
|
||||
try:
|
||||
socket.gethostbyname('host.docker.internal')
|
||||
host_candidates.append('host.docker.internal')
|
||||
print("Found host.docker.internal")
|
||||
except socket.gaierror:
|
||||
pass
|
||||
|
||||
# 2. Docker bridge gateway (typically 172.17.0.1 on Linux)
|
||||
try:
|
||||
with open('/proc/net/route', 'r') as f:
|
||||
for line in f:
|
||||
fields = line.strip().split()
|
||||
if fields[1] == '00000000': # Default route
|
||||
gateway_hex = fields[2]
|
||||
gateway_ip = socket.inet_ntoa(bytes.fromhex(gateway_hex)[::-1])
|
||||
host_candidates.append(gateway_ip)
|
||||
print(f"Found Docker gateway: {gateway_ip}")
|
||||
break
|
||||
except (FileNotFoundError, IndexError, ValueError):
|
||||
pass
|
||||
|
||||
# 3. Fallback to common Docker bridge IP
|
||||
if not host_candidates:
|
||||
host_candidates.append('172.17.0.1')
|
||||
print("Using fallback Docker bridge IP: 172.17.0.1")
|
||||
|
||||
# Return the first available candidate
|
||||
return host_candidates[0]
|
||||
|
||||
|
||||
def _fix_ollama_urls(config_section):
|
||||
"""
|
||||
Fix Ollama URLs for Docker environment.
|
||||
Replaces localhost URLs with appropriate Docker host URLs.
|
||||
Sets default ollama_base_url if not provided.
|
||||
"""
|
||||
if not config_section or "config" not in config_section:
|
||||
return config_section
|
||||
|
||||
ollama_config = config_section["config"]
|
||||
|
||||
# Set default ollama_base_url if not provided
|
||||
if "ollama_base_url" not in ollama_config:
|
||||
ollama_config["ollama_base_url"] = "http://host.docker.internal:11434"
|
||||
else:
|
||||
# Check for ollama_base_url and fix if it's localhost
|
||||
url = ollama_config["ollama_base_url"]
|
||||
if "localhost" in url or "127.0.0.1" in url:
|
||||
docker_host = _get_docker_host_url()
|
||||
if docker_host != "localhost":
|
||||
new_url = url.replace("localhost", docker_host).replace("127.0.0.1", docker_host)
|
||||
ollama_config["ollama_base_url"] = new_url
|
||||
print(f"Adjusted Ollama URL from {url} to {new_url}")
|
||||
|
||||
return config_section
|
||||
|
||||
|
||||
def reset_memory_client():
|
||||
"""Reset the global memory client to force reinitialization with new config."""
|
||||
global _memory_client, _config_hash
|
||||
_memory_client = None
|
||||
_config_hash = None
|
||||
|
||||
|
||||
# --- LLM provider config factories ---
|
||||
|
||||
def _build_ollama_llm_config(model, api_key, base_url, ollama_base_url):
|
||||
config = {"model": model or "llama3.1:latest"}
|
||||
# OLLAMA_BASE_URL takes precedence, then LLM_BASE_URL, then default
|
||||
config["ollama_base_url"] = ollama_base_url or base_url or "http://localhost:11434"
|
||||
return config
|
||||
|
||||
|
||||
def _build_openai_llm_config(model, api_key, base_url, ollama_base_url):
|
||||
config = {
|
||||
"model": model or "gpt-4o-mini",
|
||||
"api_key": api_key or "env:OPENAI_API_KEY",
|
||||
}
|
||||
if base_url:
|
||||
config["openai_base_url"] = base_url
|
||||
return config
|
||||
|
||||
|
||||
_LLM_CONFIG_FACTORIES = {
|
||||
"ollama": _build_ollama_llm_config,
|
||||
"openai": _build_openai_llm_config,
|
||||
}
|
||||
|
||||
|
||||
def _create_llm_config(provider, model, api_key, base_url, ollama_base_url):
|
||||
"""Build LLM config using registered provider factory or generic fallback."""
|
||||
base_config = {
|
||||
"temperature": 0.1,
|
||||
"max_tokens": 2000,
|
||||
}
|
||||
|
||||
factory = _LLM_CONFIG_FACTORIES.get(provider)
|
||||
if factory:
|
||||
base_config.update(factory(model, api_key, base_url, ollama_base_url))
|
||||
else:
|
||||
# Generic provider (anthropic, groq, together, deepseek, etc.)
|
||||
if not model:
|
||||
raise ValueError(
|
||||
f"LLM_MODEL environment variable is required when using LLM_PROVIDER='{provider}'. "
|
||||
f"Set LLM_MODEL to a valid model name for the '{provider}' provider."
|
||||
)
|
||||
base_config["model"] = model
|
||||
if api_key:
|
||||
base_config["api_key"] = api_key
|
||||
|
||||
return base_config
|
||||
|
||||
|
||||
# --- Embedder provider config factories ---
|
||||
|
||||
def _build_ollama_embedder_config(model, api_key, base_url, ollama_base_url, llm_base_url):
|
||||
config = {"model": model or "nomic-embed-text"}
|
||||
config["ollama_base_url"] = base_url or ollama_base_url or llm_base_url or "http://localhost:11434"
|
||||
return config
|
||||
|
||||
|
||||
def _build_openai_embedder_config(model, api_key, base_url, ollama_base_url, llm_base_url):
|
||||
config = {
|
||||
"model": model or "text-embedding-3-small",
|
||||
"api_key": api_key or "env:OPENAI_API_KEY",
|
||||
}
|
||||
if base_url:
|
||||
config["openai_base_url"] = base_url
|
||||
return config
|
||||
|
||||
|
||||
_EMBEDDER_CONFIG_FACTORIES = {
|
||||
"ollama": _build_ollama_embedder_config,
|
||||
"openai": _build_openai_embedder_config,
|
||||
}
|
||||
|
||||
|
||||
def _create_embedder_config(provider, model, api_key, base_url, ollama_base_url, llm_base_url):
|
||||
"""Build embedder config using registered provider factory or generic fallback."""
|
||||
factory = _EMBEDDER_CONFIG_FACTORIES.get(provider)
|
||||
if factory:
|
||||
config = factory(model, api_key, base_url, ollama_base_url, llm_base_url)
|
||||
else:
|
||||
if not model:
|
||||
raise ValueError(
|
||||
f"EMBEDDER_MODEL environment variable is required when using EMBEDDER_PROVIDER='{provider}'. "
|
||||
f"Set EMBEDDER_MODEL to a valid model name for the '{provider}' provider."
|
||||
)
|
||||
config = {"model": model}
|
||||
if api_key:
|
||||
config["api_key"] = api_key
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def get_default_memory_config():
|
||||
"""Get default memory client configuration with sensible defaults."""
|
||||
# Detect vector store based on environment variables
|
||||
vector_store_config = {
|
||||
"collection_name": "openmemory",
|
||||
"host": "mem0_store",
|
||||
}
|
||||
|
||||
# Check for different vector store configurations based on environment variables
|
||||
if os.environ.get('CHROMA_HOST') and os.environ.get('CHROMA_PORT'):
|
||||
vector_store_provider = "chroma"
|
||||
vector_store_config.update({
|
||||
"host": os.environ.get('CHROMA_HOST'),
|
||||
"port": int(os.environ.get('CHROMA_PORT'))
|
||||
})
|
||||
elif os.environ.get('QDRANT_HOST') and os.environ.get('QDRANT_PORT'):
|
||||
vector_store_provider = "qdrant"
|
||||
vector_store_config.update({
|
||||
"host": os.environ.get('QDRANT_HOST'),
|
||||
"port": int(os.environ.get('QDRANT_PORT'))
|
||||
})
|
||||
elif os.environ.get('WEAVIATE_CLUSTER_URL') or (os.environ.get('WEAVIATE_HOST') and os.environ.get('WEAVIATE_PORT')):
|
||||
vector_store_provider = "weaviate"
|
||||
# Prefer an explicit cluster URL if provided; otherwise build from host/port
|
||||
cluster_url = os.environ.get('WEAVIATE_CLUSTER_URL')
|
||||
if not cluster_url:
|
||||
weaviate_host = os.environ.get('WEAVIATE_HOST')
|
||||
weaviate_port = int(os.environ.get('WEAVIATE_PORT'))
|
||||
cluster_url = f"http://{weaviate_host}:{weaviate_port}"
|
||||
vector_store_config = {
|
||||
"collection_name": "openmemory",
|
||||
"cluster_url": cluster_url
|
||||
}
|
||||
elif os.environ.get('REDIS_URL'):
|
||||
vector_store_provider = "redis"
|
||||
vector_store_config = {
|
||||
"collection_name": "openmemory",
|
||||
"redis_url": os.environ.get('REDIS_URL')
|
||||
}
|
||||
elif os.environ.get('PG_HOST') and os.environ.get('PG_PORT'):
|
||||
vector_store_provider = "pgvector"
|
||||
vector_store_config.update({
|
||||
"host": os.environ.get('PG_HOST'),
|
||||
"port": int(os.environ.get('PG_PORT')),
|
||||
"dbname": os.environ.get('PG_DB', 'mem0'),
|
||||
"user": os.environ.get('PG_USER', 'mem0'),
|
||||
"password": os.environ.get('PG_PASSWORD', 'mem0')
|
||||
})
|
||||
elif os.environ.get('MILVUS_HOST') and os.environ.get('MILVUS_PORT'):
|
||||
vector_store_provider = "milvus"
|
||||
# Construct the full URL as expected by MilvusDBConfig
|
||||
milvus_host = os.environ.get('MILVUS_HOST')
|
||||
milvus_port = int(os.environ.get('MILVUS_PORT'))
|
||||
milvus_url = f"http://{milvus_host}:{milvus_port}"
|
||||
|
||||
vector_store_config = {
|
||||
"collection_name": "openmemory",
|
||||
"url": milvus_url,
|
||||
"token": os.environ.get('MILVUS_TOKEN', ''), # Always include, empty string for local setup
|
||||
"db_name": os.environ.get('MILVUS_DB_NAME', ''),
|
||||
"embedding_model_dims": 1536,
|
||||
"metric_type": "COSINE" # Using COSINE for better semantic similarity
|
||||
}
|
||||
elif os.environ.get('ELASTICSEARCH_HOST') and os.environ.get('ELASTICSEARCH_PORT'):
|
||||
vector_store_provider = "elasticsearch"
|
||||
# Construct the full URL with scheme since Elasticsearch client expects it
|
||||
elasticsearch_host = os.environ.get('ELASTICSEARCH_HOST')
|
||||
elasticsearch_port = int(os.environ.get('ELASTICSEARCH_PORT'))
|
||||
# Use http:// scheme since we're not using SSL
|
||||
full_host = f"http://{elasticsearch_host}"
|
||||
|
||||
vector_store_config.update({
|
||||
"host": full_host,
|
||||
"port": elasticsearch_port,
|
||||
"user": os.environ.get('ELASTICSEARCH_USER', 'elastic'),
|
||||
"password": os.environ.get('ELASTICSEARCH_PASSWORD', 'changeme'),
|
||||
"verify_certs": False,
|
||||
"use_ssl": False,
|
||||
"embedding_model_dims": 1536
|
||||
})
|
||||
elif os.environ.get('OPENSEARCH_HOST') and os.environ.get('OPENSEARCH_PORT'):
|
||||
vector_store_provider = "opensearch"
|
||||
vector_store_config.update({
|
||||
"host": os.environ.get('OPENSEARCH_HOST'),
|
||||
"port": int(os.environ.get('OPENSEARCH_PORT'))
|
||||
})
|
||||
elif os.environ.get('FAISS_PATH'):
|
||||
vector_store_provider = "faiss"
|
||||
vector_store_config = {
|
||||
"collection_name": "openmemory",
|
||||
"path": os.environ.get('FAISS_PATH'),
|
||||
"embedding_model_dims": 1536,
|
||||
"distance_strategy": "cosine"
|
||||
}
|
||||
else:
|
||||
# Default fallback to Qdrant
|
||||
vector_store_provider = "qdrant"
|
||||
vector_store_config.update({
|
||||
"port": 6333,
|
||||
})
|
||||
|
||||
print(f"Auto-detected vector store: {vector_store_provider} with config: {vector_store_config}")
|
||||
|
||||
# Detect LLM provider from environment variables
|
||||
llm_provider = os.environ.get('LLM_PROVIDER', 'openai').lower()
|
||||
llm_model = os.environ.get('LLM_MODEL')
|
||||
llm_api_key = os.environ.get('LLM_API_KEY')
|
||||
llm_base_url = os.environ.get('LLM_BASE_URL')
|
||||
ollama_base_url = os.environ.get('OLLAMA_BASE_URL')
|
||||
|
||||
llm_config = _create_llm_config(
|
||||
provider=llm_provider,
|
||||
model=llm_model,
|
||||
api_key=llm_api_key,
|
||||
base_url=llm_base_url,
|
||||
ollama_base_url=ollama_base_url,
|
||||
)
|
||||
print(f"Auto-detected LLM provider: {llm_provider}")
|
||||
|
||||
# Detect embedder provider from environment variables
|
||||
embedder_provider = os.environ.get('EMBEDDER_PROVIDER', llm_provider if llm_provider == 'ollama' else 'openai').lower()
|
||||
embedder_model = os.environ.get('EMBEDDER_MODEL')
|
||||
embedder_api_key = os.environ.get('EMBEDDER_API_KEY')
|
||||
embedder_base_url = os.environ.get('EMBEDDER_BASE_URL')
|
||||
|
||||
embedder_config = _create_embedder_config(
|
||||
provider=embedder_provider,
|
||||
model=embedder_model,
|
||||
api_key=embedder_api_key,
|
||||
base_url=embedder_base_url,
|
||||
ollama_base_url=ollama_base_url,
|
||||
llm_base_url=llm_base_url,
|
||||
)
|
||||
print(f"Auto-detected embedder provider: {embedder_provider}")
|
||||
|
||||
return {
|
||||
"vector_store": {
|
||||
"provider": vector_store_provider,
|
||||
"config": vector_store_config
|
||||
},
|
||||
"llm": {
|
||||
"provider": llm_provider,
|
||||
"config": llm_config
|
||||
},
|
||||
"embedder": {
|
||||
"provider": embedder_provider,
|
||||
"config": embedder_config
|
||||
},
|
||||
"version": "v1.1"
|
||||
}
|
||||
|
||||
|
||||
def _parse_environment_variables(config_dict):
|
||||
"""
|
||||
Parse environment variables in config values.
|
||||
Converts 'env:VARIABLE_NAME' to actual environment variable values.
|
||||
"""
|
||||
if isinstance(config_dict, dict):
|
||||
parsed_config = {}
|
||||
for key, value in config_dict.items():
|
||||
if isinstance(value, str) and value.startswith("env:"):
|
||||
env_var = value.split(":", 1)[1]
|
||||
env_value = os.environ.get(env_var)
|
||||
if env_value:
|
||||
parsed_config[key] = env_value
|
||||
print(f"Loaded {env_var} from environment for {key}")
|
||||
else:
|
||||
print(f"Warning: Environment variable {env_var} not found, keeping original value")
|
||||
parsed_config[key] = value
|
||||
elif isinstance(value, dict):
|
||||
parsed_config[key] = _parse_environment_variables(value)
|
||||
else:
|
||||
parsed_config[key] = value
|
||||
return parsed_config
|
||||
return config_dict
|
||||
|
||||
|
||||
def get_memory_client(custom_instructions: str = None):
|
||||
"""
|
||||
Get or initialize the Mem0 client.
|
||||
|
||||
Args:
|
||||
custom_instructions: Optional instructions for the memory project.
|
||||
|
||||
Returns:
|
||||
Initialized Mem0 client instance or None if initialization fails.
|
||||
|
||||
Raises:
|
||||
Exception: If required API keys are not set or critical configuration is missing.
|
||||
"""
|
||||
global _memory_client, _config_hash
|
||||
|
||||
try:
|
||||
# Start with default configuration
|
||||
config = get_default_memory_config()
|
||||
|
||||
# Variable to track custom instructions
|
||||
db_custom_instructions = None
|
||||
|
||||
# Load configuration from database
|
||||
try:
|
||||
db = SessionLocal()
|
||||
db_config = db.query(ConfigModel).filter(ConfigModel.key == "main").first()
|
||||
|
||||
if db_config:
|
||||
json_config = db_config.value
|
||||
|
||||
# Extract custom instructions from openmemory settings
|
||||
if "openmemory" in json_config and "custom_instructions" in json_config["openmemory"]:
|
||||
db_custom_instructions = json_config["openmemory"]["custom_instructions"]
|
||||
|
||||
# Override defaults with configurations from the database
|
||||
if "mem0" in json_config:
|
||||
mem0_config = json_config["mem0"]
|
||||
|
||||
# Update LLM configuration if available
|
||||
if "llm" in mem0_config and mem0_config["llm"] is not None:
|
||||
config["llm"] = mem0_config["llm"]
|
||||
|
||||
# Update Embedder configuration if available
|
||||
if "embedder" in mem0_config and mem0_config["embedder"] is not None:
|
||||
config["embedder"] = mem0_config["embedder"]
|
||||
|
||||
if "vector_store" in mem0_config and mem0_config["vector_store"] is not None:
|
||||
config["vector_store"] = mem0_config["vector_store"]
|
||||
else:
|
||||
print("No configuration found in database, using defaults")
|
||||
|
||||
db.close()
|
||||
|
||||
except Exception as e:
|
||||
print(f"Warning: Error loading configuration from database: {e}")
|
||||
print("Using default configuration")
|
||||
# Continue with default configuration if database config can't be loaded
|
||||
|
||||
# Use custom_instructions parameter first, then fall back to database value
|
||||
instructions_to_use = custom_instructions or db_custom_instructions
|
||||
if instructions_to_use:
|
||||
config["custom_fact_extraction_prompt"] = instructions_to_use
|
||||
|
||||
# Fix Ollama URLs for Docker environment (applies to both env-var defaults and DB overrides)
|
||||
if config.get("llm", {}).get("provider") == "ollama":
|
||||
config["llm"] = _fix_ollama_urls(config["llm"])
|
||||
if config.get("embedder", {}).get("provider") == "ollama":
|
||||
config["embedder"] = _fix_ollama_urls(config["embedder"])
|
||||
|
||||
# ALWAYS parse environment variables in the final config
|
||||
# This ensures that even default config values like "env:OPENAI_API_KEY" get parsed
|
||||
print("Parsing environment variables in final config...")
|
||||
config = _parse_environment_variables(config)
|
||||
|
||||
# Check if config has changed by comparing hashes
|
||||
current_config_hash = _get_config_hash(config)
|
||||
|
||||
# Only reinitialize if config changed or client doesn't exist
|
||||
if _memory_client is None or _config_hash != current_config_hash:
|
||||
print(f"Initializing memory client with config hash: {current_config_hash}")
|
||||
try:
|
||||
_memory_client = Memory.from_config(config_dict=config)
|
||||
_config_hash = current_config_hash
|
||||
print("Memory client initialized successfully")
|
||||
except Exception as init_error:
|
||||
print(f"Warning: Failed to initialize memory client: {init_error}")
|
||||
print("Server will continue running with limited memory functionality")
|
||||
_memory_client = None
|
||||
_config_hash = None
|
||||
return None
|
||||
|
||||
return _memory_client
|
||||
|
||||
except Exception as e:
|
||||
print(f"Warning: Exception occurred while initializing memory client: {e}")
|
||||
print("Server will continue running with limited memory functionality")
|
||||
return None
|
||||
|
||||
|
||||
def get_default_user_id():
|
||||
return "default_user"
|
||||
@@ -0,0 +1,53 @@
|
||||
from typing import Optional
|
||||
from uuid import UUID
|
||||
|
||||
from app.models import App, Memory, MemoryState
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
|
||||
def check_memory_access_permissions(
|
||||
db: Session,
|
||||
memory: Memory,
|
||||
app_id: Optional[UUID] = None
|
||||
) -> bool:
|
||||
"""
|
||||
Check if the given app has permission to access a memory based on:
|
||||
1. Memory state (must be active)
|
||||
2. App state (must not be paused)
|
||||
3. App-specific access controls
|
||||
|
||||
Args:
|
||||
db: Database session
|
||||
memory: Memory object to check access for
|
||||
app_id: Optional app ID to check permissions for
|
||||
|
||||
Returns:
|
||||
bool: True if access is allowed, False otherwise
|
||||
"""
|
||||
# Check if memory is active
|
||||
if memory.state != MemoryState.active:
|
||||
return False
|
||||
|
||||
# If no app_id provided, only check memory state
|
||||
if not app_id:
|
||||
return True
|
||||
|
||||
# Check if app exists and is active
|
||||
app = db.query(App).filter(App.id == app_id).first()
|
||||
if not app:
|
||||
return False
|
||||
|
||||
# Check if app is paused/inactive
|
||||
if not app.is_active:
|
||||
return False
|
||||
|
||||
# Check app-specific access controls
|
||||
from app.routers.memories import get_accessible_memory_ids
|
||||
accessible_memory_ids = get_accessible_memory_ids(db, app_id)
|
||||
|
||||
# If accessible_memory_ids is None, all memories are accessible
|
||||
if accessible_memory_ids is None:
|
||||
return True
|
||||
|
||||
# Check if memory is in the accessible set
|
||||
return memory.id in accessible_memory_ids
|
||||
@@ -0,0 +1,28 @@
|
||||
MEMORY_CATEGORIZATION_PROMPT = """Your task is to assign each piece of information (or “memory”) to one or more of the following categories. Feel free to use multiple categories per item when appropriate.
|
||||
|
||||
- Personal: family, friends, home, hobbies, lifestyle
|
||||
- Relationships: social network, significant others, colleagues
|
||||
- Preferences: likes, dislikes, habits, favorite media
|
||||
- Health: physical fitness, mental health, diet, sleep
|
||||
- Travel: trips, commutes, favorite places, itineraries
|
||||
- Work: job roles, companies, projects, promotions
|
||||
- Education: courses, degrees, certifications, skills development
|
||||
- Projects: to‑dos, milestones, deadlines, status updates
|
||||
- AI, ML & Technology: infrastructure, algorithms, tools, research
|
||||
- Technical Support: bug reports, error logs, fixes
|
||||
- Finance: income, expenses, investments, billing
|
||||
- Shopping: purchases, wishlists, returns, deliveries
|
||||
- Legal: contracts, policies, regulations, privacy
|
||||
- Entertainment: movies, music, games, books, events
|
||||
- Messages: emails, SMS, alerts, reminders
|
||||
- Customer Support: tickets, inquiries, resolutions
|
||||
- Product Feedback: ratings, bug reports, feature requests
|
||||
- News: articles, headlines, trending topics
|
||||
- Organization: meetings, appointments, calendars
|
||||
- Goals: ambitions, KPIs, long‑term objectives
|
||||
|
||||
Guidelines:
|
||||
- Return only the categories under 'categories' key in the JSON format.
|
||||
- If you cannot categorize the memory, return an empty list with key 'categories'.
|
||||
- Don't limit yourself to the categories listed above only. Feel free to create new categories based on the memory. Make sure that it is a single phrase.
|
||||
"""
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"mem0": {
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0.1,
|
||||
"max_tokens": 2000,
|
||||
"api_key": "env:API_KEY"
|
||||
}
|
||||
},
|
||||
"embedder": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "text-embedding-3-small",
|
||||
"api_key": "env:API_KEY"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"mem0": {
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0.1,
|
||||
"max_tokens": 2000,
|
||||
"api_key": "env:OPENAI_API_KEY"
|
||||
}
|
||||
},
|
||||
"embedder": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "text-embedding-3-small",
|
||||
"api_key": "env:OPENAI_API_KEY"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import datetime
|
||||
from uuid import uuid4
|
||||
|
||||
from app.config import DEFAULT_APP_ID, USER_ID
|
||||
from app.database import Base, SessionLocal, engine
|
||||
from app.mcp_server import setup_mcp_server
|
||||
from app.models import App, User
|
||||
from app.routers import apps_router, backup_router, config_router, memories_router, stats_router
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi_pagination import add_pagination
|
||||
|
||||
app = FastAPI(title="OpenMemory API")
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Create all tables
|
||||
Base.metadata.create_all(bind=engine)
|
||||
|
||||
# Check for USER_ID and create default user if needed
|
||||
def create_default_user():
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Check if user exists
|
||||
user = db.query(User).filter(User.user_id == USER_ID).first()
|
||||
if not user:
|
||||
# Create default user
|
||||
user = User(
|
||||
id=uuid4(),
|
||||
user_id=USER_ID,
|
||||
name="Default User",
|
||||
created_at=datetime.datetime.now(datetime.UTC)
|
||||
)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def create_default_app():
|
||||
db = SessionLocal()
|
||||
try:
|
||||
user = db.query(User).filter(User.user_id == USER_ID).first()
|
||||
if not user:
|
||||
return
|
||||
|
||||
# Check if app already exists
|
||||
existing_app = db.query(App).filter(
|
||||
App.name == DEFAULT_APP_ID,
|
||||
App.owner_id == user.id
|
||||
).first()
|
||||
|
||||
if existing_app:
|
||||
return
|
||||
|
||||
app = App(
|
||||
id=uuid4(),
|
||||
name=DEFAULT_APP_ID,
|
||||
owner_id=user.id,
|
||||
created_at=datetime.datetime.now(datetime.UTC),
|
||||
updated_at=datetime.datetime.now(datetime.UTC),
|
||||
)
|
||||
db.add(app)
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
# Create default user on startup
|
||||
create_default_user()
|
||||
create_default_app()
|
||||
|
||||
# Setup MCP server
|
||||
setup_mcp_server(app)
|
||||
|
||||
# Include routers
|
||||
app.include_router(memories_router)
|
||||
app.include_router(apps_router)
|
||||
app.include_router(stats_router)
|
||||
app.include_router(config_router)
|
||||
app.include_router(backup_router)
|
||||
|
||||
# Add pagination support
|
||||
add_pagination(app)
|
||||
@@ -0,0 +1,20 @@
|
||||
fastapi>=0.68.0
|
||||
uvicorn>=0.15.0
|
||||
sqlalchemy>=1.4.0
|
||||
python-dotenv>=1.2.2
|
||||
alembic>=1.7.0
|
||||
psycopg2-binary>=2.9.0
|
||||
python-multipart>=0.0.27
|
||||
urllib3>=2.7.0
|
||||
fastapi-pagination>=0.12.0
|
||||
mem0ai>=0.1.92
|
||||
openai>=1.40.0
|
||||
mcp[cli]>=1.25.4
|
||||
starlette>=0.40.0
|
||||
pytest>=7.0.0
|
||||
pytest-asyncio>=0.21.0
|
||||
httpx>=0.24.0
|
||||
pytest-cov>=4.0.0
|
||||
tenacity==9.1.2
|
||||
anthropic==0.51.0
|
||||
ollama==0.4.8
|
||||
@@ -0,0 +1,396 @@
|
||||
"""Tests for the MCP server endpoints (SSE and Streamable HTTP transports).
|
||||
|
||||
Covers the Streamable HTTP transport (MCP spec 2025-03-26+) and the legacy SSE
|
||||
transport. Tests exercise the full JSON-RPC flow — initialize, tools/list,
|
||||
tools/call — as well as error handling and context-variable isolation.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
# Set dummy keys before any imports that trigger client initialization
|
||||
os.environ.setdefault("OPENAI_API_KEY", "test-key")
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
from app.mcp_server import client_name_var, mcp, mcp_router, user_id_var
|
||||
|
||||
# MCP Streamable HTTP requires the Accept header to include application/json.
|
||||
# Including text/event-stream as well satisfies GET (SSE) requests.
|
||||
MCP_HEADERS = {"Accept": "application/json, text/event-stream"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture
|
||||
def test_app():
|
||||
"""Create a minimal FastAPI app with just the MCP router for testing."""
|
||||
from fastapi import FastAPI
|
||||
|
||||
app = FastAPI()
|
||||
app.include_router(mcp_router)
|
||||
return app
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def client(test_app):
|
||||
"""Async HTTP client wired to the test app via ASGI transport."""
|
||||
transport = ASGITransport(app=test_app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as ac:
|
||||
yield ac
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _jsonrpc(method: str, params: dict | None = None, req_id: int = 1) -> dict:
|
||||
"""Build a JSON-RPC 2.0 request envelope."""
|
||||
return {
|
||||
"jsonrpc": "2.0",
|
||||
"id": req_id,
|
||||
"method": method,
|
||||
"params": params or {},
|
||||
}
|
||||
|
||||
|
||||
def _initialize_payload(req_id: int = 1) -> dict:
|
||||
return _jsonrpc(
|
||||
"initialize",
|
||||
{
|
||||
"protocolVersion": "2025-03-26",
|
||||
"capabilities": {},
|
||||
"clientInfo": {"name": "test-client", "version": "0.1.0"},
|
||||
},
|
||||
req_id=req_id,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Streamable HTTP — route existence & basic protocol
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestStreamableHTTPBasic:
|
||||
"""Verify the Streamable HTTP route is registered and responds."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_post_initialize(self, client):
|
||||
"""POST initialize should return a valid JSON-RPC result."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["jsonrpc"] == "2.0"
|
||||
assert data["id"] == 1
|
||||
assert "result" in data
|
||||
result = data["result"]
|
||||
assert "serverInfo" in result
|
||||
assert "capabilities" in result
|
||||
assert result["protocolVersion"] == "2025-03-26"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_returns_method_not_allowed(self, client):
|
||||
"""DELETE in stateless mode should return 405 (no session to terminate)."""
|
||||
resp = await client.delete(
|
||||
"/mcp/testclient/http/user1",
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 405
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_missing_accept_header_returns_406(self, client):
|
||||
"""POST without the required Accept header should return 406."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
)
|
||||
assert resp.status_code == 406
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_invalid_json_returns_400(self, client):
|
||||
"""POST with unparseable body should return 400."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
content=b"not json",
|
||||
headers={**MCP_HEADERS, "Content-Type": "application/json"},
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_route_not_found_for_wrong_path(self, client):
|
||||
"""Requests to a non-existent path should 404."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/nonexistent/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Streamable HTTP — full protocol flow
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestStreamableHTTPProtocol:
|
||||
"""End-to-end JSON-RPC flows over Streamable HTTP."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tools_list(self, client):
|
||||
"""tools/list should return all registered MCP tools."""
|
||||
init_resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert init_resp.status_code == 200
|
||||
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_jsonrpc("tools/list", req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert "result" in data
|
||||
tool_names = {t["name"] for t in data["result"]["tools"]}
|
||||
expected = {"add_memories", "search_memory", "list_memories",
|
||||
"delete_memories", "delete_all_memories"}
|
||||
assert expected.issubset(tool_names), f"Missing tools: {expected - tool_names}"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tools_list_has_descriptions(self, client):
|
||||
"""Every tool returned by tools/list should have a non-empty description."""
|
||||
await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_jsonrpc("tools/list", req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
for tool in resp.json()["result"]["tools"]:
|
||||
assert tool.get("description"), f"Tool {tool['name']} has no description"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tools_list_has_input_schemas(self, client):
|
||||
"""Every tool should declare an inputSchema."""
|
||||
await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_jsonrpc("tools/list", req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
for tool in resp.json()["result"]["tools"]:
|
||||
assert "inputSchema" in tool, f"Tool {tool['name']} missing inputSchema"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_call_unknown_tool_returns_error(self, client):
|
||||
"""Calling a non-existent tool should return a JSON-RPC error."""
|
||||
await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_jsonrpc("tools/call", {"name": "no_such_tool", "arguments": {}}, req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert "error" in data or (
|
||||
"result" in data and data["result"].get("isError")
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unknown_jsonrpc_method(self, client):
|
||||
"""An unknown JSON-RPC method should return an error."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_jsonrpc("nonexistent/method"),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code in (200, 400)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_response_content_type_is_json(self, client):
|
||||
"""Responses should have Content-Type: application/json."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
ct = resp.headers.get("content-type", "")
|
||||
assert "application/json" in ct
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Streamable HTTP — context variable handling
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestStreamableHTTPContext:
|
||||
"""Verify that user_id and client_name context variables are set correctly."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_context_vars_set_during_tool_call(self, client):
|
||||
"""Context vars should reflect the path parameters during tool execution."""
|
||||
captured = {}
|
||||
|
||||
@mcp.tool(name="__test_ctx", description="test only")
|
||||
async def _capture(query: str = "") -> str:
|
||||
captured["user_id"] = user_id_var.get(None)
|
||||
captured["client_name"] = client_name_var.get(None)
|
||||
return "ok"
|
||||
|
||||
try:
|
||||
await client.post(
|
||||
"/mcp/my-app/http/alice",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
resp = await client.post(
|
||||
"/mcp/my-app/http/alice",
|
||||
json=_jsonrpc("tools/call", {"name": "__test_ctx", "arguments": {}}, req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert captured.get("user_id") == "alice"
|
||||
assert captured.get("client_name") == "my-app"
|
||||
finally:
|
||||
mcp._tool_manager._tools.pop("__test_ctx", None)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_different_users_are_isolated(self, client):
|
||||
"""Sequential requests with different user_ids must not leak state."""
|
||||
results = []
|
||||
|
||||
@mcp.tool(name="__test_uid_iso", description="test only")
|
||||
async def _capture_uid(query: str = "") -> str:
|
||||
results.append(user_id_var.get(None))
|
||||
return "ok"
|
||||
|
||||
try:
|
||||
for uid in ("userA", "userB", "userC"):
|
||||
await client.post(
|
||||
f"/mcp/app1/http/{uid}",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
await client.post(
|
||||
f"/mcp/app1/http/{uid}",
|
||||
json=_jsonrpc("tools/call", {"name": "__test_uid_iso", "arguments": {}}, req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
|
||||
assert results == ["userA", "userB", "userC"]
|
||||
finally:
|
||||
mcp._tool_manager._tools.pop("__test_uid_iso", None)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_different_clients_are_isolated(self, client):
|
||||
"""Sequential requests with different client_names must not leak state."""
|
||||
results = []
|
||||
|
||||
@mcp.tool(name="__test_cn_iso", description="test only")
|
||||
async def _capture_cn(query: str = "") -> str:
|
||||
results.append(client_name_var.get(None))
|
||||
return "ok"
|
||||
|
||||
try:
|
||||
for cn in ("cursor", "windsurf", "claude"):
|
||||
await client.post(
|
||||
f"/mcp/{cn}/http/user1",
|
||||
json=_initialize_payload(),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
await client.post(
|
||||
f"/mcp/{cn}/http/user1",
|
||||
json=_jsonrpc("tools/call", {"name": "__test_cn_iso", "arguments": {}}, req_id=2),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
|
||||
assert results == ["cursor", "windsurf", "claude"]
|
||||
finally:
|
||||
mcp._tool_manager._tools.pop("__test_cn_iso", None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Streamable HTTP — response correctness
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestStreamableHTTPResponses:
|
||||
"""Verify that captured responses are returned correctly to the caller."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_error_status_codes_are_preserved(self, client):
|
||||
"""Transport error codes (e.g. 406) must be forwarded, not masked as 200."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(),
|
||||
)
|
||||
assert resp.status_code == 406
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_status_code_preserved(self, client):
|
||||
"""DELETE 405 from stateless transport must not be masked."""
|
||||
resp = await client.delete(
|
||||
"/mcp/testclient/http/user1",
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 405
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_sequential_requests(self, client):
|
||||
"""Multiple requests in sequence should each get independent responses."""
|
||||
for i in range(5):
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
json=_initialize_payload(req_id=i + 1),
|
||||
headers=MCP_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["id"] == i + 1
|
||||
assert "result" in data
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wrong_content_type_returns_error(self, client):
|
||||
"""POST with wrong Content-Type should return an error status."""
|
||||
resp = await client.post(
|
||||
"/mcp/testclient/http/user1",
|
||||
content=b'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}',
|
||||
headers={**MCP_HEADERS, "Content-Type": "text/plain"},
|
||||
)
|
||||
assert resp.status_code in (400, 415)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Route registration
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestRouteRegistration:
|
||||
"""Verify all expected routes are registered in the router."""
|
||||
|
||||
def test_sse_route_is_registered(self, test_app):
|
||||
routes = [r.path for r in test_app.routes if hasattr(r, "path")]
|
||||
assert "/mcp/{client_name}/sse/{user_id}" in routes
|
||||
|
||||
def test_sse_post_messages_route_is_registered(self, test_app):
|
||||
routes = [r.path for r in test_app.routes if hasattr(r, "path")]
|
||||
assert "/mcp/messages/" in routes or "/mcp/{client_name}/sse/{user_id}/messages/" in routes
|
||||
|
||||
def test_streamable_http_route_is_registered(self, test_app):
|
||||
routes = [r.path for r in test_app.routes if hasattr(r, "path")]
|
||||
assert "/mcp/{client_name}/http/{user_id}" in routes
|
||||
Reference in New Issue
Block a user