chore: import zh skill memory-systems

This commit is contained in:
wehub-skill-sync
2026-07-13 21:36:25 +08:00
commit 89f0a1771f
4 changed files with 1409 additions and 0 deletions
+9
View File
@@ -0,0 +1,9 @@
# WeHub 来源说明
- Skill 名称:`memory-systems`
- 中文类目:Agent 长期/语义记忆架构
- 上游仓库:`muratcankoylan__Agent-Skills-for-Context-Engineering`
- 上游路径:`skills/memory-systems/SKILL.md`
- 上游链接:https://github.com/muratcankoylan/Agent-Skills-for-Context-Engineering/blob/HEAD/skills/memory-systems/SKILL.md
- 本仓库为 WeHub 中文 Skill 汉化包,基于 skill 市场筛选 Top200 清单整理
- 原作者、版权和许可证信息以上游仓库为准
+234
View File
@@ -0,0 +1,234 @@
---
name: memory-systems
description: >
该技能应用于智能体系统中的持久化语义记忆:
跨会话知识保留、实体追踪、时间有效性、
图或向量检索、记忆整合、以及记忆基准测试的选择。
文件型临时存储请交给 filesystem-context,交接摘要请交给 context-compression
令牌效率策略请交给 context-optimization。
---
# 记忆系统设计
记忆提供了持久化层,使智能体能够跨会话保持连续性,并基于累积的知识进行推理。简单的智能体完全依赖上下文作为记忆,会话结束时所有状态都会丢失。复杂的智能体则实现分层记忆架构,在即时上下文需求与长期知识保留之间取得平衡。从向量存储到知识图谱再到时序知识图谱的演进,代表着在结构化记忆方面投入不断增加,以改善检索和推理能力。
## 何时激活
在以下情况激活该技能:
- 构建必须跨会话持久化知识的智能体时
- 在记忆框架(Mem0、Zep/Graphiti、Letta、LangMem、Cognee)之间做选择时
- 需要在跨对话中保持实体一致性时
- 实现基于累积知识的推理时
- 设计可在生产环境中扩展的记忆架构时
- 根据基准测试(LoCoMo、LongMemEval、DMR)评估记忆系统时
- 构建具有自动实体/关系提取和自我改进记忆的动态记忆系统(Cognee)时
对于归属于其他技能的相邻工作,请勿激活该技能:
- 文件型临时存储、运行日志和工具输出卸载:`filesystem-context`
- 对话压缩或人类可读的交接摘要:`context-compression`
- 掩码、前缀缓存、令牌预算或单次轨迹内的检索范围控制:`context-optimization`
- 基于 RDF 状态的正式信念/愿望/意图模型:`bdi-mental-states`
## 核心概念
将记忆视为从易失的上下文窗口到持久存储的一个光谱。默认选择能满足检索需求的最简单层级,因为基准测试证据表明,对于某些记忆工作负载,工具复杂度的重要性低于可靠的检索(claim-memory-locomo-filesystem-baseline)。只有在检索质量下降,或者智能体需要多跳推理、关系遍历或时间旅行查询时,才增加结构(图、时间有效性)。
## 详细主题
### 生产框架全景
根据智能体所需的主要检索模式来选择框架。使用下表缩小候选范围,然后通过下面的基准测试数据进行验证。
| 框架 | 架构 | 最适合 | 权衡 |
|-----------|-------------|----------|-----------|
| **Mem0** | 向量存储 + 图记忆,可插拔后端 | 多租户系统,广泛的集成 | 在多智能体场景下不够专业 |
| **Zep/Graphiti** | 时序知识图谱,双时态模型 | 需要关系建模 + 时序推理的企业级场景 | 高级功能锁定在云端 |
| **Letta** | 自编辑记忆,分层存储(上下文内/核心/归档) | 完整的智能体内省,有状态服务 | 对简单用例过于复杂 |
| **Cognee** | 通过可定制的 ECL 管道与可定制任务构建的多层语义图 | 不断演化的智能体记忆,能够自适应和学习;多跳推理 | 摄入时处理更重 |
| **LangMem** | 用于 LangGraph 工作流的记忆工具 | 已在 LangGraph 上的团队 | 与 LangGraph 紧密耦合 |
| **文件系统** | 带命名约定的纯文本文件 | 简单智能体、原型开发 | 无语义搜索,无关系 |
当智能体需要双时态建模(同时追踪事件发生时间与数据摄入时间)时,选择 Zep/Graphiti,因为其三层级知识图谱(事件、语义实体、社区子图)在时序查询方面表现出色。当优先考虑通过托管基础设施快速上线时,选择 Mem0。当智能体需要通过其智能体开发环境进行深度自我内省时,选择 Letta。当智能体必须构建密集的多层语义图时,选择 Cognee——它将文本块和实体类型作为节点,并带有详细的关系边,每个核心组件(摄入、实体提取、后处理、检索)都是可定制的。
**基准测试性能对比**
参考以下基准测试来设定预期,但应将其视为检索维度上的特定来源信号,而非绝对排名。没有单一基准测试是决定性的。
| 系统 | DMR 准确率 | LoCoMo | HotPotQA(多跳) | 延迟 |
|--------|-------------|--------|---------------------|---------|
| Cognee | — | — | 公布的分数较高 | 视情况而定 |
| Zep(时序知识图谱) | 公布的分数较高 | — | 各项指标中等 | 报告为低延迟 |
| Letta(文件系统) | — | 公布的文件系统基线 | — | — |
| Mem0 | — | 公布的专业工具基线 | 在某项对比中较低 | — |
| MemGPT | 公布的分数较高 | — | — | 视情况而定 |
| GraphRAG | 公布的中/高范围 | — | — | 视情况而定 |
| 向量 RAG 基线 | 公布的较低范围 | — | — | 快速 |
关键结论:根据检索形态来比较记忆系统,而不是看品牌。将基准测试数据视为过时的证据,在用于产品宣传之前必须重新核实;稳定的设计原则是:从浅层开始,衡量检索质量,然后仅在更简单的层级无法满足需求时才添加语义或图结构。
### 记忆层级(决策要点)
选择能满足持久化需求的最浅记忆层级。每增加一个层级都会带来基础设施成本和运维复杂度,因此只有在浅层无法满足检索或持久化需求时才进行升级。
| 层级 | 持久性 | 实现方式 | 何时使用 |
|-------|------------|----------------|-------------|
| **工作记忆** | 仅上下文窗口 | 系统提示中的临时记录 | 始终使用——通过注意力优先位置进行优化 |
| **短期记忆** | 会话范围 | 文件系统、内存缓存 | 中间工具结果、对话状态 |
| **长期记忆** | 跨会话 | 键值存储 → 图数据库 | 用户偏好、领域知识、实体注册表 |
| **实体记忆** | 跨会话 | 实体注册表 + 属性 | 维护身份一致性("张三"在跨对话中为同一个人) |
| **时序知识图谱** | 跨会话 + 历史记录 | 带有效时间区间的图 | 随时间变化的事实、时间旅行查询、防止上下文冲突 |
### 检索策略
根据查询形态匹配检索策略。语义搜索擅长处理直接事实查询,但在多跳推理上表现不佳;基于实体的遍历擅长处理"关于 X 的一切"查询,但需要图结构;时序过滤擅长处理变化中的事实,但需要有效性元数据。当准确率至关重要且基础设施预算允许时,将多种策略结合为混合检索。
| 策略 | 何时使用 | 局限性 |
|----------|----------|------------|
| **语义**(嵌入相似度) | 直接事实查询 | 多跳推理时表现不佳 |
| **基于实体**(图遍历) | "告诉我关于 X 的一切" | 需要图结构 |
| **时序**(有效性过滤) | 事实随时间变化 | 需要有效性元数据 |
| **混合**(语义 + 关键词 + 图) | 总体准确率最佳 | 基础设施需求最大 |
混合方法通过仅检索相关的子图或记忆来减少活跃上下文。Cognee 通过在图、向量和关系存储之间提供多种搜索模式来实现混合检索,使智能体能够根据查询类型选择合适的检索策略,而不是采用一刀切的方法。
### 记忆整合
定期执行记忆整合以防止无限制增长,因为不受控制的记忆累积会随着时间推移降低检索质量。**标记为无效而非丢弃**——保留历史记录对于需要重建过去状态的时序查询至关重要。在记忆数量达到阈值、检索质量下降或按预设时间间隔触发整合。参见[实现参考](./references/implementation.md)获取可工作的整合代码。
## 实践指导
### 选择记忆架构
**从最简单的可行层级开始,仅当检索质量下降时才增加复杂度。** 大多数智能体在第一天并不需要时序知识图谱。遵循以下升级路径:
1. **原型阶段**:使用文件系统记忆。将事实存储为带时间戳的结构化 JSON。这能在投入基础设施之前验证智能体行为。
2. **扩展阶段**:当智能体需要语义搜索和多租户隔离时,迁移到 Mem0 或向量存储(带元数据),因为基于文件的查找无法处理相似度查询。
3. **复杂推理**:当智能体需要关系遍历、时间有效性或跨会话综合时,添加 Zep/Graphiti。Graphiti 使用带有通用关系的结构化连接,保持图简单且易于推理;Cognee 构建更密集的多层语义图,带有详细的关系边——根据智能体是需要时序双模型(Graphiti)还是更丰富的互联知识结构(Cognee)来选择。
4. **完全控制**:当智能体必须通过深度内省来自我管理其记忆时,使用 Letta 或 Cognee,因为这些框架将记忆操作暴露为智能体的一等动作。
### 与上下文的集成
采用即时加载记忆的方式,而非预先加载全部内容,因为大的上下文负载成本高昂且会降低注意力质量。将检索到的记忆放置在注意力优先的位置(上下文开头或结尾),以最大化其对生成结果的影响力。
### 错误恢复
优雅地处理检索失败,因为记忆系统天生具有噪声。按顺序应用以下恢复策略:
- **检索为空**:回退到更广泛的搜索(移除实体过滤条件,扩大时间范围)。如果仍然为空,提示用户澄清。
- **结果过时**:检查 `valid_until` 时间戳。如果大多数结果已过期,在重试之前触发整合。
- **事实冲突**:优先选择 `valid_from` 最近的事实。如果置信度较低,向用户展示冲突。
- **存储失败**:将写入操作排队等待重试。绝不能让记忆写入阻塞智能体的响应。
## 示例
**示例 1Mem0 集成**
```python
from mem0 import Memory
m = Memory()
m.add("用户偏好深色模式和 Python 3.12", user_id="alice")
m.add("用户切换到浅色模式", user_id="alice")
# 检索当前偏好(浅色模式),而非过时的
results = m.search("用户偏好什么主题?", user_id="alice")
```
**示例 2:时序查询**
```python
# 使用有效时间区间追踪实体
graph.create_temporal_relationship(
source_id=user_node,
rel_type="LIVES_AT",
target_id=address_node,
valid_from=datetime(2024, 1, 15),
valid_until=datetime(2024, 9, 1), # 已搬走
)
# 查询:用户在 2024 年 3 月 1 日住在哪里?
results = graph.query_at_time(
{"type": "LIVES_AT", "source_label": "User"},
query_time=datetime(2024, 3, 1)
)
```
**示例 3:Cognee 记忆摄入与搜索**
```python
import cognee
from cognee.modules.search.types import SearchType
# 摄入并构建知识图谱
await cognee.add("./docs/")
await cognee.add("任何数据")
await cognee.cognify()
# 丰富记忆
await cognee.memify()
# 智能体检索具有关系感知能力的上下文
results = await cognee.search(
query_text="对你的记忆进行任意查询",
query_type=SearchType.GRAPH_COMPLETION,
)
```
## 指导原则
1. 从文件系统记忆开始;只有在检索质量要求时才增加复杂度
2. 对任何可能随时间变化的事实追踪时间有效性
3. 使用混合检索(语义 + 关键词 + 图)以获得最佳准确率
4. 定期整合记忆——标记为无效但不丢弃
5. 为检索失败做好设计:当记忆查找未返回结果时,始终准备一个回退方案
6. 考虑持久化记忆的隐私影响(保留策略、删除权限)
7. 在每次变更前后,使用 LoCoMo 或 LongMemEval 对你的记忆系统进行基准测试
8. 在生产环境中监控记忆增长和检索延迟
## 注意事项
1. **将所有内容塞入上下文**:将可用的记忆全部加载到提示中成本高昂且会降低注意力质量。使用即时检索并配合相关性过滤。
2. **忽略时间有效性**:事实会过时。如果没有时间有效性追踪,过时的信息会污染上下文,智能体会基于错误的假设行事。
3. **过早过度设计**:在部分基准测试中,简单的文件系统记忆可能优于更专业的工具(claim-memory-locomo-filesystem-baseline)。只有在简单方法明显失败时才增加复杂度。
4. **没有整合策略**:不受控制的记忆增长会随时间推移降低检索质量。设置记忆数量阈值或定时计划来触发整合。
5. **嵌入模型不匹配**:用一个嵌入模型写入记忆,用另一个模型读取记忆,会导致检索效果不佳,因为向量空间不可互换。为每个记忆存储固定一个嵌入模型,如果模型变更,则重新嵌入所有条目。
6. **图谱模式僵化**:过度结构化的图谱模式(固定的节点类型、固定的关系标签)在领域演化时会失效。优先使用通用关系类型和灵活的属性包,以便新实体类型无需模式迁移。
7. **过时记忆中毒**:与当前状态相矛盾的旧记忆会静默地破坏智能体行为。实施过期策略或置信度衰减机制,使智能体降低对过时事实的优先级排序,并在检测到矛盾时显式呈现。
8. **记忆-上下文不匹配**:检索到的记忆在主题上相关但在上下文上错误(例如,当智能体讨论 Python 编程语言时,检索到了关于"蟒蛇"的记忆)。通过在记忆条目中包含会话或领域元数据,并在检索时进行过滤来缓解此问题。
## 集成
该技能负责持久化语义记忆。相邻技能负责临时存储、压缩和上下文策略:
- `filesystem-context`:在需要语义检索之前的文件型临时记录、日志和简单运行状态。
- `context-compression`:以散文形式保存会话状态的摘要和交接记录。
- `context-optimization`:在活跃上下文预算内进行即时记忆加载和检索范围控制。
- `context-degradation`:将过时或冲突的记忆视为上下文中毒或冲突。
- `bdi-mental-states`:当信念、愿望、意图和来源链很重要时的正式心理状态建模。
- `multi-agent-patterns`:跨智能体的共享记忆。
- `evaluation`:记忆质量、检索正确性和基准测试选择。
## 参考资料
内部参考:
- [实现参考](./references/implementation.md) - 阅读时机:从头实现向量存储、属性图、时序查询或记忆整合逻辑时
本集合中的相关技能:
- context-fundamentals - 阅读时机:设计记忆所馈入的上下文层时
- multi-agent-patterns - 阅读时机:多个智能体需要共享或协调记忆状态时
外部资源:
- Zep 时序知识图谱论文(arXiv:2501.13956 - 阅读时机:评估双时态建模或 Graphiti 架构时
- Mem0 生产架构论文(arXiv:2504.19413) - 阅读时机:评估托管记忆基础设施的权衡时
- Cognee 优化知识图谱 + 大语言模型推理论文(arXiv:2505.24478) - 阅读时机:比较多层语义图方法时
- LoCoMo 基准测试(Snap Research) - 阅读时机:评估长对话记忆保持能力时
- MemBench 评估框架(ACL 2025) - 阅读时机:设计记忆评估套件时
- Graphiti 开源时序知识图谱引擎(github.com/getzep/graphiti - 阅读时机:实现时序知识图谱时
- Cognee 开源知识图谱记忆(github.com/topoteretes/cognee - 阅读时机:为记忆构建可定制的 ECL 管道时
- [Cognee 对比:形态与功能](https://www.cognee.ai/blog/deep-dives/competition-comparison-form-vs-function) - 阅读时机:比较 Mem0、Graphiti、LightRAG、Cognee 之间的图谱结构时
---
## 技能元数据
**创建日期**2025-12-20
**最后更新**2026-05-15
**作者**:面向上下文工程的智能体技能贡献者
**版本**4.1.0
+550
View File
@@ -0,0 +1,550 @@
# 记忆系统:技术参考
本文档提供了记忆系统组件的实现细节。
## 向量存储实现
### 基础向量存储
```python
import numpy as np
from typing import List, Dict, Any
import json
def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
"""Compute cosine similarity between two vectors."""
norm_a = np.linalg.norm(a)
norm_b = np.linalg.norm(b)
if norm_a == 0 or norm_b == 0:
return 0.0
return float(np.dot(a, b) / (norm_a * norm_b))
class VectorStore:
def __init__(self, dimension=768):
self.dimension = dimension
self.vectors = []
self.metadata = []
self.texts = []
def add(self, text: str, metadata: Dict[str, Any] = None):
"""Add document to store."""
embedding = self._embed(text)
self.vectors.append(embedding)
self.metadata.append(metadata or {})
self.texts.append(text)
return len(self.vectors) - 1
def search(self, query: str, limit: int = 5,
filters: Dict[str, Any] = None) -> List[Dict]:
"""Search for similar documents."""
query_embedding = self._embed(query)
scores = []
for i, vec in enumerate(self.vectors):
score = cosine_similarity(query_embedding, vec)
# Apply filters
if filters and not self._matches_filters(self.metadata[i], filters):
score = -1 # Exclude
scores.append((i, score))
# Sort by score
scores.sort(key=lambda x: x[1], reverse=True)
# Return top k
results = []
for idx, score in scores[:limit]:
if score > 0: # Only include positive matches
results.append({
"index": idx,
"score": score,
"text": self._get_text(idx),
"metadata": self.metadata[idx]
})
return results
def _embed(self, text: str) -> np.ndarray:
"""Generate deterministic pseudo-embedding for demonstration.
In production, replace with actual embedding model."""
np.random.seed(hash(text) % (2**32))
vec = np.random.randn(self.dimension)
return vec / (np.linalg.norm(vec) + 1e-8)
def _matches_filters(self, metadata: Dict, filters: Dict) -> bool:
"""Check if metadata matches filters."""
for key, value in filters.items():
if key not in metadata:
return False
if isinstance(value, list):
if metadata[key] not in value:
return False
elif metadata[key] != value:
return False
return True
def _get_text(self, index: int) -> str:
"""Retrieve original text for index."""
return self.texts[index] if index < len(self.texts) else ""
```
### 元数据增强向量存储
```python
class MetadataVectorStore(VectorStore):
def __init__(self, dimension=768):
super().__init__(dimension)
self.entity_index = {} # entity -> [indices]
self.time_index = {} # time_range -> [indices]
def add(self, text: str, metadata: Dict[str, Any] = None):
"""Add with enhanced indexing."""
metadata = metadata or {}
index = super().add(text, metadata)
# Index by entity
if "entity" in metadata:
entity = metadata["entity"]
if entity not in self.entity_index:
self.entity_index[entity] = []
self.entity_index[entity].append(index)
# Index by time
if "valid_from" in metadata:
time_key = self._time_range_key(
metadata.get("valid_from"),
metadata.get("valid_until")
)
if time_key not in self.time_index:
self.time_index[time_key] = []
self.time_index[time_key].append(index)
return index
def search_by_entity(self, query: str, entity: str, limit: int = 5) -> List[Dict]:
"""Search within specific entity."""
indices = self.entity_index.get(entity, [])
filtered = [self.metadata[i] for i in indices]
# Score and rank
query_embedding = self._embed(query)
scored = []
for i, meta in zip(indices, filtered):
vec = self.vectors[i]
score = cosine_similarity(query_embedding, vec)
scored.append((i, score, meta))
scored.sort(key=lambda x: x[1], reverse=True)
return [{
"index": idx,
"score": score,
"metadata": meta
} for idx, score, meta in scored[:limit]]
```
## 知识图谱实现
### 属性图存储
```python
from typing import Dict, List, Optional
import uuid
class PropertyGraph:
def __init__(self):
self.nodes = {} # id -> properties
self.edges = [] # list of edge dicts
self.entity_registry = {} # name -> node_id (maintains identity)
self.indexes = {
"node_label": {}, # label -> [node_ids]
"edge_type": {} # type -> [edge_ids]
}
def get_or_create_node(self, name: str, label: str, properties: Dict = None) -> str:
"""Get existing node by name, or create a new one.
Uses entity_registry to ensure identity across interactions."""
if name in self.entity_registry:
return self.entity_registry[name]
node_id = self.create_node(label, {**(properties or {}), "name": name})
self.entity_registry[name] = node_id
return node_id
def create_node(self, label: str, properties: Dict = None) -> str:
"""Create node with label and properties."""
node_id = str(uuid.uuid4())
self.nodes[node_id] = {
"label": label,
"properties": properties or {}
}
# Index by label
if label not in self.indexes["node_label"]:
self.indexes["node_label"][label] = []
self.indexes["node_label"][label].append(node_id)
return node_id
def create_relationship(self, source_id: str, rel_type: str,
target_id: str, properties: Dict = None) -> str:
"""Create directed relationship between nodes."""
edge_id = str(uuid.uuid4())
self.edges.append({
"id": edge_id,
"source": source_id,
"target": target_id,
"type": rel_type,
"properties": properties or {}
})
# Index by type
if rel_type not in self.indexes["edge_type"]:
self.indexes["edge_type"][rel_type] = []
self.indexes["edge_type"][rel_type].append(edge_id)
return edge_id
def query(self, cypher_like: str, params: Dict = None) -> List[Dict]:
"""
Simple query matching.
Supports patterns like:
MATCH (e)-[r]->(o) WHERE e.id = $id RETURN r
"""
# In production, use actual graph database
# This is a simplified pattern matcher
results = []
if cypher_like.startswith("MATCH"):
# Parse basic pattern
pattern = self._parse_pattern(cypher_like)
results = self._match_pattern(pattern, params or {})
return results
def _parse_pattern(self, query: str) -> Dict:
"""Parse simplified MATCH pattern."""
# Simplified parser for demonstration
return {
"source_label": self._extract_label(query, "source"),
"rel_type": self._extract_type(query),
"target_label": self._extract_label(query, "target"),
"where": self._extract_where(query)
}
def _match_pattern(self, pattern: Dict, params: Dict) -> List[Dict]:
"""Match pattern against graph."""
results = []
for edge in self.edges:
# Match relationship type
if pattern["rel_type"] and edge["type"] != pattern["rel_type"]:
continue
source = self.nodes.get(edge["source"], {})
target = self.nodes.get(edge["target"], {})
# Match labels
if pattern["source_label"] and source.get("label") != pattern["source_label"]:
continue
if pattern["target_label"] and target.get("label") != pattern["target_label"]:
continue
# Match where clause
if pattern["where"] and not self._match_where(edge, source, target, params):
continue
results.append({
"source": source,
"relationship": edge,
"target": target
})
return results
```
## 时序知识图谱
```python
from datetime import datetime
from typing import Optional
class TemporalKnowledgeGraph(PropertyGraph):
def __init__(self):
super().__init__()
self.temporal_index = {} # time_range -> [edge_ids]
def create_temporal_relationship(
self,
source_id: str,
rel_type: str,
target_id: str,
valid_from: datetime,
valid_until: Optional[datetime] = None,
properties: Dict = None
) -> str:
"""Create relationship with temporal validity."""
edge_id = super().create_relationship(
source_id, rel_type, target_id, properties
)
# Index temporally
time_key = self._time_range_key(valid_from, valid_until)
if time_key not in self.temporal_index:
self.temporal_index[time_key] = []
self.temporal_index[time_key].append(edge_id)
# Store validity on edge
edge = self._get_edge(edge_id)
edge["valid_from"] = valid_from.isoformat()
edge["valid_until"] = valid_until.isoformat() if valid_until else None
return edge_id
def query_at_time(self, query: str, query_time: datetime) -> List[Dict]:
"""Query graph state at specific time."""
# Find edges valid at query time
valid_edges = []
for edge in self.edges:
valid_from = datetime.fromisoformat(edge.get("valid_from", "1970-01-01"))
valid_until = edge.get("valid_until")
if valid_from <= query_time:
if valid_until is None or datetime.fromisoformat(valid_until) > query_time:
valid_edges.append(edge)
# Match against pattern
pattern = self._parse_pattern(query)
results = []
for edge in valid_edges:
if pattern["rel_type"] and edge["type"] != pattern["rel_type"]:
continue
source = self.nodes.get(edge["source"], {})
target = self.nodes.get(edge["target"], {})
results.append({
"source": source,
"relationship": edge,
"target": target
})
return results
def _time_range_key(self, start: datetime, end: Optional[datetime]) -> str:
"""Create time range key for indexing."""
start_str = start.isoformat()
end_str = end.isoformat() if end else "infinity"
return f"{start_str}::{end_str}"
```
## 记忆整合
```python
class MemoryConsolidator:
def __init__(self, graph: PropertyGraph, vector_store: VectorStore):
self.graph = graph
self.vector_store = vector_store
self.consolidation_threshold = 1000 # memories before consolidation
def should_consolidate(self) -> bool:
"""Check if consolidation should trigger."""
total_memories = len(self.graph.nodes) + len(self.graph.edges)
return total_memories > self.consolidation_threshold
def consolidate(self):
"""Run consolidation process."""
# Step 1: Identify duplicate or merged facts
duplicates = self.find_duplicates()
# Step 2: Merge related facts
for group in duplicates:
self.merge_fact_group(group)
# Step 3: Update validity periods
self.update_validity_periods()
# Step 4: Rebuild indexes
self.rebuild_indexes()
def find_duplicates(self) -> List[List]:
"""Find groups of potentially duplicate facts."""
# Group by subject and predicate
groups = {}
for edge in self.graph.edges:
key = (edge["source"], edge["type"])
if key not in groups:
groups[key] = []
groups[key].append(edge)
# Return groups with multiple edges
return [edges for edges in groups.values() if len(edges) > 1]
def merge_fact_group(self, edges: List[Dict]):
"""Merge group of duplicate edges."""
if len(edges) == 1:
return
# Keep most recent/relevant
keeper = max(edges, key=lambda e: e.get("properties", {}).get("confidence", 0))
# Merge metadata
for edge in edges:
if edge["id"] != keeper["id"]:
self.merge_properties(keeper, edge)
self.graph.edges.remove(edge)
def merge_properties(self, target: Dict, source: Dict):
"""Merge properties from source into target."""
for key, value in source.get("properties", {}).items():
if key not in target["properties"]:
target["properties"][key] = value
elif isinstance(value, list):
target["properties"][key].extend(value)
```
## 记忆-上下文集成
```python
class MemoryContextIntegrator:
def __init__(self, memory_system, context_limit=100000):
self.memory_system = memory_system
self.context_limit = context_limit
def build_context(self, task: str, current_context: str = "") -> str:
"""Build context including relevant memories."""
# Extract entities from task
entities = self._extract_entities(task)
# Retrieve memories for each entity
memories = []
for entity in entities:
entity_memories = self.memory_system.retrieve_entity(entity)
memories.extend(entity_memories)
# Format memories for context
memory_section = self._format_memories(memories)
# Combine with current context
combined = current_context + "\n\n" + memory_section
# Check limit and truncate if needed
if self._token_count(combined) > self.context_limit:
combined = self._truncate_context(combined, self.context_limit)
return combined
def _extract_entities(self, task: str) -> List[str]:
"""Extract entity mentions from task."""
# In production, use NER or entity extraction
import re
pattern = r"\[([^\]]+)\]" # [[entity_name]] convention
return re.findall(pattern, task)
def _format_memories(self, memories: List[Dict]) -> str:
"""Format memories for context injection."""
sections = ["## Relevant Memories"]
for memory in memories:
formatted = f"- {memory.get('content', '')}"
if "source" in memory:
formatted += f" (Source: {memory['source']})"
if "timestamp" in memory:
formatted += f" [Time: {memory['timestamp']}]"
sections.append(formatted)
return "\n".join(sections)
def _token_count(self, text: str) -> int:
"""Estimate token count."""
return len(text) // 4 # Rough approximation
def _truncate_context(self, context: str, limit: int) -> str:
"""Truncate context to fit limit."""
tokens = context.split()
truncated = []
count = 0
for token in tokens:
if count + 1 > limit:
break
truncated.append(token)
count += 1
return " ".join(truncated)
```
## 框架集成示例
### Mem0 快速入门
```python
from mem0 import Memory
# Initialize with default config (uses local storage)
m = Memory()
# Store memories with user scoping
m.add("Prefers Python 3.12 with type hints", user_id="dev-alice")
m.add("Working on microservices migration", user_id="dev-alice")
# Search with natural language
results = m.search("What language does the user prefer?", user_id="dev-alice")
# Batch operations
m.add([
"Sprint goal: complete auth service",
"Blocked on database schema review"
], user_id="dev-alice")
```
### GraphitiZep 开源时序知识图谱引擎)
```python
from graphiti_core import Graphiti
from graphiti_core.nodes import EpisodeType
# Initialize with Neo4j backend
graphiti = Graphiti("bolt://localhost:7687", "neo4j", "password")
# Add episodes (conversations, events)
await graphiti.add_episode(
name="user_conversation_42",
episode_body="Alice mentioned she moved to Berlin in January.",
source=EpisodeType.message,
source_description="Chat with Alice"
)
# Search combines semantic, keyword, and graph traversal
results = await graphiti.search("Where does Alice live?")
```
### CogneeAI 记忆开源知识引擎)
```python
import cognee
from cognee.modules.search.types import SearchType
# ECL pipeline: add → cognify → memify → search
await cognee.add("./docs/")
await cognee.add("any-data")
await cognee.cognify()
await cognee.memify()
# Graph-aware retrieval (default: GRAPH_COMPLETION)
results = await cognee.search(
query_text="any query to search in memory",
query_type=SearchType.GRAPH_COMPLETION,
)
# Raw chunks when agent reasons over text itself
chunks = await cognee.search(
query_text="any query to search in memory",
query_type=SearchType.CHUNKS,
)
```
+616
View File
@@ -0,0 +1,616 @@
"""Memory System Implementation.
Provides composable building blocks for agent memory: vector stores with
metadata indexing, property graphs for entity relationships, and temporal
knowledge graphs for facts that change over time.
Use when:
- Building a memory persistence layer for an agent that must retain
knowledge across sessions.
- Prototyping memory architectures before committing to a production
framework (Mem0, Zep/Graphiti, Letta, Cognee).
- Combining semantic search with graph-based entity retrieval in a
single integrated system.
Typical usage::
from memory_store import IntegratedMemorySystem
mem = IntegratedMemorySystem()
mem.start_session("session-001")
mem.store_fact("Alice prefers dark mode", entity="Alice")
results = mem.retrieve_memories("theme preference")
"""
import hashlib
import json
from datetime import datetime
from typing import Any, Dict, List, Optional
import numpy as np
__all__ = [
"VectorStore",
"PropertyGraph",
"TemporalKnowledgeGraph",
"IntegratedMemorySystem",
]
class VectorStore:
"""Simple vector store with metadata indexing.
Use when: the agent needs semantic similarity search over stored facts
with optional entity and temporal filtering.
"""
def __init__(self, dimension: int = 768) -> None:
self.dimension: int = dimension
self.vectors: List[np.ndarray] = []
self.metadata: List[Dict[str, Any]] = []
self.entity_index: Dict[str, List[int]] = {}
self.time_index: Dict[str, List[int]] = {}
def add(self, text: str, metadata: Optional[Dict[str, Any]] = None) -> int:
"""Add document to store.
Use when: persisting a new fact or observation that the agent should
be able to retrieve later via semantic search.
"""
metadata = metadata or {}
embedding: np.ndarray = self._embed(text)
index: int = len(self.vectors)
self.vectors.append(embedding)
self.metadata.append(metadata)
# Index by entity
if "entity" in metadata:
entity: str = metadata["entity"]
if entity not in self.entity_index:
self.entity_index[entity] = []
self.entity_index[entity].append(index)
# Index by time
if "valid_from" in metadata:
time_key: str = self._time_key(metadata["valid_from"])
if time_key not in self.time_index:
self.time_index[time_key] = []
self.time_index[time_key].append(index)
return index
def search(
self,
query: str,
limit: int = 5,
filters: Optional[Dict[str, Any]] = None,
) -> List[Dict[str, Any]]:
"""Search for similar documents.
Use when: retrieving memories relevant to a query, optionally
narrowed by metadata filters (entity, session, time range).
"""
query_embedding: np.ndarray = self._embed(query)
scores: List[tuple[int, float]] = []
for i, vec in enumerate(self.vectors):
score: float = float(
np.dot(query_embedding, vec)
/ (np.linalg.norm(query_embedding) * np.linalg.norm(vec) + 1e-8)
)
# Apply filters
if filters and not self._matches_filters(self.metadata[i], filters):
score = -1.0
scores.append((i, score))
scores.sort(key=lambda x: x[1], reverse=True)
results: List[Dict[str, Any]] = []
for idx, score in scores[:limit]:
if score > 0:
results.append(
{
"index": idx,
"score": score,
"text": self.metadata[idx].get("text", ""),
"metadata": self.metadata[idx],
}
)
return results
def search_by_entity(
self, entity: str, query: str = "", limit: int = 5
) -> List[Dict[str, Any]]:
"""Search within specific entity.
Use when: the agent needs all memories associated with a known
entity, optionally ranked by relevance to a query.
"""
indices: List[int] = self.entity_index.get(entity, [])
if not indices:
return []
if query:
query_embedding: np.ndarray = self._embed(query)
scored: List[tuple[int, float, Dict[str, Any]]] = []
for i in indices:
vec: np.ndarray = self.vectors[i]
score: float = float(
np.dot(query_embedding, vec)
/ (np.linalg.norm(query_embedding) * np.linalg.norm(vec) + 1e-8)
)
scored.append((i, score, self.metadata[i]))
scored.sort(key=lambda x: x[1], reverse=True)
return [
{"index": i, "score": s, "metadata": m}
for i, s, m in scored[:limit]
]
else:
return [
{"index": i, "score": 1.0, "metadata": self.metadata[i]}
for i in indices[:limit]
]
def _embed(self, text: str) -> np.ndarray:
"""Generate embedding for text.
In production, replace with an actual embedding model. This
deterministic stub uses the text hash as a random seed so that
identical texts always produce identical vectors. Uses a local
RNG to avoid corrupting global numpy random state.
"""
rng = np.random.default_rng(hash(text) % (2**32))
return rng.standard_normal(self.dimension)
def _time_key(self, timestamp: Any) -> str:
"""Create time key for indexing."""
if isinstance(timestamp, datetime):
return timestamp.strftime("%Y-%m")
return str(timestamp)
def _matches_filters(self, metadata: Dict[str, Any], filters: Dict[str, Any]) -> bool:
"""Check if metadata matches filters."""
for key, value in filters.items():
if key not in metadata:
return False
if isinstance(value, list):
if metadata[key] not in value:
return False
elif metadata[key] != value:
return False
return True
class PropertyGraph:
"""Simple property graph storage.
Use when: the agent needs to maintain entity relationships and
traverse connections between nodes (e.g., "find all projects
associated with this user").
"""
def __init__(self) -> None:
self.nodes: Dict[str, Dict[str, Any]] = {}
self.edges: Dict[str, Dict[str, Any]] = {}
self.entity_registry: Dict[str, str] = {} # name -> node_id
self.node_index: Dict[str, List[str]] = {} # label -> node_ids
self.edge_index: Dict[str, List[str]] = {} # type -> edge_ids
def get_or_create_node(
self, name: str, label: str = "Entity", properties: Optional[Dict[str, Any]] = None
) -> str:
"""Get existing node by name, or create a new one.
Use when: storing an entity that may already exist. The entity
registry ensures identity is maintained across interactions
("John Doe" always maps to the same node).
"""
if name in self.entity_registry:
node_id: str = self.entity_registry[name]
if properties:
self.nodes[node_id]["properties"].update(properties)
return node_id
node_id = self.create_node(label, {**(properties or {}), "name": name})
self.entity_registry[name] = node_id
return node_id
def create_node(self, label: str, properties: Optional[Dict[str, Any]] = None) -> str:
"""Create node with label and properties.
Use when: adding a new entity to the graph that does not need
identity deduplication (prefer get_or_create_node otherwise).
"""
node_id: str = hashlib.md5(f"{label}{datetime.now().isoformat()}".encode()).hexdigest()[:16]
self.nodes[node_id] = {
"id": node_id,
"label": label,
"properties": properties or {},
"created_at": datetime.now().isoformat(),
}
if label not in self.node_index:
self.node_index[label] = []
self.node_index[label].append(node_id)
return node_id
def create_relationship(
self,
source_id: str,
rel_type: str,
target_id: str,
properties: Optional[Dict[str, Any]] = None,
) -> str:
"""Create directed relationship between nodes.
Use when: recording a connection between two entities (e.g.,
WORKS_AT, LIVES_IN, DEPENDS_ON).
"""
if source_id not in self.nodes:
raise ValueError(f"Unknown source node: {source_id}")
if target_id not in self.nodes:
raise ValueError(f"Unknown target node: {target_id}")
edge_id: str = hashlib.md5(
f"{source_id}{rel_type}{target_id}{datetime.now().isoformat()}".encode()
).hexdigest()[:16]
self.edges[edge_id] = {
"id": edge_id,
"source": source_id,
"target": target_id,
"type": rel_type,
"properties": properties or {},
"created_at": datetime.now().isoformat(),
}
if rel_type not in self.edge_index:
self.edge_index[rel_type] = []
self.edge_index[rel_type].append(edge_id)
return edge_id
def query(self, pattern: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Query graph with simple pattern matching.
Use when: finding relationships that match a structural pattern
(e.g., all WORKS_AT edges from Person nodes).
"""
results: List[Dict[str, Any]] = []
# Match by edge type
if "type" in pattern:
edge_ids: List[str] = self.edge_index.get(pattern["type"], [])
for eid in edge_ids:
edge: Dict[str, Any] = self.edges[eid]
source: Dict[str, Any] = self.nodes.get(edge["source"], {})
target: Dict[str, Any] = self.nodes.get(edge["target"], {})
# Match source label
if "source_label" in pattern:
if source.get("label") != pattern["source_label"]:
continue
# Match target label
if "target_label" in pattern:
if target.get("label") != pattern["target_label"]:
continue
results.append({"source": source, "edge": edge, "target": target})
return results
def get_node(self, node_id: str) -> Optional[Dict[str, Any]]:
"""Get node by ID."""
return self.nodes.get(node_id)
def get_relationships(
self, node_id: str, direction: str = "both"
) -> List[Dict[str, Any]]:
"""Get relationships for a node.
Use when: retrieving all connections for a given entity to build
a complete entity context.
"""
relationships: List[Dict[str, Any]] = []
for edge in self.edges.values():
if direction in ["outgoing", "both"] and edge["source"] == node_id:
relationships.append(
{
"edge": edge,
"target": self.nodes.get(edge["target"]),
"direction": "outgoing",
}
)
if direction in ["incoming", "both"] and edge["target"] == node_id:
relationships.append(
{
"edge": edge,
"source": self.nodes.get(edge["source"]),
"direction": "incoming",
}
)
return relationships
class TemporalKnowledgeGraph(PropertyGraph):
"""Property graph with temporal validity for facts.
Use when: the agent must track facts that change over time and
answer time-scoped queries (e.g., "where did the user live in
March 2024?").
"""
def create_temporal_relationship(
self,
source_id: str,
rel_type: str,
target_id: str,
valid_from: datetime,
valid_until: Optional[datetime] = None,
properties: Optional[Dict[str, Any]] = None,
) -> str:
"""Create relationship with temporal validity.
Use when: recording a fact that has a known start time and
may expire (e.g., employment, address, subscription status).
"""
edge_id: str = super().create_relationship(
source_id, rel_type, target_id, properties
)
# Add temporal properties
self.edges[edge_id]["valid_from"] = valid_from.isoformat()
self.edges[edge_id]["valid_until"] = (
valid_until.isoformat() if valid_until else None
)
return edge_id
def query_at_time(
self, query: Dict[str, Any], query_time: datetime
) -> List[Dict[str, Any]]:
"""Query graph state at specific time.
Use when: answering point-in-time questions about entities
(e.g., "what was true on date X?").
"""
results: List[Dict[str, Any]] = []
# Get base query results
base_results: List[Dict[str, Any]] = self.query(query)
for result in base_results:
edge: Dict[str, Any] = result["edge"]
valid_from: datetime = datetime.fromisoformat(
edge.get("valid_from", "1970-01-01")
)
valid_until: Optional[str] = edge.get("valid_until")
# Check temporal validity
if valid_from <= query_time:
if valid_until is None or datetime.fromisoformat(valid_until) > query_time:
results.append(
{
**result,
"valid_from": valid_from,
"valid_until": valid_until,
}
)
return results
def query_time_range(
self,
query: Dict[str, Any],
start_time: datetime,
end_time: datetime,
) -> List[Dict[str, Any]]:
"""Query facts valid during time range.
Use when: retrieving all facts that overlap with a given time
window (e.g., "what changed between January and June?").
"""
results: List[Dict[str, Any]] = []
base_results: List[Dict[str, Any]] = self.query(query)
for result in base_results:
edge: Dict[str, Any] = result["edge"]
valid_from: datetime = datetime.fromisoformat(
edge.get("valid_from", "1970-01-01")
)
valid_until: Optional[str] = edge.get("valid_until")
# Check if overlaps with query range
until_dt: datetime = (
datetime.fromisoformat(valid_until) if valid_until else datetime.max
)
if until_dt >= start_time and valid_from <= end_time:
results.append(
{
**result,
"valid_from": valid_from,
"valid_until": valid_until,
}
)
return results
# ---------------------------------------------------------------------------
# Memory System Integration
# ---------------------------------------------------------------------------
class IntegratedMemorySystem:
"""Integrated memory system combining vector store and graph.
Use when: the agent needs both semantic search over facts and
graph-based entity relationship traversal in a single unified
interface. This class composes VectorStore and TemporalKnowledgeGraph,
enriching vector search results with graph context.
"""
def __init__(self) -> None:
self.vector_store: VectorStore = VectorStore()
self.graph: TemporalKnowledgeGraph = TemporalKnowledgeGraph()
self.session_id: str = ""
def start_session(self, session_id: str) -> None:
"""Start a new memory session.
Use when: beginning a new conversation or task that should
scope its memories to a distinct session identifier.
"""
self.session_id = session_id
def store_fact(
self,
fact: str,
entity: str,
timestamp: Optional[datetime] = None,
relationships: Optional[List[Dict[str, Any]]] = None,
) -> None:
"""Store a fact with entity and relationships.
Use when: the agent observes a new piece of information that
should be persisted for future retrieval. Stores in both the
vector store (for semantic search) and the graph (for entity
traversal).
"""
# Store in vector store
self.vector_store.add(
fact,
{
"text": fact,
"entity": entity,
"valid_from": (timestamp or datetime.now()).isoformat(),
"session_id": self.session_id,
},
)
# Get or create entity node (uses registry for identity)
entity_node_id: str = self.graph.get_or_create_node(entity)
# Create relationships
if relationships:
for rel in relationships:
target_node_id: str = self.graph.get_or_create_node(rel["target"])
self.graph.create_relationship(
entity_node_id,
rel["type"],
target_node_id,
properties=rel.get("properties", {}),
)
def retrieve_memories(
self,
query: str,
entity_filter: Optional[str] = None,
time_filter: Optional[Dict[str, Any]] = None,
limit: int = 5,
) -> List[Dict[str, Any]]:
"""Retrieve memories matching query.
Use when: the agent needs to recall previously stored facts,
optionally filtered by entity or time. Results are enriched
with graph relationships for each matched entity.
"""
# Vector search
filters: Dict[str, Any] = {"session_id": self.session_id}
if entity_filter:
filters["entity"] = entity_filter
results: List[Dict[str, Any]] = self.vector_store.search(
query, limit=limit, filters=filters
)
# Enrich with graph relationships
for result in results:
entity: Optional[str] = result["metadata"].get("entity")
if entity:
node_id: Optional[str] = self.graph.entity_registry.get(entity)
if node_id:
result["relationships"] = self.graph.get_relationships(node_id)
return results
def retrieve_entity_context(self, entity: str) -> Dict[str, Any]:
"""Retrieve complete context for an entity.
Use when: the agent needs a full picture of a single entity
including its properties, all relationships, and associated
vector memories.
"""
node_id: Optional[str] = self.graph.entity_registry.get(entity)
# Get entity node
entity_node: Optional[Dict[str, Any]] = (
self.graph.get_node(node_id) if node_id else None
)
# Get relationships
relationships: List[Dict[str, Any]] = (
self.graph.get_relationships(node_id) if node_id else []
)
# Get vector memories
memories: List[Dict[str, Any]] = self.vector_store.search_by_entity(
entity, limit=10
)
return {
"entity": entity_node,
"relationships": relationships,
"memories": memories,
}
def consolidate(self) -> None:
"""Consolidate memories and remove outdated information.
Use when: memory count exceeds a threshold, retrieval quality
degrades, or on a scheduled interval. In production, implement:
- Merge related facts into summaries
- Update validity periods on stale entries
- Archive obsolete facts (invalidate, do not discard)
"""
pass
if __name__ == "__main__":
# Quick smoke test demonstrating the integrated memory system.
mem = IntegratedMemorySystem()
mem.start_session("demo-session")
# Store facts with entity relationships
mem.store_fact(
"Alice prefers dark mode",
entity="Alice",
relationships=[{"target": "dark mode", "type": "PREFERS"}],
)
mem.store_fact(
"Alice works at Acme Corp",
entity="Alice",
relationships=[{"target": "Acme Corp", "type": "WORKS_AT"}],
)
# Semantic retrieval
results = mem.retrieve_memories("theme preference")
print(f"Search results: {len(results)} memories found")
for r in results:
print(f" score={r['score']:.3f} text={r['text']}")
# Entity context
context = mem.retrieve_entity_context("Alice")
print(f"\nAlice context: {len(context['relationships'])} relationships, "
f"{len(context['memories'])} memories")