chore: import upstream snapshot with attribution
Python Build and Type Check / python-ci (ubuntu-latest, 3.11) (push) Has been cancelled
Python Build and Type Check / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Build and Type Check / python-ci (windows-latest, 3.11) (push) Has been cancelled
Python Build and Type Check / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Integration Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Integration Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Notebook Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Notebook Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Smoke Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Smoke Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Unit Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Unit Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
gh-pages / build (push) Has been cancelled
Python Publish (pypi) / Upload release to PyPI (push) Has been cancelled
Spellcheck / spellcheck (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:37:31 +08:00
commit 6b7e6b44f1
897 changed files with 94808 additions and 0 deletions
+42
View File
@@ -0,0 +1,42 @@
# GraphRAG Cache
This package contains a collection of utilities to handle GraphRAG caching implementation.
### Basic
This example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system.
[Open the notebook to explore the basic example code](example-notebooks/basic_cache_example.ipynb)
### Custom Cache
This example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.
[Open the notebook to explore the basic custom example code](example-notebooks/custom_cache_example.ipynb)
#### Details
By default, the `create_cache` comes with the following cache providers registered that correspond to the entries in the `CacheType` enum.
- `JsonCache`
- `MemoryCache`
- `NoopCache`
The preregistration happens dynamically, e.g., `JsonCache` is only imported and registered if you request a `JsonCache` with `create_cache(CacheType.Json, ...)`. There is no need to manually import and register builtin cache providers when using `create_cache`.
If you want a clean factory with no preregistered cache providers then directly import `cache_factory` and bypass using `create_cache`. The downside is that `cache_factory.create` uses a dict for init args instead of the strongly typed `CacheConfig` used with `create_cache`.
```python
from graphrag_cache.cache_factory import cache_factory
from graphrag_cache.json_cache import JsonCache
# cache_factory has no preregistered providers so you must register any
# providers you plan on using.
# May also register a custom implementation, see above for example.
cache_factory.register("my_cache_impl", JsonCache)
cache = cache_factory.create(strategy="my_cache_impl", init_args={"some_setting": "..."})
...
```
@@ -0,0 +1,96 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "fcb917cf",
"metadata": {},
"outputs": [],
"source": [
"# Copyright (c) 2026 Microsoft Corporation.\n",
"# Licensed under the MIT License."
]
},
{
"cell_type": "markdown",
"id": "33461307",
"metadata": {},
"source": [
"## Basic cache example\n",
"\n",
"This example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system. "
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "ee33abd6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Value stored in cache for 'my_key':\n",
"{'k1': 'object to cache'}\n",
"\n",
"Value stored in cache for cache_key using data dict:\n",
"{'k2': 'object to cache'}\n"
]
}
],
"source": [
"from graphrag_cache import CacheConfig, CacheType, create_cache, create_cache_key\n",
"from graphrag_storage import StorageConfig, StorageType\n",
"\n",
"\n",
"async def run():\n",
" \"\"\"Demonstrate basic cache usage with graphrag_cache.\"\"\"\n",
" cache = create_cache()\n",
"\n",
" # The above is equivalent to the following:\n",
" cache = create_cache(\n",
" CacheConfig(\n",
" type=CacheType.Json,\n",
" storage=StorageConfig(type=StorageType.File, base_dir=\"cache\"),\n",
" ),\n",
" )\n",
"\n",
" await cache.set(\"my_key\", {\"k1\": \"object to cache\"})\n",
" print(\"Value stored in cache for 'my_key':\")\n",
" print(await cache.get(\"my_key\"))\n",
"\n",
" # create cache key from data dict.\n",
" cache_key = create_cache_key({\"some_arg\": \"some_value\", \"something_else\": 5})\n",
" await cache.set(cache_key, {\"k2\": \"object to cache\"})\n",
" print(\"\\nValue stored in cache for cache_key using data dict:\")\n",
" print(await cache.get(cache_key))\n",
"\n",
"\n",
"if __name__ == \"__main__\":\n",
" await run()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
@@ -0,0 +1,129 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "42524e97",
"metadata": {},
"outputs": [],
"source": [
"# Copyright (c) 2026 Microsoft Corporation.\n",
"# Licensed under the MIT License."
]
},
{
"cell_type": "markdown",
"id": "a7cd6743",
"metadata": {},
"source": [
"## Custom cache example\n",
"\n",
"This example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "e7ec8dc1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Value stored in cache for 'my_key':\n",
"{'k1': 'object to cache'}\n"
]
}
],
"source": [
"from typing import Any\n",
"\n",
"from graphrag_cache import Cache, CacheConfig, create_cache, register_cache\n",
"\n",
"\n",
"class MyCache(Cache):\n",
" \"\"\"Custom cache implementation for storing and retrieving cached data.\"\"\"\n",
"\n",
" def __init__(\n",
" self,\n",
" some_setting: str,\n",
" optional_setting: str = \"default setting\",\n",
" **kwargs: Any,\n",
" ):\n",
" # Validate settings and initialize\n",
" # View the JsonCache implementation to see how to create a cache that relies on a Storage provider.\n",
" self.some_setting = some_setting\n",
" self.optional_setting = optional_setting\n",
" self._cache: dict[str, Any] = {}\n",
" self._child: Cache = self\n",
"\n",
" async def get(self, key: str) -> Any:\n",
" \"\"\"Retrieve a value from the cache by key.\"\"\"\n",
" return self._cache.get(key)\n",
"\n",
" async def set(self, key: str, value: Any, debug_data: Any = None) -> None:\n",
" \"\"\"Store a value in the cache with the specified key.\"\"\"\n",
" self._cache[key] = value\n",
"\n",
" async def has(self, key: str) -> bool:\n",
" \"\"\"Check if a key exists in the cache.\"\"\"\n",
" return key in self._cache\n",
"\n",
" async def delete(self, key: str) -> None:\n",
" \"\"\"Remove a key and its value from the cache.\"\"\"\n",
" self._cache.pop(key, None)\n",
"\n",
" async def clear(self) -> None:\n",
" \"\"\"Clear all items from the cache.\"\"\"\n",
" self._cache.clear()\n",
"\n",
" def child(self, name: str) -> Cache:\n",
" \"\"\"Create or access a child cache (not implemented in this example).\"\"\"\n",
" return self._child\n",
"\n",
"\n",
"register_cache(\"MyCache\", MyCache)\n",
"\n",
"\n",
"async def run():\n",
" \"\"\"Demonstrate usage of the custom cache implementation.\"\"\"\n",
" cache = create_cache(\n",
" CacheConfig(\n",
" type=\"MyCache\",\n",
" some_setting=\"important setting value\", # type: ignore\n",
" )\n",
" )\n",
"\n",
" await cache.set(\"my_key\", {\"k1\": \"object to cache\"})\n",
" print(\"Value stored in cache for 'my_key':\")\n",
" print(await cache.get(\"my_key\"))\n",
"\n",
"\n",
"if __name__ == \"__main__\":\n",
" await run()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
@@ -0,0 +1,20 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""The GraphRAG Cache package."""
from graphrag_cache.cache import Cache
from graphrag_cache.cache_config import CacheConfig
from graphrag_cache.cache_factory import create_cache, register_cache
from graphrag_cache.cache_key import CacheKeyCreator, create_cache_key
from graphrag_cache.cache_type import CacheType
__all__ = [
"Cache",
"CacheConfig",
"CacheKeyCreator",
"CacheType",
"create_cache",
"create_cache_key",
"register_cache",
]
@@ -0,0 +1,74 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""Abstract base class for cache."""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from graphrag_storage import Storage
class Cache(ABC):
"""Provide a cache interface for the pipeline."""
@abstractmethod
def __init__(self, *, storage: Storage | None, **kwargs: Any) -> None:
"""Create a cache instance."""
@abstractmethod
async def get(self, key: str) -> Any:
"""Get the value for the given key.
Args:
- key - The key to get the value for.
- as_bytes - Whether or not to return the value as bytes.
Returns
-------
- output - The value for the given key.
"""
@abstractmethod
async def set(self, key: str, value: Any, debug_data: dict | None = None) -> None:
"""Set the value for the given key.
Args:
- key - The key to set the value for.
- value - The value to set.
"""
@abstractmethod
async def has(self, key: str) -> bool:
"""Return True if the given key exists in the cache.
Args:
- key - The key to check for.
Returns
-------
- output - True if the key exists in the cache, False otherwise.
"""
@abstractmethod
async def delete(self, key: str) -> None:
"""Delete the given key from the cache.
Args:
- key - The key to delete.
"""
@abstractmethod
async def clear(self) -> None:
"""Clear the cache."""
@abstractmethod
def child(self, name: str) -> Cache:
"""Create a child cache with the given name.
Args:
- name - The name to create the sub cache with.
"""
@@ -0,0 +1,26 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""Cache configuration model."""
from graphrag_storage import StorageConfig, StorageType
from pydantic import BaseModel, ConfigDict, Field
from graphrag_cache.cache_type import CacheType
class CacheConfig(BaseModel):
"""The configuration section for cache."""
model_config = ConfigDict(extra="allow")
"""Allow extra fields to support custom cache implementations."""
type: str = Field(
description="The cache type to use. Builtin types include 'Json', 'Memory', and 'Noop'.",
default=CacheType.Json,
)
storage: StorageConfig | None = Field(
description="The storage configuration to use for file-based caches such as 'Json'.",
default_factory=lambda: StorageConfig(type=StorageType.File, base_dir="cache"),
)
@@ -0,0 +1,89 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""Cache factory implementation."""
from collections.abc import Callable
from graphrag_common.factory import Factory, ServiceScope
from graphrag_storage import Storage, create_storage
from graphrag_cache.cache import Cache
from graphrag_cache.cache_config import CacheConfig
from graphrag_cache.cache_type import CacheType
class CacheFactory(Factory["Cache"]):
"""A factory class for cache implementations."""
cache_factory = CacheFactory()
def register_cache(
cache_type: str,
cache_initializer: Callable[..., Cache],
scope: ServiceScope = "transient",
) -> None:
"""Register a custom cache implementation.
Args
----
- cache_type: str
The cache id to register.
- cache_initializer: Callable[..., Cache]
The cache initializer to register.
"""
cache_factory.register(cache_type, cache_initializer, scope)
def create_cache(
config: CacheConfig | None = None, storage: Storage | None = None
) -> "Cache":
"""Create a cache implementation based on the given configuration.
Args
----
- config: CacheConfig
The cache configuration to use.
- storage: Storage | None
The storage implementation to use for file-based caches such as 'Json'.
Returns
-------
Cache
The created cache implementation.
"""
config = config or CacheConfig()
config_model = config.model_dump()
cache_strategy = config.type
if not storage and config.storage:
storage = create_storage(config.storage)
if cache_strategy not in cache_factory:
match cache_strategy:
case CacheType.Json:
from graphrag_cache.json_cache import JsonCache
register_cache(CacheType.Json, JsonCache)
case CacheType.Memory:
from graphrag_cache.memory_cache import MemoryCache
register_cache(CacheType.Memory, MemoryCache)
case CacheType.Noop:
from graphrag_cache.noop_cache import NoopCache
register_cache(CacheType.Noop, NoopCache)
case _:
msg = f"CacheConfig.type '{cache_strategy}' is not registered in the CacheFactory. Registered types: {', '.join(cache_factory.keys())}."
raise ValueError(msg)
if storage:
config_model["storage"] = storage
return cache_factory.create(strategy=cache_strategy, init_args=config_model)
@@ -0,0 +1,36 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""Create cache key."""
from typing import Any, Protocol, runtime_checkable
from graphrag_common.hasher import hash_data
@runtime_checkable
class CacheKeyCreator(Protocol):
"""Create cache key function protocol.
Args
----
input_args: dict[str, Any]
The input arguments for creating the cache key.
Returns
-------
str
The generated cache key.
"""
def __call__(
self,
input_args: dict[str, Any],
) -> str:
"""Create cache key."""
...
def create_cache_key(input_args: dict[str, Any]) -> str:
"""Create a cache key based on the input arguments."""
return hash_data(input_args)
@@ -0,0 +1,15 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""Builtin cache implementation types."""
from enum import StrEnum
class CacheType(StrEnum):
"""Enum for cache types."""
Json = "json"
Memory = "memory"
Noop = "none"
@@ -0,0 +1,72 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""A module containing 'JsonCache' model."""
import json
from typing import Any
from graphrag_storage import Storage, StorageConfig, create_storage
from graphrag_cache.cache import Cache
class JsonCache(Cache):
"""File pipeline cache class definition."""
_storage: Storage
def __init__(
self,
storage: Storage | dict[str, Any] | None = None,
**kwargs: Any,
) -> None:
"""Init method definition."""
if storage is None:
msg = "JsonCache requires either a Storage instance to be provided or a StorageConfig to create one."
raise ValueError(msg)
if isinstance(storage, Storage):
self._storage = storage
else:
self._storage = create_storage(StorageConfig(**storage))
async def get(self, key: str) -> Any | None:
"""Get method definition."""
if await self.has(key):
try:
data = await self._storage.get(key)
data = json.loads(data)
except UnicodeDecodeError:
await self._storage.delete(key)
return None
except json.decoder.JSONDecodeError:
await self._storage.delete(key)
return None
else:
return data.get("result")
return None
async def set(self, key: str, value: Any, debug_data: dict | None = None) -> None:
"""Set method definition."""
if value is None:
return
data = {"result": value, **(debug_data or {})}
await self._storage.set(key, json.dumps(data, ensure_ascii=False))
async def has(self, key: str) -> bool:
"""Has method definition."""
return await self._storage.has(key)
async def delete(self, key: str) -> None:
"""Delete method definition."""
if await self.has(key):
await self._storage.delete(key)
async def clear(self) -> None:
"""Clear method definition."""
await self._storage.clear()
def child(self, name: str) -> "Cache":
"""Child method definition."""
return JsonCache(storage=self._storage.child(name))
@@ -0,0 +1,69 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""MemoryCache implementation."""
from typing import Any
from graphrag_cache.cache import Cache
class MemoryCache(Cache):
"""In memory cache class definition."""
_cache: dict[str, Any]
_name: str
def __init__(self, **kwargs: Any) -> None:
"""Init method definition."""
self._cache = {}
async def get(self, key: str) -> Any:
"""Get the value for the given key.
Args:
- key - The key to get the value for.
- as_bytes - Whether or not to return the value as bytes.
Returns
-------
- output - The value for the given key.
"""
return self._cache.get(key)
async def set(self, key: str, value: Any, debug_data: dict | None = None) -> None:
"""Set the value for the given key.
Args:
- key - The key to set the value for.
- value - The value to set.
"""
self._cache[key] = value
async def has(self, key: str) -> bool:
"""Return True if the given key exists in the storage.
Args:
- key - The key to check for.
Returns
-------
- output - True if the key exists in the storage, False otherwise.
"""
return key in self._cache
async def delete(self, key: str) -> None:
"""Delete the given key from the storage.
Args:
- key - The key to delete.
"""
del self._cache[key]
async def clear(self) -> None:
"""Clear the storage."""
self._cache.clear()
def child(self, name: str) -> "Cache":
"""Create a sub cache with the given name."""
return MemoryCache()
@@ -0,0 +1,68 @@
# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License
"""NoopCache implementation."""
from typing import Any
from graphrag_cache.cache import Cache
class NoopCache(Cache):
"""A no-op implementation of Cache, usually useful for testing."""
def __init__(self, **kwargs: Any) -> None:
"""Init method definition."""
async def get(self, key: str) -> Any:
"""Get the value for the given key.
Args:
- key - The key to get the value for.
- as_bytes - Whether or not to return the value as bytes.
Returns
-------
- output - The value for the given key.
"""
return None
async def set(
self, key: str, value: str | bytes | None, debug_data: dict | None = None
) -> None:
"""Set the value for the given key.
Args:
- key - The key to set the value for.
- value - The value to set.
"""
async def has(self, key: str) -> bool:
"""Return True if the given key exists in the cache.
Args:
- key - The key to check for.
Returns
-------
- output - True if the key exists in the cache, False otherwise.
"""
return False
async def delete(self, key: str) -> None:
"""Delete the given key from the cache.
Args:
- key - The key to delete.
"""
async def clear(self) -> None:
"""Clear the cache."""
def child(self, name: str) -> "Cache":
"""Create a child cache with the given name.
Args:
- name - The name to create the sub cache with.
"""
return self
+43
View File
@@ -0,0 +1,43 @@
[project]
name = "graphrag-cache"
version = "3.1.0"
description = "GraphRAG cache package."
authors = [
{name = "Alonso Guevara Fernández", email = "alonsog@microsoft.com"},
{name = "Andrés Morales Esquivel", email = "andresmor@microsoft.com"},
{name = "Chris Trevino", email = "chtrevin@microsoft.com"},
{name = "David Tittsworth", email = "datittsw@microsoft.com"},
{name = "Dayenne de Souza", email = "ddesouza@microsoft.com"},
{name = "Derek Worthen", email = "deworthe@microsoft.com"},
{name = "Gaudy Blanco Meneses", email = "gaudyb@microsoft.com"},
{name = "Ha Trinh", email = "trinhha@microsoft.com"},
{name = "Jonathan Larson", email = "jolarso@microsoft.com"},
{name = "Josh Bradley", email = "joshbradley@microsoft.com"},
{name = "Kate Lytvynets", email = "kalytv@microsoft.com"},
{name = "Kenny Zhang", email = "zhangken@microsoft.com"},
{name = "Mónica Carvajal"},
{name = "Nathan Evans", email = "naevans@microsoft.com"},
{name = "Rodrigo Racanicci", email = "rracanicci@microsoft.com"},
{name = "Sarah Smith", email = "smithsarah@microsoft.com"},
]
license = "MIT"
readme = "README.md"
license-files = ["LICENSE"]
requires-python = ">=3.11,<3.14"
classifiers = [
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
dependencies = [
"graphrag-common==3.1.0",
"graphrag-storage==3.1.0",
]
[project.urls]
Source = "https://github.com/microsoft/graphrag"
[build-system]
requires = ["hatchling>=1.27.0,<2.0.0"]
build-backend = "hatchling.build"