chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:30:44 +08:00
commit bcbd1bdb22
5748 changed files with 562488 additions and 0 deletions
+31
View File
@@ -0,0 +1,31 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.base import BaseResource
from mirage.resource.disk import DiskResource
from mirage.resource.ram import RAMResource
__all__ = [
"BaseResource",
"DiskResource",
"RAMResource",
"RedisResource",
]
def __getattr__(name: str):
if name == "RedisResource":
from mirage.resource.redis import RedisResource
return RedisResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+18
View File
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.aliyun.aliyun import AliyunResource
from mirage.resource.aliyun.config import AliyunConfig
__all__ = ["AliyunConfig", "AliyunResource"]
+32
View File
@@ -0,0 +1,32 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.aliyun.config import AliyunConfig
from mirage.resource.aliyun.prompt import PROMPT
from mirage.resource.s3 import S3Resource
class AliyunResource(S3Resource):
PROMPT: str = PROMPT
def __init__(self, config: AliyunConfig) -> None:
self.aliyun_config = config
super().__init__(config.to_s3_config())
def get_state(self) -> dict:
return self.config_state(self.aliyun_config)
def load_state(self, state: dict) -> None:
pass
+45
View File
@@ -0,0 +1,45 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, ConfigDict, SecretStr
from mirage.resource.s3 import S3Config
class AliyunConfig(BaseModel):
model_config = ConfigDict(frozen=True)
bucket: str
region: str
endpoint_url: str | None = None
access_key_id: SecretStr
secret_access_key: SecretStr
timeout: int = 30
proxy: str | None = None
def resolved_endpoint_url(self) -> str:
if self.endpoint_url:
return self.endpoint_url
return f"https://s3.oss-{self.region}.aliyuncs.com"
def to_s3_config(self) -> S3Config:
return S3Config(
bucket=self.bucket,
region=self.region,
endpoint_url=self.resolved_endpoint_url(),
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
timeout=self.timeout,
proxy=self.proxy,
)
+20
View File
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Alibaba Cloud OSS bucket (S3-compatible).
IMPORTANT: This is a remote mount. Prefer targeted reads over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat.
cat on .parquet/.orc/.feather returns a formatted table."""
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.backblaze.backblaze import BackblazeResource
from mirage.resource.backblaze.config import BackblazeConfig
__all__ = ["BackblazeConfig", "BackblazeResource"]
@@ -0,0 +1,32 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.backblaze.config import BackblazeConfig
from mirage.resource.backblaze.prompt import PROMPT
from mirage.resource.s3 import S3Resource
class BackblazeResource(S3Resource):
PROMPT: str = PROMPT
def __init__(self, config: BackblazeConfig) -> None:
self.backblaze_config = config
super().__init__(config.to_s3_config())
def get_state(self) -> dict:
return self.config_state(self.backblaze_config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,45 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, ConfigDict, SecretStr
from mirage.resource.s3 import S3Config
class BackblazeConfig(BaseModel):
model_config = ConfigDict(frozen=True)
bucket: str
region: str
endpoint_url: str | None = None
access_key_id: SecretStr
secret_access_key: SecretStr
timeout: int = 30
proxy: str | None = None
def resolved_endpoint_url(self) -> str:
if self.endpoint_url:
return self.endpoint_url
return f"https://s3.{self.region}.backblazeb2.com"
def to_s3_config(self) -> S3Config:
return S3Config(
bucket=self.bucket,
region=self.region,
endpoint_url=self.resolved_endpoint_url(),
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
timeout=self.timeout,
proxy=self.proxy,
)
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Backblaze B2 bucket (S3-compatible).
IMPORTANT: This is a remote mount. Prefer targeted reads over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat.
cat on .parquet/.orc/.feather returns a formatted table."""
+129
View File
@@ -0,0 +1,129 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from functools import partial
from typing import Any, Callable
from pydantic import BaseModel
from mirage.accessor.base import Accessor
from mirage.cache.index import (IndexCacheStore, IndexConfig,
RAMIndexCacheStore, RedisIndexConfig)
from mirage.resource.secrets import redacted_config_dump
try:
from mirage.cache.index import RedisIndexCacheStore
except ImportError:
RedisIndexCacheStore = None # type: ignore[misc, assignment]
class BaseResource:
name: str = "base"
caches_reads: bool = False
accessor: Accessor = Accessor()
_ops: dict[str, Callable[..., Any]] = {}
PROMPT: str = ""
WRITE_PROMPT: str = ""
index_ttl: float = 600
# Whether this resource carries enough version information for
# snapshot+replay drift detection. When True, the resource's stat()
# must populate FileStat.fingerprint with a stable per-path marker
# (ETag, md5, commit SHA, etc.) that distinguishes content versions.
# When False (the default), reads are treated as live-only at replay
# time: no fingerprint is recorded at snapshot, no drift check fires
# at load. See docs/home/snapshot.mdx for the contract.
SUPPORTS_SNAPSHOT: bool = False
def __init__(
self,
index: IndexConfig | None = None,
**kwargs,
) -> None:
super().__init__(**kwargs)
self._commands: list = []
self._ops_list: list = []
self.set_index(index)
def set_index(self, config: IndexConfig | None = None) -> None:
cfg = config or IndexConfig(ttl=self.index_ttl)
if isinstance(cfg, RedisIndexConfig):
if RedisIndexCacheStore is None:
raise ImportError(
"RedisIndexConfig requires the 'redis' extra. "
"Install with: pip install mirage-ai[redis]")
self._index = RedisIndexCacheStore(
ttl=cfg.ttl,
url=cfg.url,
key_prefix=cfg.key_prefix,
)
else:
self._index = RAMIndexCacheStore(ttl=cfg.ttl)
@property
def index(self) -> IndexCacheStore:
return self._index
async def resolve_glob(self, paths: list, prefix: str = "") -> list[str]:
raise NotImplementedError
def __getattr__(self, name: str) -> Any:
fn = type(self)._ops.get(name)
if fn is not None:
return partial(fn, self.accessor)
raise AttributeError(
f"'{type(self).__name__}' has no attribute '{name}'")
async def fingerprint(self, path: str) -> str | None:
"""Return current remote fingerprint for freshness comparison.
Args:
path (str): Backend-relative path.
Returns:
str | None: Fingerprint string, or None if always fresh.
"""
return None
def register_op(self, fn) -> None:
for ro in fn._registered_ops:
self._ops_list.append(ro)
def ops_list(self) -> list:
return self._ops_list
def register(self, fn) -> None:
for rc in fn._registered_commands:
self._commands.append(rc)
def commands(self) -> list:
return self._commands
def get_state(self) -> dict:
return {
"type": self.name,
}
def config_state(self, config: BaseModel, **extra: Any) -> dict:
cfg = redacted_config_dump(config)
return {
"type": self.name,
"config": cfg,
**extra,
}
def load_state(self, state: dict) -> None:
pass
+18
View File
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.ceph.ceph import CephResource
from mirage.resource.ceph.config import CephConfig
__all__ = ["CephConfig", "CephResource"]
+32
View File
@@ -0,0 +1,32 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.ceph.config import CephConfig
from mirage.resource.ceph.prompt import PROMPT
from mirage.resource.s3 import S3Resource
class CephResource(S3Resource):
PROMPT: str = PROMPT
def __init__(self, config: CephConfig) -> None:
self.ceph_config = config
super().__init__(config.to_s3_config())
def get_state(self) -> dict:
return self.config_state(self.ceph_config)
def load_state(self, state: dict) -> None:
pass
+42
View File
@@ -0,0 +1,42 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, ConfigDict, SecretStr
from mirage.resource.s3 import S3Config
class CephConfig(BaseModel):
model_config = ConfigDict(frozen=True)
bucket: str
endpoint_url: str
access_key_id: SecretStr
secret_access_key: SecretStr
region: str = "us-east-1"
path_style: bool = True
timeout: int = 30
proxy: str | None = None
def to_s3_config(self) -> S3Config:
return S3Config(
bucket=self.bucket,
region=self.region,
endpoint_url=self.endpoint_url,
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
path_style=self.path_style,
timeout=self.timeout,
proxy=self.proxy,
)
+20
View File
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Ceph Rados Gateway bucket (S3-compatible).
IMPORTANT: This is a remote mount. Prefer targeted reads over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat.
cat on .parquet/.orc/.feather returns a formatted table."""
+10
View File
@@ -0,0 +1,10 @@
from mirage.resource.chroma.config import ChromaConfig
__all__ = ["ChromaConfig", "ChromaResource"]
def __getattr__(name: str):
if name == "ChromaResource":
from mirage.resource.chroma.chroma import ChromaResource
return ChromaResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+54
View File
@@ -0,0 +1,54 @@
from mirage.accessor.chroma import ChromaAccessor
from mirage.commands.builtin.chroma import COMMANDS
from mirage.core.chroma.glob import resolve_glob as _resolve_glob
from mirage.core.chroma.read import read_bytes, read_stream
from mirage.core.chroma.readdir import readdir
from mirage.core.chroma.stat import stat
from mirage.ops.chroma import OPS as CHROMA_VFS_OPS
from mirage.resource.base import BaseResource
from mirage.resource.chroma.config import ChromaConfig
from mirage.resource.chroma.prompt import PROMPT
from mirage.types import ResourceName
_CHROMA_OPS = {
"read_bytes": read_bytes,
"read_stream": read_stream,
"readdir": readdir,
"stat": stat,
}
class ChromaResource(BaseResource):
name: str = ResourceName.CHROMA
caches_reads: bool = False
_ops = _CHROMA_OPS
PROMPT: str = PROMPT
SUPPORTS_SNAPSHOT: bool = False
def __init__(self, config: ChromaConfig) -> None:
super().__init__()
self.config = config
self.accessor = ChromaAccessor(config)
for fn in COMMANDS:
self.register(fn)
for fn in CHROMA_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
return None
def get_state(self) -> dict:
return {
"type": self.name,
"needs_override": True,
"redacted_fields": [],
"config": self.config.model_dump(),
}
def load_state(self, state: dict) -> None:
pass
+20
View File
@@ -0,0 +1,20 @@
from pydantic import BaseModel, ConfigDict, field_validator
class ChromaConfig(BaseModel):
model_config = ConfigDict(frozen=True)
host: str = "localhost"
port: int = 8000
ssl: bool = False
collection_name: str
slug_field: str = "page_slug"
chunk_index_field: str = "chunk_index"
@field_validator("collection_name", "slug_field", "chunk_index_field")
@classmethod
def normalize_non_empty(cls, value: str) -> str:
normalized = value.strip()
if not normalized:
raise ValueError("value cannot be empty")
return normalized
+3
View File
@@ -0,0 +1,3 @@
PROMPT = """Chroma is mounted as a read-only virtual filesystem backed by a ChromaDB collection.
Use cat, head, tail, ls, find, tree, grep, and search to inspect indexed documents. Paths are document slugs from the collection path tree. Writes are not supported."""
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.databricks_volume.config import DatabricksVolumeConfig
__all__ = ["DatabricksVolumeConfig", "DatabricksVolumeResource"]
def __getattr__(name: str):
if name == "DatabricksVolumeResource":
from mirage.resource.databricks_volume.databricks_volume import \
DatabricksVolumeResource
return DatabricksVolumeResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -0,0 +1,52 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import posixpath
from pydantic import BaseModel, ConfigDict, Field, field_validator
class DatabricksVolumeConfig(BaseModel):
model_config = ConfigDict(frozen=True,
populate_by_name=True,
serialize_by_alias=True)
catalog: str
# Public config key is still "schema"
# internal name avoids warning overwriting BaseModel.schema.
schema_name: str = Field(alias="schema")
volume: str
root_path: str = "/"
host: str | None = None
token: str | None = None
profile: str | None = None
timeout: int = 30
@field_validator("catalog", "schema_name", "volume")
@classmethod
def validate_volume_part(cls, value: str) -> str:
if not value or "/" in value:
raise ValueError("must be a non-empty path segment")
return value
@field_validator("root_path")
@classmethod
def normalize_root_path(cls, value: str) -> str:
stripped = value.strip("/")
if not stripped:
return "/"
if any(part == ".." for part in stripped.split("/")):
raise ValueError("root_path must not contain '..' segments")
normalized = posixpath.normpath("/" + stripped)
return "/" if normalized == "/." else normalized
@@ -0,0 +1,103 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from typing import Any
from mirage.accessor.databricks_volume import DatabricksVolumeAccessor
from mirage.commands.builtin.databricks_volume import \
COMMANDS as DATABRICKS_VOLUME_COMMANDS
from mirage.core.databricks_volume.copy import copy
from mirage.core.databricks_volume.create import create
from mirage.core.databricks_volume.exists import exists
from mirage.core.databricks_volume.glob import resolve_glob as _resolve_glob
from mirage.core.databricks_volume.mkdir import mkdir
from mirage.core.databricks_volume.read import read_bytes
from mirage.core.databricks_volume.readdir import readdir
from mirage.core.databricks_volume.rename import rename
from mirage.core.databricks_volume.rm import rm_recursive
from mirage.core.databricks_volume.rmdir import rmdir
from mirage.core.databricks_volume.stat import stat as databricks_stat
from mirage.core.databricks_volume.stream import range_read, read_stream
from mirage.core.databricks_volume.unlink import unlink
from mirage.core.databricks_volume.write import write_bytes
from mirage.ops.databricks_volume import OPS as DATABRICKS_VOLUME_OPS
from mirage.resource.base import BaseResource
from mirage.resource.databricks_volume.config import DatabricksVolumeConfig
from mirage.resource.databricks_volume.prompt import PROMPT
from mirage.types import PathSpec, ResourceName
from mirage.utils.key_prefix import mount_key
_DATABRICKS_VOLUME_OPS = {
"read_bytes": read_bytes,
"readdir": readdir,
"stat": databricks_stat,
"read_stream": read_stream,
"range_read": range_read,
"exists": exists,
"write": write_bytes,
"create": create,
"unlink": unlink,
"mkdir": mkdir,
"rmdir": rmdir,
"copy": copy,
"rename": rename,
"rm_recursive": rm_recursive,
}
class DatabricksVolumeResource(BaseResource):
name: str = ResourceName.DATABRICKS_VOLUME
caches_reads: bool = True
_ops: dict[str, Any] = _DATABRICKS_VOLUME_OPS
PROMPT: str = PROMPT
def __init__(
self,
config: DatabricksVolumeConfig,
client: Any | None = None,
) -> None:
super().__init__()
self.config = config
self.accessor = DatabricksVolumeAccessor(self.config, client)
for fn in DATABRICKS_VOLUME_COMMANDS:
self.register(fn)
for fn in DATABRICKS_VOLUME_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
if prefix:
paths = [
dataclasses.replace(p,
resource_path=mount_key(p.virtual, prefix))
if isinstance(p, PathSpec) else p for p in paths
]
return await _resolve_glob(self.accessor, paths, self._index)
def get_state(self) -> dict:
redacted = ["token"]
cfg = self.config.model_dump()
for field in redacted:
if cfg.get(field) is not None:
cfg[field] = "<REDACTED>"
return {
"type": self.name,
"needs_override": True,
"redacted_fields": redacted,
"config": cfg,
}
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,26 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Databricks Unity Catalog volume filesystem.
IMPORTANT: This is a remote mount - every file/dir access is an API round-trip.
Prefer targeted reads (ls, head, grep, rg, find with a path/glob) over full
scans; recursive ops (rm -r, cp -r, tree, grep -r) over a large tree are slow.
Create parent dirs before writing: `mkdir -p /path && cmd > /path/out`.
mv/cp are non-atomic full copies (no server-side rename) - avoid on large files.
sed -i is not supported; transform and redirect to a new file instead.
Read/analyze: ls, cat, head, tail, grep, rg, find, tree, stat, wc, sort, uniq,
cut, nl, tr, sed, awk, jq, diff.
Write: touch, rm, rm -r, mkdir, mkdir -p, cp, mv, and >/>> redirects."""
+17
View File
@@ -0,0 +1,17 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.dev.dev import DevResource
__all__ = ["DevResource"]
+71
View File
@@ -0,0 +1,71 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.ram import RAMAccessor
from mirage.commands.builtin.ram import COMMANDS
from mirage.ops.ram import OPS as RAM_OPS
from mirage.resource.base import BaseResource
from mirage.types import ResourceName
_DEV_NAMES = frozenset({"null", "zero"})
_ZERO_CHUNK_SIZE = 1 << 20
class _DevFiles:
def __contains__(self, key: str) -> bool:
name = key.strip("/")
return name in _DEV_NAMES
def __getitem__(self, key: str) -> bytes:
name = key.strip("/")
if name == "null":
return b""
if name == "zero":
return b"\x00" * _ZERO_CHUNK_SIZE
raise KeyError(key)
def __setitem__(self, key: str, value: bytes) -> None:
pass
def pop(self, key: str, default=None):
return default
def __iter__(self):
return iter(["/null", "/zero"])
def keys(self):
return ["/null", "/zero"]
class DevStore:
def __init__(self) -> None:
self.files: _DevFiles = _DevFiles()
self.dirs: set[str] = {"/"}
self.modified: dict[str, str] = {}
class DevResource(BaseResource):
name: str = ResourceName.RAM
def __init__(self) -> None:
super().__init__()
self._store = DevStore()
self.accessor = RAMAccessor(self._store)
for fn in COMMANDS:
self.register(fn)
for fn in RAM_OPS:
self.register_op(fn)
+10
View File
@@ -0,0 +1,10 @@
from mirage.resource.dify.config import DifyConfig
__all__ = ["DifyConfig", "DifyResource"]
def __getattr__(name: str):
if name == "DifyResource":
from mirage.resource.dify.dify import DifyResource
return DifyResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+23
View File
@@ -0,0 +1,23 @@
from pydantic import BaseModel, ConfigDict, field_validator
class DifyConfig(BaseModel):
model_config = ConfigDict(frozen=True)
api_key: str
base_url: str
dataset_id: str
slug_metadata_name: str = "slug"
@field_validator("base_url")
@classmethod
def normalize_base_url(cls, value: str) -> str:
return value.rstrip("/")
@field_validator("slug_metadata_name")
@classmethod
def normalize_slug_metadata_name(cls, value: str) -> str:
name = value.strip()
if not name:
raise ValueError("slug_metadata_name cannot be empty")
return name
+58
View File
@@ -0,0 +1,58 @@
from mirage.accessor.dify import DifyAccessor
from mirage.commands.builtin.dify import COMMANDS
from mirage.core.dify.glob import resolve_glob as _resolve_glob
from mirage.core.dify.read import read_bytes, read_stream
from mirage.core.dify.readdir import readdir
from mirage.core.dify.stat import stat
from mirage.ops.dify import OPS as DIFY_VFS_OPS
from mirage.resource.base import BaseResource
from mirage.resource.dify.config import DifyConfig
from mirage.resource.dify.prompt import PROMPT
from mirage.types import ResourceName
_DIFY_OPS = {
"read_bytes": read_bytes,
"read_stream": read_stream,
"readdir": readdir,
"stat": stat,
}
class DifyResource(BaseResource):
name: str = ResourceName.DIFY
caches_reads: bool = True
_ops = _DIFY_OPS
PROMPT: str = PROMPT
SUPPORTS_SNAPSHOT: bool = False
def __init__(self, config: DifyConfig) -> None:
super().__init__()
self.config = config
self.accessor = DifyAccessor(config)
for fn in COMMANDS:
self.register(fn)
for fn in DIFY_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
return None
def get_state(self) -> dict:
redacted = ["api_key"]
config = self.config.model_dump()
if config.get("api_key") is not None:
config["api_key"] = "<REDACTED>"
return {
"type": self.name,
"needs_override": True,
"redacted_fields": redacted,
"config": config,
}
def load_state(self, state: dict) -> None:
pass
+19
View File
@@ -0,0 +1,19 @@
PROMPT = """This is a Dify Knowledge Base mounted as a read-only filesystem.
All files are plain text assembled from Dify document segments, regardless of
their original extension. You can use cat, head, tail, grep, find, ls, wc, and
search on these files.
For semantic or relevance-based queries, use:
search "<query>" [path ...] [--method semantic|fulltext|hybrid|keyword] [--top-k N] [--threshold F]
Use grep for exact pattern or regex matching; use search when meaning matters.
Search can target multiple files, folders, or globs. Each result is returned as
an absolute workspace path, optionally followed by the Dify relevance score,
then the matched chunk content. Multiple hits from the same document stay as
separate results.
Scoped search requires documents with the configured slug metadata field
(default: slug) or Dify Built-in Fields enabled for name-based documents.
Otherwise, scoped searches against name-based paths may return empty results."""
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.digitalocean.config import DigitalOceanConfig
from mirage.resource.digitalocean.digitalocean import DigitalOceanResource
__all__ = ["DigitalOceanConfig", "DigitalOceanResource"]
@@ -0,0 +1,45 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, ConfigDict, SecretStr
from mirage.resource.s3 import S3Config
class DigitalOceanConfig(BaseModel):
model_config = ConfigDict(frozen=True)
bucket: str
region: str
endpoint_url: str | None = None
access_key_id: SecretStr
secret_access_key: SecretStr
timeout: int = 30
proxy: str | None = None
def resolved_endpoint_url(self) -> str:
if self.endpoint_url:
return self.endpoint_url
return f"https://{self.region}.digitaloceanspaces.com"
def to_s3_config(self) -> S3Config:
return S3Config(
bucket=self.bucket,
region=self.region,
endpoint_url=self.resolved_endpoint_url(),
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
timeout=self.timeout,
proxy=self.proxy,
)
@@ -0,0 +1,32 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.digitalocean.config import DigitalOceanConfig
from mirage.resource.digitalocean.prompt import PROMPT
from mirage.resource.s3 import S3Resource
class DigitalOceanResource(S3Resource):
PROMPT: str = PROMPT
def __init__(self, config: DigitalOceanConfig) -> None:
self.digitalocean_config = config
super().__init__(config.to_s3_config())
def get_state(self) -> dict:
return self.config_state(self.digitalocean_config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote DigitalOcean Spaces bucket (S3-compatible).
IMPORTANT: This is a remote mount. Prefer targeted reads over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat.
cat on .parquet/.orc/.feather returns a formatted table."""
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.discord.config import DiscordConfig
__all__ = ["DiscordConfig", "DiscordResource"]
def __getattr__(name: str):
if name == "DiscordResource":
from mirage.resource.discord.discord import DiscordResource
return DiscordResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, SecretStr
class DiscordConfig(BaseModel):
token: SecretStr
+53
View File
@@ -0,0 +1,53 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.discord import DiscordAccessor
from mirage.core.discord.glob import resolve_glob as _resolve_glob
from mirage.resource.base import BaseResource
from mirage.resource.discord.config import DiscordConfig
from mirage.resource.discord.prompt import PROMPT, WRITE_PROMPT
from mirage.types import ResourceName
class DiscordResource(BaseResource):
name: str = ResourceName.DISCORD
caches_reads: bool = True
PROMPT: str = PROMPT
WRITE_PROMPT: str = WRITE_PROMPT
def __init__(self, config: DiscordConfig) -> None:
super().__init__()
self.config = config
self.accessor = DiscordAccessor(self.config)
from mirage.commands.builtin.discord import COMMANDS
from mirage.ops.discord import OPS as DISCORD_VFS_OPS
for fn in COMMANDS:
self.register(fn)
for fn in DISCORD_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+44
View File
@@ -0,0 +1,44 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
<guild-name>__<guild-id>/
channels/
<channel-name>__<channel-id>/
<yyyy-mm-dd>/
chat.jsonl # messages for that date
files/ # attachments shared that day (may be empty)
<name>__<att-id>.<ext> # cat to download bytes from Discord CDN
members/
<username>__<user-id>.json # member profile
Naming: guild, channel, member, attachment names are `<display-name>__<id>`.
The display name keeps the original spelling from Discord (spaces,
apostrophes, emoji); only `/` is replaced with `` (U+2215) so paths
don't break. Quote names containing spaces in shell commands. Always
ls the parent dir first to discover exact entry names (they include
IDs).
Messages are JSONL; use jq to extract fields like .content, .author.username,
.attachments.
grep / rg at channel or guild scope uses Discord's `/messages/search` API
and ONLY searches message text. It does NOT index attachment content
(unlike Slack's `search.files`). To search inside text attachments
(.txt, .md, .log), target the date dir, files/ dir, or specific blob path
so grep falls back to per-file scan via CDN download. Binary attachments
(JPG, PDF) will produce noise."""
WRITE_PROMPT = """\
Write commands:
discord-send-message <channel-path> "message"
discord-add-reaction <message-path> "emoji" """
+17
View File
@@ -0,0 +1,17 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.disk.disk import DiskResource
__all__ = ["DiskResource"]
+107
View File
@@ -0,0 +1,107 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from pathlib import Path
from mirage.accessor.disk import DiskAccessor
from mirage.commands.builtin.disk import COMMANDS as DISK_COMMANDS
from mirage.core.disk.append import append_bytes
from mirage.core.disk.copy import copy
from mirage.core.disk.create import create
from mirage.core.disk.glob import resolve_glob as _resolve_glob
from mirage.core.disk.mkdir import mkdir
from mirage.core.disk.read import read_bytes
from mirage.core.disk.readdir import readdir
from mirage.core.disk.rename import rename
from mirage.core.disk.rm import rm_r
from mirage.core.disk.rmdir import rmdir
from mirage.core.disk.stat import stat as disk_stat
from mirage.core.disk.stream import read_stream
from mirage.core.disk.truncate import truncate
from mirage.core.disk.unlink import unlink
from mirage.core.disk.write import write_bytes
from mirage.ops.disk import OPS as DISK_OPS
from mirage.resource.base import BaseResource
from mirage.resource.disk.prompt import PROMPT
from mirage.types import PathSpec, ResourceName
from mirage.utils.key_prefix import mount_key
_DISK_OPS = {
"read_bytes": read_bytes,
"write": write_bytes,
"readdir": readdir,
"stat": disk_stat,
"unlink": unlink,
"rmdir": rmdir,
"copy": copy,
"rename": rename,
"mkdir": mkdir,
"read_stream": read_stream,
"rm_recursive": rm_r,
"create": create,
"truncate": truncate,
"append": append_bytes,
}
class DiskResource(BaseResource):
name: str = ResourceName.DISK
index_ttl: float = 60
_ops: dict = _DISK_OPS
PROMPT: str = PROMPT
def __init__(self, root: str) -> None:
super().__init__()
self.root = Path(root).resolve()
self.accessor = DiskAccessor(self.root)
for fn in DISK_COMMANDS:
self.register(fn)
for fn in DISK_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
if prefix:
paths = [
dataclasses.replace(p,
resource_path=mount_key(p.virtual, prefix))
if isinstance(p, PathSpec) else p for p in paths
]
return await _resolve_glob(self.accessor, paths, self._index)
async def fingerprint(self, path: str) -> str | None:
try:
remote = await disk_stat(self.accessor, path)
return remote.modified
except FileNotFoundError:
return None
def get_state(self) -> dict:
files: dict[str, bytes] = {}
for p in self.root.rglob("*"):
if p.is_file():
rel = p.relative_to(self.root).as_posix()
files[rel] = p.read_bytes()
return {
"type": self.name,
"files": files,
}
def load_state(self, state: dict) -> None:
files = state.get("files", {})
for rel, data in files.items():
target = self.root / rel
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(data)
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Local disk filesystem. Mirrors the configured root directory.
Standard commands: ls, cat, head, tail, grep, wc, find, tree, mkdir, \
touch, cp, mv, rm, tee."""
+25
View File
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.email.config import EmailConfig
__all__ = ["EmailConfig", "EmailResource"]
def __getattr__(name: str):
if name == "EmailResource":
from mirage.resource.email.email import EmailResource
return EmailResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+26
View File
@@ -0,0 +1,26 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, SecretStr
class EmailConfig(BaseModel):
imap_host: str
imap_port: int = 993
smtp_host: str
smtp_port: int = 587
username: str
password: SecretStr
use_ssl: bool = True
max_messages: int = 200
+53
View File
@@ -0,0 +1,53 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.email import EmailAccessor
from mirage.core.email.glob import resolve_glob as _resolve_glob
from mirage.resource.base import BaseResource
from mirage.resource.email.config import EmailConfig
from mirage.resource.email.prompt import PROMPT, WRITE_PROMPT
from mirage.types import ResourceName
class EmailResource(BaseResource):
name: str = ResourceName.EMAIL
caches_reads: bool = True
PROMPT: str = PROMPT
WRITE_PROMPT: str = WRITE_PROMPT
def __init__(self, config: EmailConfig) -> None:
super().__init__()
self.config = config
self.accessor = EmailAccessor(config)
from mirage.commands.builtin.email import COMMANDS
from mirage.ops.email import OPS
for fn in COMMANDS:
self.register(fn)
for fn in OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+30
View File
@@ -0,0 +1,30 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
<folder>/
<yyyy-mm-dd>/
<subject>__<uid>.email.json
<subject>__<uid>/ # if attachments exist
<attachment-filename>
Folders include: INBOX, Sent, Drafts, etc. cat shows email as JSON.
<subject> is sanitized — don't construct it; ls the date dir."""
WRITE_PROMPT = """\
Write commands:
email-send "to@email.com" "subject" "body"
email-reply <email-path> "reply body"
email-forward <email-path> "to@email.com" """
+18
View File
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gcs.config import GCSConfig
from mirage.resource.gcs.gcs import GCSResource
__all__ = ["GCSConfig", "GCSResource"]
+42
View File
@@ -0,0 +1,42 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, ConfigDict, SecretStr
from mirage.resource.s3 import S3Config
GCS_ENDPOINT = "https://storage.googleapis.com"
class GCSConfig(BaseModel):
model_config = ConfigDict(frozen=True)
bucket: str
access_key_id: SecretStr
secret_access_key: SecretStr
endpoint_url: str = GCS_ENDPOINT
region: str = "auto"
timeout: int = 30
proxy: str | None = None
def to_s3_config(self) -> S3Config:
return S3Config(
bucket=self.bucket,
region=self.region,
endpoint_url=self.endpoint_url,
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
timeout=self.timeout,
proxy=self.proxy,
)
+32
View File
@@ -0,0 +1,32 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gcs.config import GCSConfig
from mirage.resource.gcs.prompt import PROMPT
from mirage.resource.s3 import S3Resource
class GCSResource(S3Resource):
PROMPT: str = PROMPT
def __init__(self, config: GCSConfig) -> None:
self.gcs_config = config
super().__init__(config.to_s3_config())
def get_state(self) -> dict:
return self.config_state(self.gcs_config)
def load_state(self, state: dict) -> None:
pass
+20
View File
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Google Cloud Storage bucket.
IMPORTANT: This is a remote mount. Prefer targeted reads over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat.
cat on .parquet/.orc/.feather returns a formatted table."""
+25
View File
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gdocs.config import GDocsConfig
from mirage.resource.gdocs.doc_entry import DocEntry
__all__ = ["GDocsConfig", "DocEntry", "GDocsResource"]
def __getattr__(name: str):
if name == "GDocsResource":
from mirage.resource.gdocs.gdocs import GDocsResource
return GDocsResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.core.google.config import GoogleConfig
class GDocsConfig(GoogleConfig):
pass
+69
View File
@@ -0,0 +1,69 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import re
from dataclasses import dataclass
TITLE_MAX_CHARS = 100
UNSAFE_CHARS = re.compile(r'[^\w\s\-.]')
MULTI_UNDERSCORE = re.compile(r'_+')
@dataclass
class DocEntry:
id: str
name: str
modified_time: str
created_time: str
owner: str | None
owned_by_me: bool
can_edit: bool
filename: str
def sanitize_title(title: str) -> str:
"""Sanitize a document title for use in filenames.
Args:
title (str): raw document title.
Returns:
str: sanitized title, max 100 chars.
"""
if not title.strip():
return "Untitled"
cleaned = UNSAFE_CHARS.sub("_", title)
cleaned = cleaned.replace(" ", "_")
cleaned = MULTI_UNDERSCORE.sub("_", cleaned)
cleaned = cleaned.strip("_")
if len(cleaned) > TITLE_MAX_CHARS:
cleaned = cleaned[:TITLE_MAX_CHARS - 3] + "..."
return cleaned
def make_filename(title: str, doc_id: str, modified_time: str = "") -> str:
"""Build a filename from title, doc ID, and modified date.
Args:
title (str): raw document title.
doc_id (str): Google Docs document ID.
modified_time (str): ISO 8601 timestamp.
Returns:
str: filename in format "YYYY-MM-DD_Sanitized_Title__docid.json".
"""
date_prefix = modified_time[:10] if len(modified_time) >= 10 else ""
if date_prefix:
return f"{date_prefix}_{sanitize_title(title)}__{doc_id}.gdoc.json"
return f"{sanitize_title(title)}__{doc_id}.gdoc.json"
+55
View File
@@ -0,0 +1,55 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.gdocs import GDocsAccessor
from mirage.core.gdocs.glob import resolve_glob as _resolve_glob
from mirage.core.google._client import TokenManager
from mirage.resource.base import BaseResource
from mirage.resource.gdocs.config import GDocsConfig
from mirage.resource.gdocs.prompt import PROMPT, WRITE_PROMPT
from mirage.types import ResourceName
class GDocsResource(BaseResource):
name: str = ResourceName.GDOCS
caches_reads: bool = True
PROMPT: str = PROMPT
WRITE_PROMPT: str = WRITE_PROMPT
def __init__(self, config: GDocsConfig) -> None:
super().__init__()
self.config = config
self._token_manager = TokenManager(config)
self.accessor = GDocsAccessor(self.config, self._token_manager)
from mirage.commands.builtin.gdocs import COMMANDS
from mirage.ops.gdocs import OPS as GDOCS_VFS_OPS
for fn in COMMANDS:
self.register(fn)
for fn in GDOCS_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+84
View File
@@ -0,0 +1,84 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
owned/
<date>_<title>__<doc-id>.gdoc.json
shared/
<date>_<title>__<doc-id>.gdoc.json
Filename: <YYYY-MM-DD>_<title>__<doc-id>.gdoc.json
<YYYY-MM-DD> modifiedTime, used for date-glob (e.g. 2026-05-*)
<title> sanitized: spaces->_, non-[A-Za-z0-9_.-]->_, <=100 chars
<doc-id> Google Docs document ID
Buckets:
owned/ docs you created
shared/ docs shared with you by others
- does NOT include docs you own and shared with others;
those are still in owned/.
gdoc.json structure (matches the Google Docs API documents.get response):
{{
"documentId": "...",
"title": "...",
"body": {{
"content": [
{{ # one element per block
"paragraph": {{
"elements": [
{{ "textRun": {{ "content": "the actual text\\n",
"textStyle": {{...}} }} }}
],
"paragraphStyle": {{...}}
}}
}},
{{ "table": {{...}} }},
{{ "sectionBreak": {{...}} }}
]
}},
"documentStyle": {{...}},
"namedStyles": {{...}},
"revisionId": "...",
"suggestionsViewMode": "..."
}}
Useful jq paths:
.title
.body.content[].paragraph.elements[].textRun.content # all text
[.body.content[] | select(.table)] | length # table count
.revisionId"""
WRITE_PROMPT = """\
Write commands:
gws-docs-write --document <doc-id> --text "text to append"
gws-docs-documents-create --json '{"title": "My Doc"}'
gws-docs-documents-batchUpdate \\
--params '{"documentId": "<id>"}' \\
--json '{"requests": [{"insertText": {"location": {"index": 1}, "text": "Hello"}}]}'
Newline gotcha: bash double-quoted "...\\n..." is NOT a newline; the
literal characters \\ + n end up in the doc. Either:
--text $'line1\\nline2' # ANSI-C quoting
--text "$(printf 'line1\\nline2')" # printf interprets
gws-docs-documents-batchUpdate --json '{...}' # JSON handles escapes
Delete:
rm {prefix}/owned/<file>.gdoc.json # permanent delete from Drive
rm -f <path> # ignore if missing
Only operates on .gdoc.json files; owned/ and shared/ themselves
cannot be removed."""
+24
View File
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gdrive.config import GoogleDriveConfig
__all__ = ["GoogleDriveConfig", "GoogleDriveResource"]
def __getattr__(name: str):
if name == "GoogleDriveResource":
from mirage.resource.gdrive.gdrive import GoogleDriveResource
return GoogleDriveResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.core.google.config import GoogleConfig
class GoogleDriveConfig(GoogleConfig):
pass
+54
View File
@@ -0,0 +1,54 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.gdrive import GDriveAccessor
from mirage.core.gdrive.glob import resolve_glob as _resolve_glob
from mirage.core.google._client import TokenManager
from mirage.resource.base import BaseResource
from mirage.resource.gdrive.config import GoogleDriveConfig
from mirage.resource.gdrive.prompt import PROMPT
from mirage.types import ResourceName
class GoogleDriveResource(BaseResource):
name: str = ResourceName.GDRIVE
caches_reads: bool = True
PROMPT: str = PROMPT
def __init__(self, config: GoogleDriveConfig) -> None:
super().__init__()
self.config = config
self._token_manager = TokenManager(config)
self.accessor = GDriveAccessor(self.config, self._token_manager)
from mirage.commands.builtin.gdrive import COMMANDS
from mirage.ops.gdrive import OPS as GDRIVE_VFS_OPS
for fn in COMMANDS:
self.register(fn)
for fn in GDRIVE_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+31
View File
@@ -0,0 +1,31 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Mirrors Google Drive folder hierarchy. May contain:
<name>.gdoc.json Google Docs (cat returns gdoc.json - see /gdocs prompt)
<name>.gsheet.json Google Sheets (cat returns gsheet.json - see /gsheets prompt)
<name>.gslide.json Google Slides (cat returns gslide.json - see /gslides prompt)
<other-files> PDFs, images, etc. - cat returns raw bytes
No owned/ vs shared/ split here: gdrive shows the user's full Drive view,
including files shared with the user that have been added to My Drive.
Shared drives visible to the user are included as top-level directories.
IMPORTANT: This is a remote mount. Prefer targeted reads over full scans.
Date-prefixed globs (2026-05-*) push to a Drive modifiedTime range query.
All gws-* commands from /gdocs, /gsheets, /gslides also work here
(the per-service prompts have flag examples)."""
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.core.github.config import GitHubConfig
from mirage.core.github.tree_entry import TreeEntry
from mirage.resource.github.github import GitHubResource
__all__ = ["GitHubConfig", "GitHubResource", "TreeEntry"]
+119
View File
@@ -0,0 +1,119 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from collections import defaultdict
from datetime import datetime, timedelta, timezone
from mirage.accessor.github import GitHubAccessor
from mirage.cache.index import IndexEntry
from mirage.core.github.config import GitHubConfig
from mirage.core.github.glob import resolve_glob as _resolve_glob
from mirage.core.github.repo import fetch_default_branch_sync
from mirage.core.github.tree import fetch_tree_sync
from mirage.core.github.tree_entry import TreeEntry
from mirage.resource.base import BaseResource
from mirage.resource.github.prompt import PROMPT
from mirage.types import ResourceName
class GitHubResource(BaseResource):
name: str = ResourceName.GITHUB
caches_reads: bool = True
PROMPT: str = PROMPT
def __init__(
self,
config: GitHubConfig,
owner: str | None = None,
repo: str | None = None,
ref: str | None = None,
) -> None:
super().__init__()
owner = owner or config.owner
repo = repo or config.repo
ref = ref or config.ref
if owner is None or repo is None:
raise ValueError(
"GitHubResource requires owner and repo, either as "
"constructor kwargs or in GitHubConfig")
default_branch = fetch_default_branch_sync(config, owner, repo)
tree, truncated = fetch_tree_sync(config, owner, repo, ref)
self.accessor = GitHubAccessor(config,
owner,
repo,
ref,
default_branch,
truncated=truncated)
self._populate_index_sync(tree)
from mirage.commands.builtin.github import COMMANDS as _github_cmds
from mirage.ops.github import OPS as _github_vfs_ops
for fn in _github_cmds:
self.register(fn)
for fn in _github_vfs_ops:
self.register_op(fn)
def _populate_index_sync(self, tree: dict[str, TreeEntry]) -> None:
dirs: dict[str, list[tuple[str, IndexEntry]]] = defaultdict(list)
for path, entry in tree.items():
parts = path.rsplit("/", 1)
if len(parts) == 2:
parent, name = "/" + parts[0], parts[1]
else:
parent, name = "/", parts[0]
resource_type = "folder" if entry.type == "tree" else "file"
idx_entry = IndexEntry(
id=entry.sha,
name=name,
resource_type=resource_type,
size=entry.size,
)
dirs[parent].append((name, idx_entry))
for parent, entries in dirs.items():
self._index._entries.update({
("/" + parent.strip("/") + "/" + name).replace("//", "/"):
e
for name, e in entries
})
child_keys = sorted(
("/" + parent.strip("/") + "/" + name).replace("//", "/")
for name, _ in entries)
self._index._children[parent] = child_keys
self._index._expiry[parent] = (datetime.now(timezone.utc) +
timedelta(days=365))
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, self._index)
@property
def is_default_branch(self) -> bool:
return self.accessor.ref == self.accessor.default_branch
async def fingerprint(self, path: str) -> str | None:
result = await self._index.get("/" + path.strip("/"))
return result.entry.id if result.entry else None
def get_state(self) -> dict:
return self.config_state(
self.accessor.config,
owner=self.accessor.owner,
repo=self.accessor.repo,
ref=self.accessor.ref,
default_branch=self.accessor.default_branch,
truncated=self.accessor.truncated,
)
def load_state(self, state: dict) -> None:
pass
+18
View File
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Mirrors the GitHub repository file tree.
Standard commands: ls, cat, head, tail, grep, rg, wc, find, tree."""
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.github_ci.config import GitHubCIConfig
__all__ = ["GitHubCIConfig", "GitHubCIResource"]
def __getattr__(name: str):
if name == "GitHubCIResource":
from mirage.resource.github_ci.github_ci import GitHubCIResource
return GitHubCIResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -0,0 +1,23 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, SecretStr
class GitHubCIConfig(BaseModel):
token: SecretStr
owner: str
repo: str
days: int = 30
max_runs: int = 300
@@ -0,0 +1,52 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.github_ci import GitHubCIAccessor
from mirage.core.github_ci.glob import resolve_glob as _resolve_glob
from mirage.resource.base import BaseResource
from mirage.resource.github_ci.config import GitHubCIConfig
from mirage.resource.github_ci.prompt import PROMPT
from mirage.types import ResourceName
class GitHubCIResource(BaseResource):
name: str = ResourceName.GITHUB_CI
caches_reads: bool = True
PROMPT: str = PROMPT
def __init__(self, config: GitHubCIConfig) -> None:
super().__init__()
self.config = config
self.accessor = GitHubCIAccessor(self.config)
from mirage.commands.builtin.github_ci import COMMANDS
from mirage.ops.github_ci import OPS
for fn in COMMANDS:
self.register(fn)
for fn in OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,27 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
workflows/
<workflow-name>_<workflow-id>.json
runs/
<workflow-name>_<run-id>/
run.json
jobs/
<job-name>_<job-id>.json
<job-name>_<job-id>.log
annotations.jsonl
artifacts/
<artifact-name>_<artifact-id>.zip"""
+24
View File
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gmail.config import GmailConfig
__all__ = ["GmailConfig", "GmailResource"]
def __getattr__(name: str):
if name == "GmailResource":
from mirage.resource.gmail.gmail import GmailResource
return GmailResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.core.google.config import GoogleConfig
class GmailConfig(GoogleConfig):
pass
+55
View File
@@ -0,0 +1,55 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.gmail import GmailAccessor
from mirage.core.gmail.glob import resolve_glob as _resolve_glob
from mirage.core.google._client import TokenManager
from mirage.resource.base import BaseResource
from mirage.resource.gmail.config import GmailConfig
from mirage.resource.gmail.prompt import PROMPT, WRITE_PROMPT
from mirage.types import ResourceName
class GmailResource(BaseResource):
name: str = ResourceName.GMAIL
caches_reads: bool = True
PROMPT: str = PROMPT
WRITE_PROMPT: str = WRITE_PROMPT
def __init__(self, config: GmailConfig) -> None:
super().__init__()
self.config = config
self._token_manager = TokenManager(config)
self.accessor = GmailAccessor(self.config, self._token_manager)
from mirage.commands.builtin.gmail import COMMANDS
from mirage.ops.gmail import OPS as GMAIL_VFS_OPS
for fn in COMMANDS:
self.register(fn)
for fn in GMAIL_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+98
View File
@@ -0,0 +1,98 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
<label>/
<yyyy-mm-dd>/
<subject>__<message-id>.gmail.json # email JSON file (the email itself)
<subject>__<message-id>/ # attachments dir, only if any
<attachment-filename>
Email body is in the .gmail.json file. The sibling dir holds attachments
only. Read with `cat`/`head`/`jq` on `<path>.gmail.json` (keep the suffix).
Commands: cat, ls, head, tail, nl, wc, stat, find, tree, grep, rg, jq,
basename, dirname, realpath, gws-gmail-read, gws-gmail-triage.
No others (no readFile, etc.).
Path: <label>/<yyyy-mm-dd>/<subject>__<message-id>.gmail.json
<label> Gmail label (INBOX, SENT, DRAFT, IMPORTANT, STARRED,
TRASH, SPAM, or any user label)
<yyyy-mm-dd> date the message was received, used for date narrowing
(ls {prefix}/INBOX/2026-05-03/ pushes after:/before: into the
Gmail query - much cheaper than scanning the whole label)
<subject> sanitized subject (don't construct it; ls the date dir)
<message-id> Gmail message ID
email JSON structure (mirage-processed, NOT the raw Gmail API response):
{{
"id": "...",
"thread_id": "...",
"from": {{ "name": "...", "email": "..." }},
"to": [ {{ "name": "...", "email": "..." }} ],
"cc": [ ... ],
"subject": "...",
"date": "Mon, 3 May 2026 10:00:00 -0700",
"body_text": "decoded plain-text body",
"snippet": "first ~200 chars from Gmail",
"labels": [ "INBOX", "IMPORTANT", ... ],
"attachments": [
{{ "id": "...", "filename": "invoice.pdf",
"mime_type": "application/pdf", "size": 12345 }}
]
}}
Attachments: each message with attachments has a sibling directory at
<subject>__<message-id>/ (same name as the .gmail.json file but without
the extension). Cat a file inside to download bytes. grep -r over a
date dir skips these binary files automatically.
After a grep/rg hit, the result line points to .../<subject>__<id>.gmail.json
— the sibling attachments dir is NOT in grep output. Once you have the
message id (or the .gmail.json path), ls the same path with .gmail.json
stripped to list attachments directly:
ls {prefix}/INBOX/2026-05-03/<subject>__<message-id>/
ENOENT means the message has no attachments.
Useful jq paths:
.subject
.from.email
.body_text
.labels[]
.attachments[] | .filename
Read commands:
gws-gmail-read --id <message-id> # same shape as cat
gws-gmail-triage --query "is:unread" --max 20 # summary list (id, from,
# subject, date, snippet)"""
WRITE_PROMPT = """\
Write commands:
gws-gmail-send --to "to@email.com" --subject "Hi" --body "..."
gws-gmail-reply --message-id <id> --body "..."
gws-gmail-reply-all --message-id <id> --body "..."
gws-gmail-forward --message-id <id> --to "to@email.com"
Body gotcha: bash double-quoted "...\\n..." is NOT a newline.
Use $'line1\\nline2' (ANSI-C quoting) or "$(printf '...\\n...')"
if you need real newlines in the body.
Delete:
rm {prefix}/<label>/<yyyy-mm-dd>/<subject>__<message-id>.gmail.json
# moves the message to Trash
rm -f <path> # ignore if missing
Only operates on .gmail.json files; label/, date/, and the
attachments folder cannot be removed (they are virtual)."""
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gsheets.config import GSheetsConfig
from mirage.resource.gsheets.sheet_entry import SheetEntry
__all__ = ["GSheetsConfig", "SheetEntry", "GSheetsResource"]
def __getattr__(name: str):
if name == "GSheetsResource":
from mirage.resource.gsheets.gsheets import GSheetsResource
return GSheetsResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.core.google.config import GoogleConfig
class GSheetsConfig(GoogleConfig):
pass
+55
View File
@@ -0,0 +1,55 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.gsheets import GSheetsAccessor
from mirage.core.google._client import TokenManager
from mirage.core.gsheets.glob import resolve_glob as _resolve_glob
from mirage.resource.base import BaseResource
from mirage.resource.gsheets.config import GSheetsConfig
from mirage.resource.gsheets.prompt import PROMPT, WRITE_PROMPT
from mirage.types import ResourceName
class GSheetsResource(BaseResource):
name: str = ResourceName.GSHEETS
caches_reads: bool = True
PROMPT: str = PROMPT
WRITE_PROMPT: str = WRITE_PROMPT
def __init__(self, config: GSheetsConfig) -> None:
super().__init__()
self.config = config
self._token_manager = TokenManager(config)
self.accessor = GSheetsAccessor(self.config, self._token_manager)
from mirage.commands.builtin.gsheets import COMMANDS
from mirage.ops.gsheets import OPS as GSHEETS_VFS_OPS
for fn in COMMANDS:
self.register(fn)
for fn in GSHEETS_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+91
View File
@@ -0,0 +1,91 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
owned/
<date>_<title>__<spreadsheet-id>.gsheet.json
shared/
<date>_<title>__<spreadsheet-id>.gsheet.json
Filename: <YYYY-MM-DD>_<title>__<spreadsheet-id>.gsheet.json
<YYYY-MM-DD> modifiedTime, used for date-glob (e.g. 2026-05-*)
<title> sanitized: spaces->_, non-[A-Za-z0-9_.-]->_, <=100 chars
<spreadsheet-id> Google Sheets spreadsheet ID
Buckets:
owned/ sheets you created
shared/ sheets shared with you by others
- does NOT include sheets you own and shared with others;
those are still in owned/.
gsheet.json structure (matches the Google Sheets API spreadsheets.get response):
{{
"spreadsheetId": "...",
"spreadsheetUrl": "...",
"properties": {{ "title": "...", "locale": "...", "timeZone": "..." }},
"sheets": [
{{ # one element per tab
"properties": {{
"sheetId": 0, "title": "...", "index": 0,
"gridProperties": {{ "rowCount": 1000, "columnCount": 26 }}
}},
"data": [
{{
"rowData": [
{{ "values": [
{{ "formattedValue": "...",
"userEnteredValue": {{...}},
"effectiveValue": {{...}} }}
]}} # empty cells are omitted, not nullified
]
}}
]
}}
],
"namedRanges": [...]
}}
Useful jq paths:
.properties.title
.sheets[].properties.title # tab names
.sheets[0].data[0].rowData[].values[].formattedValue # cell strings
.namedRanges[]
Read commands (alternative to cat for range-scoped reads — lighter):
gws-sheets-read --spreadsheet <id> --range Sheet1!A1:C10"""
WRITE_PROMPT = """\
Write commands:
gws-sheets-write \\
--params '{"spreadsheetId": "<id>", "range": "Sheet1!A1:B2", "valueInputOption": "USER_ENTERED"}' \\
--json '{"values": [["Name", "Score"], ["Alice", 42]]}'
gws-sheets-append --spreadsheet <id> --range Sheet1!A1 \\
--values "Bob,37" # comma-separated single row
gws-sheets-append --spreadsheet <id> --range Sheet1!A1 \\
--json-values '[["Bob", 37], ["Carol", 51]]' # multiple rows
gws-sheets-spreadsheets-create \\
--json '{"properties": {"title": "Q2 Budget"}}'
gws-sheets-spreadsheets-batchUpdate \\
--params '{"spreadsheetId": "<id>"}' \\
--json '{"requests": [{"addSheet": {"properties": {"title": "Q3"}}}]}'
Delete:
rm {prefix}/owned/<file>.gsheet.json # permanent delete from Drive
rm -f <path> # ignore if missing
Only operates on .gsheet.json files; owned/ and shared/ themselves
cannot be removed."""
@@ -0,0 +1,69 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import re
from dataclasses import dataclass
TITLE_MAX_CHARS = 100
UNSAFE_CHARS = re.compile(r'[^\w\s\-.]')
MULTI_UNDERSCORE = re.compile(r'_+')
@dataclass
class SheetEntry:
id: str
name: str
modified_time: str
created_time: str
owner: str | None
owned_by_me: bool
can_edit: bool
filename: str
def sanitize_title(title: str) -> str:
"""Sanitize a document title for use in filenames.
Args:
title (str): raw document title.
Returns:
str: sanitized title, max 100 chars.
"""
if not title.strip():
return "Untitled"
cleaned = UNSAFE_CHARS.sub("_", title)
cleaned = cleaned.replace(" ", "_")
cleaned = MULTI_UNDERSCORE.sub("_", cleaned)
cleaned = cleaned.strip("_")
if len(cleaned) > TITLE_MAX_CHARS:
cleaned = cleaned[:TITLE_MAX_CHARS - 3] + "..."
return cleaned
def make_filename(title: str, doc_id: str, modified_time: str = "") -> str:
"""Build a filename from title, doc ID, and modified date.
Args:
title (str): raw document title.
doc_id (str): Google Sheets spreadsheet ID.
modified_time (str): ISO 8601 timestamp.
Returns:
str: filename in format "YYYY-MM-DD_Sanitized_Title__docid.json".
"""
date_prefix = modified_time[:10] if len(modified_time) >= 10 else ""
if date_prefix:
return f"{date_prefix}_{sanitize_title(title)}__{doc_id}.gsheet.json"
return f"{sanitize_title(title)}__{doc_id}.gsheet.json"
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.gslides.config import GSlidesConfig
from mirage.resource.gslides.slide_entry import SlideEntry
__all__ = ["GSlidesConfig", "SlideEntry", "GSlidesResource"]
def __getattr__(name: str):
if name == "GSlidesResource":
from mirage.resource.gslides.gslides import GSlidesResource
return GSlidesResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+19
View File
@@ -0,0 +1,19 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.core.google.config import GoogleConfig
class GSlidesConfig(GoogleConfig):
pass
+55
View File
@@ -0,0 +1,55 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.gslides import GSlidesAccessor
from mirage.commands.builtin.gslides import COMMANDS
from mirage.core.google._client import TokenManager
from mirage.core.gslides.glob import resolve_glob as _resolve_glob
from mirage.ops.gslides import OPS as GSLIDES_VFS_OPS
from mirage.resource.base import BaseResource
from mirage.resource.gslides.config import GSlidesConfig
from mirage.resource.gslides.prompt import PROMPT, WRITE_PROMPT
from mirage.types import ResourceName
class GSlidesResource(BaseResource):
name: str = ResourceName.GSLIDES
caches_reads: bool = True
PROMPT: str = PROMPT
WRITE_PROMPT: str = WRITE_PROMPT
def __init__(self, config: GSlidesConfig) -> None:
super().__init__()
self.config = config
self._token_manager = TokenManager(config)
self.accessor = GSlidesAccessor(self.config, self._token_manager)
for fn in COMMANDS:
self.register(fn)
for fn in GSLIDES_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
lookup = await self._index.get(path)
return lookup.entry.remote_time if lookup.entry else None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+81
View File
@@ -0,0 +1,81 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
owned/
<date>_<title>__<presentation-id>.gslide.json
shared/
<date>_<title>__<presentation-id>.gslide.json
Filename: <YYYY-MM-DD>_<title>__<presentation-id>.gslide.json
<YYYY-MM-DD> modifiedTime, used for date-glob (e.g. 2026-05-*)
<title> sanitized: spaces->_, non-[A-Za-z0-9_.-]->_, <=100 chars
<presentation-id> Google Slides presentation ID
Buckets:
owned/ presentations you created
shared/ presentations shared with you by others
- does NOT include presentations you own and shared with others;
those are still in owned/.
gslide.json structure (matches the Google Slides API presentations.get response):
{{
"presentationId": "...",
"title": "...",
"pageSize": {{ "width": {{...}}, "height": {{...}} }},
"slides": [
{{ # one element per slide
"objectId": "...",
"pageElements": [
{{
"objectId": "...",
"shape": {{
"shapeType": "TEXT_BOX",
"text": {{
"textElements": [
{{ "textRun": {{ "content": "the actual text\\n",
"style": {{...}} }} }}
]
}}
}}
}},
{{ "image": {{...}} }},
{{ "table": {{...}} }}
]
}}
],
"masters": [...],
"layouts": [...]
}}
Useful jq paths:
.title
.slides | length # slide count
.slides[].pageElements[].shape.text.textElements[].textRun.content # all text
.slides[0].objectId"""
WRITE_PROMPT = """\
Write commands:
gws-slides-presentations-create --json '{"title": "My Deck"}'
gws-slides-presentations-batchUpdate \\
--params '{"presentationId": "<id>"}' \\
--json '{"requests": [{"createSlide": {"insertionIndex": 1, "slideLayoutReference": {"predefinedLayout": "BLANK"}}}]}'
Delete:
rm {prefix}/owned/<file>.gslide.json # permanent delete from Drive
rm -f <path> # ignore if missing
Only operates on .gslide.json files; owned/ and shared/ themselves
cannot be removed."""
@@ -0,0 +1,69 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import re
from dataclasses import dataclass
TITLE_MAX_CHARS = 100
UNSAFE_CHARS = re.compile(r'[^\w\s\-.]')
MULTI_UNDERSCORE = re.compile(r'_+')
@dataclass
class SlideEntry:
id: str
name: str
modified_time: str
created_time: str
owner: str | None
owned_by_me: bool
can_edit: bool
filename: str
def sanitize_title(title: str) -> str:
"""Sanitize a document title for use in filenames.
Args:
title (str): raw document title.
Returns:
str: sanitized title, max 100 chars.
"""
if not title.strip():
return "Untitled"
cleaned = UNSAFE_CHARS.sub("_", title)
cleaned = cleaned.replace(" ", "_")
cleaned = MULTI_UNDERSCORE.sub("_", cleaned)
cleaned = cleaned.strip("_")
if len(cleaned) > TITLE_MAX_CHARS:
cleaned = cleaned[:TITLE_MAX_CHARS - 3] + "..."
return cleaned
def make_filename(title: str, doc_id: str, modified_time: str = "") -> str:
"""Build a filename from title, doc ID, and modified date.
Args:
title (str): raw document title.
doc_id (str): Google Slides presentation ID.
modified_time (str): ISO 8601 timestamp.
Returns:
str: filename in format "YYYY-MM-DD_Sanitized_Title__docid.json".
"""
date_prefix = modified_time[:10] if len(modified_time) >= 10 else ""
if date_prefix:
return f"{date_prefix}_{sanitize_title(title)}__{doc_id}.gslide.json"
return f"{sanitize_title(title)}__{doc_id}.gslide.json"
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.hf_buckets.hf_buckets import (HfBucketsConfig,
HfBucketsResource)
__all__ = ["HfBucketsConfig", "HfBucketsResource"]
@@ -0,0 +1,92 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from typing import Any
from mirage.accessor.hf_buckets import HfBucketsAccessor, HfBucketsConfig
from mirage.commands.builtin.hf_buckets import COMMANDS as HF_COMMANDS
from mirage.core.hf_buckets.create import create
from mirage.core.hf_buckets.du import du, du_all
from mirage.core.hf_buckets.exists import exists
from mirage.core.hf_buckets.find import find
from mirage.core.hf_buckets.glob import resolve_glob as _resolve_glob
from mirage.core.hf_buckets.mkdir import mkdir
from mirage.core.hf_buckets.read import read_bytes
from mirage.core.hf_buckets.readdir import readdir
from mirage.core.hf_buckets.stat import stat as hf_stat
from mirage.core.hf_buckets.stream import range_read, read_stream
from mirage.core.hf_buckets.unlink import unlink
from mirage.core.hf_buckets.write import write_bytes
from mirage.ops.hf_buckets import OPS as HF_OPS
from mirage.resource.base import BaseResource
from mirage.resource.hf_buckets.prompt import PROMPT
from mirage.types import PathSpec, ResourceName
from mirage.utils.key_prefix import mount_key
_OPS = {
"read_bytes": read_bytes,
"readdir": readdir,
"stat": hf_stat,
"read_stream": read_stream,
"range_read": range_read,
"du_total": du,
"du_all": du_all,
"exists": exists,
"find_flat": find,
"write": write_bytes,
"create": create,
"unlink": unlink,
"mkdir": mkdir,
}
class HfBucketsResource(BaseResource):
name: str = ResourceName.HF_BUCKETS
caches_reads: bool = True
_ops: dict[str, Any] = _OPS
PROMPT: str = PROMPT
SUPPORTS_SNAPSHOT: bool = True
def __init__(self, config: HfBucketsConfig) -> None:
super().__init__()
self.config = config
self.accessor = HfBucketsAccessor(self.config)
for fn in HF_COMMANDS:
self.register(fn)
for fn in HF_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
if prefix:
paths = [
dataclasses.replace(p,
resource_path=mount_key(p.virtual, prefix))
if isinstance(p, PathSpec) else p for p in paths
]
return await _resolve_glob(self.accessor, paths, self._index)
async def fingerprint(self, path: str) -> str | None:
try:
s = await hf_stat(self.accessor, path)
return s.fingerprint
except FileNotFoundError:
return None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Hugging Face Bucket (read-only). Maps bucket object keys to virtual paths.
IMPORTANT: This is a remote mount. Prefer targeted reads (grep, head) \
over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat."""
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.hf_datasets.hf_datasets import (HfDatasetsConfig,
HfDatasetsResource)
__all__ = ["HfDatasetsConfig", "HfDatasetsResource"]
@@ -0,0 +1,92 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from typing import Any
from mirage.accessor.hf_datasets import HfDatasetsAccessor, HfDatasetsConfig
from mirage.commands.builtin.hf_buckets import COMMANDS as HF_COMMANDS
from mirage.core.hf_buckets.create import create
from mirage.core.hf_buckets.du import du, du_all
from mirage.core.hf_buckets.exists import exists
from mirage.core.hf_buckets.find import find
from mirage.core.hf_buckets.glob import resolve_glob as _resolve_glob
from mirage.core.hf_buckets.mkdir import mkdir
from mirage.core.hf_buckets.read import read_bytes
from mirage.core.hf_buckets.readdir import readdir
from mirage.core.hf_buckets.stat import stat as hf_stat
from mirage.core.hf_buckets.stream import range_read, read_stream
from mirage.core.hf_buckets.unlink import unlink
from mirage.core.hf_buckets.write import write_bytes
from mirage.ops.hf_buckets import OPS as HF_OPS
from mirage.resource.base import BaseResource
from mirage.resource.hf_datasets.prompt import PROMPT
from mirage.types import PathSpec, ResourceName
from mirage.utils.key_prefix import mount_key
_OPS = {
"read_bytes": read_bytes,
"readdir": readdir,
"stat": hf_stat,
"read_stream": read_stream,
"range_read": range_read,
"du_total": du,
"du_all": du_all,
"exists": exists,
"find_flat": find,
"write": write_bytes,
"create": create,
"unlink": unlink,
"mkdir": mkdir,
}
class HfDatasetsResource(BaseResource):
name: str = ResourceName.HF_DATASETS
caches_reads: bool = True
_ops: dict[str, Any] = _OPS
PROMPT: str = PROMPT
SUPPORTS_SNAPSHOT: bool = True
def __init__(self, config: HfDatasetsConfig) -> None:
super().__init__()
self.config = config
self.accessor = HfDatasetsAccessor(self.config)
for fn in HF_COMMANDS:
self.register(fn)
for fn in HF_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
if prefix:
paths = [
dataclasses.replace(p,
resource_path=mount_key(p.virtual, prefix))
if isinstance(p, PathSpec) else p for p in paths
]
return await _resolve_glob(self.accessor, paths, self._index)
async def fingerprint(self, path: str) -> str | None:
try:
s = await hf_stat(self.accessor, path)
return s.fingerprint
except FileNotFoundError:
return None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,21 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Hugging Face Dataset. Maps dataset repo files to virtual paths.
IMPORTANT: This is a remote mount. Prefer targeted reads (grep, head) \
over full scans. Large parquet/safetensors files stream lazily \
via byte-range reads.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat."""
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.hf_models.hf_models import (HfModelsConfig,
HfModelsResource)
__all__ = ["HfModelsConfig", "HfModelsResource"]
@@ -0,0 +1,92 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from typing import Any
from mirage.accessor.hf_models import HfModelsAccessor, HfModelsConfig
from mirage.commands.builtin.hf_buckets import COMMANDS as HF_COMMANDS
from mirage.core.hf_buckets.create import create
from mirage.core.hf_buckets.du import du, du_all
from mirage.core.hf_buckets.exists import exists
from mirage.core.hf_buckets.find import find
from mirage.core.hf_buckets.glob import resolve_glob as _resolve_glob
from mirage.core.hf_buckets.mkdir import mkdir
from mirage.core.hf_buckets.read import read_bytes
from mirage.core.hf_buckets.readdir import readdir
from mirage.core.hf_buckets.stat import stat as hf_stat
from mirage.core.hf_buckets.stream import range_read, read_stream
from mirage.core.hf_buckets.unlink import unlink
from mirage.core.hf_buckets.write import write_bytes
from mirage.ops.hf_buckets import OPS as HF_OPS
from mirage.resource.base import BaseResource
from mirage.resource.hf_models.prompt import PROMPT
from mirage.types import PathSpec, ResourceName
from mirage.utils.key_prefix import mount_key
_OPS = {
"read_bytes": read_bytes,
"readdir": readdir,
"stat": hf_stat,
"read_stream": read_stream,
"range_read": range_read,
"du_total": du,
"du_all": du_all,
"exists": exists,
"find_flat": find,
"write": write_bytes,
"create": create,
"unlink": unlink,
"mkdir": mkdir,
}
class HfModelsResource(BaseResource):
name: str = ResourceName.HF_MODELS
caches_reads: bool = True
_ops: dict[str, Any] = _OPS
PROMPT: str = PROMPT
SUPPORTS_SNAPSHOT: bool = True
def __init__(self, config: HfModelsConfig) -> None:
super().__init__()
self.config = config
self.accessor = HfModelsAccessor(self.config)
for fn in HF_COMMANDS:
self.register(fn)
for fn in HF_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
if prefix:
paths = [
dataclasses.replace(p,
resource_path=mount_key(p.virtual, prefix))
if isinstance(p, PathSpec) else p for p in paths
]
return await _resolve_glob(self.accessor, paths, self._index)
async def fingerprint(self, path: str) -> str | None:
try:
s = await hf_stat(self.accessor, path)
return s.fingerprint
except FileNotFoundError:
return None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,22 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Hugging Face Model. Maps model repo files (weights, configs, \
tokenizer) to virtual paths.
IMPORTANT: This is a remote mount. Prefer config/tokenizer inspection \
over downloading full weights. Safetensors/bin files stream lazily \
via byte-range reads.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat."""
@@ -0,0 +1,18 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.hf_spaces.hf_spaces import (HfSpacesConfig,
HfSpacesResource)
__all__ = ["HfSpacesConfig", "HfSpacesResource"]
@@ -0,0 +1,92 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from typing import Any
from mirage.accessor.hf_spaces import HfSpacesAccessor, HfSpacesConfig
from mirage.commands.builtin.hf_buckets import COMMANDS as HF_COMMANDS
from mirage.core.hf_buckets.create import create
from mirage.core.hf_buckets.du import du, du_all
from mirage.core.hf_buckets.exists import exists
from mirage.core.hf_buckets.find import find
from mirage.core.hf_buckets.glob import resolve_glob as _resolve_glob
from mirage.core.hf_buckets.mkdir import mkdir
from mirage.core.hf_buckets.read import read_bytes
from mirage.core.hf_buckets.readdir import readdir
from mirage.core.hf_buckets.stat import stat as hf_stat
from mirage.core.hf_buckets.stream import range_read, read_stream
from mirage.core.hf_buckets.unlink import unlink
from mirage.core.hf_buckets.write import write_bytes
from mirage.ops.hf_buckets import OPS as HF_OPS
from mirage.resource.base import BaseResource
from mirage.resource.hf_spaces.prompt import PROMPT
from mirage.types import PathSpec, ResourceName
from mirage.utils.key_prefix import mount_key
_OPS = {
"read_bytes": read_bytes,
"readdir": readdir,
"stat": hf_stat,
"read_stream": read_stream,
"range_read": range_read,
"du_total": du,
"du_all": du_all,
"exists": exists,
"find_flat": find,
"write": write_bytes,
"create": create,
"unlink": unlink,
"mkdir": mkdir,
}
class HfSpacesResource(BaseResource):
name: str = ResourceName.HF_SPACES
caches_reads: bool = True
_ops: dict[str, Any] = _OPS
PROMPT: str = PROMPT
SUPPORTS_SNAPSHOT: bool = True
def __init__(self, config: HfSpacesConfig) -> None:
super().__init__()
self.config = config
self.accessor = HfSpacesAccessor(self.config)
for fn in HF_COMMANDS:
self.register(fn)
for fn in HF_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
if prefix:
paths = [
dataclasses.replace(p,
resource_path=mount_key(p.virtual, prefix))
if isinstance(p, PathSpec) else p for p in paths
]
return await _resolve_glob(self.accessor, paths, self._index)
async def fingerprint(self, path: str) -> str | None:
try:
s = await hf_stat(self.accessor, path)
return s.fingerprint
except FileNotFoundError:
return None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
@@ -0,0 +1,21 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
Remote Hugging Face Space. Maps the space's repo files (app code, \
README, requirements) to virtual paths.
IMPORTANT: This is a remote mount. Prefer targeted reads (grep, head) \
over full scans.
Supports: ls, cat, head, tail, grep, rg, wc, find, tree, jq, stat."""
@@ -0,0 +1,17 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.history.history import HISTORY_PREFIX, HistoryViewResource
__all__ = ["HISTORY_PREFIX", "HistoryViewResource"]
+42
View File
@@ -0,0 +1,42 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.history import HistoryAccessor
from mirage.commands.builtin.history import COMMANDS
from mirage.ops.history import OPS
from mirage.resource.base import BaseResource
HISTORY_PREFIX = "/.bash_history"
class HistoryViewResource(BaseResource):
"""Read-only view resource backing the /.bash_history mount.
Renders GNU views from the workspace's hidden recorder on every
read; holds no storage of its own.
Args:
observer (Observer): The workspace's hidden recorder.
"""
name = "history"
def __init__(self, observer) -> None:
super().__init__()
self.observer = observer
self.accessor = HistoryAccessor(observer)
for fn in COMMANDS:
self.register(fn)
for fn in OPS:
self.register_op(fn)
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.lancedb.config import LanceDBConfig
__all__ = ["LanceDBConfig", "LanceDBResource"]
def __getattr__(name: str):
if name == "LanceDBResource":
from mirage.resource.lancedb.lancedb import LanceDBResource
return LanceDBResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+33
View File
@@ -0,0 +1,33 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, SecretStr
class LanceDBConfig(BaseModel):
uri: str
api_key: SecretStr | None = None
region: str = "us-east-1"
host_override: str | None = None
storage_options: dict[str, str] | None = None
table: str | None = None
group_by: list[str] = []
id_column: str = "id"
title_column: str | None = None
blob_column: str | None = None
blob_ext: str = "bin"
text_column: str | None = None
vector_column: str | None = None
search_limit: int = 10
max_rows: int = 1000
+53
View File
@@ -0,0 +1,53 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.lancedb import LanceDBAccessor
from mirage.core.lancedb.glob import resolve_glob as _resolve_glob
from mirage.resource.base import BaseResource
from mirage.resource.lancedb.config import LanceDBConfig
from mirage.resource.lancedb.prompt import PROMPT
from mirage.types import ResourceName
_REMOTE_SCHEMES = ("s3://", "gs://", "az://", "hf://", "db://")
class LanceDBResource(BaseResource):
name: str = ResourceName.LANCEDB
PROMPT: str = PROMPT
def __init__(self, config: LanceDBConfig) -> None:
super().__init__()
self.config = config
self.caches_reads = config.uri.startswith(_REMOTE_SCHEMES)
self.accessor = LanceDBAccessor(self.config)
from mirage.commands.builtin.lancedb import COMMANDS
from mirage.ops.lancedb import OPS as LANCEDB_OPS
for fn in COMMANDS:
self.register(fn)
for fn in LANCEDB_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(self.accessor, paths, index=self._index)
async def fingerprint(self, path: str) -> str | None:
return None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+30
View File
@@ -0,0 +1,30 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
This mount is a LanceDB table exposed as a filesystem.
Layout:
- Directories are the configured group-by columns. Descending narrows a filter,
e.g. `ls Men/Tshirts/Blue` lists rows where gender=Men, articleType=Tshirts,
baseColour=Blue.
- Each matching row appears as two files: `<id>.md` (a readable card with all
attributes) and `<id>.<ext>` (the raw blob/image bytes), when a blob column is
configured.
- Semantic search is a virtual folder named `_search`. Read a query as a path
segment: `ls "_search/red running shoes"` returns the top matches as row
files; `cat "_search/red running shoes/<id>.md"` shows the card with a score.
Use ls/cd/cat/tree/find/grep as usual. Quote queries that contain spaces.\
"""
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.langfuse.config import LangfuseConfig
__all__ = ["LangfuseConfig", "LangfuseResource"]
def __getattr__(name: str):
if name == "LangfuseResource":
from mirage.resource.langfuse.langfuse import LangfuseResource
return LangfuseResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+23
View File
@@ -0,0 +1,23 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from pydantic import BaseModel, SecretStr
class LangfuseConfig(BaseModel):
public_key: str
secret_key: SecretStr
host: str = "https://cloud.langfuse.com"
default_trace_limit: int = 100
default_search_limit: int = 50
@@ -0,0 +1,55 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.accessor.langfuse import LangfuseAccessor
from mirage.core.langfuse.glob import resolve_glob as _resolve_glob
from mirage.resource.base import BaseResource
from mirage.resource.langfuse.config import LangfuseConfig
from mirage.resource.langfuse.prompt import PROMPT
from mirage.types import ResourceName
class LangfuseResource(BaseResource):
name: str = ResourceName.LANGFUSE
caches_reads: bool = True
PROMPT: str = PROMPT
def __init__(self, config: LangfuseConfig) -> None:
super().__init__()
self.config = config
self.accessor = LangfuseAccessor(self.config)
from mirage.commands.builtin.langfuse import COMMANDS
from mirage.ops.langfuse import OPS as LANGFUSE_VFS_OPS
for fn in COMMANDS:
self.register(fn)
for fn in LANGFUSE_VFS_OPS:
self.register_op(fn)
async def resolve_glob(self, paths, prefix: str = ""):
return await _resolve_glob(
self.accessor,
paths,
index=self._index,
)
async def fingerprint(self, path: str) -> str | None:
return None
def get_state(self) -> dict:
return self.config_state(self.config)
def load_state(self, state: dict) -> None:
pass
+29
View File
@@ -0,0 +1,29 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
PROMPT = """\
{prefix}
traces/
<trace-id>.json
sessions/
<session-id>/
<trace-id>.json
prompts/
<prompt-name>/
<version>.json
datasets/
<dataset-name>/
items.jsonl
runs/
<run-name>.jsonl"""
+24
View File
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.resource.linear.config import LinearConfig
__all__ = ["LinearConfig", "LinearResource"]
def __getattr__(name: str):
if name == "LinearResource":
from mirage.resource.linear.linear import LinearResource
return LinearResource
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

Some files were not shown because too many files have changed in this diff Show More