chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:30:44 +08:00
commit bcbd1bdb22
5748 changed files with 562488 additions and 0 deletions
+82
View File
@@ -0,0 +1,82 @@
---
title: Alibaba OSS
icon: /images/aliyun-logo.svg
description: Mount an Alibaba Cloud OSS bucket via its S3-compatible API as a virtual filesystem.
---
The Aliyun resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps an `AliyunConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just derives the right OSS endpoint from
`region`. Uses aioboto3 against Alibaba OSS's S3-compatible API.
The endpoint is computed from `region` as `s3.oss-<region>.aliyuncs.com`
(e.g. `s3.oss-cn-hangzhou.aliyuncs.com`). Note the `s3.` prefix: this is the
S3-compatible host, distinct from the native `oss-<region>.aliyuncs.com`. Pass
`endpoint_url` to override.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.aliyun import AliyunConfig, AliyunResource
config = AliyunConfig(
bucket=os.environ["OSS_BUCKET"],
region=os.environ.get("OSS_REGION", "cn-hangzhou"),
access_key_id=os.environ["OSS_ACCESS_KEY_ID"],
secret_access_key=os.environ["OSS_ACCESS_KEY_SECRET"],
# Optional:
# endpoint_url="https://s3.oss-cn-hangzhou.aliyuncs.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = AliyunResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.aliyun import AliyunConfig, AliyunResource
load_dotenv(".env.development")
config = AliyunConfig(
bucket=os.environ["OSS_BUCKET"],
region=os.environ.get("OSS_REGION", "cn-hangzhou"),
access_key_id=os.environ["OSS_ACCESS_KEY_ID"],
secret_access_key=os.environ["OSS_ACCESS_KEY_SECRET"],
)
resource = AliyunResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("tree /data/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- Aliyun reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [Alibaba OSS Setup](/home/setup/aliyun).
+80
View File
@@ -0,0 +1,80 @@
---
title: Backblaze B2
icon: /images/backblaze-logo.svg
description: Mount a Backblaze B2 bucket via its S3-compatible API as a virtual filesystem.
---
The Backblaze resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `BackblazeConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just derives the right B2 endpoint from
`region`. Uses aioboto3 against Backblaze B2's S3-compatible API.
The endpoint is computed from `region` as `s3.<region>.backblazeb2.com`. Pass
`endpoint_url` to override.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.backblaze import BackblazeConfig, BackblazeResource
config = BackblazeConfig(
bucket=os.environ["B2_BUCKET"],
region=os.environ["B2_REGION"], # e.g. us-west-004
access_key_id=os.environ["B2_ACCESS_KEY_ID"], # B2 application keyID
secret_access_key=os.environ["B2_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://s3.us-west-004.backblazeb2.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = BackblazeResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.backblaze import BackblazeConfig, BackblazeResource
load_dotenv(".env.development")
config = BackblazeConfig(
bucket=os.environ["B2_BUCKET"],
region=os.environ["B2_REGION"],
access_key_id=os.environ["B2_ACCESS_KEY_ID"],
secret_access_key=os.environ["B2_SECRET_ACCESS_KEY"],
)
resource = BackblazeResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("find /data/ -name '*.csv'")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- Backblaze reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [Backblaze B2 Setup](/home/setup/backblaze).
+81
View File
@@ -0,0 +1,81 @@
---
title: Ceph (Rados Gateway)
icon: /images/ceph-logo.svg
description: Mount a self-hosted Ceph Rados Gateway bucket as a virtual filesystem.
---
The Ceph resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `CephConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just points at your Ceph Rados Gateway
`endpoint_url` and uses path-style addressing by default.
Ceph RGW is self-hosted, so `endpoint_url` is **required** (there is no
region-derived host). Uses aioboto3 against the gateway's S3-compatible API.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.ceph import CephConfig, CephResource
config = CephConfig(
bucket=os.environ["CEPH_BUCKET"],
endpoint_url=os.environ["CEPH_ENDPOINT_URL"],
access_key_id=os.environ["CEPH_ACCESS_KEY_ID"],
secret_access_key=os.environ["CEPH_SECRET_ACCESS_KEY"],
# Optional:
# region="us-east-1", # default
# path_style=True, # default
# timeout=30,
# proxy="http://proxy:8080",
)
resource = CephResource(config)
ws = Workspace({"/ceph": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.ceph import CephConfig, CephResource
load_dotenv(".env.development")
config = CephConfig(
bucket=os.environ["CEPH_BUCKET"],
endpoint_url=os.environ["CEPH_ENDPOINT_URL"],
access_key_id=os.environ["CEPH_ACCESS_KEY_ID"],
secret_access_key=os.environ["CEPH_SECRET_ACCESS_KEY"],
)
resource = CephResource(config)
async def main() -> None:
ws = Workspace({"/ceph/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /ceph/")
print(await r.stdout_str())
r = await ws.execute("find /ceph/ -name '*.json'")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- Ceph reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [Ceph Setup](/home/setup/ceph).
+141
View File
@@ -0,0 +1,141 @@
---
title: Chroma
icon: database
description: Mount a ChromaDB collection as a read-only virtual filesystem.
---
The Chroma resource exposes an existing ChromaDB collection as text files
mounted at a prefix such as `/knowledge/`. It is useful when you already have a
chunked knowledge base in Chroma and want agents to use normal filesystem
commands like `ls`, `cat`, `grep`, `find`, and `chroma-query`.
For collection setup, see [Chroma Setup](/python/setup/chroma).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.chroma import ChromaConfig, ChromaResource
config = ChromaConfig(
host=os.environ.get("CHROMA_HOST", "localhost"),
port=int(os.environ.get("CHROMA_PORT", "8000")),
ssl=os.environ.get("CHROMA_SSL", "false").lower() == "true",
collection_name=os.environ["CHROMA_COLLECTION"],
slug_field=os.environ.get("CHROMA_SLUG_FIELD", "page_slug"),
chunk_index_field=os.environ.get("CHROMA_CHUNK_INDEX_FIELD", "chunk_index"),
)
resource = ChromaResource(config=config)
ws = Workspace({"/knowledge/": resource}, mode=MountMode.READ)
```
The resource is read-only and does not support snapshots.
## Filesystem Layout
Chroma paths come from the path tree document stored in the collection with ID
`__path_tree__`. The document body must be a JSON object whose keys are virtual
file paths below the mount prefix.
```json
{
"README.md": {
"size": 1024
},
"guides/quickstart.md": {
"size": 4096,
"updated_at": "2026-01-03T00:00:00Z"
}
}
```
Mounted at `/knowledge/`, this becomes:
```text
/knowledge/
README.md
guides/
quickstart.md
```
Mirage infers folders from path segments and stores the tree in the index cache.
The `size`, `created_at`, and `updated_at` metadata values are used by tree and
listing operations when present.
## Reading Documents
Each file is assembled from Chroma chunk documents whose metadata slug matches
the path. Chunks are sorted by the configured chunk index field and joined with
a single newline.
```bash
ls /knowledge/
tree /knowledge/
find /knowledge/ -type f
cat /knowledge/guides/quickstart.md
head -n 20 /knowledge/guides/quickstart.md
tail -n 20 /knowledge/guides/quickstart.md
grep -in "billing" /knowledge/
```
## Exact Search with Grep
`grep` uses Chroma document filtering as a coarse prefilter when possible, then
applies Mirage's grep matching over assembled file text. Scoped paths restrict
the candidate files before querying Chroma.
```bash
grep "refund" /knowledge/policies/
grep -i "quickstart" /knowledge/
grep -n "API key" /knowledge/guides/quickstart.md
```
## Vector Search
The `chroma-query` command calls Chroma's native vector query API. Use it when
semantic similarity matters more than exact text matching.
```bash
chroma-query "how do I get started" /knowledge/
chroma-query --top-k 5 "billing policy" /knowledge/policies/
chroma-query --top-k 3 "API authentication" /knowledge/guides/quickstart.md
```
Results are emitted one hit per line:
```text
/knowledge/guides/quickstart.md 0.82 Use an API key to authenticate requests.
```
Columns are path, score, and chunk text. The score is derived from Chroma's
returned distance as `1 - distance`.
## Cache
The resource uses Mirage's index cache for the virtual tree. The first
directory listing or path resolution fetches `__path_tree__`; later `ls`,
`find`, `tree`, and path resolution reuse the cached tree until the index cache
expires or the workspace is recreated.
File content is still read from Chroma when commands materialize document text.
## Examples
Runnable examples live under `examples/python/chroma/`:
- [`chroma.py`](https://github.com/StruktoAI/mirage/blob/main/examples/python/chroma/chroma.py) — command workflow with `ls`, `tree`, `find`, `cat`, `head`, `tail`, `grep`, and `chroma-query`.
- [`chroma_vfs.py`](https://github.com/StruktoAI/mirage/blob/main/examples/python/chroma/chroma_vfs.py) — in-process VFS workflow with `os.listdir()`, `open()`, and `os.path.*`.
## Shell Commands
| Command | Notes |
| ------- | ----- |
| `ls` | List folders and files from the path tree |
| `tree` | Print the mounted path tree |
| `find` | Search the virtual tree by name, type, depth, and size |
| `cat` | Read full file text assembled from Chroma chunks |
| `head` / `tail` | Read the first or last lines/bytes |
| `grep` | Exact or regex matching over assembled file text |
| `chroma-query` | Vector retrieval through Chroma |
+211
View File
@@ -0,0 +1,211 @@
---
title: Databricks Volume
icon: folder-open
description: Mount a Databricks Unity Catalog volume as a filesystem.
---
`DatabricksVolumeResource` exposes files from a Unity Catalog volume through
Mirage's standard filesystem interface. Agents can list, stat, read, stream,
glob, and write files under the configured volume root.
For auth and environment setup, see [Databricks Volume Setup](/python/setup/databricks).
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.databricks_volume import (
DatabricksVolumeConfig,
DatabricksVolumeResource,
)
resource = DatabricksVolumeResource(DatabricksVolumeConfig(
catalog="main",
schema="default",
volume="agent_files",
root_path="/reports",
))
ws = Workspace({"/dbx": resource}, mode=MountMode.READ)
```
| Field | Default | Notes |
| --- | --- | --- |
| `catalog` | required | Unity Catalog catalog name. |
| `schema` | required | Unity Catalog schema name. |
| `volume` | required | Unity Catalog volume name. |
| `root_path` | `/` | Subdirectory inside the volume to expose. |
| `host` | `None` | Optional workspace host override. |
| `token` | `None` | Optional PAT override. Redacted in snapshots. |
| `profile` | `None` | Optional Databricks SDK profile name. |
| `timeout` | `30` | Request timeout in seconds. |
## Mount mode
`read` or `write`.
## Filesystem layout
Mirage maps the configured mount prefix onto the configured volume subtree.
Given:
```python
DatabricksVolumeConfig(
catalog="main",
schema="default",
volume="agent_files",
root_path="/reports/2026",
)
```
and mount prefix `/dbx/`, the volume path:
```text
/Volumes/main/default/agent_files/reports/2026/q1/summary.md
```
appears in Mirage as:
```text
/dbx/q1/summary.md
```
`root_path` is normalized before use, and Mirage rejects any virtual path that
would escape above that configured subtree.
## Supported operations
Reads: `readdir`, `stat`, `exists`, `read_bytes`, `read_stream`,
`range_read`, and glob resolution. Writes: `write`, `create`, `mkdir`,
`rmdir`, `unlink`, recursive `rm`, `cp`, and `mv`.
`mv`/`cp` are non-atomic download + upload — the Files API has no server-side
rename.
## Shell Commands
The Databricks Volume resource supports shell commands that operate on real
file content. Reads use the Files API with range requests, so commands like
`head -c BYTES` avoid downloading the whole object. The supported set is
scoped to commands that work over the volume API (no compression, encoding,
or local-only utilities).
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
### Text Processing
| Command | Notes |
| ------- | -------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `diff` | Compare files line by line |
### File Operations
| Command | Notes |
| ------- | ---------------------------------------------------- |
| `cp` | Copy files (non-atomic download + upload) |
| `mv` | Move/rename files (non-atomic download + upload) |
| `rm` | Remove files (recursive for directories) |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
### Path Utilities
| Command | Notes |
| ------- | ----------------------- |
| `ls` | List directory contents |
## Snapshot behavior
`token` is redacted in resource state. Loading a snapshot back requires an
override config that provides fresh credentials if the runtime auth chain does
not already supply them.
## Databricks Apps
For Databricks Apps, prefer SDK-default auth and keep Mirage in-process:
```python
config = DatabricksVolumeConfig(
catalog="main",
schema="default",
volume="agent_files",
)
resource = DatabricksVolumeResource(config)
ws = Workspace({"/dbx/": resource}, mode=MountMode.READ)
```
This does not require FUSE. The agent can access the mounted workspace through
Mirage's backend adapters or `Workspace.execute(...)`.
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.databricks_volume import (
DatabricksVolumeConfig,
DatabricksVolumeResource,
)
resource = DatabricksVolumeResource(DatabricksVolumeConfig(
catalog="main",
schema="default",
volume="agent_files",
root_path="/reports",
))
async def main() -> None:
ws = Workspace({"/dbx/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /dbx/")
print(await r.stdout_str())
r = await ws.execute("find /dbx/ -name '*.md'")
print(await r.stdout_str())
r = await ws.execute('head -n 20 "/dbx/q1/summary.md"')
print(await r.stdout_str())
r = await ws.execute('stat "/dbx/q1/summary.md"')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See:
- `examples/python/databricks_volume/databricks_volume.py`
- `examples/python/agents/langchain/databricks_volume_deepagent.py`
## Use Cases
- **Agents in Databricks Apps**: mount a Unity Catalog volume in-process so an
agent can read and write governed files without FUSE or hardcoded credentials.
- **Reading governed datasets**: expose a reports or dataset subtree through
`root_path` and query it with shell commands.
- **Sandboxed volume access**: scope an agent to a single volume (and optional
`root_path`) so it cannot read or write outside that subtree.
- **Writing agent outputs back**: persist generated files to the volume with
`write`, `cp`, and `mv`.
+233
View File
@@ -0,0 +1,233 @@
---
title: Dify
icon: /images/dify-logo.svg
description: Mount a Dify Knowledge dataset as a read-only virtual filesystem.
---
The Dify resource exposes completed Dify Knowledge documents as text files
mounted at a prefix such as `/knowledge/`. Each file is assembled from the
document's completed, enabled segments.
For API key setup, see [Dify Setup](/python/setup/dify).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.dify import DifyConfig, DifyResource
config = DifyConfig(
api_key=os.environ["DIFY_API_KEY"],
base_url=os.environ.get("DIFY_BASE_URL", "https://api.dify.ai/v1"),
dataset_id=os.environ["DIFY_DATASET_ID"],
slug_metadata_name=os.environ.get("DIFY_SLUG_METADATA_NAME", "slug"),
)
resource = DifyResource(config=config)
ws = Workspace({"/knowledge/": resource}, mode=MountMode.READ)
```
The resource is read-only and does not support snapshots.
## Filesystem Layout
Dify documents are mapped to paths using the configured slug metadata field.
The default metadata field name is `slug`. If a document has no configured slug
metadata, Mirage falls back to the Dify document name.
```text
/knowledge/
README.md
guides/
quickstart.md
api.md
policies/
support.md
```
Example mapping:
| Dify document | Metadata | Mirage path |
| ------------- | -------- | ----------- |
| `Quickstart` | `slug=guides/quickstart.md` | `/knowledge/guides/quickstart.md` |
| `README.md` | no slug | `/knowledge/README.md` |
### Creating Slug Metadata in Dify
In the Dify Knowledge UI, open a document and add a metadata item whose name
matches `slug_metadata_name`. The default name is `slug`.
```text
name: slug
value: guides/quickstart.md
```
For a custom Dify metadata name:
```python
config = DifyConfig(
api_key=os.environ["DIFY_API_KEY"],
base_url=os.environ.get("DIFY_BASE_URL", "https://api.dify.ai/v1"),
dataset_id=os.environ["DIFY_DATASET_ID"],
slug_metadata_name="path",
)
```
Then use:
```text
name: path
value: guides/quickstart.md
```
Mirage reads `doc_metadata` from Dify and uses this value as the document's
virtual path below the mount prefix.
Use these rules for slug metadata values:
- Use `/` to create folders.
- Do not use empty segments, `.`, or `..`.
- Keep each slug metadata value unique across the dataset.
- Do not use a path that is also needed as a folder. For example, `guides` and `guides/quickstart.md` cannot both be document paths.
Only visible documents are included:
- `enabled` is `true`
- `indexing_status` is `completed`
- `archived` is `false`
## Reading Documents
`cat`, `head`, `tail`, and `grep` read Dify document segments. Segment content
is joined with a single newline between chunks.
```bash
ls /knowledge/
find /knowledge/ -type f
cat /knowledge/guides/quickstart.md
head -n 20 /knowledge/guides/quickstart.md
grep -i "billing" /knowledge/guides/quickstart.md
wc /knowledge/guides/quickstart.md
```
## Search
The `search` command calls Dify's dataset retrieval API. Use it when meaning
matters more than exact text matching.
```bash
search "how do I reset my password" /knowledge/
search --method hybrid --top-k 5 "billing policy" /knowledge/policies/
search --method semantic --threshold 0.4 "quickstart" /knowledge/guides/*.md
```
Supported methods:
| Method | Dify retrieval mode |
| ------ | ------------------- |
| `semantic` | semantic search |
| `fulltext` | full text search |
| `hybrid` | hybrid search |
| `keyword` | keyword search |
`top-k` is capped at `100`. `threshold` must be between `0` and `1`.
Results are emitted one retrieval hit per block:
```text
/knowledge/policies/refunds:0.82
Refunds are allowed within 30 days.
```
Mirage uses the configured slug metadata field to derive the path, and falls
back to the Dify document name when that metadata is missing. Multiple hits
from the same document remain separate records.
Scoped search uses Dify metadata filtering. Documents with the configured slug
metadata field are filtered by that field; name-based documents are filtered by
`document_name`, which requires Dify Built-in Fields to be enabled in dataset
metadata.
### Scoped Search Requirements
Mirage converts a scoped search path into Dify metadata filters:
| Mirage target | Dify metadata filter |
| ------------- | -------------------- |
| Document with configured slug metadata | `<slug_metadata_name> in [...]` |
| Document without configured slug metadata | `document_name in [...]` |
| Folder or glob | `<slug_metadata_name>` / `document_name` filters for all matched files |
To make scoped search reliable:
1. Add the configured slug metadata field to every document.
1. Enable Dify **Built-in Fields** in the dataset metadata settings.
1. Ensure `document_name` is available if you rely on name-based paths.
If Built-in Fields are disabled, scoped search against documents without the
configured slug metadata field may return empty results even though `cat`,
`grep`, and `find` can still see the same file.
## Cache
The resource uses Mirage's index cache for the virtual tree. Directory listings
and path resolution reuse the cached document tree until the index expires or
the workspace is recreated. If documents are edited directly in Dify, repeated
operations can temporarily see cached paths and metadata.
Document content is still read from Dify when commands materialize file data.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.dify import DifyConfig, DifyResource
load_dotenv(".env.development")
config = DifyConfig(
api_key=os.environ["DIFY_API_KEY"],
base_url=os.environ.get("DIFY_BASE_URL", "https://api.dify.ai/v1"),
dataset_id=os.environ["DIFY_DATASET_ID"],
slug_metadata_name=os.environ.get("DIFY_SLUG_METADATA_NAME", "slug"),
)
resource = DifyResource(config=config)
async def main() -> None:
ws = Workspace({"/knowledge/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /knowledge/")
print(await r.stdout_str())
r = await ws.execute("find /knowledge/ -type f | head -n 10")
print(await r.stdout_str())
r = await ws.execute('search --method hybrid --top-k 5 "getting started" /knowledge/')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
A runnable version is available at
`examples/python/dify/dify.py`.
## Shell Commands
| Command | Notes |
| ------- | ----- |
| `ls` | List folders and documents |
| `cat` | Read full document text from completed segments |
| `head` / `tail` | Read the first or last lines/bytes |
| `grep` | Exact or regex matching over streamed document text |
| `find` | Search the virtual tree by name, type, depth, and size |
| `wc` | Count lines, words, bytes, characters, and max line length |
| `search` | Semantic/full-text/hybrid/keyword retrieval through Dify |
+81
View File
@@ -0,0 +1,81 @@
---
title: DigitalOcean Spaces
icon: /images/digitalocean-logo.svg
description: Mount a DigitalOcean Spaces bucket via its S3-compatible API as a virtual filesystem.
---
The DigitalOcean resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `DigitalOceanConfig` to an `S3Config` and reuses the exact same
backend, commands, and behavior as S3, it just derives the right Spaces
endpoint from `region`. Uses aioboto3 against DigitalOcean Spaces'
S3-compatible API.
The endpoint is computed from `region` as `<region>.digitaloceanspaces.com`
(e.g. `nyc3.digitaloceanspaces.com`). Pass `endpoint_url` to override.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.digitalocean import DigitalOceanConfig, DigitalOceanResource
config = DigitalOceanConfig(
bucket=os.environ["DO_SPACE"],
region=os.environ.get("DO_REGION", "nyc3"),
access_key_id=os.environ["DO_ACCESS_KEY_ID"],
secret_access_key=os.environ["DO_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://nyc3.digitaloceanspaces.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = DigitalOceanResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.digitalocean import DigitalOceanConfig, DigitalOceanResource
load_dotenv(".env.development")
config = DigitalOceanConfig(
bucket=os.environ["DO_SPACE"],
region=os.environ.get("DO_REGION", "nyc3"),
access_key_id=os.environ["DO_ACCESS_KEY_ID"],
secret_access_key=os.environ["DO_SECRET_ACCESS_KEY"],
)
resource = DigitalOceanResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("tree /data/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- DigitalOcean reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [DigitalOcean Setup](/home/setup/digitalocean).
+348
View File
@@ -0,0 +1,348 @@
---
title: Discord
description: Mount Discord guilds, channels, messages, attachments, and members as a Mirage virtual filesystem for Python agents.
icon: discord
---
The Discord resource exposes guild, channel, and member data as a virtual
filesystem mounted at some prefix such as `/discord/`.
For token setup, see [Discord Setup](/python/setup/discord).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
ws = Workspace({"/discord": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/discord/
<guild-name>__<guild-id>/
channels/
<channel-name>__<channel-id>/
<yyyy-mm-dd>/
chat.jsonl
files/
<stem>__<attachment-id>.<ext>
...
...
members/
<username>__<user-id>.json
...
```
Example:
```text
/discord/
My Server__111222333444555666/
channels/
general__777888999000111222/
2026-04-04/
chat.jsonl
files/
screenshot__1488111222333444555.png
2026-04-05/
chat.jsonl
random__777888999000111223/
2026-04-11/
chat.jsonl
members/
alice__444555666777888999.json
bob__444555666777888900.json
```
Display names keep their original spelling from Discord (spaces,
apostrophes, emoji are all preserved). Only `/` is replaced with ``
(U+2215) so it cannot collide with a directory boundary. The Discord
snowflake ID is appended after `__` (double underscore) on guild,
channel, member, and attachment names so resource specific commands can
extract it without an extra lookup, and so two same named entities never
collide. Quote names containing spaces in shell commands. `stat` also
exposes the ID in the `extra` dict (see [Finding IDs](#finding-ids)).
### Guilds
The root lists one directory per guild the bot has access to.
### Channels
`/discord/<guild>/channels/` lists text channels (types 0, 5, 15).
Each channel directory contains day-partitioned **directories** for the
30 days leading up to the channel's last message. Each day directory
holds:
- `chat.jsonl`, the day's messages (one JSON object per line).
- `files/`, attachments posted on that day. Each blob is named
`<stem>__<attachment-id>.<ext>`, where the stem keeps the original
filename's spelling (only `/` is replaced). The ID suffix keeps the
filename collision-free. `cat`'ing a blob downloads it from the
Discord CDN.
The date range is derived from `last_message_id` on the channel object,
so inactive channels show dates around their last activity, not today.
Soft errors (403 missing permissions, 404 unknown channel, 429 rate
limit) on a single day are swallowed so listings, `find`, and `grep`
keep working across the rest of the tree.
### Members
`/discord/<guild>/members/` lists one `.json` file per member.
Reading a member file returns the full member payload from the
Discord API.
## Smart Commands
### grep / rg at different scopes
When `grep` or `rg` target a channel or guild directory (not a specific file),
they use the Discord search API instead of downloading every `.jsonl` file:
```bash
# FILE level - downloads the .jsonl, greps locally
grep hello "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# CHANNEL level - uses Discord search API (GET /guilds/{id}/messages/search)
grep hello "/discord/My Server__111222333444555666/channels/general__777888999000111222/"
# GUILD level - searches across all channels
grep hello "/discord/My Server__111222333444555666/"
```
Scope detection is handled by `mirage/core/discord/scope.py`.
### head / tail
`head` and `tail` on file-level paths use the Discord messages API directly
(`GET /channels/{id}/messages`) instead of downloading the full day's history.
## Cache
The Discord resource uses `IndexCacheStore` (same as RAM/S3/disk/GitHub).
Index entries store guild IDs, channel IDs, and `last_message_id` for
date range computation. There is no separate content cache - file content
caching is handled by the workspace `IOResult` mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
load_dotenv(".env.development")
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
async def main():
ws = Workspace({"/discord": resource}, mode=MountMode.READ)
# List guilds
r = await ws.execute("ls /discord/")
print(await r.stdout_str())
guild = r.stdout_str().strip().split("\n")[0].strip()
# List channels
r = await ws.execute(f'ls "/discord/{guild}/channels/"')
print(await r.stdout_str())
ch = r.stdout_str().strip().splitlines()[0].strip()
base = f"/discord/{guild}/channels/{ch}"
# Read messages from a specific date
r = await ws.execute(f'cat "{base}/2026-04-04/chat.jsonl" | head -n 3')
print(await r.stdout_str())
# Extract usernames with jq
r = await ws.execute(
f'jq -r ".[] | .author.username" "{base}/2026-04-04/chat.jsonl"')
print(await r.stdout_str())
# Count messages per user
r = await ws.execute(
f'cat "{base}/2026-04-04/chat.jsonl"'
' | jq -r ".[] | .author.username" | sort | uniq -c')
print(await r.stdout_str())
# List attachments for a day and download one
r = await ws.execute(f'ls "{base}/2026-04-04/files/"')
print(await r.stdout_str())
r = await ws.execute(
f'cat "{base}/2026-04-04/files/screenshot__1488111222333444555.png"')
blob = await r.materialize_stdout()
print(f"downloaded {len(blob)} bytes")
# Search across channel (uses Discord search API)
r = await ws.execute(f'grep hello "{base}/"')
print(await r.stdout_str())
# Search across guild
r = await ws.execute(f'grep hello "/discord/{guild}/"')
print(await r.stdout_str())
# Navigate with cd/pwd
await ws.execute(f'cd "{base}"')
r = await ws.execute("pwd")
print(await r.stdout_str())
# Relative paths after cd
r = await ws.execute("ls | tail -n 5")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/python/discord/discord.py` for the full working example.
## Finding IDs
Resource-specific commands require Discord snowflake IDs
(`channel_id`, `guild_id`, `message_id`). These can be extracted
from the filesystem:
```bash
# Guild ID - use stat
stat "/discord/My Server__111222333444555666"
# → extra={"guild_id": "1256522563555819574"}
# Channel ID - use stat
stat "/discord/My Server__111222333444555666/channels/general__777888999000111222"
# → extra={"channel_id": "1256522563555819574"}
# Message ID - inside JSONL messages
jq -r '.[] | "\(.id) [\(.author.username)] \(.content)"' \
"/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# → 1489887688978075769 [alice] hello world
# Find a message then reply
jq -r '.[] | select(.content | test("hello")) | .id' \
"/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# → 1489887688978075769
discord-send-message --channel_id 1256522563555819574 \
--text "Reply" --message_id 1489887688978075769
```
## Working with Large Channels
Tips for efficient access on busy channels:
```bash
# Check message count per day
wc -l "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# Read only recent messages
tail -n 10 "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# Search uses Discord API at channel/guild level (no file download)
grep "keyword" "/discord/My Server__111222333444555666/channels/general__777888999000111222/"
# Extract specific fields
jq -r '.[] | "\(.author.username): \(.content)"' \
"/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl" | head -n 20
# Count messages per user
cat "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl" \
| jq -r '.[] | .author.username' | sort | uniq -c
```
Note: `grep`/`rg` at channel or guild level uses the Discord search
API instead of downloading every `.jsonl` file, making it efficient
even for large channels.
## Shell Commands
Standard commands available on the mounted Discord tree:
| Command | Notes |
| --------------- | -------------------------------------------------------------- |
| `ls` | List guilds, channels, members, dates, attachments |
| `cat` | Read `chat.jsonl`, member `.json`, or download an attachment |
| `head` / `tail` | Smart: uses messages API for file scope |
| `grep` / `rg` | Smart: uses search API for channel/guild scope (with fallback) |
| `jq` | Query JSON; use `.[]` prefix for JSONL files |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, ID via `extra`) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
Resource-specific commands:
### `discord-send-message`
Post a message to a channel, optionally as a reply.
```bash
discord-send-message --channel_id 1256522563555819574 --text "Hello from MIRAGE"
discord-send-message --channel_id 1256522563555819574 --text "Reply" --message_id 1489887688978075769
```
| Option | Required | Description |
| -------------- | -------- | ---------------------------- |
| `--channel_id` | yes | Discord channel snowflake ID |
| `--text` | yes | Message text to send |
| `--message_id` | no | Message ID to reply to |
The channel ID can be found in directory names under
`/discord/<guild>/channels/` or via `stat`. Returns the
posted message JSON.
### `discord-add-reaction`
Add an emoji reaction to a message.
```bash
discord-add-reaction --channel_id 1256522563555819574 --message_id 1489887688978075769 --reaction 👍
```
| Option | Required | Description |
| -------------- | -------- | ---------------------------- |
| `--channel_id` | yes | Discord channel snowflake ID |
| `--message_id` | yes | Message snowflake ID |
| `--reaction` | yes | Emoji (unicode or name) |
### `discord-list-members`
Search guild members by name.
```bash
discord-list-members --guild_id 1256522563555819574 --query "alice"
```
| Option | Required | Description |
| ------------ | -------- | ----------------------- |
| `--guild_id` | yes | Discord guild snowflake |
| `--query` | yes | Username search query |
Returns matching members as JSON array.
### `discord-get-server-info`
Get full guild metadata from the Discord API.
```bash
discord-get-server-info --guild_id 1256522563555819574
```
| Option | Required | Description |
| ------------ | -------- | ----------------------- |
| `--guild_id` | yes | Discord guild snowflake |
Returns the guild object JSON (name, icon, member count, etc.).
+218
View File
@@ -0,0 +1,218 @@
---
title: Disk
description: Mount a local directory as a Mirage resource with read/write shell commands and path traversal protection.
icon: hard-drive
---
The Disk resource mounts a local directory at some prefix such as `/data/`.
All operations are backed by real files on disk. Path resolution validates
against the root boundary to prevent directory traversal escapes.
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.disk import DiskResource
resource = DiskResource(root="/path/to/dir")
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
`DiskResource(root=...)` takes a single `root` path argument pointing to the
directory to mount. Both `READ` and `WRITE` modes are supported.
## Filesystem Layout
The Disk resource mirrors the structure of the `root` directory. For example,
if `root="/srv/files"` contains:
```text
/srv/files/
notes.txt
config.json
reports/
q1.csv
q2.csv
```
Then mounting at `/data/` exposes:
```text
/data/
notes.txt
config.json
reports/
q1.csv
q2.csv
```
Paths like `../../etc/passwd` are rejected - resolution is always confined
to the root boundary.
## Cache
The Disk resource uses `IndexCacheStore` with `index_ttl = 60` (1 minute).
Directory listings are cached for up to 60 seconds before being refreshed
from disk.
## Example
```python
import asyncio
import shutil
import tempfile
from pathlib import Path
from mirage import MountMode, Workspace
from mirage.resource.disk import DiskResource
DATA_DIR = Path("/path/to/files")
tmp = tempfile.mkdtemp()
shutil.copytree(DATA_DIR, Path(tmp) / "files", dirs_exist_ok=True)
resource = DiskResource(root=tmp + "/files")
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("cat /data/example.json")
print(await r.stdout_str())
r = await ws.execute("tree /data/")
print(await r.stdout_str())
r = await ws.execute("find /data/ -name '*.json'")
print(await r.stdout_str())
r = await ws.execute("grep example /data/example.json")
print(await r.stdout_str())
r = await ws.execute("stat /data/example.json")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The Disk resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.):
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **Local directory access**: Mount local directories for AI agents to read and process
- **Sandboxed file access**: Restrict agent file operations to a specific directory tree
- **FUSE mounting**: Expose disk files through a virtual FUSE mount for external tools
- **Data pipelines**: Process local datasets with shell-like commands
- **Development**: Test file operations against real data before deploying to cloud resources
+322
View File
@@ -0,0 +1,322 @@
---
title: Email
description: Mount an IMAP mailbox as a Mirage filesystem and send messages through SMTP from Python workspaces.
icon: at
---
The Email resource exposes any IMAP mailbox as a virtual filesystem
mounted at some prefix such as `/email/`. Sending is handled via SMTP.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.email import EmailConfig, EmailResource
config = EmailConfig(
imap_host=os.environ["IMAP_HOST"],
smtp_host=os.environ["SMTP_HOST"],
username=os.environ["EMAIL_USERNAME"],
password=os.environ["EMAIL_PASSWORD"],
)
resource = EmailResource(config=config)
ws = Workspace({"/email": resource}, mode=MountMode.READ)
```
### Config Reference
| Field | Type | Default | Description |
| ----------- | ------ | ------- | ------------------------ |
| `imap_host` | `str` | | IMAP server hostname |
| `imap_port` | `int` | `993` | IMAP port |
| `smtp_host` | `str` | | SMTP server hostname |
| `smtp_port` | `int` | `587` | SMTP port |
| `username` | `str` | | Email address / login |
| `password` | `str` | | Password or app password |
| `use_ssl` | `bool` | `True` | Use SSL for IMAP |
### Common Resource Settings
| Resource | IMAP Host | SMTP Host | Notes |
| ----------- | ----------------------- | --------------------- | -------------------------- |
| Outlook/365 | `outlook.office365.com` | `smtp.office365.com` | App password or OAuth2 |
| Yahoo | `imap.mail.yahoo.com` | `smtp.mail.yahoo.com` | App password required |
| Fastmail | `imap.fastmail.com` | `smtp.fastmail.com` | App password |
| iCloud | `imap.mail.me.com` | `smtp.mail.me.com` | App password |
| ProtonMail | `127.0.0.1` (Bridge) | `127.0.0.1` (Bridge) | Requires ProtonMail Bridge |
| Self-hosted | Your server hostname | Your server hostname | Whatever you configured |
## Filesystem Layout
```text
/email/
<folder>/
<yyyy-mm-dd>/
<sanitized-subject>__<uid>.email.json
<sanitized-subject>__<uid>/ # only if message has attachments
<attachment-filename>
```
Example:
```text
/email/
INBOX/
2026-04-14/
Meeting_Notes__12345.email.json
Meeting_Notes__12345/
report.pdf
screenshot.png
Simple_Email__12346.email.json
2026-04-13/
Hello__12347.email.json
Sent/
2026-04-14/
Reply__12348.email.json
Drafts/
Archive/
```
Folder directories appear at the root, using IMAP folder names
(e.g., `INBOX`, `Sent`, `Drafts`).
### Date Directories
Inside each folder, messages are grouped into date subdirectories
formatted as `YYYY-MM-DD`. The date is derived from the message's
`Date` header.
### Message Files
Each message is stored as a `.email.json` file. The filename shape is:
```text
<sanitized-subject>__<uid>.email.json
```
### Attachments
Messages that have attachments get a companion subdirectory with
the same base name (without `.email.json`):
```text
Meeting_Notes__12345.email.json # message JSON
Meeting_Notes__12345/ # attachment directory
report.pdf
screenshot.png
```
## Cache
The Email resource uses `IndexCacheStore` (same as Gmail, Slack,
Discord, and other resources). Index entries store folder names,
message UIDs, and message metadata.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.email import EmailConfig, EmailResource
load_dotenv(".env.development")
config = EmailConfig(
imap_host=os.environ["IMAP_HOST"],
smtp_host=os.environ["SMTP_HOST"],
username=os.environ["EMAIL_USERNAME"],
password=os.environ["EMAIL_PASSWORD"],
)
resource = EmailResource(config=config)
async def main():
ws = Workspace({"/email": resource}, mode=MountMode.READ)
# List folders
r = await ws.execute("ls /email/")
print(await r.stdout_str())
# List date directories in INBOX
r = await ws.execute("ls /email/INBOX/")
print(await r.stdout_str())
# List messages for a specific date
r = await ws.execute("ls /email/INBOX/2026-04-14/")
print(await r.stdout_str())
# Read a message
r = await ws.execute(
"cat /email/INBOX/2026-04-14/Meeting_Notes__12345.email.json")
print(await r.stdout_str())
# Extract subject with jq
r = await ws.execute(
'jq ".subject"'
" /email/INBOX/2026-04-14/Meeting_Notes__12345.email.json")
print(await r.stdout_str())
# List attachments
r = await ws.execute("ls /email/INBOX/2026-04-14/Meeting_Notes__12345/")
print(await r.stdout_str())
# Search across all messages
r = await ws.execute('rg "quarterly" /email/INBOX/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 2 /email/INBOX/")
print(await r.stdout_str())
# Triage unread messages
r = await ws.execute("email-triage --folder INBOX --unseen --max 5")
print(await r.stdout_str())
# Send an email
r = await ws.execute(
'email-send --to "user@example.com"'
' --subject "Hello from MIRAGE"'
' --body "This email was sent via the MIRAGE email resource."')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Finding UIDs
Resource-specific commands require message UIDs. These can be
extracted from the filesystem:
```bash
# UID is embedded in filename after "__"
ls /email/INBOX/2026-04-14/
# -> Meeting_Notes__12345.email.json <- uid = 12345
# Read a message then reply
email-read --uid 12345 --folder INBOX
email-reply --uid 12345 --folder INBOX --body "Thanks for the notes"
```
## Shell Commands
Standard commands available on the mounted email tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List folders, dates, messages, attachments |
| `cat` | Read message JSON or attachment content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query message JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `basename` | Extract filename from path |
| `dirname` | Extract directory from path |
| `realpath` | Resolve path to absolute form |
| `nl` | Number lines of output |
Resource-specific commands:
### `email-send`
Send a new email.
```bash
email-send --to "user@example.com" --subject "Hello" --body "Hi there"
```
| Option | Required | Description |
| ----------- | -------- | ----------------------- |
| `--to` | yes | Recipient email address |
| `--subject` | yes | Email subject line |
| `--body` | yes | Email body text |
Returns the sent message status JSON.
### `email-reply`
Reply to a message.
```bash
email-reply --uid 12345 --folder INBOX --body "Thanks for the update"
```
| Option | Required | Description |
| ---------- | -------- | ---------------------------- |
| `--uid` | yes | Message UID |
| `--folder` | no | IMAP folder (default: INBOX) |
| `--body` | yes | Reply body text |
Returns the sent reply JSON.
### `email-reply-all`
Reply-all to a message.
```bash
email-reply-all --uid 12345 --folder INBOX --body "Acknowledged by the team"
```
| Option | Required | Description |
| ---------- | -------- | ---------------------------- |
| `--uid` | yes | Message UID |
| `--folder` | no | IMAP folder (default: INBOX) |
| `--body` | yes | Reply body text |
Returns the sent reply JSON.
### `email-forward`
Forward a message to another recipient.
```bash
email-forward --uid 12345 --folder INBOX --to "colleague@example.com"
```
| Option | Required | Description |
| ---------- | -------- | ---------------------------- |
| `--uid` | yes | Message UID |
| `--folder` | no | IMAP folder (default: INBOX) |
| `--to` | yes | Recipient email address |
Returns the forwarded message JSON.
### `email-triage`
Search and triage emails.
```bash
email-triage --folder INBOX --unseen --max 10
```
| Option | Required | Description |
| ---------- | -------- | ---------------------------- |
| `--folder` | no | IMAP folder (default: INBOX) |
| `--max` | no | Max results (default: 20) |
| `--unseen` | no | Only show unread messages |
Returns matching messages as JSON.
### `email-read`
Read a message by its UID.
```bash
email-read --uid 12345 --folder INBOX
```
| Option | Required | Description |
| ---------- | -------- | ---------------------------- |
| `--uid` | yes | Message UID |
| `--folder` | no | IMAP folder (default: INBOX) |
Returns the full message JSON.
+232
View File
@@ -0,0 +1,232 @@
---
title: GCS
description: Mount Google Cloud Storage buckets through Mirage's async S3-compatible resource layer for Python agents.
icon: google
---
The GCS resource mounts a Google Cloud Storage bucket at some prefix
such as `/gcs/`. All operations involve network I/O to the remote object
store. Uses aioboto3 against GCS's S3-compatible XML API via HMAC keys,
inheriting all S3 resource capabilities.
For credential setup, see [GCS Setup](/home/setup/gcs).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.gcs import GCSConfig, GCSResource
config = GCSConfig(
bucket=os.environ["GCS_BUCKET"],
access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://storage.googleapis.com",
# region="auto",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = GCSResource(config)
ws = Workspace({"/gcs": resource}, mode=MountMode.READ)
```
`GCSResource(config)` takes a `GCSConfig` object with the bucket name
and HMAC credentials. Both `READ` and `WRITE` modes are supported.
## Filesystem Layout
The GCS resource maps object keys to virtual paths under the mount
prefix, identical to the S3 resource. GCS "directories" are
prefix-based — there are no real directory objects.
For example, if bucket `mirage-ai` contains:
```text
data/example.json
data/example.parquet
data/example.jsonl
```
Then mounting at `/gcs/` exposes:
```text
/gcs/
data/
example.json
example.parquet
example.jsonl
```
Path mapping: virtual `/gcs/data/example.json` maps to GCS key
`data/example.json`.
## Cache
The GCS resource uses `IndexCacheStore` with `index_ttl = 600`
(10 minutes), same as S3. Directory listings are cached for up to 600
seconds before being refreshed from GCS. This reduces API calls for
repeated directory traversals.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gcs import GCSConfig, GCSResource
load_dotenv(".env.development")
config = GCSConfig(
bucket=os.environ["GCS_BUCKET"],
access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
)
resource = GCSResource(config)
async def main() -> None:
ws = Workspace({"/gcs/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /gcs/")
print(await r.stdout_str())
r = await ws.execute("cat /gcs/data/example.json | head -n 10")
print(await r.stdout_str())
r = await ws.execute("tree /gcs/")
print(await r.stdout_str())
r = await ws.execute("find /gcs/ -name '*.parquet'")
print(await r.stdout_str())
r = await ws.execute("stat /gcs/data/example.json")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The GCS resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit
from range reads to avoid downloading entire objects.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **AI agents accessing GCS data**: Mount GCS buckets for agents to read and process datasets
- **Data pipelines**: Read and write GCS objects with shell-like commands
- **FUSE mounting**: Expose GCS buckets through a virtual FUSE mount for external tools
+198
View File
@@ -0,0 +1,198 @@
---
title: Google Docs
description: Mount Google Docs as JSON-backed files so Python agents can read document structure and content through Mirage.
icon: file-lines
---
The Google Docs resource exposes Docs documents as a virtual filesystem
mounted at some prefix such as `/gdocs/`.
For Google OAuth setup, see [Google Workspace Setup](/python/setup/google).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
ws = Workspace({"/gdocs": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/gdocs/
owned/
<YYYY-MM-DD>_<sanitized-title>__<doc-id>.gdoc.json
...
shared/
<YYYY-MM-DD>_<sanitized-title>__<doc-id>.gdoc.json
...
```
Example:
```text
/gdocs/
owned/
2026-04-04_Project_Plan__1AbCdEf.gdoc.json
2026-04-10_Meeting_Notes__2BcDeFg.gdoc.json
shared/
2026-04-03_Design_Notes__9XyZ.gdoc.json
```
Documents are split into `owned` (documents you created) and `shared`
(documents shared with you). The filename shape is:
```text
<YYYY-MM-DD>_<sanitized-title>__<document-id>.gdoc.json
```
If the modified date is unavailable, the date prefix is omitted.
Reading a document file returns the full Google Docs API JSON for
that document.
## Cache
The Google Docs resource uses `IndexCacheStore`. Index entries store
document IDs and metadata. There is no separate content cache -- file
content caching is handled by the workspace `IOResult` mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
load_dotenv(".env.development")
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
async def main():
ws = Workspace({"/gdocs": resource}, mode=MountMode.READ)
# List structure
r = await ws.execute("ls /gdocs/")
print(await r.stdout_str())
# List owned documents
r = await ws.execute("ls /gdocs/owned/")
print(await r.stdout_str())
# Read a document
r = await ws.execute(
"cat /gdocs/owned/2026-04-04_Project_Plan__1AbCdEf.gdoc.json")
print(await r.stdout_str())
# Extract title with jq
r = await ws.execute(
'jq ".title"'
" /gdocs/owned/2026-04-04_Project_Plan__1AbCdEf.gdoc.json")
print(await r.stdout_str())
# Search across all documents
r = await ws.execute('rg "quarterly" /gdocs/owned/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 1 /gdocs/")
print(await r.stdout_str())
# Create a new document
r = await ws.execute(
'gws-docs-documents-create --json \'{"title":"MIRAGE Example Doc"}\'')
print(await r.stdout_str())
# Append text to a document
r = await ws.execute(
'gws-docs-write --document 1AbCdEf --text "Appended via MIRAGE."')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/gdocs/gdocs.py` for the full working example.
## Shell Commands
Standard commands available on the mounted Google Docs tree:
| Command | Notes |
| ----------------------------------- | --------------------------- |
| `ls` | List owned/shared documents |
| `cat` | Read document JSON |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata |
| `find` | Recursive search |
| `tree` | Directory tree view |
| `basename` / `dirname` / `realpath` | Path utilities |
| `nl` | Number lines |
Resource-specific commands:
### `gws-docs-documents-create`
Create a new Google Docs document.
```bash
gws-docs-documents-create --json '{"title":"MIRAGE Example Doc"}'
```
| Option | Required | Description |
| -------- | -------- | ---------------------------- |
| `--json` | yes | JSON body with `title` field |
Returns the created document JSON.
### `gws-docs-documents-batchUpdate`
Batch update a document.
```bash
gws-docs-documents-batchUpdate --params '{"documentId":"1AbCdEf"}' --json '{"requests":[]}'
```
| Option | Required | Description |
| ---------- | -------- | -------------------------- |
| `--params` | yes | JSON with `documentId` |
| `--json` | yes | JSON with `requests` array |
Returns the batch update response JSON.
### `gws-docs-write`
Append text to a document.
```bash
gws-docs-write --document 1AbCdEf --text "Appended via MIRAGE."
```
| Option | Required | Description |
| ------------ | -------- | ------------------ |
| `--document` | yes | Google document ID |
| `--text` | yes | Text to append |
Returns the update response JSON.
+398
View File
@@ -0,0 +1,398 @@
---
title: Google Drive
description: Mount Google Drive folders and files, including Docs, Sheets, and Slides, as a Mirage virtual filesystem.
icon: google
---
The Google Drive resource exposes a Google Drive account as a virtual
filesystem mounted at some prefix such as `/gdrive/`.
For Google OAuth setup, see [Google Workspace Setup](/python/setup/google).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
ws = Workspace({"/gdrive": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/gdrive/
<folder>/
<file>
<subfolder>/
...
<name>.gdoc.json
<name>.gsheet.json
<name>.gslide.json
<shared-drive>/
<file>
<folder>/
```
Example:
```text
/gdrive/
Projects/
spec.pdf
roadmap.gsheet.json
Budget/
Q1.gsheet.json
Q2.gsheet.json
Notes/
meeting.gdoc.json
Presentations/
quarterly_review.gslide.json
Team Drive/
shared-spec.pdf
data.csv
report.pdf
```
The mount mirrors the actual Google Drive folder hierarchy. The root
contains the Drive API `root` folder plus each Shared Drive visible to
the user. Shared Drives appear as top-level directories. Duplicate names
receive a `[Shared Drive]` suffix and, when needed, a numeric suffix.
Subfolders appear as directories, and regular files keep their original names.
### Synthetic Extensions
Google Workspace files cannot be downloaded as raw bytes, so they
are exposed with synthetic extensions and read via their respective
APIs:
| Extension | Type | Read via |
| -------------- | ------------- | ---------- |
| `.gdoc.json` | Google Docs | Docs API |
| `.gsheet.json` | Google Sheets | Sheets API |
| `.gslide.json` | Google Slides | Slides API |
Regular files (PDFs, images, CSVs, etc.) are downloaded directly
from Drive. Large regular files use streaming.
## Cache
The Google Drive resource uses `IndexCacheStore` (same as Slack,
Gmail, and other resources). Index entries store folder IDs, file
IDs, and file metadata. There is no separate content cache -- file
content caching is handled by the workspace `IOResult` mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
async def main():
ws = Workspace({"/gdrive": resource}, mode=MountMode.READ)
# List root
r = await ws.execute("ls /gdrive/ | head -n 10")
print(await r.stdout_str())
# Browse a subfolder
r = await ws.execute("ls /gdrive/Projects/")
print(await r.stdout_str())
# Read a regular file
r = await ws.execute("cat /gdrive/Projects/spec.pdf | head -c 200")
print(await r.stdout_str())
# Read a Google Doc title
r = await ws.execute('jq ".title" /gdrive/Notes/meeting.gdoc.json')
print(await r.stdout_str())
# Read a Google Sheet title
r = await ws.execute(
'jq ".properties.title" /gdrive/Projects/roadmap.gsheet.json')
print(await r.stdout_str())
# Read a Google Slides deck length
r = await ws.execute(
'jq ".slides | length"'
" /gdrive/Presentations/quarterly_review.gslide.json")
print(await r.stdout_str())
# Search across all files
r = await ws.execute('rg "quarterly" /gdrive/Projects/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 2 /gdrive/")
print(await r.stdout_str())
# Create a new Google Doc
r = await ws.execute(
"gws-docs-documents-create"
' --json \'{"title": "New Doc from MIRAGE"}\'')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/google/gdrive.py` for the full working example.
## Shell Commands
Standard commands available on the mounted Google Drive tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List folders and files |
| `cat` | Read file content (regular or Workspace) |
| `head` / `tail` | First/last N lines or bytes |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields on Workspace files |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `basename` | Extract filename from path |
| `dirname` | Extract directory from path |
| `realpath` | Resolve path to absolute form |
| `nl` | Number lines of output |
| `sort` | Sort lines |
| `uniq` | Deduplicate adjacent lines |
| `cut` | Extract fields/columns |
| `awk` | Pattern-directed scanning |
| `sed` | Stream editor |
| `tr` | Translate/delete characters |
| `diff` | Compare two files |
| `rev` | Reverse lines |
| `tac` | Reverse file line order |
| `paste` | Merge lines of files |
| `join` | Join lines on a common field |
| `column` | Columnate output |
| `comm` | Compare sorted files line by line |
| `fold` | Wrap lines to a given width |
| `fmt` | Reformat paragraph text |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `du` | Estimate file space usage |
| `shuf` | Randomly permute lines |
| `look` | Display lines beginning with a prefix |
| `strings` | Extract printable strings from binary |
| `base64` | Base64 encode/decode |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
| `xxd` | Hex dump |
| `zcat` | Read compressed files |
| `zgrep` | Search compressed files |
| `readlink` | Print resolved symbolic links |
| `cmp` | Compare two files byte by byte |
| `tsort` | Topological sort |
| `file` | Detect file type |
## Data Format Support
Google Drive may contain data files in binary columnar formats.
These are auto-converted to CSV on read. Specialized variants
of common commands handle them natively:
| Format | Extension | Specialized commands |
| ------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Parquet | `.parquet` | `cat-parquet`, `head-parquet`, `tail-parquet`, `wc-parquet`, `stat-parquet`, `grep-parquet`, `cut-parquet`, `ls-parquet`, `file-parquet` |
| Feather | `.feather` | `cat-feather`, `head-feather`, `tail-feather`, `wc-feather`, `stat-feather`, `grep-feather`, `cut-feather`, `ls-feather`, `file-feather` |
| HDF5 | `.hdf5` | `cat-hdf5`, `head-hdf5`, `tail-hdf5`, `wc-hdf5`, `stat-hdf5`, `grep-hdf5`, `cut-hdf5`, `ls-hdf5`, `file-hdf5` |
| ORC | `.orc` | `cat-orc`, `head-orc`, `tail-orc`, `wc-orc`, `stat-orc`, `grep-orc`, `cut-orc`, `ls-orc`, `file-orc` |
Example:
```bash
cat-parquet /gdrive/data/sales.parquet
head-parquet -n 5 /gdrive/data/sales.parquet
grep-parquet "revenue" /gdrive/data/sales.parquet
wc-parquet /gdrive/data/sales.parquet
```
## Resource-Specific Commands
Google Drive registers Google Workspace write commands so that
Google-native files can be created and updated from the Drive mount.
### `gws-docs-documents-create`
Create a new Google Doc.
```bash
gws-docs-documents-create --json '{"title": "My Doc"}'
```
| Option | Required | Description |
| -------- | -------- | ---------------------------- |
| `--json` | yes | JSON body with `title` field |
Returns the created document JSON.
### `gws-docs-documents-batchUpdate`
Apply batch updates to a Google Doc.
```bash
gws-docs-documents-batchUpdate \
--params '{"documentId": "DOC_ID"}' \
--json '{"requests": [...]}'
```
| Option | Required | Description |
| ---------- | -------- | ------------------------------- |
| `--params` | yes | JSON with `documentId` |
| `--json` | yes | JSON body with `requests` array |
Returns the batch update response JSON.
### `gws-docs-write`
Append text to a Google Doc.
```bash
gws-docs-write --document DOC_ID --text "Hello from MIRAGE"
```
| Option | Required | Description |
| ------------ | -------- | -------------- |
| `--document` | yes | Google Doc ID |
| `--text` | yes | Text to append |
Returns the update response JSON.
### `gws-sheets-read`
Read values from a spreadsheet range.
```bash
gws-sheets-read --spreadsheet SHEET_ID --range "Sheet1!A1:C10"
```
| Option | Required | Description |
| --------------- | -------- | ----------------- |
| `--spreadsheet` | yes | Spreadsheet ID |
| `--range` | yes | A1 notation range |
Returns the range values.
### `gws-sheets-write`
Write values to a spreadsheet range.
```bash
gws-sheets-write \
--params '{"spreadsheetId": "SHEET_ID", "range": "Sheet1!A1:C3", "valueInputOption": "USER_ENTERED"}' \
--json '{"values": [["a", "b", "c"]]}'
```
| Option | Required | Description |
| ---------- | -------- | --------------------------------------------------------------- |
| `--params` | yes | JSON with `spreadsheetId`, `range`, optional `valueInputOption` |
| `--json` | yes | JSON body with `values` array |
Returns the update response JSON.
### `gws-sheets-append`
Append rows to a spreadsheet.
```bash
gws-sheets-append --spreadsheet SHEET_ID --range "Sheet1!A1" --values "a,b,c"
```
| Option | Required | Description |
| --------------- | -------- | ---------------------------------------------- |
| `--spreadsheet` | yes | Spreadsheet ID |
| `--range` | no | A1 notation range (defaults to `A1`) |
| `--values` | no | Comma-separated values for a single row |
| `--json-values` | no | JSON array of rows (alternative to `--values`) |
One of `--values` or `--json-values` is required. Returns the
append response JSON.
### `gws-sheets-spreadsheets-create`
Create a new spreadsheet.
```bash
gws-sheets-spreadsheets-create --json '{"properties": {"title": "My Sheet"}}'
```
| Option | Required | Description |
| -------- | -------- | --------------------------------- |
| `--json` | yes | JSON body with `properties.title` |
Returns the created spreadsheet JSON.
### `gws-sheets-spreadsheets-batchUpdate`
Apply batch updates to a spreadsheet.
```bash
gws-sheets-spreadsheets-batchUpdate \
--params '{"spreadsheetId": "SHEET_ID"}' \
--json '{"requests": [...]}'
```
| Option | Required | Description |
| ---------- | -------- | ------------------------------- |
| `--params` | yes | JSON with `spreadsheetId` |
| `--json` | yes | JSON body with `requests` array |
Returns the batch update response JSON.
### `gws-slides-presentations-create`
Create a new Google Slides presentation.
```bash
gws-slides-presentations-create --json '{"title": "My Deck"}'
```
| Option | Required | Description |
| -------- | -------- | ---------------------------- |
| `--json` | yes | JSON body with `title` field |
Returns the created presentation JSON.
### `gws-slides-presentations-batchUpdate`
Apply batch updates to a presentation.
```bash
gws-slides-presentations-batchUpdate \
--params '{"presentationId": "PRES_ID"}' \
--json '{"requests": [...]}'
```
| Option | Required | Description |
| ---------- | -------- | ------------------------------- |
| `--params` | yes | JSON with `presentationId` |
| `--json` | yes | JSON body with `requests` array |
Returns the batch update response JSON.
+171
View File
@@ -0,0 +1,171 @@
---
title: GitHub
description: Mount a GitHub repository as a read-only Mirage filesystem for agents to list, read, search, and inspect code.
icon: github
---
The GitHub resource mounts a GitHub repository as a read-only virtual
filesystem.
For token setup, see [GitHub Setup](/python/setup/github).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.github import GitHubConfig, GitHubResource
config = GitHubConfig(token=os.environ["GITHUB_TOKEN"])
resource = GitHubResource(
config=config, owner="my-org", repo="my-repo", ref="main")
ws = Workspace({"/github": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/github/
README.md
pyproject.toml
src/
__init__.py
main.py
utils.py
models/
user.py
item.py
tests/
test_main.py
```
The filesystem mirrors the repository tree. No owner/repo/branch in
the path - those are specified at mount time.
## Tree Fetching
The resource fetches the full recursive tree at init. For repos with
> 100K entries, it falls back to per-directory fetching.
## Cache
The GitHub resource uses `IndexCacheStore` with SHA-based fingerprinting. Content is content-addressed - if the SHA matches, the content is identical.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.github import GitHubConfig, GitHubResource
load_dotenv(".env.development")
async def main():
config = GitHubConfig(token=os.environ["GITHUB_TOKEN"])
resource = GitHubResource(
config=config, owner="my-org", repo="my-repo", ref="main")
ws = Workspace({"/github": resource}, mode=MountMode.READ)
# List repository root
r = await ws.execute("ls /github/")
print(await r.stdout_str())
# Read a file
r = await ws.execute("cat /github/README.md")
print(await r.stdout_str())
# Search for a pattern
r = await ws.execute('rg "def main" /github/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 2 /github/")
print(await r.stdout_str())
# File metadata with SHA
r = await ws.execute("stat /github/README.md")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/code/github.py` for the full working example.
## Finding SHAs
Git blob SHAs are available via the `stat` command:
```bash
stat /github/README.md
# -> extra={"sha": "a1b2c3d4e5f6..."}
stat /github/src/main.py
# -> extra={"sha": "f6e5d4c3b2a1..."}
```
## Working with Large Repos
Tips for efficient access on large repositories:
```bash
# Find files by name
find /github/ -name "*.py"
# Search with rg (uses GitHub code search API when applicable)
rg "TODO" /github/
# Read only the first lines of a file
head -n 20 /github/src/main.py
# Check file sizes
du /github/src/
# List deeply nested directories
tree -L 3 /github/src/
```
## Shell Commands
Standard commands available on the mounted GitHub tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List files and directories |
| `cat` | Read file contents |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search; rg uses code search API |
| `jq` | Query JSON files |
| `wc` | Line/word/byte counts |
| `stat` | File metadata including git SHA |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `diff` | Compare files |
| `du` | Disk usage / file sizes |
| `awk` | Text processing |
| `sed` | Stream editing |
| `sort` | Sort lines |
| `uniq` | Deduplicate lines |
| `cut` | Extract columns |
| `tr` | Translate characters |
| `nl` | Number lines |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
| `file` | Detect file type |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
## Search Optimization
`rg` uses the GitHub code search API when the search scope exceeds
100 files and the mounted ref is the repository's default branch.
This avoids downloading file contents and returns results significantly
faster for large repositories.
+194
View File
@@ -0,0 +1,194 @@
---
title: GitHub CI
description: Mount GitHub Actions workflows, runs, jobs, logs, and artifacts as a read-only Mirage filesystem.
icon: circle-play
---
The GitHub CI resource mounts GitHub Actions workflows and runs as a
read-only virtual filesystem.
For token setup, see [GitHub CI Setup](/python/setup/github_ci).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.github_ci import GitHubCIConfig, GitHubCIResource
config = GitHubCIConfig(
token=os.environ["GITHUB_TOKEN"],
owner="my-org",
repo="my-repo",
)
resource = GitHubCIResource(config=config)
ws = Workspace({"/ci": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/ci/
workflows/
<workflow-name>_<workflow-id>.json
runs/
<workflow-name>_<run-id>/
run.json
jobs/
<job-name>_<job-id>.json
<job-name>_<job-id>.log
annotations.jsonl
artifacts/
<artifact-name>_<artifact-id>.zip
```
Example:
```text
/ci/
workflows/
CI_12345678.json
Deploy_87654321.json
runs/
CI_9876543210/
run.json
jobs/
build_11111111.json
build_11111111.log
test_22222222.json
test_22222222.log
annotations.jsonl
artifacts/
coverage-report_55555.zip
```
### Workflows
`/ci/workflows/` lists all workflows defined in the repository.
Each `.json` file contains the workflow metadata (name, path, state).
### Runs
`/ci/runs/` lists recent workflow runs within the configured time window
(default 30 days). The `created` API parameter filters server-side.
Each run directory contains:
- `run.json` - run metadata (status, conclusion, event, branch, actor, timing)
- `jobs/` - one `.json` and `.log` pair per job
- `annotations.jsonl` - check annotations (warnings, errors) across all jobs
- `artifacts/` - downloadable build artifacts
### Jobs
Each job has two files:
- `<job-name>_<job-id>.json` - job metadata with steps and timing
- `<job-name>_<job-id>.log` - full job log (plain text)
### Artifacts
Artifacts are served as `.zip` files matching the GitHub API download format.
## Cache
The GitHub CI resource uses `IndexCacheStore` with `remote_time`-based
fingerprinting. Completed runs and jobs are effectively immutable and
benefit from long cache TTL.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.github_ci import GitHubCIConfig, GitHubCIResource
load_dotenv(".env.development")
async def main():
config = GitHubCIConfig(
token=os.environ["GITHUB_TOKEN"],
owner="my-org",
repo="my-repo",
)
resource = GitHubCIResource(config=config)
ws = Workspace({"/ci": resource}, mode=MountMode.READ)
# List top-level
r = await ws.execute("ls /ci/")
print(await r.stdout_str())
# List workflows
r = await ws.execute("ls /ci/workflows/")
print(await r.stdout_str())
# List recent runs
r = await ws.execute("ls /ci/runs/")
print(await r.stdout_str())
# Read run metadata
r = await ws.execute("cat /ci/runs/CI_9876543210/run.json")
print(await r.stdout_str())
# List jobs for a run
r = await ws.execute("ls /ci/runs/CI_9876543210/jobs/")
print(await r.stdout_str())
# Read job log
r = await ws.execute("cat /ci/runs/CI_9876543210/jobs/build_11111111.log")
print(await r.stdout_str())
# Read annotations
r = await ws.execute("cat /ci/runs/CI_9876543210/annotations.jsonl")
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 2 /ci/")
print(await r.stdout_str())
# Find all failed job logs
r = await ws.execute("find /ci/runs/ -name '*.log'")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
Standard commands available on the mounted GitHub CI tree:
| Command | Notes |
| --------------- | ------------------------------ |
| `ls` | List workflows, runs, jobs |
| `cat` | Read JSON metadata or job logs |
| `head` / `tail` | First/last N lines |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (type, IDs) |
| `find` | Recursive search with `-name` |
| `tree` | Directory tree view |
## Working with CI Data
```bash
# Check run status via jq
cat /ci/runs/CI_9876543210/run.json | jq '.conclusion'
# List all job names
ls /ci/runs/CI_9876543210/jobs/ | grep '.json$'
# Tail a job log for recent output
tail -n 50 /ci/runs/CI_9876543210/jobs/build_11111111.log
# Count annotations
wc -l /ci/runs/CI_9876543210/annotations.jsonl
# Find all .log files
find /ci/ -name "*.log"
```
+337
View File
@@ -0,0 +1,337 @@
---
title: Gmail
description: Mount Gmail mailboxes as a Mirage filesystem so Python agents can browse labels, messages, threads, and attachments.
icon: envelope
---
The Gmail resource exposes a Gmail account as a virtual filesystem
mounted at some prefix such as `/gmail/`.
For Google OAuth setup, see [Google Workspace Setup](/python/setup/google).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
ws = Workspace({"/gmail": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/gmail/
<label>/
<yyyy-mm-dd>/
<sanitized-subject>__<message-id>.gmail.json
<sanitized-subject>__<message-id>/ # only if message has attachments
<attachment-filename>
...
```
Example:
```text
/gmail/
INBOX/
2026-04-12/
Meeting_Notes__msg123.gmail.json
Meeting_Notes__msg123/
report.pdf
screenshot.png
Simple_Email__msg456.gmail.json
2026-04-11/
Hello__msg789.gmail.json
SENT/
2026-04-12/
Reply__msg012.gmail.json
STARRED/
My_Custom_Label/
```
Label directories appear at the root. System labels use the Gmail
label ID (e.g., `INBOX`, `SENT`, `STARRED`). User-created labels
use their label name with spaces replaced by underscores.
### Date Directories
Inside each label, messages are grouped into date subdirectories
formatted as `YYYY-MM-DD`. The date is derived from the message's
`internalDate` (epoch milliseconds) converted to a calendar date.
### Message Files
Each message is stored as a `.gmail.json` file. The filename shape is:
```text
<sanitized-subject>__<message-id>.gmail.json
```
Subjects are sanitized for filesystem safety and truncated when
necessary. The message ID is embedded after `__` and before
`.gmail.json`.
### Attachments
Messages that have attachments get a companion subdirectory with
the same base name (without `.gmail.json`). Decoded attachment
files are placed inside:
```text
Meeting_Notes__msg123.gmail.json # message JSON
Meeting_Notes__msg123/ # attachment directory
report.pdf
screenshot.png
```
## Cache
The Gmail resource uses `IndexCacheStore` (same as Slack, Discord,
and other resources). Index entries store label IDs, message IDs,
and message metadata. There is no separate content cache -- file
content caching is handled by the workspace `IOResult` mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
async def main():
ws = Workspace({"/gmail": resource}, mode=MountMode.READ)
# List labels
r = await ws.execute("ls /gmail/")
print(await r.stdout_str())
# List date directories in INBOX
r = await ws.execute("ls /gmail/INBOX/")
print(await r.stdout_str())
# List messages for a specific date
r = await ws.execute("ls /gmail/INBOX/2026-04-12/")
print(await r.stdout_str())
# Read a message
r = await ws.execute(
"cat /gmail/INBOX/2026-04-12/Meeting_Notes__msg123.gmail.json")
print(await r.stdout_str())
# Extract subject with jq
r = await ws.execute(
'jq ".subject"'
" /gmail/INBOX/2026-04-12/Meeting_Notes__msg123.gmail.json")
print(await r.stdout_str())
# List attachments
r = await ws.execute("ls /gmail/INBOX/2026-04-12/Meeting_Notes__msg123/")
print(await r.stdout_str())
# Search across all messages
r = await ws.execute('rg "quarterly" /gmail/INBOX/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 2 /gmail/INBOX/")
print(await r.stdout_str())
# Triage unread messages
r = await ws.execute('gws-gmail-triage --query "is:unread" --max 5')
print(await r.stdout_str())
# Send an email
r = await ws.execute(
'gws-gmail-send --to "user@example.com"'
' --subject "Hello from MIRAGE"'
' --body "This email was sent via the MIRAGE Gmail resource."')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/gmail/gmail.py` for the full working example.
## Finding IDs
Resource-specific commands require message IDs. These can be
extracted from the filesystem:
```bash
# Message ID -- embedded in filename after "__"
ls /gmail/INBOX/2026-04-12/
# -> Meeting_Notes__msg123.gmail.json <- message_id = msg123
# Extract message ID from filename
basename /gmail/INBOX/2026-04-12/Meeting_Notes__msg123.gmail.json .gmail.json
# -> Meeting_Notes__msg123
# The part after "__" is the message ID: msg123
# Read a message then reply
gws-gmail-read --id msg123
gws-gmail-reply --message-id msg123 --body "Thanks for the notes"
```
## Working with Large Labels
Labels with many messages are split into date directories. Tips
for efficient access:
```bash
# List available dates
ls /gmail/INBOX/
# Check message count for a specific date
ls /gmail/INBOX/2026-04-12/ | wc -l
# Read only the most recent date
ls /gmail/INBOX/ | tail -n 1
# Search across all dates in a label
rg "keyword" /gmail/INBOX/
# Extract specific fields to reduce output
jq -r '.subject' /gmail/INBOX/2026-04-12/*.gmail.json | head -n 20
# Find messages with attachments
find /gmail/INBOX/2026-04-12/ -type d
# Tree view of a label
tree -L 2 /gmail/INBOX/
```
## Shell Commands
Standard commands available on the mounted Gmail tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List labels, dates, messages, attachments |
| `cat` | Read message JSON or attachment content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query message JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `basename` | Extract filename from path |
| `dirname` | Extract directory from path |
| `realpath` | Resolve path to absolute form |
| `nl` | Number lines of output |
Resource-specific commands:
### `gws-gmail-send`
Send a new email.
```bash
gws-gmail-send --to "user@example.com" --subject "Hello" --body "Hi there"
```
| Option | Required | Description |
| ----------- | -------- | ----------------------- |
| `--to` | yes | Recipient email address |
| `--subject` | yes | Email subject line |
| `--body` | yes | Email body text |
Returns the sent message JSON.
### `gws-gmail-reply`
Reply to a message.
```bash
gws-gmail-reply --message-id msg123 --body "Thanks for the update"
```
| Option | Required | Description |
| -------------- | -------- | ---------------- |
| `--message-id` | yes | Gmail message ID |
| `--body` | yes | Reply body text |
Returns the sent reply JSON.
### `gws-gmail-reply-all`
Reply-all to a message.
```bash
gws-gmail-reply-all --message-id msg123 --body "Acknowledged by the team"
```
| Option | Required | Description |
| -------------- | -------- | ---------------- |
| `--message-id` | yes | Gmail message ID |
| `--body` | yes | Reply body text |
Returns the sent reply JSON.
### `gws-gmail-forward`
Forward a message to another recipient.
```bash
gws-gmail-forward --message-id msg123 --to "colleague@example.com"
```
| Option | Required | Description |
| -------------- | -------- | ----------------------- |
| `--message-id` | yes | Gmail message ID |
| `--to` | yes | Recipient email address |
Returns the forwarded message JSON.
### `gws-gmail-triage`
Search and triage emails using Gmail query syntax.
```bash
gws-gmail-triage --query "is:unread" --max 10
```
| Option | Required | Description |
| --------- | -------- | ----------------------------------- |
| `--query` | yes | Gmail search query |
| `--max` | no | Maximum number of results to return |
Returns matching messages as JSON.
### `gws-gmail-read`
Read a message by its ID.
```bash
gws-gmail-read --id msg123
```
| Option | Required | Description |
| ------ | -------- | ---------------- |
| `--id` | yes | Gmail message ID |
Returns the full message JSON.
+239
View File
@@ -0,0 +1,239 @@
---
title: Google Sheets
description: Mount Google Sheets spreadsheets as JSON-backed files that Python agents can read through Mirage shell commands.
icon: table
---
The Google Sheets resource exposes Sheets spreadsheets as a virtual
filesystem mounted at some prefix such as `/gsheets/`.
For Google OAuth setup, see [Google Workspace Setup](/python/setup/google).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
ws = Workspace({"/gsheets": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/gsheets/
owned/
<YYYY-MM-DD>_<sanitized-title>__<spreadsheet-id>.gsheet.json
...
shared/
<YYYY-MM-DD>_<sanitized-title>__<spreadsheet-id>.gsheet.json
...
```
Example:
```text
/gsheets/
owned/
2026-04-04_Roadmap__1AbCdEf.gsheet.json
2026-04-10_Expenses__2BcDeFg.gsheet.json
shared/
2026-04-03_Budget__9XyZ.gsheet.json
```
Spreadsheets are split into `owned` (spreadsheets you created) and
`shared` (spreadsheets shared with you). The filename shape is:
```text
<YYYY-MM-DD>_<sanitized-title>__<spreadsheet-id>.gsheet.json
```
If the modified date is unavailable, the date prefix is omitted.
Reading a spreadsheet file returns the full Google Sheets API JSON
for that spreadsheet, including sheet metadata.
## Cache
The Google Sheets resource uses `IndexCacheStore`. Index entries
store spreadsheet IDs and metadata. There is no separate content
cache -- file content caching is handled by the workspace `IOResult`
mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
load_dotenv(".env.development")
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
async def main():
ws = Workspace({"/gsheets": resource}, mode=MountMode.READ)
# List structure
r = await ws.execute("ls /gsheets/")
print(await r.stdout_str())
# List owned spreadsheets
r = await ws.execute("ls /gsheets/owned/")
print(await r.stdout_str())
# Read a spreadsheet
r = await ws.execute(
"cat /gsheets/owned/2026-04-04_Roadmap__1AbCdEf.gsheet.json")
print(await r.stdout_str())
# Extract title with jq
r = await ws.execute(
'jq ".properties.title"'
" /gsheets/owned/2026-04-04_Roadmap__1AbCdEf.gsheet.json")
print(await r.stdout_str())
# Search across all spreadsheets
r = await ws.execute('rg "revenue" /gsheets/owned/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 1 /gsheets/")
print(await r.stdout_str())
# Read cell values
r = await ws.execute(
'gws-sheets-read --spreadsheet 1AbCdEf --range "Sheet1!A1:C3"')
print(await r.stdout_str())
# Append rows
r = await ws.execute(
"gws-sheets-append --spreadsheet 1AbCdEf --values Alice,30,NYC")
print(await r.stdout_str())
# Create a new spreadsheet
r = await ws.execute(
'gws-sheets-spreadsheets-create'
' --json \'{"properties":{"title":"MIRAGE Sheet"}}\'')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/gsheets/gsheets.py` for the full working example.
## Shell Commands
Standard commands available on the mounted Google Sheets tree:
| Command | Notes |
| ----------------------------------- | ------------------------------ |
| `ls` | List owned/shared spreadsheets |
| `cat` | Read spreadsheet JSON |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata |
| `find` | Recursive search |
| `tree` | Directory tree view |
| `basename` / `dirname` / `realpath` | Path utilities |
| `nl` | Number lines |
Resource-specific commands:
### `gws-sheets-read`
Read cell values from a spreadsheet range.
```bash
gws-sheets-read --spreadsheet 1AbCdEf --range "Sheet1!A1:C3"
```
| Option | Required | Description |
| --------------- | -------- | ------------------------- |
| `--spreadsheet` | yes | Google spreadsheet ID |
| `--range` | yes | A1 notation range to read |
Returns the cell values as JSON.
### `gws-sheets-write`
Write cell values to a spreadsheet range.
```bash
gws-sheets-write --params '{"spreadsheetId":"1AbCdEf","range":"Sheet1!A1"}' --json '{"values":[["x"]]}'
```
| Option | Required | Description |
| ---------- | -------- | --------------------------------------------------------------- |
| `--params` | yes | JSON with `spreadsheetId`, `range`, optional `valueInputOption` |
| `--json` | yes | JSON with `values` 2D array |
Returns the update response JSON.
### `gws-sheets-append`
Append rows to a spreadsheet.
```bash
gws-sheets-append --spreadsheet 1AbCdEf --values Alice,30,NYC
gws-sheets-append --spreadsheet 1AbCdEf --range "Sheet1!A1" --json-values '[["Bob",25,"LA"]]'
```
| Option | Required | Description |
| --------------- | -------- | ------------------------------------ |
| `--spreadsheet` | yes | Google spreadsheet ID |
| `--range` | no | A1 notation range (defaults to `A1`) |
| `--values` | no | Comma-separated values for one row |
| `--json-values` | no | JSON 2D array of values |
Either `--values` or `--json-values` is required. Returns the
append response JSON.
### `gws-sheets-spreadsheets-create`
Create a new spreadsheet.
```bash
gws-sheets-spreadsheets-create --json '{"properties":{"title":"MIRAGE Sheet"}}'
```
| Option | Required | Description |
| -------- | -------- | --------------------------------------- |
| `--json` | yes | JSON body with `properties.title` field |
Returns the created spreadsheet JSON.
### `gws-sheets-spreadsheets-batchUpdate`
Batch update a spreadsheet.
```bash
gws-sheets-spreadsheets-batchUpdate --params '{"spreadsheetId":"1AbCdEf"}' --json '{"requests":[]}'
```
| Option | Required | Description |
| ---------- | -------- | -------------------------- |
| `--params` | yes | JSON with `spreadsheetId` |
| `--json` | yes | JSON with `requests` array |
Returns the batch update response JSON.
+186
View File
@@ -0,0 +1,186 @@
---
title: Google Slides
description: Mount Google Slides presentations as JSON-backed files that Python agents can read through Mirage shell commands.
icon: presentation
---
The Google Slides resource exposes presentations as a virtual
filesystem mounted at some prefix such as `/gslides/`.
For Google OAuth setup, see [Google Workspace Setup](/python/setup/google).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
ws = Workspace({"/gslides": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/gslides/
owned/
<YYYY-MM-DD>_<sanitized-title>__<presentation-id>.gslide.json
...
shared/
<YYYY-MM-DD>_<sanitized-title>__<presentation-id>.gslide.json
...
```
Example:
```text
/gslides/
owned/
2026-04-04_QBR_Deck__1AbCdEf.gslide.json
2026-04-10_Team_Update__2BcDeFg.gslide.json
shared/
2026-04-03_Design_Review__9XyZ.gslide.json
```
Presentations are split into `owned` (presentations you created)
and `shared` (presentations shared with you). The filename shape is:
```text
<YYYY-MM-DD>_<sanitized-title>__<presentation-id>.gslide.json
```
If the modified date is unavailable, the date prefix is omitted.
Reading a presentation file returns the full Google Slides API JSON
for that presentation.
## Cache
The Google Slides resource uses `IndexCacheStore`. Index entries
store presentation IDs and metadata. There is no separate content
cache -- file content caching is handled by the workspace `IOResult`
mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
load_dotenv(".env.development")
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
async def main():
ws = Workspace({"/gslides": resource}, mode=MountMode.READ)
# List structure
r = await ws.execute("ls /gslides/")
print(await r.stdout_str())
# List owned presentations
r = await ws.execute("ls /gslides/owned/")
print(await r.stdout_str())
# Read a presentation
r = await ws.execute(
"cat /gslides/owned/2026-04-04_QBR_Deck__1AbCdEf.gslide.json")
print(await r.stdout_str())
# Extract title with jq
r = await ws.execute(
'jq ".title"'
" /gslides/owned/2026-04-04_QBR_Deck__1AbCdEf.gslide.json")
print(await r.stdout_str())
# Search across all presentations
r = await ws.execute('rg "quarterly" /gslides/owned/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 1 /gslides/")
print(await r.stdout_str())
# Create a new presentation
r = await ws.execute(
'gws-slides-presentations-create --json \'{"title":"MIRAGE Deck"}\'')
print(await r.stdout_str())
# Batch update a presentation
r = await ws.execute(
'gws-slides-presentations-batchUpdate'
' --params \'{"presentationId":"1AbCdEf"}\''
' --json \'{"requests":[]}\'')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/gslides/gslides.py` for the full working example.
## Shell Commands
Standard commands available on the mounted Google Slides tree:
| Command | Notes |
| ----------------------------------- | ------------------------------- |
| `ls` | List owned/shared presentations |
| `cat` | Read presentation JSON |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata |
| `find` | Recursive search |
| `tree` | Directory tree view |
| `basename` / `dirname` / `realpath` | Path utilities |
| `nl` | Number lines |
Resource-specific commands:
### `gws-slides-presentations-create`
Create a new presentation.
```bash
gws-slides-presentations-create --json '{"title":"MIRAGE Deck"}'
```
| Option | Required | Description |
| -------- | -------- | ---------------------------- |
| `--json` | yes | JSON body with `title` field |
Returns the created presentation JSON.
### `gws-slides-presentations-batchUpdate`
Batch update a presentation.
```bash
gws-slides-presentations-batchUpdate --params '{"presentationId":"1AbCdEf"}' --json '{"requests":[]}'
```
| Option | Required | Description |
| ---------- | -------- | -------------------------- |
| `--params` | yes | JSON with `presentationId` |
| `--json` | yes | JSON with `requests` array |
Returns the batch update response JSON.
+252
View File
@@ -0,0 +1,252 @@
---
title: HF Buckets
description: Mount Hugging Face Buckets as a Mirage filesystem with async reads and S3-compatible writes for Python agents.
icon: /images/huggingface-logo.svg
---
The HF Buckets resource mounts a [Hugging Face Bucket](https://huggingface.co/docs/buckets)
at some prefix such as `/hf/`. Speaks HF's HTTP API natively (async,
streaming, no Python SDK dependency).
For credential setup, see [HF Buckets Setup](/home/setup/hf_buckets).
## Install
```bash
uv add "mirage-ai[hf]"
```
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.hf_buckets import HfBucketsConfig, HfBucketsResource
config = HfBucketsConfig(
bucket=os.environ["HF_BUCKET_NAME"], # "namespace/bucket-name"
token=os.environ["HF_TOKEN"],
# Optional:
# endpoint="https://huggingface.co",
# timeout=30,
# key_prefix="data/",
)
resource = HfBucketsResource(config)
ws = Workspace({"/hf": resource}, mode=MountMode.READ)
```
`HfBucketsResource(config)` takes an `HfBucketsConfig` object with the
bucket in `namespace/bucket-name` form plus an optional access token.
Both `READ` and `WRITE` modes are supported out of the box.
## Filesystem Layout
The HF Buckets resource maps bucket object keys to virtual paths under
the mount prefix.
For example, if bucket `your-user/my-data` contains:
```text
data/file.txt
data/config.json
reports/q1.csv
reports/q2.csv
```
Then mounting at `/hf/` exposes:
```text
/hf/
data/
file.txt
config.json
reports/
q1.csv
q2.csv
```
Path mapping: virtual `/hf/data/file.txt` maps to bucket key
`data/file.txt`.
## Cache
The HF Buckets resource uses `IndexCacheStore` with `index_ttl = 600`
(10 minutes). Directory listings are cached and populate file-size/type
entries that `stat` reads via a fast path, so a `readdir` followed by
per-entry `stat` calls (which is what `ls`, FUSE `getattr`, and most
shell commands trigger) costs one HTTP request instead of N.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_buckets import HfBucketsConfig, HfBucketsResource
load_dotenv(".env.development")
config = HfBucketsConfig(
bucket=os.environ["HF_BUCKET_NAME"],
token=os.environ["HF_TOKEN"],
)
resource = HfBucketsResource(config)
async def main() -> None:
ws = Workspace({"/hf/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /hf/")
print(await r.stdout_str())
r = await ws.execute("cat /hf/data/file.txt")
print(await r.stdout_str())
r = await ws.execute("tree /hf/")
print(await r.stdout_str())
r = await ws.execute("find /hf/ -name '*.json'")
print(await r.stdout_str())
r = await ws.execute("grep example /hf/data/config.json")
print(await r.stdout_str())
r = await ws.execute("stat /hf/data/file.txt")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The HF Buckets resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit
from range reads to avoid downloading entire objects.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `rm` | Remove files |
| `touch` | Create empty file or update timestamp |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **AI agents accessing HF datasets**: Mount HF Buckets for agents to read and process datasets stored on the Hub
- **Data pipelines**: Read and write HF bucket objects with shell-like commands
- **Sandboxed bucket access**: Restrict agent operations to a specific bucket and prefix
- **FUSE mounting**: Expose HF Buckets through a virtual FUSE mount for external tools
## Scoping a resource to a key prefix
Pass `key_prefix: str | None = None` to `HfBucketsConfig` to transparently scope every operation to a subpath of the bucket:
```python
HfBucketsResource(HfBucketsConfig(
bucket="your-user/app-data",
token=hf_token,
key_prefix=f"users/{user_id}/",
))
```
When set, every read/write/list/stat operation is transparently scoped to that bucket subpath. Agents see clean paths like `/data/notes.md`; the underlying bucket key is `users/{user_id}/data/notes.md`. Useful for multi-tenant systems.
**Normalization:** leading slashes are stripped and a trailing slash is added automatically. Both `None` and an empty string are treated as "no prefix."
+119
View File
@@ -0,0 +1,119 @@
---
title: HF Datasets
icon: /images/huggingface-logo.svg
---
The HF Datasets resource mounts a [Hugging Face Dataset](https://huggingface.co/datasets)
repo at some prefix such as `/ds/`.
All reads are lazy: only the bytes you actually `cat`/`head` get transferred.
For credential setup, see [HF Datasets Setup](/home/setup/hf_datasets).
## Install
```bash
uv add "mirage-ai[hf]"
```
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.hf_datasets import HfDatasetsConfig, HfDatasetsResource
config = HfDatasetsConfig(
repo_id=os.environ["HF_DATASET_REPO"], # "namespace/dataset-name"
token=os.environ.get("HF_TOKEN"),
# Optional:
# endpoint="https://huggingface.co",
# revision="main",
# key_prefix="train/",
)
resource = HfDatasetsResource(config)
ws = Workspace({"/ds": resource}, mode=MountMode.READ)
```
`HfDatasetsConfig` takes `repo_id` in `namespace/dataset-name` form plus an
optional access token. Public datasets need no token.
## Filesystem Layout
Maps dataset repo files to virtual paths under the mount prefix.
For example, if dataset `AlienKevin/SWE-ZERO-12M-trajectories` contains:
```text
README.md
data/train-00000-of-01000.parquet
data/train-00001-of-01000.parquet
```
Then mounting at `/ds/` exposes:
```text
/ds/
README.md
data/
train-00000-of-01000.parquet
train-00001-of-01000.parquet
```
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_datasets import HfDatasetsConfig, HfDatasetsResource
load_dotenv(".env.development")
config = HfDatasetsConfig(
repo_id=os.environ.get("HF_DATASET_REPO",
"AlienKevin/SWE-ZERO-12M-trajectories"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfDatasetsResource(config)
async def main() -> None:
ws = Workspace({"/ds": resource}, mode=MountMode.READ)
r = await ws.execute("ls /ds/")
print(await r.stdout_str())
r = await ws.execute("cat /ds/README.md | head -n 20")
print(await r.stdout_str())
r = await ws.execute("find /ds/ -name '*.parquet' | head -n 5")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
Same set as [HF Buckets](/python/resource/hf_buckets#shell-commands) — read,
text-processing, file ops, path utilities, compression, encoding, and
format-specific variants for parquet/feather/orc/hdf5.
## Cache
Uses `IndexCacheStore` with `index_ttl = 600` (10 minutes). Directory
listings are cached and populate file-size/type entries for `stat`'s fast
path, so a `readdir` + per-entry `stat` (which `ls`, FUSE `getattr`, and
most shell commands trigger) costs one HTTP request instead of N.
## Use Cases
- **AI agents inspecting datasets**: Mount, browse the README, sample a few
rows from parquet shards without downloading the whole dataset
- **Dataset triage**: `ls`, `stat`, `find` to see what's in a repo before
committing to a full local copy
- **Sandboxed access**: Pin a `revision` for reproducibility
+94
View File
@@ -0,0 +1,94 @@
---
title: HF Models
icon: /images/huggingface-logo.svg
---
The HF Models resource mounts a [Hugging Face Model](https://huggingface.co/models)
repo at some prefix such as `/m/`.
You can inspect configs, tokenizers, and READMEs without downloading the
weights, weights stream only when you actually `cat` them.
For credential setup, see [HF Models Setup](/home/setup/hf_models).
## Install
```bash
uv add "mirage-ai[hf]"
```
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.hf_models import HfModelsConfig, HfModelsResource
config = HfModelsConfig(
repo_id=os.environ["HF_MODEL_REPO"], # "namespace/model-name"
token=os.environ.get("HF_TOKEN"),
# Optional:
# endpoint="https://huggingface.co",
# revision="main",
)
resource = HfModelsResource(config)
ws = Workspace({"/m": resource}, mode=MountMode.READ)
```
## Filesystem Layout
Maps model repo files (config, tokenizer, weights, etc.) to virtual paths.
For example, `sapientinc/HRM-Text-1B` mounted at `/m/` exposes:
```text
/m/
README.md
config.json
tokenizer.json
tokenizer_config.json
model.safetensors ← never downloaded unless you cat it
```
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.hf_models import HfModelsConfig, HfModelsResource
config = HfModelsConfig(repo_id="sapientinc/HRM-Text-1B")
resource = HfModelsResource(config)
async def main() -> None:
ws = Workspace({"/m": resource}, mode=MountMode.READ)
# ls is cheap: one HTTP list call
r = await ws.execute("ls -lh /m/")
print(await r.stdout_str())
# Read the config (small, fast)
r = await ws.execute("cat /m/config.json")
print(await r.stdout_str())
# Stat the weights without downloading them
r = await ws.execute("stat /m/model.safetensors")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
Same set as [HF Buckets](/python/resource/hf_buckets#shell-commands).
## Use Cases
- **Model card inspection**: Read configs, tokenizers, READMEs without
pulling multi-GB weight files
- **Compatibility checks**: `jq` over `config.json` to verify architecture
before committing to a download
- **Pinned revisions**: Mount a specific commit/tag for reproducibility
+84
View File
@@ -0,0 +1,84 @@
---
title: HF Spaces
icon: /images/huggingface-logo.svg
---
The HF Spaces resource mounts a [Hugging Face Space](https://huggingface.co/spaces)
repo (app code, README, config, requirements) at some prefix such as `/s/`.
For credential setup, see [HF Spaces Setup](/home/setup/hf_spaces).
## Install
```bash
uv add "mirage-ai[hf]"
```
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.hf_spaces import HfSpacesConfig, HfSpacesResource
config = HfSpacesConfig(
repo_id=os.environ["HF_SPACE_REPO"], # "namespace/space-name"
token=os.environ.get("HF_TOKEN"),
# Optional:
# endpoint="https://huggingface.co",
# revision="main",
)
resource = HfSpacesResource(config)
ws = Workspace({"/s": resource}, mode=MountMode.READ)
```
## Filesystem Layout
For example, `HuggingFaceBio/carbon-demo` mounted at `/s/` might expose:
```text
/s/
README.md
app.py
requirements.txt
Dockerfile
```
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.hf_spaces import HfSpacesConfig, HfSpacesResource
async def main() -> None:
resource = HfSpacesResource(
HfSpacesConfig(repo_id="HuggingFaceBio/carbon-demo"))
ws = Workspace({"/s": resource}, mode=MountMode.READ)
r = await ws.execute("ls /s/")
print(await r.stdout_str())
r = await ws.execute("cat /s/README.md | head -n 20")
print(await r.stdout_str())
r = await ws.execute("grep -l import /s/*.py 2>/dev/null")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
Same set as [HF Buckets](/python/resource/hf_buckets#shell-commands).
## Use Cases
- **Space introspection**: Browse an app's structure, requirements, and
config before forking/cloning
- **App code review**: `grep` / `find` across a Space's Python without
cloning the repo
+68
View File
@@ -0,0 +1,68 @@
---
title: Resources
icon: grid-2
description: Python resources you can mount in a Mirage workspace.
---
Each page describes a Python-supported resource: what config it needs, what the mounted tree looks like, and any resource-specific behavior. Setup docs (credentials, OAuth, etc.) live under [Setup](/home/setup/s3), these pages focus on mounted layout and behavior.
## Infrastructure
- [Disk](/python/resource/disk)
- [RAM](/python/resource/ram)
- [Redis](/python/resource/redis)
- [SSH](/python/resource/ssh)
## Object Storage
- [S3](/python/resource/s3)
- [R2](/python/resource/r2)
- [GCS](/python/resource/gcs)
- [OCI](/python/resource/oci)
- [Supabase](/python/resource/supabase)
## Cloud Files
- [Databricks Volume](/python/resource/databricks_volume)
## Google Workspace
- [Gmail](/python/resource/gmail)
- [Google Drive](/python/resource/gdrive)
- [Google Docs](/python/resource/gdocs)
- [Google Sheets](/python/resource/gsheets)
- [Google Slides](/python/resource/gslides)
## Code & DevOps
- [GitHub](/python/resource/github)
- [GitHub CI](/python/resource/github_ci)
- [Linear](/python/resource/linear)
- [Langfuse](/python/resource/langfuse)
## Messaging
- [Slack](/python/resource/slack)
- [Discord](/python/resource/discord)
- [Email](/python/resource/email)
## Database
- [MongoDB](/python/resource/mongodb)
- [Postgres](/python/resource/postgres)
- [LanceDB](/python/resource/lancedb)
- [Qdrant](/python/resource/qdrant)
## Knowledge
- [Dify](/python/resource/dify)
- [Chroma](/python/resource/chroma)
## Notes
- [Notion](/python/resource/notion)
## Others
- [Trello](/python/resource/trello)
- [Adding a New Resource](/python/resource/new)
+137
View File
@@ -0,0 +1,137 @@
---
title: LanceDB
description: Mount a LanceDB table as a Mirage filesystem, with label folders, multimodal blobs, and a semantic search command for Python agents.
icon: database
---
The LanceDB resource exposes a LanceDB table as a virtual filesystem mounted at
some prefix such as `/fashion/`. Group-by columns become nested folders, each
row becomes a card plus an optional blob file, and semantic search is the
`search` command, which returns ranked rows as canonical file paths.
For connection setup (LanceDB OSS, object storage, Cloud, Enterprise), see
[LanceDB Setup](/python/setup/lancedb).
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.lancedb import LanceDBConfig, LanceDBResource
config = LanceDBConfig(
uri="/data/fashion.lancedb",
table="fashion",
group_by=["gender", "articleType", "baseColour"],
id_column="id",
title_column="productDisplayName",
blob_column="image_bytes",
blob_ext="jpg",
vector_column="vector",
search_limit=5,
)
resource = LanceDBResource(config)
ws = Workspace({"/fashion/": resource}, mode=MountMode.READ)
```
The mapping is config-driven; nothing about the dataset is hardcoded. Point
`group_by` at different columns and the folder tree changes. See the full
[config reference](/python/setup/lancedb#config-reference).
## Filesystem layout
Every path is translated into a LanceDB query. Descending a folder adds one
`WHERE` clause; the leaf level lists rows.
```text
/ # list tables (omitted when `table` is pinned)
/<table>/ # distinct group_by[0] values
<v1>/ # distinct group_by[1] WHERE group_by[0]=v1
.../<vN>/ # all group-by columns bound -> row files
<id>.md # rendered card (text)
<id>.<ext> # raw blob / image bytes
```
When `table` is set the table level is elided, so the mount root is that table:
```text
/fashion/
Men/
Shoes/
White/
3.md
3.jpg
```
### Row cards
A `<id>.md` card renders the row's columns as readable text and points at its
blob. The vector and blob columns are omitted from the card body.
```text
# Nike Men White Running Sneakers
id: 3
gender: Men
articleType: Shoes
baseColour: White
productDisplayName: Nike Men White Running Sneakers
blob: 3.jpg
```
## Semantic search
Search is a command, not a path. It returns each ranked row as its **canonical
file path** (the same `<id>.md` you would `cat` while browsing) annotated with
the vector distance, followed by the card body. Results point back at the real
files, so search composes with `cat`, pipes, and `wc`:
```text
$ search "white running sneakers" /fashion
/fashion/Men/Shoes/White/3.md:0.2679
# Nike Men White Running Sneakers
id: 3
gender: Men
articleType: Shoes
baseColour: White
productDisplayName: Nike Men White Running Sneakers
blob: 3.jpg
...
```
Flags: `--top-k <n>` (default `search_limit`), `--threshold <max-distance>`,
`--method semantic` (the only supported method; `grep`/`rg` stay lexical).
## Supported commands
All commands delegate to Mirage's shared implementations.
| Command | Behaviour on a LanceDB mount |
| ------------- | ------------------------------------------------------------- |
| `ls` | list tables, label folders, or row files |
| `cd` | navigate (each level narrows the filter) |
| `tree` | render the label hierarchy |
| `cat` | print a row card, or dump raw blob/image bytes |
| `stat` | directory vs file, blob size, image mime type |
| `find` | walk the tree (e.g. `find /fashion -name '*.md'`) |
| `grep` / `rg` | lexical search over the rendered cards |
| `search` | semantic (vector) search -> ranked canonical paths + score |
| `head` / `tail` | first/last lines of a card |
| `wc` | count lines/bytes of a card |
`grep`/`rg` stay lexical (literal/regex). `search` is the semantic path: it
auto-embeds the query via the table's embedding function and returns ranked
rows as canonical file paths, which compose with `cat`, `wc`, and pipes.
## Access pattern
The mount is **read-only** (`MountMode.READ`); writes are not supported. The two
read modes are:
- **Browse** by label folders: pure metadata `WHERE` filters, no embedding.
- **Search** by meaning: `search "<query>" <path>` runs vector search using the
table's embedding function and returns canonical row paths.
Folder listings scan one column with `SELECT DISTINCT` and are capped by
`max_rows`, so very large tables should keep `group_by` to low-cardinality
columns.
+227
View File
@@ -0,0 +1,227 @@
---
title: Langfuse
description: Mount Langfuse traces, observations, prompts, datasets, and scores as a Mirage virtual filesystem.
icon: chart-line
---
The Langfuse resource exposes LLM observability data (traces, observations,
prompts, datasets, scores) as a virtual filesystem mounted at some prefix
such as `/langfuse/`.
For credential setup, see [Langfuse Setup](/python/setup/langfuse).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.langfuse import LangfuseConfig, LangfuseResource
config = LangfuseConfig(
public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
secret_key=os.environ["LANGFUSE_SECRET_KEY"],
host=os.environ.get("LANGFUSE_HOST", "https://cloud.langfuse.com"),
)
resource = LangfuseResource(config=config)
ws = Workspace({"/langfuse": resource}, mode=MountMode.READ)
```
| Config field | Required | Default | Description |
| ---------------------- | -------- | ---------------------------- | ------------------------------------------------ |
| `public_key` | yes | | Langfuse project public key |
| `secret_key` | yes | | Langfuse project secret key |
| `host` | no | `https://cloud.langfuse.com` | Langfuse API host URL |
| `default_trace_limit` | no | 100 | Default trace/observation cap for listings |
| `default_search_limit` | no | 50 | Default result cap for search at directory scope |
## Filesystem Layout
```text
/langfuse/
traces/
<trace-id>.json
...
sessions/
<session-id>/
<trace-id>.json
...
prompts/
<prompt-name>/
<version>.json
...
datasets/
<dataset-name>/
items.jsonl
runs/
<run-name>.jsonl
...
```
Example:
```text
/langfuse/
traces/
abc123.json
def456.json
sessions/
chat-session-1/
abc123.json
ghi789.json
prompts/
summarize/
1.json
2.json
classify/
1.json
datasets/
eval-v1/
items.jsonl
runs/
run-2026-04-01.jsonl
```
### Traces
`/langfuse/traces/` lists recent traces (capped at `default_trace_limit`).
Each trace is a `.json` file containing the full trace object with nested
observations (spans, generations, events).
### Sessions
`/langfuse/sessions/` groups traces by session ID. Each session directory
contains trace `.json` files belonging to that session.
### Prompts
`/langfuse/prompts/` lists prompt names as directories. Each prompt directory
contains version files (`1.json`, `2.json`, etc.) with the prompt content
and metadata.
### Datasets
`/langfuse/datasets/` lists dataset names as directories. Each contains
`items.jsonl` (dataset items) and `runs/` with run results.
## Smart Commands
### grep at different scopes
```bash
# FILE level - downloads the file, greps locally
grep "error" "/langfuse/traces/abc123.json"
# TRACES level - uses Langfuse trace filter API (name, tags, metadata)
grep "error" "/langfuse/traces/"
# SESSIONS level - uses Langfuse session listing
grep "chat" "/langfuse/sessions/"
# PROMPTS level - filters prompt names
grep "classify" "/langfuse/prompts/"
# ROOT level - searches across all resource types
grep "error" "/langfuse/"
```
At directory scope, the resource uses Langfuse's filter API instead of
downloading data:
- **traces/**: filters by trace name, user ID, or tags
- **sessions/**: filters by session listing
- **prompts/**: filters by prompt name
### head / tail
`head` and `tail` on `traces/` use Langfuse API pagination with sort order
instead of downloading all traces:
```bash
# Returns 10 most recent traces
head -n 10 "/langfuse/traces/"
# Returns 10 oldest traces
tail -n 10 "/langfuse/traces/"
```
## Limits
| Command | Limit behavior |
| ------------------------ | ---------------------------------------------------- |
| `ls traces/` | Returns up to `default_trace_limit` traces |
| `cat traces/<id>.json` | Full trace (single doc, no limit needed) |
| `grep` (directory level) | Server-side filter, capped at `default_search_limit` |
| `wc -l traces/` | Not supported (Langfuse has no count endpoint) |
## Cache
The Langfuse resource uses `IndexCacheStore` (same as other resources).
Index entries store trace IDs, session IDs, and prompt names.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.langfuse import LangfuseConfig, LangfuseResource
load_dotenv(".env.development")
config = LangfuseConfig(
public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
secret_key=os.environ["LANGFUSE_SECRET_KEY"],
)
resource = LangfuseResource(config=config)
async def main():
ws = Workspace({"/langfuse": resource}, mode=MountMode.READ)
# List top-level resources
r = await ws.execute("ls /langfuse/")
print(await r.stdout_str())
# List recent traces
r = await ws.execute("ls /langfuse/traces/")
print(await r.stdout_str())
# Read a specific trace
traces = r.stdout_str().strip().splitlines()
if traces:
r = await ws.execute(f'cat "/langfuse/traces/{traces[0]}"')
print(r.stdout_str()[:500])
# List prompts
r = await ws.execute("ls /langfuse/prompts/")
print(await r.stdout_str())
# Search traces (uses Langfuse filter API)
r = await ws.execute('grep "error" "/langfuse/traces/"')
print(await r.stdout_str())
# List datasets
r = await ws.execute("ls /langfuse/datasets/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
| Command | Notes |
| --------------- | ------------------------------------------------------ |
| `ls` | List traces, sessions, prompts, datasets |
| `cat` | Read trace JSON, prompt version, dataset items |
| `head` / `tail` | Paginated trace listing with sort |
| `grep` / `rg` | Smart: uses Langfuse filter API at directory scope |
| `jq` | Query JSON trace/prompt data |
| `stat` | Metadata (trace count, prompt versions, dataset items) |
| `find` | Search across resources with `-name` |
| `tree` | Directory tree view |
+392
View File
@@ -0,0 +1,392 @@
---
title: Linear
description: Mount Linear workspaces, teams, projects, issues, and comments as a Mirage filesystem for Python agents.
icon: chart-gantt
---
The Linear resource exposes Linear workspace data as a virtual filesystem
mounted at some prefix such as `/linear/`.
For API key setup, see [Linear Setup](/python/setup/linear).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.linear import LinearConfig, LinearResource
config = LinearConfig(api_key=os.environ["LINEAR_API_KEY"])
resource = LinearResource(config=config)
ws = Workspace({"/linear": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/linear/
teams/
<team-key>__<team-name>__<team-id>/
team.json
members/
<display-name>__<user-id>.json
...
issues/
<issue-key>__<issue-id>/
issue.json
comments.jsonl
...
projects/
<name>__<project-id>.json
...
cycles/
<name>__<cycle-id>.json
...
```
Example:
```text
/linear/
teams/
ENG__Engineering__abc123def/
team.json
members/
alice__usr_001.json
bob__usr_002.json
issues/
ENG-123__iss_001/
issue.json
comments.jsonl
ENG-124__iss_002/
issue.json
comments.jsonl
projects/
Mirage__proj_001.json
cycles/
Sprint-12__cyc_001.json
```
Directory and file names embed the Linear ID after `__` so that
resource-specific commands can reference the correct resource without
extra lookups.
### Teams
Each team directory is named:
```text
<team-key>__<team-name>__<team-id>
```
Inside the team directory:
- `team.json` is the normalized team metadata
- `members/` contains one JSON file per member
- `issues/` contains one directory per issue
- `projects/` contains one JSON file per project, including lightweight
related issue references
- `cycles/` contains one JSON file per cycle
### Issues
Each issue directory is named:
```text
<issue-key>__<issue-id>
```
Each issue directory contains:
- `issue.json` -- normalized issue metadata with command-aligned fields such as
`issue_id`, `issue_key`, `team_id`, `state_id`, and `assignee_id`
- `comments.jsonl` -- normalized comment stream ordered by `created_at`
`comments.jsonl` is a Mirage representation chosen for shell-friendly
workflows. It is not a native Linear file format.
### Members
Each member file is named:
```text
<display-name>__<user-id>.json
```
Member JSON includes command-aligned identifiers such as `user_id` and `email`
so the value can be reused directly in commands like `linear-issue-assign`.
### Projects and Cycles
Project and cycle files are named:
```text
<name>__<id>.json
```
Project JSON includes lightweight related issue references. Cycle JSON
includes cycle metadata such as start and end dates.
## Cache
The Linear resource uses `IndexCacheStore` (same as Discord and other
resources). There is no separate content cache -- file content caching
is handled by the workspace `IOResult` mechanism.
## Example
See `examples/linear/linear.py` for the full working example.
```bash
# List teams
ls /linear/teams/
# Read team metadata
cat /linear/teams/ENG__Engineering__abc123/team.json
# List issues
ls /linear/teams/ENG__Engineering__abc123/issues/
# Read an issue
cat /linear/teams/ENG__Engineering__abc123/issues/ENG-123__iss_001/issue.json
# Read issue comments
cat /linear/teams/ENG__Engineering__abc123/issues/ENG-123__iss_001/comments.jsonl
# Extract issue ID with jq
cat /linear/teams/ENG__Engineering__abc123/issues/ENG-123__iss_001/issue.json \
| jq '.issue_id'
# Read last 5 comments
tail -n 5 /linear/teams/ENG__Engineering__abc123/issues/ENG-123__iss_001/comments.jsonl
# List members
ls /linear/teams/ENG__Engineering__abc123/members/
# Read a project with its issues
cat /linear/teams/ENG__Engineering__abc123/projects/Mirage__proj_001.json \
| jq '.issues'
# Tree view
tree -L 2 /linear/teams/
```
## Finding IDs
IDs are embedded in directory and file names after `__`:
```bash
# Team ID -- embedded in directory name
ls /linear/teams/
# -> ENG__Engineering__abc123 <- team_id = abc123
# Issue ID -- embedded in issue directory name
ls /linear/teams/ENG__Engineering__abc123/issues/
# -> ENG-123__iss_001 <- issue_id = iss_001
# Member ID -- embedded in member file name
ls /linear/teams/ENG__Engineering__abc123/members/
# -> alice__usr_001.json <- user_id = usr_001
# Use stat for structured metadata
stat /linear/teams/ENG__Engineering__abc123/issues/ENG-123__iss_001/issue.json
# Extract fields with jq
cat /linear/teams/ENG__Engineering__abc123/issues/ENG-123__iss_001/issue.json \
| jq '{issue_id, issue_key, team_id, state_id, assignee_id}'
```
## Shell Commands
Standard commands available on the mounted Linear tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List teams, issues, members, projects |
| `cat` | Read .json metadata or .jsonl comments |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `basename` | Extract file name from path |
| `dirname` | Extract directory from path |
| `realpath` | Resolve path to absolute form |
## Resource-Specific Commands
### `linear-issue-create`
Create a new issue. Description can be passed via `--description`, `--description_file`, or stdin.
```bash
linear-issue-create --team_key ENG --title "New bug"
linear-issue-create --team_id abc123 --title "New bug" --description "Details here"
cat brief.md | linear-issue-create --team_key ENG --title "Agent bug"
```
| Option | Required | Description |
| -------------------- | -------- | ----------------------------------- |
| `--team_id` | \* | Linear team ID |
| `--team_key` | \* | Linear team key (e.g. ENG) |
| `--title` | yes | Issue title |
| `--description` | no | Inline description text |
| `--description_file` | no | Path to file containing description |
\* One of `--team_id` or `--team_key` is required.
### `linear-issue-update`
Update an existing issue. Description can be passed via `--description`, `--description_file`, or stdin.
```bash
linear-issue-update --issue_key ENG-123 --title "Updated title"
linear-issue-update --issue_id iss_001 --description "New description"
```
| Option | Required | Description |
| -------------------- | -------- | ----------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--title` | no | New title |
| `--description` | no | Inline description text |
| `--description_file` | no | Path to file containing description |
\* One of `--issue_id` or `--issue_key` is required.
### `linear-issue-assign`
Assign an issue to a user.
```bash
linear-issue-assign --issue_key ENG-123 --assignee_email "alice@example.com"
linear-issue-assign --issue_id iss_001 --assignee_id usr_001
```
| Option | Required | Description |
| ------------------ | -------- | ------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--assignee_id` | \*\* | Linear user ID |
| `--assignee_email` | \*\* | User email address |
\* One of `--issue_id` or `--issue_key` is required.
\*\* One of `--assignee_id` or `--assignee_email` is required.
### `linear-issue-transition`
Transition an issue to a different workflow state.
```bash
linear-issue-transition --issue_key ENG-123 --state_name "In Progress"
linear-issue-transition --issue_id iss_001 --state_id state_001
```
| Option | Required | Description |
| -------------- | -------- | ------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--state_id` | \*\* | Linear workflow state ID |
| `--state_name` | \*\* | Workflow state name |
\* One of `--issue_id` or `--issue_key` is required.
\*\* One of `--state_id` or `--state_name` is required.
### `linear-issue-set-priority`
Set the priority of an issue.
```bash
linear-issue-set-priority --issue_key ENG-123 --priority 1
```
| Option | Required | Description |
| ------------- | -------- | ---------------------------------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--priority` | yes | Priority level (0=none, 1=urgent, 2=high, 3=medium, 4=low) |
\* One of `--issue_id` or `--issue_key` is required.
### `linear-issue-set-project`
Assign an issue to a project.
```bash
linear-issue-set-project --issue_key ENG-123 --project_id proj_001
```
| Option | Required | Description |
| -------------- | -------- | ------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--project_id` | yes | Linear project ID |
\* One of `--issue_id` or `--issue_key` is required.
### `linear-issue-add-label`
Add a label to an issue.
```bash
linear-issue-add-label --issue_key ENG-123 --label_id lbl_001
```
| Option | Required | Description |
| ------------- | -------- | ------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--label_id` | yes | Linear label ID |
\* One of `--issue_id` or `--issue_key` is required.
### `linear-issue-comment-add`
Add a comment to an issue. Body can be passed via `--body`, `--body_file`, or stdin.
```bash
linear-issue-comment-add --issue_key ENG-123 --body "Looks good"
cat note.md | linear-issue-comment-add --issue_key ENG-123
```
| Option | Required | Description |
| ------------- | -------- | ------------------------------- |
| `--issue_id` | \* | Linear issue ID |
| `--issue_key` | \* | Linear issue key (e.g. ENG-123) |
| `--body` | \*\* | Inline comment text |
| `--body_file` | \*\* | Path to file containing body |
\* One of `--issue_id` or `--issue_key` is required.
\*\* One of `--body`, `--body_file`, or stdin is required.
### `linear-issue-comment-update`
Update an existing comment. Body can be passed via `--body`, `--body_file`, or stdin.
```bash
linear-issue-comment-update --comment_id cmt_001 --body "Updated text"
```
| Option | Required | Description |
| -------------- | -------- | ---------------------------- |
| `--comment_id` | yes | Linear comment ID |
| `--body` | \*\* | Inline comment text |
| `--body_file` | \*\* | Path to file containing body |
\*\* One of `--body`, `--body_file`, or stdin is required.
### `linear-search`
Search issues across the workspace.
```bash
linear-search --query "authentication bug"
```
| Option | Required | Description |
| --------- | -------- | ----------------- |
| `--query` | yes | Search query text |
Returns matching issues as a JSON array.
+86
View File
@@ -0,0 +1,86 @@
---
title: MinIO
icon: /images/minio-logo.svg
description: Mount a self-hosted MinIO bucket as a virtual filesystem.
---
The MinIO resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `MinIOConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just points at your MinIO `endpoint_url` and
uses path-style addressing by default.
MinIO is self-hosted, so `endpoint_url` is **required** (there is no
region-derived host). Uses aioboto3 against MinIO's S3-compatible API.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.minio import MinIOConfig, MinIOResource
config = MinIOConfig(
bucket=os.environ["MINIO_BUCKET"],
endpoint_url=os.environ.get("MINIO_ENDPOINT", "http://localhost:9000"),
access_key_id=os.environ["MINIO_ACCESS_KEY"],
secret_access_key=os.environ["MINIO_SECRET_KEY"],
# Optional:
# region="us-east-1", # default
# path_style=True, # default; MinIO requires path-style addressing
# timeout=30,
# proxy="http://proxy:8080",
)
resource = MinIOResource(config)
ws = Workspace({"/minio": resource}, mode=MountMode.WRITE)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.minio import MinIOConfig, MinIOResource
load_dotenv(".env.development")
config = MinIOConfig(
bucket=os.environ.get("MINIO_BUCKET", "mirage-demo"),
endpoint_url=os.environ.get("MINIO_ENDPOINT", "http://localhost:9000"),
access_key_id=os.environ.get("MINIO_ACCESS_KEY", "minioadmin"),
secret_access_key=os.environ.get("MINIO_SECRET_KEY", "minioadmin"),
)
resource = MinIOResource(config)
async def main() -> None:
ws = Workspace({"/minio/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /minio/")
print(await r.stdout_str())
r = await ws.execute("tree /minio/")
print(await r.stdout_str())
r = await ws.execute("cat /minio/data/config.json")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- MinIO reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies (`ls`, `cat`,
`head`, `tail`, `grep`, `rg`, `wc`, `find`, `tree`, `jq`, `stat`, plus
parquet/orc/feather table rendering). See the [S3 resource](/python/resource/s3)
for the complete command reference, range reads, streaming, and the index
cache fast path.
- For credential setup, see [MinIO Setup](/home/setup/minio).
+351
View File
@@ -0,0 +1,351 @@
---
title: MongoDB
description: Mount MongoDB databases, collections, and documents as a Mirage filesystem with JSON access for Python agents.
icon: database
---
The MongoDB resource exposes MongoDB databases, collections, and documents
as a virtual filesystem mounted at some prefix such as `/mongodb/`.
For connection setup, see [MongoDB Setup](/python/setup/mongodb).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.mongodb import MongoDBConfig, MongoDBResource
config = MongoDBConfig(
uri=os.environ["MONGODB_URI"],
default_doc_limit=1000,
default_search_limit=100,
max_doc_limit=5000,
elide_fields={"sample_mflix.movies": ["plot_embedding"]},
)
resource = MongoDBResource(config=config)
ws = Workspace({"/mongodb": resource}, mode=MountMode.READ)
```
| Config field | Required | Default | Description |
| ---------------------- | -------- | ------- | -------------------------------------------------------------------------- |
| `uri` | yes | | MongoDB connection URI |
| `databases` | no | | List of database names to mount (omit for all) |
| `default_doc_limit` | no | 1000 | Default doc cap for one-shot reads (not used by streaming `cat`) |
| `default_search_limit` | no | 100 | Default result cap for collection/db-level `grep` |
| `max_doc_limit` | no | 5000 | Hard cap for `head -n K` / `tail -n K` |
| `elide_fields` | no | `{}` | `{"<db>.<coll>": ["field", "nested.path"]}`; listed fields are dropped from `documents.jsonl` |
## Filesystem Layout
The mount mirrors MongoDB's `cluster → database → collection / view`
model. The database directory always appears in the path, even when
`databases` filters to a single entry.
```text
/mongodb/
<database>/
database.json # collections + views + counts
collections/
<collection>/
schema.json # sampled types + indexes + validator
documents.jsonl # streamed BSON Extended JSON
views/
<view>/
schema.json # view-aware (no indexes / validator)
documents.jsonl # streamed, same format as collections
```
Example:
```text
/mongodb/
sample_mflix/
database.json
collections/
movies/
schema.json
documents.jsonl
comments/
schema.json
documents.jsonl
views/
top_rated_movies/
schema.json
documents.jsonl
```
### documents.jsonl
One JSON object per line, encoded with BSON
[Relaxed Extended JSON](https://www.mongodb.com/docs/manual/reference/mongodb-extended-json/).
BSON-specific types round-trip through their canonical `$` wrappers:
```json
{"_id":{"$oid":"573a1390f29313caabcd4135"},"title":"Casablanca","released":{"$date":"1943-01-23T00:00:00Z"},"runtime":102}
```
`cat` / `head` / `tail` / `grep` / `jq` all stream from this file; nothing
materializes the full collection in memory.
Because the file is rendered on demand, `stat` / `ls -l` report no size and
`du` returns `0` for it (computing the real size would require rendering the
whole collection). Use `wc -c documents.jsonl` for the actual rendered byte
count; `stat` still exposes `document_count` for the number of documents.
### schema.json
Generated from a 100-document `$sample`. Includes:
- field path → observed BSON type frequencies (nested paths unioned across the sample)
- indexes from `listIndexes` plus access counts from `$indexStats`
- the `$jsonSchema` validator if one is registered
Views skip the index and validator sections.
### database.json
Lists every collection and view under the database with their document
counts. Useful for `cat /mongodb/<db>/database.json` to get an overview
without recursing into each entity.
### Elided fields
Fields listed under `elide_fields` are dropped entirely from
`documents.jsonl` output. The type stays documented in `schema.json`, so
heavy fields (embeddings, large binary, raw text blobs) can be hidden
from agent reads without losing the schema signal:
```python
config = MongoDBConfig(
uri=os.environ["MONGODB_URI"],
elide_fields={
"sample_mflix.movies": ["plot_embedding"],
"rag.docs": ["metadata.embedding"],
},
)
```
Nested paths use dot notation. Elision applies to both `cat` (one-shot
streaming reads) and `tail -f` (live change-stream follows).
## Streaming and Limits
`cat`, `grep`, `head`, and `tail -f` consume documents lazily through a
batched PyMongo async cursor (or change stream); the consumer cancels to stop
fetching. There is no truncation notice because nothing is forced into
memory ahead of the consumer.
| Command | Behavior |
| ---------------------------- | ----------------------------------------------------------------- |
| `cat` | Streams the whole collection sorted by `_id`; pipe to `head` to cap |
| `head -n K` / `tail -n K` | K is capped at `max_doc_limit`; server-side sort + limit |
| `tail -f` | Opens a change stream; yields each new insert as a JSONL line |
| `grep` (file level) | Streams from `documents.jsonl`; supports `-m` for short-circuit |
| `grep` (collection/db level) | Server-side query, capped at `default_search_limit` |
| `jq` | Inherits the streaming `cat` source |
| `wc` | Uses `countDocuments()` server-side; zero download |
| `stat` | Metadata only (doc count, indexes; views skip the index lookup) |
## Smart Commands
### grep at different scopes
`grep` uses MongoDB's query engine at directory scopes instead of
streaming all documents through the regex pipeline:
```bash
# FILE level - streams documents.jsonl, runs the regex locally
grep "Godfather" "/mongodb/sample_mflix/collections/movies/documents.jsonl"
# COLLECTION level - server-side query against the collection
grep "Godfather" "/mongodb/sample_mflix/collections/movies/"
# DATABASE level - searches across every collection in sample_mflix
grep "Godfather" "/mongodb/sample_mflix/"
# ROOT level - fans out across every mounted database
grep "Godfather" "/mongodb/"
```
At collection or higher scope the resource picks the best server-side
strategy from the indexes available:
1. **Text index exists** → uses `$text` (ranked by relevance)
1. **Atlas Search index exists** → uses `$search` (fuzzy, Lucene-based)
1. **Neither** → falls back to `$regex` on sampled string fields
Scope detection is handled by `mirage/core/mongodb/scope.py`.
### head / tail / tail -f
`head` and `tail` use server-side `sort` + `limit`; the requested count
is capped at `max_doc_limit`. `tail -f` opens a Mongo change stream
filtered to insert events and yields each new document as a JSONL line
in the same format as `cat`:
```bash
# First 10 docs (sorted by _id ascending)
head -n 10 "/mongodb/sample_mflix/collections/movies/documents.jsonl"
# Last 10 docs (sorted by _id descending)
tail -n 10 "/mongodb/sample_mflix/collections/movies/documents.jsonl"
# Live-follow new inserts; consumer cancels to stop
tail -f "/mongodb/sample_mflix/collections/movies/documents.jsonl"
```
`tail -f` requires the cluster to be a replica set; Atlas already
satisfies this. Views fall through to the non-streaming path because
change streams aren't defined on views.
## Cache
The MongoDB resource uses `IndexCacheStore` (same as RAM/S3/disk/GitHub)
for listings: database names, collection names, and document counts.
Document content is **not** cached. The resource leaves `caches_reads`
at its default of `False`, so `cat`, `grep`, `head`, and `tail` always
query the live collection
instead of serving a stored snapshot. This keeps reads consistent with a
mutable database and ensures `tail -f` follows the live change stream
rather than replaying cached bytes.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.mongodb import MongoDBConfig, MongoDBResource
load_dotenv(".env.development")
config = MongoDBConfig(uri=os.environ["MONGODB_URI"])
resource = MongoDBResource(config=config)
async def main():
ws = Workspace({"/mongodb": resource}, mode=MountMode.READ)
# List all databases
r = await ws.execute("ls /mongodb/")
print(await r.stdout_str())
# List the entities under a database (database.json, collections/, views/)
r = await ws.execute("ls /mongodb/sample_mflix/")
print(await r.stdout_str())
# List collections
r = await ws.execute("ls /mongodb/sample_mflix/collections/")
print(await r.stdout_str())
# Read first 5 movies
r = await ws.execute(
'head -n 5 "/mongodb/sample_mflix/collections/movies/documents.jsonl"')
print(await r.stdout_str())
# Read last 5 movies
r = await ws.execute(
'tail -n 5 "/mongodb/sample_mflix/collections/movies/documents.jsonl"')
print(await r.stdout_str())
# Inspect the sampled schema and indexes
r = await ws.execute(
'cat "/mongodb/sample_mflix/collections/movies/schema.json"')
print(await r.stdout_str())
# Extract titles with jq
r = await ws.execute(
'jq -r ".[] | .title" "/mongodb/sample_mflix/collections/movies/documents.jsonl"'
)
print(await r.stdout_str())
# Search across a database (uses MongoDB query engine)
r = await ws.execute('grep "Godfather" "/mongodb/sample_mflix/"')
print(await r.stdout_str())
# Count documents (server-side, no download)
r = await ws.execute(
'wc -l "/mongodb/sample_mflix/collections/movies/documents.jsonl"')
print(await r.stdout_str())
# View the database overview without recursing
r = await ws.execute('cat "/mongodb/sample_mflix/database.json"')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Runnable examples
Three working examples live under `examples/python/mongodb/`:
- [`mongodb.py`](https://github.com/StruktoAI/mirage/blob/main/examples/python/mongodb/mongodb.py) — agent-shell workflow: `ls`, `tree`, `cat`, `head`, `tail`, `wc`, `stat`, `grep`/`rg` at every scope, `jq`, `find`, `cd` + relative paths.
- [`mongodb_vfs.py`](https://github.com/StruktoAI/mirage/blob/main/examples/python/mongodb/mongodb_vfs.py) — in-process VFS: `os.listdir` and `open()` walk every readdir level (root, database, `collections/`, `views/`, entity) and read `database.json`, `schema.json`, `documents.jsonl` (collection + view).
- [`mongodb_fuse.py`](https://github.com/StruktoAI/mirage/blob/main/examples/python/mongodb/mongodb_fuse.py) — same coverage as the VFS example, but the tree is mounted as a real filesystem so other processes can `cat`/`ls`/`head` the mountpoint directly.
All three default to the `mirage_test` database seeded by `python/scripts/seed_mongodb_test.py`.
## Finding IDs
`_id` is serialized as `{"$oid": "..."}` under Extended JSON:
```bash
# List the first 10 ObjectId values
jq -r '.[] | ._id["$oid"]' \
"/mongodb/sample_mflix/collections/movies/documents.jsonl" | head -n 10
# Find a specific document by ID
grep "573a1390f29313caabcd42e8" \
"/mongodb/sample_mflix/collections/movies/documents.jsonl"
# Extract a few fields together
jq -r '.[] | "\(._id["$oid"]) \(.title) \(.year)"' \
"/mongodb/sample_mflix/collections/movies/documents.jsonl"
```
## Working with Large Collections
Streaming is the default; reach for these patterns when you want to keep
the round trip small:
```bash
# Document count (server-side, no download)
wc -l "/mongodb/sample_mflix/collections/comments/documents.jsonl"
# Most recent documents (sorted by _id desc)
tail -n 10 "/mongodb/sample_mflix/collections/comments/documents.jsonl"
# Server-side query at collection scope avoids streaming the whole jsonl
grep "love" "/mongodb/sample_mflix/collections/comments/"
# Drop heavy fields via elide_fields, then take the slice you want
head -n 20 "/mongodb/sample_mflix/collections/movies/documents.jsonl"
```
Hide embeddings or large blobs from agent reads with `elide_fields`;
`schema.json` still documents the original type so the agent can decide
when to ask for the raw bytes through a different path.
## Shell Commands
Standard commands available on the mounted MongoDB tree:
| Command | Notes |
| --------------- | ----------------------------------------------------------------------- |
| `ls` | List databases, collections, views, and per-entity files |
| `cat` | Stream `documents.jsonl` (or read `schema.json` / `database.json`) |
| `head` / `tail` | Smart: server-side sort + limit (capped at `max_doc_limit`) |
| `tail -f` | Live-follow new inserts via Mongo change stream |
| `grep` / `rg` | Smart: MongoDB query engine at collection/db scope, regex at file scope |
| `jq` | Query JSON; use `.[]` prefix when iterating JSONL files |
| `wc` | Smart: uses `countDocuments()` server-side |
| `stat` | Metadata (doc count, indexes; views skip the index lookup) |
| `find` | List databases/collections/views with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
+218
View File
@@ -0,0 +1,218 @@
---
title: Adding a New Resource
icon: plus
description: Step-by-step guide for implementing a new MIRAGE resource.
---
This guide covers what you need to implement when adding a new resource
to MIRAGE. Use the Discord or Slack resources as reference.
## File Structure
```text
mirage/
resource/<name>/
__init__.py # lazy-loading exports
config.py # Pydantic config (credentials)
<name>.py # BaseResource subclass
accessor/<name>.py # Accessor wrapping config
core/<name>/
__init__.py
_client.py # HTTP client (get/post with rate limiting)
readdir.py # directory listing
read.py # file reading
stat.py # file metadata
scope.py # scope detection
glob.py # glob pattern resolution
... # data fetching modules (history, search, post, etc.)
ops/<name>/
__init__.py # exports OPS list
read.py # @op wrapper
readdir.py # @op wrapper
stat.py # @op wrapper
commands/builtin/<name>/
__init__.py # exports COMMANDS list
_plan.py # cost estimation helpers
cat.py, ls.py, ... # standard commands
<name>_send.py, ... # resource-specific commands
```
## Implementation Steps
### 1. Config, Accessor, ResourceName
Create a Pydantic config model to hold credentials and an accessor
class that wraps it.
```python
# mirage/resource/<name>/config.py
from pydantic import BaseModel
class MyConfig(BaseModel):
token: str
```
```python
# mirage/accessor/<name>.py
from mirage.accessor.base import Accessor
from mirage.resource.<name>.config import MyConfig
class MyAccessor(Accessor):
def __init__(self, config: MyConfig) -> None:
self.config = config
```
Add `MY_RESOURCE = "<name>"` to the `ResourceName` enum in `mirage/types.py`.
### 2. HTTP Client
Wrap the resource's API with rate-limit handling. All resources follow
the same pattern: async get/post functions with retry on 429.
```python
# mirage/core/<name>/_client.py
async def my_get(config, endpoint, params=None) -> dict: ...
async def my_post(config, endpoint, body=None) -> dict: ...
```
### 3. Core VFS
Implement the three VFS operations that map API data to a filesystem:
- **`readdir.py`** -- Returns `list[str]` of child paths for a directory.
- **`read.py`** -- Returns `bytes` content of a file.
- **`stat.py`** -- Returns `FileStat` with name, type, and extras (e.g., IDs).
All three accept `(accessor, path, index, prefix)` and use `IndexCacheStore`
to cache name-to-ID mappings.
### 4. Scope Detection and GlobScope Optimization
`GlobScope` carries the raw path and pattern **before** expansion. This
lets commands decide how to resolve paths efficiently -- skipping
expensive glob expansion when the resource has a native API for the
operation.
**`scope.py`** parses the unexpanded path to determine the level:
```python
@dataclass
class MyScope:
level: str # "root", "category", "item", "file"
item_id: str | None = None
```
**How commands use scope for optimization:**
```python
# Example: grep at different scopes
@command("grep", resource="my_resource", spec=SPECS["grep"])
async def grep(accessor, paths, *texts, **_extra):
pattern = texts[0]
scope = detect_scope(paths[0], index)
if scope.level in ("category", "item"):
# CHEAP: use native search API (1 API call)
results = await search_api(accessor.config, scope.item_id, pattern)
return format_results(results), IOResult()
# EXPENSIVE fallback: expand glob, download each file, grep locally
paths = await resolve_glob(accessor, paths, index=index)
for p in paths:
data = await read(accessor, p.original, index, prefix=p.prefix)
# ... grep the bytes ...
```
**When to use this pattern:**
| Scenario | Approach |
| ------------------------------------------ | --------------------------------------------------------- |
| Resource has a search API (Discord, Slack) | Use scope to route to native API at directory level |
| Resource has no search API | Always fall through to file-level reads; scope is a noop |
| `head`/`tail` with direct message fetch | Use scope to detect file level, fetch N messages directly |
**When the resource has no search API**, keep `scope.py` as a noop for
structural consistency. The file still parses path parts but does not
trigger any API calls:
```python
# Noop scope -- no search API, no resource-relative paths
def detect_scope(path: str | GlobScope) -> MyScope:
key = path.strip("/")
parts = key.split("/")
# Just parse path structure, no index lookups
...
```
### 5. Glob Resolution
```python
# mirage/core/<name>/glob.py
async def resolve_glob(accessor, paths, index=None) -> list[GlobScope]:
# Expand patterns via readdir + fnmatch
```
### 6. Ops Layer
Thin wrappers that bridge core functions to the command framework:
```python
@op("read", resource="<name>")
async def read(accessor, scope, **kwargs) -> bytes:
return await core_read(accessor, scope.original,
kwargs.get("index"), prefix=scope.prefix)
```
### 7. Commands
Copy from an existing resource (Discord/Slack), then:
1. Replace accessor and core imports.
1. Set `resource="<name>"` in decorators.
1. Add or remove scope-based optimizations depending on API capabilities.
1. Add resource-specific commands (send message, etc.).
**Provision (dry-run estimates).** Factory-built commands
(`make_generic_commands`) get family-default estimators for free, so
`ws.execute(cmd, provision=True)` works without extra wiring. For
bespoke commands, pass `provision=` to the decorator, reusing the
shared helpers from `mirage.commands.builtin.generic_bind`:
```python
from mirage.commands.builtin.generic_bind.provision import (
make_file_read_provision, metadata_provision)
from mirage.core.<name>.stat import stat as my_stat
@command("cat", resource="<name>", spec=SPECS["cat"],
provision=make_file_read_provision(my_stat))
async def cat(...): ...
```
Omit `provision=` and the planner honestly reports
`precision=unknown`. If your resource renders virtual files (a
`chat.jsonl` built from messages), have `stat` report `size=None`
rather than 0 so estimates degrade to floors instead of lying.
### 8. Resource Class
```python
class MyResource(BaseResource):
name: str = ResourceName.MY_RESOURCE
caches_reads: bool = True
def __init__(self, config: MyConfig) -> None:
super().__init__()
self.config = config
self.accessor = MyAccessor(self.config)
from mirage.commands.builtin.<name> import COMMANDS
from mirage.ops.<name> import OPS
for fn in COMMANDS:
self.register(fn)
for fn in OPS:
self.register_op(fn)
```
+264
View File
@@ -0,0 +1,264 @@
---
title: Nextcloud
icon: cloud
---
The Nextcloud resource mounts a [WebDAV](http://www.webdav.org/specs/rfc4918.html)
server (Nextcloud, ownCloud, Hetzner Storage Share, or any generic WebDAV
endpoint) at some prefix such as `/nc/`. Reads are async and streaming, with
range requests for partial reads.
For credential setup, see [Nextcloud Setup](/home/setup/nextcloud).
## Install
```bash
uv add "mirage-ai[nextcloud]"
```
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.nextcloud import NextcloudConfig, NextcloudResource
config = NextcloudConfig(
url=os.environ["NEXTCLOUD_URL"],
username=os.environ["NEXTCLOUD_USERNAME"],
password=os.environ["NEXTCLOUD_PASSWORD"],
# Optional:
# verify_ssl=True,
# timeout=30,
)
resource = NextcloudResource(config)
ws = Workspace({"/nc/": resource}, mode=MountMode.WRITE)
```
`NextcloudResource(config)` takes a `NextcloudConfig` object with the
WebDAV URL plus optional Basic Auth credentials. Both `READ` and `WRITE`
modes are supported. Use an **app password** (Settings → Security in
Nextcloud) rather than your account password.
## Filesystem Layout
WebDAV resources map directly to virtual paths under the mount prefix.
For example, if your Nextcloud root contains:
```text
Documents/notes.md
Documents/contract.pdf
Photos/2024/cat.jpg
```
Then mounting at `/nc/` exposes:
```text
/nc/
Documents/
notes.md
contract.pdf
Photos/
2024/
cat.jpg
```
Path mapping: virtual `/nc/Documents/notes.md` issues HTTP requests
against `<NEXTCLOUD_URL>/Documents/notes.md`.
## Cache
The Nextcloud resource uses `IndexCacheStore`. Directory listings come
from a single `PROPFIND Depth: 1` and populate file size, type, and
ETag entries that `stat` reads via a fast path, so a `readdir` followed
by per-entry `stat` calls (which is what `ls`, FUSE `getattr`, and most
shell commands trigger) costs one HTTP request instead of N.
## Fingerprinting and Snapshots
`SUPPORTS_SNAPSHOT = True`. Per-file fingerprints come from the WebDAV
`getetag` property, so snapshot drift detection works automatically:
when a remote file changes, its ETag changes, and Mirage notices on the
next access.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.nextcloud import NextcloudConfig, NextcloudResource
load_dotenv(".env.development")
config = NextcloudConfig(
url=os.environ["NEXTCLOUD_URL"],
username=os.environ["NEXTCLOUD_USERNAME"],
password=os.environ["NEXTCLOUD_PASSWORD"],
)
resource = NextcloudResource(config)
async def main() -> None:
ws = Workspace({"/nc/": resource}, mode=MountMode.WRITE)
r = await ws.execute("ls /nc/")
print(await r.stdout_str())
r = await ws.execute("tree /nc/")
print(await r.stdout_str())
r = await ws.execute("find /nc/ -name '*.md'")
print(await r.stdout_str())
r = await ws.execute("cat /nc/Documents/notes.md")
print(await r.stdout_str())
r = await ws.execute("grep TODO /nc/Documents/notes.md")
print(await r.stdout_str())
r = await ws.execute("stat /nc/Documents/notes.md")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The Nextcloud resource supports the full set of shell commands since it
operates on real file content (text, binary, JSON, CSV, etc.). Reads
benefit from HTTP `Range` requests so commands like `head -c BYTES`
don't pull the whole file.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | --------------------------------------------- |
| `cp` | Copy files (WebDAV `COPY` method) |
| `mv` | Move/rename files (WebDAV `MOVE` method) |
| `rm` | Remove files (WebDAV `DELETE`; recursive for directories) |
| `mkdir` | Create directories (WebDAV `MKCOL`) |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Streaming
- **Reads**: streamed in chunks. Downstream commands like `head -n 1`
early-cancel, so you only pay for the first chunk over the wire.
- **Range reads**: `head -c BYTES` and other partial reads issue HTTP
`Range` requests that Nextcloud and most WebDAV servers honor.
- **Writes**: buffer the payload before upload. Streaming uploads
(Nextcloud's `uploads/` resumable protocol) are a possible future
follow-up.
## Use Cases
- **AI agents accessing personal cloud storage**: Mount Nextcloud so
agents can read documents, notes, and structured data on a self-hosted
cloud.
- **Self-hosted alternative to Dropbox/Box**: Same shell-command surface,
with full read+write support.
- **Multi-protocol WebDAV**: Works against any RFC 4918-compliant server,
not just Nextcloud.
- **FUSE mounting**: Expose Nextcloud through a virtual FUSE mount for
external tools.
+268
View File
@@ -0,0 +1,268 @@
---
title: Notion
icon: book
description: Mount Notion pages as a virtual filesystem.
---
The Notion resource exposes a Notion workspace as a virtual filesystem
mounted at a prefix such as `/notion/`.
For API key setup, see [Notion Setup](/python/setup/notion).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.notion import NotionConfig, NotionResource
config = NotionConfig(api_key=os.environ["NOTION_API_KEY"])
resource = NotionResource(config=config)
ws = Workspace({"/notion": resource}, mode=MountMode.WRITE)
```
| Field | Required | Default | Description |
| ---------- | -------- | ------------------------------ | ----------------------- |
| `api_key` | yes | | Notion integration token |
| `base_url` | no | `https://api.notion.com/v1` | API base URL |
## Filesystem Layout
```text
/notion/
pages/
<page-title>__<page-id>/
page.json
<child-page-title>__<child-id>/
page.json
...
databases/
<database-title>__<database-id>/
database.json
<row-page-title>__<page-id>/
page.json
...
```
Example:
```text
/notion/
pages/
Project-Roadmap__a1b2c3d4/
page.json
Q1-Goals__e5f6g7h8/
page.json
Q2-Goals__i9j0k1l2/
page.json
Meeting-Notes__m3n4o5p6/
page.json
databases/
Tasks__4a3b21915e77/
database.json
Write-proposal__62212c5affe6/
page.json
Build-dashboards__f988c5a145ef/
page.json
Roadmap__2c4d6a7f5bf3/
database.json
```
The `pages/` hierarchy mirrors Notion's standalone page tree. The
`databases/` hierarchy lists databases shared with the integration and
then the row pages returned by querying each database. Each database
directory contains `database.json` with the database metadata and its
typed property schema (the columns), but not the rows themselves; the
rows are the row-page directories alongside it, so `ls` the database
directory to enumerate them. Each page directory contains a `page.json`
file with the page metadata and content. Child pages appear as nested
directories.
`page.json` carries the page metadata (`page_id`, `title`, `url`,
timestamps, `parent_type`/`parent_id`, `created_by`/`last_edited_by`),
a `markdown` field with the page body rendered as Markdown, and the raw
`blocks`. Blocks with children embed them recursively under a
`children` key, and the Markdown renders nested blocks with
indentation.
`database.json` carries the database metadata (`database_id`, `title`,
`url`, timestamps, `parent`, `archived`, `is_inline`) and `properties`,
the database's typed column schema. It does not embed the rows; read a
row's `page.json` for its content, since a database row is itself a page
with `parent_type` of `database_id`.
### database.json
The database's identity plus its typed column schema (Notion's own
property objects), with no rows inline. `ls` the database directory to
enumerate row pages:
```json
{
"database_id": "2c4d6a7f-5bf3-8036-bed2-d1c95826f76b",
"title": "Item",
"url": "https://www.notion.so/2c4d6a7f5bf38036bed2d1c95826f76b",
"created_time": "2025-12-09T23:36:00.000Z",
"last_edited_time": "2026-04-15T12:30:00.000Z",
"parent": { "type": "workspace", "workspace": true },
"archived": false,
"is_inline": false,
"properties": {
"Name": { "id": "title", "type": "title", "title": {} },
"Number": { "id": "e%5BNQ", "type": "number", "number": { "format": "number" } },
"Date": { "id": "%3BgQq", "type": "date", "date": {} }
}
}
```
To create a row, `notion-page-create` with a `database_id` parent and a
`properties` object whose keys match this schema.
### page.json
A page (including a database row). Notion blocks render to `markdown`
and stay available raw under `blocks`:
```json
{
"page_id": "2c4d6a7f-5bf3-8022-826b-ed4700254ed2",
"title": "item_0",
"url": "https://www.notion.so/item_0-2c4d6a7f5bf38022826bed4700254ed2",
"created_time": "2025-12-09T23:36:00.000Z",
"last_edited_time": "2025-12-09T23:44:00.000Z",
"parent_type": "database_id",
"parent_id": "2c4d6a7f-5bf3-8036-bed2-d1c95826f76b",
"archived": false,
"created_by": "faa6ebe0-0686-466d-96b3-2480ddda715b",
"last_edited_by": "faa6ebe0-0686-466d-96b3-2480ddda715b",
"markdown": "",
"blocks": []
}
```
## Cache
Uses `IndexCacheStore` for page metadata. No separate content
cache - file content caching is handled by the workspace `IOResult`
mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.notion import NotionConfig, NotionResource
load_dotenv(".env.development")
config = NotionConfig(api_key=os.environ["NOTION_API_KEY"])
resource = NotionResource(config=config)
async def main():
ws = Workspace({"/notion": resource}, mode=MountMode.WRITE)
# List top-level pages
r = await ws.execute("ls /notion/pages/")
print(await r.stdout_str())
# List shared databases and rows in a database
r = await ws.execute("ls /notion/databases/")
print(await r.stdout_str())
# Read a page
r = await ws.execute(
'cat "/notion/pages/Project-Roadmap__a1b2c3d4/page.json"'
)
print(await r.stdout_str())
# Search across all pages
r = await ws.execute('grep "deadline" /notion/pages/')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 2 /notion/")
print(await r.stdout_str())
# Search pages with the Notion search API
r = await ws.execute('notion-search --query "Roadmap" --limit 5')
print(await r.stdout_str())
# Create a new page
r = await ws.execute(
'notion-page-create --json \'{"parent":{"page_id":"a1b2c3d4"},'
'"properties":{"title":[{"text":{"content":"New Page"}}]}}\''
)
print(await r.stdout_str())
# Append content to an existing page
r = await ws.execute(
'notion-block-append --params \'{"block_id":"a1b2c3d4"}\''
' --json \'{"children":[{"type":"paragraph","paragraph":'
'{"rich_text":[{"text":{"content":"Appended paragraph"}}]}}]}\''
)
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
Standard commands available on the mounted Notion tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List pages and child pages |
| `cat` | Read page.json content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Search across pages |
| `jq` | Query page JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
Resource-specific commands:
### `notion-page-create`
Create a new page under a parent page. Prints the new page normalized as JSON.
| Option | Required | Description |
| -------- | -------- | ---------------------------------------------------------------------- |
| `--json` | yes | Page body with `parent` (required) and `properties`, per the Notion API |
In the TypeScript SDK this command takes `--parent <parent-path>` and
`--title "title"` instead of a raw `--json` body.
### `notion-block-append`
Append content blocks to an existing page. Prints the refreshed page as JSON.
| Option | Required | Description |
| ---------- | -------- | ------------------------------------------------- |
| `--params` | yes | JSON object with the target `block_id` (page ID) |
| `--json` | yes | JSON object with the `children` blocks to append |
### `notion-comment-add`
Add a comment to a page. Prints the created comment as JSON.
| Option | Required | Description |
| -------- | -------- | ---------------------------------------------------- |
| `--json` | yes | Comment body with `parent` (required) and `rich_text` |
### `notion-search`
Search pages with the Notion search API.
| Option | Required | Description |
| --------- | -------- | ---------------------------- |
| `--query` | yes | Search query |
| `--limit` | no | Max results (default 20) |
+238
View File
@@ -0,0 +1,238 @@
---
title: OCI Object Storage
icon: server
description: Mount Oracle Cloud Infrastructure Object Storage as a virtual filesystem.
---
The OCI resource mounts an Oracle Cloud Infrastructure Object Storage
bucket at some prefix such as `/oci/`. All operations involve network
I/O to the remote object store. Uses aioboto3 against OCI's
S3-compatible API.
For credential setup, see [OCI Setup](/home/setup/oci).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.oci import OCIConfig, OCIResource
config = OCIConfig(
bucket=os.environ["OCI_BUCKET"],
namespace=os.environ["OCI_NAMESPACE"],
region=os.environ["OCI_REGION"],
access_key_id=os.environ["OCI_ACCESS_KEY_ID"],
secret_access_key=os.environ["OCI_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://{namespace}.compat.objectstorage.{region}.oraclecloud.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = OCIResource(config=config)
ws = Workspace({"/oci": resource}, mode=MountMode.READ)
```
`OCIResource(config)` takes an `OCIConfig` object with the bucket name,
OCI tenancy `namespace`, `region`, and Customer Secret Key credentials.
The S3-compatible endpoint is auto-built from `namespace` + `region`
unless `endpoint_url` is supplied. Both `READ` and `WRITE` modes are
supported.
## Filesystem Layout
The OCI resource maps object keys to virtual paths under the mount
prefix. OCI "directories" are prefix-based — there are no real
directory objects.
For example, if bucket `my-bucket` contains:
```text
data/users.csv
data/logs/2026-04-01.jsonl
config.json
```
Then mounting at `/oci/` exposes:
```text
/oci/
data/
users.csv
logs/
2026-04-01.jsonl
config.json
```
Path mapping: virtual `/oci/data/users.csv` maps to OCI key
`data/users.csv`.
## Cache
The OCI resource uses `IndexCacheStore` with `index_ttl = 600`
(10 minutes). Directory listings are cached for up to 600 seconds before
being refreshed from OCI.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.oci import OCIConfig, OCIResource
load_dotenv(".env.development")
config = OCIConfig(
bucket=os.environ["OCI_BUCKET"],
namespace=os.environ["OCI_NAMESPACE"],
region=os.environ["OCI_REGION"],
access_key_id=os.environ["OCI_ACCESS_KEY_ID"],
secret_access_key=os.environ["OCI_SECRET_ACCESS_KEY"],
)
resource = OCIResource(config=config)
async def main() -> None:
ws = Workspace({"/oci/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /oci/")
print(await r.stdout_str())
r = await ws.execute("cat /oci/data/users.csv | head -n 5")
print(await r.stdout_str())
r = await ws.execute("tree /oci/")
print(await r.stdout_str())
r = await ws.execute("find /oci/ -name '*.jsonl'")
print(await r.stdout_str())
r = await ws.execute("stat /oci/config.json")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The OCI resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit
from range reads to avoid downloading entire objects.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **AI agents accessing OCI data**: Mount OCI buckets for agents to read and process datasets
- **Data pipelines**: Read and write OCI objects with shell-like commands
- **FUSE mounting**: Expose OCI buckets through a virtual FUSE mount for external tools
+216
View File
@@ -0,0 +1,216 @@
---
title: OneDrive
description: Mount Microsoft OneDrive or SharePoint document libraries as a Mirage filesystem with async access, versioning, and shell commands.
icon: microsoft
---
The OneDrive resource mounts a Microsoft OneDrive (or SharePoint document
library) at some prefix such as `/onedrive/`. All operations involve network
I/O to Microsoft Graph. Files are served as raw bytes (no Office filetype
conversion). Uses aiohttp for async Graph access.
Compatible with: personal OneDrive, OneDrive for Business, and SharePoint
document libraries (driveItems on Microsoft Graph v1.0).
For credential setup (tokens, drive ids, app vs delegated auth), see the
[OneDrive Setup](/home/setup/onedrive) guide.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.onedrive import OneDriveResource, OneDriveConfig
config = OneDriveConfig(
access_token=os.environ["MS_GRAPH_DRIVE_TOKEN"],
# Optional:
# drive_id="b!...", # target a specific drive (required for app-only tokens)
# site_id="contoso.sharepoint.com,...", # use a SharePoint site's default drive
# key_prefix="Documents/", # mount a sub-folder as the root
# timeout=30,
)
resource = OneDriveResource(config)
ws = Workspace({"/onedrive": resource}, mode=MountMode.WRITE)
```
`OneDriveResource(config)` takes an `OneDriveConfig` object with a Microsoft
Graph bearer token and optional drive/site targeting. Both `READ` and `WRITE`
modes are supported.
### Config Reference
| Field | Required | Description |
| -------------- | -------- | --------------------------------------------------------------------- |
| `access_token` | Yes | Microsoft Graph OAuth2 bearer token |
| `drive_id` | No | Target a specific drive. Required for app-only tokens (no `/me/drive`) |
| `site_id` | No | Resolve a SharePoint site's default drive instead of `/me/drive` |
| `key_prefix` | No | Mount a sub-folder of the drive as the root |
| `timeout` | No | Request timeout in seconds (default `30`) |
Drive resolution order: `drive_id`, then the `site_id` site's default drive,
then `/me/drive`.
## Filesystem Layout
The OneDrive resource maps Graph driveItems (path-addressed) to virtual paths
under the mount prefix. Folders are real driveItems, so the tree matches what
you see in OneDrive.
For example, if the drive contains:
```text
Documents/file.txt
Documents/config.json
Reports/q1.csv
Reports/q2.csv
```
Then mounting at `/onedrive/` exposes:
```text
/onedrive/
Documents/
file.txt
config.json
Reports/
q1.csv
q2.csv
```
Path mapping: virtual `/onedrive/Documents/file.txt` maps to the driveItem at
`/root:/Documents/file.txt`.
## Versioning and Snapshots
OneDrive keeps per-file version history. The resource exposes it the same way
the S3 backend does:
- **Fingerprint** is the driveItem `cTag`, so normal reads and `stat` reflect
the current content without extra Graph calls.
- **Snapshots** pin each path to a Graph driveItem version id. Replaying a
snapshot reads `/versions/{id}/content`, giving time-travel to the exact
bytes captured at snapshot time.
- **Restore** writes a previous version back as the current one.
This makes the OneDrive resource `SUPPORTS_SNAPSHOT = True`.
## Cache
The OneDrive resource caches directory listings via `IndexCacheStore` to reduce
repeated Graph calls during traversal.
## Example
```python
import asyncio
import os
from mirage import MountMode, Workspace
from mirage.resource.onedrive import OneDriveResource, OneDriveConfig
config = OneDriveConfig(
access_token=os.environ["MS_GRAPH_DRIVE_TOKEN"],
drive_id=os.environ.get("MS_GRAPH_DRIVE_ID") or None,
)
resource = OneDriveResource(config)
async def main() -> None:
ws = Workspace({"/onedrive/": resource}, mode=MountMode.WRITE)
r = await ws.execute("ls /onedrive/")
print(await r.stdout_str())
await ws.execute("echo 'hello from mirage' > /onedrive/note.txt")
r = await ws.execute("cat /onedrive/note.txt")
print(await r.stdout_str())
r = await ws.execute("stat /onedrive/note.txt")
print(await r.stdout_str())
r = await ws.execute("tree /onedrive/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
A runnable version lives at `examples/python/onedrive/onedrive.py`.
## Shell Commands
The OneDrive resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit from
range reads to avoid downloading entire items.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `tee` | Write stdin to file and stdout |
### Path Utilities
| Command | Notes |
| ---------- | ------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `ls` | List directory contents |
## Use Cases
- **AI agents accessing org documents**: Mount OneDrive/SharePoint for agents to read and process files
- **Versioned document workflows**: Snapshot and replay exact file states over time
- **Sandboxed access**: Restrict agent operations to a specific drive and sub-folder via `key_prefix`
- **FUSE mounting**: Expose a OneDrive through a virtual FUSE mount for external tools
## Scoping a resource to a folder
Pass `key_prefix: str | None = None` to `OneDriveConfig` to transparently scope
every operation to a sub-folder of the drive:
```python
OneDriveResource(OneDriveConfig(
access_token=token,
drive_id=drive_id,
key_prefix=f"users/{user_id}/",
))
```
When set, every read/write/list/stat/copy/rename/delete operation is scoped to
that sub-folder. Agents see clean paths like `/data/notes.md`; the underlying
driveItem is `users/{user_id}/data/notes.md`. Useful for multi-tenant systems.
**Normalization:** leading slashes are stripped and a trailing slash is added
automatically. Both `None` and an empty string are treated as "no prefix."
+45
View File
@@ -0,0 +1,45 @@
---
title: Postgres
icon: database
description: Mount a Postgres database as a read-only filesystem of schemas, tables, and rows.
---
`PostgresResource` connects to a Postgres database via a DSN and exposes its schemas and tables as a tree the agent can list, read, and grep.
## Config
```python
from mirage.resource.postgres import PostgresConfig, PostgresResource
resource = PostgresResource(PostgresConfig(
dsn="postgres://user:pass@host:5432/db?sslmode=require",
schemas=["public", "analytics"], # optional allowlist
default_row_limit=1000,
max_read_rows=10_000,
max_read_bytes=10 * 1024 * 1024,
default_search_limit=100,
))
```
| Field | Default | Notes |
| --- | --- | --- |
| `dsn` | required | Postgres connection string. Redacted in snapshots. |
| `schemas` | `None` | Optional list to limit which schemas appear in the tree. |
| `default_row_limit` | `1000` | Default LIMIT applied to ad-hoc reads. |
| `max_read_rows` | `10_000` | Hard ceiling per read. |
| `max_read_bytes` | `10 MiB` | Hard ceiling per read. |
| `default_search_limit` | `100` | Default LIMIT for search-style queries. |
For DSN format and permissions, see [Postgres Setup](/home/setup/postgres).
## Mount mode
`read` only. Postgres mounts are read-only at the resource level.
## Snapshot behavior
The DSN is redacted on snapshot. Loading a snapshot back requires a config file that supplies a fresh DSN.
## File size and `du`
`rows.jsonl` is rendered on demand, so `stat` / `ls -l` / `du` report the table's **physical storage size** (`pg_total_relation_size`: heap + indexes + TOAST + page padding), not the size of the rendered JSONL. That number can differ a lot from the content and shifts with Postgres version and vacuum state. For the actual rendered byte count, use `wc -c rows.jsonl` (which streams and counts the bytes).
+130
View File
@@ -0,0 +1,130 @@
---
title: Qdrant
description: Mount a Qdrant collection as a Mirage filesystem, with payload folders, multimodal blobs, and a semantic search command for Python agents.
icon: database
---
The Qdrant resource exposes a [Qdrant](https://qdrant.tech/) collection as a virtual filesystem mounted
at some prefix such as `/q/`. Group-by payload fields become nested folders,
each point becomes a `.json` payload file (plus a `.txt` text file and an optional blob), and semantic search is the
`search` command, which returns ranked points as canonical file paths.
For connection setup (self-hosted, Qdrant Cloud, search), see
[Qdrant Setup](/python/setup/qdrant).
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.qdrant import QdrantConfig, QdrantResource
config = QdrantConfig(
url="https://xyz.cloud.qdrant.io",
api_key="...",
collection="fashion",
group_by=["gender", "articleType", "baseColour"],
id_field="id",
text_field="productDisplayName",
blob_field="image_b64",
blob_ext="jpg",
search_limit=5,
)
resource = QdrantResource(config)
ws = Workspace({"/fashion/": resource}, mode=MountMode.READ)
```
The mapping is config-driven; nothing about the dataset is hardcoded. Point
`group_by` at different payload fields and the folder tree changes.
## Filesystem layout
Every path is translated into a Qdrant query. Descending a folder adds one
payload filter; the leaf level lists points.
```text
/ # list collections (omitted when `collection` is pinned)
/<collection>/ # distinct group_by[0] values
<v1>/ # distinct group_by[1] where group_by[0]=v1
.../<vN>/ # all group-by fields bound -> point files
<id>.json # full payload (the metadata)
<id>.txt # embedded source text (when text_field set)
<id>.<ext> # raw blob / image bytes (when blob_field set)
```
`<id>` is the Qdrant point id. When `collection` is set the collection level is
elided, so the mount root is that collection:
```text
/fashion/
Men/
Shoes/
White/
3.json
3.txt
3.jpg
```
### Point files
A point is shown as its underlying data in its original format, never as the
embedding vector:
- `<id>.txt` is the embedded source text (the `text_field` value), exactly what
the vector was built from.
- `<id>.json` is the full payload as compact JSON (the metadata), with the vector
and the raw blob omitted.
- `<id>.<ext>` is the raw blob bytes when `blob_field` is configured.
```text
$ cat /fashion/Men/Shoes/White/3.txt
Nike Men White Running Sneakers
$ cat /fashion/Men/Shoes/White/3.json
{"gender":"Men","articleType":"Shoes","baseColour":"White","productDisplayName":"Nike Men White Running Sneakers","id":3}
```
## Semantic search
Search is a command, not a path. It returns each ranked point as its **canonical
content path** (the `<id>.txt`, or `<id>.json` when no `text_field` is set)
annotated with the similarity score, followed by the content:
```text
$ search "white running sneakers" /fashion
/fashion/Men/Shoes/White/3.txt:0.7421
Nike Men White Running Sneakers
```
Flags: `--top-k <n>` (default `search_limit`), `--threshold <min-score>`,
`--method semantic` (the only supported method; `grep`/`rg` stay lexical).
## Supported commands
All commands delegate to Mirage's shared implementations.
| Command | Behaviour on a Qdrant mount |
| ------------- | ------------------------------------------------------------- |
| `ls` | list collections, payload folders, or point files |
| `cd` | navigate (each level narrows the filter) |
| `tree` | render the folder hierarchy |
| `cat` | print a point's text/JSON, or dump raw blob/image bytes |
| `stat` | directory vs file, blob size, image mime type |
| `find` | walk the tree (e.g. `find /fashion -name '*.txt'`) |
| `grep` / `rg` | lexical search over the text/JSON files |
| `search` | semantic (vector) search -> ranked content paths + score |
| `head` / `tail` | first/last lines of a file |
| `wc` | count lines/bytes of a file |
## Access pattern
The mount is **read-only** (`MountMode.READ`); writes are not supported. The two
read modes are:
- **Browse** by payload folders: scroll filters on `group_by` fields, no embedding.
- **Search** by meaning: `search "<query>" <path>` runs vector search and returns
canonical point paths.
Folder listings are capped by `max_rows`. A filtered listing scrolls first and
only creates keyword payload indexes for the `group_by` fields if Qdrant reports
one is required, so already-indexed collections work under read-only keys. Keep
`group_by` to low-cardinality fields for large collections.
+82
View File
@@ -0,0 +1,82 @@
---
title: QingStor
icon: cloud
description: Mount a QingStor (QingCloud) Object Storage bucket via its S3-compatible API.
---
The QingStor resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `QingStorConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just derives the right endpoint from `region`
(the QingStor zone). Uses aioboto3 against QingStor's S3-compatible API.
The endpoint is computed from `region` as `s3.<region>.qingstor.com`
(e.g. `s3.pek3a.qingstor.com`). This is QingStor's S3-compatible host, distinct
from the native `<zone>.qingstor.com` API. Pass `endpoint_url` to override.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.qingstor import QingStorConfig, QingStorResource
config = QingStorConfig(
bucket=os.environ["QINGSTOR_BUCKET"],
region=os.environ.get("QINGSTOR_ZONE", "pek3a"),
access_key_id=os.environ["QINGSTOR_ACCESS_KEY_ID"],
secret_access_key=os.environ["QINGSTOR_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://s3.pek3a.qingstor.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = QingStorResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.qingstor import QingStorConfig, QingStorResource
load_dotenv(".env.development")
config = QingStorConfig(
bucket=os.environ["QINGSTOR_BUCKET"],
region=os.environ.get("QINGSTOR_ZONE", "pek3a"),
endpoint_url=os.environ.get("QINGSTOR_ENDPOINT_URL"),
access_key_id=os.environ["QINGSTOR_ACCESS_KEY_ID"],
secret_access_key=os.environ["QINGSTOR_SECRET_ACCESS_KEY"],
)
resource = QingStorResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("find /data/ -name '*.json' | head -n 5")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- QingStor reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [QingStor Setup](/home/setup/qingstor).
+239
View File
@@ -0,0 +1,239 @@
---
title: Cloudflare R2
icon: cloud
description: Mount Cloudflare R2 storage as a virtual filesystem.
---
The R2 resource mounts a Cloudflare R2 bucket at some prefix such as
`/r2/`. All operations involve network I/O to the remote object store.
Uses aioboto3 against R2's S3-compatible API.
For credential setup, see [R2 Setup](/home/setup/r2).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.r2 import R2Config, R2Resource
config = R2Config(
bucket=os.environ["R2_BUCKET"],
account_id=os.environ["R2_ACCOUNT_ID"],
access_key_id=os.environ["R2_ACCESS_KEY_ID"],
secret_access_key=os.environ["R2_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://...r2.cloudflarestorage.com",
# aws_profile="my-profile",
# region="auto",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = R2Resource(config=config)
ws = Workspace({"/r2": resource}, mode=MountMode.READ)
```
`R2Resource(config)` takes an `R2Config` object with the bucket name and
either `account_id` (endpoint is auto-built as
`https://{account_id}.r2.cloudflarestorage.com`) or an explicit
`endpoint_url`. Both `READ` and `WRITE` modes are supported.
## Filesystem Layout
The R2 resource maps object keys to virtual paths under the mount
prefix. R2 "directories" are prefix-based — there are no real directory
objects.
For example, if bucket `my-bucket` contains:
```text
assets/logo.png
assets/style.css
data/export.parquet
data/metrics.csv
```
Then mounting at `/r2/` exposes:
```text
/r2/
assets/
logo.png
style.css
data/
export.parquet
metrics.csv
```
Path mapping: virtual `/r2/assets/logo.png` maps to R2 key
`assets/logo.png`.
## Cache
The R2 resource uses `IndexCacheStore` with `index_ttl = 600`
(10 minutes). Directory listings are cached for up to 600 seconds before
being refreshed from R2. This reduces API calls for repeated directory
traversals.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.r2 import R2Config, R2Resource
load_dotenv(".env.development")
config = R2Config(
bucket=os.environ["R2_BUCKET"],
account_id=os.environ["R2_ACCOUNT_ID"],
access_key_id=os.environ["R2_ACCESS_KEY_ID"],
secret_access_key=os.environ["R2_SECRET_ACCESS_KEY"],
)
resource = R2Resource(config=config)
async def main() -> None:
ws = Workspace({"/r2/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /r2/")
print(await r.stdout_str())
r = await ws.execute("cat /r2/data/metrics.csv | head -n 10")
print(await r.stdout_str())
r = await ws.execute("tree /r2/")
print(await r.stdout_str())
r = await ws.execute("find /r2/ -name '*.csv'")
print(await r.stdout_str())
r = await ws.execute("stat /r2/assets/logo.png")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The R2 resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit
from range reads to avoid downloading entire objects.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **AI agents accessing R2 data**: Mount R2 buckets for agents to read and process datasets
- **Data pipelines**: Read and write R2 objects with shell-like commands
- **FUSE mounting**: Expose R2 buckets through a virtual FUSE mount for external tools
+218
View File
@@ -0,0 +1,218 @@
---
title: RAM
description: Use an in-memory Mirage filesystem for temporary files, tests, examples, and writable agent scratch space.
icon: memory
---
The RAM resource exposes an in-memory filesystem mounted at some prefix
such as `/data/`. All data lives in process memory and is lost on exit.
No setup or credentials required.
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.ram import RAMResource
resource = RAMResource()
ws = Workspace({"/data": resource}, mode=MountMode.WRITE)
```
No configuration object needed - `RAMResource()` takes no arguments.
## Filesystem Layout
The RAM filesystem starts empty. Files and directories are created
on demand via commands like `touch`, `mkdir`, `tee`, `cp`, etc.
```text
/data/
notes.txt
config.json
reports/
q1.csv
q2.csv
uploads/
image.png
```
Paths are normalized with a leading `/`. The root directory `/` always
exists.
## Cache
The RAM resource uses `IndexCacheStore` with `index_ttl = 0` (no
expiry). Since all data is in-memory, the index is always fresh,
no network calls or stale cache concerns.
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.ram import RAMResource
resource = RAMResource()
async def main() -> None:
ws = Workspace({"/data": resource}, mode=MountMode.WRITE)
# Create files
await ws.execute('echo "hello world" | tee /data/hello.txt')
await ws.execute('echo \'{"name": "alice", "age": 30}\' | tee /data/user.json')
await ws.execute("mkdir /data/reports")
await ws.execute('echo "revenue,100" | tee /data/reports/q1.csv')
# Read
r = await ws.execute("cat /data/hello.txt")
print(await r.stdout_str())
# List
r = await ws.execute("ls /data/")
print(await r.stdout_str())
# Query JSON
r = await ws.execute('jq ".name" /data/user.json')
print(await r.stdout_str())
# Search
r = await ws.execute("grep hello /data/hello.txt")
print(await r.stdout_str())
# Tree
r = await ws.execute("tree /data/")
print(await r.stdout_str())
# File info
r = await ws.execute("stat /data/hello.txt")
print(await r.stdout_str())
# Copy, move, remove
await ws.execute("cp /data/hello.txt /data/hello_copy.txt")
await ws.execute("mv /data/hello_copy.txt /data/renamed.txt")
await ws.execute("rm /data/renamed.txt")
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The RAM resource supports the full set of shell commands since it
handles real file content (text, binary, JSON, CSV, etc.):
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **Testing**: Fast filesystem for unit tests without I/O
- **Staging**: Temporary data area during pipelines
- **Prototyping**: Develop against a filesystem before switching to S3/disk
- **Caching**: In-process data exchange between commands
- **Ephemeral workspaces**: Create, process, discard - no cleanup needed
+122
View File
@@ -0,0 +1,122 @@
---
title: Redis
icon: bolt
description: Mount a Redis-backed virtual filesystem.
---
The Redis resource creates a persistent virtual filesystem backed by
Redis. It supports full read and write operations.
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.redis import RedisResource
resource = RedisResource(
url="redis://localhost:6379/0",
key_prefix="mirage:fs:",
)
ws = Workspace({"/redis": resource}, mode=MountMode.WRITE)
```
| Parameter | Required | Default | Description |
| ------------ | -------- | ------------------------ | ----------------------- |
| `url` | no | `redis://localhost:6379/0` | Redis connection URL |
| `key_prefix` | no | `mirage:fs:` | Key namespace prefix |
Keys are stored as:
- `{prefix}file:{path}` - file content (raw bytes)
- `{prefix}dir` - Redis set of directory paths
- `{prefix}modified:{path}` - modification timestamps
## Filesystem Layout
```text
/redis/
<file>
<directory>/
<file>
<subdirectory>/
<file>
```
The filesystem is fully user-defined - create any directory structure
with `mkdir`, `touch`, `tee`, etc.
Example:
```text
/redis/
config.json
logs/
app.log
error.log
cache/
session-abc.json
```
## Cache
Index TTL is `0` - always reads fresh state from Redis.
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.redis import RedisResource
resource = RedisResource(url="redis://localhost:6379/0")
async def main():
ws = Workspace({"/redis": resource}, mode=MountMode.WRITE)
# Create directories and files
await ws.execute("mkdir -p /redis/data/")
await ws.execute('echo "hello world" | tee /redis/data/hello.txt')
# Read back
r = await ws.execute("cat /redis/data/hello.txt")
print(await r.stdout_str())
# List files
r = await ws.execute("ls /redis/data/")
print(await r.stdout_str())
# Copy and move
await ws.execute("cp /redis/data/hello.txt /redis/data/copy.txt")
await ws.execute("mv /redis/data/copy.txt /redis/data/renamed.txt")
# Search
r = await ws.execute('grep "hello" /redis/data/')
print(await r.stdout_str())
# Clean up
await ws.execute("rm -r /redis/data/")
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
| Command | Notes |
| ---------------------- | ------------------------------------------ |
| `ls` | List files and directories |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search |
| `wc` | Line/word/byte counts |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `mkdir` | Create directories |
| `touch` | Create empty files |
| `cp` / `mv` / `rm` | Copy, move, delete |
| `tee` | Write stdin to file |
| `diff` / `cmp` | Compare files |
| `sort` / `cut` / `tr` | Text processing |
| `tar` / `zip` / `gzip` | Compression |
| `base64` / `md5` / `sha256sum` | Encoding and checksums |
+248
View File
@@ -0,0 +1,248 @@
---
title: S3
description: Mount AWS S3 or S3-compatible buckets as a Mirage filesystem with async access, caching, and shell commands.
icon: aws
---
The S3 resource mounts an Amazon S3 bucket (or any S3-compatible service) at
some prefix such as `/s3/`. All operations involve network I/O to the remote
object store. Uses aioboto3 for async S3 access.
Compatible with: AWS S3, MinIO, Cloudflare R2, Supabase Storage,
DigitalOcean Spaces, and any S3-compatible service.
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.s3 import S3Resource, S3Config
config = S3Config(
bucket="my-bucket",
region="us-east-1",
# Optional:
# aws_access_key_id="...",
# aws_secret_access_key="...",
# aws_profile="my-profile",
# endpoint_url="http://localhost:9000", # MinIO
# path_style=True, # For MinIO/R2
# timeout=30,
# proxy="http://proxy:8080",
)
resource = S3Resource(config)
ws = Workspace({"/s3": resource}, mode=MountMode.READ)
```
`S3Resource(config)` takes an `S3Config` object with the bucket name and
optional credentials, endpoint, and connection settings. Both `READ` and
`WRITE` modes are supported.
## Filesystem Layout
The S3 resource maps S3 object keys to virtual paths under the mount prefix.
S3 "directories" are prefix-based - there are no real directory objects.
For example, if bucket `my-bucket` contains:
```text
data/file.txt
data/config.json
reports/q1.csv
reports/q2.csv
```
Then mounting at `/s3/` exposes:
```text
/s3/
data/
file.txt
config.json
reports/
q1.csv
q2.csv
```
Path mapping: virtual `/s3/data/file.txt` maps to S3 key `data/file.txt`.
## Cache
The S3 resource uses `IndexCacheStore` with `index_ttl = 600` (10 minutes).
Directory listings are cached for up to 600 seconds before being refreshed
from S3. This reduces API calls for repeated directory traversals.
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.s3 import S3Resource, S3Config
config = S3Config(
bucket="my-bucket",
region="us-east-1",
)
resource = S3Resource(config)
async def main() -> None:
ws = Workspace({"/s3/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /s3/")
print(await r.stdout_str())
r = await ws.execute("cat /s3/data/file.txt")
print(await r.stdout_str())
r = await ws.execute("tree /s3/")
print(await r.stdout_str())
r = await ws.execute("find /s3/ -name '*.json'")
print(await r.stdout_str())
r = await ws.execute("grep example /s3/data/config.json")
print(await r.stdout_str())
r = await ws.execute("stat /s3/data/file.txt")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The S3 resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit
from range reads to avoid downloading entire objects.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **AI agents accessing cloud data**: Mount S3 buckets for agents to read and process remote datasets
- **Data pipelines**: Read and write S3 objects with shell-like commands
- **Sandboxed cloud storage access**: Restrict agent operations to a specific bucket and prefix
- **FUSE mounting**: Expose S3 buckets through a virtual FUSE mount for external tools
## Scoping a resource to a key prefix
Pass `key_prefix: str | None = None` to `S3Config` to transparently scope every operation to a subpath of the bucket:
```python
S3Resource(S3Config(
bucket="app-data",
region="eu-west-1",
key_prefix=f"users/{user_id}/",
))
```
When set, every read/write/list/stat/copy/rename/delete operation is transparently scoped to that bucket subpath. Agents see clean paths like `/data/notes.md`; the underlying bucket key is `users/{user_id}/data/notes.md`. Useful for multi-tenant systems. Pairs naturally with STS AssumeRole session policies for AWS-side enforcement. Unset behavior is unchanged.
**Normalization:** leading slashes are stripped and a trailing slash is added automatically. Both `None` and an empty string are treated as "no prefix."
+80
View File
@@ -0,0 +1,80 @@
---
title: Scaleway
icon: /images/scaleway-logo.svg
description: Mount a Scaleway Object Storage bucket via its S3-compatible API.
---
The Scaleway resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `ScalewayConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just derives the right endpoint from `region`.
Uses aioboto3 against Scaleway Object Storage's S3-compatible API.
The endpoint is computed from `region` as `s3.<region>.scw.cloud`
(e.g. `s3.fr-par.scw.cloud`). Pass `endpoint_url` to override.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.scaleway import ScalewayConfig, ScalewayResource
config = ScalewayConfig(
bucket=os.environ["SCW_BUCKET"],
region=os.environ.get("SCW_REGION", "fr-par"),
access_key_id=os.environ["SCW_ACCESS_KEY"],
secret_access_key=os.environ["SCW_SECRET_KEY"],
# Optional:
# endpoint_url="https://s3.fr-par.scw.cloud",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = ScalewayResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.scaleway import ScalewayConfig, ScalewayResource
load_dotenv(".env.development")
config = ScalewayConfig(
bucket=os.environ["SCW_BUCKET"],
region=os.environ.get("SCW_REGION", "fr-par"),
access_key_id=os.environ["SCW_ACCESS_KEY"],
secret_access_key=os.environ["SCW_SECRET_KEY"],
)
resource = ScalewayResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("tree /data/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- Scaleway reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [Scaleway Setup](/home/setup/scaleway).
+89
View File
@@ -0,0 +1,89 @@
---
title: SeaweedFS
icon: /images/seaweedfs-logo.svg
description: Mount a SeaweedFS bucket as a virtual filesystem.
---
The SeaweedFS resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `SeaweedFSConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just points at your SeaweedFS S3 gateway
`endpoint_url` and uses path-style addressing by default.
SeaweedFS exposes an S3-compatible gateway (default port `8333`), so
`endpoint_url` is **required** (there is no region-derived host). Uses aioboto3
against the SeaweedFS S3 API.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.seaweedfs import SeaweedFSConfig, SeaweedFSResource
config = SeaweedFSConfig(
bucket=os.environ["SEAWEEDFS_BUCKET"],
endpoint_url=os.environ.get("SEAWEEDFS_ENDPOINT", "http://localhost:8333"),
access_key_id=os.environ["SEAWEEDFS_ACCESS_KEY"],
secret_access_key=os.environ["SEAWEEDFS_SECRET_KEY"],
# Optional:
# region="us-east-1", # default
# path_style=True, # default; SeaweedFS uses path-style addressing
# timeout=30,
# proxy="http://proxy:8080",
)
resource = SeaweedFSResource(config)
ws = Workspace({"/seaweedfs": resource}, mode=MountMode.WRITE)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.seaweedfs import SeaweedFSConfig, SeaweedFSResource
load_dotenv(".env.development")
config = SeaweedFSConfig(
bucket=os.environ.get("SEAWEEDFS_BUCKET", "mirage-demo"),
endpoint_url=os.environ.get("SEAWEEDFS_ENDPOINT", "http://localhost:8333"),
access_key_id=os.environ.get("SEAWEEDFS_ACCESS_KEY", "any"),
secret_access_key=os.environ.get("SEAWEEDFS_SECRET_KEY", "any"),
)
resource = SeaweedFSResource(config)
async def main() -> None:
ws = Workspace({"/seaweedfs/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /seaweedfs/")
print(await r.stdout_str())
r = await ws.execute("tree /seaweedfs/")
print(await r.stdout_str())
r = await ws.execute("cat /seaweedfs/data/config.json")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- SeaweedFS reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies (`ls`, `cat`,
`head`, `tail`, `grep`, `rg`, `wc`, `find`, `tree`, `jq`, `stat`, plus
parquet/orc/feather table rendering). See the [S3 resource](/python/resource/s3)
for the complete command reference, range reads, streaming, and the index
cache fast path.
- Blaxel Agent Drive is backed by SeaweedFS. To mount a drive's S3 gateway,
point `endpoint_url` at the drive's `s3Url` and supply credentials for it.
- For credential setup, see [SeaweedFS Setup](/home/setup/seaweedfs).
+207
View File
@@ -0,0 +1,207 @@
---
title: SharePoint
description: Mount Microsoft SharePoint Online sites and document libraries as a Mirage filesystem with multi-site discovery, versioning, and shell commands.
icon: microsoft
---
The SharePoint resource mounts Microsoft SharePoint Online at some prefix such
as `/sharepoint/`. Unlike the single-drive OneDrive resource, it exposes the
full enterprise topology: many sites, each with many document libraries. Site
and drive names are resolved to Microsoft Graph ids automatically, so you mount
once and browse everything you have access to. All operations involve network
I/O to Microsoft Graph. Files are served as raw bytes (no Office filetype
conversion). Uses aiohttp for async Graph access.
For credential setup (tokens, app vs delegated auth, permissions), see the
[SharePoint Setup](/home/setup/sharepoint) guide.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.sharepoint import SharePointResource, SharePointConfig
config = SharePointConfig(
access_token=os.environ["MS_GRAPH_DRIVE_TOKEN"],
# Optional:
# tenant_host="contoso.sharepoint.com", # restrict to one tenant host
# site_filter="Engineering", # narrow the site search
# timeout=30,
# max_retries=5,
)
resource = SharePointResource(config)
ws = Workspace({"/sharepoint": resource}, mode=MountMode.WRITE)
```
`SharePointResource(config)` takes a `SharePointConfig` object with a Microsoft
Graph bearer token. No drive id is required: the resource discovers sites and
drives for you. Both `READ` and `WRITE` modes are supported.
### Config Reference
| Field | Required | Description |
| -------------- | -------- | ----------------------------------------------------------------- |
| `access_token` | Yes | Microsoft Graph OAuth2 bearer token (string or callable provider) |
| `tenant_host` | No | Restrict resolution to a single SharePoint host |
| `site_filter` | No | Search term passed to `/sites?search=` to narrow discovery |
| `timeout` | No | Request timeout in seconds (default `30`) |
| `max_retries` | No | Max retries on `429`/`5xx` with backoff (default `5`) |
`access_token` accepts either a static string or a `Callable[[], str]` provider.
With a callable, the resource refreshes the token on `401`, so long-running
mounts survive token expiry.
## Filesystem Layout
SharePoint paths follow the structure `/{site}/{library}/{path}`. The first two
levels are virtual (sites and drives), and everything below is a real Graph
driveItem.
- **Level 0** (`/sharepoint/`): lists all accessible sites
- **Level 1** (`/sharepoint/{site}/`): lists document libraries (drives) in that site
- **Level 2+** (`/sharepoint/{site}/{library}/...`): lists files and folders
For example:
```text
/sharepoint/
Engineering/
Documents/
spec.md
deck.pptx
Reports/
q1.csv
Marketing/
Shared Documents/
brand.pdf
```
Path mapping: virtual `/sharepoint/Engineering/Documents/spec.md` resolves the
site `Engineering` to a site id, the library `Documents` to a drive id, then
reads the driveItem at `/root:/spec.md`.
## Versioning and Snapshots
SharePoint keeps per-file version history. The resource exposes it the same way
the OneDrive and S3 backends do:
- **Fingerprint** is the driveItem `cTag`, so normal reads and `stat` reflect
the current content without extra Graph calls.
- **Snapshots** pin each path to a Graph driveItem version id. Replaying a
snapshot reads `/versions/{id}/content`, giving time-travel to the exact bytes
captured at snapshot time.
- **Restore** writes a previous version back as the current one.
This makes the SharePoint resource `SUPPORTS_SNAPSHOT = True`.
## Cache
The SharePoint resource caches site, drive, and directory listings via
`IndexCacheStore` to reduce repeated Graph calls during traversal.
## Example
```python
import asyncio
import os
from mirage import MountMode, Workspace
from mirage.resource.sharepoint import SharePointResource, SharePointConfig
config = SharePointConfig(access_token=os.environ["MS_GRAPH_DRIVE_TOKEN"])
resource = SharePointResource(config)
async def main() -> None:
ws = Workspace({"/sharepoint/": resource}, mode=MountMode.WRITE)
# list sites, then drives in the first site
r = await ws.execute("ls /sharepoint/")
print(await r.stdout_str())
r = await ws.execute("ls /sharepoint/Engineering/")
print(await r.stdout_str())
await ws.execute(
"echo 'hello from mirage' > /sharepoint/Engineering/Documents/note.txt")
r = await ws.execute("cat /sharepoint/Engineering/Documents/note.txt")
print(await r.stdout_str())
r = await ws.execute("tree /sharepoint/Engineering/Documents/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
Runnable versions live at `examples/python/sharepoint/sharepoint.py` (command
matrix) and `examples/python/sharepoint/sharepoint_vfs.py` (FUSE mount).
## Shell Commands
The SharePoint resource supports the full set of shell commands since it
operates on real file content (text, binary, JSON, CSV, etc.). Large files
benefit from range reads to avoid downloading entire items.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### File Operations
| Command | Notes |
| ------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `tee` | Write stdin to file and stdout |
### Path Utilities
| Command | Notes |
| ---------- | ------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `ls` | List directory contents |
## Use Cases
- **AI agents accessing org documents**: Mount enterprise SharePoint for agents to read and process files across many sites
- **Multi-site discovery**: Browse every accessible site and library from a single mount, no drive ids to manage
- **Versioned document workflows**: Snapshot and replay exact file states over time
- **FUSE mounting**: Expose SharePoint through a virtual FUSE mount for external tools
## SharePoint vs OneDrive
Both backends talk to Microsoft Graph driveItems and behave identically below
the drive level. Choose by topology:
- Use **[OneDrive](/python/resource/onedrive)** for a single drive (personal
OneDrive, OneDrive for Business, or one targeted SharePoint library via
`drive_id` / `site_id`).
- Use **SharePoint** for enterprise scenarios with many sites and libraries that
you want to discover and traverse without knowing drive ids up front.
+331
View File
@@ -0,0 +1,331 @@
---
title: Slack
description: Mount Slack channels, DMs, messages, users, and shared files as a Mirage virtual filesystem for Python agents.
icon: slack
---
The Slack resource exposes a Slack workspace as a virtual filesystem
mounted at some prefix such as `/slack/`.
For token setup, see [Slack Setup](/python/setup/slack).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.slack import SlackConfig, SlackResource
config = SlackConfig(token=os.environ["SLACK_BOT_TOKEN"])
resource = SlackResource(config=config)
ws = Workspace({"/slack": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/slack/
channels/
<channel-name>__<channel-id>/
<yyyy-mm-dd>/
chat.jsonl
files/
<name>__<F-id>.<ext>
dms/
<user-name>__<dm-id>/
<yyyy-mm-dd>/
chat.jsonl
files/
<name>__<F-id>.<ext>
users/
<username>__<user-id>.json
...
```
Example:
```text
/slack/
channels/
general__C04KEPWF6V7/
2026-04-04/
chat.jsonl
files/
2026-04-05/
chat.jsonl
files/
random__C04JVGZM7UN/
2026-04-11/
chat.jsonl
files/
dms/
alice__D0AQDJ5FP9V/
2026-04-04/
chat.jsonl
files/
slackbot__D0ARPA1QSPJ/
2026-04-04/
chat.jsonl
files/
users/
alice__U12345678.json
bob__U87654321.json
```
Directory names embed the Slack ID so that write commands
(`slack-post-message --channel_id`, etc.) can reference the correct
resource without extra lookups.
### Channels
`/slack/channels/` lists public and private channels the bot has
access to. The channel ID is appended after `__`. Each channel
directory contains day-partitioned directories for the last 90 days
(or since channel creation, whichever is shorter). Each date directory
contains `chat.jsonl` plus a `files/` directory for attachments shared
that day.
The date range is derived from the channel's `created` timestamp.
### DMs
`/slack/dms/` lists direct message conversations. The DM ID is
appended after `__`. Like channels, DM directories contain daily
directories with `chat.jsonl` and `files/`.
### Users
`/slack/users/` lists one `.json` file per non-deleted, non-bot user.
Reading a user file calls `get_user_profile()` and returns the full
profile JSON from the Slack API.
## Cache
The Slack resource uses `IndexCacheStore` (same as Discord and other
resources). Index entries store channel IDs, DM IDs, user IDs, and
channel creation timestamps for date range computation. There is no
separate content cache - file content caching is handled by the
workspace `IOResult` mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
config = SlackConfig(token=os.environ["SLACK_BOT_TOKEN"])
resource = SlackResource(config=config)
async def main():
ws = Workspace({"/slack": resource}, mode=MountMode.READ)
# List structure
r = await ws.execute("ls /slack/")
print(await r.stdout_str())
# List channels
r = await ws.execute("ls /slack/channels/")
print(await r.stdout_str())
ch = r.stdout_str().strip().splitlines()[0].strip()
base = f"/slack/channels/{ch}"
# Read messages from a specific date
r = await ws.execute(f'cat "{base}/2026-04-04/chat.jsonl" | head -n 3')
print(await r.stdout_str())
# Read user profile
r = await ws.execute("cat /slack/users/alice__U12345678.json")
print(await r.stdout_str())
# Extract message text with jq
r = await ws.execute(
f'cat "{base}/2026-04-04/chat.jsonl"'
' | jq -r ".[] | .text" | head -n 5')
print(await r.stdout_str())
# Search across channel (scans all date files)
r = await ws.execute(f'rg message "{base}/"')
print(await r.stdout_str())
# Tree view
r = await ws.execute("tree -L 1 /slack/")
print(await r.stdout_str())
# Navigate with cd/pwd
await ws.execute(f'cd "{base}"')
r = await ws.execute("pwd")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/python/slack/slack.py` for the full working example.
## Finding IDs
Resource-specific commands require Slack IDs (`channel_id`, `user_id`,
`ts`). These can be extracted from the filesystem:
```bash
# Channel ID - embedded in directory name after "__"
ls /slack/channels/
# → general__C04KEPWF6V7 ← channel_id = C04KEPWF6V7
# User ID - embedded in filename after "__"
ls /slack/users/
# → alice__U04K21SEVR9.json ← user_id = U04K21SEVR9
# Message timestamp (ts) - inside JSONL messages
jq -r '.[] | "\(.ts) [\(.user)] \(.text)"' \
"/slack/channels/general__C04KEPWF6V7/2026-04-04/chat.jsonl"
# → 1712345678.123456 [U04K21SEVR9] hello world
# Find a specific message then reply
jq -r '.[] | select(.text | test("hello")) | .ts' \
"/slack/channels/general__C04KEPWF6V7/2026-04-04/chat.jsonl"
# → 1712345678.123456
slack-reply-to-thread --channel_id C04KEPWF6V7 \
--ts 1712345678.123456 --text "Reply to hello"
```
## Working with Large Channels
Channels with many messages produce large `chat.jsonl` files per day.
Tips for efficient access:
```bash
# Check message count per day
wc -l "/slack/channels/general__C04KEPWF6V7/2026-04-04/chat.jsonl"
# Read only recent messages
tail -n 10 "/slack/channels/general__C04KEPWF6V7/2026-04-04/chat.jsonl"
# Search across all dates (scans each file)
rg "keyword" "/slack/channels/general__C04KEPWF6V7/"
# Extract specific fields to reduce output
jq -r '.[] | "\(.ts) [\(.user)] \(.text)"' \
"/slack/channels/general__C04KEPWF6V7/2026-04-04/chat.jsonl" | head -n 20
# Count messages per user
cat "/slack/channels/general__C04KEPWF6V7/2026-04-04/chat.jsonl" \
| jq -r '.[] | .user' | sort | uniq -c
```
## Shell Commands
Standard commands available on the mounted Slack tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List channels, DMs, users, dates |
| `cat` | Read `chat.jsonl`, user `.json`, or files |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON; use `.[]` prefix for JSONL |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
Resource-specific commands:
### `slack-post-message`
Post a message to a channel.
```bash
slack-post-message --channel_id C04KEPWF6V7 --text "Hello from MIRAGE"
```
| Option | Required | Description |
| -------------- | -------- | -------------------- |
| `--channel_id` | yes | Slack channel ID |
| `--text` | yes | Message text to send |
Returns the posted message JSON.
### `slack-reply-to-thread`
Reply to a specific thread.
```bash
slack-reply-to-thread --channel_id C04KEPWF6V7 --ts 1712345678.123456 --text "Thread reply"
```
| Option | Required | Description |
| -------------- | -------- | ----------------------- |
| `--channel_id` | yes | Slack channel ID |
| `--ts` | yes | Thread parent timestamp |
| `--text` | yes | Reply text |
The `--ts` value is the message timestamp (e.g., from a `.jsonl`
entry's `ts` field). Returns the posted reply JSON.
### `slack-add-reaction`
Add an emoji reaction to a message.
```bash
slack-add-reaction --channel_id C04KEPWF6V7 --ts 1712345678.123456 --reaction thumbsup
```
| Option | Required | Description |
| -------------- | -------- | ----------------------------- |
| `--channel_id` | yes | Slack channel ID |
| `--ts` | yes | Message timestamp to react to |
| `--reaction` | yes | Emoji name (without colons) |
### `slack-get-users`
Search for users by name, real name, or email.
```bash
slack-get-users --query "alice"
```
| Option | Required | Description |
| --------- | -------- | ------------ |
| `--query` | yes | Search query |
Returns matching users as JSON array.
### `slack-get-user-profile`
Get a single user's full profile.
```bash
slack-get-user-profile --user_id U04K21SEVR9
```
| Option | Required | Description |
| ----------- | -------- | ------------- |
| `--user_id` | yes | Slack user ID |
Returns the user profile JSON. The user ID can be found in
filenames under `/slack/users/` (e.g., `alice__U04K21SEVR9.json`).
### `slack-search`
Search messages across the workspace.
```bash
slack-search --query "incident report"
```
| Option | Required | Description |
| --------- | -------- | ------------ |
| `--query` | yes | Search query |
Returns search results as JSON.
+132
View File
@@ -0,0 +1,132 @@
---
title: SSH
icon: terminal
description: Mount a remote filesystem over SSH/SFTP.
---
The SSH resource mounts a remote server's filesystem over SFTP.
It supports full read and write operations.
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.ssh import SSHConfig, SSHResource
config = SSHConfig(
host="myserver",
username="deploy",
identity_file="~/.ssh/id_ed25519",
root="/var/data",
)
resource = SSHResource(config=config)
ws = Workspace({"/remote": resource}, mode=MountMode.WRITE)
```
| Field | Required | Default | Description |
| --------------- | -------- | ------- | ------------------------------------ |
| `host` | yes | | SSH host (name or IP) |
| `hostname` | no | | Override resolved hostname |
| `port` | no | `22` | SSH port |
| `username` | no | | SSH username |
| `identity_file` | no | | Path to private key |
| `root` | no | `/` | Remote directory to mount |
| `timeout` | no | `30` | Connection timeout in seconds |
| `known_hosts` | no | | Path to known_hosts file |
The `host` field matches entries in `~/.ssh/config`, so existing SSH
configurations are automatically picked up.
## Filesystem Layout
```text
/remote/
<remote-directory-tree>
```
The mounted tree mirrors the remote filesystem starting at `root`.
Example with `root="/var/data"`:
```text
/remote/
logs/
app.log
nginx/
access.log
error.log
config/
app.yaml
uploads/
image.png
```
## Cache
Uses `IndexCacheStore` for directory listings. Freshness is checked
via `{mtime}:{size}` fingerprints - files are re-fetched only when
the remote has changed.
## Example
```python
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.ssh import SSHConfig, SSHResource
config = SSHConfig(
host="myserver",
username="deploy",
identity_file="~/.ssh/id_ed25519",
root="/var/log",
)
resource = SSHResource(config=config)
async def main():
ws = Workspace({"/logs": resource}, mode=MountMode.READ)
# List remote directory
r = await ws.execute("ls /logs/")
print(await r.stdout_str())
# Read last 20 lines of a log
r = await ws.execute("tail -n 20 /logs/nginx/access.log")
print(await r.stdout_str())
# Search across log files
r = await ws.execute('grep "ERROR" /logs/app.log')
print(await r.stdout_str())
# Find large files
r = await ws.execute("find /logs/ -name '*.log'")
print(await r.stdout_str())
# File metadata
r = await ws.execute("stat /logs/app.log")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
| Command | Notes |
| ---------------------- | ------------------------------------------ |
| `ls` | List remote files and directories |
| `cat` | Read remote file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (use targeted paths) |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (size, mtime) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `mkdir` | Create remote directories |
| `touch` | Create empty remote files |
| `cp` / `mv` / `rm` | Copy, move, delete on remote |
| `tee` | Write stdin to remote file |
| `diff` / `cmp` | Compare remote files |
| `sort` / `cut` / `tr` | Text processing |
| `tar` / `zip` / `gzip` | Compression |
+239
View File
@@ -0,0 +1,239 @@
---
title: Supabase Storage
icon: bolt
description: Mount Supabase Storage as a virtual filesystem.
---
The Supabase resource mounts a Supabase Storage bucket at some prefix
such as `/supabase/`. All operations involve network I/O to the remote
object store. Uses aioboto3 against Supabase's S3-compatible API.
For credential setup, see [Supabase Setup](/home/setup/supabase).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.supabase import SupabaseConfig, SupabaseResource
config = SupabaseConfig(
bucket=os.environ["SUPABASE_BUCKET"],
region=os.environ["SUPABASE_REGION"],
project_ref=os.environ["SUPABASE_PROJECT_REF"],
access_key_id=os.environ["SUPABASE_ACCESS_KEY_ID"],
secret_access_key=os.environ["SUPABASE_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://{project_ref}.storage.supabase.co/storage/v1/s3",
# session_token="...",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = SupabaseResource(config=config)
ws = Workspace({"/supabase": resource}, mode=MountMode.READ)
```
`SupabaseResource(config)` takes a `SupabaseConfig` object with the
bucket name, `region`, and either `project_ref` (endpoint is auto-built
as `https://{project_ref}.storage.supabase.co/storage/v1/s3`) or an
explicit `endpoint_url`. Both `READ` and `WRITE` modes are supported.
## Filesystem Layout
The Supabase resource maps object keys to virtual paths under the mount
prefix. Supabase "directories" are prefix-based — there are no real
directory objects.
For example, if bucket `my-bucket` contains:
```text
avatars/user-001.png
avatars/user-002.png
documents/report.pdf
documents/data.csv
```
Then mounting at `/supabase/` exposes:
```text
/supabase/
avatars/
user-001.png
user-002.png
documents/
report.pdf
data.csv
```
Path mapping: virtual `/supabase/avatars/user-001.png` maps to Supabase
key `avatars/user-001.png`.
## Cache
The Supabase resource uses `IndexCacheStore` with `index_ttl = 600`
(10 minutes). Directory listings are cached for up to 600 seconds before
being refreshed from Supabase.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.supabase import SupabaseConfig, SupabaseResource
load_dotenv(".env.development")
config = SupabaseConfig(
bucket=os.environ["SUPABASE_BUCKET"],
region=os.environ["SUPABASE_REGION"],
project_ref=os.environ["SUPABASE_PROJECT_REF"],
access_key_id=os.environ["SUPABASE_ACCESS_KEY_ID"],
secret_access_key=os.environ["SUPABASE_SECRET_ACCESS_KEY"],
)
resource = SupabaseResource(config=config)
async def main() -> None:
ws = Workspace({"/supabase/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /supabase/")
print(await r.stdout_str())
r = await ws.execute("cat /supabase/documents/data.csv | head -n 10")
print(await r.stdout_str())
r = await ws.execute("tree -L 2 /supabase/")
print(await r.stdout_str())
r = await ws.execute("find /supabase/ -name '*.pdf'")
print(await r.stdout_str())
r = await ws.execute("stat /supabase/avatars/user-001.png")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Shell Commands
The Supabase resource supports the full set of shell commands since it operates
on real file content (text, binary, JSON, CSV, etc.). Large files benefit
from range reads to avoid downloading entire objects.
### Read Commands
| Command | Notes |
| --------------- | ------------------------------------------ |
| `cat` | Read file content |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, modified) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `nl` | Number lines |
| `du` | Disk usage summary |
| `file` | Detect file type |
| `strings` | Extract printable strings from binary |
| `xxd` | Hex dump |
| `md5` | MD5 checksum |
| `sha256sum` | SHA-256 checksum |
### Text Processing
| Command | Notes |
| ---------- | ------------------------------------------- |
| `awk` | Pattern scanning and processing |
| `sed` | Stream editor |
| `tr` | Translate or delete characters |
| `sort` | Sort lines |
| `uniq` | Remove duplicate lines |
| `cut` | Extract fields/columns |
| `join` | Join lines on a common field |
| `paste` | Merge lines side by side |
| `column` | Columnate output |
| `fold` | Wrap lines to a specified width |
| `expand` | Convert tabs to spaces |
| `unexpand` | Convert spaces to tabs |
| `fmt` | Simple text formatter |
| `rev` | Reverse lines |
| `tac` | Concatenate and print in reverse |
| `look` | Display lines beginning with a given string |
| `shuf` | Shuffle lines |
| `tsort` | Topological sort |
| `comm` | Compare two sorted files |
| `cmp` | Compare two files byte by byte |
| `diff` | Compare files line by line |
| `patch` | Apply a diff patch |
| `iconv` | Character encoding conversion |
### File Operations
| Command | Notes |
| -------- | ------------------------------------- |
| `cp` | Copy files |
| `mv` | Move/rename files |
| `rm` | Remove files |
| `mkdir` | Create directories |
| `touch` | Create empty file or update timestamp |
| `ln` | Create symbolic links |
| `tee` | Write stdin to file and stdout |
| `mktemp` | Create temporary file |
| `split` | Split file into pieces |
| `csplit` | Split file by context |
### Path Utilities
| Command | Notes |
| ---------- | -------------------------- |
| `basename` | Strip directory from path |
| `dirname` | Strip filename from path |
| `realpath` | Resolve path |
| `readlink` | Print symbolic link target |
| `ls` | List directory contents |
### Compression
| Command | Notes |
| -------- | --------------------- |
| `gzip` | Compress files |
| `gunzip` | Decompress gzip files |
| `zip` | Create zip archives |
| `unzip` | Extract zip archives |
| `tar` | Archive files |
| `zcat` | Cat compressed files |
| `zgrep` | Grep compressed files |
### Encoding
| Command | Notes |
| -------- | -------------------- |
| `base64` | Base64 encode/decode |
### Data Format Support
Commands with format-specific variants for structured data files:
| Format | Extension | Variants |
| ------- | ---------- | ---------------------------------------------- |
| Parquet | `.parquet` | cat, head, tail, wc, stat, cut, grep, ls, file |
| Feather | `.feather` | cat, head, tail, wc, stat, cut, grep, ls, file |
| ORC | `.orc` | cat, head, tail, wc, stat, cut, grep, ls, file |
| HDF5 | `.hdf5` | cat, head, tail, wc, stat, cut, grep, ls, file |
These variants auto-detect the format by extension and convert to
tabular text (CSV) for processing.
## Use Cases
- **AI agents accessing Supabase data**: Mount Supabase buckets for agents to read and process datasets
- **Data pipelines**: Read and write Supabase objects with shell-like commands
- **FUSE mounting**: Expose Supabase buckets through a virtual FUSE mount for external tools
+85
View File
@@ -0,0 +1,85 @@
---
title: Tencent COS
icon: /images/tencent-logo.svg
description: Mount a Tencent Cloud Object Storage bucket via its S3-compatible API.
---
The Tencent resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `TencentConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just derives the right COS endpoint from
`region`. Uses aioboto3 against Tencent COS's S3-compatible API.
The endpoint is computed from `region` as `cos.<region>.myqcloud.com`
(e.g. `cos.ap-guangzhou.myqcloud.com`). Pass `endpoint_url` to override.
<Note>
Tencent COS bucket names include the APPID suffix, e.g.
`my-bucket-1250000000`. Use the full name (with the suffix) for `bucket`.
</Note>
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.tencent import TencentConfig, TencentResource
config = TencentConfig(
bucket=os.environ["COS_BUCKET"], # e.g. my-bucket-1250000000
region=os.environ.get("COS_REGION", "ap-guangzhou"),
access_key_id=os.environ["COS_SECRET_ID"],
secret_access_key=os.environ["COS_SECRET_KEY"],
# Optional:
# endpoint_url="https://cos.ap-guangzhou.myqcloud.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = TencentResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.tencent import TencentConfig, TencentResource
load_dotenv(".env.development")
config = TencentConfig(
bucket=os.environ["COS_BUCKET"],
region=os.environ.get("COS_REGION", "ap-guangzhou"),
access_key_id=os.environ["COS_SECRET_ID"],
secret_access_key=os.environ["COS_SECRET_KEY"],
)
resource = TencentResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("find /data/ -name '*.json' | head -n 5")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- Tencent COS reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [Tencent COS Setup](/home/setup/tencent).
+396
View File
@@ -0,0 +1,396 @@
---
title: Trello
description: Mount Trello workspaces, boards, lists, cards, members, and labels as a Mirage filesystem for Python agents.
icon: table-columns
---
The Trello resource exposes workspaces, boards, lists, cards, members, and
labels as a virtual filesystem mounted at some prefix such as `/trello/`.
For API key setup, see [Trello Setup](/python/setup/trello).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.trello import TrelloConfig, TrelloResource
config = TrelloConfig(
api_key=os.environ["TRELLO_API_KEY"],
api_token=os.environ["TRELLO_API_TOKEN"],
)
resource = TrelloResource(config=config)
ws = Workspace({"/trello": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/trello/
workspaces/
<workspace-name>__<workspace-id>/
workspace.json
boards/
<board-name>__<board-id>/
board.json
members/
<full-name>__<member-id>.json
...
labels/
<label-name>__<label-id>.json
...
lists/
<list-name>__<list-id>/
list.json
cards/
<card-name>__<card-id>/
card.json
comments.jsonl
...
...
```
Example:
```text
/trello/
workspaces/
engineering__abc123def/
workspace.json
boards/
product-roadmap__brd_001/
board.json
members/
alice__mem_001.json
bob__mem_002.json
labels/
bug__lbl_001.json
feature__lbl_002.json
lists/
backlog__lst_001/
list.json
cards/
fix-login__crd_001/
card.json
comments.jsonl
add-search__crd_002/
card.json
comments.jsonl
in-progress__lst_002/
list.json
cards/
...
```
Directory and file names embed the Trello ID after `__` so that
resource-specific commands can reference the correct resource without
extra lookups.
### Workspaces
Each workspace directory is named:
```text
<workspace-name>__<workspace-id>
```
Inside the workspace directory:
- `workspace.json` is the normalized workspace metadata
- `boards/` contains one directory per board
### Boards
Each board directory is named:
```text
<board-name>__<board-id>
```
Inside the board directory:
- `board.json` is the normalized board metadata
- `members/` contains one JSON file per board member
- `labels/` contains one JSON file per board label
- `lists/` contains one directory per list
### Lists
Each list directory is named:
```text
<list-name>__<list-id>
```
Each list directory contains:
- `list.json` -- normalized list metadata with fields such as `list_id`,
`list_name`, `board_id`, `closed`, `pos`
- `cards/` contains one directory per card
### Cards
Each card directory is named:
```text
<card-name>__<card-id>
```
Each card directory contains:
- `card.json` -- normalized card metadata with command-aligned fields such as
`card_id`, `board_id`, `list_id`, `member_ids`, `label_ids`, `due`, `url`
- `comments.jsonl` -- normalized comment stream ordered by `created_at`
`comments.jsonl` is a Mirage representation chosen for shell-friendly
workflows. It is not a native Trello file format.
### Members
Each member file is named:
```text
<full-name>__<member-id>.json
```
Member JSON includes command-aligned identifiers such as `member_id` and
`username` so the value can be reused directly in commands like
`trello-card-assign`.
### Labels
Each label file is named:
```text
<label-name>__<label-id>.json
```
Label JSON includes `label_id`, `label_name`, `color`, and `board_id`.
## Cache
Uses `IndexCacheStore` (same as other resources).
## Example
See `examples/trello/trello.py` for the full working example.
```bash
# List workspaces
ls /trello/workspaces/
# Read workspace metadata
cat /trello/workspaces/engineering__abc123/workspace.json
# List boards
ls /trello/workspaces/engineering__abc123/boards/
# Read board metadata
cat /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/board.json
# List lists
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/
# Read a card
cat /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/fix-login__crd_001/card.json
# Read card comments
cat /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/fix-login__crd_001/comments.jsonl
# Extract card ID with jq
cat /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/fix-login__crd_001/card.json \
| jq '.card_id'
# Read last 5 comments
tail -n 5 /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/fix-login__crd_001/comments.jsonl
# List members
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/members/
# List labels
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/labels/
# Tree view
tree -L 2 /trello/workspaces/
```
## Finding IDs
IDs are embedded in directory and file names after `__`:
```bash
# Workspace ID -- embedded in directory name
ls /trello/workspaces/
# -> engineering__abc123 <- workspace_id = abc123
# Board ID -- embedded in board directory name
ls /trello/workspaces/engineering__abc123/boards/
# -> roadmap__brd_001 <- board_id = brd_001
# List ID -- embedded in list directory name
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/
# -> backlog__lst_001 <- list_id = lst_001
# Card ID -- embedded in card directory name
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/
# -> fix-login__crd_001 <- card_id = crd_001
# Member ID -- embedded in member file name
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/members/
# -> alice__mem_001.json <- member_id = mem_001
# Label ID -- embedded in label file name
ls /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/labels/
# -> bug__lbl_001.json <- label_id = lbl_001
# Use stat for structured metadata
stat /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/fix-login__crd_001/card.json
# Extract fields with jq
cat /trello/workspaces/engineering__abc123/boards/roadmap__brd_001/lists/backlog__lst_001/cards/fix-login__crd_001/card.json \
| jq '{card_id, board_id, list_id, member_ids, label_ids}'
```
## Shell Commands
Standard commands available on the mounted Trello tree:
| Command | Notes |
| --------------- | ------------------------------------------ |
| `ls` | List workspaces, boards, lists, cards |
| `cat` | Read .json metadata or .jsonl comments |
| `head` / `tail` | First/last N lines |
| `grep` / `rg` | Pattern search (file or directory level) |
| `jq` | Query JSON fields |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
| `basename` | Extract file name from path |
| `dirname` | Extract directory from path |
| `realpath` | Resolve path to absolute form |
## Resource-Specific Commands
### `trello-card-create`
Create a new card. Description can be passed via `--desc`, `--desc_file`, or stdin.
```bash
trello-card-create --list_id LST123 --name "Fix login"
trello-card-create --list_id LST123 --name "Fix login" --desc "Details here"
cat brief.md | trello-card-create --list_id LST123 --name "Agent bug"
```
| Option | Required | Description |
| ------------- | -------- | ----------------------------------- |
| `--list_id` | yes | Trello list ID |
| `--name` | yes | Card name |
| `--desc` | no | Inline description text |
| `--desc_file` | no | Path to file containing description |
### `trello-card-update`
Update an existing card. Description can be passed via `--desc`, `--desc_file`, or stdin.
```bash
trello-card-update --card_id CRD123 --name "Updated title"
trello-card-update --card_id CRD123 --desc "New description"
trello-card-update --card_id CRD123 --closed true
```
| Option | Required | Description |
| ------------- | -------- | ----------------------------------- |
| `--card_id` | yes | Trello card ID |
| `--name` | no | New card name |
| `--desc` | no | Inline description text |
| `--desc_file` | no | Path to file containing description |
| `--due` | no | Due date |
| `--closed` | no | Archive card (true/false) |
### `trello-card-move`
Move a card to a different list.
```bash
trello-card-move --card_id CRD123 --list_id LST999
```
| Option | Required | Description |
| ----------- | -------- | -------------- |
| `--card_id` | yes | Trello card ID |
| `--list_id` | yes | Trello list ID |
### `trello-card-assign`
Assign a member to a card.
```bash
trello-card-assign --card_id CRD123 --member_id MEM001
```
| Option | Required | Description |
| ------------- | -------- | ---------------- |
| `--card_id` | yes | Trello card ID |
| `--member_id` | yes | Trello member ID |
### `trello-card-comment-add`
Add a comment to a card. Text can be passed via `--text`, `--text_file`, or stdin.
```bash
trello-card-comment-add --card_id CRD123 --text "Looks good"
cat note.md | trello-card-comment-add --card_id CRD123
```
| Option | Required | Description |
| ------------- | -------- | ---------------------------- |
| `--card_id` | yes | Trello card ID |
| `--text` | \* | Inline comment text |
| `--text_file` | \* | Path to file containing text |
\* One of `--text`, `--text_file`, or stdin is required.
### `trello-card-comment-update`
Update an existing comment. Text can be passed via `--text`, `--text_file`, or stdin.
```bash
trello-card-comment-update --comment_id CMT001 --card_id CRD123 --text "Updated text"
```
| Option | Required | Description |
| -------------- | -------- | ---------------------------- |
| `--comment_id` | yes | Trello comment ID |
| `--card_id` | yes | Trello card ID |
| `--text` | \* | Inline comment text |
| `--text_file` | \* | Path to file containing text |
\* One of `--text`, `--text_file`, or stdin is required.
### `trello-card-label-add`
Add a label to a card.
```bash
trello-card-label-add --card_id CRD123 --label_id LBL001
```
| Option | Required | Description |
| ------------ | -------- | --------------- |
| `--card_id` | yes | Trello card ID |
| `--label_id` | yes | Trello label ID |
### `trello-card-label-remove`
Remove a label from a card.
```bash
trello-card-label-remove --card_id CRD123 --label_id LBL001
```
| Option | Required | Description |
| ------------ | -------- | --------------- |
| `--card_id` | yes | Trello card ID |
| `--label_id` | yes | Trello label ID |
+80
View File
@@ -0,0 +1,80 @@
---
title: Wasabi
icon: /images/wasabi-logo.svg
description: Mount a Wasabi cloud storage bucket as a virtual filesystem.
---
The Wasabi resource is a thin wrapper over the [S3 resource](/python/resource/s3).
It maps a `WasabiConfig` to an `S3Config` and reuses the exact same backend,
commands, and behavior as S3, it just derives the right Wasabi endpoint from
`region`. Uses aioboto3 against Wasabi's S3-compatible API.
The endpoint is computed from `region` as `s3.<region>.wasabisys.com`
(`s3.wasabisys.com` for `us-east-1`). Pass `endpoint_url` to override.
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.wasabi import WasabiConfig, WasabiResource
config = WasabiConfig(
bucket=os.environ["WASABI_BUCKET"],
region=os.environ.get("WASABI_REGION", "us-east-2"),
access_key_id=os.environ["WASABI_ACCESS_KEY_ID"],
secret_access_key=os.environ["WASABI_SECRET_ACCESS_KEY"],
# Optional:
# endpoint_url="https://s3.us-east-2.wasabisys.com",
# timeout=30,
# proxy="http://proxy:8080",
)
resource = WasabiResource(config)
ws = Workspace({"/data": resource}, mode=MountMode.READ)
```
`region` defaults to `us-east-1`. Both `READ` and `WRITE` modes are supported.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.wasabi import WasabiConfig, WasabiResource
load_dotenv(".env.development")
config = WasabiConfig(
bucket=os.environ["WASABI_BUCKET"],
region=os.environ.get("WASABI_REGION", "us-east-2"),
access_key_id=os.environ["WASABI_ACCESS_KEY_ID"],
secret_access_key=os.environ["WASABI_SECRET_ACCESS_KEY"],
)
resource = WasabiResource(config)
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
r = await ws.execute("ls /data/")
print(await r.stdout_str())
r = await ws.execute("tree /data/")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
## Notes
- Wasabi reports `ResourceName.S3` and routes through the same `core/s3`
implementation, so the full S3 shell-command set applies. See the
[S3 resource](/python/resource/s3) for the complete command reference,
range reads, streaming, and the index cache fast path.
- For credential setup, see [Wasabi Setup](/home/setup/wasabi).