chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
@@ -0,0 +1,48 @@
|
||||
R2_ACCOUNT_ID=
|
||||
R2_BUCKET=
|
||||
R2_ACCESS_KEY_ID=
|
||||
R2_SECRET_ACCESS_KEY=
|
||||
R2_REGION=auto
|
||||
|
||||
AWS_ACCESS_KEY_ID=
|
||||
AWS_SECRET_ACCESS_KEY=
|
||||
AWS_DEFAULT_REGION=us-east-1
|
||||
AWS_S3_BUCKET=mirage-ai-test
|
||||
|
||||
BOX_CLIENT_ID=
|
||||
BOX_CLIENT_SECRET=
|
||||
BOX_DEVELOPER_TOKEN=
|
||||
|
||||
OPENAI_API_KEY=
|
||||
OPENAI_MODEL=gpt-5-mini
|
||||
|
||||
TRELLO_API_KEY=
|
||||
TRELLO_API_TOKEN=
|
||||
|
||||
NOTION_API_KEY=
|
||||
|
||||
SLACK_BOT_TOKEN=
|
||||
SLACK_USER_TOKEN=
|
||||
|
||||
IMAP_HOST=
|
||||
SMTP_HOST=
|
||||
EMAIL_USERNAME=
|
||||
EMAIL_PASSWORD=
|
||||
|
||||
DROPBOX_APP_KEY=
|
||||
DROPBOX_APP_SECRET=
|
||||
DROPBOX_REFRESH_TOKEN=
|
||||
|
||||
BOX_CLIENT_ID=
|
||||
BOX_CLIENT_SECRET=
|
||||
BOX_REFRESH_TOKEN=
|
||||
BOX_DEVELOPER_TOKEN=
|
||||
|
||||
DAYTONA_API_KEY=
|
||||
|
||||
HF_TOKEN=
|
||||
HF_BUCKET_NAME=
|
||||
|
||||
NEXTCLOUD_URL=
|
||||
NEXTCLOUD_USERNAME=
|
||||
NEXTCLOUD_PASSWORD=
|
||||
@@ -0,0 +1 @@
|
||||
* @zechengz
|
||||
@@ -0,0 +1,110 @@
|
||||
name: Report an issue
|
||||
description: Report a bug, docs problem, feature request, question, or other issue.
|
||||
title: "[Issue]: "
|
||||
labels: ["triage"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for taking the time to report this. Clear details help us reproduce, prioritize, and fix issues faster.
|
||||
|
||||
- type: dropdown
|
||||
id: report_type
|
||||
attributes:
|
||||
label: What type of report is this?
|
||||
options:
|
||||
- Bug
|
||||
- Feature request
|
||||
- Documentation
|
||||
- Example
|
||||
- Question
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: area
|
||||
attributes:
|
||||
label: Area
|
||||
options:
|
||||
- Python package
|
||||
- TypeScript packages
|
||||
- CLI
|
||||
- Docs
|
||||
- Examples
|
||||
- GitHub workflows
|
||||
- Not sure
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: summary
|
||||
attributes:
|
||||
label: Summary
|
||||
description: Tell us what happened, what you expected, or what you want to improve.
|
||||
placeholder: Describe the issue or request in a few sentences.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: steps
|
||||
attributes:
|
||||
label: Steps to reproduce
|
||||
description: For bugs, include the smallest command, code snippet, or example that reproduces the issue.
|
||||
placeholder: |
|
||||
1. ...
|
||||
2. ...
|
||||
3. ...
|
||||
|
||||
- type: textarea
|
||||
id: expected
|
||||
attributes:
|
||||
label: Expected behavior
|
||||
description: What should have happened instead?
|
||||
|
||||
- type: textarea
|
||||
id: actual
|
||||
attributes:
|
||||
label: Actual behavior
|
||||
description: What happened? Include errors, stack traces, logs, or screenshots when useful.
|
||||
render: shell
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: Mirage version or commit
|
||||
placeholder: "0.0.2a0, npm package version, or git SHA"
|
||||
|
||||
- type: input
|
||||
id: python_version
|
||||
attributes:
|
||||
label: Python version
|
||||
placeholder: "3.12.0"
|
||||
|
||||
- type: input
|
||||
id: node_version
|
||||
attributes:
|
||||
label: Node.js version
|
||||
placeholder: "22.0.0"
|
||||
|
||||
- type: dropdown
|
||||
id: os
|
||||
attributes:
|
||||
label: Operating system
|
||||
options:
|
||||
- macOS
|
||||
- Linux
|
||||
- Windows
|
||||
- Other
|
||||
|
||||
- type: input
|
||||
id: browser
|
||||
attributes:
|
||||
label: Browser
|
||||
placeholder: "Chrome 126, Safari 17, Firefox 127, or N/A"
|
||||
|
||||
- type: textarea
|
||||
id: context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add anything else that may help, including links to related issues, PRs, docs, or examples.
|
||||
@@ -0,0 +1,6 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
@@ -0,0 +1,60 @@
|
||||
name: Pre-commit
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
"on":
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
pre-commit:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: pnpm
|
||||
cache-dependency-path: typescript/pnpm-lock.yaml
|
||||
|
||||
- name: Install node-gyp
|
||||
run: npm install -g node-gyp
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Install TypeScript dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build TypeScript packages
|
||||
working-directory: typescript
|
||||
run: pnpm -r build
|
||||
|
||||
- name: Run pre-commit
|
||||
run: SKIP=no-commit-to-branch ./python/.venv/bin/pre-commit run --all-files
|
||||
@@ -0,0 +1,733 @@
|
||||
name: CLI exit codes
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
"on":
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- python/**
|
||||
- typescript/**
|
||||
- .github/workflows/test_cli.yml
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
permissions:
|
||||
pull-requests: read
|
||||
outputs:
|
||||
hit: ${{ steps.filter.outputs.hit }}
|
||||
steps:
|
||||
- uses: dorny/paths-filter@v4
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
hit:
|
||||
- 'python/**'
|
||||
- 'typescript/**'
|
||||
- 'integ/cross.sh'
|
||||
- 'integ/cross.yaml'
|
||||
- 'integ/parity.sh'
|
||||
- 'integ/cli_fuse.sh'
|
||||
- 'integ/cli_config.sh'
|
||||
- '.github/workflows/test_cli.yml'
|
||||
|
||||
python:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
name: Python CLI
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Install mirage package
|
||||
working-directory: python
|
||||
run: uv pip install .
|
||||
|
||||
- name: Smoke test exit code propagation
|
||||
working-directory: python
|
||||
shell: bash
|
||||
run: |
|
||||
set -u
|
||||
MIRAGE=".venv/bin/mirage"
|
||||
cat > /tmp/ws.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
YAML
|
||||
|
||||
$MIRAGE workspace delete smoke > /dev/null 2>&1 || true
|
||||
$MIRAGE workspace create /tmp/ws.yaml --id smoke
|
||||
|
||||
fail=0
|
||||
check() {
|
||||
local desc="$1"
|
||||
local expected_exit="$2"
|
||||
local expected_field="$3"
|
||||
shift 3
|
||||
echo
|
||||
echo "--- $desc"
|
||||
echo "+ $*"
|
||||
set +e
|
||||
out=$("$@")
|
||||
got_exit=$?
|
||||
set -e
|
||||
echo "stdout:"
|
||||
echo "$out" | sed 's/^/ /'
|
||||
echo "process exit: $got_exit (expected $expected_exit)"
|
||||
if [ "$got_exit" != "$expected_exit" ]; then
|
||||
echo "FAIL: process exit code mismatch"
|
||||
fail=1
|
||||
return
|
||||
fi
|
||||
if [ -n "$expected_field" ]; then
|
||||
field_val=$(echo "$out" | jq -r '.exit_code // .result.exit_code // empty')
|
||||
echo "json exit_code: $field_val (expected $expected_field)"
|
||||
if [ "$field_val" != "$expected_field" ]; then
|
||||
echo "FAIL: json exit_code mismatch"
|
||||
fail=1
|
||||
return
|
||||
fi
|
||||
fi
|
||||
echo "PASS"
|
||||
}
|
||||
|
||||
check "echo ok succeeds" 0 0 $MIRAGE execute -w smoke -c 'echo ok'
|
||||
check "false exits 1" 1 1 $MIRAGE execute -w smoke -c 'false'
|
||||
check "false | echo: pipefail off => last cmd wins" 0 0 \
|
||||
$MIRAGE execute -w smoke -c 'false | echo done'
|
||||
check "pipefail on => rightmost failure wins" 1 1 \
|
||||
$MIRAGE execute -w smoke -c 'set -o pipefail; false | echo done'
|
||||
|
||||
echo
|
||||
echo "--- background + job wait"
|
||||
submit=$($MIRAGE execute -w smoke --bg -c 'false')
|
||||
echo "submit response:"
|
||||
echo "$submit" | sed 's/^/ /'
|
||||
JID=$(echo "$submit" | jq -r .job_id)
|
||||
echo "job id: $JID"
|
||||
check "job wait on failing job" 1 1 $MIRAGE job wait "$JID"
|
||||
|
||||
echo
|
||||
echo "--- shell composition (&& and ||)"
|
||||
if $MIRAGE execute -w smoke -c 'false' > /dev/null && echo "should not print"; then
|
||||
echo "FAIL: && short-circuit did not work"
|
||||
fail=1
|
||||
else
|
||||
echo "PASS: && correctly short-circuited on inner failure"
|
||||
fi
|
||||
if $MIRAGE execute -w smoke -c 'echo ok' > /dev/null && echo "should print"; then
|
||||
echo "PASS: && correctly proceeded on inner success"
|
||||
else
|
||||
echo "FAIL: && incorrectly short-circuited on success"
|
||||
fail=1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "--- per-mount command safeguards"
|
||||
cat > /tmp/sg.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
command_safeguards:
|
||||
cat:
|
||||
max_lines: 2
|
||||
on_exceed: truncate
|
||||
YAML
|
||||
cat > /tmp/sgerr.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
command_safeguards:
|
||||
cat:
|
||||
max_lines: 2
|
||||
on_exceed: error
|
||||
YAML
|
||||
$MIRAGE workspace delete sg > /dev/null 2>&1 || true
|
||||
$MIRAGE workspace delete sgerr > /dev/null 2>&1 || true
|
||||
$MIRAGE workspace create /tmp/sg.yaml --id sg
|
||||
$MIRAGE workspace create /tmp/sgerr.yaml --id sgerr
|
||||
$MIRAGE execute -w sg -c "printf '1\n2\n3\n4\n5\n' > /f.txt" > /dev/null
|
||||
$MIRAGE execute -w sgerr -c "printf '1\n2\n3\n4\n5\n' > /f.txt" > /dev/null
|
||||
|
||||
check "cat over max_lines, on_exceed=truncate => exit 0" 0 0 \
|
||||
$MIRAGE execute -w sg -c 'cat /f.txt'
|
||||
echo "+ verify truncated to 2 lines"
|
||||
lines=$($MIRAGE execute -w sg -c 'cat /f.txt' | jq -r '.stdout' | grep -c .)
|
||||
echo " truncated stdout lines: $lines (expected 2)"
|
||||
if [ "$lines" != "2" ]; then
|
||||
echo "FAIL: truncate did not cap to 2 lines"
|
||||
fail=1
|
||||
else
|
||||
echo "PASS"
|
||||
fi
|
||||
check "cat over max_lines, on_exceed=error => exit 1" 1 1 \
|
||||
$MIRAGE execute -w sgerr -c 'cat /f.txt'
|
||||
$MIRAGE workspace delete sg
|
||||
$MIRAGE workspace delete sgerr
|
||||
|
||||
echo
|
||||
echo "--- top-level snake_case config keys"
|
||||
cat > /tmp/cfgkeys.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
default_session_id: mysess
|
||||
default_agent_id: myagent
|
||||
cache:
|
||||
type: ram
|
||||
limit: 256MB
|
||||
max_drain_bytes: 1048576
|
||||
index:
|
||||
type: ram
|
||||
ttl: 600
|
||||
YAML
|
||||
$MIRAGE workspace delete cfgkeys > /dev/null 2>&1 || true
|
||||
sid=$($MIRAGE workspace create /tmp/cfgkeys.yaml --id cfgkeys | jq -r '.sessions[0].sessionId // .sessions[0].session_id')
|
||||
echo " default_session_id (snake) => $sid (expected mysess)"
|
||||
if [ "$sid" != "mysess" ]; then
|
||||
echo "FAIL: snake_case top-level keys not applied"
|
||||
fail=1
|
||||
else
|
||||
echo "PASS"
|
||||
fi
|
||||
$MIRAGE execute -w cfgkeys -c "printf 'x\ny\n' > /f.txt" > /dev/null
|
||||
check "ops work with snake_case cache/index configured => exit 0" 0 0 \
|
||||
$MIRAGE execute -w cfgkeys -c 'cat /f.txt'
|
||||
$MIRAGE workspace delete cfgkeys
|
||||
|
||||
curl_check() {
|
||||
local desc="$1"
|
||||
local expected="$2"
|
||||
shift 2
|
||||
local got
|
||||
got=$(curl -s -o /dev/null -w "%{http_code}" "$@")
|
||||
echo " $desc: got $got (expected $expected)"
|
||||
if [ "$got" != "$expected" ]; then
|
||||
echo " FAIL"
|
||||
fail=1
|
||||
fi
|
||||
}
|
||||
|
||||
echo
|
||||
echo "===== auth: mode=local (daemon spawned by CLI, token auto-minted) ====="
|
||||
TOKEN=$(cat "$HOME/.mirage/auth_token")
|
||||
MODE=$(stat -c "%a" "$HOME/.mirage/auth_token" 2>/dev/null || stat -f "%Lp" "$HOME/.mirage/auth_token")
|
||||
echo "token file mode: $MODE (expected 600)"
|
||||
if [ "$MODE" != "600" ]; then
|
||||
echo "FAIL: token file must be 0o600"
|
||||
fail=1
|
||||
fi
|
||||
curl_check "correct bearer => 200" 200 \
|
||||
-H "Authorization: Bearer $TOKEN" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "wrong bearer => 401" 401 \
|
||||
-H "Authorization: Bearer wrong-token" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "no header => 401" 401 http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "/v1/health bypass => 200" 200 http://127.0.0.1:8765/v1/health
|
||||
curl_check "JWT-shaped value treated as opaque => 401" 401 \
|
||||
-H "Authorization: Bearer aaaa.bbbb.cccc" http://127.0.0.1:8765/v1/workspaces
|
||||
|
||||
$MIRAGE workspace delete smoke
|
||||
$MIRAGE daemon stop || true
|
||||
sleep 2
|
||||
|
||||
echo
|
||||
echo "===== auth: mode=token (operator PAT) ====="
|
||||
MIRAGE_AUTH_MODE=token MIRAGE_AUTH_TOKEN=operator-pat-xyz \
|
||||
.venv/bin/python -m uvicorn mirage.server.daemon:app \
|
||||
--host 127.0.0.1 --port 8765 --log-level warning >/tmp/auth-token.log 2>&1 &
|
||||
TOKEN_PID=$!
|
||||
sleep 2
|
||||
curl_check "correct PAT => 200" 200 \
|
||||
-H "Authorization: Bearer operator-pat-xyz" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "wrong PAT => 401" 401 \
|
||||
-H "Authorization: Bearer something-else" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "no header => 401" 401 http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "/v1/health bypass => 200" 200 http://127.0.0.1:8765/v1/health
|
||||
kill $TOKEN_PID 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
echo
|
||||
echo "===== auth: mode=jwt (RS256, external issuer) ====="
|
||||
mkdir -p /tmp/jwt-test
|
||||
.venv/bin/python - <<'PYEOF'
|
||||
import time, jwt
|
||||
from pathlib import Path
|
||||
from cryptography.hazmat.primitives import serialization
|
||||
from cryptography.hazmat.primitives.asymmetric import rsa
|
||||
d = Path("/tmp/jwt-test")
|
||||
priv = rsa.generate_private_key(public_exponent=65537, key_size=2048)
|
||||
priv_pem = priv.private_bytes(
|
||||
encoding=serialization.Encoding.PEM,
|
||||
format=serialization.PrivateFormat.PKCS8,
|
||||
encryption_algorithm=serialization.NoEncryption())
|
||||
(d / "pub.pem").write_bytes(priv.public_key().public_bytes(
|
||||
encoding=serialization.Encoding.PEM,
|
||||
format=serialization.PublicFormat.SubjectPublicKeyInfo))
|
||||
now = int(time.time())
|
||||
(d / "good.jwt").write_text(jwt.encode(
|
||||
{"sub": "agent", "exp": now + 60}, priv_pem, algorithm="RS256"))
|
||||
(d / "expired.jwt").write_text(jwt.encode(
|
||||
{"sub": "agent", "exp": now - 60}, priv_pem, algorithm="RS256"))
|
||||
PYEOF
|
||||
|
||||
MIRAGE_AUTH_MODE=jwt \
|
||||
MIRAGE_JWT_PUBKEY_FILE=/tmp/jwt-test/pub.pem \
|
||||
MIRAGE_JWT_ALG=RS256 \
|
||||
.venv/bin/python -m uvicorn mirage.server.daemon:app \
|
||||
--host 127.0.0.1 --port 8765 --log-level warning >/tmp/auth-jwt.log 2>&1 &
|
||||
JWT_PID=$!
|
||||
sleep 2
|
||||
GOOD=$(cat /tmp/jwt-test/good.jwt)
|
||||
EXPIRED=$(cat /tmp/jwt-test/expired.jwt)
|
||||
curl_check "valid signed JWT => 200" 200 \
|
||||
-H "Authorization: Bearer $GOOD" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "expired JWT => 401" 401 \
|
||||
-H "Authorization: Bearer $EXPIRED" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "opaque (not 3-part) => 401" 401 \
|
||||
-H "Authorization: Bearer opaque-not-a-jwt" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "/v1/health bypass => 200" 200 http://127.0.0.1:8765/v1/health
|
||||
kill $JWT_PID 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
if [ "$fail" != "0" ]; then
|
||||
echo
|
||||
echo "One or more checks failed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
typescript:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
name: TypeScript CLI
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: pnpm
|
||||
cache-dependency-path: typescript/pnpm-lock.yaml
|
||||
|
||||
- name: Install node-gyp
|
||||
run: npm install -g node-gyp
|
||||
|
||||
- name: Install TypeScript dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build TypeScript packages
|
||||
working-directory: typescript
|
||||
run: pnpm -r build
|
||||
|
||||
- name: Smoke test exit code propagation
|
||||
working-directory: typescript
|
||||
shell: bash
|
||||
run: |
|
||||
set -u
|
||||
MIRAGE="node packages/cli/dist/bin/mirage.js"
|
||||
cat > /tmp/ws.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
YAML
|
||||
|
||||
$MIRAGE workspace delete smoke > /dev/null 2>&1 || true
|
||||
$MIRAGE workspace create /tmp/ws.yaml --id smoke
|
||||
|
||||
fail=0
|
||||
check() {
|
||||
local desc="$1"
|
||||
local expected_exit="$2"
|
||||
local expected_field="$3"
|
||||
shift 3
|
||||
echo
|
||||
echo "--- $desc"
|
||||
echo "+ $*"
|
||||
set +e
|
||||
out=$("$@")
|
||||
got_exit=$?
|
||||
set -e
|
||||
echo "stdout:"
|
||||
echo "$out" | sed 's/^/ /'
|
||||
echo "process exit: $got_exit (expected $expected_exit)"
|
||||
if [ "$got_exit" != "$expected_exit" ]; then
|
||||
echo "FAIL: process exit code mismatch"
|
||||
fail=1
|
||||
return
|
||||
fi
|
||||
if [ -n "$expected_field" ]; then
|
||||
field_val=$(echo "$out" | jq -r '.exitCode // .result.exitCode // empty')
|
||||
echo "json exitCode: $field_val (expected $expected_field)"
|
||||
if [ "$field_val" != "$expected_field" ]; then
|
||||
echo "FAIL: json exitCode mismatch"
|
||||
fail=1
|
||||
return
|
||||
fi
|
||||
fi
|
||||
echo "PASS"
|
||||
}
|
||||
|
||||
check "echo ok succeeds" 0 0 $MIRAGE execute -w smoke -c 'echo ok'
|
||||
check "false exits 1" 1 1 $MIRAGE execute -w smoke -c 'false'
|
||||
check "false | echo: pipefail off => last cmd wins" 0 0 \
|
||||
$MIRAGE execute -w smoke -c 'false | echo done'
|
||||
check "pipefail on => rightmost failure wins" 1 1 \
|
||||
$MIRAGE execute -w smoke -c 'set -o pipefail; false | echo done'
|
||||
|
||||
echo
|
||||
echo "--- background + job wait"
|
||||
submit=$($MIRAGE execute -w smoke --bg -c 'false')
|
||||
echo "submit response:"
|
||||
echo "$submit" | sed 's/^/ /'
|
||||
JID=$(echo "$submit" | jq -r .jobId)
|
||||
echo "job id: $JID"
|
||||
check "job wait on failing job" 1 1 $MIRAGE job wait "$JID"
|
||||
|
||||
echo
|
||||
echo "--- shell composition (&& and ||)"
|
||||
if $MIRAGE execute -w smoke -c 'false' > /dev/null && echo "should not print"; then
|
||||
echo "FAIL: && short-circuit did not work"
|
||||
fail=1
|
||||
else
|
||||
echo "PASS: && correctly short-circuited on inner failure"
|
||||
fi
|
||||
if $MIRAGE execute -w smoke -c 'echo ok' > /dev/null && echo "should print"; then
|
||||
echo "PASS: && correctly proceeded on inner success"
|
||||
else
|
||||
echo "FAIL: && incorrectly short-circuited on success"
|
||||
fail=1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "--- per-mount command safeguards"
|
||||
cat > /tmp/sg.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
command_safeguards:
|
||||
cat:
|
||||
max_lines: 2
|
||||
on_exceed: truncate
|
||||
YAML
|
||||
cat > /tmp/sgerr.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
command_safeguards:
|
||||
cat:
|
||||
max_lines: 2
|
||||
on_exceed: error
|
||||
YAML
|
||||
$MIRAGE workspace delete sg > /dev/null 2>&1 || true
|
||||
$MIRAGE workspace delete sgerr > /dev/null 2>&1 || true
|
||||
$MIRAGE workspace create /tmp/sg.yaml --id sg
|
||||
$MIRAGE workspace create /tmp/sgerr.yaml --id sgerr
|
||||
$MIRAGE execute -w sg -c "printf '1\n2\n3\n4\n5\n' > /f.txt" > /dev/null
|
||||
$MIRAGE execute -w sgerr -c "printf '1\n2\n3\n4\n5\n' > /f.txt" > /dev/null
|
||||
|
||||
check "cat over max_lines, on_exceed=truncate => exit 0" 0 0 \
|
||||
$MIRAGE execute -w sg -c 'cat /f.txt'
|
||||
echo "+ verify truncated to 2 lines"
|
||||
lines=$($MIRAGE execute -w sg -c 'cat /f.txt' | jq -r '.stdout' | grep -c .)
|
||||
echo " truncated stdout lines: $lines (expected 2)"
|
||||
if [ "$lines" != "2" ]; then
|
||||
echo "FAIL: truncate did not cap to 2 lines"
|
||||
fail=1
|
||||
else
|
||||
echo "PASS"
|
||||
fi
|
||||
check "cat over max_lines, on_exceed=error => exit 1" 1 1 \
|
||||
$MIRAGE execute -w sgerr -c 'cat /f.txt'
|
||||
$MIRAGE workspace delete sg
|
||||
$MIRAGE workspace delete sgerr
|
||||
|
||||
echo
|
||||
echo "--- top-level snake_case config keys"
|
||||
cat > /tmp/cfgkeys.yaml <<'YAML'
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
default_session_id: mysess
|
||||
default_agent_id: myagent
|
||||
cache:
|
||||
type: ram
|
||||
limit: 256MB
|
||||
max_drain_bytes: 1048576
|
||||
index:
|
||||
type: ram
|
||||
ttl: 600
|
||||
YAML
|
||||
$MIRAGE workspace delete cfgkeys > /dev/null 2>&1 || true
|
||||
sid=$($MIRAGE workspace create /tmp/cfgkeys.yaml --id cfgkeys | jq -r '.sessions[0].sessionId // .sessions[0].session_id')
|
||||
echo " default_session_id (snake) => $sid (expected mysess)"
|
||||
if [ "$sid" != "mysess" ]; then
|
||||
echo "FAIL: snake_case top-level keys not applied"
|
||||
fail=1
|
||||
else
|
||||
echo "PASS"
|
||||
fi
|
||||
$MIRAGE execute -w cfgkeys -c "printf 'x\ny\n' > /f.txt" > /dev/null
|
||||
check "ops work with snake_case cache/index configured => exit 0" 0 0 \
|
||||
$MIRAGE execute -w cfgkeys -c 'cat /f.txt'
|
||||
$MIRAGE workspace delete cfgkeys
|
||||
|
||||
curl_check() {
|
||||
local desc="$1"
|
||||
local expected="$2"
|
||||
shift 2
|
||||
local got
|
||||
got=$(curl -s -o /dev/null -w "%{http_code}" "$@")
|
||||
echo " $desc: got $got (expected $expected)"
|
||||
if [ "$got" != "$expected" ]; then
|
||||
echo " FAIL"
|
||||
fail=1
|
||||
fi
|
||||
}
|
||||
|
||||
echo
|
||||
echo "===== auth: mode=local (TS daemon spawned by CLI, token auto-minted) ====="
|
||||
TOKEN=$(cat "$HOME/.mirage/auth_token")
|
||||
MODE=$(stat -c "%a" "$HOME/.mirage/auth_token" 2>/dev/null || stat -f "%Lp" "$HOME/.mirage/auth_token")
|
||||
echo "token file mode: $MODE (expected 600)"
|
||||
if [ "$MODE" != "600" ]; then
|
||||
echo "FAIL: token file must be 0o600"
|
||||
fail=1
|
||||
fi
|
||||
curl_check "correct bearer => 200" 200 \
|
||||
-H "Authorization: Bearer $TOKEN" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "wrong bearer => 401" 401 \
|
||||
-H "Authorization: Bearer wrong-token" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "no header => 401" 401 http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "/v1/health bypass => 200" 200 http://127.0.0.1:8765/v1/health
|
||||
curl_check "JWT-shaped value treated as opaque => 401" 401 \
|
||||
-H "Authorization: Bearer aaaa.bbbb.cccc" http://127.0.0.1:8765/v1/workspaces
|
||||
|
||||
$MIRAGE workspace delete smoke
|
||||
$MIRAGE daemon stop || true
|
||||
sleep 2
|
||||
|
||||
echo
|
||||
echo "===== auth: mode=token (TS daemon, operator PAT) ====="
|
||||
MIRAGE_AUTH_MODE=token MIRAGE_AUTH_TOKEN=operator-pat-xyz \
|
||||
node packages/server/dist/bin/daemon.js >/tmp/auth-token-ts.log 2>&1 &
|
||||
TOKEN_PID=$!
|
||||
sleep 2
|
||||
curl_check "correct PAT => 200" 200 \
|
||||
-H "Authorization: Bearer operator-pat-xyz" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "wrong PAT => 401" 401 \
|
||||
-H "Authorization: Bearer something-else" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "no header => 401" 401 http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "/v1/health bypass => 200" 200 http://127.0.0.1:8765/v1/health
|
||||
kill $TOKEN_PID 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
echo
|
||||
echo "===== auth: mode=jwt (TS daemon, RS256, external issuer) ====="
|
||||
mkdir -p /tmp/jwt-test-ts
|
||||
cat > packages/server/mint-jwt.mjs <<'NODEEOF'
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import { generateKeyPair, SignJWT, exportSPKI } from 'jose'
|
||||
const { publicKey, privateKey } = await generateKeyPair('RS256', { modulusLength: 2048 })
|
||||
writeFileSync('/tmp/jwt-test-ts/pub.pem', await exportSPKI(publicKey))
|
||||
const now = Math.floor(Date.now() / 1000)
|
||||
const sign = (exp) => new SignJWT({ sub: 'agent' })
|
||||
.setProtectedHeader({ alg: 'RS256' })
|
||||
.setExpirationTime(exp)
|
||||
.sign(privateKey)
|
||||
writeFileSync('/tmp/jwt-test-ts/good.jwt', await sign(now + 60))
|
||||
writeFileSync('/tmp/jwt-test-ts/expired.jwt', await sign(now - 60))
|
||||
NODEEOF
|
||||
node packages/server/mint-jwt.mjs
|
||||
rm packages/server/mint-jwt.mjs
|
||||
|
||||
MIRAGE_AUTH_MODE=jwt \
|
||||
MIRAGE_JWT_PUBKEY_FILE=/tmp/jwt-test-ts/pub.pem \
|
||||
MIRAGE_JWT_ALG=RS256 \
|
||||
node packages/server/dist/bin/daemon.js >/tmp/auth-jwt-ts.log 2>&1 &
|
||||
JWT_PID=$!
|
||||
sleep 2
|
||||
GOOD=$(cat /tmp/jwt-test-ts/good.jwt)
|
||||
EXPIRED=$(cat /tmp/jwt-test-ts/expired.jwt)
|
||||
curl_check "valid signed JWT => 200" 200 \
|
||||
-H "Authorization: Bearer $GOOD" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "expired JWT => 401" 401 \
|
||||
-H "Authorization: Bearer $EXPIRED" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "opaque (not 3-part) => 401" 401 \
|
||||
-H "Authorization: Bearer opaque-not-a-jwt" http://127.0.0.1:8765/v1/workspaces
|
||||
curl_check "/v1/health bypass => 200" 200 http://127.0.0.1:8765/v1/health
|
||||
kill $JWT_PID 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
if [ "$fail" != "0" ]; then
|
||||
echo
|
||||
echo "One or more checks failed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cross:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
name: Cross-language snapshot interop
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
services:
|
||||
redis:
|
||||
image: redis:7
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
S3_BUCKET: mirage-cross
|
||||
S3_ENDPOINT: http://localhost:9000
|
||||
S3_REGION: us-east-1
|
||||
AWS_ACCESS_KEY_ID: minio
|
||||
AWS_SECRET_ACCESS_KEY: minio123
|
||||
CROSS_DISK_ROOT: /tmp/mirage-cross-disk
|
||||
CROSS_REDIS_PREFIX: mirage-cross/
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install libfuse (mfusepy + @zkochan/fuse-native, for the CLI FUSE e2e)
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y fuse3 libfuse3-dev libfuse2t64 || \
|
||||
sudo apt-get install -y fuse3 libfuse3-dev libfuse2
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Install mirage package
|
||||
working-directory: python
|
||||
run: uv pip install .
|
||||
|
||||
- name: Install pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "24"
|
||||
|
||||
- name: Install TypeScript dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile=false
|
||||
|
||||
- name: Build TypeScript packages
|
||||
working-directory: typescript
|
||||
run: pnpm -r build
|
||||
|
||||
- name: Start MinIO
|
||||
run: |
|
||||
for i in $(seq 1 5); do
|
||||
docker pull minio/minio:latest && break
|
||||
echo "docker pull failed (attempt $i), retrying"; sleep 10
|
||||
done
|
||||
docker run -d --name minio -p 9000:9000 \
|
||||
-e MINIO_ROOT_USER=minio -e MINIO_ROOT_PASSWORD=minio123 \
|
||||
minio/minio:latest server /data
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf http://localhost:9000/minio/health/live && break
|
||||
sleep 1
|
||||
done
|
||||
|
||||
- name: Create MinIO bucket
|
||||
run: |
|
||||
curl -sSL https://dl.min.io/client/mc/release/linux-amd64/mc -o /tmp/mc
|
||||
chmod +x /tmp/mc
|
||||
/tmp/mc alias set local http://localhost:9000 minio minio123
|
||||
/tmp/mc mb local/mirage-cross || true
|
||||
|
||||
- name: Run cross-language snapshot interop (both directions)
|
||||
run: |
|
||||
bash integ/cross.sh \
|
||||
"python/.venv/bin/mirage" \
|
||||
"node typescript/packages/cli/dist/bin/mirage.js"
|
||||
|
||||
- name: Run CLI feature parity (subshell, sessions, versioning, fuse)
|
||||
run: |
|
||||
bash integ/parity.sh \
|
||||
"python/.venv/bin/mirage" \
|
||||
"node typescript/packages/cli/dist/bin/mirage.js"
|
||||
|
||||
- name: Run CLI config battery (relocation, validation, resolved view)
|
||||
run: |
|
||||
bash integ/cli_config.sh \
|
||||
"python/.venv/bin/mirage" \
|
||||
"node typescript/packages/cli/dist/bin/mirage.js"
|
||||
|
||||
- name: Run CLI end-to-end FUSE (two per-mount subtrees, real libfuse)
|
||||
run: |
|
||||
bash integ/cli_fuse.sh \
|
||||
"python/.venv/bin/mirage" \
|
||||
"node typescript/packages/cli/dist/bin/mirage.js"
|
||||
|
||||
gate:
|
||||
name: cli-gate
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes, python, typescript, cross]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check required jobs
|
||||
run: |
|
||||
results="${{ join(needs.*.result, ' ') }}"
|
||||
echo "$results"
|
||||
case "$results" in
|
||||
*failure*|*cancelled*) exit 1 ;;
|
||||
esac
|
||||
@@ -0,0 +1,237 @@
|
||||
name: Test (Install)
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
"on":
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- python/**
|
||||
- typescript/**
|
||||
- .github/workflows/test_install.yml
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
permissions:
|
||||
pull-requests: read
|
||||
outputs:
|
||||
hit: ${{ steps.filter.outputs.hit }}
|
||||
steps:
|
||||
- uses: dorny/paths-filter@v4
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
hit:
|
||||
- 'python/**'
|
||||
- 'typescript/**'
|
||||
- '.github/workflows/test_install.yml'
|
||||
|
||||
python-minimal:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.11", "3.12"]
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install base package only (no extras)
|
||||
working-directory: python
|
||||
run: |
|
||||
uv venv .venv-min
|
||||
uv pip install --python .venv-min/bin/python .
|
||||
|
||||
- name: Verify optional dependencies stay out of the base install
|
||||
working-directory: python
|
||||
run: |
|
||||
.venv-min/bin/python - <<'PY'
|
||||
import importlib.util
|
||||
banned = ["numpy", "pypdfium2", "PIL", "mfusepy", "opendal", "aioboto3",
|
||||
"pandas", "redis", "asyncssh", "motor", "asyncpg", "av"]
|
||||
present = [m for m in banned if importlib.util.find_spec(m) is not None]
|
||||
assert not present, f"optional deps leaked into base install: {present}"
|
||||
print("OK: no optional deps in base install")
|
||||
PY
|
||||
|
||||
- name: Core workspace smoke test on base install
|
||||
working-directory: python
|
||||
run: |
|
||||
.venv-min/bin/python - <<'PY'
|
||||
import asyncio
|
||||
|
||||
import mirage
|
||||
|
||||
async def main():
|
||||
ws = mirage.Workspace({"/ram": mirage.RAMResource()},
|
||||
mode=mirage.MountMode.WRITE)
|
||||
out = await ws.execute("echo hello | wc -c")
|
||||
assert out.exit_code == 0 and out.stdout == b"6\n", out
|
||||
out = await ws.execute("echo hi > /ram/a.txt")
|
||||
assert out.exit_code == 0, out
|
||||
out = await ws.execute("cat /ram/a.txt")
|
||||
assert out.exit_code == 0 and out.stdout == b"hi\n", out
|
||||
print("OK: core smoke passed on base install")
|
||||
|
||||
asyncio.run(main())
|
||||
PY
|
||||
|
||||
- name: CLI entry point works on base install
|
||||
working-directory: python
|
||||
run: .venv-min/bin/mirage --help > /dev/null
|
||||
|
||||
python-extra:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- { extra: s3, module: mirage.resource.s3 }
|
||||
- { extra: ssh, module: mirage.resource.ssh }
|
||||
- { extra: mongodb, module: mirage.resource.mongodb }
|
||||
- { extra: postgres, module: mirage.resource.postgres }
|
||||
- { extra: redis, module: mirage.resource.redis }
|
||||
- { extra: email, module: mirage.resource.email }
|
||||
- { extra: fuse, module: mirage.fuse.mount }
|
||||
- { extra: pdf, module: mirage.core.filetype.pdf }
|
||||
- { extra: parquet, module: mirage.core.filetype.parquet }
|
||||
- { extra: hdf5, module: mirage.core.filetype.hdf5 }
|
||||
- { extra: nextcloud, module: mirage.resource.nextcloud }
|
||||
- { extra: hf, module: mirage.resource.hf_buckets }
|
||||
- { extra: langfuse, module: mirage.resource.langfuse }
|
||||
- { extra: chroma, module: mirage.resource.chroma }
|
||||
- { extra: lancedb, module: mirage.resource.lancedb }
|
||||
- { extra: qdrant, module: mirage.resource.qdrant }
|
||||
- { extra: databricks, module: mirage.resource.databricks_volume }
|
||||
- { extra: pydantic-ai, module: mirage.agents.pydantic_ai }
|
||||
- { extra: openai, module: mirage.agents.openai_agents }
|
||||
- { extra: agno, module: mirage.agents.agno }
|
||||
- { extra: deepagents, module: mirage.agents.langchain }
|
||||
- { extra: openhands, module: mirage.agents.openhands, python: "3.12" }
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ matrix.python || '3.11' }}
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install base package + single extra
|
||||
working-directory: python
|
||||
run: |
|
||||
uv venv .venv-x
|
||||
uv pip install --python .venv-x/bin/python ".[${{ matrix.extra }}]"
|
||||
|
||||
- name: Import the extra's entry module
|
||||
working-directory: python
|
||||
run: |
|
||||
.venv-x/bin/python -c "import importlib; importlib.import_module('${{ matrix.module }}'); print('OK: ${{ matrix.module }}')"
|
||||
|
||||
ts-minimal:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: pnpm
|
||||
cache-dependency-path: typescript/pnpm-lock.yaml
|
||||
|
||||
- name: Install TypeScript dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build core and node packages
|
||||
working-directory: typescript
|
||||
run: pnpm -r --filter ./packages/core --filter ./packages/node build
|
||||
|
||||
- name: Pack tarballs
|
||||
working-directory: typescript
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/minimal"
|
||||
pnpm --filter @struktoai/mirage-core pack --pack-destination "$RUNNER_TEMP/minimal"
|
||||
pnpm --filter @struktoai/mirage-node pack --pack-destination "$RUNNER_TEMP/minimal"
|
||||
|
||||
- name: Install as a bare consumer (no optional peers)
|
||||
run: |
|
||||
cd "$RUNNER_TEMP/minimal"
|
||||
npm init -y > /dev/null
|
||||
npm install ./struktoai-mirage-core-*.tgz ./struktoai-mirage-node-*.tgz
|
||||
|
||||
- name: Verify optional peers stay out of the bare install
|
||||
run: |
|
||||
cd "$RUNNER_TEMP/minimal"
|
||||
for dep in @lancedb opendal mongodb pg redis ssh2 imapflow mailparser nodemailer @zkochan; do
|
||||
if [ -e "node_modules/$dep" ]; then
|
||||
echo "FAIL: optional peer $dep leaked into bare install"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
echo "OK: no optional peers in bare install"
|
||||
|
||||
- name: Core workspace smoke test on bare install
|
||||
run: |
|
||||
cd "$RUNNER_TEMP/minimal"
|
||||
node --input-type=module - <<'JS'
|
||||
import { Workspace, RAMResource, MountMode } from '@struktoai/mirage-node'
|
||||
const dec = new TextDecoder()
|
||||
const ws = new Workspace({ '/ram': new RAMResource() }, { mode: MountMode.WRITE })
|
||||
let out = await ws.execute('echo hello | wc -c')
|
||||
if (out.exitCode !== 0 || dec.decode(out.stdout).trim() !== '6') throw new Error('pipe smoke failed')
|
||||
out = await ws.execute('echo hi > /ram/a.txt')
|
||||
if (out.exitCode !== 0) throw new Error('write smoke failed')
|
||||
out = await ws.execute('cat /ram/a.txt')
|
||||
if (out.exitCode !== 0 || dec.decode(out.stdout) !== 'hi\n') throw new Error('read smoke failed')
|
||||
console.log('OK: core smoke passed on bare install')
|
||||
JS
|
||||
|
||||
gate:
|
||||
name: test-install-gate
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes, python-minimal, python-extra, ts-minimal]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check required jobs
|
||||
run: |
|
||||
results="${{ join(needs.*.result, ' ') }}"
|
||||
echo "$results"
|
||||
case "$results" in
|
||||
*failure*|*cancelled*) exit 1 ;;
|
||||
esac
|
||||
@@ -0,0 +1,775 @@
|
||||
name: Integ
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
"on":
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- integ/**
|
||||
- data/**
|
||||
- python/**
|
||||
- typescript/**
|
||||
- .github/workflows/test_integ.yml
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
permissions:
|
||||
pull-requests: read
|
||||
outputs:
|
||||
core: ${{ steps.filter.outputs.core }}
|
||||
ts: ${{ steps.filter.outputs.ts }}
|
||||
database: ${{ steps.filter.outputs.database }}
|
||||
data: ${{ steps.filter.outputs.data }}
|
||||
ssh: ${{ steps.filter.outputs.ssh }}
|
||||
fuse: ${{ steps.filter.outputs.fuse }}
|
||||
steps:
|
||||
- uses: dorny/paths-filter@v4
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
core:
|
||||
- 'integ/**'
|
||||
- 'data/**'
|
||||
- 'python/**'
|
||||
- '.github/workflows/test_integ.yml'
|
||||
ts:
|
||||
- 'integ/**'
|
||||
- 'typescript/**'
|
||||
- '.github/workflows/test_integ.yml'
|
||||
database:
|
||||
- 'integ/mongodb.py'
|
||||
- 'integ/mongodb.ts'
|
||||
- 'integ/truth_mongodb.txt'
|
||||
- 'integ/postgres.py'
|
||||
- 'integ/postgres.ts'
|
||||
- 'integ/truth_postgres.txt'
|
||||
- 'integ/chroma.py'
|
||||
- 'integ/chroma.ts'
|
||||
- 'integ/truth_chroma.txt'
|
||||
- 'integ/check_lines.sh'
|
||||
- 'python/mirage/accessor/mongodb.py'
|
||||
- 'python/mirage/core/mongodb/**'
|
||||
- 'python/mirage/commands/builtin/mongodb/**'
|
||||
- 'python/mirage/resource/mongodb/**'
|
||||
- 'python/mirage/accessor/postgres.py'
|
||||
- 'python/mirage/core/postgres/**'
|
||||
- 'python/mirage/commands/builtin/postgres/**'
|
||||
- 'python/mirage/resource/postgres/**'
|
||||
- 'python/mirage/accessor/chroma.py'
|
||||
- 'python/mirage/core/chroma/**'
|
||||
- 'python/mirage/commands/builtin/chroma/**'
|
||||
- 'python/mirage/resource/chroma/**'
|
||||
- 'integ/qdrant.py'
|
||||
- 'integ/qdrant.ts'
|
||||
- 'integ/truth_qdrant.txt'
|
||||
- 'python/mirage/accessor/qdrant.py'
|
||||
- 'python/mirage/core/qdrant/**'
|
||||
- 'python/mirage/commands/builtin/qdrant/**'
|
||||
- 'python/mirage/resource/qdrant/**'
|
||||
- 'typescript/**'
|
||||
- '.github/workflows/test_integ.yml'
|
||||
data:
|
||||
- 'integ/**'
|
||||
- 'python/**'
|
||||
- '.github/workflows/test_integ.yml'
|
||||
ssh:
|
||||
- 'integ/ssh.py'
|
||||
- 'integ/ssh.ts'
|
||||
- 'integ/truth.txt'
|
||||
- 'integ/cases.py'
|
||||
- 'integ/cases.ts'
|
||||
- 'python/mirage/accessor/ssh.py'
|
||||
- 'python/mirage/core/ssh/**'
|
||||
- 'python/mirage/commands/builtin/ssh/**'
|
||||
- 'python/mirage/resource/ssh/**'
|
||||
- 'typescript/packages/node/src/accessor/ssh.ts'
|
||||
- 'typescript/packages/node/src/core/ssh/**'
|
||||
- 'typescript/packages/node/src/commands/builtin/ssh/**'
|
||||
- 'typescript/packages/node/src/resource/ssh/**'
|
||||
- 'typescript/packages/node/src/ops/ssh/**'
|
||||
- '.github/workflows/test_integ.yml'
|
||||
fuse:
|
||||
- 'integ/fuse.py'
|
||||
- 'integ/fuse.ts'
|
||||
- 'integ/truth_fuse.txt'
|
||||
- 'integ/check_lines.sh'
|
||||
- 'integ/package.json'
|
||||
- 'python/mirage/fuse/**'
|
||||
- 'python/mirage/workspace/fuse.py'
|
||||
- 'typescript/packages/node/src/fuse/**'
|
||||
- 'typescript/packages/node/src/workspace/fuse.ts'
|
||||
- 'typescript/packages/node/src/workspace.ts'
|
||||
- '.github/workflows/test_integ.yml'
|
||||
|
||||
integ:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.core == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
redis:
|
||||
image: redis:7
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Run RAM backend and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/ram.py > /tmp/ram.out
|
||||
diff integ/truth.txt /tmp/ram.out
|
||||
|
||||
- name: Run Disk backend and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/disk.py > /tmp/disk.out
|
||||
diff integ/truth.txt /tmp/disk.out
|
||||
|
||||
- name: Run Redis backend and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/redis.py > /tmp/redis.out
|
||||
diff integ/truth.txt /tmp/redis.out
|
||||
|
||||
- name: Run Safeguard scenarios and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/safeguard.py > /tmp/safeguard.out
|
||||
diff integ/truth_safeguard.txt /tmp/safeguard.out
|
||||
|
||||
- name: Run History scenarios and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/history.py > /tmp/history.out
|
||||
diff integ/truth_history.txt /tmp/history.out
|
||||
|
||||
- name: Run s3 + gcs + minio integ (moto) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/s3.py > /tmp/s3.out
|
||||
diff integ/truth_s3.txt /tmp/s3.out
|
||||
|
||||
- name: Run shared command suite on s3 (moto) and diff against shared truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/s3_cases.py > /tmp/s3_cases.out
|
||||
diff integ/truth.txt /tmp/s3_cases.out
|
||||
|
||||
- name: Run onedrive integ (fake Graph) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/onedrive.py > /tmp/onedrive.out
|
||||
diff integ/truth_onedrive.txt /tmp/onedrive.out
|
||||
|
||||
- name: Run shared command suite on onedrive (fake Graph) and diff against shared truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/onedrive_cases.py > /tmp/onedrive_cases.out
|
||||
diff integ/truth.txt /tmp/onedrive_cases.out
|
||||
|
||||
- name: Run lancedb integ (embedded) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/lancedb.py > /tmp/lancedb.out
|
||||
diff integ/truth_lancedb.txt /tmp/lancedb.out
|
||||
|
||||
- name: Run find arg-error suite (SaaS backends, dummy creds) and diff
|
||||
run: |
|
||||
./python/.venv/bin/python integ/find_arg_errors.py > /tmp/fae.out
|
||||
diff integ/truth_find_arg_errors.txt /tmp/fae.out
|
||||
|
||||
- name: Run cross-mount commands (ram -> ram/redis/s3 via moto)
|
||||
run: ./python/.venv/bin/python integ/cross_commands.py
|
||||
|
||||
- name: Run virtual-root mount switching (ram/disk/yaml)
|
||||
run: ./python/.venv/bin/python integ/root.py
|
||||
|
||||
integ-ts:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.ts == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
redis:
|
||||
image: redis:7
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
S3_BUCKET: mirage-integ
|
||||
S3_ENDPOINT: http://localhost:9000
|
||||
S3_REGION: us-east-1
|
||||
AWS_ACCESS_KEY_ID: minio
|
||||
AWS_SECRET_ACCESS_KEY: minio123
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '24'
|
||||
|
||||
- name: Set up pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile=false
|
||||
|
||||
- name: Build mirage-core
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-core build
|
||||
|
||||
- name: Build mirage-node
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-node build
|
||||
|
||||
- name: Build mirage-browser
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-browser build
|
||||
|
||||
- name: Start MinIO
|
||||
run: |
|
||||
for i in $(seq 1 5); do
|
||||
docker pull minio/minio:latest && break
|
||||
echo "docker pull failed (attempt $i), retrying"; sleep 10
|
||||
done
|
||||
docker run -d --name minio -p 9000:9000 \
|
||||
-e MINIO_ROOT_USER=minio -e MINIO_ROOT_PASSWORD=minio123 \
|
||||
minio/minio:latest server /data
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf http://localhost:9000/minio/health/live && break
|
||||
sleep 1
|
||||
done
|
||||
|
||||
- name: Create MinIO bucket
|
||||
run: |
|
||||
for i in 1 2 3; do
|
||||
if curl -fsSL https://dl.min.io/client/mc/release/linux-amd64/mc -o /tmp/mc \
|
||||
&& chmod +x /tmp/mc && /tmp/mc --version >/dev/null 2>&1; then
|
||||
break
|
||||
fi
|
||||
echo "mc download attempt $i failed; retrying"; sleep 3
|
||||
done
|
||||
/tmp/mc --version
|
||||
/tmp/mc alias set local http://localhost:9000 minio minio123
|
||||
/tmp/mc mb local/mirage-integ || true
|
||||
/tmp/mc mb local/mirage-integ-s3 || true
|
||||
/tmp/mc mb local/mirage-integ-gcs || true
|
||||
/tmp/mc mb local/mirage-integ-minio || true
|
||||
|
||||
- name: Run RAM backend and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx ram.ts > /tmp/ts-ram.out
|
||||
diff truth.txt /tmp/ts-ram.out
|
||||
|
||||
- name: Run Disk backend and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx disk.ts > /tmp/ts-disk.out
|
||||
diff truth.txt /tmp/ts-disk.out
|
||||
|
||||
- name: Run OPFS backend and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx opfs.ts > /tmp/ts-opfs.out
|
||||
diff truth.txt /tmp/ts-opfs.out
|
||||
|
||||
- name: Run Redis backend and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx redis.ts > /tmp/ts-redis.out
|
||||
diff truth.txt /tmp/ts-redis.out
|
||||
|
||||
- name: Run Safeguard scenarios and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx safeguard.ts > /tmp/ts-safeguard.out
|
||||
diff truth_safeguard.txt /tmp/ts-safeguard.out
|
||||
|
||||
- name: Run shared command suite on S3 (MinIO) and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx s3_cases.ts > /tmp/ts-s3.out
|
||||
diff truth.txt /tmp/ts-s3.out
|
||||
|
||||
- name: Run s3-native suite (multi-cloud/MinIO) and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx s3.ts > /tmp/ts-s3-native.out
|
||||
diff truth_s3_ts.txt /tmp/ts-s3-native.out
|
||||
|
||||
- name: Run cross-mount commands (ram -> ram/s3 via MinIO)
|
||||
working-directory: integ
|
||||
run: pnpm exec tsx cross_commands.ts
|
||||
|
||||
- name: Run notion integ (TS, mock server) and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx notion.ts > /tmp/notion_ts.out
|
||||
diff truth_notion.txt /tmp/notion_ts.out
|
||||
|
||||
- name: Run lancedb integ (TS, embedded) and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx lancedb.ts > /tmp/ts-lancedb.out
|
||||
diff truth_lancedb.txt /tmp/ts-lancedb.out
|
||||
|
||||
- name: Run find arg-error suite (SaaS backends, dummy creds) and diff
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx find_arg_errors.ts > /tmp/ts-fae.out
|
||||
diff truth_find_arg_errors.txt /tmp/ts-fae.out
|
||||
|
||||
integ-database:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.database == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_USER: mirage
|
||||
POSTGRES_PASSWORD: mirage
|
||||
POSTGRES_DB: mirage_integ
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 10
|
||||
chroma:
|
||||
image: chromadb/chroma:latest
|
||||
ports:
|
||||
- 8000:8000
|
||||
qdrant:
|
||||
image: qdrant/qdrant:latest
|
||||
ports:
|
||||
- 6333:6333
|
||||
env:
|
||||
MONGODB_URI: mongodb://localhost:27017/?replicaSet=rs0
|
||||
POSTGRES_DSN: postgres://mirage:mirage@localhost:5432/mirage_integ
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Wait for ChromaDB OSS
|
||||
run: |
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf http://localhost:8000/api/v2/heartbeat && exit 0
|
||||
sleep 1
|
||||
done
|
||||
echo "ChromaDB did not become healthy" >&2
|
||||
exit 1
|
||||
|
||||
- name: Wait for Qdrant
|
||||
run: |
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf http://localhost:6333/readyz && exit 0
|
||||
sleep 1
|
||||
done
|
||||
echo "Qdrant did not become healthy" >&2
|
||||
exit 1
|
||||
|
||||
- name: Start MongoDB replica set
|
||||
run: |
|
||||
docker run -d --name mongo -p 27017:27017 mongo:8 \
|
||||
mongod --replSet rs0 --bind_ip_all
|
||||
for i in $(seq 1 30); do
|
||||
docker exec mongo mongosh --quiet --eval 'db.runCommand({ping:1}).ok' >/dev/null 2>&1 && break
|
||||
sleep 1
|
||||
done
|
||||
docker exec mongo mongosh --quiet --eval \
|
||||
"rs.initiate({_id:'rs0',members:[{_id:0,host:'localhost:27017'}]})"
|
||||
for i in $(seq 1 30); do
|
||||
[ "$(docker exec mongo mongosh --quiet --eval 'db.hello().isWritablePrimary' 2>/dev/null)" = "true" ] && break
|
||||
sleep 1
|
||||
done
|
||||
|
||||
- name: Run MongoDB backend and check against truth
|
||||
shell: bash
|
||||
run: |
|
||||
set -o pipefail
|
||||
./python/.venv/bin/python integ/mongodb.py 2>&1 \
|
||||
| bash integ/check_lines.sh integ/truth_mongodb.txt
|
||||
|
||||
- name: Run Postgres backend and check against truth
|
||||
shell: bash
|
||||
run: |
|
||||
set -o pipefail
|
||||
./python/.venv/bin/python integ/postgres.py 2>&1 \
|
||||
| bash integ/check_lines.sh integ/truth_postgres.txt
|
||||
|
||||
- name: Run chroma integ (ChromaDB OSS) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/chroma.py > /tmp/chroma.out
|
||||
diff integ/truth_chroma.txt /tmp/chroma.out
|
||||
|
||||
- name: Run qdrant integ and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/qdrant.py > /tmp/qdrant.out
|
||||
diff integ/truth_qdrant.txt /tmp/qdrant.out
|
||||
|
||||
integ-database-ts:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.database == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_USER: mirage
|
||||
POSTGRES_PASSWORD: mirage
|
||||
POSTGRES_DB: mirage_integ
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 10
|
||||
chroma:
|
||||
image: chromadb/chroma:latest
|
||||
ports:
|
||||
- 8000:8000
|
||||
qdrant:
|
||||
image: qdrant/qdrant:latest
|
||||
ports:
|
||||
- 6333:6333
|
||||
env:
|
||||
MONGODB_URI: mongodb://localhost:27017/?replicaSet=rs0
|
||||
POSTGRES_DSN: postgres://mirage:mirage@localhost:5432/mirage_integ
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '24'
|
||||
|
||||
- name: Set up pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile=false
|
||||
|
||||
- name: Build mirage-core
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-core build
|
||||
|
||||
- name: Build mirage-node
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-node build
|
||||
|
||||
- name: Wait for ChromaDB OSS
|
||||
run: |
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf http://localhost:8000/api/v2/heartbeat && exit 0
|
||||
sleep 1
|
||||
done
|
||||
echo "ChromaDB did not become healthy" >&2
|
||||
exit 1
|
||||
|
||||
- name: Start MongoDB replica set
|
||||
run: |
|
||||
docker run -d --name mongo -p 27017:27017 mongo:8 \
|
||||
mongod --replSet rs0 --bind_ip_all
|
||||
for i in $(seq 1 30); do
|
||||
docker exec mongo mongosh --quiet --eval 'db.runCommand({ping:1}).ok' >/dev/null 2>&1 && break
|
||||
sleep 1
|
||||
done
|
||||
docker exec mongo mongosh --quiet --eval \
|
||||
"rs.initiate({_id:'rs0',members:[{_id:0,host:'localhost:27017'}]})"
|
||||
for i in $(seq 1 30); do
|
||||
[ "$(docker exec mongo mongosh --quiet --eval 'db.hello().isWritablePrimary' 2>/dev/null)" = "true" ] && break
|
||||
sleep 1
|
||||
done
|
||||
|
||||
- name: Wait for Qdrant
|
||||
run: |
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf http://localhost:6333/readyz && exit 0
|
||||
sleep 1
|
||||
done
|
||||
echo "Qdrant did not become healthy" >&2
|
||||
exit 1
|
||||
|
||||
- name: Run MongoDB backend (TS) and check against truth
|
||||
shell: bash
|
||||
working-directory: integ
|
||||
run: |
|
||||
set -o pipefail
|
||||
pnpm exec tsx mongodb.ts 2>&1 \
|
||||
| bash check_lines.sh truth_mongodb.txt
|
||||
|
||||
- name: Run Postgres backend (TS) and check against truth
|
||||
shell: bash
|
||||
working-directory: integ
|
||||
run: |
|
||||
set -o pipefail
|
||||
pnpm exec tsx postgres.ts 2>&1 \
|
||||
| bash check_lines.sh truth_postgres.txt
|
||||
|
||||
- name: Run TS chroma integ (ChromaDB OSS) and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx chroma.ts > /tmp/ts-chroma.out
|
||||
diff truth_chroma.txt /tmp/ts-chroma.out
|
||||
|
||||
- name: Run TS qdrant integ and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx qdrant.ts > /tmp/ts-qdrant.out
|
||||
diff truth_qdrant.txt /tmp/ts-qdrant.out
|
||||
|
||||
integ-data:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.data == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
nextcloud:
|
||||
image: nextcloud:30-apache
|
||||
ports:
|
||||
- 8080:80
|
||||
env:
|
||||
SQLITE_DATABASE: nextcloud
|
||||
NEXTCLOUD_ADMIN_USER: admin
|
||||
NEXTCLOUD_ADMIN_PASSWORD: admin123
|
||||
NEXTCLOUD_TRUSTED_DOMAINS: localhost
|
||||
env:
|
||||
NEXTCLOUD_URL: http://localhost:8080/remote.php/dav/files/admin/
|
||||
NEXTCLOUD_USERNAME: admin
|
||||
NEXTCLOUD_PASSWORD: admin123
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Run dify integ (mock server) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/dify.py > /tmp/dify.out
|
||||
diff integ/truth_dify.txt /tmp/dify.out
|
||||
|
||||
- name: Run notion integ (mock server) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/notion.py > /tmp/notion.out
|
||||
diff integ/truth_notion.txt /tmp/notion.out
|
||||
|
||||
- name: Wait for Nextcloud install
|
||||
shell: bash
|
||||
run: |
|
||||
for i in $(seq 1 60); do
|
||||
if curl -sf http://localhost:8080/status.php | grep -q '"installed":true'; then
|
||||
echo "Nextcloud is installed"
|
||||
exit 0
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
echo "Nextcloud did not finish installing in time" >&2
|
||||
curl -s http://localhost:8080/status.php || true
|
||||
exit 1
|
||||
|
||||
- name: Run Nextcloud backend and check against truth
|
||||
shell: bash
|
||||
run: |
|
||||
set -o pipefail
|
||||
./python/.venv/bin/python integ/nextcloud.py 2>&1 \
|
||||
| bash integ/check_lines.sh integ/truth_nextcloud.txt
|
||||
|
||||
- name: Run shared cases on Nextcloud and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/nextcloud_cases.py > /tmp/nextcloud_cases.out
|
||||
diff integ/truth.txt /tmp/nextcloud_cases.out
|
||||
|
||||
integ-ssh:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.ssh == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Run ssh integ (mock sftp server) and diff against truth
|
||||
run: |
|
||||
./python/.venv/bin/python integ/ssh.py > /tmp/ssh.out
|
||||
diff integ/truth.txt /tmp/ssh.out
|
||||
|
||||
integ-ssh-ts:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.ssh == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '24'
|
||||
|
||||
- name: Set up pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile=false
|
||||
|
||||
- name: Build mirage-core
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-core build
|
||||
|
||||
- name: Build mirage-node
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-node build
|
||||
|
||||
- name: Run TS ssh integ (mock sftp server) and diff against truth
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx ssh.ts > /tmp/ts-ssh.out
|
||||
diff truth.txt /tmp/ts-ssh.out
|
||||
|
||||
integ-fuse:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.fuse == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install libfuse (fusermount3 + libfuse2 for mfusepy, libfuse3 headers for fuse-native)
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y fuse3 libfuse3-dev libfuse2t64 || \
|
||||
sudo apt-get install -y fuse3 libfuse3-dev libfuse2
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Run Python FUSE mount and check (real libfuse)
|
||||
run: |
|
||||
./python/.venv/bin/python integ/fuse.py 2>&1 \
|
||||
| bash integ/check_lines.sh integ/truth_fuse.txt
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '24'
|
||||
|
||||
- name: Set up pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Install dependencies (builds @zkochan/fuse-native against libfuse3)
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile=false
|
||||
|
||||
- name: Build mirage-core
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-core build
|
||||
|
||||
- name: Build mirage-node
|
||||
working-directory: typescript
|
||||
run: pnpm --filter @struktoai/mirage-node build
|
||||
|
||||
- name: Run TypeScript FUSE mount and check (real libfuse)
|
||||
working-directory: integ
|
||||
run: |
|
||||
pnpm exec tsx fuse.ts 2>&1 \
|
||||
| bash check_lines.sh truth_fuse.txt
|
||||
|
||||
gate:
|
||||
name: integ-gate
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes, integ, integ-ts, integ-database, integ-database-ts, integ-data, integ-ssh, integ-ssh-ts, integ-fuse]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check required jobs
|
||||
run: |
|
||||
results="${{ join(needs.*.result, ' ') }}"
|
||||
echo "$results"
|
||||
case "$results" in
|
||||
*failure*|*cancelled*) exit 1 ;;
|
||||
esac
|
||||
@@ -0,0 +1,208 @@
|
||||
name: Test (Python)
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
"on":
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- python/**
|
||||
- examples/python/**
|
||||
- integ/**
|
||||
- .github/workflows/test_python.yml
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
permissions:
|
||||
pull-requests: read
|
||||
outputs:
|
||||
hit: ${{ steps.filter.outputs.hit }}
|
||||
steps:
|
||||
- uses: dorny/paths-filter@v4
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
hit:
|
||||
- 'python/**'
|
||||
- 'examples/python/**'
|
||||
- 'integ/**'
|
||||
- '.github/workflows/test_python.yml'
|
||||
|
||||
test:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
redis:
|
||||
image: redis:7
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
mongodb:
|
||||
image: mongo:8
|
||||
ports:
|
||||
- 27017:27017
|
||||
options: >-
|
||||
--health-cmd "mongosh --quiet --eval 'db.runCommand({ping:1}).ok'"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 10
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: python
|
||||
run: uv sync --all-extras --no-extra camel
|
||||
|
||||
- name: Install package
|
||||
working-directory: python
|
||||
run: uv pip install .
|
||||
|
||||
- name: Run pytest (main suite, camel excluded)
|
||||
working-directory: python
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
MONGODB_URI: mongodb://localhost:27017
|
||||
run: uv run pytest --ignore=tests/agents/camel
|
||||
|
||||
- name: Examples (output checked against integ/ truth files)
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
run: |
|
||||
run() { echo "## $1"; ./python/.venv/bin/python "$1" 2>&1 | bash integ/check_lines.sh "$2"; }
|
||||
run examples/python/ram/ram.py integ/truth/python/ram.txt
|
||||
run examples/python/ram/ram_vfs.py integ/truth/python/ram_vfs.txt
|
||||
run examples/python/ram/ram_python.py integ/truth/python/ram_python.txt
|
||||
run examples/python/disk/disk.py integ/truth/python/disk.txt
|
||||
run examples/python/disk/disk_vfs.py integ/truth/python/disk_vfs.txt
|
||||
run examples/python/other/custom_command.py integ/truth/python/custom_command.txt
|
||||
run examples/python/redis_resource/example_redis.py integ/truth/python/redis.txt
|
||||
run examples/python/redis_resource/example_redis_index.py integ/truth/python/redis_index.txt
|
||||
run examples/python/redis_resource/example_redis_cache.py integ/truth/python/redis_cache.txt
|
||||
run examples/python/redis_resource/example_redis_vfs.py integ/truth/python/redis_vfs.txt
|
||||
run examples/python/version/branching.py integ/truth/version_branching.txt
|
||||
|
||||
- name: Sync Python dependencies for camel
|
||||
working-directory: python
|
||||
run: uv sync --extra camel
|
||||
|
||||
- name: Run pytest (camel toolkit tests)
|
||||
working-directory: python
|
||||
run: uv run pytest tests/agents/camel
|
||||
|
||||
import-isolation:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- extra: pydantic-ai
|
||||
module: mirage.agents.pydantic_ai
|
||||
blocked: deepagents
|
||||
- extra: openai
|
||||
module: mirage.agents.openai_agents
|
||||
blocked: deepagents
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Create venv and install only the target extra
|
||||
working-directory: python
|
||||
run: |
|
||||
uv venv .venv-iso
|
||||
uv pip install --python .venv-iso/bin/python ".[${{ matrix.extra }}]"
|
||||
|
||||
- name: Verify blocked package is not installed
|
||||
working-directory: python
|
||||
run: |
|
||||
if .venv-iso/bin/python -c "import ${{ matrix.blocked }}" 2>/dev/null; then
|
||||
echo "FAIL: ${{ matrix.blocked }} should not be installed for extra=${{ matrix.extra }}"
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: ${{ matrix.blocked }} is absent"
|
||||
|
||||
- name: Import agent package and prompt without blocked dep
|
||||
working-directory: python
|
||||
run: |
|
||||
.venv-iso/bin/python - <<'PY'
|
||||
import importlib, sys
|
||||
mod = importlib.import_module("${{ matrix.module }}")
|
||||
prompt = importlib.import_module("${{ matrix.module }}.prompt")
|
||||
assert isinstance(prompt.MIRAGE_SYSTEM_PROMPT, str)
|
||||
assert callable(prompt.build_system_prompt)
|
||||
assert "${{ matrix.blocked }}" not in sys.modules, \
|
||||
f"${{ matrix.blocked }} got imported transitively"
|
||||
print(f"OK: imported ${{ matrix.module }} cleanly without ${{ matrix.blocked }}")
|
||||
PY
|
||||
|
||||
audit:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Audit dependencies for known vulnerabilities (informational)
|
||||
working-directory: python
|
||||
run: uv audit --preview-features audit --no-extra camel
|
||||
|
||||
gate:
|
||||
name: test-python-gate
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes, test, import-isolation]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check required jobs
|
||||
run: |
|
||||
results="${{ join(needs.*.result, ' ') }}"
|
||||
echo "$results"
|
||||
case "$results" in
|
||||
*failure*|*cancelled*) exit 1 ;;
|
||||
esac
|
||||
@@ -0,0 +1,161 @@
|
||||
name: Test (TypeScript)
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
"on":
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- typescript/**
|
||||
- examples/typescript/**
|
||||
- integ/**
|
||||
- .github/workflows/test_typescript.yml
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
permissions:
|
||||
pull-requests: read
|
||||
outputs:
|
||||
hit: ${{ steps.filter.outputs.hit }}
|
||||
steps:
|
||||
- uses: dorny/paths-filter@v4
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
hit:
|
||||
- 'typescript/**'
|
||||
- 'examples/typescript/**'
|
||||
- 'integ/**'
|
||||
- '.github/workflows/test_typescript.yml'
|
||||
|
||||
test:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
redis:
|
||||
image: redis:7
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: pnpm
|
||||
cache-dependency-path: typescript/pnpm-lock.yaml
|
||||
|
||||
- name: Install node-gyp
|
||||
run: npm install -g node-gyp
|
||||
|
||||
- name: Install TypeScript dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build TypeScript packages
|
||||
working-directory: typescript
|
||||
run: pnpm -r build
|
||||
|
||||
- name: Typecheck
|
||||
working-directory: typescript
|
||||
run: pnpm -r typecheck
|
||||
|
||||
- name: Run TypeScript tests
|
||||
working-directory: typescript
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
run: pnpm test
|
||||
|
||||
- name: Examples (output checked against integ/ truth files)
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
run: |
|
||||
run() { echo "## $1"; pnpm -C examples/typescript exec tsx "$1" 2>&1 | bash integ/check_lines.sh "$2"; }
|
||||
run ram/ram.ts integ/truth/typescript/ram.txt
|
||||
run ram/ram_vfs.ts integ/truth/typescript/ram_vfs.txt
|
||||
run disk/disk.ts integ/truth/typescript/disk.txt
|
||||
run disk/disk_vfs.ts integ/truth/typescript/disk_vfs.txt
|
||||
run other/custom_command.ts integ/truth/typescript/custom_command.txt
|
||||
run pyodide/basic.ts integ/truth/typescript/pyodide_basic.txt
|
||||
run pyodide/env.ts integ/truth/typescript/pyodide_env.txt
|
||||
run pyodide/heredoc.ts integ/truth/typescript/pyodide_heredoc.txt
|
||||
run pyodide/script.ts integ/truth/typescript/pyodide_script.txt
|
||||
run pyodide/ram.ts integ/truth/typescript/pyodide_ram.txt
|
||||
run redis/redis.ts integ/truth/typescript/redis.txt
|
||||
run redis/redis_index.ts integ/truth/typescript/redis_index.txt
|
||||
run redis/redis_cache.ts integ/truth/typescript/redis_cache.txt
|
||||
run redis/redis_vfs.ts integ/truth/typescript/redis_vfs.txt
|
||||
run version/branching.ts integ/truth/version_branching.txt
|
||||
|
||||
python-fs-shim:
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.1
|
||||
|
||||
- name: Set up Node 24 (for --experimental-wasm-jspi)
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "24"
|
||||
cache: pnpm
|
||||
cache-dependency-path: typescript/pnpm-lock.yaml
|
||||
|
||||
- name: Install node-gyp
|
||||
run: npm install -g node-gyp
|
||||
|
||||
- name: Install TypeScript dependencies
|
||||
working-directory: typescript
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build TypeScript packages
|
||||
working-directory: typescript
|
||||
run: pnpm -r --filter './packages/*' build
|
||||
|
||||
- name: Python FS shim example (output checked against integ/ truth file)
|
||||
working-directory: examples/typescript
|
||||
run: |
|
||||
node --experimental-wasm-jspi --import tsx/esm pyodide/vfs.ts 2>&1 \
|
||||
| bash ../../integ/check_lines.sh ../../integ/truth/typescript/pyodide_vfs.txt
|
||||
|
||||
gate:
|
||||
name: test-typescript-gate
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes, test, python-fs-shim]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check required jobs
|
||||
run: |
|
||||
results="${{ join(needs.*.result, ' ') }}"
|
||||
echo "$results"
|
||||
case "$results" in
|
||||
*failure*|*cancelled*) exit 1 ;;
|
||||
esac
|
||||
@@ -0,0 +1,234 @@
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[codz]
|
||||
*$py.class
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
/lib/
|
||||
/lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
share/python-wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
.cache
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
*.cover
|
||||
*.py.cover
|
||||
.hypothesis/
|
||||
.pytest_cache/
|
||||
cover/
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
local_settings.py
|
||||
db.sqlite3
|
||||
db.sqlite3-journal
|
||||
|
||||
# Flask stuff:
|
||||
instance/
|
||||
.webassets-cache
|
||||
|
||||
# Scrapy stuff:
|
||||
.scrapy
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
.pybuilder/
|
||||
target/
|
||||
|
||||
# Jupyter Notebook
|
||||
.ipynb_checkpoints
|
||||
|
||||
# IPython
|
||||
profile_default/
|
||||
ipython_config.py
|
||||
|
||||
# pyenv
|
||||
# For a library or package, you might want to ignore these files since the code is
|
||||
# intended to run in multiple environments; otherwise, check them in:
|
||||
# .python-version
|
||||
|
||||
# pipenv
|
||||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
||||
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
||||
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
||||
# install all needed dependencies.
|
||||
#Pipfile.lock
|
||||
|
||||
# UV
|
||||
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
|
||||
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
||||
# commonly ignored for libraries.
|
||||
#uv.lock
|
||||
|
||||
# poetry
|
||||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
||||
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
||||
# commonly ignored for libraries.
|
||||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
||||
#poetry.lock
|
||||
#poetry.toml
|
||||
|
||||
# pdm
|
||||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
||||
# pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
|
||||
# https://pdm-project.org/en/latest/usage/project/#working-with-version-control
|
||||
#pdm.lock
|
||||
#pdm.toml
|
||||
.pdm-python
|
||||
.pdm-build/
|
||||
|
||||
# pixi
|
||||
# Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
|
||||
#pixi.lock
|
||||
# Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
|
||||
# in the .venv directory. It is recommended not to include this directory in version control.
|
||||
.pixi
|
||||
|
||||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
||||
__pypackages__/
|
||||
|
||||
# Celery stuff
|
||||
celerybeat-schedule
|
||||
celerybeat.pid
|
||||
|
||||
# SageMath parsed files
|
||||
*.sage.py
|
||||
|
||||
# Environments
|
||||
.env
|
||||
.env.development
|
||||
.envrc
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# Spyder project settings
|
||||
.spyderproject
|
||||
.spyproject
|
||||
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
|
||||
# mkdocs documentation
|
||||
/site
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
dmypy.json
|
||||
|
||||
# Pyre type checker
|
||||
.pyre/
|
||||
|
||||
# pytype static type analyzer
|
||||
.pytype/
|
||||
|
||||
# Cython debug symbols
|
||||
cython_debug/
|
||||
|
||||
# PyCharm
|
||||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
||||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
||||
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
||||
#.idea/
|
||||
|
||||
# Abstra
|
||||
# Abstra is an AI-powered process automation framework.
|
||||
# Ignore directories containing user credentials, local state, and settings.
|
||||
# Learn more at https://abstra.io/docs
|
||||
.abstra/
|
||||
|
||||
# Visual Studio Code
|
||||
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
|
||||
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
|
||||
# and can be added to the global gitignore or merged into this file. However, if you prefer,
|
||||
# you could uncomment the following to ignore the entire vscode folder
|
||||
# .vscode/
|
||||
|
||||
# Ruff stuff:
|
||||
.ruff_cache/
|
||||
|
||||
# PyPI configuration file
|
||||
.pypirc
|
||||
|
||||
# Cursor
|
||||
# Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
|
||||
# exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
|
||||
# refer to https://docs.cursor.com/context/ignore-files
|
||||
.cursorignore
|
||||
.cursorindexingignore
|
||||
|
||||
# Marimo
|
||||
marimo/_static/
|
||||
marimo/_lsp/
|
||||
__marimo__/
|
||||
|
||||
.DS_Store
|
||||
|
||||
# Audio models
|
||||
models/
|
||||
|
||||
# Internal docs
|
||||
.docs/
|
||||
|
||||
# Git worktrees
|
||||
.worktrees/
|
||||
|
||||
# Local symlink for the TS CLI bin (not committed)
|
||||
/mirage-ts
|
||||
|
||||
# OG image generator (local only, lint-noisy)
|
||||
/scripts/generate_og_images.py
|
||||
|
||||
# Local-only working dirs (untracked but kept on disk)
|
||||
/.claude/
|
||||
/logo/
|
||||
/paper/
|
||||
/plan/
|
||||
|
||||
docs/plans/
|
||||
docs/learnings/
|
||||
@@ -0,0 +1,2 @@
|
||||
[settings]
|
||||
known_first_party = mirage,tests
|
||||
@@ -0,0 +1,142 @@
|
||||
ci:
|
||||
# https://pre-commit.ci/#configuration
|
||||
autofix_prs: true
|
||||
autoupdate_commit_msg: '[pre-commit.ci] pre-commit suggestions'
|
||||
autoupdate_schedule: monthly
|
||||
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v6.0.0
|
||||
hooks:
|
||||
- id: no-commit-to-branch
|
||||
name: No commits to master
|
||||
- id: end-of-file-fixer
|
||||
name: End-of-file fixer
|
||||
exclude: |
|
||||
(?x)^(
|
||||
typescript/.*|
|
||||
integ/truth\.txt
|
||||
)$
|
||||
- name: mixed-line-ending
|
||||
id: mixed-line-ending
|
||||
args: [--fix, lf]
|
||||
exclude: |
|
||||
(?x)^(
|
||||
typescript/.*|
|
||||
integ/truth\.txt
|
||||
)$
|
||||
- id: trailing-whitespace
|
||||
name: Remove trailing whitespaces
|
||||
exclude: |
|
||||
(?x)^(
|
||||
typescript/.*|
|
||||
integ/truth\.txt
|
||||
)$
|
||||
- id: check-toml
|
||||
name: Check toml
|
||||
- id: check-yaml
|
||||
name: Check yaml
|
||||
exclude: |
|
||||
(?x)^(
|
||||
conda/pytorch-geometric/meta.yaml|
|
||||
conda/pyg/meta.yaml|
|
||||
typescript/.*
|
||||
)$
|
||||
|
||||
- repo: https://github.com/adrienverge/yamllint.git
|
||||
rev: v1.37.1
|
||||
hooks:
|
||||
- id: yamllint
|
||||
name: Lint yaml
|
||||
args: [-d, '{extends: default, rules: {line-length: disable, document-start: disable, truthy: {level: error}, braces: {max-spaces-inside: 1}}}']
|
||||
exclude: ^typescript/
|
||||
|
||||
- repo: https://github.com/asottile/pyupgrade
|
||||
rev: v3.21.0
|
||||
hooks:
|
||||
- id: pyupgrade
|
||||
name: Upgrade Python syntax
|
||||
args: [--py38-plus]
|
||||
|
||||
- repo: https://github.com/PyCQA/autoflake
|
||||
rev: v2.3.1
|
||||
hooks:
|
||||
- id: autoflake
|
||||
name: Remove unused imports and variables
|
||||
args: [
|
||||
--remove-all-unused-imports,
|
||||
--remove-unused-variables,
|
||||
--remove-duplicate-keys,
|
||||
--ignore-init-module-imports,
|
||||
--in-place,
|
||||
]
|
||||
|
||||
- repo: https://github.com/google/yapf
|
||||
rev: v0.43.0
|
||||
hooks:
|
||||
- id: yapf
|
||||
name: Format code
|
||||
additional_dependencies: [toml]
|
||||
|
||||
- repo: https://github.com/pycqa/isort
|
||||
rev: 7.0.0
|
||||
hooks:
|
||||
- id: isort
|
||||
name: Sort imports
|
||||
|
||||
- repo: https://github.com/PyCQA/flake8
|
||||
rev: 7.3.0
|
||||
hooks:
|
||||
- id: flake8
|
||||
name: Check PEP8
|
||||
args: [--toml-config=python/pyproject.toml]
|
||||
additional_dependencies: [Flake8-pyproject]
|
||||
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.14.3
|
||||
hooks:
|
||||
- id: ruff
|
||||
name: Ruff formatting
|
||||
args: [--fix, --exit-non-zero-on-fix]
|
||||
|
||||
- repo: https://github.com/executablebooks/mdformat
|
||||
rev: 0.7.22
|
||||
hooks:
|
||||
- id: mdformat
|
||||
name: Format Markdown
|
||||
additional_dependencies:
|
||||
- mdformat-gfm
|
||||
- mdformat_frontmatter
|
||||
- mdformat_footnote
|
||||
exclude: |
|
||||
(?x)^(
|
||||
test/data/cursor\.md|
|
||||
typescript/.*
|
||||
)$
|
||||
|
||||
- repo: https://github.com/sphinx-contrib/sphinx-lint
|
||||
rev: v1.0.1
|
||||
hooks:
|
||||
- id: sphinx-lint
|
||||
name: Check Sphinx
|
||||
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: ts-prettier
|
||||
name: Prettier (typescript)
|
||||
entry: typescript/scripts/precommit-run.sh prettier --write
|
||||
language: system
|
||||
files: ^typescript/.*\.(ts|tsx|js|mjs|cjs|json|md|yaml|yml)$
|
||||
pass_filenames: true
|
||||
- id: ts-eslint
|
||||
name: ESLint (typescript)
|
||||
entry: typescript/scripts/precommit-run.sh eslint --fix
|
||||
language: system
|
||||
files: ^typescript/.*\.(ts|tsx|js|mjs|cjs)$
|
||||
pass_filenames: true
|
||||
- id: ts-knip
|
||||
name: Knip (typescript dead code)
|
||||
entry: bash -c 'cd typescript && pnpm exec knip'
|
||||
language: system
|
||||
files: ^typescript/.*\.(ts|tsx|js|mjs|cjs|json)$
|
||||
pass_filenames: false
|
||||
@@ -0,0 +1,8 @@
|
||||
cff-version: 1.2.0
|
||||
message: "Please cite Mirage if you use this code in your own work."
|
||||
title: "Mirage: A Unified Virtual Filesystem for AI Agents"
|
||||
authors:
|
||||
- family-names: "Zhang"
|
||||
given-names: "Zecheng"
|
||||
license: Apache-2.0
|
||||
url: "https://github.com/strukto-ai/mirage"
|
||||
@@ -0,0 +1,119 @@
|
||||
# CLAUDE.md
|
||||
|
||||
MIRAGE is a package that allows you to mount anything as a filesystem and make it usable by AI Agents.
|
||||
|
||||
## Repo Layout
|
||||
|
||||
This monorepo hosts two sibling implementations:
|
||||
|
||||
- `python/` — the Python package (`mirage/`, `tests/`, `pyproject.toml`, `uv.lock`).
|
||||
- `typescript/` — the TypeScript monorepo (`packages/core`, `packages/browser`, `packages/node`, etc.).
|
||||
- `docs/`, `examples/`, `.github/` — shared across both.
|
||||
|
||||
Run Python commands from `python/`, TypeScript commands from `typescript/`.
|
||||
|
||||
### TypeScript packages
|
||||
|
||||
- `typescript/packages/core` contains runtime-agnostic primitives and shared logic. Code in `core` must work in both browser and Node.js runtimes; do not put browser-only or Node-only APIs there.
|
||||
- `typescript/packages/browser` contains browser-only resources, commands, and workspace wiring. It depends on `@struktoai/mirage-core`.
|
||||
- `typescript/packages/node` contains Node.js-only resources, commands, and workspace wiring. It depends on `@struktoai/mirage-core`.
|
||||
- Put shared TypeScript behavior in `core` only when it works in both runtime packages. Put runtime-specific behavior in `browser` or `node`.
|
||||
|
||||
## Python/TypeScript Parity
|
||||
|
||||
- Keep Python and TypeScript layout, architecture, and semantics mirrored as much as practical.
|
||||
- When changing one implementation, check the other for the matching pattern or feature. If one side is more correct, use it to improve the weaker side instead of copying a bad design.
|
||||
- For major Python or TypeScript changes, consider adding or updating integration coverage under `integ/`.
|
||||
- Known gap: TypeScript does not support ORC files. Python registers `.orc` in its filetype factory (`mirage/core/filetype/orc.py` plus per-backend `read_orc` ops); the TypeScript filetype factory only covers parquet, feather/arrow/ipc, and hdf5/h5. Do not assume `.orc` commands work in TypeScript.
|
||||
|
||||
## History
|
||||
|
||||
Command history is a recording, not a command log. A hidden `Observer` records every top-level command as timestamp-ordered events (`COMMAND`, `CLEAR`, `DELETE`, op events); the user-facing surfaces are just views of those events.
|
||||
|
||||
- **Observer + ObserverStore.** The `Observer` owns a storage-agnostic `ObserverStore` (`append`/`write`/`readAll`/`readMatching`/`clear`/`close`), not a mount. Stores: `RAMObserverStore` (core, default), `DiskObserverStore` and `RedisObserverStore` (node). RAM is just the default, history can persist to disk or Redis.
|
||||
- **Two views over the same events.** `/.bash_history` is a read-only view mount (`HistoryViewResource`) rendered in GNU bash histfile format (`#<epoch>` line then the command), so `cat`/`grep`/`tail`/`find` work on it for free. The `history` shell builtin (GNU `-c -d -a -n -r -w -s -p` + count) routes through the same mount, so file and builtin never disagree.
|
||||
- **Recording scope.** Top-level lines record; nested evals (`$()`, `eval`, `source`, `xargs`) run with `record: false`, so their inner ops bubble to the parent and no spurious command is logged (mirrors GNU's line reader).
|
||||
- **Snapshots.** History is captured as events into snapshot state and restored on load.
|
||||
- **Format is GNU bash, not zsh** (`#<epoch>`, not `: <ts>:<dur>;<cmd>`).
|
||||
|
||||
## FUSE
|
||||
|
||||
- **Directory and unknown sizes.** `getattr` reports `st_size` 0 for directories and for API-backed size-unknown files that have not been opened recently. Reads stay correct because Python mounts with `direct_io` (kernel reads to EOF regardless of st_size) AND `attr_timeout=0` (post-open fstat routes to `getattr(path, fh)`, which serves the real size of the open-hydrated content); prefetched bytes live in a 30s TTL cache (`PREFETCH_TTL`) so release-then-stat does not refetch. All three pieces are load-bearing: without `attr_timeout=0`, `wc -c` prints 0, BSD `cp` copies 0 bytes, and `tail -c` dumps the whole file; without `direct_io`, `cat` reads 0 bytes on macOS. Do not "fix" getattr to report real sizes eagerly (one API fetch per `ls -l` entry), and do not report fake sizes: stat-only tools (`tar`, `rsync`, `test -s`) seeing 0 matches procfs precedent. TypeScript's `fuse-native` lacks `direct_io`, so it still reports a 100 MiB sentinel until first open; on Linux its `attrTimeout: '0'` + fgetattr path makes the sentinel unnecessary (removal planned with a pnpm patch adding `direct_io` for macOS).
|
||||
- **macOS allows only one FUSE mount per process.** The second mount dies with `fuse: cannot register signal source` (mfusepy registers libfuse signal handlers, which only the first mount in a process can claim). Multi-mount scenarios (`integ/fuse.py` mounts two) pass only on Linux; do not debug them as regressions on macOS. A failed run leaks the first mount: list with `mount | grep MirageFS`, clean with `umount <mountpoint>`.
|
||||
|
||||
## Development Setup
|
||||
|
||||
This project uses `uv` for Python dependency management. Install dependencies with:
|
||||
|
||||
```bash
|
||||
cd python && uv sync --all-extras --no-extra camel
|
||||
```
|
||||
|
||||
`camel` is declared as conflicting with `openai` (and other extras) in `pyproject.toml`, so `uv sync --all-extras` fails. Exclude `camel` to keep the `openai` stack.
|
||||
|
||||
### Running examples
|
||||
|
||||
Examples under `examples/python/` load `.env.development` from the repo root (cwd-relative). To keep cwd at the root while using the `python/` venv, invoke the venv interpreter directly:
|
||||
|
||||
```bash
|
||||
./python/.venv/bin/python examples/python/s3/s3.py
|
||||
```
|
||||
|
||||
Avoid `uv --directory python run ...` for examples — it changes cwd to `python/` and breaks `load_dotenv(".env.development")`.
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- No need to consider backward compatibility for the code.
|
||||
|
||||
## Create a PR
|
||||
|
||||
When asked to create a PR, please follow the following steps:
|
||||
|
||||
1. Run `pre-commit run --all-files` from the repo root to lint and format the code.
|
||||
1. Run `cd python && uv run pytest` to run the Python tests.
|
||||
1. Run `git add -A` to add all changes.
|
||||
1. Run `git checkout -b <branch-name>` to create a new branch.
|
||||
1. Run `git commit -m "<commit-message>"` to commit the changes.
|
||||
1. Run `git push origin <branch-name>` to push the changes to the remote repository.
|
||||
1. Run `gh pr create --title "<pr-title>" --body "<pr-body>"` to create a PR.
|
||||
|
||||
## Commands
|
||||
|
||||
### Linting and Formatting
|
||||
|
||||
After making major changes, run pre-commit from the repo root to ensure code quality:
|
||||
|
||||
```bash
|
||||
./python/.venv/bin/pre-commit run --all-files
|
||||
```
|
||||
|
||||
Invoke the venv's `pre-commit` binary directly (not via `uv --directory python run`) so cwd stays at the repo root — otherwise `git ls-files` only lists files under `python/` and `examples/` gets silently skipped.
|
||||
|
||||
## Type Conventions
|
||||
|
||||
- Paths must always be represented as `PathSpec`, never raw strings. All functions that accept or return paths use `list[str | PathSpec]` where `str` is for text arguments and `PathSpec` is for paths. Never pass a path as a plain `str` — wrap it in `PathSpec`.
|
||||
|
||||
## Rules
|
||||
|
||||
- **Shell-style commands** (cat, grep, du, find, head, tail, wc, ls, etc.) follow POSIX / Unix coreutils semantics as much as possible; match BSD/GNU behavior and document any deliberate divergence. Pin exact GNU behavior with docker (`debian:stable-slim`) before changing command semantics.
|
||||
- **`find -size` is strict and rounds up.** GNU `+N` keeps `ceil(size/unit) > N`, `-N` keeps `ceil(size/unit) < N`, bare `N` keeps `ceil(size/unit) == N` (so `-size -1k` matches only empty files and `+0c` excludes empty ones). The parsers (`_parse_size` / `parseSize`) translate this once into inclusive byte bounds; backend cores just keep `min_size <= size <= max_size` and must not re-interpret the spec. Deliberate divergence: directories count as size 0 (GNU compares the inode size, e.g. 4096 on ext4), which matches what `find` sees over a mirage FUSE mount.
|
||||
- **Async-native by default.** I/O uses `aiofiles` / `redis.asyncio` / `aioboto3`, and command pipelines are async generators.
|
||||
- **Python unit tests mirror src 1:1 where reasonable.** Try to have a matching `tests/<path>/test_a.py` for each source file `mirage/<path>/a.py`. `__init__.py`, pure type-stub modules, and trivial re-exports are fine to skip; modules with real logic should have one.
|
||||
- **Do not add `__init__.py` files under `tests/`.** Tests are namespace packages and pytest discovers them without `__init__.py`. Don't create one when adding a new test directory.
|
||||
- **Monkeypatching a backend command module in tests:** the command imports its helpers by value (`from mirage.core.<backend>.read import read_bytes`), so to intercept them you must rebind the name inside the command module, not the core source module. But the command module is hard to reach: the backend package re-exports the command function in `__init__.py` (`from .cat import cat`), which shadows the submodule of the same name, so `import mirage.commands.builtin.<backend>.cat as mod`, `from ...<backend> import cat as mod`, and even pytest's string target `monkeypatch.setattr("mirage.commands.builtin.<backend>.cat.read_bytes", fake)` all resolve to the function, not the module (`AttributeError`). The command is also wrapped by `@command`, so `cat.__globals__` is the decorator's module. Reach the real command-module namespace through the unwrapped function and patch the dict: `monkeypatch.setitem(cat.__wrapped__.__globals__, "read_bytes", fake)`.
|
||||
- Avoid add any comments or docstrings on the top of the file.
|
||||
- Do not create nested functions.
|
||||
- Add type to Args for docstring.
|
||||
- Do not add comment after each line of code in the format of "# 10MB - trigger segmentation for files larger than this". The most you can add is "# 10MB".
|
||||
- For all imports you need to put to the top of the file. Don't have imports within each function.
|
||||
- **No circular imports.** If putting an import at the top would cause a cycle, that's a sign the dependency direction is wrong — fix the design (dependency injection, splitting modules, moving the shared piece to a leaf), don't paper over it with function-local lazy imports. Verify by checking that running `cd python && uv run python -c "import <every changed module>"` succeeds without ImportError.
|
||||
- **Never silently swallow exceptions.** `try: ... except: pass` (or `except SomeError: pass`) hides real bugs. If you genuinely need to ignore an error, log it (`logger.debug(...)`) or document loudly why it's safe. Default behavior should be: let the exception propagate. Especially never silently swallow `RuntimeError` — it usually signals something deeper (event loop in wrong state, recursion limit, etc.) that you need to actually fix.
|
||||
- **Never call `asyncio.run()` inside a sync function that might be invoked under an outer event loop.** It will raise `RuntimeError: asyncio.run() cannot be called from a running event loop`. If you need async behavior from a sync API, either: (a) make the calling function `async`, (b) operate on the underlying sync state directly (e.g. write to a dict instead of calling an async setter), or (c) use a sync alternative of the same library (e.g. `redis.Redis` instead of `redis.asyncio.Redis`). Do NOT wrap with `try/except RuntimeError: pass` — that masks the bug AND leaks the unawaited coroutine.
|
||||
- Please don't change any file name unless I ask you to do so.
|
||||
- Don't add too many printings or comments in the code.
|
||||
- Don't add README.md unless I ask you to do so.
|
||||
- Use uv add to install new dependencies.
|
||||
- **Command wrappers and flags.** The dispatcher passes parsed command-line flags as keyword arguments. Wrappers must declare dispatcher-injected parameters (`stdin`, `index`, `prefix`) explicitly in their signature — never fish them out of `**flags` with `.get()`. Treat `**flags: object` as an opaque bag of true command-line flags and forward it wholesale to the generic command. When a wrapper genuinely needs a flag value itself (e.g. a search push-down), read it through `FlagView` (`fl = FlagView(flags)` then `fl.bool("F")`, `fl.int("m")`, `fl.str("type")`, `fl.list("e")`) or a shared domain accessor like `pattern_arg` — never raw `flags.get(...)` / isinstance chains.
|
||||
- **Generic commands own flag interpretation.** Backend wrappers are wiring only (glob resolution, backend I/O injection, pass-through of `texts` and `flags`); all flag semantics live in the generic command for that family, mirroring the TS generics. Adding or changing a flag should touch the spec and the generic, not N wrappers.
|
||||
- **Generics parse flags once into a frozen struct.** Each generic defines a `@dataclass(frozen=True, slots=True)` flag struct plus a module-level `parse_flags(fl, ...)` (mirroring the TS `parseFlags` struct); the function body reads only struct attributes, never string keys. Construct the FlagView with the command's spec (`FlagView(flags, spec=SPECS["grep"])`) so a typo in a flag name raises KeyError instead of silently reading as False/None.
|
||||
- **Never annotate a parameter as `object`.** Use the real type: a backend handle is `accessor: Accessor` (`mirage.accessor.base`), an index is `index: IndexCacheStore | None` (`mirage.cache.index`). Ignored variadics are still typed (`*texts: str`). `object` is only acceptable as the value type of an opaque flag bag (`**flags: object`).
|
||||
@@ -0,0 +1,124 @@
|
||||
# Contributing to Mirage
|
||||
|
||||
Thank you for your interest in contributing to Mirage. Mirage is a unified
|
||||
virtual filesystem for AI agents, with Python and TypeScript implementations in
|
||||
one repository.
|
||||
|
||||
## Contribution Guidelines
|
||||
|
||||
We prefer small, well-tested contributions that solve real user problems.
|
||||
|
||||
- Open an issue or discussion before starting non-trivial features, API changes,
|
||||
or large refactors.
|
||||
- Link related issues or discussions in your pull request.
|
||||
- Keep pull requests focused. Split unrelated changes into separate PRs.
|
||||
- Include tests for bug fixes and new behavior when practical.
|
||||
- Update affected docs and examples when behavior changes.
|
||||
- AI-assisted contributions are fine, but they must be reviewed, edited, and
|
||||
tested by a human before submission. Bulk low-signal generated submissions
|
||||
may be closed.
|
||||
|
||||
## Repository Layout
|
||||
|
||||
- `python/` contains the Python package, tests, `pyproject.toml`, and `uv.lock`.
|
||||
- `typescript/` contains the TypeScript monorepo and packages.
|
||||
- `docs/`, `examples/`, and `.github/` are shared across both implementations.
|
||||
|
||||
Run Python commands from `python/`. Run TypeScript commands from `typescript/`.
|
||||
|
||||
## Development Setup
|
||||
|
||||
Install Python dependencies:
|
||||
|
||||
```bash
|
||||
cd python
|
||||
uv sync --all-extras
|
||||
```
|
||||
|
||||
Install TypeScript dependencies:
|
||||
|
||||
```bash
|
||||
cd typescript
|
||||
pnpm install
|
||||
```
|
||||
|
||||
Examples under `examples/python/` load `.env.development` from the repository
|
||||
root. Run them from the root with the Python virtualenv interpreter:
|
||||
|
||||
```bash
|
||||
./python/.venv/bin/python examples/python/s3/s3.py
|
||||
```
|
||||
|
||||
## Common Commands
|
||||
|
||||
Run Python tests:
|
||||
|
||||
```bash
|
||||
cd python
|
||||
uv run pytest
|
||||
```
|
||||
|
||||
Run Python formatting and linting from the repository root:
|
||||
|
||||
```bash
|
||||
./python/.venv/bin/pre-commit run --all-files
|
||||
```
|
||||
|
||||
Run TypeScript checks:
|
||||
|
||||
```bash
|
||||
cd typescript
|
||||
pnpm typecheck
|
||||
pnpm test
|
||||
pnpm lint
|
||||
pnpm format:check
|
||||
```
|
||||
|
||||
Build TypeScript packages:
|
||||
|
||||
```bash
|
||||
cd typescript
|
||||
pnpm build
|
||||
```
|
||||
|
||||
If you change Python dependencies, use `uv add` and commit the updated lock
|
||||
file. If you change TypeScript dependencies, use `pnpm` and commit the updated
|
||||
lock file.
|
||||
|
||||
## Code Style
|
||||
|
||||
- Represent filesystem paths with `PathSpec`, not raw strings, in Python APIs.
|
||||
- Keep imports at the top of files.
|
||||
- Avoid circular imports. Fix the dependency direction instead of using lazy
|
||||
function-local imports.
|
||||
- Do not silently swallow exceptions. Let unexpected errors propagate, or log
|
||||
clearly when an error is intentionally ignored.
|
||||
- Do not call `asyncio.run()` from sync code that may run inside an existing
|
||||
event loop.
|
||||
- Keep comments and docstrings useful and concise. When adding docstring
|
||||
`Args`, include argument types.
|
||||
|
||||
## Pull Request Checklist
|
||||
|
||||
Before opening a PR:
|
||||
|
||||
- The change is scoped to one bug, feature, or documentation improvement.
|
||||
- Related issues or discussions are linked.
|
||||
- Tests were added or updated when relevant.
|
||||
- Affected docs and examples were updated.
|
||||
- Formatting, linting, and tests pass locally, or the PR explains why a check
|
||||
could not be run.
|
||||
|
||||
Useful PR title prefixes:
|
||||
|
||||
- `feat:` for new features
|
||||
- `fix:` for bug fixes
|
||||
- `docs:` for documentation changes
|
||||
- `refactor:` for code restructuring
|
||||
- `test:` for test-only changes
|
||||
- `chore:` for maintenance work
|
||||
|
||||
## Getting Help
|
||||
|
||||
Use GitHub issues for bugs, feature requests, and design discussions. For faster
|
||||
conversation, join the Mirage Discord linked from the README.
|
||||
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2026 @ Strukto.AI
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,193 @@
|
||||
<p align="center">
|
||||
<img src="assets/mirage-og-light@2x.png" alt="Mirage: A Unified Virtual File System for AI Agents" width="900">
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://docs.mirage.strukto.ai" alt="Documentation">
|
||||
<img src="https://img.shields.io/badge/mirage-docs-0C0C0C?labelColor=FAFAFA" /></a>
|
||||
<a href="https://www.strukto.ai" alt="Website">
|
||||
<img src="https://img.shields.io/badge/made by-strukto.ai-0C0C0C?labelColor=FAFAFA" /></a>
|
||||
<a href="https://github.com/strukto-ai/mirage/blob/main/LICENSE" alt="License">
|
||||
<img src="https://img.shields.io/badge/license-Apache--2.0-0C0C0C?labelColor=FAFAFA" /></a>
|
||||
<a href="https://discord.gg/u8BPQ65KsS" alt="Discord">
|
||||
<img src="https://img.shields.io/badge/discord-join-0C0C0C?labelColor=FAFAFA&logo=discord&logoColor=0C0C0C" /></a>
|
||||
<br/>
|
||||
<a href="https://docs.mirage.strukto.ai/python/quickstart" alt="Python docs">
|
||||
<img src="https://img.shields.io/badge/python-docs-0C0C0C?labelColor=FAFAFA&logo=python&logoColor=0C0C0C" alt="Python docs"></a>
|
||||
<a href="https://pypi.org/project/mirage-ai/" alt="PyPI Version">
|
||||
<img src="https://img.shields.io/pypi/v/mirage-ai.svg?color=0C0C0C&labelColor=FAFAFA"/></a>
|
||||
<br/>
|
||||
<a href="https://docs.mirage.strukto.ai/typescript/quickstart" alt="TypeScript docs">
|
||||
<img src="https://img.shields.io/badge/typescript-docs-0C0C0C?labelColor=FAFAFA&logo=typescript&logoColor=0C0C0C" alt="TypeScript docs"></a>
|
||||
<a href="https://www.npmjs.com/package/@struktoai/mirage-node" alt="NPM Version">
|
||||
<img src="https://img.shields.io/npm/v/@struktoai/mirage-node.svg?color=0C0C0C&labelColor=FAFAFA"/></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>
|
||||
<a href="./readme/README.zh-CN.md"><img alt="简体中文 README" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
|
||||
<a href="./readme/README.zh-TW.md"><img alt="繁體中文 README" src="https://img.shields.io/badge/繁體中文-d9d9d9"></a>
|
||||
<a href="./readme/README.fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-d9d9d9"></a>
|
||||
<a href="./readme/README.vi.md"><img alt="README Tiếng Việt" src="https://img.shields.io/badge/Ti%E1%BA%BFng%20Vi%E1%BB%87t-d9d9d9"></a>
|
||||
<a href="./readme/README.ko.md"><img alt="README 한국어" src="https://img.shields.io/badge/%ED%95%9C%EA%B5%AD%EC%96%B4-d9d9d9"></a>
|
||||
</p>
|
||||
|
||||
Mirage is **a Unified Virtual File System for AI Agents**: it mounts services and data sources like S3, Google Drive, Slack, Gmail, and Redis side-by-side as one filesystem. Any LLM that already knows bash can read, grep, and pipe across every backend out of the box, with zero new vocabulary.
|
||||
|
||||
```ts
|
||||
const ws = new Workspace({
|
||||
'/data': new RAMResource(),
|
||||
'/s3': new S3Resource({ bucket: 'logs' }),
|
||||
'/slack': new SlackResource({ token: process.env.SLACK_BOT_TOKEN! }),
|
||||
})
|
||||
|
||||
await ws.execute('grep -r alert /slack/channels/general__C04QX/ | wc -l')
|
||||
await ws.execute('cp /s3/report.csv /data/local.csv')
|
||||
await ws.execute('wc -l $(find /s3/data -name "*.jsonl")')
|
||||
|
||||
// Commands are extensible: register new commands, or override one per
|
||||
// resource + filetype, e.g. `cat` on S3 Parquet renders rows as JSON.
|
||||
ws.command('summarize', ...)
|
||||
ws.command('cat', { resource: 's3', filetype: 'parquet' }, ...)
|
||||
|
||||
await ws.execute('summarize /data/local.csv')
|
||||
await ws.execute('cat /s3/events/2026-05-06.parquet | jq .user')
|
||||
```
|
||||
|
||||
## About
|
||||
|
||||
- **One interface instead of N SDKs and M MCPs.** Every service speaks the same filesystem semantics, and pipelines compose across services as naturally as on a local disk.
|
||||
- **Around 50 built-in backends:** RAM, Disk, Redis, S3 / R2 / OCI / Supabase / GCS, Gmail / GDrive / GDocs / GSheets / GSlides, GitHub / Linear / Notion / Trello, Slack / Discord / Email, MongoDB / Postgres / LanceDB / Qdrant, SSH, and more, mounted side-by-side under a single root.
|
||||
- **Portable workspaces:** clone, snapshot, and version a workspace; agent runs move between machines without restarting or reconfiguring the system.
|
||||
- **Embeddable:** the Python and TypeScript SDKs run in-process inside FastAPI, Express, browser apps, or any async runtime; no separate process required.
|
||||
- **Agent integrations:** OpenAI Agents SDK, Vercel AI SDK, LangChain, Pydantic AI, CAMEL, and OpenHands via the SDKs; coding agents like Claude Code and Codex via the lightweight CLI + daemon.
|
||||
|
||||
## Architecture
|
||||
|
||||
<p align="center">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="assets/mirage-arch-dark.svg">
|
||||
<img src="assets/mirage-arch-light.svg" alt="Mirage architecture: AI Agent and Application → Mirage Bash and VFS → Dispatcher & Cache → Infrastructure and Remote" width="720">
|
||||
</picture>
|
||||
</p>
|
||||
|
||||
## Installation
|
||||
|
||||
- **Python** ≥ 3.11 for the `mirage-ai` package and the `mirage` CLI
|
||||
- **Node.js** ≥ 20 for the TypeScript SDK
|
||||
- **macOS** or **Linux** (FUSE-based mounts require platform support)
|
||||
|
||||
### Python
|
||||
|
||||
```bash
|
||||
uv add mirage-ai # installs the `mirage` library and the `mirage` CLI binary
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```bash
|
||||
npm install @struktoai/mirage-node # Node.js servers and CLIs
|
||||
npm install @struktoai/mirage-browser # browser / edge runtimes
|
||||
npm install @struktoai/mirage-agents # OpenAI / Vercel AI / LangChain / Mastra adapters
|
||||
```
|
||||
|
||||
Both runtime packages pull in `@struktoai/mirage-core` automatically.
|
||||
|
||||
### CLI
|
||||
|
||||
```bash
|
||||
curl -fsSL https://strukto.ai/mirage/install.sh | sh
|
||||
# or
|
||||
npm install -g @struktoai/mirage-cli
|
||||
# or
|
||||
uvx mirage-ai
|
||||
# or
|
||||
npx @struktoai/mirage-cli
|
||||
```
|
||||
|
||||
## Quickstart
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
from mirage import Workspace
|
||||
from mirage.resource.ram import RAMResource
|
||||
from mirage.resource.s3 import S3Config, S3Resource
|
||||
|
||||
ws = Workspace({
|
||||
"/data": RAMResource(),
|
||||
"/s3": S3Resource(S3Config(bucket="my-bucket")),
|
||||
})
|
||||
|
||||
await ws.execute("cp /s3/report.csv /data/report.csv")
|
||||
await ws.execute("grep alert /s3/data/log.jsonl | wc -l")
|
||||
|
||||
await ws.snapshot("demo.tar")
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```ts
|
||||
import { Workspace, RAMResource, S3Resource } from '@struktoai/mirage-node'
|
||||
|
||||
const ws = new Workspace({
|
||||
'/data': new RAMResource(),
|
||||
'/s3': new S3Resource({ bucket: 'my-bucket' }),
|
||||
})
|
||||
|
||||
await ws.execute('cp /s3/report.csv /data/report.csv')
|
||||
await ws.execute('grep alert /s3/data/log.jsonl | wc -l')
|
||||
|
||||
await ws.snapshot('demo.tar')
|
||||
```
|
||||
|
||||
### CLI
|
||||
|
||||
```bash
|
||||
mirage workspace create ws.yaml --id demo
|
||||
mirage execute --workspace_id demo --command "cp /s3/report.csv /data/report.csv"
|
||||
mirage provision --workspace_id demo --command "cat /s3/data/large.jsonl"
|
||||
mirage workspace snapshot demo demo.tar
|
||||
mirage workspace load demo.tar --id demo-restored
|
||||
```
|
||||
|
||||
## Agent Frameworks
|
||||
|
||||
Mirage plugs into agent frameworks as a sandbox or tool layer. POSIX operations such as `read` can also be customized per resource and filetype, e.g. reading a PDF returns parsed pages instead of raw bytes.
|
||||
|
||||
| | Integrations |
|
||||
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Python | [OpenAI Agents SDK](https://docs.mirage.strukto.ai/python/agents/openai-agents), [LangChain](https://docs.mirage.strukto.ai/python/agents/langchain), [Pydantic AI](https://docs.mirage.strukto.ai/python/agents/pydantic-ai), [CAMEL](https://docs.mirage.strukto.ai/python/agents/camel), [OpenHands](https://docs.mirage.strukto.ai/python/agents/openhands), [Agno](https://docs.mirage.strukto.ai/python/agents/agno) |
|
||||
| TypeScript | [Vercel AI SDK](https://docs.mirage.strukto.ai/typescript/agents/vercel), [OpenAI Agents SDK](https://docs.mirage.strukto.ai/typescript/agents/openai), [LangChain](https://docs.mirage.strukto.ai/typescript/agents/langchain), [Mastra](https://docs.mirage.strukto.ai/typescript/agents/mastra) |
|
||||
| Coding agents | [Claude Code](https://docs.mirage.strukto.ai/python/agents/claude-code), [Codex](https://docs.mirage.strukto.ai/python/agents/codex), [OpenCode](https://docs.mirage.strukto.ai/typescript/agents/opencode), [Pi](https://docs.mirage.strukto.ai/typescript/agents/pi) |
|
||||
|
||||
## Cache
|
||||
|
||||
Every `Workspace` has a two-layer cache so repeated work against remote backends hits local state instead of the network:
|
||||
|
||||
- **Index cache:** listings and metadata. The first directory walk hits the API; later ones serve from the index until the TTL expires (default 10 minutes).
|
||||
- **File cache:** object bytes. The first read streams from origin; later pipelines read from cache (default 512 MB).
|
||||
|
||||
Both layers default to in-process RAM with zero setup. A Redis store shares cache state across workers, processes, and machines:
|
||||
|
||||
```ts
|
||||
import { RedisFileCacheStore, S3Resource, Workspace } from '@struktoai/mirage-node'
|
||||
|
||||
const ws = new Workspace(
|
||||
{ '/s3': new S3Resource({ bucket: 'my-bucket' }) },
|
||||
{
|
||||
cache: new RedisFileCacheStore({ url: 'redis://localhost:6379/0', cacheLimit: '8GB' }),
|
||||
index: { type: 'redis', url: 'redis://localhost:6379/0', ttl: 600 },
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
See the [cache docs](https://docs.mirage.strukto.ai/home/cache) for the full miss/hit lifecycle.
|
||||
|
||||
## Contributors
|
||||
|
||||
Thanks to everyone who has contributed to Mirage.
|
||||
|
||||
<a href="https://github.com/strukto-ai/mirage/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=strukto-ai/mirage" alt="Mirage contributors" />
|
||||
</a>
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目:`strukto-ai/mirage`
|
||||
- 原始仓库:https://github.com/strukto-ai/mirage
|
||||
- 导入方式:上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源,不代表 WeHub 是原项目作者
|
||||
@@ -0,0 +1,33 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
The following versions of Mirage are currently supported with security updates:
|
||||
|
||||
| Version | Supported |
|
||||
| ------- | ------------------ |
|
||||
| 0.0.x | :white_check_mark: |
|
||||
| < 0.0 | :x: |
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
If you discover a security vulnerability in Mirage, please report it responsibly:
|
||||
|
||||
### How to Report
|
||||
|
||||
- **Email**: Send details to zecheng@strukto.ai
|
||||
- **GitHub**: Use GitHub's private security advisory feature
|
||||
- **Include**: Detailed description, steps to reproduce, and potential impact
|
||||
|
||||
### What to Expect
|
||||
|
||||
- **Response Time**: We aim to acknowledge reports within 48 hours
|
||||
- **Updates**: We will provide updates on the investigation progress weekly
|
||||
- **Resolution**: Critical vulnerabilities will be addressed within 7 days
|
||||
- **Credit**: We will credit security researchers in our security advisories (if desired)
|
||||
|
||||
### Security Disclosure Policy
|
||||
|
||||
- We follow responsible disclosure practices
|
||||
- We request 90 days to address the vulnerability before public disclosure
|
||||
- We will coordinate disclosure timing with the reporter
|
||||
@@ -0,0 +1,181 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 1010" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif">
|
||||
<defs>
|
||||
<style type="text/css"><![CDATA[
|
||||
@import url('https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@500;600;700;800&display=swap');
|
||||
.display {
|
||||
font-family: 'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace;
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
]]></style>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
|
||||
<path d="M0 0 L10 5 L0 10 z" fill="#A1A1AA"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<rect width="1200" height="1010" fill="#0E0E10"/>
|
||||
|
||||
<!-- Title: mirage icon (cube) + "mirage" wordmark in diagram font -->
|
||||
<svg x="407" y="4" width="96" height="110" viewBox="16 7 128 146">
|
||||
<polygon points="19,45 80,10 80,45 50,63" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<polygon points="80,10 141,45 110,63 80,45" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<polygon points="141,45 141,115 110,98 110,63" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<polygon points="141,115 80,150 80,115 110,98" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<polygon points="80,150 19,115 50,98 80,115" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<polygon points="19,115 19,45 50,63 50,98" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<polygon points="80,45 110,63 110,98 80,115 50,98 50,63" fill="#FFFFFF" stroke="#000000" stroke-width="2.5"/>
|
||||
<line x1="80" y1="80" x2="50" y2="63" stroke="#000000" stroke-width="2.5"/>
|
||||
<line x1="80" y1="80" x2="110" y2="63" stroke="#000000" stroke-width="2.5"/>
|
||||
<line x1="80" y1="80" x2="80" y2="115" stroke="#000000" stroke-width="2.5"/>
|
||||
</svg>
|
||||
<text x="515" y="84" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif" font-size="72" font-weight="700" letter-spacing="-1">mirage</text>
|
||||
<text x="600" y="140" text-anchor="middle" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif" font-size="22" font-weight="500" letter-spacing="0.4">A Unified Virtual Filesystem for AI Agents</text>
|
||||
|
||||
<!-- ===================== LAYER 1: AGENT ===================== -->
|
||||
<rect x="20" y="160" width="1160" height="170" rx="8" fill="#1A1A1D" stroke="#FFFFFF" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="198" font-size="22" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">AI Agent and Application</text>
|
||||
<text x="44" y="228" font-size="18" fill="#A1A1AA">
|
||||
<tspan x="44" dy="0">Bash commands, VFS</tspan>
|
||||
<tspan x="44" dy="24">calls, or syscalls</tspan>
|
||||
</text>
|
||||
|
||||
<!-- AI Agent (centered over connectors at x=470 and x=680: equal 30px margins on each side) -->
|
||||
<rect x="440" y="210" width="270" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="575" y="242" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">AI Agent</text>
|
||||
|
||||
<!-- OS Processes (positioned over FUSE Adapter x=890 for vertical connector) -->
|
||||
<rect x="791" y="210" width="198" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="890" y="242" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">App & CLI</text>
|
||||
|
||||
|
||||
<!-- ===================== LAYER 2: WORKSPACE ===================== -->
|
||||
<rect x="20" y="376" width="1160" height="200" rx="8" fill="#1A1A1D" stroke="#FFFFFF" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="414" font-size="22" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage Bash and VFS</text>
|
||||
<text x="44" y="444" font-size="18" fill="#A1A1AA">
|
||||
<tspan x="44" dy="0">Mirage Bash &</tspan>
|
||||
<tspan x="44" dy="24">registered commands;</tspan>
|
||||
<tspan x="44" dy="24">Mirage VFS &</tspan>
|
||||
<tspan x="44" dy="24">registered VFS ops</tspan>
|
||||
</text>
|
||||
|
||||
<!-- Mirage Shell -->
|
||||
<rect x="376" y="409" width="188" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="470" y="441" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage Bash</text>
|
||||
|
||||
<!-- Mirage VFS -->
|
||||
<rect x="596" y="409" width="168" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="680" y="441" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage VFS</text>
|
||||
|
||||
<!-- FUSE Adapter -->
|
||||
<rect x="796" y="409" width="188" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="890" y="441" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">FUSE Adapter</text>
|
||||
|
||||
<!-- Command Registry -->
|
||||
<rect x="376" y="491" width="188" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="470" y="523" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Command Registry</text>
|
||||
|
||||
<!-- VFS Registry -->
|
||||
<rect x="596" y="491" width="168" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="680" y="523" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">VFS Registry</text>
|
||||
|
||||
<!-- Shell -> Command Registry -->
|
||||
<line x1="470" y1="461" x2="470" y2="491" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Mirage VFS -> VFS Registry -->
|
||||
<line x1="680" y1="461" x2="680" y2="491" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- FUSE -> Mirage VFS -->
|
||||
<line x1="796" y1="435" x2="764" y2="435" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- ===================== LAYER 3: DISPATCHER & CACHE ===================== -->
|
||||
<rect x="20" y="622" width="1160" height="140" rx="8" fill="#1A1A1D" stroke="#FFFFFF" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="660" font-size="22" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Dispatcher & Cache</text>
|
||||
<text x="44" y="690" font-size="18" fill="#A1A1AA">
|
||||
<tspan x="44" dy="0">Dispatches by mount;</tspan>
|
||||
<tspan x="44" dy="24">caches repeats</tspan>
|
||||
</text>
|
||||
|
||||
<!-- Mirage Dispatcher (matches AI Agent dimensions: w=270, h=52, x=440) -->
|
||||
<rect x="440" y="666" width="270" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="575" y="698" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage Dispatcher</text>
|
||||
|
||||
<!-- Index & File Cache (merged; aligned x with Application & CLI: x=800, w=180) -->
|
||||
<rect x="791" y="666" width="198" height="52" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="890" y="698" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Index & File Cache</text>
|
||||
|
||||
<!-- Dispatcher -> Cache -->
|
||||
<line x1="710" y1="692" x2="791" y2="692" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- ===================== LAYER 4: BACKEND ===================== -->
|
||||
<rect x="20" y="808" width="1160" height="190" rx="8" fill="#1A1A1D" stroke="#FFFFFF" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="846" font-size="22" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Infrastructure and Remote</text>
|
||||
<text x="44" y="876" font-size="18" fill="#A1A1AA">
|
||||
<tspan x="44" dy="0">Infrastructure & remote,</tspan>
|
||||
<tspan x="44" dy="24">mounted as one tree</tspan>
|
||||
</text>
|
||||
|
||||
<!-- Infrastructure (renamed from Local Backends; aligned x with Mirage Shell at 380, w=180) -->
|
||||
<rect x="380" y="833" width="180" height="140" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="470" y="861" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Infrastructure</text>
|
||||
<line x1="420" y1="873" x2="520" y2="873" stroke="#3F3F46" stroke-width="1"/>
|
||||
<text x="470" y="897" text-anchor="middle" font-size="15" fill="#A1A1AA">RAM</text>
|
||||
<text x="470" y="919" text-anchor="middle" font-size="15" fill="#A1A1AA">Disk</text>
|
||||
<text x="470" y="941" text-anchor="middle" font-size="15" fill="#A1A1AA">OPFS</text>
|
||||
<text x="470" y="963" text-anchor="middle" font-size="15" fill="#A1A1AA">Redis</text>
|
||||
|
||||
<!-- Remote Services (widened by 10px each side: x=790, w=200; center stays at 890) -->
|
||||
<rect x="790" y="833" width="200" height="140" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1.2"/>
|
||||
<text x="890" y="861" text-anchor="middle" font-size="19" font-weight="500" fill="#FFFFFF" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Remote Service</text>
|
||||
<line x1="830" y1="873" x2="950" y2="873" stroke="#3F3F46" stroke-width="1"/>
|
||||
|
||||
<!-- Icon grid: 6 per row, 22x22 colored rect with inlined simple-icons brand SVG -->
|
||||
<g transform="translate(804,883)"><rect width="22" height="22" rx="5" fill="#FF9900"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="m20.913 13.147l.12-.895c.947.576 1.258.922 1.354 1.071c-.16.031-.562.046-1.474-.176m-2.174 7.988l-.005.073c0 .084-.207.405-1.124.768a10 10 0 0 1-1.438.432c-1.405.325-3.128.504-4.853.504c-4.612 0-7.412-1.184-7.412-1.704l-.005-.073L1.81 5.602q.203.117.432.227q.064.03.128.057q.2.093.417.18l.179.069q.232.087.478.168l.13.043q.311.097.646.187l.176.044q.263.066.534.127a23 23 0 0 0 .843.17l.121.023q.378.067.768.122q.107.016.216.03q.299.04.604.077l.24.027q.366.04.74.07l.081.009q.413.033.83.056l.233.012q.316.015.633.025a33 33 0 0 0 2.795-.026l.232-.011q.417-.023.83-.056l.08-.008q.375-.03.742-.072l.238-.026q.307-.036.609-.077l.211-.03q.392-.056.772-.122l.111-.02q.323-.06.634-.125l.212-.047q.279-.062.546-.13l.166-.042q.338-.09.654-.189l.115-.038a11 11 0 0 0 .493-.173q.087-.032.17-.066q.225-.089.43-.185q.059-.025.116-.052q.23-.11.436-.228l-.976 7.245c-2.488-.78-5.805-2.292-7.311-3a1.09 1.09 0 0 0-1.088-1.085c-.6 0-1.088.489-1.088 1.088s.488 1.089 1.088 1.089c.196 0 .378-.056.537-.148c1.72.812 5.144 2.367 7.715 3.15zm-7.42-20.047c5.677 0 9.676 1.759 9.75 2.736l-.014.113c-.01.033-.031.067-.048.101c-.015.028-.026.057-.047.087c-.024.033-.058.068-.09.102c-.028.03-.051.06-.084.09c-.038.035-.087.07-.133.105c-.04.03-.074.06-.119.091c-.053.036-.116.071-.177.107c-.05.03-.095.06-.15.09c-.068.036-.147.073-.222.11c-.059.028-.114.057-.177.085c-.084.038-.177.074-.268.111c-.068.027-.13.054-.203.082c-.097.036-.205.072-.31.107c-.075.026-.148.053-.228.079c-.111.035-.233.069-.35.103c-.085.024-.165.05-.253.073c-.124.034-.258.065-.389.098c-.093.022-.181.046-.278.068c-.139.032-.287.061-.433.091c-.098.02-.191.041-.293.06c-.155.03-.32.057-.482.084c-.1.018-.198.036-.302.052c-.166.026-.342.048-.515.072c-.11.014-.213.03-.325.044c-.181.023-.372.041-.56.06q-.163.019-.332.036c-.188.016-.386.029-.58.043c-.122.009-.24.02-.364.028c-.207.012-.422.02-.635.028c-.12.005-.234.012-.354.016a36 36 0 0 1-2.069 0c-.12-.004-.234-.011-.352-.016c-.214-.008-.43-.016-.637-.028c-.122-.008-.238-.02-.36-.027c-.195-.015-.394-.028-.584-.044c-.11-.01-.215-.024-.324-.035c-.19-.02-.384-.038-.568-.06l-.315-.044c-.176-.024-.355-.046-.525-.073c-.1-.015-.192-.033-.29-.05c-.167-.028-.335-.055-.494-.086c-.096-.018-.183-.038-.276-.056c-.151-.032-.305-.062-.45-.095c-.09-.02-.173-.043-.26-.064c-.138-.034-.277-.067-.407-.102c-.082-.022-.157-.046-.235-.069a12 12 0 0 1-.368-.108c-.075-.024-.141-.049-.213-.073c-.11-.037-.223-.075-.325-.113c-.067-.025-.125-.051-.188-.077c-.096-.038-.195-.076-.282-.115c-.06-.027-.11-.054-.166-.08c-.08-.039-.162-.077-.233-.116c-.052-.028-.094-.055-.142-.084c-.063-.038-.13-.075-.185-.113c-.043-.029-.075-.058-.113-.086c-.048-.037-.098-.073-.139-.11c-.032-.029-.054-.057-.08-.087c-.033-.035-.069-.07-.093-.104c-.02-.03-.031-.058-.046-.086c-.018-.035-.039-.068-.049-.102l-.015-.113c.076-.977 4.074-2.736 9.748-2.736m12.182 12.124c-.118-.628-.84-1.291-2.31-2.128l.963-7.16l.005-.073C22.16 1.581 16.447 0 11.32 0C6.194 0 .482 1.581.482 3.851l.005.072L2.819 21.25c.071 2.002 5.236 2.75 8.5 2.75c1.805 0 3.615-.188 5.098-.531c.598-.138 1.133-.3 1.592-.48c1.18-.467 1.789-1.053 1.813-1.739l.945-7.018c.557.131 1.016.197 1.389.197c.54 0 .902-.137 1.134-.413a.96.96 0 0 0 .21-.804Z"/></svg></g> <!-- S3 -->
|
||||
<g transform="translate(834,883)"><rect width="22" height="22" rx="5" fill="#F38020"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M16.509 16.845c.147-.507.09-.971-.155-1.316c-.225-.316-.605-.499-1.062-.52l-8.66-.113a.16.16 0 0 1-.133-.07a.2.2 0 0 1-.02-.156a.24.24 0 0 1 .203-.156l8.736-.113c1.035-.049 2.16-.886 2.554-1.913l.499-1.302a.27.27 0 0 0 .014-.168a5.689 5.689 0 0 0-10.937-.584a2.58 2.58 0 0 0-1.794-.498a2.56 2.56 0 0 0-2.223 3.18A3.634 3.634 0 0 0 0 16.751q.002.264.035.527a.174.174 0 0 0 .17.148h15.98a.22.22 0 0 0 .204-.155zm2.757-5.564c-.077 0-.161 0-.239.011c-.056 0-.105.042-.127.098l-.337 1.174c-.148.507-.092.971.154 1.317c.225.316.605.498 1.062.52l1.844.113c.056 0 .105.026.133.07a.2.2 0 0 1 .021.156a.24.24 0 0 1-.204.156l-1.92.112c-1.042.049-2.159.887-2.553 1.914l-.141.358c-.028.072.021.142.099.142h6.597a.174.174 0 0 0 .17-.126a5 5 0 0 0 .175-1.28a4.74 4.74 0 0 0-4.734-4.727"/></svg></g> <!-- R2 (Cloudflare) -->
|
||||
<g transform="translate(864,883)"><rect width="22" height="22" rx="5" fill="#C74634"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M16.412 4.412h-8.82a7.588 7.588 0 0 0-.008 15.176h8.828a7.588 7.588 0 0 0 0-15.176m-.193 12.502H7.786a4.915 4.915 0 0 1 0-9.828h8.433a4.914 4.914 0 1 1 0 9.828"/></svg></g> <!-- OCI -->
|
||||
<g transform="translate(894,883)"><rect width="22" height="22" rx="5" fill="#4285F4"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M12.19 2.38a9.344 9.344 0 0 0-9.234 6.893c.053-.02-.055.013 0 0c-3.875 2.551-3.922 8.11-.247 10.941l.006-.007l-.007.03a6.7 6.7 0 0 0 4.077 1.356h5.173l.03.03h5.192c6.687.053 9.376-8.605 3.835-12.35a9.37 9.37 0 0 0-2.821-4.552l-.043.043l.006-.05A9.34 9.34 0 0 0 12.19 2.38m-.358 4.146c1.244-.04 2.518.368 3.486 1.15a5.19 5.19 0 0 1 1.862 4.078v.518c3.53-.07 3.53 5.262 0 5.193h-5.193l-.008.009v-.04H6.785a2.6 2.6 0 0 1-1.067-.23h.001a2.597 2.597 0 1 1 3.437-3.437l3.013-3.012A6.75 6.75 0 0 0 8.11 8.24c.018-.01.04-.026.054-.023a5.2 5.2 0 0 1 3.67-1.69z"/></svg></g> <!-- GCS -->
|
||||
<g transform="translate(924,883)"><rect width="22" height="22" rx="5" fill="#3FCF8E"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M11.9 1.036c-.015-.986-1.26-1.41-1.874-.637L.764 12.05C-.33 13.427.65 15.455 2.409 15.455h9.579l.113 7.51c.014.985 1.259 1.408 1.873.636l9.262-11.653c1.093-1.375.113-3.403-1.645-3.403h-9.642z"/></svg></g> <!-- Supabase -->
|
||||
<g transform="translate(954,883)"><rect width="22" height="22" rx="5" fill="#EA4335"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M24 5.457v13.909c0 .904-.732 1.636-1.636 1.636h-3.819V11.73L12 16.64l-6.545-4.91v9.273H1.636A1.636 1.636 0 0 1 0 19.366V5.457c0-2.023 2.309-3.178 3.927-1.964L5.455 4.64L12 9.548l6.545-4.91l1.528-1.145C21.69 2.28 24 3.434 24 5.457"/></svg></g> <!-- Gmail -->
|
||||
<g transform="translate(804,911)"><rect width="22" height="22" rx="5" fill="#1FA463"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M12.01 1.485c-2.082 0-3.754.02-3.743.047c.01.02 1.708 3.001 3.774 6.62l3.76 6.574h3.76c2.081 0 3.753-.02 3.742-.047c-.005-.02-1.708-3.001-3.775-6.62l-3.76-6.574zm-4.76 1.73a789.828 789.861 0 0 0-3.63 6.319L0 15.868l1.89 3.298l1.885 3.297l3.62-6.335l3.618-6.33l-1.88-3.287C8.1 4.704 7.255 3.22 7.25 3.214zm2.259 12.653l-.203.348c-.114.198-.96 1.672-1.88 3.287a423.93 423.948 0 0 1-1.698 2.97c-.01.026 3.24.042 7.222.042h7.244l1.796-3.157c.992-1.734 1.85-3.23 1.906-3.323l.104-.167h-7.249z"/></svg></g> <!-- GDrive -->
|
||||
<g transform="translate(834,911)"><rect width="22" height="22" rx="5" fill="#4285F4"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M14.727 6.727H14V0H4.91c-.905 0-1.637.732-1.637 1.636v20.728c0 .904.732 1.636 1.636 1.636h14.182c.904 0 1.636-.732 1.636-1.636V6.727zm-.545 10.455H7.09v-1.364h7.09v1.364zm2.727-3.273H7.091v-1.364h9.818zm0-3.273H7.091V9.273h9.818zM14.727 6h6l-6-6z"/></svg></g> <!-- GDocs -->
|
||||
<g transform="translate(864,911)"><rect width="22" height="22" rx="5" fill="#34A853"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M11.318 12.545H7.91v-1.909h3.41v1.91zM14.728 0v6h6zm1.363 10.636h-3.41v1.91h3.41zm0 3.273h-3.41v1.91h3.41zM20.727 6.5v15.864c0 .904-.732 1.636-1.636 1.636H4.909a1.636 1.636 0 0 1-1.636-1.636V1.636C3.273.732 4.005 0 4.909 0h9.318v6.5zm-3.273 2.773H6.545v7.909h10.91v-7.91zm-6.136 4.636H7.91v1.91h3.41v-1.91z"/></svg></g> <!-- GSheets -->
|
||||
<g transform="translate(894,911)"><rect width="22" height="22" rx="5" fill="#FBBC04"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M16.09 15.273H7.91v-4.637h8.18zm1.728-8.523h2.91v15.614c0 .904-.733 1.636-1.637 1.636H4.909a1.636 1.636 0 0 1-1.636-1.636V1.636C3.273.732 4.005 0 4.909 0h9.068v6.75zm-.363 2.523H6.545v7.363h10.91zm-2.728-5.979V6h6.001l-6-6v3.294z"/></svg></g> <!-- GSlides -->
|
||||
<g transform="translate(924,911)"><rect width="22" height="22" rx="5" fill="#4A154B"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M5.042 15.165a2.53 2.53 0 0 1-2.52 2.523A2.53 2.53 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52zm1.271 0a2.527 2.527 0 0 1 2.521-2.52a2.527 2.527 0 0 1 2.521 2.52v6.313A2.53 2.53 0 0 1 8.834 24a2.53 2.53 0 0 1-2.521-2.522zM8.834 5.042a2.53 2.53 0 0 1-2.521-2.52A2.53 2.53 0 0 1 8.834 0a2.53 2.53 0 0 1 2.521 2.522v2.52zm0 1.271a2.53 2.53 0 0 1 2.521 2.521a2.53 2.53 0 0 1-2.521 2.521H2.522A2.53 2.53 0 0 1 0 8.834a2.53 2.53 0 0 1 2.522-2.521zm10.122 2.521a2.53 2.53 0 0 1 2.522-2.521A2.53 2.53 0 0 1 24 8.834a2.53 2.53 0 0 1-2.522 2.521h-2.522zm-1.268 0a2.53 2.53 0 0 1-2.523 2.521a2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.165 0a2.53 2.53 0 0 1 2.523 2.522zm-2.523 10.122a2.53 2.53 0 0 1 2.523 2.522A2.53 2.53 0 0 1 15.165 24a2.527 2.527 0 0 1-2.52-2.522v-2.522zm0-1.268a2.527 2.527 0 0 1-2.52-2.523a2.526 2.526 0 0 1 2.52-2.52h6.313A2.527 2.527 0 0 1 24 15.165a2.53 2.53 0 0 1-2.522 2.523z"/></svg></g> <!-- Slack -->
|
||||
<g transform="translate(954,911)"><rect width="22" height="22" rx="5" fill="#5865F2"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M20.317 4.37a19.8 19.8 0 0 0-4.885-1.515a.074.074 0 0 0-.079.037c-.21.375-.444.864-.608 1.25a18.3 18.3 0 0 0-5.487 0a13 13 0 0 0-.617-1.25a.08.08 0 0 0-.079-.037A19.7 19.7 0 0 0 3.677 4.37a.1.1 0 0 0-.032.027C.533 9.046-.32 13.58.099 18.057a.08.08 0 0 0 .031.057a19.9 19.9 0 0 0 5.993 3.03a.08.08 0 0 0 .084-.028a14 14 0 0 0 1.226-1.994a.076.076 0 0 0-.041-.106a13 13 0 0 1-1.872-.892a.077.077 0 0 1-.008-.128a10 10 0 0 0 .372-.292a.07.07 0 0 1 .077-.01c3.928 1.793 8.18 1.793 12.062 0a.07.07 0 0 1 .078.01q.181.149.373.292a.077.077 0 0 1-.006.127a12.3 12.3 0 0 1-1.873.892a.077.077 0 0 0-.041.107c.36.698.772 1.362 1.225 1.993a.08.08 0 0 0 .084.028a19.8 19.8 0 0 0 6.002-3.03a.08.08 0 0 0 .032-.054c.5-5.177-.838-9.674-3.549-13.66a.06.06 0 0 0-.031-.03M8.02 15.33c-1.182 0-2.157-1.085-2.157-2.419c0-1.333.956-2.419 2.157-2.419c1.21 0 2.176 1.096 2.157 2.42c0 1.333-.956 2.418-2.157 2.418m7.975 0c-1.183 0-2.157-1.085-2.157-2.419c0-1.333.955-2.419 2.157-2.419c1.21 0 2.176 1.096 2.157 2.42c0 1.333-.946 2.418-2.157 2.418"/></svg></g> <!-- Discord -->
|
||||
<g transform="translate(804,939)"><rect width="22" height="22" rx="5" fill="#26A5E4"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M11.944 0A12 12 0 0 0 0 12a12 12 0 0 0 12 12a12 12 0 0 0 12-12A12 12 0 0 0 12 0zm4.962 7.224c.1-.002.321.023.465.14a.5.5 0 0 1 .171.325c.016.093.036.306.02.472c-.18 1.898-.962 6.502-1.36 8.627c-.168.9-.499 1.201-.82 1.23c-.696.065-1.225-.46-1.9-.902c-1.056-.693-1.653-1.124-2.678-1.8c-1.185-.78-.417-1.21.258-1.91c.177-.184 3.247-2.977 3.307-3.23c.007-.032.014-.15-.056-.212s-.174-.041-.249-.024q-.159.037-5.061 3.345q-.72.495-1.302.48c-.428-.008-1.252-.241-1.865-.44c-.752-.245-1.349-.374-1.297-.789q.04-.324.893-.663q5.247-2.286 6.998-3.014c3.332-1.386 4.025-1.627 4.476-1.635"/></svg></g> <!-- Telegram -->
|
||||
<g transform="translate(834,939)"><rect width="22" height="22" rx="5" fill="#181717"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M12 .297c-6.63 0-12 5.373-12 12c0 5.303 3.438 9.8 8.205 11.385c.6.113.82-.258.82-.577c0-.285-.01-1.04-.015-2.04c-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729c1.205.084 1.838 1.236 1.838 1.236c1.07 1.835 2.809 1.305 3.495.998c.108-.776.417-1.305.76-1.605c-2.665-.3-5.466-1.332-5.466-5.93c0-1.31.465-2.38 1.235-3.22c-.135-.303-.54-1.523.105-3.176c0 0 1.005-.322 3.3 1.23c.96-.267 1.98-.399 3-.405c1.02.006 2.04.138 3 .405c2.28-1.552 3.285-1.23 3.285-1.23c.645 1.653.24 2.873.12 3.176c.765.84 1.23 1.91 1.23 3.22c0 4.61-2.805 5.625-5.475 5.92c.42.36.81 1.096.81 2.22c0 1.606-.015 2.896-.015 3.286c0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12"/></svg></g> <!-- GitHub -->
|
||||
<g transform="translate(864,939)"><rect width="22" height="22" rx="5" fill="#5E6AD2"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M2.886 4.18A11.98 11.98 0 0 1 11.99 0C18.624 0 24 5.376 24 12.009c0 3.64-1.62 6.903-4.18 9.105L2.887 4.18ZM1.817 5.626l16.556 16.556q-.787.496-1.65.866L.951 7.277q.371-.863.866-1.65ZM.322 9.163l14.515 14.515q-1.066.26-2.195.322L0 11.358a12 12 0 0 1 .322-2.195m-.17 4.862l9.823 9.824a12.02 12.02 0 0 1-9.824-9.824Z"/></svg></g> <!-- Linear -->
|
||||
<g transform="translate(894,939)"><rect width="22" height="22" rx="5" fill="#000000"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M4.459 4.208c.746.606 1.026.56 2.428.466l13.215-.793c.28 0 .047-.28-.046-.326L17.86 1.968c-.42-.326-.981-.7-2.055-.607L3.01 2.295c-.466.046-.56.28-.374.466zm.793 3.08v13.904c0 .747.373 1.027 1.214.98l14.523-.84c.841-.046.935-.56.935-1.167V6.354c0-.606-.233-.933-.748-.887l-15.177.887c-.56.047-.747.327-.747.933zm14.337.745c.093.42 0 .84-.42.888l-.7.14v10.264c-.608.327-1.168.514-1.635.514c-.748 0-.935-.234-1.495-.933l-4.577-7.186v6.952L12.21 19s0 .84-1.168.84l-3.222.186c-.093-.186 0-.653.327-.746l.84-.233V9.854L7.822 9.76c-.094-.42.14-1.026.793-1.073l3.456-.233l4.764 7.279v-6.44l-1.215-.139c-.093-.514.28-.887.747-.933zM1.936 1.035l13.31-.98c1.634-.14 2.055-.047 3.082.7l4.249 2.986c.7.513.934.653.934 1.213v16.378c0 1.026-.373 1.634-1.68 1.726l-15.458.934c-.98.047-1.448-.093-1.962-.747l-3.129-4.06c-.56-.747-.793-1.306-.793-1.96V2.667c0-.839.374-1.54 1.447-1.632"/></svg></g> <!-- Notion -->
|
||||
<g transform="translate(924,939)"><rect width="22" height="22" rx="5" fill="#47A248"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M17.193 9.555c-1.264-5.58-4.252-7.414-4.573-8.115c-.28-.394-.53-.954-.735-1.44c-.036.495-.055.685-.523 1.184c-.723.566-4.438 3.682-4.74 10.02c-.282 5.912 4.27 9.435 4.888 9.884l.07.05A74 74 0 0 1 11.91 24h.481a29 29 0 0 1 .51-3.07c.417-.296.604-.463.85-.693a11.34 11.34 0 0 0 3.639-8.464c.01-.814-.103-1.662-.197-2.218m-5.336 8.195s0-8.291.275-8.29c.213 0 .49 10.695.49 10.695c-.381-.045-.765-1.76-.765-2.405"/></svg></g> <!-- MongoDB -->
|
||||
<text x="965" y="957" text-anchor="middle" font-size="16" font-weight="700" fill="#A1A1AA" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">…</text>
|
||||
|
||||
<!-- ===================== CROSS-LAYER ARROWS ===================== -->
|
||||
<!-- AI Agent -> Mirage Shell (vertical) -->
|
||||
<line x1="470" y1="262" x2="470" y2="409" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
<rect x="176" y="282" width="282" height="28" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1"/>
|
||||
<text x="450" y="302" text-anchor="end" font-size="18" font-family="'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace" fill="#A1A1AA">grep alert /s3/log.jsonl</text>
|
||||
|
||||
<!-- AI Agent -> Mirage VFS (vertical) -->
|
||||
<line x1="680" y1="262" x2="680" y2="409" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
<rect x="490" y="282" width="180" height="28" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1"/>
|
||||
<text x="662" y="302" text-anchor="end" font-size="18" font-family="'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace" fill="#A1A1AA">stat /paper.pdf</text>
|
||||
|
||||
<!-- Application & CLI -> FUSE (vertical) -->
|
||||
<line x1="890" y1="262" x2="890" y2="409" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
<rect x="902" y="282" width="200" height="28" rx="4" fill="#0E0E10" stroke="#FFFFFF" stroke-width="1"/>
|
||||
<text x="910" y="302" font-size="18" font-family="'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace" fill="#A1A1AA">read /r2/data.csv</text>
|
||||
|
||||
<!-- Command Registry -> Dispatcher (vertical) -->
|
||||
<line x1="470" y1="543" x2="470" y2="666" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- VFS Registry -> Dispatcher (vertical) -->
|
||||
<line x1="680" y1="543" x2="680" y2="666" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Dispatcher -> Infrastructure (vertical, aligned with CR -> Dispatcher connector at x=470) -->
|
||||
<line x1="470" y1="718" x2="470" y2="833" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Dispatcher -> Remote Services (start x=680 matches PR -> Dispatcher; end x=890 matches Cache -> RS) -->
|
||||
<line x1="680" y1="718" x2="890" y2="833" stroke="#A1A1AA" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Cache - Remote Services (cache-hit short-circuit; dotted, no arrowhead) -->
|
||||
<line x1="890" y1="718" x2="890" y2="833" stroke="#A1A1AA" stroke-width="1.2" stroke-dasharray="4 3"/>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 26 KiB |
@@ -0,0 +1,181 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 1010" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif">
|
||||
<defs>
|
||||
<style type="text/css"><![CDATA[
|
||||
@import url('https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@500;600;700;800&display=swap');
|
||||
.display {
|
||||
font-family: 'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace;
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
]]></style>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
|
||||
<path d="M0 0 L10 5 L0 10 z" fill="#27272A"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<rect width="1200" height="1010" fill="#FFFFFF"/>
|
||||
|
||||
<!-- Title: mirage icon (cube) + "mirage" wordmark in diagram font -->
|
||||
<svg x="407" y="4" width="96" height="110" viewBox="16 7 128 146">
|
||||
<polygon points="19,45 80,10 80,45 50,63" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<polygon points="80,10 141,45 110,63 80,45" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<polygon points="141,45 141,115 110,98 110,63" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<polygon points="141,115 80,150 80,115 110,98" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<polygon points="80,150 19,115 50,98 80,115" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<polygon points="19,115 19,45 50,63 50,98" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<polygon points="80,45 110,63 110,98 80,115 50,98 50,63" fill="#000000" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<line x1="80" y1="80" x2="50" y2="63" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<line x1="80" y1="80" x2="110" y2="63" stroke="#ffffff" stroke-width="2.5"/>
|
||||
<line x1="80" y1="80" x2="80" y2="115" stroke="#ffffff" stroke-width="2.5"/>
|
||||
</svg>
|
||||
<text x="515" y="84" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif" font-size="72" font-weight="700" letter-spacing="-1">mirage</text>
|
||||
<text x="600" y="140" text-anchor="middle" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif" font-size="22" font-weight="500" letter-spacing="0.4">A Unified Virtual Filesystem for AI Agents</text>
|
||||
|
||||
<!-- ===================== LAYER 1: AGENT ===================== -->
|
||||
<rect x="20" y="160" width="1160" height="170" rx="8" fill="#FAF9F6" stroke="#000000" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="198" font-size="22" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">AI Agent and Application</text>
|
||||
<text x="44" y="228" font-size="18" fill="#27272A">
|
||||
<tspan x="44" dy="0">Bash commands, VFS</tspan>
|
||||
<tspan x="44" dy="24">calls, or syscalls</tspan>
|
||||
</text>
|
||||
|
||||
<!-- AI Agent (centered over connectors at x=470 and x=680: equal 30px margins on each side) -->
|
||||
<rect x="440" y="210" width="270" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="575" y="242" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">AI Agent</text>
|
||||
|
||||
<!-- OS Processes (positioned over FUSE Adapter x=890 for vertical connector) -->
|
||||
<rect x="791" y="210" width="198" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="890" y="242" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">App & CLI</text>
|
||||
|
||||
|
||||
<!-- ===================== LAYER 2: WORKSPACE ===================== -->
|
||||
<rect x="20" y="376" width="1160" height="200" rx="8" fill="#FAF9F6" stroke="#000000" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="414" font-size="22" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage Bash and VFS</text>
|
||||
<text x="44" y="444" font-size="18" fill="#27272A">
|
||||
<tspan x="44" dy="0">Mirage Bash &</tspan>
|
||||
<tspan x="44" dy="24">registered commands;</tspan>
|
||||
<tspan x="44" dy="24">Mirage VFS &</tspan>
|
||||
<tspan x="44" dy="24">registered VFS ops</tspan>
|
||||
</text>
|
||||
|
||||
<!-- Mirage Shell -->
|
||||
<rect x="376" y="409" width="188" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="470" y="441" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage Bash</text>
|
||||
|
||||
<!-- Mirage VFS -->
|
||||
<rect x="596" y="409" width="168" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="680" y="441" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage VFS</text>
|
||||
|
||||
<!-- FUSE Adapter -->
|
||||
<rect x="796" y="409" width="188" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="890" y="441" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">FUSE Adapter</text>
|
||||
|
||||
<!-- Command Registry -->
|
||||
<rect x="376" y="491" width="188" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="470" y="523" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Command Registry</text>
|
||||
|
||||
<!-- VFS Registry -->
|
||||
<rect x="596" y="491" width="168" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="680" y="523" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">VFS Registry</text>
|
||||
|
||||
<!-- Shell -> Command Registry -->
|
||||
<line x1="470" y1="461" x2="470" y2="491" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Mirage VFS -> VFS Registry -->
|
||||
<line x1="680" y1="461" x2="680" y2="491" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- FUSE -> Mirage VFS -->
|
||||
<line x1="796" y1="435" x2="764" y2="435" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- ===================== LAYER 3: DISPATCHER & CACHE ===================== -->
|
||||
<rect x="20" y="622" width="1160" height="140" rx="8" fill="#FAF9F6" stroke="#000000" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="660" font-size="22" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Dispatcher & Cache</text>
|
||||
<text x="44" y="690" font-size="18" fill="#27272A">
|
||||
<tspan x="44" dy="0">Dispatches by mount;</tspan>
|
||||
<tspan x="44" dy="24">caches repeats</tspan>
|
||||
</text>
|
||||
|
||||
<!-- Mirage Dispatcher (matches AI Agent dimensions: w=270, h=52, x=440) -->
|
||||
<rect x="440" y="666" width="270" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="575" y="698" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Mirage Dispatcher</text>
|
||||
|
||||
<!-- Index & File Cache (merged; aligned x with Application & CLI: x=800, w=180) -->
|
||||
<rect x="791" y="666" width="198" height="52" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="890" y="698" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Index & File Cache</text>
|
||||
|
||||
<!-- Dispatcher -> Cache -->
|
||||
<line x1="710" y1="692" x2="791" y2="692" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- ===================== LAYER 4: BACKEND ===================== -->
|
||||
<rect x="20" y="808" width="1160" height="190" rx="8" fill="#FAF9F6" stroke="#000000" stroke-width="1" stroke-dasharray="6 4"/>
|
||||
<text x="44" y="846" font-size="22" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Infrastructure and Remote</text>
|
||||
<text x="44" y="876" font-size="18" fill="#27272A">
|
||||
<tspan x="44" dy="0">Infrastructure & remote,</tspan>
|
||||
<tspan x="44" dy="24">mounted as one tree</tspan>
|
||||
</text>
|
||||
|
||||
<!-- Infrastructure (renamed from Local Backends; aligned x with Mirage Shell at 380, w=180) -->
|
||||
<rect x="380" y="833" width="180" height="140" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="470" y="861" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Infrastructure</text>
|
||||
<line x1="420" y1="873" x2="520" y2="873" stroke="#E4E4E7" stroke-width="1"/>
|
||||
<text x="470" y="897" text-anchor="middle" font-size="15" fill="#27272A">RAM</text>
|
||||
<text x="470" y="919" text-anchor="middle" font-size="15" fill="#27272A">Disk</text>
|
||||
<text x="470" y="941" text-anchor="middle" font-size="15" fill="#27272A">OPFS</text>
|
||||
<text x="470" y="963" text-anchor="middle" font-size="15" fill="#27272A">Redis</text>
|
||||
|
||||
<!-- Remote Services (widened by 10px each side: x=790, w=200; center stays at 890) -->
|
||||
<rect x="790" y="833" width="200" height="140" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1.2"/>
|
||||
<text x="890" y="861" text-anchor="middle" font-size="19" font-weight="500" fill="#000000" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">Remote Service</text>
|
||||
<line x1="830" y1="873" x2="950" y2="873" stroke="#E4E4E7" stroke-width="1"/>
|
||||
|
||||
<!-- Icon grid: 6 per row, 22x22 colored rect with inlined simple-icons brand SVG -->
|
||||
<g transform="translate(804,883)"><rect width="22" height="22" rx="5" fill="#FF9900"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="m20.913 13.147l.12-.895c.947.576 1.258.922 1.354 1.071c-.16.031-.562.046-1.474-.176m-2.174 7.988l-.005.073c0 .084-.207.405-1.124.768a10 10 0 0 1-1.438.432c-1.405.325-3.128.504-4.853.504c-4.612 0-7.412-1.184-7.412-1.704l-.005-.073L1.81 5.602q.203.117.432.227q.064.03.128.057q.2.093.417.18l.179.069q.232.087.478.168l.13.043q.311.097.646.187l.176.044q.263.066.534.127a23 23 0 0 0 .843.17l.121.023q.378.067.768.122q.107.016.216.03q.299.04.604.077l.24.027q.366.04.74.07l.081.009q.413.033.83.056l.233.012q.316.015.633.025a33 33 0 0 0 2.795-.026l.232-.011q.417-.023.83-.056l.08-.008q.375-.03.742-.072l.238-.026q.307-.036.609-.077l.211-.03q.392-.056.772-.122l.111-.02q.323-.06.634-.125l.212-.047q.279-.062.546-.13l.166-.042q.338-.09.654-.189l.115-.038a11 11 0 0 0 .493-.173q.087-.032.17-.066q.225-.089.43-.185q.059-.025.116-.052q.23-.11.436-.228l-.976 7.245c-2.488-.78-5.805-2.292-7.311-3a1.09 1.09 0 0 0-1.088-1.085c-.6 0-1.088.489-1.088 1.088s.488 1.089 1.088 1.089c.196 0 .378-.056.537-.148c1.72.812 5.144 2.367 7.715 3.15zm-7.42-20.047c5.677 0 9.676 1.759 9.75 2.736l-.014.113c-.01.033-.031.067-.048.101c-.015.028-.026.057-.047.087c-.024.033-.058.068-.09.102c-.028.03-.051.06-.084.09c-.038.035-.087.07-.133.105c-.04.03-.074.06-.119.091c-.053.036-.116.071-.177.107c-.05.03-.095.06-.15.09c-.068.036-.147.073-.222.11c-.059.028-.114.057-.177.085c-.084.038-.177.074-.268.111c-.068.027-.13.054-.203.082c-.097.036-.205.072-.31.107c-.075.026-.148.053-.228.079c-.111.035-.233.069-.35.103c-.085.024-.165.05-.253.073c-.124.034-.258.065-.389.098c-.093.022-.181.046-.278.068c-.139.032-.287.061-.433.091c-.098.02-.191.041-.293.06c-.155.03-.32.057-.482.084c-.1.018-.198.036-.302.052c-.166.026-.342.048-.515.072c-.11.014-.213.03-.325.044c-.181.023-.372.041-.56.06q-.163.019-.332.036c-.188.016-.386.029-.58.043c-.122.009-.24.02-.364.028c-.207.012-.422.02-.635.028c-.12.005-.234.012-.354.016a36 36 0 0 1-2.069 0c-.12-.004-.234-.011-.352-.016c-.214-.008-.43-.016-.637-.028c-.122-.008-.238-.02-.36-.027c-.195-.015-.394-.028-.584-.044c-.11-.01-.215-.024-.324-.035c-.19-.02-.384-.038-.568-.06l-.315-.044c-.176-.024-.355-.046-.525-.073c-.1-.015-.192-.033-.29-.05c-.167-.028-.335-.055-.494-.086c-.096-.018-.183-.038-.276-.056c-.151-.032-.305-.062-.45-.095c-.09-.02-.173-.043-.26-.064c-.138-.034-.277-.067-.407-.102c-.082-.022-.157-.046-.235-.069a12 12 0 0 1-.368-.108c-.075-.024-.141-.049-.213-.073c-.11-.037-.223-.075-.325-.113c-.067-.025-.125-.051-.188-.077c-.096-.038-.195-.076-.282-.115c-.06-.027-.11-.054-.166-.08c-.08-.039-.162-.077-.233-.116c-.052-.028-.094-.055-.142-.084c-.063-.038-.13-.075-.185-.113c-.043-.029-.075-.058-.113-.086c-.048-.037-.098-.073-.139-.11c-.032-.029-.054-.057-.08-.087c-.033-.035-.069-.07-.093-.104c-.02-.03-.031-.058-.046-.086c-.018-.035-.039-.068-.049-.102l-.015-.113c.076-.977 4.074-2.736 9.748-2.736m12.182 12.124c-.118-.628-.84-1.291-2.31-2.128l.963-7.16l.005-.073C22.16 1.581 16.447 0 11.32 0C6.194 0 .482 1.581.482 3.851l.005.072L2.819 21.25c.071 2.002 5.236 2.75 8.5 2.75c1.805 0 3.615-.188 5.098-.531c.598-.138 1.133-.3 1.592-.48c1.18-.467 1.789-1.053 1.813-1.739l.945-7.018c.557.131 1.016.197 1.389.197c.54 0 .902-.137 1.134-.413a.96.96 0 0 0 .21-.804Z"/></svg></g> <!-- S3 -->
|
||||
<g transform="translate(834,883)"><rect width="22" height="22" rx="5" fill="#F38020"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M16.509 16.845c.147-.507.09-.971-.155-1.316c-.225-.316-.605-.499-1.062-.52l-8.66-.113a.16.16 0 0 1-.133-.07a.2.2 0 0 1-.02-.156a.24.24 0 0 1 .203-.156l8.736-.113c1.035-.049 2.16-.886 2.554-1.913l.499-1.302a.27.27 0 0 0 .014-.168a5.689 5.689 0 0 0-10.937-.584a2.58 2.58 0 0 0-1.794-.498a2.56 2.56 0 0 0-2.223 3.18A3.634 3.634 0 0 0 0 16.751q.002.264.035.527a.174.174 0 0 0 .17.148h15.98a.22.22 0 0 0 .204-.155zm2.757-5.564c-.077 0-.161 0-.239.011c-.056 0-.105.042-.127.098l-.337 1.174c-.148.507-.092.971.154 1.317c.225.316.605.498 1.062.52l1.844.113c.056 0 .105.026.133.07a.2.2 0 0 1 .021.156a.24.24 0 0 1-.204.156l-1.92.112c-1.042.049-2.159.887-2.553 1.914l-.141.358c-.028.072.021.142.099.142h6.597a.174.174 0 0 0 .17-.126a5 5 0 0 0 .175-1.28a4.74 4.74 0 0 0-4.734-4.727"/></svg></g> <!-- R2 (Cloudflare) -->
|
||||
<g transform="translate(864,883)"><rect width="22" height="22" rx="5" fill="#C74634"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M16.412 4.412h-8.82a7.588 7.588 0 0 0-.008 15.176h8.828a7.588 7.588 0 0 0 0-15.176m-.193 12.502H7.786a4.915 4.915 0 0 1 0-9.828h8.433a4.914 4.914 0 1 1 0 9.828"/></svg></g> <!-- OCI -->
|
||||
<g transform="translate(894,883)"><rect width="22" height="22" rx="5" fill="#4285F4"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M12.19 2.38a9.344 9.344 0 0 0-9.234 6.893c.053-.02-.055.013 0 0c-3.875 2.551-3.922 8.11-.247 10.941l.006-.007l-.007.03a6.7 6.7 0 0 0 4.077 1.356h5.173l.03.03h5.192c6.687.053 9.376-8.605 3.835-12.35a9.37 9.37 0 0 0-2.821-4.552l-.043.043l.006-.05A9.34 9.34 0 0 0 12.19 2.38m-.358 4.146c1.244-.04 2.518.368 3.486 1.15a5.19 5.19 0 0 1 1.862 4.078v.518c3.53-.07 3.53 5.262 0 5.193h-5.193l-.008.009v-.04H6.785a2.6 2.6 0 0 1-1.067-.23h.001a2.597 2.597 0 1 1 3.437-3.437l3.013-3.012A6.75 6.75 0 0 0 8.11 8.24c.018-.01.04-.026.054-.023a5.2 5.2 0 0 1 3.67-1.69z"/></svg></g> <!-- GCS -->
|
||||
<g transform="translate(924,883)"><rect width="22" height="22" rx="5" fill="#3FCF8E"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M11.9 1.036c-.015-.986-1.26-1.41-1.874-.637L.764 12.05C-.33 13.427.65 15.455 2.409 15.455h9.579l.113 7.51c.014.985 1.259 1.408 1.873.636l9.262-11.653c1.093-1.375.113-3.403-1.645-3.403h-9.642z"/></svg></g> <!-- Supabase -->
|
||||
<g transform="translate(954,883)"><rect width="22" height="22" rx="5" fill="#EA4335"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M24 5.457v13.909c0 .904-.732 1.636-1.636 1.636h-3.819V11.73L12 16.64l-6.545-4.91v9.273H1.636A1.636 1.636 0 0 1 0 19.366V5.457c0-2.023 2.309-3.178 3.927-1.964L5.455 4.64L12 9.548l6.545-4.91l1.528-1.145C21.69 2.28 24 3.434 24 5.457"/></svg></g> <!-- Gmail -->
|
||||
<g transform="translate(804,911)"><rect width="22" height="22" rx="5" fill="#1FA463"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M12.01 1.485c-2.082 0-3.754.02-3.743.047c.01.02 1.708 3.001 3.774 6.62l3.76 6.574h3.76c2.081 0 3.753-.02 3.742-.047c-.005-.02-1.708-3.001-3.775-6.62l-3.76-6.574zm-4.76 1.73a789.828 789.861 0 0 0-3.63 6.319L0 15.868l1.89 3.298l1.885 3.297l3.62-6.335l3.618-6.33l-1.88-3.287C8.1 4.704 7.255 3.22 7.25 3.214zm2.259 12.653l-.203.348c-.114.198-.96 1.672-1.88 3.287a423.93 423.948 0 0 1-1.698 2.97c-.01.026 3.24.042 7.222.042h7.244l1.796-3.157c.992-1.734 1.85-3.23 1.906-3.323l.104-.167h-7.249z"/></svg></g> <!-- GDrive -->
|
||||
<g transform="translate(834,911)"><rect width="22" height="22" rx="5" fill="#4285F4"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M14.727 6.727H14V0H4.91c-.905 0-1.637.732-1.637 1.636v20.728c0 .904.732 1.636 1.636 1.636h14.182c.904 0 1.636-.732 1.636-1.636V6.727zm-.545 10.455H7.09v-1.364h7.09v1.364zm2.727-3.273H7.091v-1.364h9.818zm0-3.273H7.091V9.273h9.818zM14.727 6h6l-6-6z"/></svg></g> <!-- GDocs -->
|
||||
<g transform="translate(864,911)"><rect width="22" height="22" rx="5" fill="#34A853"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M11.318 12.545H7.91v-1.909h3.41v1.91zM14.728 0v6h6zm1.363 10.636h-3.41v1.91h3.41zm0 3.273h-3.41v1.91h3.41zM20.727 6.5v15.864c0 .904-.732 1.636-1.636 1.636H4.909a1.636 1.636 0 0 1-1.636-1.636V1.636C3.273.732 4.005 0 4.909 0h9.318v6.5zm-3.273 2.773H6.545v7.909h10.91v-7.91zm-6.136 4.636H7.91v1.91h3.41v-1.91z"/></svg></g> <!-- GSheets -->
|
||||
<g transform="translate(894,911)"><rect width="22" height="22" rx="5" fill="#FBBC04"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M16.09 15.273H7.91v-4.637h8.18zm1.728-8.523h2.91v15.614c0 .904-.733 1.636-1.637 1.636H4.909a1.636 1.636 0 0 1-1.636-1.636V1.636C3.273.732 4.005 0 4.909 0h9.068v6.75zm-.363 2.523H6.545v7.363h10.91zm-2.728-5.979V6h6.001l-6-6v3.294z"/></svg></g> <!-- GSlides -->
|
||||
<g transform="translate(924,911)"><rect width="22" height="22" rx="5" fill="#4A154B"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M5.042 15.165a2.53 2.53 0 0 1-2.52 2.523A2.53 2.53 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52zm1.271 0a2.527 2.527 0 0 1 2.521-2.52a2.527 2.527 0 0 1 2.521 2.52v6.313A2.53 2.53 0 0 1 8.834 24a2.53 2.53 0 0 1-2.521-2.522zM8.834 5.042a2.53 2.53 0 0 1-2.521-2.52A2.53 2.53 0 0 1 8.834 0a2.53 2.53 0 0 1 2.521 2.522v2.52zm0 1.271a2.53 2.53 0 0 1 2.521 2.521a2.53 2.53 0 0 1-2.521 2.521H2.522A2.53 2.53 0 0 1 0 8.834a2.53 2.53 0 0 1 2.522-2.521zm10.122 2.521a2.53 2.53 0 0 1 2.522-2.521A2.53 2.53 0 0 1 24 8.834a2.53 2.53 0 0 1-2.522 2.521h-2.522zm-1.268 0a2.53 2.53 0 0 1-2.523 2.521a2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.165 0a2.53 2.53 0 0 1 2.523 2.522zm-2.523 10.122a2.53 2.53 0 0 1 2.523 2.522A2.53 2.53 0 0 1 15.165 24a2.527 2.527 0 0 1-2.52-2.522v-2.522zm0-1.268a2.527 2.527 0 0 1-2.52-2.523a2.526 2.526 0 0 1 2.52-2.52h6.313A2.527 2.527 0 0 1 24 15.165a2.53 2.53 0 0 1-2.522 2.523z"/></svg></g> <!-- Slack -->
|
||||
<g transform="translate(954,911)"><rect width="22" height="22" rx="5" fill="#5865F2"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M20.317 4.37a19.8 19.8 0 0 0-4.885-1.515a.074.074 0 0 0-.079.037c-.21.375-.444.864-.608 1.25a18.3 18.3 0 0 0-5.487 0a13 13 0 0 0-.617-1.25a.08.08 0 0 0-.079-.037A19.7 19.7 0 0 0 3.677 4.37a.1.1 0 0 0-.032.027C.533 9.046-.32 13.58.099 18.057a.08.08 0 0 0 .031.057a19.9 19.9 0 0 0 5.993 3.03a.08.08 0 0 0 .084-.028a14 14 0 0 0 1.226-1.994a.076.076 0 0 0-.041-.106a13 13 0 0 1-1.872-.892a.077.077 0 0 1-.008-.128a10 10 0 0 0 .372-.292a.07.07 0 0 1 .077-.01c3.928 1.793 8.18 1.793 12.062 0a.07.07 0 0 1 .078.01q.181.149.373.292a.077.077 0 0 1-.006.127a12.3 12.3 0 0 1-1.873.892a.077.077 0 0 0-.041.107c.36.698.772 1.362 1.225 1.993a.08.08 0 0 0 .084.028a19.8 19.8 0 0 0 6.002-3.03a.08.08 0 0 0 .032-.054c.5-5.177-.838-9.674-3.549-13.66a.06.06 0 0 0-.031-.03M8.02 15.33c-1.182 0-2.157-1.085-2.157-2.419c0-1.333.956-2.419 2.157-2.419c1.21 0 2.176 1.096 2.157 2.42c0 1.333-.956 2.418-2.157 2.418m7.975 0c-1.183 0-2.157-1.085-2.157-2.419c0-1.333.955-2.419 2.157-2.419c1.21 0 2.176 1.096 2.157 2.42c0 1.333-.946 2.418-2.157 2.418"/></svg></g> <!-- Discord -->
|
||||
<g transform="translate(804,939)"><rect width="22" height="22" rx="5" fill="#26A5E4"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M11.944 0A12 12 0 0 0 0 12a12 12 0 0 0 12 12a12 12 0 0 0 12-12A12 12 0 0 0 12 0zm4.962 7.224c.1-.002.321.023.465.14a.5.5 0 0 1 .171.325c.016.093.036.306.02.472c-.18 1.898-.962 6.502-1.36 8.627c-.168.9-.499 1.201-.82 1.23c-.696.065-1.225-.46-1.9-.902c-1.056-.693-1.653-1.124-2.678-1.8c-1.185-.78-.417-1.21.258-1.91c.177-.184 3.247-2.977 3.307-3.23c.007-.032.014-.15-.056-.212s-.174-.041-.249-.024q-.159.037-5.061 3.345q-.72.495-1.302.48c-.428-.008-1.252-.241-1.865-.44c-.752-.245-1.349-.374-1.297-.789q.04-.324.893-.663q5.247-2.286 6.998-3.014c3.332-1.386 4.025-1.627 4.476-1.635"/></svg></g> <!-- Telegram -->
|
||||
<g transform="translate(834,939)"><rect width="22" height="22" rx="5" fill="#181717"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M12 .297c-6.63 0-12 5.373-12 12c0 5.303 3.438 9.8 8.205 11.385c.6.113.82-.258.82-.577c0-.285-.01-1.04-.015-2.04c-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729c1.205.084 1.838 1.236 1.838 1.236c1.07 1.835 2.809 1.305 3.495.998c.108-.776.417-1.305.76-1.605c-2.665-.3-5.466-1.332-5.466-5.93c0-1.31.465-2.38 1.235-3.22c-.135-.303-.54-1.523.105-3.176c0 0 1.005-.322 3.3 1.23c.96-.267 1.98-.399 3-.405c1.02.006 2.04.138 3 .405c2.28-1.552 3.285-1.23 3.285-1.23c.645 1.653.24 2.873.12 3.176c.765.84 1.23 1.91 1.23 3.22c0 4.61-2.805 5.625-5.475 5.92c.42.36.81 1.096.81 2.22c0 1.606-.015 2.896-.015 3.286c0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12"/></svg></g> <!-- GitHub -->
|
||||
<g transform="translate(864,939)"><rect width="22" height="22" rx="5" fill="#5E6AD2"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M2.886 4.18A11.98 11.98 0 0 1 11.99 0C18.624 0 24 5.376 24 12.009c0 3.64-1.62 6.903-4.18 9.105L2.887 4.18ZM1.817 5.626l16.556 16.556q-.787.496-1.65.866L.951 7.277q.371-.863.866-1.65ZM.322 9.163l14.515 14.515q-1.066.26-2.195.322L0 11.358a12 12 0 0 1 .322-2.195m-.17 4.862l9.823 9.824a12.02 12.02 0 0 1-9.824-9.824Z"/></svg></g> <!-- Linear -->
|
||||
<g transform="translate(894,939)"><rect width="22" height="22" rx="5" fill="#000000"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M4.459 4.208c.746.606 1.026.56 2.428.466l13.215-.793c.28 0 .047-.28-.046-.326L17.86 1.968c-.42-.326-.981-.7-2.055-.607L3.01 2.295c-.466.046-.56.28-.374.466zm.793 3.08v13.904c0 .747.373 1.027 1.214.98l14.523-.84c.841-.046.935-.56.935-1.167V6.354c0-.606-.233-.933-.748-.887l-15.177.887c-.56.047-.747.327-.747.933zm14.337.745c.093.42 0 .84-.42.888l-.7.14v10.264c-.608.327-1.168.514-1.635.514c-.748 0-.935-.234-1.495-.933l-4.577-7.186v6.952L12.21 19s0 .84-1.168.84l-3.222.186c-.093-.186 0-.653.327-.746l.84-.233V9.854L7.822 9.76c-.094-.42.14-1.026.793-1.073l3.456-.233l4.764 7.279v-6.44l-1.215-.139c-.093-.514.28-.887.747-.933zM1.936 1.035l13.31-.98c1.634-.14 2.055-.047 3.082.7l4.249 2.986c.7.513.934.653.934 1.213v16.378c0 1.026-.373 1.634-1.68 1.726l-15.458.934c-.98.047-1.448-.093-1.962-.747l-3.129-4.06c-.56-.747-.793-1.306-.793-1.96V2.667c0-.839.374-1.54 1.447-1.632"/></svg></g> <!-- Notion -->
|
||||
<g transform="translate(924,939)"><rect width="22" height="22" rx="5" fill="#47A248"/><svg x="3" y="3" width="16" height="16" viewBox="0 0 24 24"><path fill="#FFFFFF" d="M17.193 9.555c-1.264-5.58-4.252-7.414-4.573-8.115c-.28-.394-.53-.954-.735-1.44c-.036.495-.055.685-.523 1.184c-.723.566-4.438 3.682-4.74 10.02c-.282 5.912 4.27 9.435 4.888 9.884l.07.05A74 74 0 0 1 11.91 24h.481a29 29 0 0 1 .51-3.07c.417-.296.604-.463.85-.693a11.34 11.34 0 0 0 3.639-8.464c.01-.814-.103-1.662-.197-2.218m-5.336 8.195s0-8.291.275-8.29c.213 0 .49 10.695.49 10.695c-.381-.045-.765-1.76-.765-2.405"/></svg></g> <!-- MongoDB -->
|
||||
<text x="965" y="957" text-anchor="middle" font-size="16" font-weight="700" fill="#27272A" font-family="Avenir Next, Helvetica Neue, Arial, sans-serif">…</text>
|
||||
|
||||
<!-- ===================== CROSS-LAYER ARROWS ===================== -->
|
||||
<!-- AI Agent -> Mirage Shell (vertical) -->
|
||||
<line x1="470" y1="262" x2="470" y2="409" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
<rect x="176" y="282" width="282" height="28" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1"/>
|
||||
<text x="450" y="302" text-anchor="end" font-size="18" font-family="'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace" fill="#27272A">grep alert /s3/log.jsonl</text>
|
||||
|
||||
<!-- AI Agent -> Mirage VFS (vertical) -->
|
||||
<line x1="680" y1="262" x2="680" y2="409" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
<rect x="490" y="282" width="180" height="28" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1"/>
|
||||
<text x="662" y="302" text-anchor="end" font-size="18" font-family="'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace" fill="#27272A">stat /paper.pdf</text>
|
||||
|
||||
<!-- Application & CLI -> FUSE (vertical) -->
|
||||
<line x1="890" y1="262" x2="890" y2="409" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
<rect x="902" y="282" width="200" height="28" rx="4" fill="#FFFFFF" stroke="#000000" stroke-width="1"/>
|
||||
<text x="910" y="302" font-size="18" font-family="'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, monospace" fill="#27272A">read /r2/data.csv</text>
|
||||
|
||||
<!-- Command Registry -> Dispatcher (vertical) -->
|
||||
<line x1="470" y1="543" x2="470" y2="666" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- VFS Registry -> Dispatcher (vertical) -->
|
||||
<line x1="680" y1="543" x2="680" y2="666" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Dispatcher -> Infrastructure (vertical, aligned with CR -> Dispatcher connector at x=470) -->
|
||||
<line x1="470" y1="718" x2="470" y2="833" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Dispatcher -> Remote Services (start x=680 matches PR -> Dispatcher; end x=890 matches Cache -> RS) -->
|
||||
<line x1="680" y1="718" x2="890" y2="833" stroke="#27272A" stroke-width="1.2" marker-end="url(#arrow)"/>
|
||||
|
||||
<!-- Cache - Remote Services (cache-hit short-circuit; dotted, no arrowhead) -->
|
||||
<line x1="890" y1="718" x2="890" y2="833" stroke="#27272A" stroke-width="1.2" stroke-dasharray="4 3"/>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 26 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 69 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 70 KiB |
@@ -0,0 +1,90 @@
|
||||
# Command Conformance Spec
|
||||
|
||||
One declarative spec for command behavior, run by both the Python and
|
||||
TypeScript implementations (issue #151). Each case pins the exact bytes of
|
||||
stdout, the exact bytes of stderr, and the exit code — so trailing newlines,
|
||||
empty output, and binary output are all part of the contract.
|
||||
|
||||
## Runners
|
||||
|
||||
- Python: `python/tests/conformance/test_conformance.py` runs every case
|
||||
against `ram` and `disk`, plus `redis` when `REDIS_URL` is set. Runs as part
|
||||
of `uv run pytest`.
|
||||
- TypeScript: `typescript/packages/node/src/conformance.test.ts` runs every
|
||||
case against `ram`. Runs as part of `pnpm test`.
|
||||
|
||||
## Files
|
||||
|
||||
- `seeds.json` — files written into the workspace before every case, via the
|
||||
ops/fs write API (not shell commands). Values are `{"text": ...}` for UTF-8
|
||||
content or `{"base64": ...}` for binary content.
|
||||
- `cases/<command>.json` — cases for one command:
|
||||
|
||||
```json
|
||||
{
|
||||
"command": "wc",
|
||||
"cases": [
|
||||
{
|
||||
"id": "wc_default",
|
||||
"cmd": "wc /data/a.txt",
|
||||
"matrix": { "python": ["ram", "disk", "redis"], "typescript": ["ram"] },
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": " 5 5 24 /data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `stdout` / `stderr` use exactly one of `*_text` (UTF-8) or `*_base64`
|
||||
(arbitrary bytes). Trailing newlines are significant.
|
||||
- `stdin_text` / `stdin_base64` optionally feed stdin to the command.
|
||||
- `matrix` is explicit: a case runs only on the listed backends. Listing a
|
||||
backend is a claim of support — a missing command there is a failure, not a
|
||||
skip. A case whose matrix is empty is a load-time error in both runners.
|
||||
|
||||
## Policy
|
||||
|
||||
- One expected value per case. If a backend or language legitimately diverges,
|
||||
that divergence is triaged first: either it is a bug (fix the
|
||||
implementation) or it is intended semantics (document it and add an explicit
|
||||
per-backend override mechanism — not yet needed).
|
||||
- This spec is an acceptance/parity net, not a replacement for backend tests.
|
||||
API call counts, pushdown/fallback, cache invalidation, error injection, and
|
||||
concurrency stay in hand-written per-backend tests.
|
||||
- `spec/` (command definitions exported from the registries) is a different
|
||||
artifact: it describes the command *surface*; this folder describes command
|
||||
*behavior*.
|
||||
|
||||
## Adding a backend
|
||||
|
||||
Add a backend when its command behavior should be compared against the same
|
||||
expectations as the existing implementations:
|
||||
|
||||
1. Add its name to `SUPPORTED_MATRIX` in the Python and/or TypeScript runner.
|
||||
|
||||
1. Add runner setup that constructs a fresh workspace and resets backend state
|
||||
between cases.
|
||||
|
||||
1. For API-backed resources, use mocked clients with payloads shaped like real
|
||||
provider responses. Do not call live APIs from this suite.
|
||||
|
||||
1. Add the backend to the applicable case matrices:
|
||||
|
||||
```json
|
||||
"matrix": {
|
||||
"python": ["ram", "disk", "redis", "github"],
|
||||
"typescript": ["ram", "github"]
|
||||
}
|
||||
```
|
||||
|
||||
1. Run the focused Python and TypeScript conformance tests.
|
||||
|
||||
Both runners are discovered by the existing Python and TypeScript CI test jobs;
|
||||
no workflow change is required.
|
||||
|
||||
Only add a backend to a case when both should produce the same exact stdout,
|
||||
stderr, and exit code. Keep provider-specific API calls, cache behavior, error
|
||||
injection, and concurrency in backend tests.
|
||||
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"command": "cat",
|
||||
"cases": [
|
||||
{
|
||||
"id": "cat_no_trailing_newline",
|
||||
"cmd": "cat /data/no_nl.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "no trailing newline",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cat_empty",
|
||||
"cmd": "cat /data/empty.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cat_binary",
|
||||
"cmd": "cat /data/binary.bin",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_base64": "AAECaGVsbG///g==",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"command": "cmp",
|
||||
"cases": [
|
||||
{
|
||||
"id": "cmp_identical",
|
||||
"cmd": "cmp /data/same_a.txt /data/same_b.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cmp_differ",
|
||||
"cmd": "cmp /data/a.txt /data/b.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 1,
|
||||
"stdout_text": "/data/a.txt /data/b.txt differ: char 1, line 1\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"command": "du",
|
||||
"cases": [
|
||||
{
|
||||
"id": "du_file",
|
||||
"cmd": "du /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "24\t/data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "du_dir_summary",
|
||||
"cmd": "du -s /data/sub",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "20\t/data/sub\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"command": "file",
|
||||
"cases": [
|
||||
{
|
||||
"id": "file_text",
|
||||
"cmd": "file /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "/data/a.txt: text\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "file_binary",
|
||||
"cmd": "file /data/binary.bin",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "/data/binary.bin: binary\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"command": "find",
|
||||
"cases": [
|
||||
{
|
||||
"id": "find_subdir",
|
||||
"cmd": "find /data/sub",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "/data/sub\n/data/sub/deep\n/data/sub/deep/deeper.txt\n/data/sub/nested.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "find_name_glob",
|
||||
"cmd": "find /data/sub -name \"*.txt\"",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "/data/sub/deep/deeper.txt\n/data/sub/nested.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"command": "grep",
|
||||
"cases": [
|
||||
{
|
||||
"id": "grep_match",
|
||||
"cmd": "grep world /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "world\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "grep_no_match",
|
||||
"cmd": "grep zzz /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 1,
|
||||
"stdout_text": "",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"command": "head",
|
||||
"cases": [
|
||||
{
|
||||
"id": "head_bytes",
|
||||
"cmd": "head -c 5 /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "hello",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"command": "ls",
|
||||
"cases": [
|
||||
{
|
||||
"id": "ls_dir",
|
||||
"cmd": "ls /data",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "a.txt\nb.txt\nbinary.bin\nempty.txt\nno_nl.txt\nsame_a.txt\nsame_b.txt\nsub\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "ls_subdir",
|
||||
"cmd": "ls /data/sub",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "deep\nnested.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"command": "md5",
|
||||
"cases": [
|
||||
{
|
||||
"id": "md5_file",
|
||||
"cmd": "md5 /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "191c53e00627438f88824862472a3f59 /data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"command": "stat",
|
||||
"cases": [
|
||||
{
|
||||
"id": "stat_name_size",
|
||||
"cmd": "stat -c \"%n %s\" /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "/data/a.txt 24\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"command": "tree",
|
||||
"cases": [
|
||||
{
|
||||
"id": "tree_subdir",
|
||||
"cmd": "tree /data/sub",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "\u251c\u2500\u2500 deep\n\u2502 \u2514\u2500\u2500 deeper.txt\n\u2514\u2500\u2500 nested.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,140 @@
|
||||
{
|
||||
"command": "wc",
|
||||
"cases": [
|
||||
{
|
||||
"id": "wc_default",
|
||||
"cmd": "wc /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": " 5 5 24 /data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wc_lines",
|
||||
"cmd": "wc -l /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "5 /data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wc_words",
|
||||
"cmd": "wc -w /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "5 /data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wc_bytes",
|
||||
"cmd": "wc -c /data/a.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "24 /data/a.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wc_no_trailing_newline",
|
||||
"cmd": "wc /data/no_nl.txt",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": " 0 3 19 /data/no_nl.txt\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wc_stdin",
|
||||
"cmd": "wc",
|
||||
"stdin_text": "a\nb\n",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": " 2 2 4\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wc_lines_stdin",
|
||||
"cmd": "wc -l",
|
||||
"stdin_text": "a\nb\n",
|
||||
"matrix": {
|
||||
"python": [
|
||||
"ram",
|
||||
"disk",
|
||||
"redis"
|
||||
],
|
||||
"typescript": [
|
||||
"ram"
|
||||
]
|
||||
},
|
||||
"expect": {
|
||||
"exit": 0,
|
||||
"stdout_text": "2\n",
|
||||
"stderr_text": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"/data/a.txt": {
|
||||
"text": "hello\nworld\nfoo\nbar\nbaz\n"
|
||||
},
|
||||
"/data/b.txt": {
|
||||
"text": "1\n2\n3\n"
|
||||
},
|
||||
"/data/no_nl.txt": {
|
||||
"text": "no trailing newline"
|
||||
},
|
||||
"/data/empty.txt": {
|
||||
"text": ""
|
||||
},
|
||||
"/data/binary.bin": {
|
||||
"base64": "AAECaGVsbG///g=="
|
||||
},
|
||||
"/data/same_a.txt": {
|
||||
"text": "identical\n"
|
||||
},
|
||||
"/data/same_b.txt": {
|
||||
"text": "identical\n"
|
||||
},
|
||||
"/data/sub/nested.txt": {
|
||||
"text": "nested\ncontent\n"
|
||||
},
|
||||
"/data/sub/deep/deeper.txt": {
|
||||
"text": "deep\n"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
{
|
||||
"company": "Strukto",
|
||||
"founded": "2024-03-01",
|
||||
"valuation": 45000000,
|
||||
"departments": [
|
||||
{
|
||||
"name": "Engineering",
|
||||
"budget": 2500000,
|
||||
"quarterly_spend": [580000, 620000, 640000, 660000],
|
||||
"teams": [
|
||||
{
|
||||
"name": "Platform",
|
||||
"lead": "Alice Chen",
|
||||
"slack_channel": "#platform-eng",
|
||||
"oncall_rotation": ["Alice Chen", "Dan Lee", "Bob Park"],
|
||||
"members": [
|
||||
{"name": "Alice Chen", "role": "lead", "level": "senior", "salary": 185000, "skills": ["python", "go", "kubernetes"], "joined": "2024-03-15", "reviews": [{"quarter": "Q1-2025", "score": 4.8, "feedback": "exceptional technical leadership"}, {"quarter": "Q2-2025", "score": 4.7, "feedback": "drove mirage architecture"}]},
|
||||
{"name": "Bob Park", "role": "backend", "level": "mid", "salary": 140000, "skills": ["python", "postgres"], "joined": "2024-06-01", "reviews": [{"quarter": "Q1-2025", "score": 4.2, "feedback": "solid contributor"}, {"quarter": "Q2-2025", "score": 4.5, "feedback": "growing into senior scope"}]},
|
||||
{"name": "Carol Wu", "role": "backend", "level": "junior", "salary": 105000, "skills": ["python"], "joined": "2025-01-10", "reviews": [{"quarter": "Q1-2025", "score": 3.8, "feedback": "ramping up well"}]},
|
||||
{"name": "Dan Lee", "role": "devops", "level": "senior", "salary": 175000, "skills": ["terraform", "kubernetes", "aws", "gcp"], "joined": "2024-04-01", "reviews": [{"quarter": "Q1-2025", "score": 4.6, "feedback": "infra reliability improved 40%"}, {"quarter": "Q2-2025", "score": 4.4, "feedback": "mentoring junior engineers"}]}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"name": "mirage",
|
||||
"status": "active",
|
||||
"priority": 1,
|
||||
"started": "2025-09-15",
|
||||
"milestones": [
|
||||
{"name": "v0.1-alpha", "date": "2025-11-01", "completed": true},
|
||||
{"name": "v0.5-beta", "date": "2026-02-01", "completed": true},
|
||||
{"name": "v1.0-ga", "date": "2026-06-01", "completed": false}
|
||||
],
|
||||
"metrics": {"commits": 847, "open_issues": 23, "closed_issues": 312, "test_coverage": 0.87},
|
||||
"dependencies": ["pipeline-v2", "agent-sdk"]
|
||||
},
|
||||
{
|
||||
"name": "pipeline-v2",
|
||||
"status": "planning",
|
||||
"priority": 2,
|
||||
"started": null,
|
||||
"milestones": [],
|
||||
"metrics": {"commits": 0, "open_issues": 5, "closed_issues": 0, "test_coverage": null},
|
||||
"dependencies": []
|
||||
}
|
||||
],
|
||||
"incidents": [
|
||||
{"id": "INC-042", "severity": "p1", "title": "S3 mount timeout under load", "date": "2026-01-15", "resolved": true, "ttd_minutes": 12, "ttr_minutes": 45, "root_cause": "connection pool exhaustion", "owner": "Dan Lee"},
|
||||
{"id": "INC-051", "severity": "p2", "title": "Cache invalidation race condition", "date": "2026-02-20", "resolved": true, "ttd_minutes": 35, "ttr_minutes": 120, "root_cause": "missing lock on concurrent reads", "owner": "Alice Chen"},
|
||||
{"id": "INC-063", "severity": "p3", "title": "Memory leak in long-running grep pipes", "date": "2026-03-10", "resolved": false, "ttd_minutes": null, "ttr_minutes": null, "root_cause": null, "owner": "Bob Park"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "AI",
|
||||
"lead": "Eve Santos",
|
||||
"slack_channel": "#ai-team",
|
||||
"oncall_rotation": ["Eve Santos", "Grace Kim"],
|
||||
"members": [
|
||||
{"name": "Eve Santos", "role": "lead", "level": "senior", "salary": 195000, "skills": ["python", "pytorch", "mlops", "cuda"], "joined": "2024-05-01", "reviews": [{"quarter": "Q1-2025", "score": 4.9, "feedback": "built ML platform from scratch"}, {"quarter": "Q2-2025", "score": 4.8, "feedback": "agent-sdk is company-defining"}]},
|
||||
{"name": "Frank Liu", "role": "ml-engineer", "level": "mid", "salary": 155000, "skills": ["python", "pytorch", "onnx"], "joined": "2024-09-15", "reviews": [{"quarter": "Q1-2025", "score": 4.0, "feedback": "good model optimization work"}, {"quarter": "Q2-2025", "score": 4.3, "feedback": "latency improvements impressive"}]},
|
||||
{"name": "Grace Kim", "role": "ml-engineer", "level": "senior", "salary": 180000, "skills": ["python", "jax", "triton", "cuda"], "joined": "2024-07-01", "reviews": [{"quarter": "Q1-2025", "score": 4.7, "feedback": "triton kernels are exceptional"}, {"quarter": "Q2-2025", "score": 4.6, "feedback": "eval framework design is solid"}]}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"name": "agent-sdk",
|
||||
"status": "active",
|
||||
"priority": 1,
|
||||
"started": "2025-11-01",
|
||||
"milestones": [
|
||||
{"name": "tool-use-v1", "date": "2026-01-15", "completed": true},
|
||||
{"name": "multi-agent", "date": "2026-04-01", "completed": false},
|
||||
{"name": "production-hardening", "date": "2026-07-01", "completed": false}
|
||||
],
|
||||
"metrics": {"commits": 523, "open_issues": 41, "closed_issues": 189, "test_coverage": 0.82},
|
||||
"dependencies": ["eval-framework"]
|
||||
},
|
||||
{
|
||||
"name": "eval-framework",
|
||||
"status": "active",
|
||||
"priority": 2,
|
||||
"started": "2026-01-10",
|
||||
"milestones": [
|
||||
{"name": "baseline-evals", "date": "2026-02-15", "completed": true},
|
||||
{"name": "regression-suite", "date": "2026-04-15", "completed": false}
|
||||
],
|
||||
"metrics": {"commits": 156, "open_issues": 12, "closed_issues": 67, "test_coverage": 0.91},
|
||||
"dependencies": []
|
||||
},
|
||||
{
|
||||
"name": "model-serving",
|
||||
"status": "completed",
|
||||
"priority": 3,
|
||||
"started": "2025-06-01",
|
||||
"milestones": [
|
||||
{"name": "single-model", "date": "2025-08-01", "completed": true},
|
||||
{"name": "multi-model", "date": "2025-10-15", "completed": true},
|
||||
{"name": "auto-scaling", "date": "2025-12-01", "completed": true}
|
||||
],
|
||||
"metrics": {"commits": 412, "open_issues": 3, "closed_issues": 201, "test_coverage": 0.94},
|
||||
"dependencies": []
|
||||
}
|
||||
],
|
||||
"incidents": [
|
||||
{"id": "INC-038", "severity": "p1", "title": "Model serving OOM on large batch", "date": "2025-12-05", "resolved": true, "ttd_minutes": 5, "ttr_minutes": 30, "root_cause": "unbounded batch accumulation", "owner": "Eve Santos"},
|
||||
{"id": "INC-055", "severity": "p2", "title": "Eval pipeline stale cache", "date": "2026-02-28", "resolved": true, "ttd_minutes": 60, "ttr_minutes": 180, "root_cause": "redis TTL misconfigured", "owner": "Frank Liu"}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Product",
|
||||
"budget": 1200000,
|
||||
"quarterly_spend": [280000, 290000, 310000, 320000],
|
||||
"teams": [
|
||||
{
|
||||
"name": "Growth",
|
||||
"lead": "Hank Brown",
|
||||
"slack_channel": "#growth",
|
||||
"oncall_rotation": ["Hank Brown"],
|
||||
"members": [
|
||||
{"name": "Hank Brown", "role": "lead", "level": "senior", "salary": 165000, "skills": ["analytics", "sql", "figma", "amplitude"], "joined": "2024-08-01", "reviews": [{"quarter": "Q1-2025", "score": 4.3, "feedback": "strong product instincts"}, {"quarter": "Q2-2025", "score": 4.5, "feedback": "onboarding conversion up 25%"}]},
|
||||
{"name": "Iris Zhang", "role": "designer", "level": "mid", "salary": 130000, "skills": ["figma", "css", "framer"], "joined": "2024-10-01", "reviews": [{"quarter": "Q1-2025", "score": 4.1, "feedback": "clean design system"}, {"quarter": "Q2-2025", "score": 4.4, "feedback": "user research driving decisions"}]},
|
||||
{"name": "Jack Patel", "role": "pm", "level": "mid", "salary": 145000, "skills": ["sql", "analytics", "linear"], "joined": "2025-02-01", "reviews": [{"quarter": "Q2-2025", "score": 4.0, "feedback": "good stakeholder management"}]}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"name": "onboarding-v3",
|
||||
"status": "active",
|
||||
"priority": 1,
|
||||
"started": "2026-02-01",
|
||||
"milestones": [
|
||||
{"name": "user-research", "date": "2026-02-15", "completed": true},
|
||||
{"name": "prototype", "date": "2026-03-15", "completed": true},
|
||||
{"name": "launch", "date": "2026-05-01", "completed": false}
|
||||
],
|
||||
"metrics": {"commits": 89, "open_issues": 8, "closed_issues": 34, "test_coverage": 0.72},
|
||||
"dependencies": ["agent-sdk"]
|
||||
},
|
||||
{
|
||||
"name": "dashboard-redesign",
|
||||
"status": "planning",
|
||||
"priority": 2,
|
||||
"started": null,
|
||||
"milestones": [],
|
||||
"metrics": {"commits": 0, "open_issues": 3, "closed_issues": 0, "test_coverage": null},
|
||||
"dependencies": ["onboarding-v3"]
|
||||
}
|
||||
],
|
||||
"incidents": []
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Infrastructure",
|
||||
"budget": 1800000,
|
||||
"quarterly_spend": [420000, 440000, 460000, 480000],
|
||||
"teams": [
|
||||
{
|
||||
"name": "SRE",
|
||||
"lead": "Karen Okafor",
|
||||
"slack_channel": "#sre",
|
||||
"oncall_rotation": ["Karen Okafor", "Leo Martinez", "Mia Tanaka"],
|
||||
"members": [
|
||||
{"name": "Karen Okafor", "role": "lead", "level": "senior", "salary": 190000, "skills": ["kubernetes", "prometheus", "grafana", "terraform"], "joined": "2024-03-15", "reviews": [{"quarter": "Q1-2025", "score": 4.7, "feedback": "observability stack is world-class"}, {"quarter": "Q2-2025", "score": 4.8, "feedback": "zero p1 incidents in Q2"}]},
|
||||
{"name": "Leo Martinez", "role": "sre", "level": "mid", "salary": 150000, "skills": ["kubernetes", "python", "prometheus"], "joined": "2025-03-01", "reviews": [{"quarter": "Q2-2025", "score": 4.2, "feedback": "good runbook documentation"}]},
|
||||
{"name": "Mia Tanaka", "role": "sre", "level": "senior", "salary": 178000, "skills": ["aws", "gcp", "terraform", "ansible"], "joined": "2024-06-15", "reviews": [{"quarter": "Q1-2025", "score": 4.5, "feedback": "multi-cloud migration on track"}, {"quarter": "Q2-2025", "score": 4.6, "feedback": "cost optimization saved 30%"}]}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"name": "multi-cloud",
|
||||
"status": "active",
|
||||
"priority": 1,
|
||||
"started": "2025-07-01",
|
||||
"milestones": [
|
||||
{"name": "aws-baseline", "date": "2025-09-01", "completed": true},
|
||||
{"name": "gcp-parity", "date": "2026-03-01", "completed": true},
|
||||
{"name": "failover-testing", "date": "2026-06-01", "completed": false}
|
||||
],
|
||||
"metrics": {"commits": 334, "open_issues": 15, "closed_issues": 178, "test_coverage": 0.79},
|
||||
"dependencies": []
|
||||
},
|
||||
{
|
||||
"name": "cost-optimization",
|
||||
"status": "active",
|
||||
"priority": 2,
|
||||
"started": "2026-01-01",
|
||||
"milestones": [
|
||||
{"name": "audit", "date": "2026-01-15", "completed": true},
|
||||
{"name": "reserved-instances", "date": "2026-02-15", "completed": true},
|
||||
{"name": "spot-fleet", "date": "2026-04-01", "completed": false}
|
||||
],
|
||||
"metrics": {"commits": 67, "open_issues": 4, "closed_issues": 29, "test_coverage": 0.85},
|
||||
"dependencies": ["multi-cloud"]
|
||||
}
|
||||
],
|
||||
"incidents": [
|
||||
{"id": "INC-029", "severity": "p1", "title": "Full cluster outage during upgrade", "date": "2025-10-22", "resolved": true, "ttd_minutes": 3, "ttr_minutes": 90, "root_cause": "incompatible CRD version", "owner": "Karen Okafor"},
|
||||
{"id": "INC-044", "severity": "p1", "title": "DNS resolution failure", "date": "2026-01-20", "resolved": true, "ttd_minutes": 8, "ttr_minutes": 25, "root_cause": "coredns pods evicted by OOM", "owner": "Leo Martinez"},
|
||||
{"id": "INC-058", "severity": "p2", "title": "Prometheus storage full", "date": "2026-03-05", "resolved": true, "ttd_minutes": 45, "ttr_minutes": 60, "root_cause": "retention policy too aggressive", "owner": "Mia Tanaka"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"locations": [
|
||||
{"city": "San Francisco", "country": "US", "timezone": "America/Los_Angeles", "headcount": 7, "office_type": "hq", "monthly_cost": 85000, "amenities": ["gym", "cafeteria", "parking"]},
|
||||
{"city": "Seoul", "country": "KR", "timezone": "Asia/Seoul", "headcount": 3, "office_type": "coworking", "monthly_cost": 12000, "amenities": ["meeting_rooms"]},
|
||||
{"city": "Toronto", "country": "CA", "timezone": "America/Toronto", "headcount": 2, "office_type": "remote", "monthly_cost": 3000, "amenities": []},
|
||||
{"city": "London", "country": "GB", "timezone": "Europe/London", "headcount": 1, "office_type": "remote", "monthly_cost": 2500, "amenities": []}
|
||||
],
|
||||
"vendor_contracts": [
|
||||
{"vendor": "AWS", "service": "cloud", "annual_cost": 960000, "renewal": "2027-01-01", "tier": "enterprise"},
|
||||
{"vendor": "GCP", "service": "cloud", "annual_cost": 480000, "renewal": "2026-09-01", "tier": "premium"},
|
||||
{"vendor": "Datadog", "service": "observability", "annual_cost": 120000, "renewal": "2026-06-01", "tier": "pro"},
|
||||
{"vendor": "Linear", "service": "project-management", "annual_cost": 7200, "renewal": "2026-04-01", "tier": "business"},
|
||||
{"vendor": "Figma", "service": "design", "annual_cost": 4800, "renewal": "2026-07-01", "tier": "organization"},
|
||||
{"vendor": "Anthropic", "service": "ai-api", "annual_cost": 240000, "renewal": "2026-12-01", "tier": "enterprise"}
|
||||
],
|
||||
"okrs": [
|
||||
{
|
||||
"quarter": "Q1-2026",
|
||||
"objectives": [
|
||||
{
|
||||
"title": "Launch mirage v1.0",
|
||||
"key_results": [
|
||||
{"description": "Ship GA release", "target": 1, "actual": 0, "unit": "release"},
|
||||
{"description": "Onboard 10 enterprise customers", "target": 10, "actual": 4, "unit": "customers"},
|
||||
{"description": "Achieve 99.9% uptime", "target": 99.9, "actual": 99.7, "unit": "percent"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Build world-class AI capabilities",
|
||||
"key_results": [
|
||||
{"description": "Launch agent-sdk multi-agent", "target": 1, "actual": 0, "unit": "release"},
|
||||
{"description": "Eval coverage across 50 benchmarks", "target": 50, "actual": 32, "unit": "benchmarks"},
|
||||
{"description": "P95 inference latency under 200ms", "target": 200, "actual": 185, "unit": "ms"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Operational excellence",
|
||||
"key_results": [
|
||||
{"description": "Mean time to detect under 15 min", "target": 15, "actual": 22, "unit": "minutes"},
|
||||
{"description": "Mean time to resolve under 60 min", "target": 60, "actual": 75, "unit": "minutes"},
|
||||
{"description": "Cloud cost reduction 20%", "target": 20, "actual": 30, "unit": "percent"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"version": "3.0",
|
||||
"last_updated": "2026-03-25T00:00:00Z",
|
||||
"total_headcount": 13,
|
||||
"total_budget": 5500000,
|
||||
"fiscal_year": "2026",
|
||||
"data_classification": "internal"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
plans/
|
||||
@@ -0,0 +1,540 @@
|
||||
{
|
||||
"$schema": "https://mintlify.com/docs.json",
|
||||
"theme": "mint",
|
||||
"name": "Mirage",
|
||||
"description": "Mirage gives AI agents one virtual filesystem for S3, Google Drive, Slack, GitHub, Notion, Postgres, and SSH, so shell commands work across services.",
|
||||
"colors": {
|
||||
"primary": "#0C0C0C",
|
||||
"light": "#FAFAFA",
|
||||
"dark": "#000000"
|
||||
},
|
||||
"logo": {
|
||||
"light": "/logo/light.svg",
|
||||
"dark": "/logo/dark.svg"
|
||||
},
|
||||
"favicon": "/favicon.svg",
|
||||
"contextual": {
|
||||
"options": [
|
||||
"copy",
|
||||
"view",
|
||||
"chatgpt",
|
||||
"claude",
|
||||
"perplexity",
|
||||
"mcp",
|
||||
"cursor",
|
||||
"vscode"
|
||||
]
|
||||
},
|
||||
"seo": {
|
||||
"metatags": {
|
||||
"keywords": "Mirage, filesystem for AI agents, virtual filesystem, AI agent tools, agent infrastructure, MCP alternative, bash for agents, S3 filesystem, Google Drive filesystem, Slack filesystem, GitHub filesystem, agent sandbox, Strukto",
|
||||
"author": "Strukto",
|
||||
"robots": "index, follow",
|
||||
"googlebot": "index, follow",
|
||||
"canonical": "https://docs.mirage.strukto.ai",
|
||||
"theme-color": "#0C0C0C",
|
||||
"color-scheme": "light dark",
|
||||
"language": "en",
|
||||
"category": "Technology",
|
||||
"target": "developers",
|
||||
"apple-mobile-web-app-capable": "yes",
|
||||
"apple-mobile-web-app-status-bar-style": "black-translucent",
|
||||
"og:type": "website",
|
||||
"og:url": "https://docs.mirage.strukto.ai",
|
||||
"og:image": "https://raw.githubusercontent.com/strukto-ai/mirage/main/assets/mirage-og-light.png",
|
||||
"og:image:alt": "Mirage virtual filesystem for AI agents",
|
||||
"og:site_name": "Mirage",
|
||||
"og:locale": "en_US",
|
||||
"twitter:card": "summary_large_image",
|
||||
"twitter:site": "@struktoai",
|
||||
"twitter:creator": "@struktoai",
|
||||
"twitter:image": "https://raw.githubusercontent.com/strukto-ai/mirage/main/assets/mirage-og-light.png",
|
||||
"twitter:image:alt": "Mirage · Unified Virtual Filesystem for AI Agents | Strukto"
|
||||
}
|
||||
},
|
||||
"navigation": {
|
||||
"tabs": [
|
||||
{
|
||||
"tab": "Home",
|
||||
"icon": "/images/mirage-icon-light.svg",
|
||||
"groups": [
|
||||
{
|
||||
"group": "Getting Started",
|
||||
"pages": [
|
||||
"home/introduction",
|
||||
"home/install",
|
||||
"home/cli"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Learn",
|
||||
"pages": [
|
||||
"home/architecture",
|
||||
"home/bash",
|
||||
"home/resource-matrix",
|
||||
"home/cache",
|
||||
"home/snapshot",
|
||||
"home/observer",
|
||||
"home/auth"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Help",
|
||||
"pages": [
|
||||
"home/troubleshooting"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Setup",
|
||||
"pages": [
|
||||
{
|
||||
"group": "FUSE",
|
||||
"icon": "hard-drive",
|
||||
"pages": [
|
||||
"home/setup/macos",
|
||||
"home/setup/linux"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Object Storage",
|
||||
"icon": "cloud",
|
||||
"pages": [
|
||||
"home/setup/s3",
|
||||
"home/setup/r2",
|
||||
"home/setup/gcs",
|
||||
"home/setup/oci",
|
||||
"home/setup/supabase",
|
||||
"home/setup/minio",
|
||||
"home/setup/ceph",
|
||||
"home/setup/seaweedfs",
|
||||
"home/setup/wasabi",
|
||||
"home/setup/backblaze",
|
||||
"home/setup/digitalocean",
|
||||
"home/setup/tencent",
|
||||
"home/setup/aliyun",
|
||||
"home/setup/scaleway",
|
||||
"home/setup/qingstor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Hugging Face",
|
||||
"icon": "/images/huggingface-logo.svg",
|
||||
"pages": [
|
||||
"home/setup/hf_buckets",
|
||||
"home/setup/hf_datasets",
|
||||
"home/setup/hf_models",
|
||||
"home/setup/hf_spaces"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Google Workspace",
|
||||
"icon": "google",
|
||||
"pages": [
|
||||
"home/setup/google"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Microsoft",
|
||||
"icon": "microsoft",
|
||||
"pages": [
|
||||
"home/setup/onedrive",
|
||||
"home/setup/sharepoint"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Cloud Files",
|
||||
"icon": "folder-open",
|
||||
"pages": [
|
||||
"home/setup/dropbox",
|
||||
"home/setup/box",
|
||||
"home/setup/nextcloud",
|
||||
"home/setup/databricks"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Code & DevOps",
|
||||
"icon": "code",
|
||||
"pages": [
|
||||
"home/setup/github",
|
||||
"home/setup/github_ci",
|
||||
"home/setup/linear",
|
||||
"home/setup/langfuse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Messaging",
|
||||
"icon": "comments",
|
||||
"pages": [
|
||||
"home/setup/slack",
|
||||
"home/setup/discord",
|
||||
"home/setup/email"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Database",
|
||||
"icon": "database",
|
||||
"pages": [
|
||||
"home/setup/mongodb",
|
||||
"home/setup/postgres",
|
||||
"home/setup/chroma",
|
||||
"home/setup/lancedb",
|
||||
"home/setup/qdrant"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Knowledge",
|
||||
"icon": "brain",
|
||||
"pages": [
|
||||
"home/setup/dify"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Notes",
|
||||
"icon": "book",
|
||||
"pages": [
|
||||
"home/setup/notion"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Others",
|
||||
"icon": "ellipsis",
|
||||
"pages": [
|
||||
"home/setup/trello"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "Python",
|
||||
"icon": "/images/python-logo.svg",
|
||||
"groups": [
|
||||
{
|
||||
"group": "Getting Started",
|
||||
"pages": [
|
||||
"python/install",
|
||||
"python/quickstart",
|
||||
"python/setup/fuse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Agents",
|
||||
"icon": "robot",
|
||||
"pages": [
|
||||
"python/agents/index",
|
||||
"python/agents/openai-agents",
|
||||
"python/agents/langchain",
|
||||
"python/agents/openhands",
|
||||
"python/agents/pydantic-ai",
|
||||
"python/agents/camel",
|
||||
"python/agents/agno",
|
||||
"python/agents/claude-code",
|
||||
"python/agents/claude-agent-sdk",
|
||||
"python/agents/codex"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Resources",
|
||||
"pages": [
|
||||
"python/resource/index",
|
||||
{
|
||||
"group": "Infrastructure",
|
||||
"icon": "server",
|
||||
"pages": [
|
||||
"python/resource/disk",
|
||||
"python/resource/ram",
|
||||
"python/resource/redis",
|
||||
"python/resource/ssh"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Object Storage",
|
||||
"icon": "cloud",
|
||||
"pages": [
|
||||
"python/resource/s3",
|
||||
"python/resource/r2",
|
||||
"python/resource/gcs",
|
||||
"python/resource/supabase",
|
||||
"python/resource/oci",
|
||||
"python/resource/minio",
|
||||
"python/resource/ceph",
|
||||
"python/resource/seaweedfs",
|
||||
"python/resource/wasabi",
|
||||
"python/resource/backblaze",
|
||||
"python/resource/digitalocean",
|
||||
"python/resource/tencent",
|
||||
"python/resource/aliyun",
|
||||
"python/resource/scaleway",
|
||||
"python/resource/qingstor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Hugging Face",
|
||||
"icon": "/images/huggingface-logo.svg",
|
||||
"pages": [
|
||||
"python/resource/hf_buckets",
|
||||
"python/resource/hf_datasets",
|
||||
"python/resource/hf_models",
|
||||
"python/resource/hf_spaces"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Google Workspace",
|
||||
"icon": "google",
|
||||
"pages": [
|
||||
"python/resource/gmail",
|
||||
"python/resource/gdrive",
|
||||
"python/resource/gdocs",
|
||||
"python/resource/gsheets",
|
||||
"python/resource/gslides"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Microsoft",
|
||||
"icon": "microsoft",
|
||||
"pages": [
|
||||
"python/resource/onedrive",
|
||||
"python/resource/sharepoint"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Cloud Files",
|
||||
"icon": "folder-open",
|
||||
"pages": [
|
||||
"python/resource/nextcloud",
|
||||
"python/resource/databricks_volume"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Code & DevOps",
|
||||
"icon": "code",
|
||||
"pages": [
|
||||
"python/resource/github",
|
||||
"python/resource/github_ci",
|
||||
"python/resource/linear",
|
||||
"python/resource/langfuse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Messaging",
|
||||
"icon": "comments",
|
||||
"pages": [
|
||||
"python/resource/slack",
|
||||
"python/resource/discord",
|
||||
"python/resource/email"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Database",
|
||||
"icon": "database",
|
||||
"pages": [
|
||||
"python/resource/mongodb",
|
||||
"python/resource/postgres",
|
||||
"python/resource/chroma",
|
||||
"python/resource/lancedb",
|
||||
"python/resource/qdrant"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Knowledge",
|
||||
"icon": "brain",
|
||||
"pages": [
|
||||
"python/resource/dify"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Notes",
|
||||
"icon": "book",
|
||||
"pages": [
|
||||
"python/resource/notion"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Others",
|
||||
"icon": "ellipsis",
|
||||
"pages": [
|
||||
"python/resource/trello"
|
||||
]
|
||||
},
|
||||
"python/resource/new"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "TS",
|
||||
"icon": "/images/typescript-logo.svg",
|
||||
"groups": [
|
||||
{
|
||||
"group": "Getting Started",
|
||||
"pages": [
|
||||
"typescript/install",
|
||||
"typescript/quickstart",
|
||||
"typescript/server-and-cli",
|
||||
"typescript/setup/fuse",
|
||||
"typescript/python",
|
||||
"typescript/python-fs",
|
||||
"typescript/limitations"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Agents",
|
||||
"icon": "robot",
|
||||
"pages": [
|
||||
"typescript/agents/index",
|
||||
"typescript/agents/openai",
|
||||
"typescript/agents/langchain",
|
||||
"typescript/agents/pi",
|
||||
"typescript/agents/vercel",
|
||||
"typescript/agents/mastra",
|
||||
"typescript/agents/opencode",
|
||||
"typescript/agents/claude-code",
|
||||
"typescript/agents/claude-agent-sdk",
|
||||
"typescript/agents/codex"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Resources",
|
||||
"pages": [
|
||||
{
|
||||
"group": "Infrastructure",
|
||||
"icon": "server",
|
||||
"pages": [
|
||||
"typescript/setup/disk",
|
||||
"typescript/setup/opfs",
|
||||
"typescript/setup/redis",
|
||||
"typescript/setup/ssh"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Object Storage",
|
||||
"icon": "cloud",
|
||||
"pages": [
|
||||
"typescript/setup/s3",
|
||||
"typescript/setup/r2",
|
||||
"typescript/setup/gcs",
|
||||
"typescript/setup/oci",
|
||||
"typescript/setup/supabase",
|
||||
"typescript/setup/minio",
|
||||
"typescript/setup/ceph",
|
||||
"typescript/setup/seaweedfs",
|
||||
"typescript/setup/wasabi",
|
||||
"typescript/setup/backblaze",
|
||||
"typescript/setup/digitalocean",
|
||||
"typescript/setup/tencent",
|
||||
"typescript/setup/aliyun",
|
||||
"typescript/setup/scaleway",
|
||||
"typescript/setup/qingstor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Hugging Face",
|
||||
"icon": "/images/huggingface-logo.svg",
|
||||
"pages": [
|
||||
"typescript/setup/hf_buckets",
|
||||
"typescript/setup/hf_datasets",
|
||||
"typescript/setup/hf_models",
|
||||
"typescript/setup/hf_spaces"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Google Workspace",
|
||||
"icon": "google",
|
||||
"pages": [
|
||||
"typescript/setup/gmail",
|
||||
"typescript/setup/gdrive",
|
||||
"typescript/setup/gdocs",
|
||||
"typescript/setup/gsheets",
|
||||
"typescript/setup/gslides"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Cloud Files",
|
||||
"icon": "folder-open",
|
||||
"pages": [
|
||||
"typescript/dropbox",
|
||||
"typescript/box",
|
||||
"typescript/setup/databricks_volume"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Code & DevOps",
|
||||
"icon": "code",
|
||||
"pages": [
|
||||
"typescript/setup/github",
|
||||
"typescript/setup/github_ci",
|
||||
"typescript/setup/linear",
|
||||
"typescript/setup/langfuse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Messaging",
|
||||
"icon": "comments",
|
||||
"pages": [
|
||||
"typescript/slack",
|
||||
"typescript/discord",
|
||||
"typescript/setup/email"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Database",
|
||||
"icon": "database",
|
||||
"pages": [
|
||||
"typescript/setup/mongodb",
|
||||
"typescript/setup/postgres",
|
||||
"typescript/setup/chroma",
|
||||
"typescript/setup/lancedb",
|
||||
"typescript/setup/qdrant"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Notes",
|
||||
"icon": "book",
|
||||
"pages": [
|
||||
"typescript/setup/notion"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Others",
|
||||
"icon": "ellipsis",
|
||||
"pages": [
|
||||
"typescript/setup/trello"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"navbar": {
|
||||
"links": [
|
||||
{
|
||||
"label": "mirage",
|
||||
"icon": "globe",
|
||||
"href": "https://www.strukto.ai/mirage"
|
||||
},
|
||||
{
|
||||
"label": "Discord",
|
||||
"icon": "discord",
|
||||
"href": "https://discord.gg/u8BPQ65KsS"
|
||||
},
|
||||
{
|
||||
"label": "",
|
||||
"icon": "x-twitter",
|
||||
"href": "https://x.com/struktoai"
|
||||
},
|
||||
{
|
||||
"label": "strukto-ai/mirage",
|
||||
"icon": "github",
|
||||
"href": "https://github.com/strukto-ai/mirage"
|
||||
}
|
||||
]
|
||||
},
|
||||
"footer": {
|
||||
"socials": {
|
||||
"github": "https://github.com/strukto-ai/mirage"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg"
|
||||
viewBox="0 0 160 160"
|
||||
width="160" height="160">
|
||||
|
||||
<polygon points="19,45 80,10 80,45 50,63"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
<polygon points="80,10 141,45 110,63 80,45"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
<polygon points="141,45 141,115 110,98 110,63"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
<polygon points="141,115 80,150 80,115 110,98"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
<polygon points="80,150 19,115 50,98 80,115"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
<polygon points="19,115 19,45 50,63 50,98"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
|
||||
<polygon points="80,45 110,63 110,98 80,115 50,98 50,63"
|
||||
fill="#000000" stroke="#ffffff" stroke-width="3"/>
|
||||
|
||||
<line x1="80" y1="80" x2="50" y2="63"
|
||||
stroke="#ffffff" stroke-width="3"/>
|
||||
<line x1="80" y1="80" x2="110" y2="63"
|
||||
stroke="#ffffff" stroke-width="3"/>
|
||||
<line x1="80" y1="80" x2="80" y2="115"
|
||||
stroke="#ffffff" stroke-width="3"/>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.1 KiB |
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: Architecture
|
||||
description: Four layers, one filesystem. How Mirage turns mounted services into one bash-driven environment.
|
||||
icon: cube
|
||||
---
|
||||
|
||||
<div className="my-6">
|
||||
<img noZoom src="/images/mirage-arch-light.svg" alt="Mirage architecture" className="block dark:hidden w-full" />
|
||||
<img noZoom src="/images/mirage-arch-dark.svg" alt="Mirage architecture" className="hidden dark:block w-full" />
|
||||
</div>
|
||||
|
||||
Mirage stacks four thin layers between an agent and the services it touches.
|
||||
|
||||
## 1. AI Agent and Application
|
||||
|
||||
The agent (or any application embedding Mirage) issues bash commands, VFS calls, or syscalls. One vocabulary, every backend.
|
||||
|
||||
## 2. Mirage Bash and VFS
|
||||
|
||||
The action surface. **Mirage Bash** parses commands with tree-sitter and runs them against the **Mirage VFS**, a unified filesystem API over every mount. A **FUSE Adapter** exposes the same tree to host tools when you want it. A **Command Registry** and **VFS Registry** describe what verbs and resources are available.
|
||||
|
||||
## 3. Dispatcher & Cache
|
||||
|
||||
The **Mirage Dispatcher** routes each operation to the mount that owns the path, joining pipelines that span systems. The **Index & File Cache** absorbs repeats: the first directory walk hits the API, the next serves from cache; the first read streams bytes, later reads are local.
|
||||
|
||||
## 4. Infrastructure and Remote
|
||||
|
||||
Whatever you mount: RAM, Disk, Redis, S3 / R2 / GCS / OCI / Supabase, Gmail / GDrive / GDocs / GSheets / GSlides, GitHub / Linear / Notion / Trello, Slack / Discord / Email, MongoDB / Postgres / LanceDB / Qdrant, SSH, and more. Each speaks the same filesystem semantics from the agent's point of view.
|
||||
|
||||
Browse the [Resource Matrix](/home/resource-matrix) for the full list.
|
||||
|
||||
## Where to go next
|
||||
|
||||
- [Resource Matrix](/home/resource-matrix) to pick a backend to mount.
|
||||
- [Python Quickstart](/python/quickstart) for working code in minutes.
|
||||
- [TypeScript Quickstart](/typescript/quickstart) for the same Workspace API in Node, browser, or edge.
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
title: Daemon Auth
|
||||
description: How the Mirage daemon authenticates HTTP clients in local, token, and JWT modes.
|
||||
icon: shield-halved
|
||||
---
|
||||
|
||||
## What It Does
|
||||
|
||||
The daemon serves a local HTTP API on `127.0.0.1:8765`. Every request except `/v1/health` must present a bearer token. Which token is accepted is decided by `MIRAGE_AUTH_MODE`:
|
||||
|
||||
| Mode | When to use | What the daemon accepts |
|
||||
| --- | --- | --- |
|
||||
| `local` (default) | One user, one machine. Zero config. | A random token the CLI mints into `~/.mirage/auth_token` (mode 0o600). |
|
||||
| `token` | Shared daemon, operator-issued PAT. | The exact string in `MIRAGE_AUTH_TOKEN`. |
|
||||
| `jwt` | Multi-tenant, external issuer (Clerk, Auth0, your own). | Any RS256-signed JWT that verifies against `MIRAGE_JWT_PUBKEY` / `_FILE`. |
|
||||
|
||||
`/v1/health` is always reachable without a token so load balancers and process supervisors can probe it.
|
||||
|
||||
## Local Mode (Default)
|
||||
|
||||
You usually do nothing. The first time the CLI spawns the daemon it writes a random 32-byte token to `~/.mirage/auth_token` at mode 0o600 and uses it on every subsequent request.
|
||||
|
||||
```bash
|
||||
mirage workspace create workspace.yaml --id demo
|
||||
cat ~/.mirage/auth_token # the token the daemon expects
|
||||
```
|
||||
|
||||
Probe it directly:
|
||||
|
||||
```bash
|
||||
TOKEN=$(cat ~/.mirage/auth_token)
|
||||
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8765/v1/workspaces # 200
|
||||
curl http://127.0.0.1:8765/v1/workspaces # 401
|
||||
curl http://127.0.0.1:8765/v1/health # 200
|
||||
```
|
||||
|
||||
## Token Mode (Operator PAT)
|
||||
|
||||
For a daemon you run yourself (Docker, systemd, a shared dev box), pin one token across all clients.
|
||||
|
||||
```bash
|
||||
export MIRAGE_AUTH_MODE=token
|
||||
export MIRAGE_AUTH_TOKEN=<your-pat>
|
||||
|
||||
# Python
|
||||
python -m uvicorn mirage.server.daemon:app --host 127.0.0.1 --port 8765
|
||||
|
||||
# TypeScript
|
||||
node typescript/packages/cli/dist/bin/daemon.js
|
||||
```
|
||||
|
||||
Clients pass the same value:
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer $MIRAGE_AUTH_TOKEN" http://127.0.0.1:8765/v1/workspaces
|
||||
```
|
||||
|
||||
Both server and CLI use a constant-time compare, so timing leakage is not a concern.
|
||||
|
||||
## JWT Mode (External Issuer)
|
||||
|
||||
Hand the daemon a public key and it will accept any non-expired JWT signed by the matching private key. Verification is networkless: no JWKS fetch, no callback to the issuer.
|
||||
|
||||
```bash
|
||||
export MIRAGE_AUTH_MODE=jwt
|
||||
export MIRAGE_JWT_PUBKEY_FILE=/etc/mirage/issuer-pub.pem
|
||||
export MIRAGE_JWT_ALG=RS256
|
||||
|
||||
# Optional hardening
|
||||
export MIRAGE_JWT_ISSUER=https://your-issuer.example
|
||||
export MIRAGE_JWT_AUDIENCE=mirage-daemon
|
||||
export MIRAGE_JWT_AUTHORIZED_PARTIES=https://app.example,https://cli.example
|
||||
export MIRAGE_JWT_CLOCK_SKEW_SECONDS=5
|
||||
```
|
||||
|
||||
Hard rules the daemon enforces:
|
||||
|
||||
- `alg` is pinned to `MIRAGE_JWT_ALG`. A token signed with a different algorithm is rejected, which defeats alg-confusion attacks.
|
||||
- `alg=none` is always rejected.
|
||||
- `exp` is mandatory.
|
||||
- `typ`, if present, must be `JWT`.
|
||||
- Opaque (non-three-segment) values in `Authorization: Bearer` are rejected before key work, so probing is cheap.
|
||||
|
||||
## Environment Reference
|
||||
|
||||
| Variable | Modes | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `MIRAGE_AUTH_MODE` | all | `local` (default), `token`, or `jwt`. |
|
||||
| `MIRAGE_AUTH_TOKEN` | local, token | Local-mode override; required in `token` mode. |
|
||||
| `MIRAGE_JWT_PUBKEY` | jwt | PEM string of the public key. |
|
||||
| `MIRAGE_JWT_PUBKEY_FILE` | jwt | Path to a PEM file (alternative to inline). |
|
||||
| `MIRAGE_JWT_ALG` | jwt | Signing algorithm to pin, e.g. `RS256`. |
|
||||
| `MIRAGE_JWT_ISSUER` | jwt | Required `iss` claim. |
|
||||
| `MIRAGE_JWT_AUDIENCE` | jwt | Required `aud` claim. |
|
||||
| `MIRAGE_JWT_AUTHORIZED_PARTIES` | jwt | Comma-separated allow-list for `azp`. |
|
||||
| `MIRAGE_JWT_CLOCK_SKEW_SECONDS` | jwt | Default `5`. |
|
||||
|
||||
## Where to Go Next
|
||||
|
||||
- [CLI](/home/cli) walks the daily Workspace flow that uses local-mode automatically.
|
||||
- [Architecture](/home/architecture) shows where the auth middleware sits in the request path.
|
||||
@@ -0,0 +1,227 @@
|
||||
---
|
||||
title: Bash
|
||||
description: Run bash commands with `execute()`, per-call `cwd`/`env` overrides, and mid-flight cancellation.
|
||||
icon: terminal
|
||||
---
|
||||
|
||||
Mirage Bash is how agents act on the workspace. `execute()` parses a bash-style command, looks up the target session, resolves mounts, runs the executor, applies I/O side effects, and records history through the [Observer](/home/observer).
|
||||
|
||||
## Per-call overrides: `cwd`, `env`
|
||||
|
||||
Providing `cwd` or `env` runs the command in an ephemeral session clone, like a bash subshell `(cd /data && cmd)`. Mutations like `cd` or `export` inside the call do NOT persist back to the workspace's session. To change persistent state, run the command without these options.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python" icon="/images/python-logo.svg">
|
||||
```python
|
||||
# Persistent mutation (no options): like `cd /data; cmd`
|
||||
await ws.execute("cd /data")
|
||||
await ws.execute("ls") # sees /data
|
||||
|
||||
# One-shot subshell (with cwd): like `(cd /data && cmd)`
|
||||
await ws.execute("ls", cwd="/data")
|
||||
# ws.cwd is unchanged; mutations inside don't leak
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="TypeScript" icon="/images/typescript-logo.svg">
|
||||
```typescript
|
||||
// Persistent mutation
|
||||
await ws.execute("cd /data")
|
||||
await ws.execute("ls") // sees /data
|
||||
|
||||
// One-shot subshell
|
||||
await ws.execute("ls", { cwd: "/data" })
|
||||
// ws.cwd is unchanged; mutations inside don't leak
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="CLI" icon="terminal">
|
||||
```bash
|
||||
# Use a real bash subshell inside the command string
|
||||
mirage execute -w demo -c "(cd /data && ls)"
|
||||
mirage execute -w demo -c "(export FOO=bar; printenv FOO)"
|
||||
```
|
||||
The CLI doesn't have `--cwd` / `--env` flags, but bash subshell syntax `(cd ... && cmd)` and `(export FOO=bar; cmd)` give the same per-call isolation. Mutations inside the parens don't leak.
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
This makes per-call overrides safe under concurrent calls on the same session. Two parallel `execute()` calls with different `cwd` see their own cwd without cross-contamination, even on the same session.
|
||||
|
||||
## Subshells `(...)`
|
||||
|
||||
Wrapping commands in `( ... )` runs them in an isolated copy of the session: `cd`, `export`, and other mutations inside the parens do not leak back. It is the same isolation as the `cwd` / `env` overrides above, and the CLI's stand-in for them (there are no `--cwd` / `--env` flags).
|
||||
|
||||
```bash
|
||||
(cd /data && ls) # cwd change scoped to the subshell
|
||||
(export TOKEN=xyz; printenv) # env var gone once the parens close
|
||||
```
|
||||
|
||||
The isolation holds under concurrency, and subshells are still covered by the per-session mount allowlist and the cancellation boundaries below.
|
||||
|
||||
## Mid-flight cancellation: `cancel` / `signal`
|
||||
|
||||
Both bindings support cooperative cancellation observed at recursion boundaries (LIST, PIPELINE, FOR/WHILE/UNTIL iterations, COMMAND, subshells, command substitution) and inside `sleep`. On cancel, the call raises an abort error.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python" icon="/images/python-logo.svg">
|
||||
```python
|
||||
import asyncio
|
||||
from mirage.workspace.abort import MirageAbortError
|
||||
|
||||
cancel = asyncio.Event()
|
||||
|
||||
async def trigger():
|
||||
await asyncio.sleep(0.1)
|
||||
cancel.set()
|
||||
|
||||
asyncio.create_task(trigger())
|
||||
try:
|
||||
await ws.execute("sleep 5", cancel=cancel)
|
||||
except MirageAbortError:
|
||||
print("aborted")
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="TypeScript" icon="/images/typescript-logo.svg">
|
||||
```typescript
|
||||
try {
|
||||
await ws.execute("sleep 5", { signal: AbortSignal.timeout(100) })
|
||||
} catch (e) {
|
||||
if (e instanceof DOMException && e.name === "AbortError") {
|
||||
console.log("aborted")
|
||||
}
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="CLI" icon="terminal">
|
||||
```bash
|
||||
# Background the job, then cancel it
|
||||
JOB_ID=$(mirage execute -w demo -c "sleep 60" --bg)
|
||||
mirage job cancel "$JOB_ID"
|
||||
```
|
||||
Per-call timeout is not a CLI flag yet. Use `--bg` to get a job id and `mirage job cancel` to terminate, or wrap the command in the `timeout` builtin: `mirage execute -w demo -c "timeout 30 <cmd>"` exits `124` on overrun.
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Three Scopes for State
|
||||
|
||||
| Need | API | Bash equivalent |
|
||||
| --- | --- | --- |
|
||||
| One isolated command | `execute(cmd, cwd=..., env=...)` | `(cd /data && cmd)` |
|
||||
| Many isolated commands sharing scoped state | `session_id=...` (Py) / `sessionId` (TS) | a separate terminal |
|
||||
| Persistent shell mutations | run without options | `cd /data; cmd` |
|
||||
|
||||
## Supported bash syntax
|
||||
|
||||
Mirage Bash is a tree-sitter-bash parser plus a custom executor. It implements the constructs LLMs reach for most often. What is not supported returns a clear, parseable error so an agent can self-correct on its next turn.
|
||||
|
||||
### Supported
|
||||
|
||||
- **Operators:** pipes `|`, `|&`; lists `&&`, `||`, `;`; background `&`.
|
||||
- **Redirects:** `>`, `>>`, `<`, `2>`, `2>&1`, `&>`, `&>>`, heredoc `<<`, herestring `<<<`.
|
||||
- **Substitutions:** command substitution `` `cmd` `` and `$(cmd)`; arithmetic `$((expr))`; parameter expansion `${VAR}`, `${VAR:-default}`, `${VAR%suffix}`, etc.; input-direction process substitution `<(cmd)`.
|
||||
- **Control flow:** `if`/`elif`/`else`/`fi`, `for`, `while`, `until`, `case`, `select`, `function name() {}`, `break`, `continue`, `return`.
|
||||
- **Grouping:** subshells `(cmd)`, compound `{ cmd; }`, negation `! cmd`.
|
||||
- **Builtins:** `cd`, `pwd`, `echo`, `printf`, `printenv`, `read`, `source`, `.`, `eval`, `export`, `unset`, `local`, `set`, `shift`, `trap` (no-op), `test`, `[`, `[[`, `true`, `false`, `sleep`, `xargs`, `timeout`, `bash`, `sh`, `python`, `python3`.
|
||||
- **Builtin options (GNU semantics):** `echo -n/-e/-E` (leading-word option rule: `echo hi -n` prints `hi -n`), `read -r`, `xargs -n/-0/-d/-r/--` (batching, GNU exit codes: `123` when an invocation fails, `126`/`127` stop the run), `timeout DURATION` with `s`/`m`/`h`/`d` suffixes (kills at the deadline with exit `124`, usage errors exit `125`). `shift` and `return` report bash's `numeric argument required` errors.
|
||||
- **Globs:** `*`, `?`, `[...]` classes and `[!...]` negation (Python `fnmatch` semantics in both implementations), resolved by the shell or pushed down to the resource.
|
||||
- **Comments:** `#`.
|
||||
|
||||
### Unsupported (returns clear error)
|
||||
|
||||
- **Job control:** `bg`, `disown`. (`fg`, `jobs`, `wait`, `kill`, `ps` work; use the `--background` flag and `mirage job` CLI for long-running work.)
|
||||
- **Shell internals:** `exec`, `complete`, `compgen`, `ulimit`.
|
||||
- **Output process substitution:** `>(cmd)` (the `<(cmd)` direction works).
|
||||
- **Builtin options with no process backing:** `xargs -I`/`-P` (exit `1`) and `timeout -s`/`-k`/`--preserve-status` (exit `125`) return an `unsupported option` error: commands run as coroutines inside the workspace, so there is no process to signal or parallelize.
|
||||
|
||||
Each returns `exit_code 2` with stderr `mirage: unsupported builtin: <name>` or `mirage: unsupported: process substitution >(...)`, except the builtin options above, which use the listed GNU-shaped exit codes.
|
||||
|
||||
### Syntax errors
|
||||
|
||||
Commands the parser cannot make sense of return `exit_code 2` with stderr `mirage: syntax error near '<token>'`. Earlier versions silently ran whatever fragment did parse; that no longer happens.
|
||||
|
||||
### What `--background` is and isn't
|
||||
|
||||
The daemon's `--background` flag detaches a job and returns a job id. It is not the same as the bash `&` operator, which the shell does support inline (`sleep 30 &`). Use `&` for in-shell job parallelism, `--background` (or `mirage job`) for long-lived work that should outlive the request.
|
||||
|
||||
## Per-session mount capability
|
||||
|
||||
A session can be created with an explicit allowlist of mount prefixes. Any command whose path resolves to a mount outside that list is rejected with `mirage: session 'agent' not allowed to access mount '/X'` and exit code 1. Default sessions (no allowlist) keep their current unrestricted behavior, so existing code is unaffected.
|
||||
|
||||
This is a soft boundary, enforced inside the daemon process, not an OS or process-level isolation. Use it to shrink the blast radius of prompt-injection in multi-agent workspaces: a Slack-only agent cannot pivot to read `/linear`, `/github`, or any other mount it was not given.
|
||||
|
||||
The check fires for every code path that reaches a mount: shell commands (`cat`, `ls`, ...), redirects (`>`, `<`), cross-mount `cp`/`mv`, `wget -O`, `curl -o`, command substitution `$(...)`, subshells `(...)`, pipes, `&&`/`||` chains, background jobs, and the programmatic `ws.ops.read/write/...` API. Two infrastructure prefixes are always allowed regardless of the allowlist: the history view (`/.bash_history`, which the `history` builtin and the GNU histfile render from) and the virtual root (`/`, where stateless text-processing commands like `wc` resolve when given no path).
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python" icon="/images/python-logo.svg">
|
||||
```python
|
||||
ws = Workspace({
|
||||
"/s3": s3,
|
||||
"/slack": slack,
|
||||
"/linear": linear,
|
||||
})
|
||||
|
||||
ws.create_session("slack-agent", allowed_mounts={"/slack"})
|
||||
ws.create_session("data-agent", allowed_mounts={"/s3"})
|
||||
|
||||
await ws.execute("ls /slack", session_id="slack-agent") # ok
|
||||
await ws.execute("cat /linear/issues/SEC-42",
|
||||
session_id="slack-agent")
|
||||
# exit_code=1, stderr=b"session 'slack-agent' not allowed to "
|
||||
# b"access mount '/linear'\n"
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="CLI" icon="terminal">
|
||||
```bash
|
||||
# Repeat --mount (or -m) per allowed prefix
|
||||
mirage session create demo --id slack-agent --mount /slack
|
||||
mirage session create demo --id data-agent -m /s3 -m /github
|
||||
|
||||
mirage execute -w demo -s slack-agent -c "cat /linear/issues/SEC-42"
|
||||
# mirage: session 'slack-agent' not allowed to access mount '/linear'
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
The allowlist is a property of the session, so it covers every command issued under that `session_id`, including subshells, pipelines, and recursive `bash -c '...'`. It does not change `MountMode`: a write to a mount in the allowlist is still rejected if the mount is `READ`. The two checks compose.
|
||||
|
||||
## Agent Pattern
|
||||
|
||||
Agent harnesses commonly fan out tool calls in parallel, each with its own `cwd`/`env`/`cancel`. The clone semantics make this race-free without per-call boilerplate. From the CLI, a subshell per call gives the same isolation.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python" icon="/images/python-logo.svg">
|
||||
```python
|
||||
async def tool_call(cmd: str, cwd: str, env: dict[str, str], timeout: float):
|
||||
cancel = asyncio.Event()
|
||||
asyncio.get_event_loop().call_later(timeout, cancel.set)
|
||||
return await ws.execute(cmd, cwd=cwd, env=env, cancel=cancel)
|
||||
|
||||
results = await asyncio.gather(
|
||||
tool_call("ls", "/data", {"DEBUG": "1"}, 5.0),
|
||||
tool_call("grep foo *.log", "/logs", {"DEBUG": "1"}, 5.0),
|
||||
)
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="TypeScript" icon="/images/typescript-logo.svg">
|
||||
```typescript
|
||||
async function toolCall(
|
||||
cmd: string,
|
||||
cwd: string,
|
||||
env: Record<string, string>,
|
||||
timeoutMs: number,
|
||||
) {
|
||||
return ws.execute(cmd, { cwd, env, signal: AbortSignal.timeout(timeoutMs) })
|
||||
}
|
||||
|
||||
const results = await Promise.all([
|
||||
toolCall("ls", "/data", { DEBUG: "1" }, 5000),
|
||||
toolCall("grep foo *.log", "/logs", { DEBUG: "1" }, 5000),
|
||||
])
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="CLI" icon="terminal">
|
||||
```bash
|
||||
# No --cwd/--env flags: isolate each parallel call in a subshell
|
||||
mirage execute -w demo -c "(cd /data && export DEBUG=1 && ls) & (cd /logs && export DEBUG=1 && grep foo *.log) & wait"
|
||||
```
|
||||
Each `( ... )` runs in its own scope, so the parallel `cd` / `export` don't collide. `&` backgrounds them inside Mirage Bash and `wait` joins.
|
||||
</Tab>
|
||||
</Tabs>
|
||||
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: Cache
|
||||
description: The two-layer workspace cache, index and file, with RAM and Redis stores.
|
||||
icon: database
|
||||
---
|
||||
|
||||
## What It Does
|
||||
|
||||
Every `Workspace` ships with a **two-layer cache** so repeated work against remote backends (S3, GDrive, Slack, ...) hits local state instead of the network:
|
||||
|
||||
- **Index cache.** Listings and metadata. The first directory walk hits the API; subsequent ones serve from the index until the TTL expires.
|
||||
- **File cache.** Object bytes. The first read streams from origin; later pipelines read from cache.
|
||||
|
||||
## Stores
|
||||
|
||||
Each layer is a pluggable store with two built-ins:
|
||||
|
||||
- **RAM** (default): in-process, zero setup, 512 MB file cache and 10-minute index TTL. Best for single-process apps and notebooks.
|
||||
- **Redis**: shared across workers, processes, and machines. Best for serverless, multi-replica services, or for cache state that survives restarts.
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```python Python
|
||||
from mirage import Workspace
|
||||
from mirage.cache.file.config import RedisCacheConfig
|
||||
from mirage.cache.index.config import RedisIndexConfig
|
||||
from mirage.resource.s3 import S3Config, S3Resource
|
||||
|
||||
ws = Workspace(
|
||||
{"/s3": S3Resource(S3Config(bucket="my-bucket"))},
|
||||
cache=RedisCacheConfig(url="redis://localhost:6379/0", limit="8GB"),
|
||||
index=RedisIndexConfig(url="redis://localhost:6379/0", ttl=600),
|
||||
)
|
||||
```
|
||||
|
||||
```typescript TypeScript
|
||||
import { RedisFileCacheStore, S3Resource, Workspace } from '@struktoai/mirage-node'
|
||||
|
||||
const ws = new Workspace(
|
||||
{ '/s3': new S3Resource({ bucket: 'my-bucket' }) },
|
||||
{
|
||||
cache: new RedisFileCacheStore({ url: 'redis://localhost:6379/0', cacheLimit: '8GB' }),
|
||||
index: { type: 'redis', url: 'redis://localhost:6379/0', ttl: 600 },
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
## Eviction & Limits
|
||||
|
||||
The two layers are bounded differently:
|
||||
|
||||
| Layer | Holds | Default | Bound | Eviction |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| **File cache** | object bytes per virtual path | RAM, 512 MB | `cache_limit` (Py) / `cacheLimit` (TS) | LRU: least-recently-used bytes drop once the total exceeds the limit |
|
||||
| **Index cache** | directory listings + `FileStat` metadata | RAM, 10-min TTL | `ttl` (seconds) | time-based: entries expire after the TTL, then re-fetch on next access |
|
||||
|
||||
Raising the file limit keeps more bytes warm at the cost of memory; lengthening the index TTL serves listings longer between API walks at the cost of staleness.
|
||||
|
||||
## Miss/Hit Lifecycle
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```python Python
|
||||
from mirage import Workspace
|
||||
from mirage.resource.s3 import S3Config, S3Resource
|
||||
|
||||
ws = Workspace({"/s3": S3Resource(S3Config(bucket="my-bucket"))})
|
||||
|
||||
# 1. Index miss → S3 LIST. Listing stored in index cache.
|
||||
await ws.execute("ls /s3/data/")
|
||||
|
||||
# 2. Index hit → 0 network calls.
|
||||
await ws.execute('find /s3/data/ -name "*.jsonl"')
|
||||
|
||||
# 3. File miss → S3 GET. Bytes stored in file cache.
|
||||
await ws.execute("cat /s3/data/log.jsonl | wc -l")
|
||||
|
||||
# 4. File hit → 0 network calls.
|
||||
await ws.execute("grep alert /s3/data/log.jsonl")
|
||||
```
|
||||
|
||||
```typescript TypeScript
|
||||
import { S3Resource, Workspace } from '@struktoai/mirage-node'
|
||||
|
||||
const ws = new Workspace({ '/s3': new S3Resource({ bucket: 'my-bucket' }) })
|
||||
|
||||
// 1. Index miss → S3 LIST. Listing stored in index cache.
|
||||
await ws.execute('ls /s3/data/')
|
||||
|
||||
// 2. Index hit → 0 network calls.
|
||||
await ws.execute('find /s3/data/ -name "*.jsonl"')
|
||||
|
||||
// 3. File miss → S3 GET. Bytes stored in file cache.
|
||||
await ws.execute('cat /s3/data/log.jsonl | wc -l')
|
||||
|
||||
// 4. File hit → 0 network calls.
|
||||
await ws.execute('grep alert /s3/data/log.jsonl')
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
## Relationship To Snapshots
|
||||
|
||||
The file cache is exactly what a [snapshot](/home/snapshot) serializes: `ws.snapshot()` writes the cached bytes for every touched path into the tar, and `Workspace.load()` restores them into the file cache so a replayed run reads from local state. The index cache is not snapshotted; it rebuilds lazily after load.
|
||||
@@ -0,0 +1,439 @@
|
||||
---
|
||||
title: CLI
|
||||
description: Drive Mirage from the shell. Spin up a workspace from YAML, run commands against your mounts, snapshot and restore.
|
||||
icon: terminal
|
||||
---
|
||||
|
||||
The `mirage` CLI is a thin httpx wrapper over the Mirage daemon. It
|
||||
auto-spawns the daemon on first `workspace create`, and the daemon
|
||||
auto-exits 30 seconds after the last workspace is deleted. Most users
|
||||
never type a daemon command directly.
|
||||
|
||||
Output is structured JSON to stdout for every verb -- pipe to `jq`,
|
||||
save to file, or read it directly.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
curl -fsSL https://strukto.ai/mirage/install.sh | sh
|
||||
# or
|
||||
npm install -g @struktoai/mirage-cli
|
||||
# or
|
||||
uvx mirage-ai
|
||||
# or
|
||||
npx @struktoai/mirage-cli
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
mirage --help
|
||||
```
|
||||
|
||||
## Define a workspace in YAML
|
||||
|
||||
A workspace is a set of prefixed mounts plus some workspace-level
|
||||
settings. Save this as `workspace.yaml`:
|
||||
|
||||
```yaml
|
||||
mode: WRITE
|
||||
|
||||
mounts:
|
||||
/:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
/s3:
|
||||
resource: s3
|
||||
mode: READ
|
||||
config:
|
||||
bucket: ${AWS_S3_BUCKET}
|
||||
region: ${AWS_DEFAULT_REGION}
|
||||
aws_access_key_id: ${AWS_ACCESS_KEY_ID}
|
||||
aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
|
||||
```
|
||||
|
||||
`${VAR}` placeholders are interpolated from your shell environment at
|
||||
`mirage workspace create` time. Missing vars fail fast with the full
|
||||
list, not lazily on first use.
|
||||
|
||||
## Walkthrough
|
||||
|
||||
A guided tour from creating a workspace to snapshotting it. Each
|
||||
step builds on the previous one; you can copy them in order.
|
||||
|
||||
For a runnable end-to-end version against a real multi-mount workspace
|
||||
(`/s3`, `/gdrive`, `/gmail`, `/slack`, `/discord`), see
|
||||
<a href="https://github.com/strukto-ai/mirage/blob/main/examples/python/cross/README.md" target="_blank" rel="noopener noreferrer"><code>examples/python/cross/README.md</code></a>.
|
||||
|
||||
### 1. Source env and create a workspace
|
||||
|
||||
The YAML's `${...}` placeholders resolve from your shell at create
|
||||
time, so source your env first. The daemon auto-spawns on the first
|
||||
`create`.
|
||||
|
||||
```bash
|
||||
set -a && source .env.development && set +a
|
||||
|
||||
mirage workspace create workspace.yaml --id demo
|
||||
```
|
||||
|
||||
### 2. Inspect
|
||||
|
||||
`list` is one line per workspace; `get` returns the full mount and
|
||||
session detail.
|
||||
|
||||
```bash
|
||||
mirage workspace list
|
||||
mirage workspace get demo
|
||||
```
|
||||
|
||||
### 3. Run commands against your mounts
|
||||
|
||||
`execute` runs a shell command inside the workspace. Paths resolve
|
||||
through the mount registry (`/s3/...` hits S3, `/` hits the RAM
|
||||
backing, etc.).
|
||||
|
||||
```bash
|
||||
mirage execute --workspace_id demo --command "ls /s3/"
|
||||
mirage execute --workspace_id demo --command "head -n 1 /s3/data/example.jsonl"
|
||||
```
|
||||
|
||||
### 4. Pipe stdin
|
||||
|
||||
When stdout isn't a TTY, the CLI forwards stdin to the command
|
||||
automatically.
|
||||
|
||||
```bash
|
||||
echo -e "a\nb\nc" | mirage execute --workspace_id demo --command "wc -l"
|
||||
```
|
||||
|
||||
### 5. Dry-run with `provision`
|
||||
|
||||
`provision` returns a `ProvisionResult` (network bytes, cache hits,
|
||||
estimated cost) without running the command -- handy for predicting
|
||||
spend before kicking off an expensive read.
|
||||
|
||||
```bash
|
||||
mirage provision --workspace_id demo \
|
||||
--command "cat /s3/data/example.jsonl | wc -l"
|
||||
```
|
||||
|
||||
Every factory-built backend estimates commands out of the box, by
|
||||
family: whole-file readers (`cat`, `sort`, `md5`, ...) charge the byte
|
||||
total from `stat`, `head`/`tail`/`file` charge a bounded range,
|
||||
`grep`/`rg` charge a worst-case full read, and metadata commands (`ls`,
|
||||
`find`, `stat`, `du`, ...) charge op counts only. Transforms (`gzip`,
|
||||
`tar`, `split`, ...) keep the read total as a floor with
|
||||
`precision=unknown` output; `cp` brackets both read and write between 0
|
||||
(server-side copy) and the source total; metadata writes (`rm`,
|
||||
`mkdir`, `touch`, ...) are zero-byte op counts, with recursive `rm`
|
||||
degrading to a floor; pure commands (`seq`, `date`, `bc`, `expr`) and
|
||||
shell builtins (`echo`, `cd`, ...) are zero-cost. Anything the planner
|
||||
cannot estimate honestly -- `mv` (free rename or full cross-mount
|
||||
copy), `tee` (stdin size), arbitrary programs -- reports
|
||||
`precision=unknown` with all totals as floors, never an error. Virtual
|
||||
files whose size cannot be resolved (for example a rendered
|
||||
`chat.jsonl`) degrade the estimate to `precision=unknown` while keeping
|
||||
the known byte total as a floor.
|
||||
|
||||
Provision is optional when you register your own commands: leave it out
|
||||
and the planner reports `precision=unknown`. To opt in with one line,
|
||||
reuse the estimator helpers (`make_file_read_provision`,
|
||||
`make_search_provision`, `metadata_provision`, ... in Python;
|
||||
`makeFileReadProvision` and friends in TypeScript), or pass
|
||||
`provision_overrides={"grep": my_estimator}` to the command factory. An
|
||||
explicit `None`/`null` override disables a default.
|
||||
|
||||
Pipelines combine field-wise: `|`, `;` and `&&` sum the estimates, `||`
|
||||
brackets the branches (cheapest low, priciest high), and `for` loops
|
||||
multiply by the iteration count. A stage downstream of an unknown stage
|
||||
is also unknown, and totals under `precision=unknown` are floors.
|
||||
|
||||
### 6. Cache: network → hit after a real read
|
||||
|
||||
After a real `cat`, `provision` flips that path from a network read
|
||||
to a cache hit (`cache_hits=1`).
|
||||
|
||||
```bash
|
||||
mirage execute --workspace_id demo --command "cat /s3/data/example.jsonl > /dev/null"
|
||||
mirage provision --workspace_id demo --command "cat /s3/data/example.jsonl"
|
||||
```
|
||||
|
||||
### 7. Command history
|
||||
|
||||
Every executed command is recorded by a hidden recorder (the
|
||||
[Observer](/home/observer)). The `history` builtin shows the calling
|
||||
session's commands (GNU bash semantics, `history -c` clears only that
|
||||
session's view), and `/.bash_history` renders the GNU histfile across all
|
||||
sessions, readable with the ordinary file commands.
|
||||
|
||||
```bash
|
||||
mirage execute --workspace_id demo --command "history 5"
|
||||
mirage execute --workspace_id demo --command "tail -n 6 /.bash_history"
|
||||
mirage execute --workspace_id demo --command "grep cat /.bash_history"
|
||||
```
|
||||
|
||||
### 8. Background jobs
|
||||
|
||||
Long-running commands take `--background` and return a `job_id`
|
||||
immediately. `mirage job wait` blocks until it's done.
|
||||
|
||||
```bash
|
||||
JOB=$(mirage execute --workspace_id demo --background \
|
||||
--command "wc -l /s3/data/example.jsonl" \
|
||||
| jq -r .job_id)
|
||||
mirage job wait $JOB
|
||||
```
|
||||
|
||||
### 9. Snapshot to disk
|
||||
|
||||
```bash
|
||||
mirage workspace snapshot demo /tmp/demo.tar
|
||||
```
|
||||
|
||||
### 10. Restore from snapshot
|
||||
|
||||
Snapshots redact cloud creds at save time, so loading needs fresh
|
||||
creds via a config file. The same workspace YAML used for create works.
|
||||
|
||||
```bash
|
||||
mirage workspace load /tmp/demo.tar workspace.yaml \
|
||||
--id demo_loaded
|
||||
|
||||
mirage workspace get demo_loaded --verbose
|
||||
mirage execute --workspace_id demo_loaded \
|
||||
--command "head -n 1 /s3/data/example.jsonl"
|
||||
```
|
||||
|
||||
<Note>
|
||||
A config file is required for any mount whose snapshot config contains
|
||||
redacted secrets (S3 with inline keys, GDrive, Slack, Discord, Redis,
|
||||
...). The snapshot stores `"<REDACTED>"` in place of those secrets at
|
||||
save time. Loading without the config 400s with the list of prefixes
|
||||
that need fresh creds. Local resources (RAM, Disk) restore as-is with
|
||||
no extra config needed.
|
||||
</Note>
|
||||
|
||||
### 11. Clean up
|
||||
|
||||
The daemon exits ~30s after the last workspace is deleted.
|
||||
|
||||
```bash
|
||||
mirage workspace delete demo
|
||||
mirage workspace delete demo_loaded
|
||||
```
|
||||
|
||||
## Versioning
|
||||
|
||||
Every workspace has its own git-backed history kept by the daemon (under
|
||||
`~/.mirage/repos/<workspace-id>` by default; set `MIRAGE_VERSION_ROOT` to
|
||||
relocate). You commit the live state as a version, then log, diff, branch,
|
||||
and restore in place. The verbs follow git.
|
||||
|
||||
### Commit and log
|
||||
|
||||
```bash
|
||||
mirage execute --workspace_id demo --command "echo v1 > /notes.txt"
|
||||
mirage workspace commit demo -m "first"
|
||||
|
||||
mirage execute --workspace_id demo --command "echo v2 > /notes.txt"
|
||||
mirage workspace commit demo -m "second"
|
||||
|
||||
mirage workspace log demo # newest first: "second", then "first"
|
||||
```
|
||||
|
||||
### Diff
|
||||
|
||||
`diff` reports changed paths (added / modified / deleted), git-style. No refs
|
||||
compares live state to the branch HEAD; one ref compares live to that ref; two
|
||||
refs compare two versions.
|
||||
|
||||
```bash
|
||||
mirage workspace diff demo # live vs HEAD
|
||||
mirage workspace diff demo <v1> # live vs <v1>
|
||||
mirage workspace diff demo <v1> <v2> # <v1> vs <v2>
|
||||
```
|
||||
|
||||
### Branch
|
||||
|
||||
`branch` forks at another branch's current version; commit onto it with `-b`.
|
||||
|
||||
```bash
|
||||
mirage workspace branch demo exp # fork exp from main
|
||||
mirage workspace branch demo exp2 --from exp # fork from a non-main branch
|
||||
mirage workspace commit demo -b exp -m "on exp" # commit onto exp
|
||||
mirage workspace log demo -b exp # log a specific branch
|
||||
```
|
||||
|
||||
### Checkout
|
||||
|
||||
`checkout` restores the live state to a past version or branch, in place.
|
||||
It overwrites uncommitted live state.
|
||||
|
||||
```bash
|
||||
mirage workspace checkout demo <version> # by version id
|
||||
mirage workspace checkout demo main # by branch name
|
||||
```
|
||||
|
||||
### Clone from a version
|
||||
|
||||
`clone --at` makes a new workspace from one of the source's past versions
|
||||
(omit `--at` to clone the live state).
|
||||
|
||||
```bash
|
||||
mirage workspace clone demo --at <version> --id demo_at_v1
|
||||
```
|
||||
|
||||
## Verbs at a glance
|
||||
|
||||
| Verb | What it does |
|
||||
| --- | --- |
|
||||
| `mirage workspace create FILE [--id NAME]` | Build resources from YAML, register a workspace, return its id. |
|
||||
| `mirage workspace list` | Brief one-line summary per active workspace. |
|
||||
| `mirage workspace get ID [--verbose]` | Full detail (mounts, sessions, optionally cache / dirty / history internals). |
|
||||
| `mirage workspace delete ID` | Stop the workspace; daemon may exit on the idle timer. |
|
||||
| `mirage workspace clone SRC_ID [--id NAME] [--at REF]` | New workspace from the source; `--at REF` clones a past version, otherwise the live state. |
|
||||
| `mirage workspace commit ID [-m MSG] [-b BRANCH]` | Commit the live state as a version; returns the version id. |
|
||||
| `mirage workspace log ID [-b BRANCH]` | List versions on a branch, newest first. |
|
||||
| `mirage workspace diff ID [A] [B] [-b BRANCH]` | Changed paths (added/modified/deleted). No refs: live vs HEAD; one ref: live vs A; two: A vs B. |
|
||||
| `mirage workspace branch ID NAME [--from BRANCH]` | Fork a branch at another branch's current version. |
|
||||
| `mirage workspace checkout ID REF` | Restore the live state in place to a version id or branch. |
|
||||
| `mirage workspace snapshot ID PATH.tar` | Snapshot to a tar file. |
|
||||
| `mirage workspace load PATH.tar [CONFIG] [--id NAME]` | Restore from tar; optional config re-supplies redacted creds. |
|
||||
| `mirage session create WS [--id NAME]` | Add a named session (own cwd + env). |
|
||||
| `mirage session list WS` | List sessions for a workspace. |
|
||||
| `mirage session delete WS SESSION` | Close a session. |
|
||||
| `mirage execute --workspace_id WS [--session_id S] [--background] --command "..."` | Run a command. Pipes stdin automatically when stdout is not a TTY. |
|
||||
| `mirage provision --workspace_id WS [--session_id S] --command "..."` | Dry-run / cost estimate -- returns a `ProvisionResult` shape (network bytes, cache hits, estimated cost) without running the command. |
|
||||
| `mirage job list [--workspace_id WS]` | List jobs the daemon has run, plus their status. |
|
||||
| `mirage job get JOB` | Detail for one job. |
|
||||
| `mirage job wait JOB [--timeout SECS]` | Block until the job is done; returns the result. |
|
||||
| `mirage job cancel JOB` | Cancel a running job. |
|
||||
|
||||
## Per-mount safeguards
|
||||
|
||||
<Note>
|
||||
Per-mount `command_safeguards` are **Python CLI only** today. The TypeScript
|
||||
CLI config schema does not carry them yet.
|
||||
</Note>
|
||||
|
||||
Cap what a command may stream back per mount with `command_safeguards`, so a
|
||||
runaway `cat`/`grep`/`rg` can't flood the agent or hang. Each entry sets
|
||||
`max_lines` / `max_bytes` (output cap) and/or `timeout_seconds` (deadline),
|
||||
with `on_exceed: truncate` (stop, exit 0, add a stderr notice) or
|
||||
`on_exceed: error` (stop, exit 1):
|
||||
|
||||
```yaml
|
||||
mounts:
|
||||
/data:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
command_safeguards:
|
||||
head: # cap output, keep going
|
||||
max_lines: 100
|
||||
on_exceed: truncate
|
||||
grep: # cap output, fail hard
|
||||
max_lines: 50
|
||||
on_exceed: error
|
||||
rg: # deadline in seconds
|
||||
timeout_seconds: 30
|
||||
```
|
||||
|
||||
Caps fire on the **terminal** command of a pipeline only, so
|
||||
`cat big.txt | head -n 30` still shows 30 lines. Truncation exits `0`, `error`
|
||||
exits `1`, and a timeout exits `124` -- each with a stderr notice. Without a
|
||||
`command_safeguards` block, `cat`/`grep`/`rg`/`head`/`tail` still cap at 2000
|
||||
lines by default. See [Output Safeguards](/python/quickstart#output-safeguards)
|
||||
for the SDK form and the same fields.
|
||||
|
||||
## Daemon control
|
||||
|
||||
Most users never run these directly -- the daemon auto-spawns on
|
||||
first `workspace create` and auto-exits after the idle timer fires
|
||||
(default 30s after the last workspace is deleted). When you need to
|
||||
intervene -- typically during development, when you want code changes
|
||||
to take effect:
|
||||
|
||||
```bash
|
||||
mirage daemon status # health, PID, uptime, workspace count
|
||||
mirage daemon stop # graceful: trip exit event, falls back to SIGTERM after --timeout (default 5s)
|
||||
mirage daemon restart # stop + lazy respawn (add --eager to spawn now)
|
||||
mirage daemon kill # SIGKILL via PID file -- last resort
|
||||
```
|
||||
|
||||
| Verb | What it does |
|
||||
| --- | --- |
|
||||
| `mirage daemon status` | Daemon health, PID, uptime, workspace count. Exit 1 if daemon not reachable. |
|
||||
| `mirage daemon stop` | `POST /v1/shutdown` to trip the daemon's exit event. Daemon closes active workspaces and exits. Falls back to SIGTERM on `--timeout`. |
|
||||
| `mirage daemon restart` | Stop, then either wait for next `workspace create` to auto-spawn (default) or `--eager` to spawn immediately. Workspaces are LOST on restart -- save any you want to keep first with `mirage workspace snapshot <id> <path>` and reload with `mirage workspace load <path>`. |
|
||||
| `mirage daemon kill` | SIGKILL via PID file at `~/.mirage/daemon.pid`. Skips graceful shutdown -- use only when `stop` hangs. |
|
||||
|
||||
<Note>
|
||||
When you change Mirage's source code (commands, providers, etc.),
|
||||
the running daemon won't see your changes -- it loaded the old code
|
||||
at startup. Run `mirage daemon restart` to pick up new code.
|
||||
Same situation as `dockerd` after recompiling Docker.
|
||||
</Note>
|
||||
|
||||
## Where the daemon lives
|
||||
|
||||
Most users never need to think about the daemon. If you do:
|
||||
|
||||
- It listens on `http://127.0.0.1:8765` by default.
|
||||
- Override via env vars or `~/.mirage/config.toml`, edited with
|
||||
`mirage config` (the `git config` of Mirage):
|
||||
```bash
|
||||
mirage config set port 9100
|
||||
mirage config set version_root /data/repos
|
||||
mirage config get version_root # one key, exit 1 if unset
|
||||
mirage config list # everything written in the file
|
||||
mirage config unset port # remove a key
|
||||
mirage config list --resolved # effective values + where each came from
|
||||
```
|
||||
```toml
|
||||
[daemon]
|
||||
url = "http://127.0.0.1:8765"
|
||||
idle_grace_seconds = 30
|
||||
port = 9100
|
||||
version_root = "/data/repos"
|
||||
```
|
||||
- Per key the precedence is: env var > `config.toml` > default.
|
||||
Because env vars silently beat the file, `mirage config list`
|
||||
(which shows only the file) can look right while the daemon uses
|
||||
something else. `--resolved` shows the value each key will
|
||||
actually get and which source won:
|
||||
|
||||
```bash
|
||||
$ mirage config set version_root /data/repos
|
||||
$ export MIRAGE_VERSION_ROOT=/tmp/other # e.g. left over in your shell profile
|
||||
|
||||
$ mirage config list # the file looks correct...
|
||||
version_root = /data/repos
|
||||
|
||||
$ mirage config list --resolved # ...but the env var wins
|
||||
version_root = /tmp/other (env MIRAGE_VERSION_ROOT)
|
||||
port = 8765 (default)
|
||||
url = http://127.0.0.1:8765 (default)
|
||||
auth_token = *** (env MIRAGE_TOKEN)
|
||||
```
|
||||
|
||||
Use it whenever a setting seems ignored: the origin column names
|
||||
the exact env var overriding you. Secrets are masked, and piped
|
||||
output is JSON (`{"version_root": {"value": ..., "origin": ...}}`)
|
||||
so it works with `jq`.
|
||||
- Allowed keys: `url`, `socket`, `auth_token`, `auth_mode`,
|
||||
`allowed_hosts`, `idle_grace_seconds`, `port`, `pid_file`,
|
||||
`version_root`, `snapshot_root`, and the `jwt_*` family
|
||||
(`jwt_alg`, `jwt_issuer`, `jwt_audience`, `jwt_pubkey_file`,
|
||||
`jwt_clock_skew`, `jwt_authorized_parties`). `MIRAGE_HOME` and raw
|
||||
secrets (`MIRAGE_AUTH_TOKEN`, `MIRAGE_JWT_PUBKEY`) stay env-only.
|
||||
- Like `dockerd` with a bad `daemon.json`, the daemon refuses to
|
||||
start on unknown keys or malformed TOML, naming the offender.
|
||||
`mirage config unset <key>` accepts unknown keys so you can repair
|
||||
the file; `mirage config list` warns about them.
|
||||
- Path settings (`pid_file`, `version_root`, `snapshot_root`, `port`)
|
||||
take effect on the next daemon start; there is no hot reload.
|
||||
- `mirage config set` chmods the file to `0600` since it may hold
|
||||
`auth_token`.
|
||||
- Logs go to `~/.mirage/daemon.log` when the CLI auto-spawns it.
|
||||
- It exits 30 seconds after the workspace count hits zero (configurable).
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: Installation
|
||||
description: Install Mirage with the Python package mirage-ai or the TypeScript packages @struktoai/mirage-node and @struktoai/mirage-browser.
|
||||
keywords: ["install Mirage", "mirage-ai", "@struktoai/mirage-node", "virtual filesystem", "AI agents"]
|
||||
icon: download
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Python** ≥ 3.11 for the `mirage-ai` package and the `mirage` CLI
|
||||
- **Node.js** ≥ 20 for the TypeScript SDK
|
||||
- **macOS** or **Linux** (FUSE-based mounts require platform support)
|
||||
|
||||
## Python
|
||||
|
||||
```bash
|
||||
uv add mirage-ai
|
||||
```
|
||||
|
||||
This installs both the `mirage` library and the `mirage` CLI binary.
|
||||
|
||||
## TypeScript
|
||||
|
||||
Pick the package that matches your runtime — `@struktoai/mirage-core` is auto-pulled by both Node and browser entrypoints.
|
||||
|
||||
```bash
|
||||
npm install @struktoai/mirage-node # Node.js servers and CLIs
|
||||
npm install @struktoai/mirage-browser # browser / edge runtimes
|
||||
npm install @struktoai/mirage-agents # OpenAI / Vercel AI / LangChain / Mastra adapters
|
||||
```
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
curl -fsSL https://strukto.ai/mirage/install.sh | sh
|
||||
# or
|
||||
npm install -g @struktoai/mirage-cli
|
||||
# or
|
||||
uvx mirage-ai
|
||||
# or
|
||||
npx @struktoai/mirage-cli
|
||||
```
|
||||
|
||||
## More Details
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Python" icon="/images/python-logo.svg" href="/python/install">
|
||||
Resource extras, virtualenv setup, and `uv` workflow.
|
||||
</Card>
|
||||
<Card title="TypeScript" icon="/images/typescript-logo.svg" href="/typescript/install">
|
||||
Native peers (FUSE, Redis) and per-runtime notes for Node, browser, and edge.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
@@ -0,0 +1,388 @@
|
||||
---
|
||||
title: Introduction
|
||||
description: Unified Virtual Filesystem for AI Agents.
|
||||
icon: /images/mirage-icon-light.svg
|
||||
mode: "frame"
|
||||
---
|
||||
|
||||
<div className="px-4 py-1 mt-0 lg:pb-16 max-w-6xl mx-auto">
|
||||
|
||||
<div className="mx-auto w-full max-w-[1200px]">
|
||||
<img noZoom src="/images/mirage-text-logo-light.svg" alt="Mirage" className="block dark:hidden w-full" />
|
||||
<img noZoom src="/images/mirage-text-logo-dark.svg" alt="Mirage" className="hidden dark:block w-full" />
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
<div className="max-w-6xl mx-auto px-4 lg:px-6 text-black dark:text-white">
|
||||
|
||||
<h2 className="text-2xl font-bold mb-6">Mirage in 30 Seconds</h2>
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mb-4">
|
||||
<p className="my-0">
|
||||
Mount your resources, then run shell commands across them. The same <code>echo</code>, <code>ls</code>, <code>grep</code> work against an in-memory mount, <Icon icon="aws" /> S3, <Icon icon="slack" /> Slack, etc.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python" icon="/images/python-logo.svg">
|
||||
```python
|
||||
import os
|
||||
|
||||
from mirage import Workspace
|
||||
from mirage.resource.ram import RAMResource
|
||||
from mirage.resource.s3 import S3Config, S3Resource
|
||||
from mirage.resource.slack import SlackConfig, SlackResource
|
||||
|
||||
ws = Workspace({
|
||||
"/data": RAMResource(),
|
||||
"/s3": S3Resource(S3Config(bucket="my-bucket")),
|
||||
"/slack": SlackResource(SlackConfig(token=os.environ["SLACK_BOT_TOKEN"])),
|
||||
})
|
||||
|
||||
# Same shell vocabulary, three different backends.
|
||||
await ws.execute('echo "hello mirage" > /data/hello.txt')
|
||||
await ws.execute("ls /s3/reports/")
|
||||
result = await ws.execute('grep -r "release" /slack/channels/eng__C04QX')
|
||||
print(await result.stdout_str())
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="TypeScript" icon="/images/typescript-logo.svg">
|
||||
```typescript
|
||||
import {
|
||||
RAMResource,
|
||||
S3Resource,
|
||||
SlackResource,
|
||||
Workspace,
|
||||
} from '@struktoai/mirage-node'
|
||||
|
||||
const ws = new Workspace({
|
||||
'/data': new RAMResource(),
|
||||
'/s3': new S3Resource({ bucket: 'my-bucket' }),
|
||||
'/slack': new SlackResource({ token: process.env.SLACK_BOT_TOKEN! }),
|
||||
})
|
||||
|
||||
// Same shell vocabulary, three different backends.
|
||||
await ws.execute('echo "hello mirage" > /data/hello.txt')
|
||||
await ws.execute('ls /s3/reports/')
|
||||
const res = await ws.execute('grep -r "release" /slack/channels/eng__C04QX')
|
||||
console.log(new TextDecoder().decode(res.stdout))
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="CLI" icon="terminal">
|
||||
```bash
|
||||
cat > workspace.yaml <<'EOF'
|
||||
mode: WRITE
|
||||
mounts:
|
||||
/data:
|
||||
resource: ram
|
||||
mode: WRITE
|
||||
/s3:
|
||||
resource: s3
|
||||
mode: READ
|
||||
config:
|
||||
bucket: ${AWS_S3_BUCKET}
|
||||
/slack:
|
||||
resource: slack
|
||||
mode: READ
|
||||
config:
|
||||
token: ${SLACK_BOT_TOKEN}
|
||||
EOF
|
||||
|
||||
mirage workspace create workspace.yaml --id demo
|
||||
mirage execute -w demo -c 'echo "hello mirage" > /data/hello.txt'
|
||||
mirage execute -w demo -c 'ls /s3/reports/'
|
||||
mirage execute -w demo -c 'grep -r "release" /slack/channels/eng__C04QX'
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mt-4 mb-2">
|
||||
<p className="my-0">
|
||||
Add GitHub, Postgres, SSH, Notion, Google Drive, ... and the same shell vocabulary keeps working. That's the whole pitch.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h2 className="text-2xl font-bold mt-16 mb-6">What is Mirage?</h2>
|
||||
|
||||
<div className="max-w-none space-y-1 text-[15px] leading-7">
|
||||
<p className="my-0">
|
||||
<img noZoom src="/images/mirage-icon-light.svg" alt="Mirage" className="inline-block h-4 align-middle mr-1 dark:hidden" /><img noZoom src="/images/mirage-icon-dark.svg" alt="Mirage" className="hidden h-4 align-middle mr-1 dark:inline-block" /><b>Mirage</b> is a <b>Unified Virtual Filesystem</b> for AI agents. It mounts your apps, services, and systems behind one filesystem interface, so an agent reaches every backend with the same handful of Unix-like tools instead of a new SDK per service.
|
||||
</p>
|
||||
<div className="space-y-3">
|
||||
<div>
|
||||
<h4 className="text-lg font-bold mt-4 mb-1">One Filesystem</h4>
|
||||
<p className="my-0">
|
||||
Every service speaks the same filesystem semantics, so agents reason about one abstraction instead of N SDKs and M MCPs. S3, R2, Google Drive, GitHub, Linear, Notion, Slack, Discord, MongoDB, Redis, SSH, and more mount side-by-side under a single root.
|
||||
</p>
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-lg font-bold mt-4 mb-1">Familiar Bash Tools</h4>
|
||||
<p className="my-0">
|
||||
Agents reuse the same handful of Unix-like tools (<code>ls</code>, <code>find</code>, <code>grep</code>, <code>cat</code>, ...) instead of learning a new API per service. Pipelines compose across services as naturally as on a local disk, the exact corpus modern LLMs are most heavily trained on.
|
||||
</p>
|
||||
|
||||
```bash
|
||||
# Find every mention of "mirage" across three services
|
||||
grep -r "mirage" /slack /gmail /github
|
||||
```
|
||||
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-lg font-bold mt-4 mb-1">Portable, Versioned Workspaces</h4>
|
||||
<p className="my-0">
|
||||
Snapshot, clone, and version a workspace the way git treats source. Move agent runs between machines without restarting, fork from any past state, and replay a run on demand.
|
||||
</p>
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-lg font-bold mt-4 mb-1">Embed in Apps and Agents</h4>
|
||||
<p className="my-0">
|
||||
Python and TypeScript SDKs give your AI agents a virtual filesystem directly inside FastAPI, Express, browser apps, or any async runtime, no separate process required. Works with the major agent frameworks (OpenAI Agents SDK, Vercel AI SDK, LangChain, Pydantic AI, CAMEL, OpenHands) and a lightweight CLI plugs into coding agents like <img noZoom src="/images/claude-logo.svg" alt="" className="inline-block h-4 align-middle mr-1" />Claude Code and <img noZoom src="/images/openai-logo.svg" alt="" className="inline-block h-4 align-middle mr-1 brightness-0 dark:invert" />Codex.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h2 className="text-2xl font-bold mt-16 mb-6">A Real-world Example</h2>
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mb-4">
|
||||
<p className="my-0">
|
||||
An agent watches your team's <Icon icon="slack" /> Slack <code>#incident</code> channel. A user posts a screenshot of <code>mirage --help</code> with the message <i>"the CLI design is confusing and hard to follow"</i>.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mt-6 mb-2">
|
||||
<p className="my-0">
|
||||
Built with the <a href="https://github.com/openai/openai-agents-python" target="_blank" rel="noopener noreferrer"><img noZoom src="/images/openai-logo.svg" alt="" className="inline-block h-4 align-middle mr-1 brightness-0 dark:invert" />OpenAI Agents SDK</a>, the agent walks <Icon icon="slack" /> Slack, <Icon icon="github" /> GitHub, and <Icon icon="linear" /> Linear through one bash tool.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h4 className="text-lg font-bold mt-8 mb-2">Code</h4>
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python" icon="/images/python-logo.svg">
|
||||
```python
|
||||
from agents import Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import SandboxAgent, SandboxRunConfig
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from mirage.agents.openai_agents import MirageSandboxClient
|
||||
from mirage.resource.github import GitHubResource
|
||||
from mirage.resource.linear import LinearResource
|
||||
from mirage.resource.slack import SlackResource
|
||||
|
||||
slack = SlackResource(...)
|
||||
github = GitHubResource(repo="strukto-ai/mirage")
|
||||
linear = LinearResource(...)
|
||||
|
||||
ws = Workspace({
|
||||
"/slack": (slack, MountMode.READ),
|
||||
"/github": (github, MountMode.READ),
|
||||
"/linear": (linear, MountMode.WRITE),
|
||||
})
|
||||
|
||||
agent = SandboxAgent(
|
||||
name="Design feedback triage",
|
||||
model="gpt-5.5",
|
||||
instructions=ws.file_prompt,
|
||||
)
|
||||
config = RunConfig(sandbox=SandboxRunConfig(client=MirageSandboxClient(ws)))
|
||||
|
||||
task = (
|
||||
"Scan recent messages in the Slack #incident channel. If anyone posts "
|
||||
"feedback about Mirage with a screenshot, read the image, locate the "
|
||||
"relevant CLI code in the Mirage GitHub repo, and file a design issue "
|
||||
"in Linear with the screenshot, the user's feedback, and links to the "
|
||||
"offending source files."
|
||||
)
|
||||
result = await Runner.run(agent, task, run_config=config)
|
||||
print(result.final_output)
|
||||
```
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mt-4 mb-4">
|
||||
<p className="my-0">
|
||||
Full runnable source: <a href="https://github.com/strukto-ai/mirage/blob/main/examples/python/demo/design_feedback.py" target="_blank" rel="noopener noreferrer"><code>examples/python/demo/design_feedback.py</code></a>.
|
||||
</p>
|
||||
</div>
|
||||
</Tab>
|
||||
<Tab title="TypeScript" icon="/images/typescript-logo.svg">
|
||||
```typescript
|
||||
import {
|
||||
GitHubResource,
|
||||
LinearResource,
|
||||
MountMode,
|
||||
SlackResource,
|
||||
Workspace,
|
||||
} from '@struktoai/mirage-node'
|
||||
import { Agent, run, shellTool } from '@openai/agents'
|
||||
import { MirageShell, buildSystemPrompt } from '@struktoai/mirage-agents/openai'
|
||||
|
||||
const slack = new SlackResource({ token: process.env.SLACK_BOT_TOKEN! })
|
||||
const github = new GitHubResource({
|
||||
token: process.env.GITHUB_TOKEN!,
|
||||
owner: 'strukto-ai',
|
||||
repo: 'mirage',
|
||||
})
|
||||
const linear = new LinearResource({ apiKey: process.env.LINEAR_API_KEY! })
|
||||
|
||||
const ws = new Workspace(
|
||||
{ '/slack': slack, '/github': github, '/linear': linear },
|
||||
{
|
||||
mode: MountMode.READ,
|
||||
modeOverrides: { '/linear': MountMode.WRITE },
|
||||
},
|
||||
)
|
||||
|
||||
const agent = new Agent({
|
||||
name: 'Design feedback triage',
|
||||
model: 'gpt-5.5',
|
||||
instructions: buildSystemPrompt({ workspace: ws }),
|
||||
tools: [shellTool({ shell: new MirageShell(ws) })],
|
||||
})
|
||||
|
||||
const task =
|
||||
'Scan recent messages in the Slack #incident channel. If anyone posts ' +
|
||||
'feedback about Mirage with a screenshot, read the image, locate the ' +
|
||||
'relevant CLI code in the Mirage GitHub repo, and file a design issue ' +
|
||||
'in Linear with the screenshot, the user\'s feedback, and links to the ' +
|
||||
'offending source files.'
|
||||
|
||||
const result = await run(agent, task)
|
||||
console.log(result.finalOutput)
|
||||
```
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mt-4 mb-4">
|
||||
<p className="my-0">
|
||||
Runnable source: <a href="https://github.com/strukto-ai/mirage/blob/main/examples/typescript/agents/openai/multi_resource_agent.ts" target="_blank" rel="noopener noreferrer"><code>examples/typescript/agents/openai/multi_resource_agent.ts</code></a>, the same shell-tool pattern over Slack and S3.
|
||||
</p>
|
||||
</div>
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
<h4 className="text-lg font-bold mt-10 mb-2">Walk-through</h4>
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mb-2">
|
||||
<p className="my-0">
|
||||
Inside the workspace shell, the agent runs three steps:
|
||||
</p>
|
||||
</div>
|
||||
|
||||
```bash
|
||||
# 1. Read the latest #incident message + list its attachments
|
||||
$ cat /slack/channels/incident__C0B0DB9K11T/2026-04-28/chat.jsonl
|
||||
$ ls /slack/channels/incident__C0B0DB9K11T/2026-04-28/files/
|
||||
# image__F0B01A3R171.png <- the agent reads this via the model's vision input
|
||||
|
||||
# 2. Find the CLI source the screenshot is complaining about
|
||||
$ rg -n "Mirage daemon CLI|workspace|session|provision" /github/typescript
|
||||
$ cat /github/typescript/packages/cli/src/main.ts
|
||||
|
||||
# 3. File a design issue in Linear with the feedback + code refs
|
||||
$ linear-issue-create --team_id <team-id> \
|
||||
--title "[Design] Rework Mirage CLI top-level command surface" \
|
||||
--description "$(cat <<'EOF'
|
||||
... feedback, screenshot summary, and links to the offending files ...
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mt-6 mb-2">
|
||||
<p className="my-0">
|
||||
The agent files a new issue in <Icon icon="linear" /> Linear with the user's feedback and links to the relevant source files.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h2 className="text-2xl font-bold mt-16 mb-6">Use Cases</h2>
|
||||
|
||||
<div className="max-w-none text-[15px] leading-7 mb-4">
|
||||
<p className="my-0">
|
||||
Mirage shows up wherever an agent needs to read, write, or stitch together data that doesn't already live on a local disk.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Cross-app triage agents" icon="bell">
|
||||
Watch Slack, search GitHub, file Linear issues. One shell, no per-service wiring.
|
||||
</Card>
|
||||
<Card title="SWE agents on remote data" icon="code">
|
||||
Point Claude Code or Codex at S3, Postgres, or SSH hosts as if they were files.
|
||||
</Card>
|
||||
<Card title="RAG with writes" icon="pen-to-square">
|
||||
READ + WRITE mounts on Notion, Drive, and Linear so the agent can edit and comment back.
|
||||
</Card>
|
||||
<Card title="Ops & observability copilots" icon="chart-line">
|
||||
<code>tail</code>, <code>grep</code>, <code>jq</code> over remote logs, metrics, and config without per-source plugins.
|
||||
</Card>
|
||||
<Card title="Sandbox-native workflows" icon="box">
|
||||
Embed inside Daytona, E2B, Modal, or Vercel sandboxes as the data plane.
|
||||
</Card>
|
||||
<Card title="Reproducible agent runs" icon="wand-magic-sparkles">
|
||||
Snapshot, restore, and clone workspaces for branchable, replayable agent runs.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
<h2 className="text-2xl font-bold mt-16 mb-6">FAQ</h2>
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="Do I need FUSE?">
|
||||
No. Mirage runs the workspace in-process: <code>ws.execute(...)</code> parses and dispatches commands without mounting anything on the host. FUSE is an optional surface if you also want host tools (editors, language servers, <code>rg</code>) to see the workspace.
|
||||
</Accordion>
|
||||
<Accordion title="Where does the shell run? Is it really bash?">
|
||||
The shell runs in your Mirage process. It's a tree-sitter bash parser plus a custom executor that routes commands to per-mount handlers, so there is no subshell to <code>/bin/bash</code> and no <code>os.system</code>. Most common Unix verbs work (<code>ls</code>, <code>cat</code>, <code>grep</code>, <code>find</code>, <code>head</code>, <code>wc</code>, <code>jq</code>, ...) plus pipes, redirects, globs, and <code>&&</code>/<code>||</code>.
|
||||
</Accordion>
|
||||
<Accordion title="Is it sandboxed?">
|
||||
The shell only sees your mounted resources: no arbitrary host filesystem, no shelling out to host binaries. For untrusted code you'd still want a real sandbox (Daytona, E2B, Modal); Mirage embeds inside those rather than replacing them.
|
||||
</Accordion>
|
||||
<Accordion title="What about latency and cost?">
|
||||
Backend-bound. <code>cat /s3/...</code> is one GetObject; <code>find /postgres/...</code> is a SQL query. Reads cache per session, and <code>mirage provision</code> returns a dry-run estimate (network bytes, cache hits, projected cost) before you commit to an expensive operation.
|
||||
</Accordion>
|
||||
<Accordion title="Hosted or self-hosted?">
|
||||
Self-hosted. Mirage is a library plus a thin local daemon. The daemon lives in your process or sandbox; data only leaves your network if a mount you configured already does (e.g. an S3 read).
|
||||
</Accordion>
|
||||
<Accordion title="Which agent frameworks are supported?">
|
||||
Anything that exposes a shell tool: Claude Code, Codex / OpenAI Agents SDK, Cursor, OpenHands, Pydantic deepagents. Direct SDK integrations live under <a href="/python/agents">Python agents</a> and <a href="/typescript/agents">TypeScript agents</a>.
|
||||
</Accordion>
|
||||
<Accordion title="Python or TypeScript?">
|
||||
Both. The Python package is the reference implementation; the TypeScript SDK (<code>@struktoai/mirage-node</code>) ships the same <code>Workspace</code>/<code>execute</code> surface and most resources. Some agent integrations land on Python first.
|
||||
</Accordion>
|
||||
<Accordion title="What if a resource I want isn't supported yet?">
|
||||
Resources are pluggable. The <a href="/python/resource/new">"Add a resource"</a> guide walks through the read/write/stat surface a new backend implements. PRs welcome.
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
<h2 className="text-2xl font-bold mt-16 mb-6">Explore Mirage</h2>
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Python Quickstart" icon="python" href="/python/quickstart">
|
||||
Create a workspace, mount resources, and run shell commands in minutes.
|
||||
</Card>
|
||||
<Card title="TypeScript Quickstart" icon="js" href="/typescript/quickstart">
|
||||
Same Workspace API in Node, browser, and edge runtimes.
|
||||
</Card>
|
||||
<Card title="CLI" icon="terminal" href="/home/cli">
|
||||
Drive workspaces from the shell: create, execute, snapshot, and restore.
|
||||
</Card>
|
||||
<Card title="Resource Matrix" icon="grid-2" href="/home/resource-matrix">
|
||||
Compare resources by mount mode, setup path, and common use cases.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
<h2 className="text-2xl font-bold mt-16 mb-6">Community & Support</h2>
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Need Help?" icon="circle-question" href="/home/troubleshooting">
|
||||
Setup failures, credential issues, and FUSE gotchas.
|
||||
</Card>
|
||||
<Card title="Book a Call" icon="calendar" href="https://cal.com/strukto/30min">
|
||||
Talk to the team directly if you are blocked.
|
||||
</Card>
|
||||
<Card title="Join our Discord" icon="discord" href="https://discord.gg/u8BPQ65KsS">
|
||||
Chat with the community and get help from the team.
|
||||
</Card>
|
||||
<Card title="GitHub Issues" icon="github" href="https://github.com/strukto-ai/mirage/issues">
|
||||
Report bugs, request features, or ask questions.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
</div>
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: Observer
|
||||
description: A hidden recorder that captures every command and file op as timestamped events, backed by a pluggable store. Powers command history.
|
||||
icon: eye
|
||||
---
|
||||
|
||||
## What It Does
|
||||
|
||||
Every workspace has one **Observer**: a hidden recorder that logs each top-level
|
||||
command and its file ops as timestamp-ordered events. It owns no mount and has no
|
||||
endpoint of its own, features like command history are just *views* over its
|
||||
events. Nested evals (`$(...)`, `eval`, `source`, `xargs`) run without recording,
|
||||
so only real top-level commands land.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Exec["ws.execute(cmd)"] --> Obs[Observer]
|
||||
Obs --> Store[("ObserverStore<br/>RAM · Disk · Redis")]
|
||||
Store --> Hist["history builtin<br/>(calling session)"]
|
||||
Store --> File["/.bash_history<br/>(all sessions)"]
|
||||
```
|
||||
|
||||
## Storage backends
|
||||
|
||||
The Observer holds a storage-agnostic `ObserverStore`. RAM is the default; swap it
|
||||
to persist events across daemon restarts. The store is chosen at construction; there
|
||||
is no runtime API to change it.
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```python Python
|
||||
from mirage import Workspace, MountMode
|
||||
from mirage.resource.ram import RAMResource
|
||||
from mirage.observe.disk_store import DiskObserverStore
|
||||
from mirage.observe.redis_store import RedisObserverStore
|
||||
|
||||
# RAM (default), nothing to configure
|
||||
ws = Workspace({"/data": RAMResource()}, mode=MountMode.WRITE)
|
||||
|
||||
# Persist to disk
|
||||
ws = Workspace({"/data": RAMResource()}, mode=MountMode.WRITE,
|
||||
observe=DiskObserverStore("/var/mirage/history"))
|
||||
|
||||
# Persist to Redis
|
||||
ws = Workspace({"/data": RAMResource()}, mode=MountMode.WRITE,
|
||||
observe=RedisObserverStore("redis://localhost:6379/0"))
|
||||
```
|
||||
|
||||
```typescript TypeScript
|
||||
import { Workspace, RAMResource } from '@struktoai/mirage-core'
|
||||
import { DiskObserverStore, RedisObserverStore } from '@struktoai/mirage-node'
|
||||
|
||||
// RAM (default), nothing to configure
|
||||
const ws = new Workspace({ '/data': new RAMResource() })
|
||||
|
||||
// Persist to disk
|
||||
const wsDisk = new Workspace(
|
||||
{ '/data': new RAMResource() },
|
||||
{ observe: new DiskObserverStore('/var/mirage/history') },
|
||||
)
|
||||
|
||||
// Persist to Redis
|
||||
const wsRedis = new Workspace(
|
||||
{ '/data': new RAMResource() },
|
||||
{ observe: new RedisObserverStore({ url: 'redis://localhost:6379/0' }) },
|
||||
)
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
## Supported: command history
|
||||
|
||||
The Observer powers a GNU-bash-compatible history, exposed two ways over the same
|
||||
events:
|
||||
|
||||
| Surface | Scope | Notes |
|
||||
| --- | --- | --- |
|
||||
| `history` builtin | calling session | GNU flags `-c -d -a -n -r -w -s -p` and a count arg |
|
||||
| `/.bash_history` mount | all sessions | read-only, GNU histfile format (`#<epoch>` then the command) |
|
||||
|
||||
Because `/.bash_history` is a real read-only mount, the ordinary file commands work
|
||||
on it directly:
|
||||
|
||||
```bash
|
||||
history 5
|
||||
tail -n 6 /.bash_history
|
||||
grep cat /.bash_history
|
||||
```
|
||||
|
||||
The format is GNU bash (`#<epoch>`), not zsh (`: <ts>:<dur>;<cmd>`).
|
||||
|
||||
## Snapshots
|
||||
|
||||
History is part of the workspace state: the Observer's command, clear, and delete
|
||||
events are captured into a [snapshot](/home/snapshot) and restored on load, so a
|
||||
restored workspace replays with the same history. The `/.bash_history` view mount
|
||||
itself is not stored, it is a live projection rebuilt from the events.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
title: Python SDK
|
||||
description: Start with the Mirage Python SDK quickstart for installing the package, creating a workspace, and mounting resources.
|
||||
url: /python/quickstart
|
||||
icon: python
|
||||
---
|
||||
@@ -0,0 +1,136 @@
|
||||
---
|
||||
title: Resource Matrix
|
||||
description: Compare Mirage resources by mount mode and setup path, organized by category.
|
||||
icon: grid-2
|
||||
---
|
||||
|
||||
## How To Read This Matrix
|
||||
|
||||
Use this page to pick the first resource to try. If you want the fastest path, start with RAM or disk. If you need real integrations, jump from the setup guide to the resource docs.
|
||||
|
||||
The **Mount Mode** column shows which `MountMode` values the resource supports (`read`, `write`, and `exec`, the last makes a mount executable so commands can launch binaries from it; typically used with Disk or RAM). The **Docs** column shows where each resource is available, with one icon per supported runtime: <Icon icon="python" /> Python, <Icon icon="node-js" /> TypeScript (Node), and <Icon icon="globe" /> TypeScript (Browser). Click an icon to jump to that runtime's docs.
|
||||
|
||||
Sections below mirror the **Setup** sidebar so you can pick a category and follow the same path through credentials → resource docs.
|
||||
|
||||
## Infrastructure
|
||||
|
||||
No external setup, these run locally or against a connection string you already have.
|
||||
|
||||
| Resource | Mount Mode | Docs | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| RAM | read, write, exec | [<Icon icon="python" />](/python/resource/ram) [<Icon icon="node-js" />](/typescript/quickstart) [<Icon icon="globe" />](/typescript/quickstart) | Best first-run option |
|
||||
| Disk | read, write, exec | [<Icon icon="python" />](/python/resource/disk) [<Icon icon="node-js" />](/typescript/setup/disk) | Local filesystem bridge |
|
||||
| OPFS | read, write, exec | [<Icon icon="globe" />](/typescript/setup/opfs) | Browser-only persistent filesystem |
|
||||
| Redis | read, write, exec | [<Icon icon="python" />](/python/resource/redis) [<Icon icon="node-js" />](/typescript/setup/redis) | Persistent cache-backed workspace |
|
||||
| SSH | read, write | [<Icon icon="python" />](/python/resource/ssh) [<Icon icon="node-js" />](/typescript/setup/ssh) | Remote filesystem access |
|
||||
|
||||
## Object Storage
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| S3 | read, write | [S3](/home/setup/s3) | [<Icon icon="python" />](/python/resource/s3) [<Icon icon="node-js" />](/typescript/setup/s3) [<Icon icon="globe" />](/typescript/setup/s3) | Common cloud object store |
|
||||
| R2 | read, write | [R2](/home/setup/r2) | [<Icon icon="python" />](/python/resource/r2) [<Icon icon="node-js" />](/typescript/setup/r2) [<Icon icon="globe" />](/typescript/setup/r2) | S3-style API |
|
||||
| GCS | read, write | [GCS](/home/setup/gcs) | [<Icon icon="python" />](/python/resource/gcs) [<Icon icon="node-js" />](/typescript/setup/gcs) [<Icon icon="globe" />](/typescript/setup/gcs) | GCP object storage |
|
||||
| OCI | read, write | [OCI](/home/setup/oci) | [<Icon icon="python" />](/python/resource/oci) [<Icon icon="node-js" />](/typescript/setup/oci) [<Icon icon="globe" />](/typescript/setup/oci) | Oracle object storage |
|
||||
| Supabase | read, write | [Supabase](/home/setup/supabase) | [<Icon icon="python" />](/python/resource/supabase) [<Icon icon="node-js" />](/typescript/setup/supabase) [<Icon icon="globe" />](/typescript/setup/supabase) | Storage-oriented workflows |
|
||||
| MinIO | read, write | [MinIO](/home/setup/minio) | [<Icon icon="python" />](/python/resource/minio) [<Icon icon="node-js" />](/typescript/setup/minio) [<Icon icon="globe" />](/typescript/setup/minio) | Self-hosted S3-compatible store |
|
||||
| Ceph | read, write | [Ceph](/home/setup/ceph) | [<Icon icon="python" />](/python/resource/ceph) [<Icon icon="node-js" />](/typescript/setup/ceph) [<Icon icon="globe" />](/typescript/setup/ceph) | Ceph Rados Gateway (self-hosted) |
|
||||
| SeaweedFS | read, write | [SeaweedFS](/home/setup/seaweedfs) | [<Icon icon="python" />](/python/resource/seaweedfs) [<Icon icon="node-js" />](/typescript/setup/seaweedfs) [<Icon icon="globe" />](/typescript/setup/seaweedfs) | SeaweedFS S3 gateway (self-hosted; backs Blaxel Agent Drive) |
|
||||
| Wasabi | read, write | [Wasabi](/home/setup/wasabi) | [<Icon icon="python" />](/python/resource/wasabi) [<Icon icon="node-js" />](/typescript/setup/wasabi) [<Icon icon="globe" />](/typescript/setup/wasabi) | Low-cost S3-compatible cloud |
|
||||
| Backblaze B2 | read, write | [Backblaze B2](/home/setup/backblaze) | [<Icon icon="python" />](/python/resource/backblaze) [<Icon icon="node-js" />](/typescript/setup/backblaze) [<Icon icon="globe" />](/typescript/setup/backblaze) | B2 via S3 API |
|
||||
| DigitalOcean Spaces | read, write | [DigitalOcean](/home/setup/digitalocean) | [<Icon icon="python" />](/python/resource/digitalocean) [<Icon icon="node-js" />](/typescript/setup/digitalocean) [<Icon icon="globe" />](/typescript/setup/digitalocean) | DO Spaces via S3 API |
|
||||
| Tencent COS | read, write | [Tencent COS](/home/setup/tencent) | [<Icon icon="python" />](/python/resource/tencent) [<Icon icon="node-js" />](/typescript/setup/tencent) [<Icon icon="globe" />](/typescript/setup/tencent) | Tencent Cloud Object Storage |
|
||||
| Alibaba OSS | read, write | [Alibaba OSS](/home/setup/aliyun) | [<Icon icon="python" />](/python/resource/aliyun) [<Icon icon="node-js" />](/typescript/setup/aliyun) [<Icon icon="globe" />](/typescript/setup/aliyun) | Aliyun Object Storage Service |
|
||||
| Scaleway | read, write | [Scaleway](/home/setup/scaleway) | [<Icon icon="python" />](/python/resource/scaleway) [<Icon icon="node-js" />](/typescript/setup/scaleway) [<Icon icon="globe" />](/typescript/setup/scaleway) | Scaleway Object Storage |
|
||||
| QingStor | read, write | [QingStor](/home/setup/qingstor) | [<Icon icon="python" />](/python/resource/qingstor) [<Icon icon="node-js" />](/typescript/setup/qingstor) [<Icon icon="globe" />](/typescript/setup/qingstor) | QingCloud Object Storage |
|
||||
| HF Buckets | read, write | [HF Buckets](/home/setup/hf_buckets) | [<Icon icon="python" />](/python/resource/hf_buckets) [<Icon icon="node-js" />](/typescript/setup/hf_buckets) | Hugging Face Buckets |
|
||||
| HF Datasets | read, write | [HF Datasets](/home/setup/hf_datasets) | [<Icon icon="python" />](/python/resource/hf_datasets) [<Icon icon="node-js" />](/typescript/setup/hf_datasets) | Hugging Face Dataset repos (lazy reads) |
|
||||
| HF Models | read, write | [HF Models](/home/setup/hf_models) | [<Icon icon="python" />](/python/resource/hf_models) [<Icon icon="node-js" />](/typescript/setup/hf_models) | Hugging Face Model repos (weights stream on demand) |
|
||||
| HF Spaces | read, write | [HF Spaces](/home/setup/hf_spaces) | [<Icon icon="python" />](/python/resource/hf_spaces) [<Icon icon="node-js" />](/typescript/setup/hf_spaces) | Hugging Face Space repos (app code, configs) |
|
||||
|
||||
## Google Workspace
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Gmail | read, partial write | [Google](/home/setup/google) | [<Icon icon="python" />](/python/resource/gmail) [<Icon icon="node-js" />](/typescript/setup/gmail) [<Icon icon="globe" />](/typescript/setup/gmail) | Mailbox access |
|
||||
| Drive | read, partial write | [Google](/home/setup/google) | [<Icon icon="python" />](/python/resource/gdrive) [<Icon icon="node-js" />](/typescript/setup/gdrive) [<Icon icon="globe" />](/typescript/setup/gdrive) | File tree and metadata |
|
||||
| Docs | read, partial write | [Google](/home/setup/google) | [<Icon icon="python" />](/python/resource/gdocs) [<Icon icon="node-js" />](/typescript/setup/gdocs) [<Icon icon="globe" />](/typescript/setup/gdocs) | Document content |
|
||||
| Sheets | read, partial write | [Google](/home/setup/google) | [<Icon icon="python" />](/python/resource/gsheets) [<Icon icon="node-js" />](/typescript/setup/gsheets) [<Icon icon="globe" />](/typescript/setup/gsheets) | Spreadsheet data |
|
||||
| Slides | read, partial write | [Google](/home/setup/google) | [<Icon icon="python" />](/python/resource/gslides) [<Icon icon="node-js" />](/typescript/setup/gslides) [<Icon icon="globe" />](/typescript/setup/gslides) | Presentation content |
|
||||
|
||||
## Cloud Files
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Databricks Volume | read, write | [Databricks Volume](/home/setup/databricks) | [<Icon icon="python" />](/python/resource/databricks_volume) [<Icon icon="node-js" />](/typescript/setup/databricks_volume) | Unity Catalog volume subtree; mv/cp are non-atomic copies |
|
||||
| Dropbox | read | [Dropbox](/home/setup/dropbox) | [<Icon icon="node-js" />](/typescript/dropbox) [<Icon icon="globe" />](/typescript/dropbox) | OAuth2 + PKCE; Node and browser |
|
||||
| Box | read | [Box](/home/setup/box) | [<Icon icon="node-js" />](/typescript/box) [<Icon icon="globe" />](/typescript/box) | OAuth2 + PKCE or developer token; `.boxnote.json` / `.boxcanvas.json` / `.gdoc.json` decoded |
|
||||
| Nextcloud | read, write | [Nextcloud](/home/setup/nextcloud) | [<Icon icon="python" />](/python/resource/nextcloud) | Self-hosted Nextcloud / ownCloud / WebDAV; HTTP Basic auth with app password |
|
||||
|
||||
## Code & DevOps
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| GitHub | read | [GitHub](/home/setup/github) | [<Icon icon="python" />](/python/resource/github) [<Icon icon="node-js" />](/typescript/setup/github) [<Icon icon="globe" />](/typescript/setup/github) | Repository browsing |
|
||||
| GitHub CI | read | [GitHub CI](/home/setup/github_ci) | [<Icon icon="python" />](/python/resource/github_ci) [<Icon icon="node-js" />](/typescript/setup/github_ci) [<Icon icon="globe" />](/typescript/setup/github_ci) | Runs, logs, artifacts |
|
||||
| Linear | read | [Linear](/home/setup/linear) | [<Icon icon="python" />](/python/resource/linear) [<Icon icon="node-js" />](/typescript/setup/linear) [<Icon icon="globe" />](/typescript/setup/linear) | Issues and projects |
|
||||
| Langfuse | read | [Langfuse](/home/setup/langfuse) | [<Icon icon="python" />](/python/resource/langfuse) [<Icon icon="node-js" />](/typescript/setup/langfuse) [<Icon icon="globe" />](/typescript/setup/langfuse) | Trace exploration |
|
||||
|
||||
## Messaging
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Slack | read | [Slack](/home/setup/slack) | [<Icon icon="python" />](/python/resource/slack) [<Icon icon="node-js" />](/typescript/slack) [<Icon icon="globe" />](/typescript/slack) | Channels, messages, files |
|
||||
| Discord | read | [Discord](/home/setup/discord) | [<Icon icon="python" />](/python/resource/discord) [<Icon icon="node-js" />](/typescript/discord) [<Icon icon="globe" />](/typescript/discord) | Guild and channel history |
|
||||
| Email | read, write | [Email](/home/setup/email) | [<Icon icon="python" />](/python/resource/email) [<Icon icon="node-js" />](/typescript/setup/email) | Mailbox workflows; browser blocked by raw TCP requirement |
|
||||
|
||||
## Database
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| MongoDB | read | [MongoDB](/home/setup/mongodb) | [<Icon icon="python" />](/python/resource/mongodb) [<Icon icon="node-js" />](/typescript/setup/mongodb) [<Icon icon="globe" />](/typescript/setup/mongodb) | Collection-backed views |
|
||||
| Postgres | read | [Postgres](/home/setup/postgres) | [<Icon icon="python" />](/python/resource/postgres) [<Icon icon="node-js" />](/typescript/setup/postgres) [<Icon icon="globe" />](/typescript/setup/postgres) | SQL tables exposed as paths |
|
||||
| LanceDB | read | [LanceDB](/home/setup/lancedb) | [<Icon icon="python" />](/python/resource/lancedb) [<Icon icon="node-js" />](/typescript/setup/lancedb) | Label folders + semantic search command |
|
||||
| Qdrant | read | [Qdrant](/home/setup/qdrant) | [<Icon icon="python" />](/python/resource/qdrant) [<Icon icon="node-js" />](/typescript/setup/qdrant) [<Icon icon="globe" />](/typescript/setup/qdrant) | Collections as folders + semantic search command |
|
||||
|
||||
## Knowledge
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Dify | read | [Dify](/home/setup/dify) | [<Icon icon="python" />](/python/resource/dify) | Knowledge documents and retrieval search |
|
||||
| Chroma | read | [Chroma](/home/setup/chroma) | [<Icon icon="python" />](/python/resource/chroma) [<Icon icon="node-js" />](/typescript/setup/chroma) | ChromaDB collection exposed as files and vector search |
|
||||
|
||||
## Notes
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Notion | read, write | [Notion](/home/setup/notion) | [<Icon icon="python" />](/python/resource/notion) [<Icon icon="node-js" />](/typescript/setup/notion) [<Icon icon="globe" />](/typescript/setup/notion) | Pages and blocks |
|
||||
|
||||
## Others
|
||||
|
||||
| Resource | Mount Mode | Setup | Docs | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Trello | read | [Trello](/home/setup/trello) | [<Icon icon="python" />](/python/resource/trello) [<Icon icon="node-js" />](/typescript/setup/trello) [<Icon icon="globe" />](/typescript/setup/trello) | Boards and cards |
|
||||
|
||||
## Agent Frameworks
|
||||
|
||||
Mirage drops into the major agent application frameworks. Each adapter exposes a `Workspace` as the agent's filesystem and shell. Click an icon to jump to the integration docs.
|
||||
|
||||
| Framework | Docs | Notes |
|
||||
| --- | --- | --- |
|
||||
| OpenAI Agents SDK | [<Icon icon="python" />](/python/agents/openai-agents) [<Icon icon="node-js" />](/typescript/agents/openai) [<Icon icon="globe" />](/typescript/agents/openai) | `MirageShell` and `MirageEditor` plug into `shellTool` and `applyPatchTool`. |
|
||||
| Vercel AI SDK | [<Icon icon="node-js" />](/typescript/agents/vercel) [<Icon icon="globe" />](/typescript/agents/vercel) | `mirageTools()` returns five typed tools for `generateText` / `streamText`. |
|
||||
| LangChain (deepagents) | [<Icon icon="python" />](/python/agents/langchain) [<Icon icon="node-js" />](/typescript/agents/langchain) | `LangchainWorkspace` backend for deepagents. Node only. |
|
||||
| Pi Coding Agent | [<Icon icon="node-js" />](/typescript/agents/pi) | Mirage extension for `@mariozechner/pi-coding-agent`. Node only. |
|
||||
| Mastra | [<Icon icon="node-js" />](/typescript/agents/mastra) | `mirageTools()` for Mastra `Agent` definitions. Node only. |
|
||||
| Pydantic AI | [<Icon icon="python" />](/python/agents/pydantic-ai) | For pydantic-ai and pydantic-deepagents. |
|
||||
| CAMEL-AI | [<Icon icon="python" />](/python/agents/camel) | For CAMEL `ChatAgent`. |
|
||||
| OpenHands | [<Icon icon="python" />](/python/agents/openhands) | For the OpenHands agent SDK. |
|
||||
| Claude Code (CLI) | [<Icon icon="python" />](/python/agents/claude-code) [<Icon icon="node-js" />](/typescript/agents/claude-code) | Mount via FUSE; run `claude` against the mountpoint. |
|
||||
| Codex (CLI) | [<Icon icon="python" />](/python/agents/codex) [<Icon icon="node-js" />](/typescript/agents/codex) | Mount via FUSE; run `codex` against the mountpoint. |
|
||||
|
||||
## Recommended Starting Points
|
||||
|
||||
- Use [RAM](/python/resource/ram) if you want to learn Mirage itself.
|
||||
- Use [Disk](/python/resource/disk) if you want local file interoperability.
|
||||
- Use [S3](/python/resource/s3) or [GitHub](/python/resource/github) if you want a realistic first external integration.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: Alibaba OSS
|
||||
icon: /images/aliyun-logo.svg
|
||||
description: Set up Alibaba Cloud OSS credentials for the Aliyun resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an AccessKey pair
|
||||
|
||||
1. Sign in to the [Alibaba Cloud console](https://www.aliyun.com/)
|
||||
1. Go to **AccessKey Management** (RAM user recommended) -> **Create AccessKey**
|
||||
1. Copy the **AccessKey ID** and **AccessKey Secret**
|
||||
1. Note your bucket's **region** (e.g. `cn-hangzhou`) and **bucket** name
|
||||
|
||||
The endpoint is derived from `region` as `s3.oss-<region>.aliyuncs.com`. Note the
|
||||
`s3.` prefix: this is the S3-compatible host, distinct from the native
|
||||
`oss-<region>.aliyuncs.com`.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
OSS_BUCKET=my-bucket
|
||||
OSS_REGION=cn-hangzhou
|
||||
OSS_ACCESS_KEY_ID=...
|
||||
OSS_ACCESS_KEY_SECRET=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Alibaba OSS resource](/python/resource/aliyun) docs.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: Backblaze B2
|
||||
icon: /images/backblaze-logo.svg
|
||||
description: Set up Backblaze B2 credentials for the Backblaze resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an application key
|
||||
|
||||
1. Sign in to [Backblaze](https://secure.backblaze.com/) -> **B2 Cloud Storage**
|
||||
1. Go to **Application Keys** -> **Add a New Application Key**
|
||||
1. Scope it to your bucket with at least **Read** access
|
||||
1. Copy the **keyID** (access key id) and **applicationKey** (secret access key)
|
||||
1. Note your bucket's **endpoint region** from the bucket details
|
||||
(e.g. `us-west-004`) and the **bucket** name
|
||||
|
||||
The endpoint is derived from `region` as `s3.<region>.backblazeb2.com`.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
B2_BUCKET=my-bucket
|
||||
B2_REGION=us-west-004
|
||||
B2_ACCESS_KEY_ID=... # B2 keyID
|
||||
B2_SECRET_ACCESS_KEY=... # B2 applicationKey
|
||||
```
|
||||
|
||||
For Python configuration, see the [Backblaze resource](/python/resource/backblaze) docs.
|
||||
@@ -0,0 +1,212 @@
|
||||
---
|
||||
title: Box
|
||||
icon: box
|
||||
description: Set up Box OAuth2 credentials and obtain a refresh token (Node and browser).
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The Box resource uses OAuth2 against the [Box v2 API](https://developer.box.com/reference/) to:
|
||||
|
||||
- **`/folders/{id}/items`**, list folder items (root id is `0`)
|
||||
- **`/files/{id}/content`**, download file bytes
|
||||
- **`/files/{id}`** / **`/folders/{id}`**, fetch entry metadata
|
||||
- **`/search`**, search files by name/content
|
||||
- **`/oauth2/token`**, refresh access tokens
|
||||
|
||||
Box is similar to Dropbox in shape but uses **numeric folder/file IDs** rather than paths.
|
||||
Mirage caches the path → ID mapping in the index store, so once you `ls /box/` the IDs are
|
||||
remembered and subsequent reads don't pay the lookup cost.
|
||||
|
||||
Three credential modes are supported:
|
||||
|
||||
- **Developer token**, fastest first-run path. Click one button in the Box app console, paste
|
||||
the 32-char access token, and you're done. Lives 60 minutes; you regenerate by hand. No
|
||||
client_id / refresh_token / OAuth flow needed. Recommended for `mirage` exploration.
|
||||
- **Code flow with client secret**, for Node/server long-running use. Needs `client_id`,
|
||||
`client_secret`, `refresh_token`.
|
||||
- **PKCE flow**, for browsers. Needs only `client_id` and `refresh_token` (no secret in the bundle).
|
||||
|
||||
<Note>
|
||||
**Box rotates the refresh token on every refresh.** The token you copy out of the initial code
|
||||
exchange is only valid until the next refresh, after that, only the new rotated token works.
|
||||
The `BoxTokenManager` keeps the latest token in memory; pass `onRefreshTokenRotated` if you
|
||||
want to persist it across restarts (the browser example does this with `localStorage`).
|
||||
</Note>
|
||||
|
||||
## Quick Start: Developer Token (60 minutes, no OAuth)
|
||||
|
||||
If you just want to try the resource against your own Box account, the developer token is the
|
||||
fastest path, no redirect URI, no client secret, no curl exchange:
|
||||
|
||||
1. Go to https://app.box.com/developers/console -> **Create New App** -> **Custom App** ->
|
||||
**User Authentication (OAuth 2.0)** -> name it "Mirage" -> **Create App**
|
||||
1. On the **Configuration** tab, scroll to the **Developer Token** section near the bottom.
|
||||
1. Click **Generate Developer Token**. Copy the 32-char token.
|
||||
1. Paste into your `.env.development`:
|
||||
```bash
|
||||
BOX_DEVELOPER_TOKEN=...
|
||||
```
|
||||
1. Run any of the examples in [`examples/typescript/box/`](https://github.com/strukto-ai/mirage/tree/main/examples/typescript/box).
|
||||
|
||||
The token expires in **60 minutes** and there is no programmatic way to renew it; you regenerate
|
||||
by hand. For long-running processes, use the OAuth flow below instead.
|
||||
|
||||
When `BOX_DEVELOPER_TOKEN` is set, the `BoxResource` skips the OAuth refresh path entirely and
|
||||
calls Box directly with the token in the `Authorization: Bearer` header.
|
||||
|
||||
## Setup (long-running OAuth flow)
|
||||
|
||||
### 1. Create a Box App
|
||||
|
||||
1. Go to https://app.box.com/developers/console
|
||||
1. Click **Create New App**
|
||||
1. Choose **Custom App**
|
||||
1. **Authentication Method**: **User Authentication (OAuth 2.0)**
|
||||
1. Name it (e.g. "Mirage") -> **Create App**
|
||||
|
||||
### 2. Configure the App
|
||||
|
||||
Open the app's **Configuration** tab:
|
||||
|
||||
- **OAuth 2.0 Redirect URIs**: add every destination you'll redirect back to. Box accepts
|
||||
multiple values, register them all up front so you don't bounce between dev and prod:
|
||||
- For the CLI flow below: `http://localhost:1`
|
||||
- For the browser PKCE example: `http://localhost:5173/box_pkce.html`
|
||||
- For production: e.g. `https://yourapp.com/box/callback`
|
||||
- **Allowed Origins** (only needed if you'll call from a browser; older Box docs call this
|
||||
"CORS Domains"):
|
||||
- `http://localhost:5173`
|
||||
- Any production origins (e.g. `https://yourapp.com`)
|
||||
- **Application Scopes**:
|
||||
- **Read all files and folders stored in Box** (required)
|
||||
- **Write all files and folders stored in Box** (only if you'll add write commands later)
|
||||
- Click **Save Changes**
|
||||
|
||||
### 3. Submit for Authorization (enterprise apps only)
|
||||
|
||||
If your Box account is part of an enterprise, the **Authorization** tab requires an admin to
|
||||
approve the app before it can issue tokens. Personal/free Box accounts skip this step.
|
||||
|
||||
### 4. Copy the Client Credentials
|
||||
|
||||
Still on **Configuration**, copy:
|
||||
|
||||
- **Client ID** -> `BOX_CLIENT_ID`
|
||||
- **Client Secret** -> `BOX_CLIENT_SECRET` (skip if you only need PKCE)
|
||||
|
||||
### 5. Get the Refresh Token (CLI flow with client secret)
|
||||
|
||||
**A) Open this URL in a browser** (replace `YOUR_CLIENT_ID`):
|
||||
|
||||
```
|
||||
https://account.box.com/api/oauth2/authorize?response_type=code&client_id=YOUR_CLIENT_ID&redirect_uri=http://localhost:1
|
||||
```
|
||||
|
||||
**B) Authorize:** sign in -> click **Grant access**.
|
||||
|
||||
**C) Copy the code:** browser redirects to `http://localhost:1?code=...` (page won't load,
|
||||
expected). Copy the `code` from the URL bar.
|
||||
|
||||
**D) Exchange the code:**
|
||||
|
||||
```bash
|
||||
curl https://api.box.com/oauth2/token \
|
||||
-d "code=THE_CODE" \
|
||||
-d "grant_type=authorization_code" \
|
||||
-d "client_id=YOUR_CLIENT_ID" \
|
||||
-d "client_secret=YOUR_CLIENT_SECRET" \
|
||||
-d "redirect_uri=http://localhost:1"
|
||||
```
|
||||
|
||||
Response shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"access_token": "T...",
|
||||
"expires_in": 3884,
|
||||
"refresh_token": "R...",
|
||||
"restricted_to": [],
|
||||
"token_type": "bearer"
|
||||
}
|
||||
```
|
||||
|
||||
Save the `refresh_token`.
|
||||
|
||||
### 6. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
BOX_CLIENT_ID=...
|
||||
BOX_CLIENT_SECRET=...
|
||||
BOX_REFRESH_TOKEN=...
|
||||
```
|
||||
|
||||
## Alternative: PKCE Flow (browser, no client secret)
|
||||
|
||||
1. Add `http://localhost:5173/box_pkce.html` to **OAuth 2.0 Redirect URIs**
|
||||
1. Add `http://localhost:5173` to **Allowed Origins**
|
||||
1. Set only `BOX_CLIENT_ID` in `.env.development`
|
||||
1. From `examples/typescript/browser/`, run `pnpm dev`
|
||||
1. Open `http://localhost:5173/box_pkce.html` and click **Connect Box**
|
||||
|
||||
The example persists the rotated refresh token to `localStorage` via the
|
||||
`onRefreshTokenRotated` callback so subsequent page loads work without re-auth.
|
||||
|
||||
## Token Lifetime
|
||||
|
||||
| Token | Lifetime |
|
||||
| ------------- | -------------------------------------------------------- |
|
||||
| Access token | ~1 hour (`expires_in: ~3600`) |
|
||||
| Refresh token | 60 days from issue, OR 60 days from last use (whichever) |
|
||||
|
||||
A refresh token gets revoked when:
|
||||
|
||||
- 60 days pass without use
|
||||
- The user revokes the app at https://app.box.com/account/security
|
||||
- The Box account password changes
|
||||
- A new refresh token is issued (the old one expires after a short grace period)
|
||||
|
||||
The `BoxTokenManager` always uses the latest token. For long-running processes restart-safe,
|
||||
provide `onRefreshTokenRotated` to persist the new token to disk / a vault.
|
||||
|
||||
## CORS Notes
|
||||
|
||||
Unlike Dropbox, Box has first-class CORS but **only for origins you've explicitly allowlisted**
|
||||
in the **Allowed Origins** section of the app configuration. If you see browser-side
|
||||
`Access-Control-Allow-Origin` errors, double-check the origin matches exactly (no trailing
|
||||
slash; protocol matters).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Fix |
|
||||
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| `400 invalid_grant` on token exchange | The `code` is single-use and short-lived, redo Step 5A with a fresh URL |
|
||||
| `403 access_denied` | Enterprise admin hasn't approved the app on the **Authorization** tab |
|
||||
| `401 unauthorized` after a few hours | Refresh token rotation, the original token is invalid; persist via `onRefreshTokenRotated` |
|
||||
| Browser: `CORS error` even though origin set | Check that the origin in **Allowed Origins** is exact (no trailing slash, http vs https) |
|
||||
| `path/not_found` on `ls /box/<folder>/` | The folder name was constructed wrong, use `ls /box/` first, then drill into a name from the list |
|
||||
|
||||
For TypeScript usage:
|
||||
|
||||
```ts
|
||||
import { BoxResource } from '@struktoai/mirage-node'
|
||||
|
||||
const box = new BoxResource({
|
||||
clientId: process.env.BOX_CLIENT_ID!,
|
||||
clientSecret: process.env.BOX_CLIENT_SECRET!,
|
||||
refreshToken: process.env.BOX_REFRESH_TOKEN!,
|
||||
})
|
||||
```
|
||||
|
||||
Or browser (PKCE):
|
||||
|
||||
```ts
|
||||
import { BoxResource } from '@struktoai/mirage-browser'
|
||||
|
||||
const box = new BoxResource({
|
||||
clientId: 'YOUR_CLIENT_ID',
|
||||
refreshToken: refreshTokenFromLocalStorage,
|
||||
onRefreshTokenRotated: (next) => localStorage.setItem('box-refresh', next),
|
||||
})
|
||||
```
|
||||
@@ -0,0 +1,33 @@
|
||||
---
|
||||
title: Ceph (Rados Gateway)
|
||||
icon: /images/ceph-logo.svg
|
||||
description: Set up Ceph Rados Gateway (RGW) credentials for the Ceph resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an RGW user
|
||||
|
||||
On a host with `radosgw-admin` access:
|
||||
|
||||
```bash
|
||||
radosgw-admin user create --uid=mirage --display-name="Mirage"
|
||||
```
|
||||
|
||||
The output includes a `keys` block with `access_key` and `secret_key`. Note your
|
||||
gateway **endpoint** (e.g. `https://ceph.example.com`) and the **bucket** name.
|
||||
|
||||
Ceph RGW is self-hosted, so there is no region-derived endpoint, the
|
||||
`endpoint_url` is required and RGW uses path-style addressing.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
CEPH_BUCKET=my-bucket
|
||||
CEPH_ENDPOINT_URL=https://ceph.example.com
|
||||
CEPH_ACCESS_KEY_ID=...
|
||||
CEPH_SECRET_ACCESS_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Ceph resource](/python/resource/ceph) docs.
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
title: Chroma
|
||||
icon: database
|
||||
description: Prepare a ChromaDB collection for the Chroma resource.
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
Mirage mounts an existing ChromaDB collection as a read-only virtual
|
||||
filesystem. The collection must already contain:
|
||||
|
||||
- One path tree document with ID `__path_tree__`.
|
||||
- One or more page chunk documents.
|
||||
- A metadata field that maps each chunk to a virtual file path slug.
|
||||
- A metadata field that orders chunks within each file.
|
||||
|
||||
The Python resource connects via Chroma's `AsyncHttpClient`.
|
||||
|
||||
## Path Tree Document
|
||||
|
||||
The path tree document defines the filesystem tree. Store it in Chroma with the
|
||||
ID `__path_tree__`. Its document content must be either plain JSON or
|
||||
gzip-compressed base64 JSON.
|
||||
|
||||
```json
|
||||
{
|
||||
"README.md": {
|
||||
"size": 1024,
|
||||
"created_at": "2026-01-01T00:00:00Z",
|
||||
"updated_at": "2026-01-02T00:00:00Z"
|
||||
},
|
||||
"guides/quickstart.md": {
|
||||
"size": 4096,
|
||||
"updated_at": "2026-01-03T00:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Path tree keys become file paths below the Mirage mount. Use forward slashes to
|
||||
create directories.
|
||||
|
||||
Path rules:
|
||||
|
||||
- Do not use an empty path.
|
||||
- Do not use `.`, `..`, or empty path segments.
|
||||
- Keep every path unique after trimming leading and trailing slashes.
|
||||
- Do not use a path both as a file and as a directory prefix. For example,
|
||||
`guides` and `guides/quickstart.md` cannot both be files.
|
||||
|
||||
Metadata fields are optional. Mirage uses `size`, `created_at`, and
|
||||
`updated_at` when present.
|
||||
|
||||
## Chunk Documents
|
||||
|
||||
Each file is materialized from one or more Chroma documents. Every chunk must
|
||||
include metadata that identifies the file slug and chunk order:
|
||||
|
||||
```json
|
||||
{
|
||||
"page_slug": "guides/quickstart.md",
|
||||
"chunk_index": 0
|
||||
}
|
||||
```
|
||||
|
||||
Default metadata fields:
|
||||
|
||||
| Field | Default | Purpose |
|
||||
| ----- | ------- | ------- |
|
||||
| Slug field | `page_slug` | File path in the virtual tree |
|
||||
| Chunk index field | `chunk_index` | Sort order when assembling file content |
|
||||
|
||||
If your collection uses different metadata names, configure
|
||||
`ChromaConfig(slug_field="...", chunk_index_field="...")`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
CHROMA_COLLECTION=knowledge
|
||||
CHROMA_HOST=localhost
|
||||
CHROMA_PORT=8000
|
||||
CHROMA_SSL=false
|
||||
CHROMA_SLUG_FIELD=page_slug
|
||||
CHROMA_CHUNK_INDEX_FIELD=chunk_index
|
||||
CHROMA_EXAMPLE_QUERY=getting started
|
||||
```
|
||||
|
||||
## Validate the Collection
|
||||
|
||||
Before mounting, verify that the collection contains the path tree document and
|
||||
at least one chunk with matching slug metadata:
|
||||
|
||||
```python
|
||||
import chromadb
|
||||
|
||||
client = chromadb.HttpClient(host="localhost", port=8000)
|
||||
collection = client.get_collection("knowledge")
|
||||
|
||||
tree = collection.get(ids=["__path_tree__"])
|
||||
chunks = collection.get(where={"page_slug": "guides/quickstart.md"})
|
||||
|
||||
print(tree["documents"][0])
|
||||
print(len(chunks["documents"]))
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python Chroma Setup](/python/setup/chroma)
|
||||
guide.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
title: Databricks Volume
|
||||
icon: folder-open
|
||||
description: Set up Databricks credentials for the Unity Catalog volume resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
The resource reads a Unity Catalog volume over the Databricks SDK. It needs a
|
||||
workspace host plus one auth method: a personal access token, a config
|
||||
profile, or the SDK defaults (handy inside Databricks Apps).
|
||||
|
||||
### 1. Get Your Workspace Host
|
||||
|
||||
Your workspace URL, e.g. `https://dbc-xxxxxxxx-xxxx.cloud.databricks.com`
|
||||
(AWS) or `https://adb-xxxxxxxxxxxx.xx.azuredatabricks.net` (Azure).
|
||||
|
||||
### 2. Create a Personal Access Token
|
||||
|
||||
1. In the workspace, open **Settings** -> **Developer** -> **Access tokens**.
|
||||
1. Click **Generate new token**, set a comment and lifetime, and copy it.
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
DATABRICKS_HOST=https://dbc-xxxxxxxx-xxxx.cloud.databricks.com
|
||||
DATABRICKS_TOKEN=dapi...
|
||||
```
|
||||
|
||||
### Alternatives
|
||||
|
||||
- **Config profile:** add a profile to `~/.databrickscfg` and pass
|
||||
`profile="DEV"` instead of host/token.
|
||||
- **SDK defaults:** omit host, token, and profile to fall through to the
|
||||
Databricks SDK default auth chain (for example inside Databricks Apps).
|
||||
|
||||
For Python configuration, see the [Python Databricks Volume Setup](/python/setup/databricks) guide.
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: Dify
|
||||
icon: /images/dify-logo.svg
|
||||
description: Set up a Dify Knowledge API key and dataset for the Dify resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
Mirage uses the Dify Knowledge Base API. You need a dataset API key and the
|
||||
dataset ID for the knowledge base you want to mount.
|
||||
|
||||
### 1. Get a Knowledge API Key
|
||||
|
||||
1. Open your Dify workspace.
|
||||
1. Go to the target knowledge base.
|
||||
1. Open **API Access** or the dataset API settings.
|
||||
1. Create or copy an API key with permission to read documents, segments, and retrieval results.
|
||||
|
||||
The key is sent as a bearer token to the Dify API.
|
||||
|
||||
### 2. Find the Dataset ID
|
||||
|
||||
Use the dataset ID from the knowledge base URL or Dify API settings. The ID is
|
||||
used in API paths such as:
|
||||
|
||||
```text
|
||||
/datasets/<dataset-id>/documents
|
||||
```
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
DIFY_API_KEY=dataset-...
|
||||
DIFY_BASE_URL=https://api.dify.ai/v1
|
||||
DIFY_DATASET_ID=...
|
||||
DIFY_SLUG_METADATA_NAME=slug
|
||||
```
|
||||
|
||||
For self-hosted Dify, set `DIFY_BASE_URL` to your API base URL, for example:
|
||||
|
||||
```bash
|
||||
DIFY_BASE_URL=https://dify.example.com/v1
|
||||
```
|
||||
|
||||
## Document Metadata
|
||||
|
||||
Mirage can mount a Dify dataset without custom metadata, but scoped search works
|
||||
best when every document has a stable slug metadata field. Mirage looks for a
|
||||
field named `slug` by default; Python users can change that field name with
|
||||
`DifyConfig(slug_metadata_name="...")`.
|
||||
|
||||
### Recommended: Add a Slug Metadata Field
|
||||
|
||||
Create a metadata field on the Dify knowledge dataset and use the value as the
|
||||
virtual path below the Mirage mount. The default field name is `slug`.
|
||||
|
||||
| Dify document | `slug` metadata value | Mirage path |
|
||||
| ------------- | --------------------- | ----------- |
|
||||
| Quickstart | `guides/quickstart.md` | `/knowledge/guides/quickstart.md` |
|
||||
| Support Policy | `policies/support.md` | `/knowledge/policies/support.md` |
|
||||
|
||||
If your Dify metadata field is named `path`, set `slug_metadata_name="path"` in
|
||||
the Dify resource config and use `name: path` in Dify metadata.
|
||||
|
||||
Guidelines:
|
||||
|
||||
- Use forward slashes to create folders.
|
||||
- Do not start with `/`; Mirage normalizes leading and trailing slashes.
|
||||
- Do not use empty segments, `.`, or `..`.
|
||||
- Keep values unique. Two documents with the same slug metadata value will raise a duplicate path error.
|
||||
- Avoid making one document's slug metadata value a parent of another document's slug metadata value. For example, do not use both `guides` and `guides/quickstart.md`; Mirage treats that as a path collision.
|
||||
|
||||
### Add or Update Slug Metadata in Dify
|
||||
|
||||
In Dify's Knowledge UI, open each document and edit its metadata. Add a custom
|
||||
metadata item:
|
||||
|
||||
```text
|
||||
name: slug
|
||||
value: guides/quickstart.md
|
||||
```
|
||||
|
||||
If you upload or sync documents programmatically, include the same metadata on
|
||||
the document. Mirage reads Dify's `doc_metadata` field and looks for an item
|
||||
whose `name` matches `slug_metadata_name`.
|
||||
|
||||
### Enable Built-in Fields for Name-Based Documents
|
||||
|
||||
If a document does not have the configured slug metadata field, Mirage falls
|
||||
back to the Dify document name for its file path. Scoped `search` can still
|
||||
target those name-based documents, but Dify must expose the `document_name`
|
||||
built-in field for metadata filtering.
|
||||
|
||||
In the Dify knowledge dataset settings:
|
||||
|
||||
1. Open the dataset metadata settings.
|
||||
1. Enable **Built-in Fields**.
|
||||
1. Ensure `document_name` is available for metadata filtering.
|
||||
|
||||
Without Built-in Fields, commands like `ls`, `find`, `cat`, and `grep` still
|
||||
work because Mirage resolves files through the document list and segment APIs.
|
||||
Scoped `search` against name-based paths may return no records because Dify
|
||||
cannot filter retrieval by `document_name`.
|
||||
|
||||
For the most predictable behavior, add the configured slug metadata field to
|
||||
every document and use slug-based paths.
|
||||
|
||||
For Python configuration, see the [Python Dify Setup](/python/setup/dify) guide.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
title: DigitalOcean Spaces
|
||||
icon: /images/digitalocean-logo.svg
|
||||
description: Set up DigitalOcean Spaces credentials for the DigitalOcean resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create a Spaces access key
|
||||
|
||||
1. Sign in to the [DigitalOcean console](https://cloud.digitalocean.com/)
|
||||
1. Go to **API** -> **Spaces Keys** -> **Generate New Key**
|
||||
1. Copy the **Access Key** and **Secret**
|
||||
1. Note your Space's **region** (e.g. `nyc3`) and **Space name** (the bucket)
|
||||
|
||||
The endpoint is derived from `region` as `<region>.digitaloceanspaces.com`.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
DO_SPACE=my-space
|
||||
DO_REGION=nyc3
|
||||
DO_ACCESS_KEY_ID=...
|
||||
DO_SECRET_ACCESS_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [DigitalOcean resource](/python/resource/digitalocean) docs.
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Discord
|
||||
icon: discord
|
||||
description: Set up a Discord Bot Token for the Discord resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create a Discord Application
|
||||
|
||||
1. Go to https://discord.com/developers/applications
|
||||
1. **New Application** -> name it (e.g., "Mirage")
|
||||
|
||||
### 2. Create a Bot
|
||||
|
||||
1. **Bot** in the left sidebar
|
||||
1. **Reset Token** -> copy the **Bot Token**
|
||||
1. Under **Privileged Gateway Intents**, enable:
|
||||
- **Message Content Intent** (required to read message content)
|
||||
|
||||
### 3. Invite the Bot to Your Server
|
||||
|
||||
1. **OAuth2** -> **URL Generator** in the left sidebar
|
||||
1. Select scopes: `bot`
|
||||
1. Select bot permissions: `Read Messages/View Channels`, `Read Message History`
|
||||
1. Copy the generated URL and open it in a browser to invite
|
||||
|
||||
### 4. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
DISCORD_BOT_TOKEN=MTIx...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python Discord Setup](/python/setup/discord) guide.
|
||||
@@ -0,0 +1,199 @@
|
||||
---
|
||||
title: Dropbox
|
||||
icon: dropbox
|
||||
description: Set up Dropbox OAuth2 credentials and obtain a long-lived refresh token.
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The Dropbox resource uses OAuth2 with a long-lived refresh token to authenticate against the
|
||||
[Dropbox v2 HTTP API](https://www.dropbox.com/developers/documentation/http/documentation):
|
||||
|
||||
- **`/2/files/list_folder`**, list folder entries
|
||||
- **`/2/files/download`**, download file bytes
|
||||
- **`/2/files/search_v2`**, search files by name/content
|
||||
- **`/2/files/get_metadata`**, fetch entry metadata
|
||||
|
||||
Two flows are supported:
|
||||
|
||||
- **Code flow with client secret**, for Node/server. Needs `client_id`, `client_secret`, `refresh_token`.
|
||||
- **PKCE flow**, for browsers. Needs only `client_id` and `refresh_token` (no secret in the bundle).
|
||||
|
||||
Both produce the same long-lived refresh token; the resource auto-refreshes short-lived access
|
||||
tokens (≈4h) behind the scenes.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Create a Dropbox App
|
||||
|
||||
1. Go to https://www.dropbox.com/developers/apps
|
||||
1. Click **Create app**
|
||||
1. Choose:
|
||||
- API: **Scoped access**
|
||||
- Access type: **App folder** (sandboxed to one folder Dropbox creates for you) or
|
||||
**Full Dropbox** (your entire account). Pick App folder for least privilege.
|
||||
- Name it (e.g., "Mirage") -> **Create app**
|
||||
|
||||
### 2. Configure Permissions
|
||||
|
||||
On the app's settings page, click the **Permissions** tab and check:
|
||||
|
||||
- `files.metadata.read`, list folders, read file metadata
|
||||
- `files.content.read`, download file bytes
|
||||
|
||||
If you also want write/delete (not used by the read-only resource yet), check
|
||||
`files.content.write` and `files.metadata.write`.
|
||||
|
||||
Click **Submit** at the bottom of the Permissions tab. **Required**: Dropbox does not include
|
||||
unchecked scopes in tokens, even if the auth URL requests them.
|
||||
|
||||
### 3. Configure OAuth 2 Redirect URI
|
||||
|
||||
On the **Settings** tab, scroll to **OAuth 2 -> Redirect URIs** and add:
|
||||
|
||||
- For the CLI flow below: `http://localhost:1`
|
||||
- For the browser PKCE example: `http://localhost:5173/dropbox_pkce.html`
|
||||
|
||||
Click **Add** after each.
|
||||
|
||||
### 4. Copy the App Key (and Secret)
|
||||
|
||||
Still on **Settings**, copy:
|
||||
|
||||
- **App key**, this is your `DROPBOX_APP_KEY`
|
||||
- **App secret**, click **Show** -> copy. This is your `DROPBOX_APP_SECRET`. **Skip if you
|
||||
only need the PKCE flow.**
|
||||
|
||||
### 5. Get the Refresh Token (CLI flow with client secret)
|
||||
|
||||
This mirrors the Google flow, easiest path for Node/server use.
|
||||
|
||||
**A) Open this URL in a browser** (replace `YOUR_APP_KEY`):
|
||||
|
||||
```
|
||||
https://www.dropbox.com/oauth2/authorize?client_id=YOUR_APP_KEY&response_type=code&token_access_type=offline&redirect_uri=http://localhost:1
|
||||
```
|
||||
|
||||
- `token_access_type=offline`, **required** to receive a refresh token. Without this you only
|
||||
get a short-lived access token.
|
||||
|
||||
**B) Authorize:** sign in -> click **Allow**.
|
||||
|
||||
**C) Copy the code:** the browser redirects to `http://localhost:1?code=ABC...` (the page won't
|
||||
load, expected). Copy the `code` value from the URL bar.
|
||||
|
||||
**D) Exchange the code for a refresh token:**
|
||||
|
||||
```bash
|
||||
curl https://api.dropboxapi.com/oauth2/token \
|
||||
-d "code=THE_CODE_FROM_STEP_C" \
|
||||
-d "grant_type=authorization_code" \
|
||||
-d "client_id=YOUR_APP_KEY" \
|
||||
-d "client_secret=YOUR_APP_SECRET" \
|
||||
-d "redirect_uri=http://localhost:1"
|
||||
```
|
||||
|
||||
The response contains `refresh_token`, save it. Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"access_token": "sl.B...",
|
||||
"expires_in": 14400,
|
||||
"refresh_token": "abc123...",
|
||||
"scope": "files.metadata.read files.content.read",
|
||||
"token_type": "bearer",
|
||||
"uid": "12345",
|
||||
"account_id": "dbid:..."
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
DROPBOX_APP_KEY=abc123app4key
|
||||
DROPBOX_APP_SECRET=def456app4secret
|
||||
DROPBOX_REFRESH_TOKEN=ghi789refresh4token
|
||||
```
|
||||
|
||||
## Alternative: PKCE Flow (browser, no client secret)
|
||||
|
||||
If you're mounting Dropbox from a browser SPA and don't want to ship a client secret, use the
|
||||
PKCE flow. The bundled example does the dance end-to-end.
|
||||
|
||||
1. In the Dropbox app **Settings** tab, ensure the redirect URI
|
||||
`http://localhost:5173/dropbox_pkce.html` is registered (Step 3).
|
||||
1. Set only `DROPBOX_APP_KEY` in `.env.development` at the repo root (no secret needed).
|
||||
1. From `examples/typescript/browser/`, run `pnpm dev`.
|
||||
1. Open `http://localhost:5173/dropbox_pkce.html` and click **Connect Dropbox**.
|
||||
|
||||
The example persists the refresh token to `localStorage`, then mounts a `DropboxResource` with
|
||||
just `{ clientId, refreshToken }` and runs `ls /dropbox/`. Inspect DevTools -> Network -> filter
|
||||
`token` to confirm refresh calls don't include `client_secret`.
|
||||
|
||||
The same `refresh_token` can be reused for headless setups, copy it out of `localStorage` and
|
||||
set it as `DROPBOX_REFRESH_TOKEN` in your env.
|
||||
|
||||
## Token Lifetime
|
||||
|
||||
| Token | Lifetime |
|
||||
| ------------- | ------------------------------------------ |
|
||||
| Access token | ~4 hours (`expires_in: 14400`) |
|
||||
| Refresh token | Long-lived; survives until manually revoked |
|
||||
|
||||
Refresh tokens get revoked if:
|
||||
|
||||
- The user revokes the app at https://www.dropbox.com/account/connected_apps
|
||||
- The Dropbox account password is changed
|
||||
- The app is deleted in the developer console
|
||||
|
||||
The `DropboxTokenManager` in the resource transparently exchanges the refresh token for a fresh
|
||||
access token whenever the cached one is within 5 minutes of expiry.
|
||||
|
||||
## Scopes Reference
|
||||
|
||||
| Scope | Purpose |
|
||||
| ------------------------ | ------------------------------------ |
|
||||
| `files.metadata.read` | List folders, read file metadata |
|
||||
| `files.metadata.write` | Move, rename, create folders |
|
||||
| `files.content.read` | Download file bytes |
|
||||
| `files.content.write` | Upload, overwrite, delete files |
|
||||
| `sharing.read` | List shared links and shared folders |
|
||||
| `account_info.read` | Account email, country, user info |
|
||||
|
||||
The Mirage resource only needs `files.metadata.read` + `files.content.read` for the read-only
|
||||
mount. Add the `*.write` scopes if/when write commands land.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Fix |
|
||||
| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
|
||||
| `400 invalid_grant` on token exchange | The `code` is single-use and expires fast, re-do Step 5A with a fresh URL |
|
||||
| No `refresh_token` in token response | Add `token_access_type=offline` to the auth URL |
|
||||
| `path/not_found/.` on `ls /dropbox/` | Account is empty, or app is **App folder** scoped and the sandbox folder hasn't been created |
|
||||
| `missing_scope` on download | Check `files.content.read` is enabled in the **Permissions** tab and **submitted** |
|
||||
| Browser PKCE: redirect URI mismatch | Dropbox is strict, `http://localhost:5173/dropbox_pkce.html` must match exactly (no trailing slash) |
|
||||
| Token works locally but fails after a while | Access token expired, the `TokenManager` should auto-refresh; if not, refresh token may be revoked |
|
||||
|
||||
For TypeScript usage, mount with:
|
||||
|
||||
```ts
|
||||
import { DropboxResource } from '@struktoai/mirage-node'
|
||||
|
||||
const dropbox = new DropboxResource({
|
||||
clientId: process.env.DROPBOX_APP_KEY!,
|
||||
clientSecret: process.env.DROPBOX_APP_SECRET!,
|
||||
refreshToken: process.env.DROPBOX_REFRESH_TOKEN!,
|
||||
})
|
||||
```
|
||||
|
||||
Or browser (PKCE):
|
||||
|
||||
```ts
|
||||
import { DropboxResource } from '@struktoai/mirage-browser'
|
||||
|
||||
const dropbox = new DropboxResource({
|
||||
clientId: 'YOUR_APP_KEY',
|
||||
refreshToken: refreshTokenFromLocalStorage,
|
||||
})
|
||||
```
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
title: Email
|
||||
icon: envelope
|
||||
description: Set up IMAP/SMTP credentials for the Email resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
The Email resource connects to any email account via IMAP (reading)
|
||||
and SMTP (sending). You need the server hostnames and a password or
|
||||
app password.
|
||||
|
||||
### 1. Find Your IMAP/SMTP Settings
|
||||
|
||||
| Resource | IMAP Host | IMAP Port | SMTP Host | SMTP Port |
|
||||
| ----------- | ----------------------- | --------- | --------------------- | --------- |
|
||||
| Outlook/365 | `outlook.office365.com` | 993 | `smtp.office365.com` | 587 |
|
||||
| Yahoo | `imap.mail.yahoo.com` | 993 | `smtp.mail.yahoo.com` | 587 |
|
||||
| Fastmail | `imap.fastmail.com` | 993 | `smtp.fastmail.com` | 587 |
|
||||
| iCloud | `imap.mail.me.com` | 993 | `smtp.mail.me.com` | 587 |
|
||||
| ProtonMail | `127.0.0.1` | 1143 | `127.0.0.1` | 1025 |
|
||||
| Self-hosted | Your server hostname | 993 | Your server hostname | 587 |
|
||||
|
||||
ProtonMail requires [ProtonMail Bridge](https://proton.me/mail/bridge)
|
||||
running locally.
|
||||
|
||||
### 2. Create an App Password
|
||||
|
||||
Most resources require an **app password** instead of your regular
|
||||
account password. This is a one-time password specifically for
|
||||
third-party apps.
|
||||
|
||||
**Outlook / Microsoft 365:**
|
||||
|
||||
1. Go to https://account.microsoft.com/security
|
||||
1. **Security** -> **Advanced security options**
|
||||
1. **App passwords** -> **Create a new app password**
|
||||
1. Copy the generated password
|
||||
|
||||
**Yahoo:**
|
||||
|
||||
1. Go to https://login.yahoo.com/account/security
|
||||
1. **Generate app password**
|
||||
1. Select **Other App**, name it (e.g., "Mirage")
|
||||
1. Copy the generated password
|
||||
|
||||
**Fastmail:**
|
||||
|
||||
1. Go to https://www.fastmail.com/settings/security/tokens
|
||||
1. **New App Password**
|
||||
1. Select **IMAP/SMTP** access
|
||||
1. Copy the generated password
|
||||
|
||||
**iCloud:**
|
||||
|
||||
1. Go to https://appleid.apple.com/account/manage
|
||||
1. **Sign-In and Security** -> **App-Specific Passwords**
|
||||
1. **Generate an app-specific password**
|
||||
1. Copy the generated password
|
||||
|
||||
**Self-hosted / Other:**
|
||||
|
||||
Use your regular email password, or whatever your mail server
|
||||
requires.
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
IMAP_HOST=imap.fastmail.com
|
||||
SMTP_HOST=smtp.fastmail.com
|
||||
EMAIL_USERNAME=you@fastmail.com
|
||||
EMAIL_PASSWORD=your-app-password
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"Authentication failed"** - Make sure you're using an app password,
|
||||
not your regular account password. Most resources block regular
|
||||
passwords for IMAP/SMTP access.
|
||||
|
||||
**"Connection refused"** - Check the hostname and port. Some networks
|
||||
block port 993/587. For ProtonMail, make sure Bridge is running.
|
||||
|
||||
**"SSL handshake failed"** - If your server uses STARTTLS on port 143
|
||||
instead of SSL on port 993, set `use_ssl=False` and `imap_port=143`.
|
||||
|
||||
For Python configuration, see the [Python Email Setup](/python/setup/email) guide.
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
title: Google Cloud Storage
|
||||
icon: google
|
||||
description: Set up Google Cloud Storage credentials for the GCS resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create HMAC Keys
|
||||
|
||||
1. Go to https://console.cloud.google.com/storage/settings -> **Interoperability**
|
||||
1. Under **Service account HMAC**, click **Create a key for a service account**
|
||||
1. Select a service account with Storage Object Viewer (read) or
|
||||
Storage Object Admin (read/write)
|
||||
1. Copy the **Access key** and **Secret**
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
GCS_BUCKET=mirage-ai
|
||||
GCS_ACCESS_KEY_ID=GOOG...
|
||||
GCS_SECRET_ACCESS_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [GCS resource](/python/resource/gcs) docs.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
title: GitHub
|
||||
icon: github
|
||||
description: Set up a GitHub Personal Access Token for the GitHub resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create a Personal Access Token
|
||||
|
||||
1. Go to https://github.com/settings/tokens
|
||||
1. **Generate new token (classic)** or **Fine-grained token**
|
||||
1. For classic tokens, select scopes:
|
||||
- `repo` (read access to repositories)
|
||||
1. For fine-grained tokens:
|
||||
- **Repository access**: select the repos you need
|
||||
- **Permissions**: Contents -> Read-only
|
||||
1. Copy the token
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
GITHUB_TOKEN=ghp_xxxx...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python GitHub Setup](/python/setup/github) guide.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
title: GitHub CI
|
||||
icon: circle-play
|
||||
description: Set up a GitHub Personal Access Token for the GitHub CI resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create a Personal Access Token
|
||||
|
||||
1. Go to https://github.com/settings/tokens
|
||||
1. **Generate new token (classic)** or **Fine-grained token**
|
||||
1. For classic tokens, select scopes:
|
||||
- `repo` (includes `actions:read`)
|
||||
1. For fine-grained tokens:
|
||||
- **Repository access**: select the repos you need
|
||||
- **Permissions**: Actions -> Read-only
|
||||
1. Copy the token
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
GITHUB_TOKEN=ghp_xxxx...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python GitHub CI Setup](/python/setup/github_ci) guide.
|
||||
@@ -0,0 +1,142 @@
|
||||
---
|
||||
title: Google Workspace
|
||||
icon: google
|
||||
description: Set up Google OAuth2 credentials for Docs, Sheets, Slides, Drive, and Gmail resources.
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The Google resources use OAuth2 with a refresh token to authenticate against:
|
||||
|
||||
- **Google Drive API v3** - lists documents, reads metadata
|
||||
- **Google Docs API v1** - reads/writes document content
|
||||
|
||||
Authentication requires three values: `client_id`, `client_secret`, and `refresh_token`.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Create a Google Cloud Project
|
||||
|
||||
1. Go to https://console.cloud.google.com/
|
||||
1. Click the project selector dropdown -> **New Project**
|
||||
1. Name it (e.g., "Mirage") -> **Create**
|
||||
1. Select it from the dropdown
|
||||
|
||||
### 2. Enable APIs
|
||||
|
||||
1. Go to https://console.cloud.google.com/apis/library
|
||||
1. Search **Google Drive API** -> click -> **Enable**
|
||||
1. Search **Google Docs API** -> click -> **Enable**
|
||||
|
||||
### 3. Configure Google Auth Platform
|
||||
|
||||
The OAuth settings are under **Google Auth Platform** in the Cloud Console left sidebar (or go to https://console.cloud.google.com/auth/overview).
|
||||
|
||||
**A) Branding** (left sidebar):
|
||||
|
||||
1. Fill in: App name (e.g., "Mirage"), support email, developer contact email
|
||||
1. Save
|
||||
|
||||
**B) Audience** (left sidebar):
|
||||
|
||||
1. Set user type to **External**
|
||||
1. Add your own Google email as a **test user**
|
||||
1. Save
|
||||
|
||||
**C) Data Access** (left sidebar) - this is where scopes are configured:
|
||||
|
||||
1. Click **Add or Remove Scopes**
|
||||
1. Search for or paste these scopes:
|
||||
- `https://www.googleapis.com/auth/drive` (full Drive access)
|
||||
- `https://www.googleapis.com/auth/documents` (Google Docs)
|
||||
- `https://www.googleapis.com/auth/presentations` (Google Slides)
|
||||
1. Select them -> Save
|
||||
|
||||
**D) Publish** (to avoid 7-day token expiry):
|
||||
|
||||
1. Go to **Audience** -> click **Publish App**
|
||||
1. See [Token Lifetime](#token-lifetime) for details
|
||||
|
||||
### 4. Create OAuth2 Client
|
||||
|
||||
1. Go to **Clients** in the left sidebar (or click **Create OAuth client** on the Overview page)
|
||||
1. Application type: **Desktop app**
|
||||
1. Name it -> **Create**
|
||||
1. Copy the **Client ID** and **Client Secret**
|
||||
|
||||
### 5. Get the Refresh Token
|
||||
|
||||
**A) Open this URL in a browser** (replace `YOUR_CLIENT_ID`):
|
||||
|
||||
```
|
||||
https://accounts.google.com/o/oauth2/v2/auth?client_id=YOUR_CLIENT_ID&redirect_uri=http://localhost:1&response_type=code&scope=https://www.googleapis.com/auth/drive%20https://www.googleapis.com/auth/documents%20https://www.googleapis.com/auth/presentations&access_type=offline&prompt=consent
|
||||
```
|
||||
|
||||
- `access_type=offline` - required to receive a refresh token
|
||||
- `prompt=consent` - forces refresh token issuance on re-auth
|
||||
|
||||
**B) Authorize:** Sign in -> click through "unverified app" warning -> grant permissions.
|
||||
|
||||
**C) Copy the code:** The browser redirects to `http://localhost:1?code=4/0AXXXX...` (page won't load - expected). Copy the `code` value from the URL bar.
|
||||
|
||||
**D) Exchange for tokens:**
|
||||
|
||||
```bash
|
||||
curl -X POST https://oauth2.googleapis.com/token \
|
||||
-d "client_id=YOUR_CLIENT_ID" \
|
||||
-d "client_secret=YOUR_CLIENT_SECRET" \
|
||||
-d "code=THE_CODE_FROM_STEP_C" \
|
||||
-d "grant_type=authorization_code" \
|
||||
-d "redirect_uri=http://localhost:1"
|
||||
```
|
||||
|
||||
The response JSON contains `refresh_token` - save it.
|
||||
|
||||
### 6. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
GOOGLE_CLIENT_ID=123456789-abc.apps.googleusercontent.com
|
||||
GOOGLE_CLIENT_SECRET=GOCSPx-xxx
|
||||
GOOGLE_REFRESH_TOKEN=1//0abc...
|
||||
```
|
||||
|
||||
## Token Lifetime
|
||||
|
||||
| App Publishing Status | Refresh Token Lifetime |
|
||||
| --------------------- | ---------------------------------- |
|
||||
| Testing (default) | 7 days - must re-authorize |
|
||||
| In Production | Never expires (with continued use) |
|
||||
|
||||
To publish: go to the OAuth consent screen page and click **Publish App**. Users will see an "unverified app" warning during auth, which is fine for personal use.
|
||||
|
||||
A published refresh token only gets revoked if:
|
||||
|
||||
- Manually revoked at https://myaccount.google.com/permissions
|
||||
- Google account password is changed
|
||||
- Token unused for 6 months
|
||||
- 100+ outstanding refresh tokens per account per client (oldest revoked)
|
||||
|
||||
For personal use with a published app, you do the auth flow **once** and it works indefinitely. The `TokenManager` in the resource automatically uses the refresh token to obtain fresh access tokens (which expire hourly) behind the scenes.
|
||||
|
||||
## Scopes Reference
|
||||
|
||||
| Scope | Purpose |
|
||||
| ---------------------------------------------------- | ---------------------------------------------- |
|
||||
| `https://www.googleapis.com/auth/drive` | Full Drive access (list, create, delete files) |
|
||||
| `https://www.googleapis.com/auth/drive.readonly` | List files, read metadata only (alternative) |
|
||||
| `https://www.googleapis.com/auth/documents` | Read and write Google Docs |
|
||||
| `https://www.googleapis.com/auth/documents.readonly` | Read-only Docs access (alternative) |
|
||||
| `https://www.googleapis.com/auth/presentations` | Read and write Google Slides |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Fix |
|
||||
| ---------------------------------- | ----------------------------------------------------------------- |
|
||||
| 403 during OAuth flow | Add yourself as a test user in Step 3.7 |
|
||||
| No `refresh_token` in response | Ensure `access_type=offline` and `prompt=consent` in the auth URL |
|
||||
| Token exchange fails | `redirect_uri` must exactly match between auth URL and curl |
|
||||
| Refresh token expires after 7 days | Publish the app (Step 3.9) |
|
||||
| Auth code doesn't work twice | Codes are single-use - re-authorize if exchange fails |
|
||||
|
||||
For Python configuration, see the [Python Google Workspace Setup](/python/setup/google) guide.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: HF Buckets
|
||||
icon: /images/huggingface-logo.svg
|
||||
description: Set up Hugging Face Buckets credentials for the HF Buckets resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create a Bucket
|
||||
|
||||
1. Go to https://huggingface.co/new-bucket and create a bucket
|
||||
2. Note the full name in `namespace/bucket-name` form
|
||||
(e.g. `your-user/my-data`)
|
||||
|
||||
### 2. Create an Access Token
|
||||
|
||||
1. Go to https://huggingface.co/settings/tokens
|
||||
2. Create a fine-grained token with **Read** (or **Write** if you need
|
||||
to upload) access to the bucket's namespace
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
HF_BUCKET_NAME=your-user/my-data
|
||||
HF_TOKEN=hf_...
|
||||
```
|
||||
|
||||
For Python configuration, see the
|
||||
[HF Buckets resource](/python/resource/hf_buckets) docs.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: HF Datasets
|
||||
icon: /images/huggingface-logo.svg
|
||||
description: Set up Hugging Face Datasets credentials.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Pick a Dataset Repo
|
||||
|
||||
Browse [huggingface.co/datasets](https://huggingface.co/datasets) and note the
|
||||
full `namespace/dataset-name` (e.g. `AlienKevin/SWE-ZERO-12M-trajectories`).
|
||||
|
||||
Public datasets need no token. Gated/private datasets do.
|
||||
|
||||
### 2. Create an Access Token (gated/private only)
|
||||
|
||||
1. Go to https://huggingface.co/settings/tokens
|
||||
2. Create a fine-grained token with **Read** access to the dataset
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
HF_DATASET_REPO=AlienKevin/SWE-ZERO-12M-trajectories
|
||||
HF_TOKEN=hf_... # optional, public datasets work without it
|
||||
```
|
||||
|
||||
For Python configuration, see the
|
||||
[HF Datasets resource](/python/resource/hf_datasets) docs.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: HF Models
|
||||
icon: /images/huggingface-logo.svg
|
||||
description: Set up Hugging Face Models credentials.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Pick a Model Repo
|
||||
|
||||
Browse [huggingface.co/models](https://huggingface.co/models) and note the
|
||||
full `namespace/model-name` (e.g. `sapientinc/HRM-Text-1B`).
|
||||
|
||||
Public models need no token. Gated/private models do.
|
||||
|
||||
### 2. Create an Access Token (gated/private only)
|
||||
|
||||
1. Go to https://huggingface.co/settings/tokens
|
||||
2. Create a fine-grained token with **Read** access to the model
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
HF_MODEL_REPO=sapientinc/HRM-Text-1B
|
||||
HF_TOKEN=hf_... # optional, public models work without it
|
||||
```
|
||||
|
||||
For Python configuration, see the
|
||||
[HF Models resource](/python/resource/hf_models) docs.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: HF Spaces
|
||||
icon: /images/huggingface-logo.svg
|
||||
description: Set up Hugging Face Spaces credentials.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Pick a Space Repo
|
||||
|
||||
Browse [huggingface.co/spaces](https://huggingface.co/spaces) and note the
|
||||
full `namespace/space-name` (e.g. `HuggingFaceBio/carbon-demo`).
|
||||
|
||||
Public spaces need no token. Private spaces do.
|
||||
|
||||
### 2. Create an Access Token (private only)
|
||||
|
||||
1. Go to https://huggingface.co/settings/tokens
|
||||
2. Create a fine-grained token with **Read** access to the space
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
HF_SPACE_REPO=HuggingFaceBio/carbon-demo
|
||||
HF_TOKEN=hf_... # optional, public spaces work without it
|
||||
```
|
||||
|
||||
For Python configuration, see the
|
||||
[HF Spaces resource](/python/resource/hf_spaces) docs.
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
title: LanceDB
|
||||
icon: database
|
||||
description: Set up a LanceDB connection for the LanceDB resource.
|
||||
---
|
||||
|
||||
LanceDB mounts a table as a read-only filesystem: group-by columns become nested
|
||||
folders, each row is a card plus an optional blob file, and semantic search is
|
||||
the `search` command.
|
||||
|
||||
## Connection
|
||||
|
||||
### Local / embedded
|
||||
|
||||
Point `uri` at a directory. LanceDB stores the table there, no server needed.
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
LANCEDB_URI=/data/fashion.lancedb
|
||||
```
|
||||
|
||||
### Object storage
|
||||
|
||||
`uri` also accepts `s3://`, `gs://`, and `az://` prefixes for tables backed by
|
||||
object storage.
|
||||
|
||||
### Cloud and Enterprise
|
||||
|
||||
`db://` URIs connect to LanceDB Cloud (`api_key` + `region`) or Enterprise
|
||||
(`api_key` + `host_override`):
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
LANCEDB_URI=db://my-database
|
||||
LANCEDB_API_KEY=sk_...
|
||||
LANCEDB_REGION=us-east-1
|
||||
# Enterprise only
|
||||
LANCEDB_HOST_OVERRIDE=https://my-database.us-east-1.api.lancedb.com
|
||||
```
|
||||
|
||||
## Search
|
||||
|
||||
Search is the `search "<query>" <path>` command. It returns ranked rows as their
|
||||
canonical `<id>.md` file paths plus a score, so results compose with `cat`, `wc`,
|
||||
and pipes (`grep`/`rg` stay lexical). It requires a table built with an embedding
|
||||
function on a source field, so `tbl.search(text)` auto-embeds the query.
|
||||
|
||||
## Limits
|
||||
|
||||
The resource is read-only and bounds how much an agent can pull:
|
||||
|
||||
- `search_limit` (10): default top-k returned by `search`.
|
||||
- `max_rows` (1,000): hard ceiling on rows listed per folder.
|
||||
|
||||
See the [Python](/python/resource/lancedb) and [TypeScript](/typescript/setup/lancedb)
|
||||
resource pages for full config.
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
title: Langfuse
|
||||
icon: chart-line
|
||||
description: Set up Langfuse API keys for the Langfuse resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Get API Keys
|
||||
|
||||
#### Langfuse Cloud
|
||||
|
||||
1. Go to https://cloud.langfuse.com
|
||||
1. Select your project -> **Settings** -> **API Keys**
|
||||
1. Copy the **Public Key** and **Secret Key**
|
||||
|
||||
#### Self-hosted
|
||||
|
||||
1. Go to your Langfuse instance (e.g., `https://langfuse.yourcompany.com`)
|
||||
1. Select your project -> **Settings** -> **API Keys**
|
||||
1. Copy the **Public Key** and **Secret Key**
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
LANGFUSE_PUBLIC_KEY=pk-lf-...
|
||||
LANGFUSE_SECRET_KEY=sk-lf-...
|
||||
LANGFUSE_HOST=https://cloud.langfuse.com
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python Langfuse Setup](/python/setup/langfuse) guide.
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
title: Linear
|
||||
icon: chart-gantt
|
||||
description: Set up a Linear personal API key for the Linear resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create a Personal API Key
|
||||
|
||||
1. Go to Settings -> Security & Access
|
||||
1. Create a new personal API key
|
||||
1. Copy the key
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
LINEAR_API_KEY=lin_api_xxxx...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python Linear Setup](/python/setup/linear) guide.
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: Linux
|
||||
icon: linux
|
||||
description: Set up FUSE on Linux for MIRAGE.
|
||||
---
|
||||
|
||||
## FUSE Setup
|
||||
|
||||
FUSE mode mounts virtual filesystems as real directories,
|
||||
allowing any tool (not just MIRAGE commands) to access mounted data.
|
||||
|
||||
### Install FUSE
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Ubuntu / Debian">
|
||||
```bash
|
||||
sudo apt-get install fuse3 libfuse3-dev
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="Fedora / RHEL">
|
||||
```bash
|
||||
sudo dnf install fuse3 fuse3-devel
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="Arch Linux">
|
||||
```bash
|
||||
sudo pacman -S fuse3
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
### Allow User Mounts
|
||||
|
||||
Edit `/etc/fuse.conf` and uncomment `user_allow_other`:
|
||||
|
||||
```bash
|
||||
sudo sed -i 's/#user_allow_other/user_allow_other/' /etc/fuse.conf
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
fusermount3 --version
|
||||
```
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
title: macOS
|
||||
icon: apple
|
||||
description: Set up FUSE on macOS for MIRAGE.
|
||||
---
|
||||
|
||||
## Install macFUSE
|
||||
|
||||
```bash
|
||||
brew install --cask macfuse
|
||||
```
|
||||
|
||||
## Enable Kernel Extension
|
||||
|
||||
<Steps>
|
||||
<Step title="Allow the Extension">
|
||||
Open **System Settings - Privacy & Security**. Scroll to the bottom, you will see:
|
||||
|
||||
> "System software from developer 'Benjamin Fleischer' was blocked from loading."
|
||||
|
||||
Click **Allow**.
|
||||
</Step>
|
||||
<Step title="Restart">
|
||||
Restart your Mac. macOS requires a reboot to load the kernel extension.
|
||||
</Step>
|
||||
<Step title="Verify">
|
||||
```bash
|
||||
ls /Library/Filesystems/macfuse.fs
|
||||
```
|
||||
If this directory exists, macFUSE is installed.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
## Apple Silicon (M1/M2/M3/M4)
|
||||
|
||||
If you don't see the **Allow** button:
|
||||
|
||||
<Steps>
|
||||
<Step title="Shut Down">
|
||||
Shut down your Mac completely.
|
||||
</Step>
|
||||
<Step title="Enter Recovery Mode">
|
||||
Hold the power button until "Loading startup options" appears. Click **Options - Continue**.
|
||||
</Step>
|
||||
<Step title="Enable Kernel Extensions">
|
||||
In Recovery: **Utilities - Startup Security Utility**. Set to **Reduced Security** and check **"Allow user management of kernel extensions"**.
|
||||
</Step>
|
||||
<Step title="Restart">
|
||||
Restart, then go back to **System Settings - Privacy & Security** to allow the extension.
|
||||
</Step>
|
||||
</Steps>
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
title: MinIO
|
||||
icon: /images/minio-logo.svg
|
||||
description: Set up MinIO credentials for the MinIO resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an access key
|
||||
|
||||
1. Open the **MinIO Console** (e.g. `http://localhost:9001`) and sign in
|
||||
1. Go to **Access Keys** -> **Create access key**
|
||||
1. Copy the **Access Key** and **Secret Key**
|
||||
1. Note your server **endpoint** (e.g. `http://localhost:9000`) and **bucket** name
|
||||
|
||||
MinIO is self-hosted, so there is no region-derived endpoint, the `endpoint_url`
|
||||
is required and MinIO uses path-style addressing.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
MINIO_BUCKET=my-bucket
|
||||
MINIO_ENDPOINT=http://localhost:9000
|
||||
MINIO_ACCESS_KEY=...
|
||||
MINIO_SECRET_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [MinIO resource](/python/resource/minio) docs.
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
title: MongoDB
|
||||
icon: database
|
||||
description: Set up a MongoDB connection for the MongoDB resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Get Your Connection URI
|
||||
|
||||
#### Local MongoDB
|
||||
|
||||
```bash
|
||||
# Default local instance
|
||||
MONGODB_URI=mongodb://localhost:27017
|
||||
```
|
||||
|
||||
#### MongoDB Atlas (Cloud)
|
||||
|
||||
1. Go to https://cloud.mongodb.com
|
||||
1. Select your cluster -> **Connect** -> **Drivers**
|
||||
1. Copy the connection string:
|
||||
```
|
||||
mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/
|
||||
```
|
||||
|
||||
#### Self-hosted with Authentication
|
||||
|
||||
```bash
|
||||
MONGODB_URI=mongodb://username:password@host:27017/?authSource=admin
|
||||
```
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
MONGODB_URI=mongodb+srv://user:pass@cluster0.xxxxx.mongodb.net/
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python MongoDB Setup](/python/setup/mongodb) guide.
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
title: Nextcloud
|
||||
icon: cloud
|
||||
description: Set up Nextcloud / ownCloud / WebDAV credentials for the Nextcloud resource.
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The Nextcloud resource talks to any [WebDAV](http://www.webdav.org/specs/rfc4918.html)
|
||||
server: self-hosted Nextcloud, ownCloud, Hetzner Storage Share, or any
|
||||
generic WebDAV endpoint. Authentication is HTTP Basic with an
|
||||
**app password** (not your account password).
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Pick a server
|
||||
|
||||
Any of these works:
|
||||
|
||||
- **Nextcloud Hosted trial** at https://try.nextcloud.com
|
||||
- **Self-hosted**: `docker run -p 8080:80 nextcloud`
|
||||
- **Hetzner Storage Share**, **ownCloud**, or any WebDAV server
|
||||
|
||||
### 2. Create an app password
|
||||
|
||||
In your Nextcloud web UI:
|
||||
|
||||
1. Open **Settings** → **Security** → **Devices & sessions**
|
||||
2. Enter an app name (e.g. `mirage`) and click **Create new app password**
|
||||
3. Copy the generated password (shown once)
|
||||
|
||||
Login password also works but **app passwords are strongly preferred** so
|
||||
you can revoke per-app access without changing your account password.
|
||||
|
||||
### 3. Find your WebDAV URL
|
||||
|
||||
The standard Nextcloud path is:
|
||||
|
||||
```text
|
||||
https://<your-server>/remote.php/dav/files/<username>/
|
||||
```
|
||||
|
||||
For example: `https://cloud.example.com/remote.php/dav/files/alice/`.
|
||||
|
||||
For generic WebDAV servers, use whatever URL the provider gives.
|
||||
|
||||
### 4. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
NEXTCLOUD_URL=https://cloud.example.com/remote.php/dav/files/alice/
|
||||
NEXTCLOUD_USERNAME=alice
|
||||
NEXTCLOUD_PASSWORD=<app-password>
|
||||
```
|
||||
|
||||
For Python configuration, see the
|
||||
[Nextcloud resource](/python/resource/nextcloud) docs.
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: Notion
|
||||
icon: book
|
||||
description: Set up a Notion internal integration for the Notion resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an Internal Integration
|
||||
|
||||
1. Go to [My Integrations](https://www.notion.com/my-integrations)
|
||||
1. Click **+ New integration**
|
||||
1. Name it (e.g. "mirage"), select your workspace
|
||||
1. Under **Content Capabilities**, enable:
|
||||
- **Read content** (required)
|
||||
- **Update content** (for write commands)
|
||||
- **Insert content** (for `notion-comment-add`)
|
||||
1. Click **Save changes**
|
||||
1. Copy the **Internal Integration Secret** (starts with `ntn_`)
|
||||
|
||||
### 2. Share Pages with the Integration
|
||||
|
||||
The integration has no access by default. You must share each page:
|
||||
|
||||
1. Open a Notion page you want to mount
|
||||
1. Click `...` (top-right) -> **+ Add connections**
|
||||
1. Search for your integration name and select it
|
||||
1. Child pages inherit the connection - share a top-level page to expose its entire subtree
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
NOTION_API_KEY=ntn_xxxx...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Python Notion Setup](/python/setup/notion) guide.
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: OCI Object Storage
|
||||
icon: server
|
||||
description: Set up Oracle Cloud Infrastructure Object Storage credentials for the OCI resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
The OCI resource needs:
|
||||
|
||||
- an Object Storage bucket
|
||||
- your Object Storage namespace
|
||||
- your OCI region
|
||||
- a Customer Secret Key pair for the S3 Compatibility API
|
||||
|
||||
### 1. Create a Bucket
|
||||
|
||||
1. Open the OCI Console
|
||||
1. Go to `Storage` -> `Buckets`
|
||||
1. Choose the target compartment
|
||||
1. Click `Create bucket`
|
||||
1. Enter the bucket name, such as `mirage`
|
||||
1. Keep the storage tier as `Standard` unless you specifically want archive
|
||||
|
||||
If you want MIRAGE data under a prefix like `data/`, create objects under that
|
||||
prefix inside the bucket. The prefix is not part of the bucket name.
|
||||
|
||||
### 2. Get the Namespace
|
||||
|
||||
1. Open the profile menu in the OCI Console
|
||||
1. Go to `Tenancy`
|
||||
1. Copy the Object Storage namespace string, such as `idsiqbdbcr4i`
|
||||
|
||||
You can also fetch it with:
|
||||
|
||||
```bash
|
||||
oci os ns get
|
||||
```
|
||||
|
||||
### 3. Get S3-compatible Keys
|
||||
|
||||
1. Open the profile menu
|
||||
1. Go to `User settings`
|
||||
1. Open `Customer secret keys`
|
||||
1. Click `Generate secret key`
|
||||
1. Copy both the `Access Key` and `Secret Key`
|
||||
|
||||
Oracle only shows the secret once, so save it immediately.
|
||||
|
||||
### 4. Find the Region
|
||||
|
||||
Use the OCI Console region selector in the top-right. For example:
|
||||
|
||||
- `us-ashburn-1`
|
||||
- `us-phoenix-1`
|
||||
- `eu-frankfurt-1`
|
||||
|
||||
### 5. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
OCI_BUCKET=mirage
|
||||
OCI_NAMESPACE=idsiqbdbcr4i
|
||||
OCI_REGION=us-ashburn-1
|
||||
OCI_ACCESS_KEY_ID=...
|
||||
OCI_SECRET_ACCESS_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [OCI resource](/python/resource/oci) docs.
|
||||
@@ -0,0 +1,103 @@
|
||||
---
|
||||
title: OneDrive
|
||||
icon: microsoft
|
||||
description: Get a Microsoft Graph access token and drive ID for the OneDrive resource.
|
||||
---
|
||||
|
||||
The OneDrive resource talks to Microsoft OneDrive and SharePoint document
|
||||
libraries through the [Microsoft Graph API](https://learn.microsoft.com/en-us/graph/).
|
||||
It authenticates with an OAuth2 **bearer access token**, optionally scoped to a
|
||||
specific **drive**.
|
||||
|
||||
## Credentials
|
||||
|
||||
You need two things: an **access token**, and (for app-only tokens or SharePoint)
|
||||
a **drive ID**. Pick whichever token path fits.
|
||||
|
||||
### Option A: Quick token (Graph Explorer)
|
||||
|
||||
Fastest way to try it. The token lasts about an hour, which is plenty for a test run.
|
||||
|
||||
1. Open [Graph Explorer](https://developer.microsoft.com/graph/graph-explorer) and **Sign in**.
|
||||
1. Run any query (for example `GET /me/drive`) and consent when prompted.
|
||||
1. Open the **Access token** tab and copy the token.
|
||||
|
||||
This is a **delegated** token, so it works against your own `/me/drive` and you do
|
||||
**not** need a drive ID.
|
||||
|
||||
### Option B: Repeatable token (app registration + device code)
|
||||
|
||||
Use this for scripted or CI runs.
|
||||
|
||||
1. In the [Microsoft Entra admin center](https://entra.microsoft.com), go to
|
||||
**Identity** -> **Applications** -> **App registrations** -> **New registration**.
|
||||
For a personal account, choose *Accounts in any organizational directory and
|
||||
personal Microsoft accounts*.
|
||||
1. Under **Authentication** -> **Advanced settings**, set
|
||||
**Allow public client flows** to **Yes**.
|
||||
1. Under **API permissions** -> **Microsoft Graph** -> **Delegated**, add
|
||||
`Files.ReadWrite.All` (add `Sites.ReadWrite.All` for SharePoint libraries).
|
||||
1. Run the device-code flow with your **Application (client) ID**:
|
||||
|
||||
```bash
|
||||
CLIENT_ID="<your app client id>"
|
||||
|
||||
# 1) request a device code
|
||||
curl -s -X POST "https://login.microsoftonline.com/common/oauth2/v2.0/devicecode" \
|
||||
-d "client_id=$CLIENT_ID" \
|
||||
-d "scope=Files.ReadWrite.All offline_access"
|
||||
# -> open https://microsoft.com/devicelogin and enter the user_code it prints
|
||||
|
||||
# 2) after you sign in, exchange the device_code for a token
|
||||
curl -s -X POST "https://login.microsoftonline.com/common/oauth2/v2.0/token" \
|
||||
-d "grant_type=urn:ietf:params:oauth:grant-type:device_code" \
|
||||
-d "client_id=$CLIENT_ID" \
|
||||
-d "device_code=<device_code from step 1>" | jq -r .access_token
|
||||
```
|
||||
|
||||
<Note>
|
||||
**App-only tokens** (client-credentials flow) have no signed-in user, so
|
||||
`/me/drive` returns an error. With an app-only token you **must** pass a
|
||||
`drive_id` (or `site_id`). Delegated tokens (Options A and B) can use `/me/drive`
|
||||
with no drive ID.
|
||||
</Note>
|
||||
|
||||
### Get your drive ID
|
||||
|
||||
Only needed for app-only tokens or to target a specific SharePoint library. With
|
||||
your token exported as `TOKEN`:
|
||||
|
||||
```bash
|
||||
# your personal OneDrive drive id
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
https://graph.microsoft.com/v1.0/me/drive | jq -r .id
|
||||
|
||||
# list every drive you can see (name + id)
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
https://graph.microsoft.com/v1.0/me/drives | jq '.value[] | {name, id}'
|
||||
|
||||
# a SharePoint site's drives
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
"https://graph.microsoft.com/v1.0/sites/{hostname}:/sites/{site-path}" | jq -r .id
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
"https://graph.microsoft.com/v1.0/sites/{site-id}/drives" | jq '.value[] | {name, id}'
|
||||
```
|
||||
|
||||
The drive ID looks like `b!a1B2c3...`.
|
||||
|
||||
### Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
ONEDRIVE_ACCESS_TOKEN=<token>
|
||||
ONEDRIVE_DRIVE_ID=<drive id> # omit when using a delegated /me/drive token
|
||||
```
|
||||
|
||||
<Note>
|
||||
Snapshots and version pinning rely on OneDrive/SharePoint **version history**,
|
||||
which is on by default. Older versions are only readable while the library
|
||||
retains them (the version cap is configurable, and an admin can disable
|
||||
versioning).
|
||||
</Note>
|
||||
|
||||
For Python configuration, see the [Python OneDrive Setup](/python/setup/onedrive) guide.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: Postgres
|
||||
icon: database
|
||||
description: Set up a Postgres connection for the Postgres resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Get Your Connection DSN
|
||||
|
||||
#### Local Postgres
|
||||
|
||||
```bash
|
||||
# Default local instance
|
||||
DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres
|
||||
```
|
||||
|
||||
#### Managed Postgres (Supabase, Neon, RDS, ...)
|
||||
|
||||
Copy the connection string from your provider's dashboard. It usually looks like:
|
||||
|
||||
```
|
||||
postgres://<user>:<password>@<host>:5432/<database>?sslmode=require
|
||||
```
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
DATABASE_URL=postgres://user:pass@host:5432/db?sslmode=require
|
||||
```
|
||||
|
||||
## Permissions
|
||||
|
||||
The Postgres resource is read-only. Grant the role you connect with at minimum:
|
||||
|
||||
```sql
|
||||
GRANT USAGE ON SCHEMA public TO mirage_reader;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA public TO mirage_reader;
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO mirage_reader;
|
||||
```
|
||||
|
||||
If you only want to expose a subset of schemas, set `schemas` in the resource config:
|
||||
|
||||
```python
|
||||
PostgresConfig(dsn=os.environ["DATABASE_URL"], schemas=["public", "analytics"])
|
||||
```
|
||||
|
||||
## Limits
|
||||
|
||||
The resource has built-in safety limits to keep an agent from accidentally pulling a whole table:
|
||||
|
||||
- `default_row_limit` (1,000): default LIMIT applied to ad-hoc reads.
|
||||
- `max_read_rows` (10,000): hard ceiling per read.
|
||||
- `max_read_bytes` (10 MiB): hard ceiling per read.
|
||||
- `default_search_limit` (100): default LIMIT for search-style queries.
|
||||
|
||||
Override in the config if you need bigger reads.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
title: Qdrant
|
||||
icon: database
|
||||
description: Set up a Qdrant connection for the Qdrant resource.
|
||||
---
|
||||
|
||||
[Qdrant](https://qdrant.tech/) mounts a collection as a read-only filesystem: group-by payload fields
|
||||
become nested folders, each point is a `.json` payload file (plus a `.txt` text
|
||||
file and an optional blob), and semantic search is the `search` command.
|
||||
|
||||
## Connection
|
||||
|
||||
### Local / self-hosted
|
||||
|
||||
Point `host`/`port` at a running Qdrant (defaults `localhost:6333`):
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
QDRANT_HOST=localhost
|
||||
QDRANT_PORT=6333
|
||||
```
|
||||
|
||||
### Qdrant Cloud
|
||||
|
||||
Use `url` plus an `api_key`:
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
QDRANT_URL=https://xyz.us-east4-0.gcp.cloud.qdrant.io
|
||||
QDRANT_API_KEY=...
|
||||
```
|
||||
|
||||
## Search
|
||||
|
||||
Search is the `search "<query>" <path>` command. It returns ranked points as
|
||||
their canonical `<id>.txt` (or `<id>.json`) file paths plus a similarity score,
|
||||
so results compose with `cat`, `wc`, and pipes (`grep`/`rg` stay lexical).
|
||||
|
||||
The query text is turned into a vector two ways:
|
||||
|
||||
- **Local (default):** the `qdrant-client[fastembed]` extra embeds the query in
|
||||
process with `embedding_model` (default `sentence-transformers/all-MiniLM-L6-v2`).
|
||||
- **Server-side:** set `cloud_inference` to let a Qdrant Cloud (inference-enabled)
|
||||
cluster embed the query. The TypeScript backend always uses this path.
|
||||
|
||||
Either way the collection must already store vectors produced by the same model.
|
||||
|
||||
## Limits
|
||||
|
||||
The resource is read-only and bounds how much an agent can pull:
|
||||
|
||||
- `search_limit` (10): default top-k returned by `search`.
|
||||
- `max_rows` (1,000): hard ceiling on points listed per folder.
|
||||
|
||||
Folder listings filter on payload fields. A filtered listing scrolls first and
|
||||
only creates keyword payload indexes for the `group_by` fields if Qdrant reports
|
||||
one is required. `max_rows` caps how many points are scanned per folder.
|
||||
|
||||
See the [Python](/python/resource/qdrant) and [TypeScript](/typescript/setup/qdrant)
|
||||
resource pages for full config.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
title: QingStor
|
||||
icon: cloud
|
||||
description: Set up QingStor (QingCloud) Object Storage credentials for the QingStor resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an access key
|
||||
|
||||
1. Sign in to the [QingCloud console](https://console.qingcloud.com/)
|
||||
1. Go to **API Keys** (under your account) -> **Create**
|
||||
1. Copy the **Access Key ID** and **Secret Access Key**
|
||||
1. Note your bucket's **zone** (e.g. `pek3a`) and **bucket** name
|
||||
|
||||
The endpoint is derived from the zone (`region`) as `s3.<region>.qingstor.com`,
|
||||
QingStor's S3-compatible host (distinct from the native `<zone>.qingstor.com`).
|
||||
Set `QINGSTOR_ENDPOINT_URL` to override.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
QINGSTOR_BUCKET=my-bucket
|
||||
QINGSTOR_ZONE=pek3a
|
||||
QINGSTOR_ACCESS_KEY_ID=...
|
||||
QINGSTOR_SECRET_ACCESS_KEY=...
|
||||
# QINGSTOR_ENDPOINT_URL=https://s3.pek3a.qingstor.com # optional override
|
||||
```
|
||||
|
||||
For Python configuration, see the [QingStor resource](/python/resource/qingstor) docs.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
title: Cloudflare R2
|
||||
icon: cloud
|
||||
description: Set up Cloudflare R2 credentials for the R2 resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create R2 API Token
|
||||
|
||||
1. Go to https://dash.cloudflare.com/ -> **R2 Object Storage**
|
||||
1. **Manage R2 API Tokens** -> **Create API Token**
|
||||
1. Set permissions: **Object Read** (or **Object Read & Write**)
|
||||
1. Scope to your bucket
|
||||
1. Copy the **Access Key ID** and **Secret Access Key**
|
||||
1. Note your **Account ID** from the Cloudflare dashboard URL
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
R2_BUCKET=my-bucket
|
||||
R2_ACCOUNT_ID=abc123...
|
||||
R2_ACCESS_KEY_ID=...
|
||||
R2_SECRET_ACCESS_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [R2 resource](/python/resource/r2) docs.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
title: S3
|
||||
icon: aws
|
||||
description: Set up AWS S3 credentials for the S3 resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
The S3 resource needs an AWS access key pair with read (and optionally write) access to your bucket.
|
||||
|
||||
### 1. Create an IAM User
|
||||
|
||||
1. Go to https://console.aws.amazon.com/iam/
|
||||
1. **Users** -> **Create user**
|
||||
1. Attach the `AmazonS3ReadOnlyAccess` policy (or `AmazonS3FullAccess` for write)
|
||||
1. **Security credentials** -> **Create access key** -> **Application running outside AWS**
|
||||
1. Copy the **Access key ID** and **Secret access key**
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
AWS_S3_BUCKET=my-bucket
|
||||
AWS_DEFAULT_REGION=us-east-1
|
||||
AWS_ACCESS_KEY_ID=AKIA...
|
||||
AWS_SECRET_ACCESS_KEY=wJal...
|
||||
```
|
||||
|
||||
### Alternative: AWS Profile
|
||||
|
||||
If you have `~/.aws/credentials` configured, you can use a profile name instead of explicit access keys.
|
||||
|
||||
For the Python resource API, see the [S3 resource doc](/python/resource/s3).
|
||||
|
||||
## Scoping a resource to a key prefix
|
||||
|
||||
Both runtimes support a key prefix option that transparently scopes every operation to a subpath of the bucket, so agents see clean paths while the underlying S3 keys carry the full prefix.
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```ts TypeScript
|
||||
const s3 = new S3Resource({
|
||||
bucket: 'app-data',
|
||||
region: 'eu-west-1',
|
||||
keyPrefix: `users/${userId}/`,
|
||||
})
|
||||
```
|
||||
|
||||
```python Python
|
||||
S3Resource(S3Config(
|
||||
bucket="app-data",
|
||||
region="eu-west-1",
|
||||
key_prefix=f"users/{user_id}/",
|
||||
))
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
Leading slashes are stripped and a trailing slash is added automatically. `None` / `undefined` and empty strings both mean "no prefix." For browser security considerations, see the [TypeScript S3 setup doc](/typescript/setup/s3).
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
title: Scaleway
|
||||
icon: /images/scaleway-logo.svg
|
||||
description: Set up Scaleway Object Storage credentials for the Scaleway resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
### 1. Create an API key
|
||||
|
||||
1. Sign in to the [Scaleway console](https://console.scaleway.com/)
|
||||
1. Go to **IAM** -> **API Keys** -> **Generate API Key**
|
||||
1. Copy the **Access Key** and **Secret Key**
|
||||
1. Note your bucket's **region** (e.g. `fr-par`) and **bucket** name
|
||||
|
||||
The endpoint is derived from `region` as `s3.<region>.scw.cloud`.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
SCW_BUCKET=my-bucket
|
||||
SCW_REGION=fr-par
|
||||
SCW_ACCESS_KEY=...
|
||||
SCW_SECRET_KEY=...
|
||||
```
|
||||
|
||||
For Python configuration, see the [Scaleway resource](/python/resource/scaleway) docs.
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: SeaweedFS
|
||||
icon: /images/seaweedfs-logo.svg
|
||||
description: Set up SeaweedFS credentials for the SeaweedFS resource.
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
SeaweedFS serves an S3-compatible gateway (the `weed s3` command, default port
|
||||
`8333`). Access keys are defined in the gateway's S3 config (`-s3.config`), or
|
||||
the gateway can run without authentication for local development.
|
||||
|
||||
### 1. Get an access key
|
||||
|
||||
1. Start the S3 gateway, e.g. `weed server -s3` or `weed s3 -filer=localhost:8888`
|
||||
1. If you configured `-s3.config`, copy an identity's **accessKey** and **secretKey**
|
||||
1. Note your gateway **endpoint** (e.g. `http://localhost:8333`) and **bucket** name
|
||||
|
||||
SeaweedFS uses path-style addressing, so the `endpoint_url` is required.
|
||||
|
||||
### 2. Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
SEAWEEDFS_BUCKET=my-bucket
|
||||
SEAWEEDFS_ENDPOINT=http://localhost:8333
|
||||
SEAWEEDFS_ACCESS_KEY=...
|
||||
SEAWEEDFS_SECRET_KEY=...
|
||||
```
|
||||
|
||||
<Tip>
|
||||
Blaxel Agent Drive is backed by SeaweedFS. Use the drive's `s3Url` as the
|
||||
endpoint and a drive access token's credentials to mount it as a SeaweedFS
|
||||
resource.
|
||||
</Tip>
|
||||
|
||||
For Python configuration, see the [SeaweedFS resource](/python/resource/seaweedfs) docs.
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
title: SharePoint
|
||||
icon: microsoft
|
||||
description: Get a Microsoft Graph access token for the SharePoint resource.
|
||||
---
|
||||
|
||||
The SharePoint resource talks to Microsoft SharePoint Online through the
|
||||
[Microsoft Graph API](https://learn.microsoft.com/en-us/graph/). It
|
||||
authenticates with an OAuth2 **bearer access token**. Unlike OneDrive, you do
|
||||
**not** need a drive id: the resource discovers sites and document libraries for
|
||||
you, so a token with the right SharePoint permissions is all it needs.
|
||||
|
||||
## Credentials
|
||||
|
||||
You only need an **access token**. Pick whichever path fits.
|
||||
|
||||
### Option A: App-only (client credentials)
|
||||
|
||||
Best for scripted, automated, and CI runs. There is no signed-in user, and it
|
||||
works for SharePoint because the resource uses `/sites` and `/sites/{id}/drives`
|
||||
(no `/me` context required).
|
||||
|
||||
1. In the [Microsoft Entra admin center](https://entra.microsoft.com), go to
|
||||
**Identity** -> **Applications** -> **App registrations** -> **New registration**.
|
||||
1. Under **API permissions** -> **Microsoft Graph** -> **Application
|
||||
permissions**, add `Sites.Read.All` for read-only, or `Sites.ReadWrite.All`
|
||||
(plus `Files.ReadWrite.All` for writes) for full access. Click
|
||||
**Grant admin consent**.
|
||||
1. Under **Certificates & secrets**, create a **client secret**.
|
||||
1. Request a token with the client-credentials flow:
|
||||
|
||||
```bash
|
||||
TENANT_ID="<your tenant id>"
|
||||
CLIENT_ID="<your app client id>"
|
||||
CLIENT_SECRET="<your client secret>"
|
||||
|
||||
curl -s -X POST "https://login.microsoftonline.com/$TENANT_ID/oauth2/v2.0/token" \
|
||||
-d "client_id=$CLIENT_ID" \
|
||||
-d "client_secret=$CLIENT_SECRET" \
|
||||
-d "scope=https://graph.microsoft.com/.default" \
|
||||
-d "grant_type=client_credentials" | jq -r .access_token
|
||||
```
|
||||
|
||||
### Option B: Delegated (Graph Explorer)
|
||||
|
||||
Fastest way to try it as yourself. The token lasts about an hour, which is
|
||||
plenty for a test run.
|
||||
|
||||
1. Open [Graph Explorer](https://developer.microsoft.com/graph/graph-explorer) and **Sign in**.
|
||||
1. Run any SharePoint query (for example `GET /sites?search=*`) and consent when prompted.
|
||||
1. Open the **Access token** tab and copy the token.
|
||||
|
||||
This is a **delegated** token scoped to your own SharePoint access.
|
||||
|
||||
<Note>
|
||||
A static token is short-lived (about 60 minutes). For long-running mounts, pass
|
||||
a `Callable[[], str]` provider as `access_token` instead of a string. The
|
||||
resource refreshes on `401`, so the mount survives token expiry.
|
||||
</Note>
|
||||
|
||||
### Verify access
|
||||
|
||||
With your token exported as `TOKEN`:
|
||||
|
||||
```bash
|
||||
# list sites you can see (name + id)
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
"https://graph.microsoft.com/v1.0/sites?search=*" | jq '.value[] | {displayName, id}'
|
||||
|
||||
# list the document libraries (drives) in a site
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
"https://graph.microsoft.com/v1.0/sites/{site-id}/drives" | jq '.value[] | {name, id}'
|
||||
```
|
||||
|
||||
### Set environment variables
|
||||
|
||||
```bash
|
||||
# .env.development
|
||||
MS_GRAPH_DRIVE_TOKEN=<token>
|
||||
```
|
||||
|
||||
<Note>
|
||||
Snapshots and version pinning rely on SharePoint **version history**, which is
|
||||
on by default. Older versions are only readable while the library retains them
|
||||
(the version cap is configurable, and an admin can disable versioning).
|
||||
</Note>
|
||||
|
||||
For Python configuration, see the [Python SharePoint resource](/python/resource/sharepoint) guide.
|
||||