chore: import upstream snapshot with attribution

examples/python/cross/README.md

@@ -0,0 +1,274 @@
+# Cross-resource workspace (CLI)
+
+Drive a multi-mount workspace (`/s3`, `/gdrive`, `/gmail`, `/slack`,
+`/discord`) end-to-end from the shell using `workspace.yaml`.
+
+The two CLIs expose the same workspace HTTP API. Each
+example below shows the **Python** CLI (`mirage`, on `$PATH`) and
+the **TypeScript** CLI (`./mirage-ts`, a symlink to
+`typescript/packages/cli/dist/bin/mirage.js` from the repo root).
+Pick whichever CLI is convenient for the run; the command shapes match.
+
+## Prereqs
+
+- `.env.development` at the repo root with `AWS_`\*, `GOOGLE_*`,
+  `SLACK_BOT_TOKEN`, `DISCORD_BOT_TOKEN`.
+- Python: `mirage` CLI on `$PATH` (e.g. `./python/.venv/bin/mirage`).
+- TypeScript: `pnpm --filter @struktoai/mirage-cli build` then
+  `ln -sf typescript/packages/cli/dist/bin/mirage.js mirage-ts`
+  at the repo root (already gitignored).
+
+## 1. Source env and create the workspace
+
+The YAML's `${...}` placeholders resolve from your shell at create
+time, so source first.
+
+```bash
+set -a && source .env.development && set +a
+```
+
+```bash
+mirage       workspace create examples/python/cross/workspace.yaml --id cross
+./mirage-ts  workspace create examples/python/cross/workspace.yaml --id cross
+```
+
+## 2. Inspect
+
+```bash
+mirage       workspace list
+./mirage-ts  workspace list
+```
+
+```bash
+mirage       workspace get cross
+./mirage-ts  workspace get cross
+```
+
+## 3. Run commands across mounts
+
+`/gdrive/` is index-first — list it once before reading individual
+files, otherwise paths resolve to ENOENT.
+
+```bash
+mirage       execute --workspace_id cross --command "ls /s3/"
+./mirage-ts  execute --workspace_id cross --command "ls /s3/"
+```
+
+```bash
+mirage       execute --workspace_id cross --command "ls /gdrive/"
+./mirage-ts  execute --workspace_id cross --command "ls /gdrive/"
+```
+
+```bash
+mirage       execute --workspace_id cross --command "head -n 1 /s3/data/example.jsonl"
+./mirage-ts  execute --workspace_id cross --command "head -n 1 /s3/data/example.jsonl"
+```
+
+```bash
+mirage       execute --workspace_id cross \
+  --command 'cat /s3/data/example.jsonl "/gdrive/AWS CDK.gdoc.json" | wc -l'
+./mirage-ts  execute --workspace_id cross \
+  --command 'cat /s3/data/example.jsonl "/gdrive/AWS CDK.gdoc.json" | wc -l'
+```
+
+## 4. Dry-run with `provision`
+
+```bash
+mirage       provision --workspace_id cross --command "cat /s3/data/example.jsonl | wc -l"
+./mirage-ts  provision --workspace_id cross --command "cat /s3/data/example.jsonl | wc -l"
+```
+
+After a real read the same path flips from a network read to a cache
+hit (`cache_hits=1`):
+
+```bash
+mirage       execute   --workspace_id cross --command "cat /s3/data/example.jsonl > /dev/null"
+./mirage-ts  execute   --workspace_id cross --command "cat /s3/data/example.jsonl > /dev/null"
+```
+
+```bash
+mirage       provision --workspace_id cross --command "cat /s3/data/example.jsonl"
+./mirage-ts  provision --workspace_id cross --command "cat /s3/data/example.jsonl"
+```
+
+## 5. Snapshot and restore
+
+Snapshots redact cloud creds at snapshot time, so loading needs fresh
+creds via a config file. The same workspace YAML used for create works.
+
+```bash
+mirage       workspace snapshot cross /tmp/cross.tar
+./mirage-ts  workspace snapshot cross /tmp/cross.tar
+```
+
+```bash
+mirage       workspace load /tmp/cross.tar examples/python/cross/workspace.yaml \
+  --id cross_loaded
+./mirage-ts  workspace load /tmp/cross.tar examples/python/cross/workspace.yaml \
+  --id cross_loaded
+```
+
+```bash
+mirage       workspace get cross_loaded --verbose
+./mirage-ts  workspace get cross_loaded --verbose
+```
+
+## 6. Clean up
+
+The daemon exits ~30s after the last workspace is deleted.
+
+```bash
+mirage       workspace delete cross
+./mirage-ts  workspace delete cross
+```
+
+```bash
+mirage       workspace delete cross_loaded
+./mirage-ts  workspace delete cross_loaded
+```
+
+## 7. Per-mount safeguards (Python CLI)
+
+A mount can cap what a command streams back with `command_safeguards`,
+so a runaway `cat`/`grep`/`rg` can't flood the agent or hang forever.
+Each entry sets `max_lines` / `max_bytes` (output cap) and/or
+`timeout_seconds` (deadline), with `on_exceed: truncate` (stop, exit 0,
+add a notice) or `on_exceed: error` (stop, exit 1, add a notice).
+
+This section is **Python-only**: the TS config schema does not yet carry
+`command_safeguards`. It runs against its own workspace
+(`cross_sg`) from [workspace_safeguards.yaml](workspace_safeguards.yaml),
+so the steps above are untouched. That file guards `/s3` with:
+`head` → 10 lines / truncate, `grep` → 20 lines / error, `rg` → a 1 ms
+timeout.
+
+```bash
+set -a && source .env.development && set +a
+mirage workspace create examples/python/cross/workspace_safeguards.yaml --id cross_sg
+```
+
+Warm the object once (the first S3 read fetches the whole object and can
+take a few seconds; later reads are cache hits):
+
+```bash
+mirage execute --workspace_id cross_sg --command "head -n 1 /s3/data/example.jsonl"
+```
+
+`max_lines` + `on_exceed: truncate` — asking for 50 lines yields 10 plus a
+notice on stderr, exit `0`:
+
+```bash
+mirage execute --workspace_id cross_sg --command "head -n 50 /s3/data/example.jsonl"
+```
+
+`max_lines` + `on_exceed: error` — `grep` matches thousands of lines, trips
+the 20-line cap, and fails with exit `1`:
+
+```bash
+mirage execute --workspace_id cross_sg --command "grep mirage /s3/data/example.jsonl"
+```
+
+`timeout_seconds` — the 1 ms deadline trips on any real read, exit `124`
+with `rg: timed out after 0.001s`:
+
+```bash
+mirage execute --workspace_id cross_sg --command "rg mirage /s3/data/example.jsonl"
+```
+
+Commands below their cap are untouched, so the earlier shapes still work
+unchanged (1 line, no notice):
+
+```bash
+mirage execute --workspace_id cross_sg --command "head -n 1 /s3/data/example.jsonl"
+```
+
+Clean up:
+
+```bash
+mirage workspace delete cross_sg
+```
+
+## 8. Versioning (Python CLI)
+
+The daemon keeps a git-backed history per workspace under
+`~/.mirage/repos/<id>` (set `MIRAGE_VERSION_ROOT` to relocate). You commit
+the live state as a version, then `log` / `diff` / `branch` / `checkout`,
+following git. See the [CLI docs](https://docs.mirage.strukto.ai) for the
+full reference.
+
+This section is **Python-only** and runs against its own scratch workspace
+(`cross_ver`) from
+[workspace_versioning.yaml](workspace_versioning.yaml), so the steps above
+are untouched. It uses a writable RAM mount: versioning tracks the workspace
+tree regardless of backend, and this keeps the demo fast and writes nothing
+to your real cloud backends.
+
+> The history outlives the workspace: `workspace delete` does **not** remove
+> `~/.mirage/repos/<id>`. Re-creating the same id resumes the old history, so
+> this walkthrough starts from a clean id (or `rm -rf ~/.mirage/repos/cross_ver`
+> first) to keep output predictable.
+
+```bash
+mirage workspace create examples/python/cross/workspace_versioning.yaml --id cross_ver
+```
+
+Commit two versions, then `log` (newest first):
+
+```bash
+mirage execute --workspace_id cross_ver --command "echo v1 > /notes.txt"
+mirage workspace commit cross_ver -m "first"
+
+mirage execute --workspace_id cross_ver --command "echo v2 > /notes.txt"
+mirage workspace commit cross_ver -m "second"
+
+mirage workspace log cross_ver
+```
+
+`diff` reports changed paths (added / modified / deleted). With no refs it
+compares the live state to the branch HEAD; with two version ids it compares
+those versions (substitute the ids `log` printed):
+
+```bash
+mirage workspace diff cross_ver                 # live vs HEAD (clean: all empty)
+mirage workspace diff cross_ver <v1> <v2>       # => modified: ["notes.txt"]
+```
+
+`branch` forks at a branch's current version; commit onto it with `-b`. `main`
+is left untouched:
+
+```bash
+mirage workspace branch cross_ver exp
+mirage execute --workspace_id cross_ver --command "echo on-exp > /notes.txt"
+mirage workspace commit cross_ver -b exp -m "on exp"
+mirage workspace log cross_ver -b exp           # on exp, second, first
+mirage workspace log cross_ver                  # main unchanged: second, first
+```
+
+`checkout` restores the live state in place to a version or branch
+(overwriting uncommitted changes):
+
+```bash
+mirage workspace checkout cross_ver <v1>
+mirage execute --workspace_id cross_ver --command "cat /notes.txt"   # => v1
+```
+
+`clone --at` builds a new workspace from one of the source's past versions
+(omit `--at` to clone the live state):
+
+```bash
+mirage workspace clone cross_ver --at <v1> --id cross_ver_at
+mirage execute --workspace_id cross_ver_at --command "cat /notes.txt" # => v1
+```
+
+Clean up (and drop the histories so a rerun starts fresh):
+
+```bash
+mirage workspace delete cross_ver
+mirage workspace delete cross_ver_at
+rm -rf ~/.mirage/repos/cross_ver ~/.mirage/repos/cross_ver_at
+```
+
+## SDK alternative
+
+The same flow driven from Python (with snapshot fingerprinting) lives
+in [example.py](example.py) + [load_check.py](load_check.py).

examples/python/cross/example.py

@@ -0,0 +1,136 @@
+# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
+
+import asyncio
+import json
+import os
+
+from dotenv import load_dotenv
+
+from mirage import MountMode, Workspace
+from mirage.resource.discord import DiscordConfig, DiscordResource
+from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
+from mirage.resource.gmail import GmailConfig, GmailResource
+from mirage.resource.s3 import S3Config, S3Resource
+from mirage.resource.slack import SlackConfig, SlackResource
+
+load_dotenv(".env.development")
+
+google_kwargs = dict(
+    client_id=os.environ["GOOGLE_CLIENT_ID"],
+    client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
+    refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
+)
+
+s3 = S3Resource(config=S3Config(
+    bucket=os.environ["AWS_S3_BUCKET"],
+    region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
+    aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
+    aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
+))
+gdrive = GoogleDriveResource(config=GoogleDriveConfig(**google_kwargs))
+gmail = GmailResource(config=GmailConfig(**google_kwargs))
+slack = SlackResource(config=SlackConfig(
+    token=os.environ["SLACK_BOT_TOKEN"],
+    search_token=os.environ.get("SLACK_USER_TOKEN"),
+))
+discord = DiscordResource(config=DiscordConfig(
+    token=os.environ["DISCORD_BOT_TOKEN"]))
+
+# Stable path that both scripts agree on. Override with the env var
+# MIRAGE_CROSS_DIR if you want a different location.
+SNAPSHOT_DIR = os.environ.get("MIRAGE_CROSS_DIR", "/tmp/mirage-cross-clone")
+SNAPSHOT_TAR = os.path.join(SNAPSHOT_DIR, "snapshot.tar")
+EXPECTED_JSON = os.path.join(SNAPSHOT_DIR, "expected.json")
+
+# Read-only commands whose output is deterministic between runs.
+# The loader script reruns the same commands and asserts identical output.
+_FINGERPRINT_COMMANDS = [
+    'head -n 1 /s3/data/example.jsonl',
+    'cat /s3/data/example.jsonl | wc -l',
+    'grep -c "mirage" /s3/data/example.jsonl',
+    'head -n 1 "/gdrive/AWS CDK.gdoc.json"',
+    'wc -l "/gdrive/AWS CDK.gdoc.json"',
+    'cat /s3/data/example.jsonl "/gdrive/AWS CDK.gdoc.json" | wc -l',
+]
+
+
+async def _capture(ws, cmd):
+    r = await ws.execute(cmd)
+    return {
+        "command": cmd,
+        "exit_code": r.exit_code,
+        "stdout": (await r.stdout_str()).strip(),
+        "stderr": (await r.stderr_str()).strip(),
+    }
+
+
+async def main():
+    ws = Workspace(
+        {
+            "/s3": s3,
+            "/gdrive": gdrive,
+            "/gmail": gmail,
+            "/slack": slack,
+            "/discord": discord,
+        },
+        mode=MountMode.WRITE,
+    )
+
+    # gdrive needs `ls` of the parent folder first so the index has
+    # the file_id mapping the loader will need too.
+    await ws.execute("ls /gdrive/")
+
+    # Warm the cache by running each fingerprint command once and
+    # discarding the output. This way the snapshot's cache state
+    # matches what the loader will see — and downstream commands like
+    # `wc -l` produce identical formatting on both sides (some commands
+    # format slightly differently between cache-hit and source-read
+    # paths; warming makes the comparison apples-to-apples).
+    for cmd in _FINGERPRINT_COMMANDS:
+        await ws.execute(cmd)
+
+    # ── exercise the workspace (read-only for determinism) ──────────
+    print("=== capturing fingerprint commands ===\n")
+    fingerprints = []
+    for cmd in _FINGERPRINT_COMMANDS:
+        cap = await _capture(ws, cmd)
+        fingerprints.append(cap)
+        head = cap["stdout"][:80].replace("\n", " ")
+        print(f"  {cmd}")
+        print(f"    exit={cap['exit_code']}  stdout={head!r}")
+
+    # ── snapshot the workspace + expected outputs ──────────────────
+    os.makedirs(SNAPSHOT_DIR, exist_ok=True)
+    await ws.snapshot(SNAPSHOT_TAR)
+    with open(EXPECTED_JSON, "w") as f:
+        json.dump(
+            {
+                "tar_path": SNAPSHOT_TAR,
+                "fingerprints": fingerprints,
+            },
+            f,
+            indent=2,
+        )
+
+    print("\n=== saved ===")
+    print(f"  tar:      {SNAPSHOT_TAR} "
+          f"({os.path.getsize(SNAPSHOT_TAR)} bytes)")
+    print(f"  expected: {EXPECTED_JSON}")
+    print("\nNow run: ./python/.venv/bin/python "
+          "examples/python/cross/load_check.py")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

examples/python/cross/load_check.py

@@ -0,0 +1,147 @@
+# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
+
+import asyncio
+import json
+import os
+import sys
+
+from dotenv import load_dotenv
+
+from mirage.resource.discord import DiscordConfig, DiscordResource
+from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
+from mirage.resource.gmail import GmailConfig, GmailResource
+from mirage.resource.s3 import S3Config, S3Resource
+from mirage.resource.slack import SlackConfig, SlackResource
+from mirage.workspace import Workspace
+
+load_dotenv(".env.development")
+
+EXPECTED_JSON = os.environ.get(
+    "MIRAGE_CROSS_EXPECTED",
+    "/tmp/mirage-cross-clone/expected.json",
+)
+
+
+def _fresh_resources():
+    google_kwargs = dict(
+        client_id=os.environ["GOOGLE_CLIENT_ID"],
+        client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
+        refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
+    )
+    return {
+        "/s3":
+        S3Resource(config=S3Config(
+            bucket=os.environ["AWS_S3_BUCKET"],
+            region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
+            aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
+            aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
+        )),
+        "/gdrive":
+        GoogleDriveResource(config=GoogleDriveConfig(**google_kwargs)),
+        "/gmail":
+        GmailResource(config=GmailConfig(**google_kwargs)),
+        "/slack":
+        SlackResource(config=SlackConfig(
+            token=os.environ["SLACK_BOT_TOKEN"],
+            search_token=os.environ.get("SLACK_USER_TOKEN"),
+        )),
+        "/discord":
+        DiscordResource(config=DiscordConfig(
+            token=os.environ["DISCORD_BOT_TOKEN"])),
+    }
+
+
+async def _capture(ws, cmd):
+    r = await ws.execute(cmd)
+    return {
+        "command": cmd,
+        "exit_code": r.exit_code,
+        "stdout": (await r.stdout_str()).strip(),
+        "stderr": (await r.stderr_str()).strip(),
+    }
+
+
+def _summarize(field, expected, got):
+    if expected == got:
+        return "OK"
+    if isinstance(expected, str) and isinstance(got, str):
+        if len(expected) > 60 or len(got) > 60:
+            return (f"DIFF (lengths exp={len(expected)} got={len(got)}; "
+                    f"first diff at char {_first_diff(expected, got)})")
+    return f"DIFF (expected={expected!r}, got={got!r})"
+
+
+def _first_diff(a, b):
+    for i, (ca, cb) in enumerate(zip(a, b)):
+        if ca != cb:
+            return i
+    return min(len(a), len(b))
+
+
+async def main():
+    if not os.path.exists(EXPECTED_JSON):
+        print(f"ERROR: {EXPECTED_JSON} not found.")
+        print("Run examples/python/cross/example.py first to create "
+              "the snapshot.")
+        sys.exit(1)
+
+    with open(EXPECTED_JSON) as f:
+        expected_doc = json.load(f)
+
+    tar_path = expected_doc["tar_path"]
+    print(f"=== loading {tar_path} ===")
+    ws = await Workspace.load(tar_path, resources=_fresh_resources())
+    mounts = sorted(m.prefix for m in ws.mounts())
+    print(f"  mounts: {mounts}")
+    print(f"  loaded history entries: {len(await ws.history())}")
+
+    # gdrive index belongs to the freshly-supplied resource (override
+    # drops the saved index). Repopulate it the same way the original
+    # script did, so the loader's commands resolve the same paths.
+    await ws.execute("ls /gdrive/")
+
+    # ── re-execute fingerprint commands and compare ─────────────────
+    print(f"\n=== re-running {len(expected_doc['fingerprints'])} commands "
+          "and comparing ===\n")
+
+    n_match = 0
+    n_diff = 0
+    for expected in expected_doc["fingerprints"]:
+        got = await _capture(ws, expected["command"])
+        all_ok = True
+        for field in ("exit_code", "stdout", "stderr"):
+            verdict = _summarize(field, expected[field], got[field])
+            if verdict != "OK":
+                all_ok = False
+
+        marker = "✓" if all_ok else "✗"
+        print(f"  {marker} {expected['command']}")
+        if not all_ok:
+            for field in ("exit_code", "stdout", "stderr"):
+                v = _summarize(field, expected[field], got[field])
+                if v != "OK":
+                    print(f"      {field}: {v}")
+        if all_ok:
+            n_match += 1
+        else:
+            n_diff += 1
+
+    print(f"\n=== summary: {n_match} match, {n_diff} differ ===")
+    if n_diff > 0:
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

examples/python/cross/workspace.yaml

@@ -0,0 +1,35 @@
+mode: WRITE
+consistency: LAZY
+
+mounts:
+  /s3:
+    resource: s3
+    config:
+      bucket: ${AWS_S3_BUCKET}
+      region: ${AWS_DEFAULT_REGION}
+      aws_access_key_id: ${AWS_ACCESS_KEY_ID}
+      aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
+
+  /gdrive:
+    resource: gdrive
+    config:
+      client_id: ${GOOGLE_CLIENT_ID}
+      client_secret: ${GOOGLE_CLIENT_SECRET}
+      refresh_token: ${GOOGLE_REFRESH_TOKEN}
+
+  /gmail:
+    resource: gmail
+    config:
+      client_id: ${GOOGLE_CLIENT_ID}
+      client_secret: ${GOOGLE_CLIENT_SECRET}
+      refresh_token: ${GOOGLE_REFRESH_TOKEN}
+
+  /slack:
+    resource: slack
+    config:
+      token: ${SLACK_BOT_TOKEN}
+
+  /discord:
+    resource: discord
+    config:
+      token: ${DISCORD_BOT_TOKEN}

examples/python/cross/workspace_safeguards.yaml

@@ -0,0 +1,28 @@
+mode: WRITE
+consistency: LAZY
+
+# Same /s3 mount as workspace.yaml, plus per-mount command_safeguards so the
+# guards can be observed firing from the CLI. Python-only: the TS config
+# schema does not yet carry command_safeguards, so this lives in its own file
+# to keep the shared workspace.yaml usable from both CLIs.
+mounts:
+  /s3:
+    resource: s3
+    config:
+      bucket: ${AWS_S3_BUCKET}
+      region: ${AWS_DEFAULT_REGION}
+      aws_access_key_id: ${AWS_ACCESS_KEY_ID}
+      aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
+    command_safeguards:
+      # Cap head output at 10 lines and keep going (exit 0 + notice).
+      head:
+        max_lines: 10
+        on_exceed: truncate
+      # Cap grep output at 20 lines and fail hard (exit 1 + notice).
+      grep:
+        max_lines: 20
+        on_exceed: error
+      # Deliberately tiny deadline so any real read trips the timeout
+      # (exit 124). A production value would be seconds, not milliseconds.
+      rg:
+        timeout_seconds: 0.001

examples/python/cross/workspace_versioning.yaml

@@ -0,0 +1,12 @@
+mode: WRITE
+consistency: LAZY
+
+# A standalone scratch workspace for the versioning walkthrough. Versioning is
+# backend-agnostic (it tracks the workspace tree, not the mount type), so this
+# uses a writable RAM mount to keep the demo fast and to avoid writing test
+# files into your real cloud backends. The same verbs work on the cross
+# workspace above.
+mounts:
+  /:
+    resource: ram
+    mode: WRITE