bcbd1bdb22
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
440 lines
16 KiB
Plaintext
440 lines
16 KiB
Plaintext
---
|
|
title: CLI
|
|
description: Drive Mirage from the shell. Spin up a workspace from YAML, run commands against your mounts, snapshot and restore.
|
|
icon: terminal
|
|
---
|
|
|
|
The `mirage` CLI is a thin httpx wrapper over the Mirage daemon. It
|
|
auto-spawns the daemon on first `workspace create`, and the daemon
|
|
auto-exits 30 seconds after the last workspace is deleted. Most users
|
|
never type a daemon command directly.
|
|
|
|
Output is structured JSON to stdout for every verb -- pipe to `jq`,
|
|
save to file, or read it directly.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
curl -fsSL https://strukto.ai/mirage/install.sh | sh
|
|
# or
|
|
npm install -g @struktoai/mirage-cli
|
|
# or
|
|
uvx mirage-ai
|
|
# or
|
|
npx @struktoai/mirage-cli
|
|
```
|
|
|
|
Verify:
|
|
|
|
```bash
|
|
mirage --help
|
|
```
|
|
|
|
## Define a workspace in YAML
|
|
|
|
A workspace is a set of prefixed mounts plus some workspace-level
|
|
settings. Save this as `workspace.yaml`:
|
|
|
|
```yaml
|
|
mode: WRITE
|
|
|
|
mounts:
|
|
/:
|
|
resource: ram
|
|
mode: WRITE
|
|
/s3:
|
|
resource: s3
|
|
mode: READ
|
|
config:
|
|
bucket: ${AWS_S3_BUCKET}
|
|
region: ${AWS_DEFAULT_REGION}
|
|
aws_access_key_id: ${AWS_ACCESS_KEY_ID}
|
|
aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
|
|
```
|
|
|
|
`${VAR}` placeholders are interpolated from your shell environment at
|
|
`mirage workspace create` time. Missing vars fail fast with the full
|
|
list, not lazily on first use.
|
|
|
|
## Walkthrough
|
|
|
|
A guided tour from creating a workspace to snapshotting it. Each
|
|
step builds on the previous one; you can copy them in order.
|
|
|
|
For a runnable end-to-end version against a real multi-mount workspace
|
|
(`/s3`, `/gdrive`, `/gmail`, `/slack`, `/discord`), see
|
|
<a href="https://github.com/strukto-ai/mirage/blob/main/examples/python/cross/README.md" target="_blank" rel="noopener noreferrer"><code>examples/python/cross/README.md</code></a>.
|
|
|
|
### 1. Source env and create a workspace
|
|
|
|
The YAML's `${...}` placeholders resolve from your shell at create
|
|
time, so source your env first. The daemon auto-spawns on the first
|
|
`create`.
|
|
|
|
```bash
|
|
set -a && source .env.development && set +a
|
|
|
|
mirage workspace create workspace.yaml --id demo
|
|
```
|
|
|
|
### 2. Inspect
|
|
|
|
`list` is one line per workspace; `get` returns the full mount and
|
|
session detail.
|
|
|
|
```bash
|
|
mirage workspace list
|
|
mirage workspace get demo
|
|
```
|
|
|
|
### 3. Run commands against your mounts
|
|
|
|
`execute` runs a shell command inside the workspace. Paths resolve
|
|
through the mount registry (`/s3/...` hits S3, `/` hits the RAM
|
|
backing, etc.).
|
|
|
|
```bash
|
|
mirage execute --workspace_id demo --command "ls /s3/"
|
|
mirage execute --workspace_id demo --command "head -n 1 /s3/data/example.jsonl"
|
|
```
|
|
|
|
### 4. Pipe stdin
|
|
|
|
When stdout isn't a TTY, the CLI forwards stdin to the command
|
|
automatically.
|
|
|
|
```bash
|
|
echo -e "a\nb\nc" | mirage execute --workspace_id demo --command "wc -l"
|
|
```
|
|
|
|
### 5. Dry-run with `provision`
|
|
|
|
`provision` returns a `ProvisionResult` (network bytes, cache hits,
|
|
estimated cost) without running the command -- handy for predicting
|
|
spend before kicking off an expensive read.
|
|
|
|
```bash
|
|
mirage provision --workspace_id demo \
|
|
--command "cat /s3/data/example.jsonl | wc -l"
|
|
```
|
|
|
|
Every factory-built backend estimates commands out of the box, by
|
|
family: whole-file readers (`cat`, `sort`, `md5`, ...) charge the byte
|
|
total from `stat`, `head`/`tail`/`file` charge a bounded range,
|
|
`grep`/`rg` charge a worst-case full read, and metadata commands (`ls`,
|
|
`find`, `stat`, `du`, ...) charge op counts only. Transforms (`gzip`,
|
|
`tar`, `split`, ...) keep the read total as a floor with
|
|
`precision=unknown` output; `cp` brackets both read and write between 0
|
|
(server-side copy) and the source total; metadata writes (`rm`,
|
|
`mkdir`, `touch`, ...) are zero-byte op counts, with recursive `rm`
|
|
degrading to a floor; pure commands (`seq`, `date`, `bc`, `expr`) and
|
|
shell builtins (`echo`, `cd`, ...) are zero-cost. Anything the planner
|
|
cannot estimate honestly -- `mv` (free rename or full cross-mount
|
|
copy), `tee` (stdin size), arbitrary programs -- reports
|
|
`precision=unknown` with all totals as floors, never an error. Virtual
|
|
files whose size cannot be resolved (for example a rendered
|
|
`chat.jsonl`) degrade the estimate to `precision=unknown` while keeping
|
|
the known byte total as a floor.
|
|
|
|
Provision is optional when you register your own commands: leave it out
|
|
and the planner reports `precision=unknown`. To opt in with one line,
|
|
reuse the estimator helpers (`make_file_read_provision`,
|
|
`make_search_provision`, `metadata_provision`, ... in Python;
|
|
`makeFileReadProvision` and friends in TypeScript), or pass
|
|
`provision_overrides={"grep": my_estimator}` to the command factory. An
|
|
explicit `None`/`null` override disables a default.
|
|
|
|
Pipelines combine field-wise: `|`, `;` and `&&` sum the estimates, `||`
|
|
brackets the branches (cheapest low, priciest high), and `for` loops
|
|
multiply by the iteration count. A stage downstream of an unknown stage
|
|
is also unknown, and totals under `precision=unknown` are floors.
|
|
|
|
### 6. Cache: network → hit after a real read
|
|
|
|
After a real `cat`, `provision` flips that path from a network read
|
|
to a cache hit (`cache_hits=1`).
|
|
|
|
```bash
|
|
mirage execute --workspace_id demo --command "cat /s3/data/example.jsonl > /dev/null"
|
|
mirage provision --workspace_id demo --command "cat /s3/data/example.jsonl"
|
|
```
|
|
|
|
### 7. Command history
|
|
|
|
Every executed command is recorded by a hidden recorder (the
|
|
[Observer](/home/observer)). The `history` builtin shows the calling
|
|
session's commands (GNU bash semantics, `history -c` clears only that
|
|
session's view), and `/.bash_history` renders the GNU histfile across all
|
|
sessions, readable with the ordinary file commands.
|
|
|
|
```bash
|
|
mirage execute --workspace_id demo --command "history 5"
|
|
mirage execute --workspace_id demo --command "tail -n 6 /.bash_history"
|
|
mirage execute --workspace_id demo --command "grep cat /.bash_history"
|
|
```
|
|
|
|
### 8. Background jobs
|
|
|
|
Long-running commands take `--background` and return a `job_id`
|
|
immediately. `mirage job wait` blocks until it's done.
|
|
|
|
```bash
|
|
JOB=$(mirage execute --workspace_id demo --background \
|
|
--command "wc -l /s3/data/example.jsonl" \
|
|
| jq -r .job_id)
|
|
mirage job wait $JOB
|
|
```
|
|
|
|
### 9. Snapshot to disk
|
|
|
|
```bash
|
|
mirage workspace snapshot demo /tmp/demo.tar
|
|
```
|
|
|
|
### 10. Restore from snapshot
|
|
|
|
Snapshots redact cloud creds at save time, so loading needs fresh
|
|
creds via a config file. The same workspace YAML used for create works.
|
|
|
|
```bash
|
|
mirage workspace load /tmp/demo.tar workspace.yaml \
|
|
--id demo_loaded
|
|
|
|
mirage workspace get demo_loaded --verbose
|
|
mirage execute --workspace_id demo_loaded \
|
|
--command "head -n 1 /s3/data/example.jsonl"
|
|
```
|
|
|
|
<Note>
|
|
A config file is required for any mount whose snapshot config contains
|
|
redacted secrets (S3 with inline keys, GDrive, Slack, Discord, Redis,
|
|
...). The snapshot stores `"<REDACTED>"` in place of those secrets at
|
|
save time. Loading without the config 400s with the list of prefixes
|
|
that need fresh creds. Local resources (RAM, Disk) restore as-is with
|
|
no extra config needed.
|
|
</Note>
|
|
|
|
### 11. Clean up
|
|
|
|
The daemon exits ~30s after the last workspace is deleted.
|
|
|
|
```bash
|
|
mirage workspace delete demo
|
|
mirage workspace delete demo_loaded
|
|
```
|
|
|
|
## Versioning
|
|
|
|
Every workspace has its own git-backed history kept by the daemon (under
|
|
`~/.mirage/repos/<workspace-id>` by default; set `MIRAGE_VERSION_ROOT` to
|
|
relocate). You commit the live state as a version, then log, diff, branch,
|
|
and restore in place. The verbs follow git.
|
|
|
|
### Commit and log
|
|
|
|
```bash
|
|
mirage execute --workspace_id demo --command "echo v1 > /notes.txt"
|
|
mirage workspace commit demo -m "first"
|
|
|
|
mirage execute --workspace_id demo --command "echo v2 > /notes.txt"
|
|
mirage workspace commit demo -m "second"
|
|
|
|
mirage workspace log demo # newest first: "second", then "first"
|
|
```
|
|
|
|
### Diff
|
|
|
|
`diff` reports changed paths (added / modified / deleted), git-style. No refs
|
|
compares live state to the branch HEAD; one ref compares live to that ref; two
|
|
refs compare two versions.
|
|
|
|
```bash
|
|
mirage workspace diff demo # live vs HEAD
|
|
mirage workspace diff demo <v1> # live vs <v1>
|
|
mirage workspace diff demo <v1> <v2> # <v1> vs <v2>
|
|
```
|
|
|
|
### Branch
|
|
|
|
`branch` forks at another branch's current version; commit onto it with `-b`.
|
|
|
|
```bash
|
|
mirage workspace branch demo exp # fork exp from main
|
|
mirage workspace branch demo exp2 --from exp # fork from a non-main branch
|
|
mirage workspace commit demo -b exp -m "on exp" # commit onto exp
|
|
mirage workspace log demo -b exp # log a specific branch
|
|
```
|
|
|
|
### Checkout
|
|
|
|
`checkout` restores the live state to a past version or branch, in place.
|
|
It overwrites uncommitted live state.
|
|
|
|
```bash
|
|
mirage workspace checkout demo <version> # by version id
|
|
mirage workspace checkout demo main # by branch name
|
|
```
|
|
|
|
### Clone from a version
|
|
|
|
`clone --at` makes a new workspace from one of the source's past versions
|
|
(omit `--at` to clone the live state).
|
|
|
|
```bash
|
|
mirage workspace clone demo --at <version> --id demo_at_v1
|
|
```
|
|
|
|
## Verbs at a glance
|
|
|
|
| Verb | What it does |
|
|
| --- | --- |
|
|
| `mirage workspace create FILE [--id NAME]` | Build resources from YAML, register a workspace, return its id. |
|
|
| `mirage workspace list` | Brief one-line summary per active workspace. |
|
|
| `mirage workspace get ID [--verbose]` | Full detail (mounts, sessions, optionally cache / dirty / history internals). |
|
|
| `mirage workspace delete ID` | Stop the workspace; daemon may exit on the idle timer. |
|
|
| `mirage workspace clone SRC_ID [--id NAME] [--at REF]` | New workspace from the source; `--at REF` clones a past version, otherwise the live state. |
|
|
| `mirage workspace commit ID [-m MSG] [-b BRANCH]` | Commit the live state as a version; returns the version id. |
|
|
| `mirage workspace log ID [-b BRANCH]` | List versions on a branch, newest first. |
|
|
| `mirage workspace diff ID [A] [B] [-b BRANCH]` | Changed paths (added/modified/deleted). No refs: live vs HEAD; one ref: live vs A; two: A vs B. |
|
|
| `mirage workspace branch ID NAME [--from BRANCH]` | Fork a branch at another branch's current version. |
|
|
| `mirage workspace checkout ID REF` | Restore the live state in place to a version id or branch. |
|
|
| `mirage workspace snapshot ID PATH.tar` | Snapshot to a tar file. |
|
|
| `mirage workspace load PATH.tar [CONFIG] [--id NAME]` | Restore from tar; optional config re-supplies redacted creds. |
|
|
| `mirage session create WS [--id NAME]` | Add a named session (own cwd + env). |
|
|
| `mirage session list WS` | List sessions for a workspace. |
|
|
| `mirage session delete WS SESSION` | Close a session. |
|
|
| `mirage execute --workspace_id WS [--session_id S] [--background] --command "..."` | Run a command. Pipes stdin automatically when stdout is not a TTY. |
|
|
| `mirage provision --workspace_id WS [--session_id S] --command "..."` | Dry-run / cost estimate -- returns a `ProvisionResult` shape (network bytes, cache hits, estimated cost) without running the command. |
|
|
| `mirage job list [--workspace_id WS]` | List jobs the daemon has run, plus their status. |
|
|
| `mirage job get JOB` | Detail for one job. |
|
|
| `mirage job wait JOB [--timeout SECS]` | Block until the job is done; returns the result. |
|
|
| `mirage job cancel JOB` | Cancel a running job. |
|
|
|
|
## Per-mount safeguards
|
|
|
|
<Note>
|
|
Per-mount `command_safeguards` are **Python CLI only** today. The TypeScript
|
|
CLI config schema does not carry them yet.
|
|
</Note>
|
|
|
|
Cap what a command may stream back per mount with `command_safeguards`, so a
|
|
runaway `cat`/`grep`/`rg` can't flood the agent or hang. Each entry sets
|
|
`max_lines` / `max_bytes` (output cap) and/or `timeout_seconds` (deadline),
|
|
with `on_exceed: truncate` (stop, exit 0, add a stderr notice) or
|
|
`on_exceed: error` (stop, exit 1):
|
|
|
|
```yaml
|
|
mounts:
|
|
/data:
|
|
resource: ram
|
|
mode: WRITE
|
|
command_safeguards:
|
|
head: # cap output, keep going
|
|
max_lines: 100
|
|
on_exceed: truncate
|
|
grep: # cap output, fail hard
|
|
max_lines: 50
|
|
on_exceed: error
|
|
rg: # deadline in seconds
|
|
timeout_seconds: 30
|
|
```
|
|
|
|
Caps fire on the **terminal** command of a pipeline only, so
|
|
`cat big.txt | head -n 30` still shows 30 lines. Truncation exits `0`, `error`
|
|
exits `1`, and a timeout exits `124` -- each with a stderr notice. Without a
|
|
`command_safeguards` block, `cat`/`grep`/`rg`/`head`/`tail` still cap at 2000
|
|
lines by default. See [Output Safeguards](/python/quickstart#output-safeguards)
|
|
for the SDK form and the same fields.
|
|
|
|
## Daemon control
|
|
|
|
Most users never run these directly -- the daemon auto-spawns on
|
|
first `workspace create` and auto-exits after the idle timer fires
|
|
(default 30s after the last workspace is deleted). When you need to
|
|
intervene -- typically during development, when you want code changes
|
|
to take effect:
|
|
|
|
```bash
|
|
mirage daemon status # health, PID, uptime, workspace count
|
|
mirage daemon stop # graceful: trip exit event, falls back to SIGTERM after --timeout (default 5s)
|
|
mirage daemon restart # stop + lazy respawn (add --eager to spawn now)
|
|
mirage daemon kill # SIGKILL via PID file -- last resort
|
|
```
|
|
|
|
| Verb | What it does |
|
|
| --- | --- |
|
|
| `mirage daemon status` | Daemon health, PID, uptime, workspace count. Exit 1 if daemon not reachable. |
|
|
| `mirage daemon stop` | `POST /v1/shutdown` to trip the daemon's exit event. Daemon closes active workspaces and exits. Falls back to SIGTERM on `--timeout`. |
|
|
| `mirage daemon restart` | Stop, then either wait for next `workspace create` to auto-spawn (default) or `--eager` to spawn immediately. Workspaces are LOST on restart -- save any you want to keep first with `mirage workspace snapshot <id> <path>` and reload with `mirage workspace load <path>`. |
|
|
| `mirage daemon kill` | SIGKILL via PID file at `~/.mirage/daemon.pid`. Skips graceful shutdown -- use only when `stop` hangs. |
|
|
|
|
<Note>
|
|
When you change Mirage's source code (commands, providers, etc.),
|
|
the running daemon won't see your changes -- it loaded the old code
|
|
at startup. Run `mirage daemon restart` to pick up new code.
|
|
Same situation as `dockerd` after recompiling Docker.
|
|
</Note>
|
|
|
|
## Where the daemon lives
|
|
|
|
Most users never need to think about the daemon. If you do:
|
|
|
|
- It listens on `http://127.0.0.1:8765` by default.
|
|
- Override via env vars or `~/.mirage/config.toml`, edited with
|
|
`mirage config` (the `git config` of Mirage):
|
|
```bash
|
|
mirage config set port 9100
|
|
mirage config set version_root /data/repos
|
|
mirage config get version_root # one key, exit 1 if unset
|
|
mirage config list # everything written in the file
|
|
mirage config unset port # remove a key
|
|
mirage config list --resolved # effective values + where each came from
|
|
```
|
|
```toml
|
|
[daemon]
|
|
url = "http://127.0.0.1:8765"
|
|
idle_grace_seconds = 30
|
|
port = 9100
|
|
version_root = "/data/repos"
|
|
```
|
|
- Per key the precedence is: env var > `config.toml` > default.
|
|
Because env vars silently beat the file, `mirage config list`
|
|
(which shows only the file) can look right while the daemon uses
|
|
something else. `--resolved` shows the value each key will
|
|
actually get and which source won:
|
|
|
|
```bash
|
|
$ mirage config set version_root /data/repos
|
|
$ export MIRAGE_VERSION_ROOT=/tmp/other # e.g. left over in your shell profile
|
|
|
|
$ mirage config list # the file looks correct...
|
|
version_root = /data/repos
|
|
|
|
$ mirage config list --resolved # ...but the env var wins
|
|
version_root = /tmp/other (env MIRAGE_VERSION_ROOT)
|
|
port = 8765 (default)
|
|
url = http://127.0.0.1:8765 (default)
|
|
auth_token = *** (env MIRAGE_TOKEN)
|
|
```
|
|
|
|
Use it whenever a setting seems ignored: the origin column names
|
|
the exact env var overriding you. Secrets are masked, and piped
|
|
output is JSON (`{"version_root": {"value": ..., "origin": ...}}`)
|
|
so it works with `jq`.
|
|
- Allowed keys: `url`, `socket`, `auth_token`, `auth_mode`,
|
|
`allowed_hosts`, `idle_grace_seconds`, `port`, `pid_file`,
|
|
`version_root`, `snapshot_root`, and the `jwt_*` family
|
|
(`jwt_alg`, `jwt_issuer`, `jwt_audience`, `jwt_pubkey_file`,
|
|
`jwt_clock_skew`, `jwt_authorized_parties`). `MIRAGE_HOME` and raw
|
|
secrets (`MIRAGE_AUTH_TOKEN`, `MIRAGE_JWT_PUBKEY`) stay env-only.
|
|
- Like `dockerd` with a bad `daemon.json`, the daemon refuses to
|
|
start on unknown keys or malformed TOML, naming the offender.
|
|
`mirage config unset <key>` accepts unknown keys so you can repair
|
|
the file; `mirage config list` warns about them.
|
|
- Path settings (`pid_file`, `version_root`, `snapshot_root`, `port`)
|
|
take effect on the next daemon start; there is no hot reload.
|
|
- `mirage config set` chmods the file to `0600` since it may hold
|
|
`auth_token`.
|
|
- Logs go to `~/.mirage/daemon.log` when the CLI auto-spawns it.
|
|
- It exits 30 seconds after the workspace count hits zero (configurable).
|