wehub-resource-sync
bcbd1bdb22
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
Wasmer + Mirage FUSE
Run an untrusted Wasmer (WASM/WASIX) sandbox that reads S3
through a host-side Mirage FUSE mount. The guest sees a normal /s3 directory
and never touches S3 credentials or the network.
How it works
WASIX Python guest --mapdir /s3--> host FUSE mountpoint --> Mirage --> S3
(wasmer run) (macFUSE / fuse3)
wasmer_fuse.py(host) FUSE-mounts anS3Resourceat a temp mountpoint.- It launches
wasmer run python/pythonwith--mapdir /s3:<mountpoint>, so the guest's/s3is backed by the host FUSE mount. remote/guest.pyruns inside the WASIX guest: itos.listdir('/s3')and reads/s3/data/example.jsonlas if they were local files.
Mirage stays on the host because the Python wasmer binding is frozen at 3.10;
we drive the actively-maintained wasmer CLI as a subprocess instead.
Prerequisites
wasmerCLI onPATH: install from https://wasmer.io (curl https://get.wasmer.io -sSfL | sh).- Host FUSE: macOS macFUSE, or Linux
fuse3. - AWS credentials in
.env.developmentat the repo root:AWS_S3_BUCKET,AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY, and optionallyAWS_DEFAULT_REGION. - The bucket must contain
data/example.jsonl(any text/JSONL file works; the guest just counts lines containingmirage).
Run
From the repo root (so .env.development loads):
./python/.venv/bin/python examples/python/runtimes/wasmer/wasmer_fuse.py
Expected tail:
=== guest output ===
--- os.listdir('/s3') ---
data
...
--- read /s3/data/example.jsonl through Wasmer -> FUSE -> Mirage -> S3 ---
5766 lines, 5678 containing 'mirage'
Mirage served 104 ops, 10432116 bytes to the sandbox
First-run note
The very first wasmer run python/python downloads the WASIX Python package from
the Wasmer registry, which can take a few minutes. The example's subprocess
timeout (180s) may fire during that cold download and surface as a
TimeoutExpired traceback. Warm the cache once, then re-run:
wasmer run python/python -- -c "print('warm')"
Platform support
Works on both macOS (macFUSE) and Linux (fuse3). Not run in CI. It
needs the wasmer CLI, host FUSE, and live AWS credentials.