chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:30:44 +08:00
commit bcbd1bdb22
5748 changed files with 562488 additions and 0 deletions
@@ -0,0 +1,59 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from dotenv import load_dotenv
from mirage import MountMode, RAMResource, Workspace
from mirage.agents.agno import MirageToolkit
load_dotenv(".env.development")
ws = Workspace({"/data": RAMResource()}, mode=MountMode.WRITE)
agent = Agent(
model=OpenAIChat(id="gpt-4o"),
tools=[MirageToolkit(ws)],
instructions=("You have access to a virtual filesystem via shell "
"tools. Use them to explore and read files."),
markdown=True,
)
TASK = "List all files under /data and show the contents of each one."
def main() -> None:
asyncio.run(ws.execute('echo "hello from mirage" | tee /data/hello.txt'))
agent.print_response(TASK)
async def amain() -> None:
await ws.execute('echo "hello from mirage" | tee /data/hello.txt')
await agent.aprint_response(TASK)
records = ws.ops.records
if records:
total = sum(r.bytes for r in records)
print(f"\n--- {len(records)} ops, {total:,} bytes ---")
for r in records:
print(f" {r.op:<8} {r.source:<8} {r.bytes:>10,} B "
f"{r.duration_ms:>5} ms {r.path}")
if __name__ == "__main__":
main()
asyncio.run(amain())
@@ -0,0 +1,67 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from camel.agents import ChatAgent
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.agents.camel import MirageFileToolkit, MirageTerminalToolkit
from mirage.resource.ram import RAMResource
load_dotenv(".env.development")
ram = RAMResource()
ws = Workspace({"/": ram}, mode=MountMode.WRITE)
terminal = MirageTerminalToolkit(ws)
files = MirageFileToolkit(ws)
model = ModelFactory.create(
model_platform=ModelPlatformType.OPENAI,
model_type=ModelType.GPT_5_MINI,
)
agent = ChatAgent(
system_message=BaseMessage.make_assistant_message(
role_name="Mirage Camel Agent",
content=("You operate over a Mirage virtual filesystem mounted at /. "
"Use the file toolkit to write structured files and the "
"terminal toolkit to run shell commands. Paths start at /."),
),
model=model,
tools=[*terminal.get_tools(), *files.get_tools()],
)
task = ("Write a CSV at /data/numbers.csv with columns name,value and 3 rows. "
"Then list /data and read the file back.")
async def main():
response = await asyncio.to_thread(agent.step, task)
print(response.msgs[-1].content)
listing = await ws.execute("find / -type f")
print((listing.stdout or b"").decode())
if __name__ == "__main__":
try:
asyncio.run(main())
finally:
terminal.close()
files.close()
@@ -0,0 +1,85 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
"""Drive every Mirage tool through the Claude Agent SDK.
Gives a Sonnet agent a task that exercises all six tools the Mirage
MCP server exposes (execute_command, read, write, edit, ls, grep)
against a RAM-backed workspace, prints each tool call, and verifies
the final file contents.
Usage:
uv add 'mirage-ai[claude-agent-sdk]'
python examples/python/agents/claude_agent_sdk/all_tools.py
"""
import asyncio
from claude_agent_sdk import (AssistantMessage, ResultMessage, ToolUseBlock,
query)
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.agents.claude_agent_sdk import build_options
from mirage.resource.ram import RAMResource
load_dotenv(".env.development")
PROMPT = """\
You are operating on a Mirage virtual filesystem via the mirage tools.
Use exactly one mirage tool per step and do them in order:
1. Use the ls tool on '/'.
2. Use the write tool to create '/notes.txt' with lines: alpha, beta, gamma.
3. Use the read tool on '/notes.txt'.
4. Use the edit tool on '/notes.txt' to replace 'beta' with 'BETA'.
5. Use the grep tool to search for 'a' in '/notes.txt'.
6. Use the execute_command tool to run: cat /notes.txt | sort | wc -l
Briefly report what each step returned.
"""
EXPECTED = {
f"mcp__mirage__{name}"
for name in ("execute_command", "read", "write", "edit", "ls", "grep")
}
async def main() -> None:
ws = Workspace({"/": RAMResource()}, mode=MountMode.WRITE)
options = build_options(ws)
options.model = "claude-sonnet-4-6"
options.permission_mode = "bypassPermissions"
used: list[str] = []
async for msg in query(prompt=PROMPT, options=options):
if isinstance(msg, AssistantMessage):
for block in msg.content:
if isinstance(block, ToolUseBlock):
used.append(block.name)
print(f" -> {block.name} {block.input}")
elif isinstance(msg, ResultMessage):
print("\n=== final report ===")
print(msg.result)
print("\n=== tools used ===")
print(used)
missing = EXPECTED - set(used)
print("all six tools exercised:", not missing, "| missing:", missing
or "none")
final = await ws.ops.read("/notes.txt")
print("\n=== /notes.txt final content (from the Mirage workspace) ===")
print(final.decode("utf-8"))
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,61 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from databricks_langchain import ChatDatabricks
from deepagents import create_deep_agent
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.agents.langchain import (LangchainWorkspace, build_system_prompt,
extract_text)
from mirage.resource.databricks_volume import (DatabricksVolumeConfig,
DatabricksVolumeResource)
load_dotenv(".env.development")
resource = DatabricksVolumeResource(
DatabricksVolumeConfig(
catalog=os.environ["DATABRICKS_VOLUME_CATALOG"],
schema=os.environ["DATABRICKS_VOLUME_SCHEMA"],
volume=os.environ["DATABRICKS_VOLUME_NAME"],
root_path=os.environ.get("DATABRICKS_VOLUME_ROOT_PATH", "/"),
host=os.environ.get("DATABRICKS_HOST"),
token=os.environ.get("DATABRICKS_TOKEN"),
profile=os.environ.get("DATABRICKS_CONFIG_PROFILE"),
))
ws = Workspace({"/dbx/": resource}, mode=MountMode.READ)
agent = create_deep_agent(
model=ChatDatabricks(endpoint=os.environ["DATABRICKS_CHAT_ENDPOINT"], ),
system_prompt=build_system_prompt(workspace=ws),
backend=LangchainWorkspace(ws),
)
task = ("Inspect /dbx/, identify the most relevant text or markdown files, "
"and summarize their contents. Use head for large files.")
result = agent.invoke({"messages": [{"role": "user", "content": task}]})
for text in extract_text(result["messages"]):
print(text)
records = ws.ops.records
if records:
total = sum(record.bytes for record in records)
print(f"\n--- {len(records)} ops, {total:,} bytes ---")
for record in records:
print(f" {record.op:<8} {record.source:<18} {record.bytes:>10,} B "
f"{record.duration_ms:>5} ms {record.path}")
@@ -0,0 +1,65 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from deepagents import create_deep_agent
from dotenv import load_dotenv
from langchain_anthropic import ChatAnthropic
from mirage import MountMode, Workspace
from mirage.agents.langchain import (LangchainWorkspace, build_system_prompt,
extract_text)
from mirage.resource.s3 import S3Config, S3Resource
load_dotenv(".env.development")
config = S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)
s3 = S3Resource(config)
ws = Workspace({"/s3/": s3}, mode=MountMode.READ)
agent = create_deep_agent(
model=ChatAnthropic(model="claude-sonnet-4-20250514"),
system_prompt=build_system_prompt(
mount_info={"/s3/": "S3 bucket (CSV, Parquet, JSONL)"}, ),
backend=LangchainWorkspace(ws),
)
task = ("Explore and summarize the data in /s3/data/."
" Use head command for large files.")
result = agent.invoke({"messages": [{"role": "user", "content": task}]})
for text in extract_text(result["messages"]):
print(text)
task2 = ("How many rows are in the parquet, orc, and h5 files"
" under /s3/data/? ")
result2 = agent.invoke({"messages": [{"role": "user", "content": task2}]})
for text in extract_text(result2["messages"]):
print(text)
records = ws.ops.records
if records:
total = sum(r.bytes for r in records)
print(f"\n--- {len(records)} ops, {total:,} bytes ---")
for r in records:
print(f" {r.op:<8} {r.source:<8} {r.bytes:>10,} B "
f"{r.duration_ms:>5} ms {r.path}")
@@ -0,0 +1,104 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from pathlib import Path
from agents import Agent
from dotenv import load_dotenv
from openai import AsyncOpenAI
from mirage import MountMode, Workspace
from mirage.agents.openai_agents import MirageRunner, build_system_prompt
from mirage.resource.disk import DiskResource
from mirage.resource.ram import RAMResource
load_dotenv(".env.development")
REPO_ROOT = Path(__file__).resolve().parents[3]
LOGO_PATH = REPO_ROOT / "logo" / "mirage-text-logo-light.svg"
ram = RAMResource()
disk = DiskResource(root=str(REPO_ROOT))
ws = Workspace({"/ram": ram, "/disk": disk}, mode=MountMode.READ)
agent = Agent(
name="Multimodal Mirage Agent",
model="gpt-5.4-mini",
instructions=build_system_prompt(
mount_info={
"/ram": "In-memory filesystem",
"/disk": "Read-only repo files",
},
extra_instructions=("You will be shown attachments inline. "
"Describe what you see in 1-2 sentences."),
),
)
async def main():
if not os.environ.get("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set; skipping live agent run.")
return
png_path = "/ram/diagram.png"
png_bytes = LOGO_PATH.read_bytes() if LOGO_PATH.exists() else b""
if png_bytes:
await ws.ops.write(png_path, png_bytes)
txt_path = "/ram/notes.txt"
await ws.ops.write(txt_path,
b"Status: green. INP < 200ms across all routes.\n")
client = AsyncOpenAI()
runner = MirageRunner(ws, client=client)
paths: list[str] = [txt_path]
if png_bytes:
paths.append(png_path)
print("=== build_blocks ===")
blocks = await runner.build_blocks(
"Summarize the attachments. List each by type.", paths)
for b in blocks:
kind = b["type"]
head = (b.get("text") or b.get("image_url") or b.get("file_id")
or "")[:60]
print(f" {kind}: {head}...")
print()
print("=== Runner.run ===")
result = await runner.run_with_attachments(
agent,
"Summarize the attachments. List each by type.",
paths,
)
print(result.final_output)
# Same flow works against any mounted resource. Example variants:
#
# from mirage.resource.s3 import S3Resource, S3Config
# ws = Workspace({"/s3": S3Resource(S3Config(...))}, mode=MountMode.READ)
# await runner.run_with_attachments(agent, "...", ["/s3/bucket/img.png"])
#
# from mirage.resource.slack import SlackResource, SlackConfig
# ws = Workspace({"/slack": SlackResource(SlackConfig(...))})
# await runner.run_with_attachments(
# agent, "Summarize the PDF",
# ["/slack/channels/general__C1/2026-04-28/files/report__F1.pdf"])
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,79 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from agents import Agent, ApplyPatchTool, Runner, ShellTool
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.agents.openai_agents import (MirageEditor, MirageShellExecutor,
build_system_prompt)
from mirage.resource.ram import RAMResource
load_dotenv(".env.development")
ram = RAMResource()
ws = Workspace({"/": ram}, mode=MountMode.WRITE)
system_prompt = build_system_prompt(
mount_info={"/": "In-memory filesystem (read/write)"},
extra_instructions=("All file paths start from /. "
"For example: /hello.txt, /data/numbers.csv. "
"Use the shell tool to run commands like: "
"echo 'content' > /hello.txt, mkdir /data, "
"cat /hello.txt, ls /."),
)
agent = Agent(
name="Mirage RAM Agent",
model="gpt-5.5-mini",
instructions=system_prompt,
tools=[
ShellTool(executor=MirageShellExecutor(ws)),
ApplyPatchTool(editor=MirageEditor(ws)),
],
)
task = ("Create a file /hello.txt with the content 'Hello from Mirage!'. "
"Then create a directory /data and write a CSV file /data/numbers.csv "
"with columns: name, value. Add 3 rows of sample data. "
"Finally, list all files and cat the CSV.")
async def main():
result = await Runner.run(agent, task)
print(result.final_output)
print("\n--- Verifying files in workspace ---")
find_all = await ws.execute("find / -type f")
print(f"find / -type f:\n{(find_all.stdout or b'').decode()}")
for path in (find_all.stdout or b"").decode().strip().split("\n"):
path = path.strip()
if not path:
continue
cat_result = await ws.execute(f"cat {path}")
print(f"cat {path}:\n{(cat_result.stdout or b'').decode()}")
records = ws.ops.records
if records:
total = sum(r.bytes for r in records)
print(f"--- {len(records)} ops, {total:,} bytes ---")
for r in records:
print(f" {r.op:<8} {r.source:<8} {r.bytes:>10,} B "
f"{r.duration_ms:>5} ms {r.path}")
asyncio.run(main())
@@ -0,0 +1,155 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from agents import Runner
from agents.run import RunConfig
from agents.sandbox import SandboxAgent, SandboxRunConfig
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.agents.openai_agents import MirageSandboxClient
from mirage.resource.ram import RAMResource
from mirage.resource.s3 import S3Config, S3Resource
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
ram = RAMResource()
s3 = S3Resource(
S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
))
slack = SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
))
ws = Workspace(
{
"/": (ram, MountMode.WRITE),
"/s3": (s3, MountMode.READ),
"/slack": (slack, MountMode.READ),
},
mode=MountMode.WRITE,
)
client = MirageSandboxClient(ws)
agent = SandboxAgent(
name="Mirage Sandbox Agent",
model="gpt-5.5",
instructions=ws.file_prompt,
)
task = ("1. Find the date of the latest Slack message in the general channel. "
"2. Summarize the parquet file in /s3/data/. "
"Write your findings to /report.txt.")
async def main():
result = await Runner.run(
agent,
task,
run_config=RunConfig(sandbox=SandboxRunConfig(client=client)),
)
print(result.final_output)
ws = client._ws
find_all = await ws.execute("find / -type f")
print("\n--- Files in workspace ---")
print((find_all.stdout or b"").decode())
# ── persist/hydrate via the OpenAI Agents sandbox API ──────────
# MirageSandboxSession.persist_workspace returns a BytesIO with a
# tar; hydrate_workspace mutates an existing session's workspace
# in place. Build a fresh session (with the same mount shape) and
# restore the snapshot into it.
print("\n--- persist / hydrate via sandbox API ---")
session = await client.create()
snapshot = await session.persist_workspace()
snapshot_size = snapshot.getbuffer().nbytes
print(f" persisted snapshot: {snapshot_size:,} bytes")
# Fresh client with the same mount shape — required so hydrate
# finds the same prefixes to restore content into.
fresh_ws = Workspace(
{
"/": (RAMResource(), MountMode.WRITE),
"/s3": (S3Resource(
S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)), MountMode.READ),
"/slack": (SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
)), MountMode.READ),
},
mode=MountMode.WRITE,
)
fresh_client = MirageSandboxClient(fresh_ws)
fresh_session = await fresh_client.create()
await fresh_session.hydrate_workspace(snapshot)
fresh_find = await fresh_ws.execute("find / -type f")
print("--- Files in hydrated workspace ---")
print((fresh_find.stdout or b"").decode())
orig_files = set((find_all.stdout or b"").decode().strip().splitlines())
fresh_files = set((fresh_find.stdout or b"").decode().strip().splitlines())
diff = orig_files.symmetric_difference(fresh_files)
print(f"--- file list diff: {len(diff)} files differ "
f"{'(OK)' if not diff else '(' + str(diff) + ')'} ---")
# Verify content (not just names) for every file the agent created.
print("\n--- content match per file ---")
n_match = 0
n_diff = 0
for path in sorted(orig_files):
if not path:
continue
orig = await ws.execute(f"cat {path}")
fresh = await fresh_ws.execute(f"cat {path}")
orig_bytes = orig.stdout or b""
fresh_bytes = fresh.stdout or b""
if orig_bytes == fresh_bytes:
print(f"{path} ({len(orig_bytes)} bytes match)")
n_match += 1
else:
print(f"{path}")
print(f" orig ({len(orig_bytes)} bytes): "
f"{orig_bytes[:120]!r}")
print(f" fresh ({len(fresh_bytes)} bytes): "
f"{fresh_bytes[:120]!r}")
n_diff += 1
print(f"\n--- content summary: {n_match} match, {n_diff} differ ---")
# Show /report.txt explicitly so the user can read what the agent wrote
if "/report.txt" in orig_files:
report = await fresh_ws.execute("cat /report.txt")
body = (report.stdout or b"").decode()
print(f"\n--- /report.txt from hydrated workspace "
f"({len(body)} chars) ---")
print(body)
asyncio.run(main())
@@ -0,0 +1,134 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from agents import Runner
from agents.run import RunConfig
from agents.sandbox import SandboxAgent, SandboxRunConfig
from dotenv import load_dotenv
from openai import AsyncOpenAI
from mirage import MountMode, Workspace
from mirage.agents.openai_agents import MirageRunner, MirageSandboxClient
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
slack = SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
))
ws = Workspace({"/slack": (slack, MountMode.READ)}, mode=MountMode.READ)
client = MirageSandboxClient(ws)
navigator = SandboxAgent(
name="path-resolver",
model="gpt-5.4-mini",
instructions=(f"{ws.file_prompt}\n\n"
"Use shell tools (ls, find) to locate files. "
"Reply with absolute paths only, one per line."),
)
analyst = SandboxAgent(
name="analyst",
model="gpt-5.4-mini",
instructions=(
f"{ws.file_prompt}\n\n"
"You have shell tools (ls, find, cat, grep, ...) and view_image. "
"Some files may already be attached to this message — read them "
"directly. For images you discover later, call view_image. "
"Answer using only attachments and confirmed file contents."),
)
async def mirage_run(task: str) -> str:
"""Run a task with both pre-attached multimodal context and live tools.
Pipeline:
1. Navigator agent uses shell tools to resolve which paths the task
needs.
2. MirageRunner pre-attaches every resolved path as a multimodal
block — input_image (PNG/JPEG/GIF, base64 data URI), input_file
(PDF, uploaded via OpenAI Files API), or input_text otherwise.
3. The analyst SandboxAgent receives those blocks AND retains all
shell tools + native view_image. It can read the pre-attached
content directly OR call view_image / cat for anything else
it discovers mid-run.
Args:
task (str): Natural-language task referring to files in the VFS.
Returns:
str: The analyst's final output.
"""
nav = await Runner.run(
navigator,
f"Find every file this request refers to: {task}",
run_config=RunConfig(sandbox=SandboxRunConfig(client=client)),
max_turns=20,
)
print(" navigator raw output:")
for line in nav.final_output.strip().splitlines():
print(f" {line!r}")
paths = [
line.strip().strip("`").strip()
for line in nav.final_output.strip().splitlines()
if line.strip().startswith("/")
]
print(f" resolved paths: {paths}")
runner = MirageRunner(ws, client=AsyncOpenAI())
blocks = await runner.build_blocks(task, paths)
out = await Runner.run(
analyst,
[{
"role": "user",
"content": blocks
}],
run_config=RunConfig(sandbox=SandboxRunConfig(client=client)),
max_turns=20,
)
return out.final_output
async def main():
task = "Summarize the latest PNG and PDF in the slack general channel."
print(f"=== Task: {task} ===")
print()
result = await mirage_run(task)
print()
print("=== Analyst output ===")
print(result)
# Why both pre-attach AND view_image?
#
# - PNG/JPEG/GIF: either path works. view_image is convenient for images
# the agent discovers mid-run; pre-attach is convenient for images
# already known up front.
# - PDF: ONLY pre-attach works. The OpenAI Agents SDK has no view_file
# builtin for tool outputs (issue #341). PDFs must be uploaded to the
# Files API and added as input_file blocks in a user message. We do
# that before the agent run so the model receives full PDF text and
# rendered pages.
# - input_text: any non-binary content the agent might want pre-loaded.
#
# Resource-agnostic: ws.ops.read(path) routes via the workspace mount
# registry, so the same flow works for /s3/...png, /disk/...pdf,
# /slack/.../files/..., etc.
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,75 @@
# OpenHands + Mirage — agents that just use a shell
This example wires the OpenHands SDK to a Mirage `Workspace` and gives the agent **one** tool: `terminal`. No SaaS-specific tools, no MCP servers, no per-vendor schemas. Slack, S3, Gmail, GitHub, Linear — Mirage mounts each as a directory tree, and the agent treats them like a filesystem.
## Run
```bash
./python/.venv/bin/python examples/python/agents/openhands/sandbox_agent.py
```
The task: *"Find Slack messages containing 'hello' in #general."* The agent finishes in **2 commands**:
```
$ ls /slack/channels/
general__C04KEPWF6V7 random__C04JVGZM7UN test__C0AS76ABXMK
$ grep -i hello /slack/channels/general__C04KEPWF6V7/*.jsonl
.../2026-04-16.jsonl:[zechengzhang97] hello
.../2026-04-04.jsonl:[demo app] Hello from MIRAGE Slack provider!
```
That's it. The agent never learned about Slack's API. It used `ls` and `grep`.
## Why this matters: Mirage vs. the alternatives
The same task, three ways. Same answer; very different agent surface.
### With Mirage (this example)
- **Agent's tool list:** `terminal`. One tool, one schema.
- **Agent's vocabulary:** every shell command it already knows — `ls`, `cat`, `grep`, `head`, `wc`, `jq`, `find`, pipes, redirection.
- **What changed when we added Slack:** mount it at `/slack`. No new tools, no new prompts, no new agent code.
### With a Slack MCP server
- **Agent's tool list:** typically 612 Slack-specific tools — `slack_search_messages`, `slack_list_channels`, `slack_get_channel_history`, `slack_get_user_info`, `slack_post_message`, `slack_add_reaction`, …
- **Agent's vocabulary per tool:** every tool has its own JSON schema, parameter names, return shape. The model has to *learn the API*, then translate user intent into the right tool + the right params.
- **Composition:** want to filter messages with `jq` then count with `wc`? You can't — MCP tools are atomic; you get back what they return.
- **Adding Discord:** another MCP server with its own 612 tools. The agent's prompt now juggles two parallel APIs.
### With the Slack CLI
- **Agent's tool list:** `terminal` (good — same as Mirage), but...
- **Agent's vocabulary:** `slack search ...`, `slack chat send ...`, `slack auth login ...`. Vendor-specific subcommands, vendor-specific output formats, vendor-specific auth handling. The agent has to know the Slack CLI exists *and* how to invoke it.
- **Composition:** the CLI's stdout is its own format. Pipe it through `jq` if it happens to emit JSON, otherwise parse text.
- **Adding Discord:** install the Discord CLI. Now the agent needs to know two CLIs and pick correctly.
### Side-by-side
| | Mirage | Slack MCP | Slack CLI |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ---------------------------------------- |
| Tools the agent sees | 1 (`terminal`) | 612 per backend | 1 (`terminal`) |
| Vocabulary the agent must learn | shell + Mirage's filesystem layout | each tool's schema | each CLI's subcommand grammar |
| Composability (pipe / redirect / loop) | yes — real shell | no — atomic calls | partial — depends on CLI's stdout format |
| Adding a new backend | mount it; nothing else changes | new MCP server, new tool list, prompt churn | install new CLI; agent must learn it |
| Pushdown to native APIs (search, etc.) | automatic, in the builtin (Mirage rewrites `grep` over a Slack channel into one `search.messages` call) | only what the MCP exposes | none — text in, text out |
## What Mirage gives the agent
- **One stable tool surface** (`terminal`) regardless of how many backends are mounted.
- **Pipes and composability** because everything is a stream of bytes — `cat /s3/data/2026-04.parquet | grep error | jq '.user' | sort | uniq -c`.
- **Format-aware reads** — `cat` on `.parquet` / `.feather` / `.orc` returns a formatted table; `head -n 5` on `.jsonl` returns the first 5 messages; `grep` on a Slack channel directory pushes down to `search.messages` automatically.
- **One mental model** for the agent: *"the workspace is a filesystem; use shell."*
- **One mental model** for you: *"if I can mount it, the agent can use it."*
## Configure
The script loads `.env.development` from the repo root. Required:
| Var | What it's for |
| ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `LLM_API_KEY` | OpenHands `LLM` (defaults to Anthropic — set to your `ANTHROPIC_API_KEY`) |
| `AWS_S3_BUCKET`, `AWS_DEFAULT_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | `/s3` mount |
| `SLACK_BOT_TOKEN` | `/slack` mount |
| `SLACK_USER_TOKEN` *(recommended)* | enables Slack's `search.messages` push-down so `grep` over `/slack/channels/<channel>/*.jsonl` runs in one API call instead of fanning out per day |
@@ -0,0 +1,80 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from openhands.sdk import LLM, Agent, Conversation, Tool
from mirage import MountMode, Workspace
from mirage.agents.openhands import MirageWorkspace, register_mirage_terminal
from mirage.resource.ram import RAMResource
from mirage.resource.s3 import S3Config, S3Resource
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
TASK = (
"Find any Slack messages containing the word 'hello' (case-insensitive) "
"in the general channel. The channel directory is at "
"/slack/channels/ and starts with 'general'. Each day's messages live "
"in a <yyyy-mm-dd>.jsonl file. Use `ls` to discover the exact channel "
"directory, then `grep -i hello` across its jsonl files. Report the "
"matching message texts and stop.")
def build_workspace() -> Workspace:
s3 = S3Resource(
S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
))
slack = SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
))
return Workspace(
{
"/": (RAMResource(), MountMode.WRITE),
"/s3": (s3, MountMode.READ),
"/slack": (slack, MountMode.READ),
},
mode=MountMode.WRITE,
)
def main() -> None:
ws = build_workspace()
llm = LLM(
model=os.getenv("LLM_MODEL", "anthropic/claude-sonnet-4-6"),
api_key=os.getenv("LLM_API_KEY"),
base_url=os.getenv("LLM_BASE_URL", None),
)
with MirageWorkspace(workspace=ws, working_dir="/") as mirage_ws:
tool_name = register_mirage_terminal(mirage_ws)
agent = Agent(
llm=llm,
tools=[Tool(name=tool_name)],
system_message=ws.file_prompt,
)
conversation = Conversation(agent=agent, workspace=mirage_ws)
conversation.send_message(TASK)
conversation.run()
if __name__ == "__main__":
main()
@@ -0,0 +1,65 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dataclasses import dataclass
from dotenv import load_dotenv
from pydantic_ai import Agent
from pydantic_ai_backends import create_console_toolset
from mirage import MountMode, Workspace
from mirage.agents.pydantic_ai import PydanticAIWorkspace, build_system_prompt
from mirage.resource.s3 import S3Config, S3Resource
load_dotenv(".env.development")
config = S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)
s3 = S3Resource(config)
ws = Workspace({"/s3/": s3}, mode=MountMode.READ)
@dataclass
class Deps:
backend: PydanticAIWorkspace
backend = PydanticAIWorkspace(ws)
agent = Agent(
"openai:gpt-4.1",
system_prompt=build_system_prompt(
mount_info={"/s3/": "S3 bucket (CSV, Parquet, JSONL)"}),
deps_type=Deps,
toolsets=[create_console_toolset()],
)
task = ("Explore and summarize the data in /s3/data/."
" Use head command for large files and do not write anything.")
result = agent.run_sync(task, deps=Deps(backend=backend))
print(result.output)
records = ws.ops.records
if records:
total = sum(r.bytes for r in records)
print(f"\n--- {len(records)} ops, {total:,} bytes ---")
for r in records:
print(f" {r.op:<8} {r.source:<8} {r.bytes:>10,} B "
f"{r.duration_ms:>5} ms {r.path}")
@@ -0,0 +1,76 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dataclasses import dataclass
import anthropic.types.beta.beta_web_search_tool_20250305_param as _ws_mod
from dotenv import load_dotenv
if not hasattr(_ws_mod, "UserLocation"):
class _UserLocation:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
_ws_mod.UserLocation = _UserLocation
from pydantic_ai import Agent
from pydantic_ai_backends import create_console_toolset
from mirage import MountMode, Workspace
from mirage.agents.pydantic_ai import PydanticAIWorkspace, build_system_prompt
from mirage.resource.s3 import S3Config, S3Resource
load_dotenv(".env.development")
config = S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)
s3 = S3Resource(config)
ws = Workspace({"/s3/": s3}, mode=MountMode.READ)
@dataclass
class Deps:
backend: PydanticAIWorkspace
backend = PydanticAIWorkspace(ws)
agent = Agent(
"anthropic:claude-sonnet-4-20250514",
system_prompt=build_system_prompt(
mount_info={"/s3/": "S3 bucket with PDF documents"}),
deps_type=Deps,
toolsets=[create_console_toolset()],
)
task = ("Read the PDF at /s3/data/example.pdf."
" Summarize the first 5 pages of the paper.")
result = agent.run_sync(task, deps=Deps(backend=backend))
print(result.output)
records = ws.ops.records
if records:
total = sum(r.bytes for r in records)
print(f"\n--- {len(records)} ops, {total:,} bytes ---")
for r in records:
print(f" {r.op:<8} {r.source:<8} {r.bytes:>10,} B "
f"{r.duration_ms:>5} ms {r.path}")
@@ -0,0 +1,91 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
import time
from dataclasses import dataclass
from dotenv import load_dotenv
from pydantic_ai import Agent
from pydantic_ai_backends import create_console_toolset
from mirage import MountMode, Workspace
from mirage.agents.pydantic_ai import PydanticAIWorkspace
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
slack = SlackResource(
config=SlackConfig(token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN")))
ws = Workspace({"/slack": slack}, mode=MountMode.READ)
@dataclass
class Deps:
backend: PydanticAIWorkspace
backend = PydanticAIWorkspace(ws)
agent = Agent(
"openai:gpt-5.4-mini",
system_prompt=ws.file_prompt,
deps_type=Deps,
toolsets=[
create_console_toolset(require_execute_approval=False,
image_support=True)
],
)
def main():
task = (
"Read and summarize the latest PNG and PDF in the slack "
"general channel. Open each file with read_file before responding.")
print(f"=== Task: {task} ===")
print()
t0 = time.perf_counter()
result = agent.run_sync(task, deps=Deps(backend=backend))
elapsed = time.perf_counter() - t0
print(result.output)
print()
print(f"--- {elapsed:.1f}s ---")
records = ws.ops.records
if records:
total = sum(r.bytes for r in records)
print(f"--- {len(records)} ops, {total:,} bytes ---")
for r in records:
print(f" {r.op:<8} {r.source:<8} {r.bytes:>10,} B "
f"{r.duration_ms:>5} ms {r.path}")
# Single-agent flow.
#
# Pydantic AI's tool channel accepts multimodal `BinaryContent` blocks in
# `ToolReturn.content`, so the agent's `read()` tool can return rendered
# PDF pages and image bytes inline in its context. No two-phase
# orchestration needed — unlike the OpenAI Agents SDK, where tool
# returns are text-only (issue #341) and PDFs require pre-attach via
# the Files API in a separate user-message turn.
#
# Mirage wiring is done by mirage.agents.pydantic_ai.PydanticAIWorkspace
# in backend.py: when `read(path)` ends in .pdf, it routes through
# `pages_to_images` and packs each page as
# `BinaryContent(media_type="image/png")`. Resource-agnostic: the same
# flow works for /s3, /disk, /slack/.../files/, etc.
if __name__ == "__main__":
main()
+58
View File
@@ -0,0 +1,58 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.aliyun import AliyunConfig, AliyunResource
load_dotenv(".env.development")
config = AliyunConfig(
bucket=os.environ["OSS_BUCKET"],
region=os.environ["OSS_REGION"], # e.g. cn-hangzhou
access_key_id=os.environ["OSS_ACCESS_KEY_ID"],
secret_access_key=os.environ["OSS_ACCESS_KEY_SECRET"],
)
resource = AliyunResource(config)
ws = Workspace({"/oss/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
return f"{len(records)} ops, {sum(r.bytes for r in records)} bytes"
async def main():
print(f"=== Alibaba OSS at {config.resolved_endpoint_url()} ===")
r = await ws.execute("ls /oss/")
print("ls /oss/:\n" + await r.stdout_str())
r = await ws.execute("find /oss/ -name '*.json' | head -n 5")
print("find *.json:\n" + await r.stdout_str())
r = await ws.execute("grep -m 1 mirage /oss/data/example.jsonl",
provision=True)
print(f"plan grep -m 1: network_read={r.network_read} "
f"precision={r.precision}")
print(f"\nStats: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
+58
View File
@@ -0,0 +1,58 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.backblaze import BackblazeConfig, BackblazeResource
load_dotenv(".env.development")
config = BackblazeConfig(
bucket=os.environ["B2_BUCKET"],
region=os.environ["B2_REGION"], # e.g. us-west-004
access_key_id=os.environ["B2_ACCESS_KEY_ID"], # keyID
secret_access_key=os.environ["B2_SECRET_ACCESS_KEY"], # applicationKey
)
resource = BackblazeResource(config)
ws = Workspace({"/b2/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
return f"{len(records)} ops, {sum(r.bytes for r in records)} bytes"
async def main():
print(f"=== Backblaze B2 at {config.resolved_endpoint_url()} ===")
r = await ws.execute("ls /b2/")
print("ls /b2/:\n" + await r.stdout_str())
r = await ws.execute("find /b2/ -name '*.json' | head -n 5")
print("find *.json:\n" + await r.stdout_str())
r = await ws.execute("grep -m 1 mirage /b2/data/example.jsonl",
provision=True)
print(f"plan grep -m 1: network_read={r.network_read} "
f"precision={r.precision}")
print(f"\nStats: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
+58
View File
@@ -0,0 +1,58 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.ceph import CephConfig, CephResource
load_dotenv(".env.development")
config = CephConfig(
bucket=os.environ["CEPH_BUCKET"],
endpoint_url=os.environ["CEPH_ENDPOINT_URL"],
access_key_id=os.environ["CEPH_ACCESS_KEY_ID"],
secret_access_key=os.environ["CEPH_SECRET_ACCESS_KEY"],
)
resource = CephResource(config)
ws = Workspace({"/ceph/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
return f"{len(records)} ops, {sum(r.bytes for r in records)} bytes"
async def main():
print(f"=== Ceph RGW at {config.endpoint_url} ===")
r = await ws.execute("ls /ceph/")
print("ls /ceph/:\n" + await r.stdout_str())
r = await ws.execute("find /ceph/ -name '*.json' | head -n 5")
print("find *.json:\n" + await r.stdout_str())
r = await ws.execute("grep -m 1 mirage /ceph/data/example.jsonl",
provision=True)
print(f"plan grep -m 1: network_read={r.network_read} "
f"precision={r.precision}")
print(f"\nStats: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
+107
View File
@@ -0,0 +1,107 @@
import asyncio
import os
import shlex
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.chroma import ChromaConfig, ChromaResource
load_dotenv(".env.development")
def int_env(name: str, default: int) -> int:
value = os.environ.get(name)
if value is None:
return default
return int(value)
def bool_env(name: str, default: bool) -> bool:
value = os.environ.get(name)
if value is None:
return default
return value.lower() in {"1", "true", "yes", "on"}
def require_env(name: str) -> str:
value = os.environ.get(name)
if not value:
raise RuntimeError(f"Missing required environment variable: {name}")
return value
def build_resource() -> ChromaResource:
config = ChromaConfig(
host=os.environ.get("CHROMA_HOST", "localhost"),
port=int_env("CHROMA_PORT", 8000),
ssl=bool_env("CHROMA_SSL", False),
collection_name=require_env("CHROMA_COLLECTION"),
slug_field=os.environ.get("CHROMA_SLUG_FIELD", "page_slug"),
chunk_index_field=os.environ.get("CHROMA_CHUNK_INDEX_FIELD",
"chunk_index"),
)
return ChromaResource(config=config)
async def run(ws: Workspace, command: str, max_chars: int = 1000) -> str:
result = await ws.execute(command)
stdout = await result.stdout_str()
stderr = (result.stderr or b"").decode(errors="replace")
print(f"$ {command}")
if stdout:
output = stdout.strip()
if len(output) > max_chars:
output = output[:max_chars] + "\n..."
print(output)
if stderr:
print(stderr.strip())
print(f"[exit={result.exit_code}]\n")
return stdout
async def first_document_path(ws: Workspace) -> str | None:
output = await run(ws, "find /knowledge/ -type f | head -n 1")
path = output.strip()
return path or None
async def main() -> None:
resource = build_resource()
ws = Workspace({"/knowledge/": resource}, mode=MountMode.READ)
print("=== Chroma Knowledge ===\n")
await run(ws, "ls /knowledge/")
await run(ws, "tree /knowledge/", max_chars=1500)
await run(ws, "find /knowledge/ -type f | head -n 10")
first_path = await first_document_path(ws)
if first_path is None:
print("No documents found in the Chroma path tree.")
return
quoted_path = shlex.quote(first_path)
print(f"First document: {first_path}\n")
await run(ws, f"cat {quoted_path}", max_chars=1500)
await run(ws, f"head -n 5 {quoted_path}")
await run(ws, f"tail -n 5 {quoted_path}")
query = os.environ.get("CHROMA_EXAMPLE_QUERY", "getting started")
quoted_query = shlex.quote(query)
await run(ws, f"grep -in {quoted_query} /knowledge/", max_chars=1500)
await run(ws,
f"chroma-query --top-k 5 {quoted_query} /knowledge/",
max_chars=1500)
records = ws.ops.records
network_bytes = ws.ops.network_bytes
cache_bytes = ws.ops.cache_bytes
print("=== Stats ===")
print(f"{len(records)} ops, {network_bytes} network bytes, "
f"{cache_bytes} cache bytes")
if __name__ == "__main__":
asyncio.run(main())
+95
View File
@@ -0,0 +1,95 @@
import asyncio
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.chroma import ChromaConfig, ChromaResource
load_dotenv(".env.development")
def int_env(name: str, default: int) -> int:
value = os.environ.get(name)
if value is None:
return default
return int(value)
def bool_env(name: str, default: bool) -> bool:
value = os.environ.get(name)
if value is None:
return default
return value.lower() in {"1", "true", "yes", "on"}
def require_env(name: str) -> str:
value = os.environ.get(name)
if not value:
raise RuntimeError(f"Missing required environment variable: {name}")
return value
def build_resource() -> ChromaResource:
config = ChromaConfig(
host=os.environ.get("CHROMA_HOST", "localhost"),
port=int_env("CHROMA_PORT", 8000),
ssl=bool_env("CHROMA_SSL", False),
collection_name=require_env("CHROMA_COLLECTION"),
slug_field=os.environ.get("CHROMA_SLUG_FIELD", "page_slug"),
chunk_index_field=os.environ.get("CHROMA_CHUNK_INDEX_FIELD",
"chunk_index"),
)
return ChromaResource(config=config)
def first_file(vos, directory: str) -> str | None:
entries = vos.listdir(directory)
for entry in entries:
path = f"{directory.rstrip('/')}/{entry}"
if vos.path.isdir(path):
found = first_file(vos, path)
if found is not None:
return found
elif vos.path.isfile(path):
return path
return None
async def main() -> None:
resource = build_resource()
with Workspace({"/knowledge/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== Chroma VFS ===\n")
print("--- os.listdir('/knowledge') ---")
for entry in vos.listdir("/knowledge")[:20]:
print(f" {entry}")
path = first_file(vos, "/knowledge")
if path is None:
print("\nNo documents found in the Chroma path tree.")
return
print(f"\n--- open('{path}') ---")
with open(path) as file:
content = file.read()
preview = content[:1500]
print(preview)
if len(content) > len(preview):
print("...")
print("\n--- os.path metadata ---")
print(f" exists: {vos.path.exists(path)}")
print(f" isfile: {vos.path.isfile(path)}")
print(f" size: {vos.path.getsize(path)}")
records = ws.ops.records
total = sum(record.bytes for record in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
if __name__ == "__main__":
asyncio.run(main())
+83
View File
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
"""
Example: Run Claude Code inside a memory-backed FUSE filesystem.
All file operations from Claude happen in-memory — nothing touches disk.
After Claude exits, the workspace still holds everything Claude created.
Usage:
python examples/claude/claude-memory.py
python examples/claude/claude-memory.py --agent-id my-bot \
--prompt "write a fibonacci.py"
"""
import argparse
import os
import subprocess
import sys
import tempfile
from mirage.fuse.mount import mount_background
from mirage.resource.ram import RAMResource
from mirage.types import MountMode
from mirage.workspace import Workspace
def unmount(mountpoint: str) -> None:
if sys.platform == "darwin":
subprocess.run(["diskutil", "unmount", "force", mountpoint],
capture_output=True)
else:
subprocess.run(["fusermount", "-u", mountpoint], capture_output=True)
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--agent-id", default="claude", dest="agent_id")
parser.add_argument("--prompt",
default="write a hello world script in /hello.py")
args = parser.parse_args()
ws = Workspace({"/": RAMResource()}, mode=MountMode.WRITE)
with tempfile.TemporaryDirectory() as mountpoint:
t = mount_background(ws, mountpoint, agent_id=args.agent_id)
print(f"Mounted memory filesystem at {mountpoint}")
print(f"Agent: {args.agent_id}")
try:
subprocess.run(
["claude", "-p", args.prompt],
cwd=mountpoint,
env={
**os.environ, "MIRAGE_AGENT_ID": args.agent_id
},
)
finally:
unmount(mountpoint)
t.join(timeout=3)
print("\n=== Files Claude created (in memory) ===")
for path in ws.ls("/"):
try:
content = ws.cat(path)
print(f"\n--- {path} ({len(content)} bytes) ---")
print(content.decode(errors="replace"))
except (IsADirectoryError, ValueError):
print(f"\n--- {path}/ (directory) ---")
if __name__ == "__main__":
main()
+122
View File
@@ -0,0 +1,122 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
"""
Example: Use the Claude Agent SDK with a Mirage FUSE-backed workspace.
The agent reads, writes, and edits files — all intercepted by Mirage's
FUSE layer. Every file operation is recorded and can be inspected after
the agent finishes.
Usage:
pip install claude-agent-sdk
python examples/claude/claude-sdk.py
python examples/claude/claude-sdk.py --agent-id code-bot \
--prompt "create a python calculator module with tests"
"""
import argparse
import subprocess
import sys
import tempfile
import anyio
from claude_agent_sdk import ClaudeAgentOptions, ResultMessage, query
from mirage.fuse.fs import MirageFS
from mirage.fuse.mount import mount_background
from mirage.resource.ram import RAMResource
from mirage.types import MountMode
from mirage.workspace import Workspace
def unmount(mountpoint: str) -> None:
if sys.platform == "darwin":
subprocess.run(["diskutil", "unmount", "force", mountpoint],
capture_output=True)
else:
subprocess.run(["fusermount", "-u", mountpoint], capture_output=True)
def print_workspace(ws: Workspace, prefix: str = "/") -> None:
for path in ws.ls(prefix):
try:
content = ws.cat(path)
print(f"\n--- {path} ({len(content)} bytes) ---")
print(content.decode(errors="replace"))
except (IsADirectoryError, ValueError):
print(f"\n--- {path}/ (directory) ---")
def print_ops(ops: list[dict]) -> None:
for op in ops:
ts = op["timestamp"]
print(f" [{ts}] {op['op']:6s} {op['path']}")
async def run_agent(mountpoint: str, prompt: str, agent_id: str) -> None:
options = ClaudeAgentOptions(
cwd=mountpoint,
allowed_tools=["Read", "Write", "Edit", "Glob", "Grep", "Bash"],
permission_mode="bypassPermissions",
allow_dangerously_skip_permissions=True,
env={"MIRAGE_AGENT_ID": agent_id},
)
async for message in query(prompt=prompt, options=options):
if isinstance(message, ResultMessage):
print(f"\n=== Agent result ===\n{message.result}")
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--agent-id", default="claude-sdk", dest="agent_id")
parser.add_argument(
"--prompt",
default=(
"Create a file called /hello.py that prints 'Hello from Mirage!' "
"and a file called /utils.py with a function"
" that reverses a string."),
)
args = parser.parse_args()
ws = Workspace({"/": RAMResource()}, mode=MountMode.WRITE)
with tempfile.TemporaryDirectory() as mountpoint:
fs = MirageFS(ws.ops, agent_id=args.agent_id)
t = mount_background(ws, mountpoint, agent_id=args.agent_id)
print(f"Mounted memory workspace at {mountpoint}")
print(f"Agent: {args.agent_id}")
print(f"Prompt: {args.prompt}\n")
try:
anyio.run(run_agent, mountpoint, args.prompt, args.agent_id)
finally:
ops = fs.drain_ops()
unmount(mountpoint)
t.join(timeout=3)
print("\n=== FUSE operations captured ===")
if ops:
print_ops(ops)
else:
print(" (no operations recorded)")
print("\n=== Files in workspace (all in-memory) ===")
try:
print_workspace(ws)
except FileNotFoundError:
print(" (workspace is empty)")
if __name__ == "__main__":
main()
+274
View File
@@ -0,0 +1,274 @@
# Cross-resource workspace (CLI)
Drive a multi-mount workspace (`/s3`, `/gdrive`, `/gmail`, `/slack`,
`/discord`) end-to-end from the shell using `workspace.yaml`.
The two CLIs expose the same workspace HTTP API. Each
example below shows the **Python** CLI (`mirage`, on `$PATH`) and
the **TypeScript** CLI (`./mirage-ts`, a symlink to
`typescript/packages/cli/dist/bin/mirage.js` from the repo root).
Pick whichever CLI is convenient for the run; the command shapes match.
## Prereqs
- `.env.development` at the repo root with `AWS_`\*, `GOOGLE_*`,
`SLACK_BOT_TOKEN`, `DISCORD_BOT_TOKEN`.
- Python: `mirage` CLI on `$PATH` (e.g. `./python/.venv/bin/mirage`).
- TypeScript: `pnpm --filter @struktoai/mirage-cli build` then
`ln -sf typescript/packages/cli/dist/bin/mirage.js mirage-ts`
at the repo root (already gitignored).
## 1. Source env and create the workspace
The YAML's `${...}` placeholders resolve from your shell at create
time, so source first.
```bash
set -a && source .env.development && set +a
```
```bash
mirage workspace create examples/python/cross/workspace.yaml --id cross
./mirage-ts workspace create examples/python/cross/workspace.yaml --id cross
```
## 2. Inspect
```bash
mirage workspace list
./mirage-ts workspace list
```
```bash
mirage workspace get cross
./mirage-ts workspace get cross
```
## 3. Run commands across mounts
`/gdrive/` is index-first — list it once before reading individual
files, otherwise paths resolve to ENOENT.
```bash
mirage execute --workspace_id cross --command "ls /s3/"
./mirage-ts execute --workspace_id cross --command "ls /s3/"
```
```bash
mirage execute --workspace_id cross --command "ls /gdrive/"
./mirage-ts execute --workspace_id cross --command "ls /gdrive/"
```
```bash
mirage execute --workspace_id cross --command "head -n 1 /s3/data/example.jsonl"
./mirage-ts execute --workspace_id cross --command "head -n 1 /s3/data/example.jsonl"
```
```bash
mirage execute --workspace_id cross \
--command 'cat /s3/data/example.jsonl "/gdrive/AWS CDK.gdoc.json" | wc -l'
./mirage-ts execute --workspace_id cross \
--command 'cat /s3/data/example.jsonl "/gdrive/AWS CDK.gdoc.json" | wc -l'
```
## 4. Dry-run with `provision`
```bash
mirage provision --workspace_id cross --command "cat /s3/data/example.jsonl | wc -l"
./mirage-ts provision --workspace_id cross --command "cat /s3/data/example.jsonl | wc -l"
```
After a real read the same path flips from a network read to a cache
hit (`cache_hits=1`):
```bash
mirage execute --workspace_id cross --command "cat /s3/data/example.jsonl > /dev/null"
./mirage-ts execute --workspace_id cross --command "cat /s3/data/example.jsonl > /dev/null"
```
```bash
mirage provision --workspace_id cross --command "cat /s3/data/example.jsonl"
./mirage-ts provision --workspace_id cross --command "cat /s3/data/example.jsonl"
```
## 5. Snapshot and restore
Snapshots redact cloud creds at snapshot time, so loading needs fresh
creds via a config file. The same workspace YAML used for create works.
```bash
mirage workspace snapshot cross /tmp/cross.tar
./mirage-ts workspace snapshot cross /tmp/cross.tar
```
```bash
mirage workspace load /tmp/cross.tar examples/python/cross/workspace.yaml \
--id cross_loaded
./mirage-ts workspace load /tmp/cross.tar examples/python/cross/workspace.yaml \
--id cross_loaded
```
```bash
mirage workspace get cross_loaded --verbose
./mirage-ts workspace get cross_loaded --verbose
```
## 6. Clean up
The daemon exits ~30s after the last workspace is deleted.
```bash
mirage workspace delete cross
./mirage-ts workspace delete cross
```
```bash
mirage workspace delete cross_loaded
./mirage-ts workspace delete cross_loaded
```
## 7. Per-mount safeguards (Python CLI)
A mount can cap what a command streams back with `command_safeguards`,
so a runaway `cat`/`grep`/`rg` can't flood the agent or hang forever.
Each entry sets `max_lines` / `max_bytes` (output cap) and/or
`timeout_seconds` (deadline), with `on_exceed: truncate` (stop, exit 0,
add a notice) or `on_exceed: error` (stop, exit 1, add a notice).
This section is **Python-only**: the TS config schema does not yet carry
`command_safeguards`. It runs against its own workspace
(`cross_sg`) from [workspace_safeguards.yaml](workspace_safeguards.yaml),
so the steps above are untouched. That file guards `/s3` with:
`head` → 10 lines / truncate, `grep` → 20 lines / error, `rg` → a 1 ms
timeout.
```bash
set -a && source .env.development && set +a
mirage workspace create examples/python/cross/workspace_safeguards.yaml --id cross_sg
```
Warm the object once (the first S3 read fetches the whole object and can
take a few seconds; later reads are cache hits):
```bash
mirage execute --workspace_id cross_sg --command "head -n 1 /s3/data/example.jsonl"
```
`max_lines` + `on_exceed: truncate` — asking for 50 lines yields 10 plus a
notice on stderr, exit `0`:
```bash
mirage execute --workspace_id cross_sg --command "head -n 50 /s3/data/example.jsonl"
```
`max_lines` + `on_exceed: error``grep` matches thousands of lines, trips
the 20-line cap, and fails with exit `1`:
```bash
mirage execute --workspace_id cross_sg --command "grep mirage /s3/data/example.jsonl"
```
`timeout_seconds` — the 1 ms deadline trips on any real read, exit `124`
with `rg: timed out after 0.001s`:
```bash
mirage execute --workspace_id cross_sg --command "rg mirage /s3/data/example.jsonl"
```
Commands below their cap are untouched, so the earlier shapes still work
unchanged (1 line, no notice):
```bash
mirage execute --workspace_id cross_sg --command "head -n 1 /s3/data/example.jsonl"
```
Clean up:
```bash
mirage workspace delete cross_sg
```
## 8. Versioning (Python CLI)
The daemon keeps a git-backed history per workspace under
`~/.mirage/repos/<id>` (set `MIRAGE_VERSION_ROOT` to relocate). You commit
the live state as a version, then `log` / `diff` / `branch` / `checkout`,
following git. See the [CLI docs](https://docs.mirage.strukto.ai) for the
full reference.
This section is **Python-only** and runs against its own scratch workspace
(`cross_ver`) from
[workspace_versioning.yaml](workspace_versioning.yaml), so the steps above
are untouched. It uses a writable RAM mount: versioning tracks the workspace
tree regardless of backend, and this keeps the demo fast and writes nothing
to your real cloud backends.
> The history outlives the workspace: `workspace delete` does **not** remove
> `~/.mirage/repos/<id>`. Re-creating the same id resumes the old history, so
> this walkthrough starts from a clean id (or `rm -rf ~/.mirage/repos/cross_ver`
> first) to keep output predictable.
```bash
mirage workspace create examples/python/cross/workspace_versioning.yaml --id cross_ver
```
Commit two versions, then `log` (newest first):
```bash
mirage execute --workspace_id cross_ver --command "echo v1 > /notes.txt"
mirage workspace commit cross_ver -m "first"
mirage execute --workspace_id cross_ver --command "echo v2 > /notes.txt"
mirage workspace commit cross_ver -m "second"
mirage workspace log cross_ver
```
`diff` reports changed paths (added / modified / deleted). With no refs it
compares the live state to the branch HEAD; with two version ids it compares
those versions (substitute the ids `log` printed):
```bash
mirage workspace diff cross_ver # live vs HEAD (clean: all empty)
mirage workspace diff cross_ver <v1> <v2> # => modified: ["notes.txt"]
```
`branch` forks at a branch's current version; commit onto it with `-b`. `main`
is left untouched:
```bash
mirage workspace branch cross_ver exp
mirage execute --workspace_id cross_ver --command "echo on-exp > /notes.txt"
mirage workspace commit cross_ver -b exp -m "on exp"
mirage workspace log cross_ver -b exp # on exp, second, first
mirage workspace log cross_ver # main unchanged: second, first
```
`checkout` restores the live state in place to a version or branch
(overwriting uncommitted changes):
```bash
mirage workspace checkout cross_ver <v1>
mirage execute --workspace_id cross_ver --command "cat /notes.txt" # => v1
```
`clone --at` builds a new workspace from one of the source's past versions
(omit `--at` to clone the live state):
```bash
mirage workspace clone cross_ver --at <v1> --id cross_ver_at
mirage execute --workspace_id cross_ver_at --command "cat /notes.txt" # => v1
```
Clean up (and drop the histories so a rerun starts fresh):
```bash
mirage workspace delete cross_ver
mirage workspace delete cross_ver_at
rm -rf ~/.mirage/repos/cross_ver ~/.mirage/repos/cross_ver_at
```
## SDK alternative
The same flow driven from Python (with snapshot fingerprinting) lives
in [example.py](example.py) + [load_check.py](load_check.py).
+136
View File
@@ -0,0 +1,136 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
from mirage.resource.gmail import GmailConfig, GmailResource
from mirage.resource.s3 import S3Config, S3Resource
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
google_kwargs = dict(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
s3 = S3Resource(config=S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
))
gdrive = GoogleDriveResource(config=GoogleDriveConfig(**google_kwargs))
gmail = GmailResource(config=GmailConfig(**google_kwargs))
slack = SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
))
discord = DiscordResource(config=DiscordConfig(
token=os.environ["DISCORD_BOT_TOKEN"]))
# Stable path that both scripts agree on. Override with the env var
# MIRAGE_CROSS_DIR if you want a different location.
SNAPSHOT_DIR = os.environ.get("MIRAGE_CROSS_DIR", "/tmp/mirage-cross-clone")
SNAPSHOT_TAR = os.path.join(SNAPSHOT_DIR, "snapshot.tar")
EXPECTED_JSON = os.path.join(SNAPSHOT_DIR, "expected.json")
# Read-only commands whose output is deterministic between runs.
# The loader script reruns the same commands and asserts identical output.
_FINGERPRINT_COMMANDS = [
'head -n 1 /s3/data/example.jsonl',
'cat /s3/data/example.jsonl | wc -l',
'grep -c "mirage" /s3/data/example.jsonl',
'head -n 1 "/gdrive/AWS CDK.gdoc.json"',
'wc -l "/gdrive/AWS CDK.gdoc.json"',
'cat /s3/data/example.jsonl "/gdrive/AWS CDK.gdoc.json" | wc -l',
]
async def _capture(ws, cmd):
r = await ws.execute(cmd)
return {
"command": cmd,
"exit_code": r.exit_code,
"stdout": (await r.stdout_str()).strip(),
"stderr": (await r.stderr_str()).strip(),
}
async def main():
ws = Workspace(
{
"/s3": s3,
"/gdrive": gdrive,
"/gmail": gmail,
"/slack": slack,
"/discord": discord,
},
mode=MountMode.WRITE,
)
# gdrive needs `ls` of the parent folder first so the index has
# the file_id mapping the loader will need too.
await ws.execute("ls /gdrive/")
# Warm the cache by running each fingerprint command once and
# discarding the output. This way the snapshot's cache state
# matches what the loader will see — and downstream commands like
# `wc -l` produce identical formatting on both sides (some commands
# format slightly differently between cache-hit and source-read
# paths; warming makes the comparison apples-to-apples).
for cmd in _FINGERPRINT_COMMANDS:
await ws.execute(cmd)
# ── exercise the workspace (read-only for determinism) ──────────
print("=== capturing fingerprint commands ===\n")
fingerprints = []
for cmd in _FINGERPRINT_COMMANDS:
cap = await _capture(ws, cmd)
fingerprints.append(cap)
head = cap["stdout"][:80].replace("\n", " ")
print(f" {cmd}")
print(f" exit={cap['exit_code']} stdout={head!r}")
# ── snapshot the workspace + expected outputs ──────────────────
os.makedirs(SNAPSHOT_DIR, exist_ok=True)
await ws.snapshot(SNAPSHOT_TAR)
with open(EXPECTED_JSON, "w") as f:
json.dump(
{
"tar_path": SNAPSHOT_TAR,
"fingerprints": fingerprints,
},
f,
indent=2,
)
print("\n=== saved ===")
print(f" tar: {SNAPSHOT_TAR} "
f"({os.path.getsize(SNAPSHOT_TAR)} bytes)")
print(f" expected: {EXPECTED_JSON}")
print("\nNow run: ./python/.venv/bin/python "
"examples/python/cross/load_check.py")
if __name__ == "__main__":
asyncio.run(main())
+147
View File
@@ -0,0 +1,147 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage.resource.discord import DiscordConfig, DiscordResource
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
from mirage.resource.gmail import GmailConfig, GmailResource
from mirage.resource.s3 import S3Config, S3Resource
from mirage.resource.slack import SlackConfig, SlackResource
from mirage.workspace import Workspace
load_dotenv(".env.development")
EXPECTED_JSON = os.environ.get(
"MIRAGE_CROSS_EXPECTED",
"/tmp/mirage-cross-clone/expected.json",
)
def _fresh_resources():
google_kwargs = dict(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
return {
"/s3":
S3Resource(config=S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)),
"/gdrive":
GoogleDriveResource(config=GoogleDriveConfig(**google_kwargs)),
"/gmail":
GmailResource(config=GmailConfig(**google_kwargs)),
"/slack":
SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
)),
"/discord":
DiscordResource(config=DiscordConfig(
token=os.environ["DISCORD_BOT_TOKEN"])),
}
async def _capture(ws, cmd):
r = await ws.execute(cmd)
return {
"command": cmd,
"exit_code": r.exit_code,
"stdout": (await r.stdout_str()).strip(),
"stderr": (await r.stderr_str()).strip(),
}
def _summarize(field, expected, got):
if expected == got:
return "OK"
if isinstance(expected, str) and isinstance(got, str):
if len(expected) > 60 or len(got) > 60:
return (f"DIFF (lengths exp={len(expected)} got={len(got)}; "
f"first diff at char {_first_diff(expected, got)})")
return f"DIFF (expected={expected!r}, got={got!r})"
def _first_diff(a, b):
for i, (ca, cb) in enumerate(zip(a, b)):
if ca != cb:
return i
return min(len(a), len(b))
async def main():
if not os.path.exists(EXPECTED_JSON):
print(f"ERROR: {EXPECTED_JSON} not found.")
print("Run examples/python/cross/example.py first to create "
"the snapshot.")
sys.exit(1)
with open(EXPECTED_JSON) as f:
expected_doc = json.load(f)
tar_path = expected_doc["tar_path"]
print(f"=== loading {tar_path} ===")
ws = await Workspace.load(tar_path, resources=_fresh_resources())
mounts = sorted(m.prefix for m in ws.mounts())
print(f" mounts: {mounts}")
print(f" loaded history entries: {len(await ws.history())}")
# gdrive index belongs to the freshly-supplied resource (override
# drops the saved index). Repopulate it the same way the original
# script did, so the loader's commands resolve the same paths.
await ws.execute("ls /gdrive/")
# ── re-execute fingerprint commands and compare ─────────────────
print(f"\n=== re-running {len(expected_doc['fingerprints'])} commands "
"and comparing ===\n")
n_match = 0
n_diff = 0
for expected in expected_doc["fingerprints"]:
got = await _capture(ws, expected["command"])
all_ok = True
for field in ("exit_code", "stdout", "stderr"):
verdict = _summarize(field, expected[field], got[field])
if verdict != "OK":
all_ok = False
marker = "" if all_ok else ""
print(f" {marker} {expected['command']}")
if not all_ok:
for field in ("exit_code", "stdout", "stderr"):
v = _summarize(field, expected[field], got[field])
if v != "OK":
print(f" {field}: {v}")
if all_ok:
n_match += 1
else:
n_diff += 1
print(f"\n=== summary: {n_match} match, {n_diff} differ ===")
if n_diff > 0:
sys.exit(1)
if __name__ == "__main__":
asyncio.run(main())
+35
View File
@@ -0,0 +1,35 @@
mode: WRITE
consistency: LAZY
mounts:
/s3:
resource: s3
config:
bucket: ${AWS_S3_BUCKET}
region: ${AWS_DEFAULT_REGION}
aws_access_key_id: ${AWS_ACCESS_KEY_ID}
aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
/gdrive:
resource: gdrive
config:
client_id: ${GOOGLE_CLIENT_ID}
client_secret: ${GOOGLE_CLIENT_SECRET}
refresh_token: ${GOOGLE_REFRESH_TOKEN}
/gmail:
resource: gmail
config:
client_id: ${GOOGLE_CLIENT_ID}
client_secret: ${GOOGLE_CLIENT_SECRET}
refresh_token: ${GOOGLE_REFRESH_TOKEN}
/slack:
resource: slack
config:
token: ${SLACK_BOT_TOKEN}
/discord:
resource: discord
config:
token: ${DISCORD_BOT_TOKEN}
@@ -0,0 +1,28 @@
mode: WRITE
consistency: LAZY
# Same /s3 mount as workspace.yaml, plus per-mount command_safeguards so the
# guards can be observed firing from the CLI. Python-only: the TS config
# schema does not yet carry command_safeguards, so this lives in its own file
# to keep the shared workspace.yaml usable from both CLIs.
mounts:
/s3:
resource: s3
config:
bucket: ${AWS_S3_BUCKET}
region: ${AWS_DEFAULT_REGION}
aws_access_key_id: ${AWS_ACCESS_KEY_ID}
aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
command_safeguards:
# Cap head output at 10 lines and keep going (exit 0 + notice).
head:
max_lines: 10
on_exceed: truncate
# Cap grep output at 20 lines and fail hard (exit 1 + notice).
grep:
max_lines: 20
on_exceed: error
# Deliberately tiny deadline so any real read trips the timeout
# (exit 124). A production value would be seconds, not milliseconds.
rg:
timeout_seconds: 0.001
@@ -0,0 +1,12 @@
mode: WRITE
consistency: LAZY
# A standalone scratch workspace for the versioning walkthrough. Versioning is
# backend-agnostic (it tracks the workspace tree, not the mount type), so this
# uses a writable RAM mount to keep the demo fast and to avoid writing test
# files into your real cloud backends. The same verbs work on the cross
# workspace above.
mounts:
/:
resource: ram
mode: WRITE
@@ -0,0 +1,80 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.databricks_volume import (DatabricksVolumeConfig,
DatabricksVolumeResource)
load_dotenv(".env.development")
config = DatabricksVolumeConfig(
catalog=os.environ["DATABRICKS_VOLUME_CATALOG"],
schema=os.environ["DATABRICKS_VOLUME_SCHEMA"],
volume=os.environ["DATABRICKS_VOLUME_NAME"],
root_path=os.environ.get("DATABRICKS_VOLUME_ROOT_PATH", "/"),
host=os.environ.get("DATABRICKS_HOST"),
token=os.environ.get("DATABRICKS_TOKEN"),
profile=os.environ.get("DATABRICKS_CONFIG_PROFILE"),
)
resource = DatabricksVolumeResource(config=config)
async def _run(ws, cmd):
print(f"\n>>> {cmd}")
result = await ws.execute(cmd)
stdout = (await result.stdout_str()).strip()
stderr = (await result.stderr_str()).strip()
if stdout:
for line in stdout.splitlines()[:12]:
print(f" {line[:140]}")
if len(stdout.splitlines()) > 12:
print(f" ... ({len(stdout.splitlines())} lines total)")
if stderr:
print(f" [stderr] {stderr[:140]}")
if not stdout and not stderr:
print(f" (empty, exit={result.exit_code})")
return result
async def main():
ws = Workspace({"/dbx/": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /dbx/__nf_missing__.txt", "head /dbx/__nf_missing__.txt",
"stat /dbx/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
await _run(ws, "ls /dbx/")
await _run(ws, "tree -L 2 /dbx/")
await _run(ws, 'find /dbx/ -name "*.md"')
target = os.environ.get("DATABRICKS_VOLUME_SAMPLE_FILE")
if target:
await _run(ws, f'stat "{target}"')
await _run(ws, f'head -n 20 "{target}"')
await _run(ws, f'grep -n TODO "{target}"')
else:
print("\nSet DATABRICKS_VOLUME_SAMPLE_FILE to run file reads.")
if __name__ == "__main__":
asyncio.run(main())
+102
View File
@@ -0,0 +1,102 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import logging
import os
from agents import Runner
from agents.run import RunConfig
from agents.sandbox import SandboxAgent, SandboxRunConfig
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.agents.openai_agents import MirageSandboxClient
from mirage.resource.github import GitHubConfig, GitHubResource
from mirage.resource.linear import LinearConfig, LinearResource
from mirage.resource.ram import RAMResource
from mirage.resource.slack import SlackConfig, SlackResource
load_dotenv(".env.development")
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(name)s %(levelname)s %(message)s",
)
logging.getLogger("openai.agents").setLevel(logging.DEBUG)
GITHUB_OWNER = os.environ.get("MIRAGE_GITHUB_OWNER", "strukto-ai")
GITHUB_REPO = os.environ.get("MIRAGE_GITHUB_REPO", "mirage")
GITHUB_REF = os.environ.get("MIRAGE_GITHUB_REF", "main")
async def main() -> None:
slack = SlackResource(config=SlackConfig(
token=os.environ["SLACK_BOT_TOKEN"],
search_token=os.environ.get("SLACK_USER_TOKEN"),
))
github = GitHubResource(
config=GitHubConfig(token=os.environ["GITHUB_TOKEN"]),
owner=GITHUB_OWNER,
repo=GITHUB_REPO,
ref=GITHUB_REF,
)
linear = LinearResource(config=LinearConfig(
api_key=os.environ["LINEAR_API_KEY"]))
ws = Workspace({
"/": (RAMResource(), MountMode.WRITE),
"/slack": (slack, MountMode.READ),
"/github": (github, MountMode.READ),
"/linear": (linear, MountMode.WRITE),
})
_orig_exec = ws.execute
async def _trace_exec(cmd_str, *args, **kwargs):
print(f"[shell] {cmd_str}", flush=True)
result = await _orig_exec(cmd_str, *args, **kwargs)
out = (result.stdout or b"")[:200]
if out:
print(f"[shell] -> {out!r}", flush=True)
return result
ws.execute = _trace_exec # type: ignore[assignment]
client = MirageSandboxClient(ws)
agent = SandboxAgent(
name="Mirage design feedback agent",
model="gpt-5.5",
instructions=ws.file_prompt,
)
task = ("Triage the latest user feedback about Mirage from the Slack "
"incident channel: read the message and any attached screenshot, "
"find the relevant code in the Mirage GitHub repo, then file a "
"design issue in the Strukto-ai team on Linear using "
"`linear-issue-create` with the feedback and code references.")
result = await Runner.run(
agent,
task,
max_turns=30,
run_config=RunConfig(sandbox=SandboxRunConfig(client=client)),
)
print("\n--- agent output ---")
print(result.final_output)
if __name__ == "__main__":
asyncio.run(main())
+90
View File
@@ -0,0 +1,90 @@
import asyncio
import os
import shlex
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.dify import DifyConfig, DifyResource
load_dotenv(".env.development")
def require_env(name: str) -> str:
value = os.environ.get(name)
if not value:
raise RuntimeError(f"Missing required environment variable: {name}")
return value
def build_resource() -> DifyResource:
config = DifyConfig(
api_key=require_env("DIFY_API_KEY"),
base_url=os.environ.get("DIFY_BASE_URL", "https://api.dify.ai/v1"),
dataset_id=require_env("DIFY_DATASET_ID"),
slug_metadata_name=os.environ.get("DIFY_SLUG_METADATA_NAME", "slug"),
)
return DifyResource(config=config)
async def run(ws: Workspace, command: str, max_chars: int = 800) -> str:
result = await ws.execute(command)
stdout = await result.stdout_str()
stderr = (result.stderr or b"").decode(errors="replace")
print(f"$ {command}")
if stdout:
output = stdout.strip()
if len(output) > max_chars:
output = output[:max_chars] + "\n..."
print(output)
if stderr:
print(stderr.strip())
print(f"[exit={result.exit_code}]\n")
return stdout
async def first_document_path(ws: Workspace) -> str | None:
output = await run(ws, "find /knowledge/ -type f | head -n 1")
path = output.strip()
return path or None
async def main() -> None:
resource = build_resource()
ws = Workspace({"/knowledge/": resource}, mode=MountMode.READ)
print("=== Dify Knowledge ===\n")
await run(ws, "ls /knowledge/")
await run(ws, "find /knowledge/ -type f | head -n 10")
first_path = await first_document_path(ws)
if first_path is None:
print("No completed documents found in the Dify dataset.")
return
quoted_path = shlex.quote(first_path)
print(f"First document: {first_path}\n")
await run(ws, f"cat {quoted_path}", max_chars=1200)
await run(ws, f"head -n 5 {quoted_path}")
await run(ws, f"tail -n 5 {quoted_path}")
await run(ws, f"wc {quoted_path}")
query = os.environ.get("DIFY_EXAMPLE_QUERY", "getting started")
quoted_query = shlex.quote(query)
await run(ws, f"grep -i {quoted_query} {quoted_path}")
await run(ws,
f"search --method hybrid --top-k 5 {quoted_query} /knowledge/",
max_chars=1200)
records = ws.ops.records
network_bytes = ws.ops.network_bytes
cache_bytes = ws.ops.cache_bytes
print("=== Stats ===")
print(f"{len(records)} ops, {network_bytes} network bytes, "
f"{cache_bytes} cache bytes")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,59 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.digitalocean import (DigitalOceanConfig,
DigitalOceanResource)
load_dotenv(".env.development")
config = DigitalOceanConfig(
bucket=os.environ["DO_SPACE"],
region=os.environ["DO_REGION"], # e.g. nyc3
access_key_id=os.environ["DO_ACCESS_KEY_ID"],
secret_access_key=os.environ["DO_SECRET_ACCESS_KEY"],
)
resource = DigitalOceanResource(config)
ws = Workspace({"/do/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
return f"{len(records)} ops, {sum(r.bytes for r in records)} bytes"
async def main():
print(f"=== DigitalOcean Spaces at {config.resolved_endpoint_url()} ===")
r = await ws.execute("ls /do/")
print("ls /do/:\n" + await r.stdout_str())
r = await ws.execute("find /do/ -name '*.json' | head -n 5")
print("find *.json:\n" + await r.stdout_str())
r = await ws.execute("grep -m 1 mirage /do/data/example.jsonl",
provision=True)
print(f"plan grep -m 1: network_read={r.network_read} "
f"precision={r.precision}")
print(f"\nStats: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
+305
View File
@@ -0,0 +1,305 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import re
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
load_dotenv(".env.development")
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
def _assert_nonempty(text: str, msg: str) -> None:
if not text.strip():
raise AssertionError(f"regression: {msg}")
async def main():
ws = Workspace({"/discord": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /discord/__nf_missing__.txt",
"head /discord/__nf_missing__.txt",
"stat /discord/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── discover structure ────────────────────────────
print("=== ls /discord/ (guilds) ===")
r = await ws.execute("ls /discord/")
print(await r.stdout_str())
_assert_nonempty(await r.stdout_str(), "ls /discord/ returned no guilds")
guilds = (await r.stdout_str()).strip().split("\n")
guild = guilds[0].strip()
print(f"=== ls /discord/{guild}/channels/ ===")
r = await ws.execute(f'ls "/discord/{guild}/channels/"')
print(await r.stdout_str())
_assert_nonempty(await r.stdout_str(), "no channels in first guild")
print(f"=== ls -l /discord/{guild}/channels/ "
"(mtime from last_message_id) ===")
long_ch = await ws.execute(
f'ls -l "/discord/{guild}/channels/" | head -n 5')
print(await long_ch.stdout_str())
channels = (await r.stdout_str()).strip().splitlines()
ch = channels[0].strip()
base = f"/discord/{guild}/channels/{ch}"
# ── pick the most recent date that has messages ────
print(f"\n=== ls {base}/ (date directories) ===")
r = await ws.execute(f'ls "{base}/" | tail -n 5')
print(await r.stdout_str())
dates = (await r.stdout_str()).strip().splitlines()
if not dates:
print(" (no date directories — channel is empty)")
return
# Walk newest → oldest, find one with non-empty chat.jsonl
target_date: str | None = None
for d in reversed(dates):
d = d.strip()
r = await ws.execute(f'cat "{base}/{d}/chat.jsonl" | head -c 1')
if (await r.stdout_str()).strip():
target_date = d
break
if target_date is None:
target_date = dates[-1].strip()
print(f" using date: {target_date}")
file_path = f"{base}/{target_date}/chat.jsonl"
# ── cat the chat.jsonl ────────────────────────────
print(f"\n=== cat {target_date}/chat.jsonl | head -n 3 ===")
r = await ws.execute(f'cat "{file_path}" | head -n 3')
print((await r.stdout_str())[:300])
# ── list date contents (chat.jsonl + files dir) ────
print(f"\n=== ls {base}/{target_date}/ ===")
r = await ws.execute(f'ls "{base}/{target_date}/"')
print(await r.stdout_str())
# ── list attachments for that date (may be empty) ─
print(f"\n=== ls {base}/{target_date}/files/ (attachments) ===")
r = await ws.execute(f'ls "{base}/{target_date}/files/"')
files_out = (await r.stdout_str()).strip()
if files_out:
for line in files_out.splitlines()[:5]:
print(f" {line}")
else:
print(" (no attachments on this date)")
# ── stat + cat first attachment (byte-exact CDN download) ──
if files_out:
first_att = files_out.splitlines()[0].strip()
att_path = f"{base}/{target_date}/files/{first_att}"
print(f"\n=== stat {first_att} ===")
r = await ws.execute(f'stat "{att_path}"')
stat_out = (await r.stdout_str()).strip()
print(f" {stat_out[:200]}")
size_match = re.search(r"size=(\d+)", stat_out)
expected_size = int(size_match.group(1)) if size_match else None
print(f"\n=== cat {first_att} (byte-exact CDN download) ===")
r = await ws.execute(f'cat "{att_path}"')
data = await r.materialize_stdout()
print(f" bytes={len(data)} expected={expected_size} "
f"exit={r.exit_code}")
if expected_size is not None and len(data) != expected_size:
raise AssertionError(
f"regression: attachment cat got {len(data)} bytes, "
f"expected {expected_size}")
# ── grep at FILE level ────────────────────────────
print(f"\n=== grep at FILE level: grep . {target_date}/chat.jsonl ===")
r = await ws.execute(f'grep -c . "{file_path}"')
print(f" line count: {(await r.stdout_str()).strip()}")
# ── grep at CHANNEL level (Discord search push-down) ──
print(f"\n=== grep at CHANNEL level: grep . {base}/ ===")
r = await ws.execute(f'grep -m 5 . "{base}/"')
print(f" exit={r.exit_code}")
out = (await r.stdout_str()).strip()
if out:
for line in out.splitlines()[:5]:
print(f" {line[:120]}")
else:
print(" (no results)")
err = await r.stderr_str()
if err:
print(f" stderr: {err[:200]}")
# ── grep at GUILD level ──────────────────────────
print(f"\n=== grep at GUILD level: grep . /discord/{guild}/ ===")
r = await ws.execute(f'grep -m 5 . "/discord/{guild}/"')
print(f" exit={r.exit_code}")
out = (await r.stdout_str()).strip()
if out:
for line in out.splitlines()[:5]:
print(f" {line[:120]}")
else:
print(" (no results)")
# ── jq pipeline ──────────────────────────────────
print(f"\n=== jq '.[] | .author.username' {target_date}/chat.jsonl ===")
r = await ws.execute(f'jq -r ".[] | .author.username" "{file_path}"'
' | head -n 5')
out = (await r.stdout_str()).strip()
if out:
for line in out.splitlines()[:5]:
print(f" {line}")
# ── stat ─────────────────────────────────────────
print(f"\n=== stat {file_path} ===")
r = await ws.execute(f'stat "{file_path}"')
print(f" {(await r.stdout_str()).strip()[:200]}")
# ── wc / head / tail ─────────────────────────────
print(f"\n=== wc -l {target_date}/chat.jsonl ===")
r = await ws.execute(f'wc -l "{file_path}"')
print(f" {(await r.stdout_str()).strip()}")
# ── basename / dirname / realpath (path ops) ─────
print(f"\n=== basename {file_path} ===")
r = await ws.execute(f'basename "{file_path}"')
out = (await r.stdout_str()).strip()
print(f" {out}")
assert out == "chat.jsonl", f"basename expected 'chat.jsonl', got {out!r}"
expected_dir = f"{base}/{target_date}"
print(f"\n=== dirname {file_path} ===")
r = await ws.execute(f'dirname "{file_path}"')
out = (await r.stdout_str()).strip()
print(f" {out}")
assert out == expected_dir, (
f"dirname expected {expected_dir!r}, got {out!r}")
print(f"\n=== realpath {file_path} ===")
r = await ws.execute(f'realpath "{file_path}"')
out = (await r.stdout_str()).strip()
print(f" {out}")
assert out == file_path, f"realpath expected {file_path!r}, got {out!r}"
print(f"\n=== realpath -e {file_path} (must exist) ===")
r = await ws.execute(f'realpath -e "{file_path}"')
print(f" exit={r.exit_code} {(await r.stdout_str()).strip()}")
assert r.exit_code == 0, (
"regression: realpath -e failed for existing file; "
f"stderr={await r.stderr_str()}")
# ── tree ─────────────────────────────────────────
print(f"\n=== tree -L 2 /discord/{guild}/ ===")
r = await ws.execute(f'tree -L 2 "/discord/{guild}/" | head -n 20')
out = (await r.stdout_str()).strip()
for line in out.splitlines()[:20]:
print(f" {line}")
# ── find chat.jsonl everywhere ───────────────────
print(f"\n=== find /discord/{guild}/ -name chat.jsonl | head -n 5 ===")
r = await ws.execute(f'find "/discord/{guild}/" -name "chat.jsonl"'
' | head -n 5')
print(f" exit={r.exit_code}")
out = (await r.stdout_str()).strip()
if out:
for line in out.splitlines():
print(f" {line}")
if r.exit_code != 0:
raise AssertionError(
f"regression: find chat.jsonl exited {r.exit_code} "
"(soft errors should not abort)")
# -path matches the display path; -size counts dirs and sizeless
# rendered files as 0 (so +0c drops them, -1k keeps them).
print(f"\n=== find /discord/{guild}/ -path '*channels*' | head -n 5 ===")
r = await ws.execute(f'find "/discord/{guild}/" -path "*channels*"'
' | head -n 5')
print(f" exit={r.exit_code}")
out = (await r.stdout_str()).strip()
if out:
for line in out.splitlines():
print(f" {line}")
if r.exit_code != 0 or not out:
raise AssertionError("regression: find -path returned nothing")
print(f"\n=== find /discord/{guild}/ -maxdepth 1 -size +0c ===")
r = await ws.execute(f'find "/discord/{guild}/" -maxdepth 1 -size +0c')
print(f" exit={r.exit_code} (dirs count as size 0, expect no output)")
if (await r.stdout_str()).strip():
raise AssertionError("regression: -size +0c matched a directory")
# ── pwd / cd / relative ──────────────────────────
print(f"\n=== cd {base} ===")
r = await ws.execute(f'cd "{base}"')
print(f" exit={r.exit_code}")
print("\n=== pwd (after cd) ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print(f"\n=== cat {target_date}/chat.jsonl (relative) | head -n 1 ===")
r = await ws.execute(f'cat "{target_date}/chat.jsonl" | head -n 1')
out = (await r.stdout_str()).strip()
if out:
print(f" {out[:120]}")
# ── members ──────────────────────────────────────
print(f"\n=== ls /discord/{guild}/members/ | head -n 5 ===")
r = await ws.execute(f'ls "/discord/{guild}/members/" | head -n 5')
mem_out = (await r.stdout_str()).strip()
if mem_out:
for line in mem_out.splitlines():
print(f" {line}")
first_member = mem_out.splitlines()[0].strip()
print(f"\n=== cat /discord/{guild}/members/{first_member} ===")
r = await ws.execute(f'cat "/discord/{guild}/members/{first_member}"')
body = (await r.stdout_str()).strip()
print(f" {body[:200]}")
else:
print(" (no members visible)")
# ── glob expansion: a mid-path glob replaces the guild segment
# (which may contain spaces) and walks into the literal tail.
print("\n=== echo /discord/*/channels (mid-path glob) ===")
r = await ws.execute("echo /discord/*/channels")
out = (await r.stdout_str()).strip()
print(f" {out[:200]}")
assert out.endswith("/channels"), "mid-path glob did not expand"
print("\n=== for f in /discord/*/channels/* (channel glob loop) ===")
r = await ws.execute(
"for f in /discord/*/channels/*; do echo found:$f; done | head -n 3")
out = (await r.stdout_str()).strip()
for line in out.splitlines():
print(f" {line[:120]}")
# A glob that matches nothing stays the literal word, so the
# command reports it like GNU coreutils.
print("\n=== cat /discord/zz-none-*/guild.json (no match) ===")
r = await ws.execute("cat /discord/zz-none-*/guild.json")
err = (await r.stderr_str()).strip()
print(f" exit={r.exit_code} {err[:120]}")
assert r.exit_code == 1 and "zz-none-*" in err
if __name__ == "__main__":
asyncio.run(main())
+152
View File
@@ -0,0 +1,152 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
import subprocess
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
load_dotenv(".env.development")
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
with Workspace({"/discord/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
# ── list guilds ──────────────────────────────
print("--- os.listdir() guilds ---")
# Skip the virtual /.mirage dir (agent metadata), it is not a guild.
guilds = [g for g in os.listdir(mp) if not g.startswith(".")]
for g in guilds:
print(f" {g}")
if not guilds:
print("no guilds found")
else:
guild = guilds[0]
# ── list guild contents ──────────────────
print(f"\n--- os.listdir() {guild} ---")
contents = os.listdir(f"{mp}/{guild}")
for c in contents:
print(f" {c}")
# ── list channels ────────────────────────
print(f"\n--- os.listdir() {guild}/channels ---")
channels = os.listdir(f"{mp}/{guild}/channels")
for ch in channels:
print(f" {ch}")
if channels:
ch = channels[0]
# ── list date files ──────────────────
print(f"\n--- os.listdir() {ch} (last 5 dates) ---")
dates = os.listdir(f"{mp}/{guild}/channels/{ch}")
for d in dates[-5:]:
print(f" {d}")
# ── read chat.jsonl ──────────────────
if dates:
target = dates[-1]
date_dir = f"{mp}/{guild}/channels/{ch}/{target}"
chat_path = f"{date_dir}/chat.jsonl"
# chat.jsonl has no size until fetched: unopened it stats as
# 0 bytes; any open (cat/wc/cp) hydrates it, and stat then
# reports the real size (see docs/python/setup/fuse.mdx).
print(
f"\n--- size-unknown semantics on {target}/chat.jsonl ---")
print(
f" stat before open: {os.stat(chat_path).st_size} bytes")
wc = subprocess.run(["wc", "-lc", chat_path],
capture_output=True,
text=True)
n_lines, n_bytes = wc.stdout.split()[:2]
print(
f" wc -lc : {n_lines} messages, {n_bytes} bytes")
print(
f" stat after read : {os.stat(chat_path).st_size} bytes")
print(f"\n--- open() + read {target}/chat.jsonl ---")
with open(chat_path) as f:
text = f.read().strip()
if text:
for i, line in enumerate(text.splitlines()):
if i >= 5:
print(" ...")
break
line = line.strip()
if not line:
continue
try:
msg = json.loads(line)
except json.JSONDecodeError:
break
author = msg.get("author", {}).get("username", "?")
content = msg.get("content", "")
print(f" [{author}] {content[:80]}")
else:
print(" (empty — no messages on this date)")
# list attachments
files_dir = f"{date_dir}/files"
try:
atts = os.listdir(files_dir)
except (FileNotFoundError, OSError):
atts = []
if atts:
print(f"\n--- os.listdir() {target}/files ---")
for a in atts[:5]:
print(f" {a}")
# ── list members ─────────────────────────
print(f"\n--- os.listdir() {guild}/members ---")
members = os.listdir(f"{mp}/{guild}/members")
for m in members:
print(f" {m}")
if members:
member_path = f"{mp}/{guild}/members/{members[0]}"
print(f"\n--- open() + read {members[0]} ---")
with open(member_path) as f:
text = f.read().strip()
if text:
try:
data = json.loads(text)
user = data.get("user", {})
print(f" username: {user.get('username')}")
print(f" id: {user.get('id')}")
except json.JSONDecodeError:
print(f" (raw: {text[:100]})")
else:
print(" (empty)")
# ── interactive: browse the mount in another terminal ──
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> cat {mp}/<guild>/channels/<ch>/<date>/chat.jsonl")
print(">>> Press Enter to unmount and exit...")
input()
# ── stats ────────────────────────────────────
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+152
View File
@@ -0,0 +1,152 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import re
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
load_dotenv(".env.development")
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
async def main():
with Workspace({"/discord/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE: open() reads from Discord transparently ===\n")
print("--- os.listdir() guilds ---")
guilds = vos.listdir("/discord")
for g in guilds:
print(f" {g}")
if not guilds:
return
guild = guilds[0]
guild_dir = f"/discord/{guild}"
print(f"\n--- os.listdir() {guild_dir} ---")
for s in vos.listdir(guild_dir):
print(f" {s}")
ch_root = f"{guild_dir}/channels"
print(f"\n--- os.listdir() {ch_root} ---")
channels = vos.listdir(ch_root)
for ch in channels[:5]:
print(f" {ch}")
if not channels:
return
ch = channels[0]
ch_dir = f"{ch_root}/{ch}"
print(f"\n--- os.listdir() {ch_dir} (last 5 dates) ---")
dates = vos.listdir(ch_dir)
for d in dates[-5:]:
print(f" {d}")
if dates:
for d in reversed(dates):
chat_path = f"{ch_dir}/{d}/chat.jsonl"
try:
with open(chat_path) as f:
content = f.read()
except FileNotFoundError:
continue
lines = [
line_text for line_text in content.strip().split("\n")
if line_text.strip()
]
if lines:
print(f"\n--- open() + read {d}/chat.jsonl ---")
print(f" messages: {len(lines)}")
for line in lines[:3]:
rec = json.loads(line)
author = rec.get("author", {}).get("username", "?")
text = rec.get("content", "")[:80]
print(f" [{author}] {text}")
# also list attachments in that day's files dir
files_dir = f"{ch_dir}/{d}/files"
try:
atts = vos.listdir(files_dir)
except FileNotFoundError:
atts = []
if atts:
print(f"\n--- os.listdir() {d}/files ---")
for a in atts[:5]:
print(f" {a}")
print("\n--- os.path.isfile / isdir / exists ---")
print(f" isfile(chat.jsonl): "
f"{vos.path.isfile(chat_path)}")
print(f" isdir(files/): "
f"{vos.path.isdir(files_dir)}")
print(f" exists(bogus): "
f"{vos.path.exists(f'{ch_dir}/{d}/nope.txt')}")
print(f"\n--- os.stat {d}/chat.jsonl ---")
st = vos.stat(chat_path)
print(f" type={st.type} size={st.size}")
if atts:
att_path = f"{files_dir}/{atts[0]}"
print(f"\n--- os.stat {atts[0]} ---")
ast = vos.stat(att_path)
print(f" type={ast.type} size={ast.size}")
print(f"\n--- open({atts[0]}, 'rb') ---")
with open(att_path, "rb") as f:
blob = f.read()
print(f" bytes={len(blob)} expected={ast.size} "
f"match={len(blob) == ast.size}")
if ast.size is not None and len(blob) != ast.size:
raise AssertionError(
f"regression: open('rb') got {len(blob)} "
f"bytes, expected {ast.size}")
print("\n--- json.loads + regex search on chat.jsonl ---")
pattern = re.compile(r"\S+")
matches = 0
for raw in lines:
rec = json.loads(raw)
text = rec.get("content", "") or ""
if pattern.search(text):
matches += 1
print(f" messages with non-whitespace content: "
f"{matches}/{len(lines)}")
break
else:
print("\n (no messages found in recent dates)")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+162
View File
@@ -0,0 +1,162 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import copy as _copy
import os
import shutil
import tempfile
from pathlib import Path
from mirage import MountMode, Workspace
from mirage.resource.disk import DiskResource
REPO_ROOT = Path(__file__).resolve().parents[3]
DATA_DIR = REPO_ROOT / "data"
tmp = tempfile.mkdtemp()
shutil.copytree(DATA_DIR, Path(tmp) / "files", dirs_exist_ok=True)
resource = DiskResource(root=tmp + "/files")
async def main() -> None:
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
print("=== ls /data/ ===")
result = await ws.execute("ls /data/")
print(await result.stdout_str())
print("=== cat /data/example.json ===")
result = await ws.execute("cat /data/example.json")
print(await result.stdout_str())
print("=== head -n 3 /data/example.jsonl ===")
result = await ws.execute("head -n 3 /data/example.jsonl")
print(await result.stdout_str())
print("=== tail -n 2 /data/example.jsonl ===")
result = await ws.execute("tail -n 2 /data/example.jsonl")
print(await result.stdout_str())
print("=== wc /data/example.json ===")
result = await ws.execute("wc /data/example.json")
print(await result.stdout_str())
print("=== stat /data/example.json ===")
result = await ws.execute("stat /data/example.json")
print(await result.stdout_str())
print("=== tree /data/ ===")
result = await ws.execute("tree /data/")
print(await result.stdout_str())
print("=== find /data/ -name '*.json' ===")
result = await ws.execute("find /data/ -name '*.json'")
print(await result.stdout_str())
print("=== grep example /data/example.json ===")
result = await ws.execute("grep example /data/example.json")
print(await result.stdout_str())
print("=== rg example /data/example.json ===")
result = await ws.execute("rg example /data/example.json")
print(await result.stdout_str())
print("=== du /data/ ===")
result = await ws.execute("du /data/")
print(await result.stdout_str())
print("=== jq .company /data/example.json ===")
result = await ws.execute('jq ".company" /data/example.json')
print(await result.stdout_str())
print("=== basename /data/example.json ===")
result = await ws.execute("basename /data/example.json")
print(await result.stdout_str())
print("=== dirname /data/example.json ===")
result = await ws.execute("dirname /data/example.json")
print(await result.stdout_str())
print("=== sort /data/example.jsonl ===")
result = await ws.execute("sort /data/example.jsonl")
print(await result.stdout_str())
print("=== cp /data/example.json /data/copy.json ===")
await ws.execute("cp /data/example.json /data/copy.json")
result = await ws.execute("ls /data/")
print(await result.stdout_str())
print("=== mv /data/copy.json /data/renamed.json ===")
await ws.execute("mv /data/copy.json /data/renamed.json")
result = await ws.execute("ls /data/")
print(await result.stdout_str())
print("=== rm /data/renamed.json ===")
await ws.execute("rm /data/renamed.json")
result = await ws.execute("ls /data/")
print(await result.stdout_str())
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /data/missing.json", "head /data/missing.json",
"stat /data/missing.json"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── persistence: save / load / copy / deepcopy ──────────────────
# Disk has no redacted config: full file tree is in the snapshot.
# Default load behavior creates a fresh tmpdir. Caller can override
# by supplying DiskResource(root=...) in resources={...}.
print("\n=== PERSISTENCE ===\n")
with tempfile.NamedTemporaryFile(suffix=".tar", delete=False) as f:
snap = f.name
custom_root = tempfile.mkdtemp(prefix="mirage-disk-restore-")
try:
await ws.snapshot(snap)
print(f" saved → {snap} ({os.path.getsize(snap)} bytes)")
# Load with default fresh tmpdir
loaded_default = await Workspace.load(snap)
r = await loaded_default.execute("ls /data/")
print(f" loaded (default tmpdir) ls: "
f"{(await r.stdout_str()).strip()[:80]}")
# Load with caller-supplied root — files written into custom_root
loaded_custom = await Workspace.load(
snap, resources={"/data": DiskResource(root=custom_root)})
r = await loaded_custom.execute("ls /data/")
print(f" loaded (root={custom_root[:40]}…) ls: "
f"{(await r.stdout_str()).strip()[:80]}")
print(f" custom_root contents: {sorted(os.listdir(custom_root))[:5]}")
cp = await ws.copy()
print(f" copy() mounts: {[m.prefix for m in cp.mounts()]}")
for op_name, op in (("deepcopy", _copy.deepcopy), ("shallow copy",
_copy.copy)):
try:
op(ws)
print(f"{op_name} should have raised")
except NotImplementedError as e:
print(f"{op_name} raises: {str(e)[:60]}")
finally:
os.unlink(snap)
shutil.rmtree(custom_root, ignore_errors=True)
if __name__ == "__main__":
asyncio.run(main())
+55
View File
@@ -0,0 +1,55 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
import shutil
import tempfile
from pathlib import Path
from mirage import Mount, MountMode, Workspace
from mirage.resource.disk import DiskResource
REPO_ROOT = Path(__file__).resolve().parents[3]
DATA_DIR = REPO_ROOT / "data"
tmp = tempfile.mkdtemp()
shutil.copytree(DATA_DIR, Path(tmp) / "files", dirs_exist_ok=True)
resource = DiskResource(root=tmp + "/files")
with Workspace({"/data/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
data_path = mp
print("--- os.listdir() ---")
entries = os.listdir(data_path)
for e in entries:
size = os.path.getsize(f"{data_path}/{e}")
print(f" {e:30s} {size:>10,} bytes")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and try:")
print(f">>> ls -la {mp}/")
print(f">>> cat {mp}/example.json")
print(f">>> cat {mp}/example.json | jq .")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+61
View File
@@ -0,0 +1,61 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import shutil
import sys
import tempfile
from pathlib import Path
from mirage import MountMode, Workspace
from mirage.resource.disk import DiskResource
REPO_ROOT = Path(__file__).resolve().parents[3]
DATA_DIR = REPO_ROOT / "data"
tmp = tempfile.mkdtemp()
shutil.copytree(DATA_DIR, Path(tmp) / "files", dirs_exist_ok=True)
resource = DiskResource(root=tmp + "/files")
async def main():
ws = Workspace({"/data/": resource}, mode=MountMode.READ)
with ws:
vos = sys.modules["os"]
print("=== VFS MODE ===\n")
print("--- os.listdir() ---")
entries = vos.listdir("/data")
for e in entries:
print(f" {e}")
print("\n--- open() + read ---")
with open("/data/example.json") as f:
print(f" {f.read().strip()}")
print("\n--- os.path.exists() ---")
print(f" example.json: {vos.path.exists('/data/example.json')}")
print(f" nope.txt: {vos.path.exists('/data/nope.txt')}")
print("\n--- os.path.isdir() ---")
print(f" /data: {vos.path.isdir('/data')}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+157
View File
@@ -0,0 +1,157 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.email import EmailConfig, EmailResource
load_dotenv(".env.development")
config = EmailConfig(
imap_host=os.environ["IMAP_HOST"],
smtp_host=os.environ["SMTP_HOST"],
username=os.environ["EMAIL_USERNAME"],
password=os.environ["EMAIL_PASSWORD"],
max_messages=20,
)
resource = EmailResource(config=config)
async def main() -> None:
ws = Workspace({"/email": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /email/__nf_missing__.txt",
"head /email/__nf_missing__.txt",
"stat /email/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=== ls /email/ ===")
result = await ws.execute("ls /email/")
print(await result.stdout_str())
folders = (await result.stdout_str()).strip().splitlines()
folder = "Inbox"
if not any("Inbox" in f or "INBOX" in f for f in folders):
folder = folders[0] if folders else ""
if not folder:
print("No folders")
return
print(f"=== ls /email/{folder}/ ===")
result = await ws.execute(f"ls /email/{folder}/")
print(await result.stdout_str())
dates = (await result.stdout_str()).strip().splitlines()
if not dates:
print("No dates")
return
first_date = dates[0]
print(f"=== ls /email/{folder}/{first_date}/ ===")
result = await ws.execute(f"ls /email/{folder}/{first_date}/")
print(await result.stdout_str())
messages = (await result.stdout_str()).strip().splitlines()
msg_files = [m for m in messages if m.endswith(".email.json")]
if not msg_files:
print("No messages")
return
first_msg = f"/email/{folder}/{first_date}/{msg_files[0]}"
print(f"=== cat {first_msg} ===")
result = await ws.execute(f"cat {first_msg}")
print((await result.stdout_str())[:500])
print(f"\n=== jq .subject {first_msg} ===")
result = await ws.execute(f'jq ".subject" {first_msg}')
print(await result.stdout_str())
print(f"=== jq .from {first_msg} ===")
result = await ws.execute(f'jq ".from" {first_msg}')
print(await result.stdout_str())
# find: -name at folder level pushes down to IMAP search; -path and
# -size run the local walk (dirs and sizeless entries count as 0, so
# +0c drops them and -1k keeps them).
print(f"=== find /email/{folder}/ -name '*.email.json' | head -n 5 ===")
result = await ws.execute(
f'find /email/{folder}/ -name "*.email.json" | head -n 5')
print(await result.stdout_str())
print(f"=== find /email/{folder}/ -path '*{first_date}*'"
" | head -n 5 ===")
result = await ws.execute(
f'find /email/{folder}/ -path "*{first_date}*" | head -n 5')
print(await result.stdout_str())
print(f"=== find /email/{folder}/ -maxdepth 1 -size +0c"
" (dirs drop out) ===")
result = await ws.execute(f"find /email/{folder}/ -maxdepth 1 -size +0c")
print(f" exit={result.exit_code}")
print(await result.stdout_str())
print("=== email-triage --unseen --max 5 ===")
result = await ws.execute(
f'email-triage --folder {folder} --unseen --max 5')
print((await result.stdout_str())[:500])
print(f"\n=== tree -L 2 /email/{folder}/ ===")
result = await ws.execute(f"tree -L 2 /email/{folder}/")
print((await result.stdout_str())[:500])
# ── native search dispatch (IMAP TEXT search via -r at folder level) ──
for label, cmd in [
(f"grep -r Hi /email/{folder}/ (folder scope, IMAP search)",
f"grep -r Hi /email/{folder}/"),
(f"rg Hi /email/{folder}/ (folder scope)", f"rg Hi /email/{folder}/"),
]:
print(f"\n=== {label} ===")
r = await ws.execute(cmd)
out = (await r.stdout_str()).strip()
err = (await r.stderr_str()).strip()
lines = out.splitlines() if out else []
print(f" exit={r.exit_code} matches: {len(lines)}")
if err:
print(f" stderr: {err[:200]}")
for line in lines[:3]:
print(f" {line[:150]}")
# ── glob expansion: the folder segment is the pattern, the date
# tail keeps walking (lists folders once, then each match's days).
glob_folder = folder[:2] + "*"
print(f"\n=== echo /email/{glob_folder}/2* (mid-path glob) ===")
r = await ws.execute(f"echo /email/{glob_folder}/2*")
out = (await r.stdout_str()).strip()
print(f" {out[:200]}")
assert f"/email/{folder}/2" in out, "mid-path glob did not expand"
# A glob that matches nothing stays the literal word, so the
# command reports it like GNU coreutils.
print("\n=== cat /email/zz-none-*/x.eml (no match) ===")
r = await ws.execute("cat /email/zz-none-*/x.eml")
err = (await r.stderr_str()).strip()
print(f" exit={r.exit_code} {err[:120]}")
assert r.exit_code == 1 and "zz-none-*" in err
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.email import EmailConfig, EmailResource
load_dotenv(".env.development")
config = EmailConfig(
imap_host=os.environ["IMAP_HOST"],
smtp_host=os.environ["SMTP_HOST"],
username=os.environ["EMAIL_USERNAME"],
password=os.environ["EMAIL_PASSWORD"],
max_messages=20,
)
resource = EmailResource(config=config)
with Workspace({"/email/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() folders ---")
folders = os.listdir(mp)
for f in folders:
print(f" {f}")
folder = "Inbox"
if not any("Inbox" in f or "INBOX" in f for f in folders):
folder = folders[0] if folders else ""
folder_path = f"{mp}/{folder}"
if os.path.isdir(folder_path):
print(f"\n--- os.listdir() {folder} (dates) ---")
dates = os.listdir(folder_path)
for d in dates[:5]:
print(f" {d}")
if dates:
date_path = f"{folder_path}/{dates[0]}"
print(f"\n--- os.listdir() {dates[0]} ---")
messages = os.listdir(date_path)
for m in messages[:5]:
print(f" {m}")
json_msgs = [m for m in messages if m.endswith(".email.json")]
if json_msgs:
first = json_msgs[0]
path = f"{date_path}/{first}"
print(f"\n--- open() + read {first[:60]} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" subject: {parsed.get('subject', 'N/A')}")
print(f" from: {parsed.get('from', 'N/A')}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> ls {mp}/{folder}/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
@@ -0,0 +1,87 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.email import EmailConfig, EmailResource
load_dotenv(".env.development")
config = EmailConfig(
imap_host=os.environ["IMAP_HOST"],
smtp_host=os.environ["SMTP_HOST"],
username=os.environ["EMAIL_USERNAME"],
password=os.environ["EMAIL_PASSWORD"],
max_messages=20,
)
resource = EmailResource(config=config)
async def main():
with Workspace({"/email/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE ===\n")
print("--- os.listdir() folders ---")
folders = vos.listdir("/email")
for f in folders:
print(f" {f}")
folder = "Inbox"
if not any("Inbox" in f or "INBOX" in f for f in folders):
folder = folders[0] if folders else ""
if not folder:
print("No folders")
return
print(f"\n--- os.listdir() {folder} dates ---")
dates = vos.listdir(f"/email/{folder}")
for d in dates[:5]:
print(f" {d}")
if dates:
first_date = dates[0]
print(f"\n--- os.listdir() {first_date} messages ---")
messages = vos.listdir(f"/email/{folder}/{first_date}")
for msg in messages[:5]:
print(f" {msg}")
json_msgs = [m for m in messages if m.endswith(".email.json")]
if json_msgs:
first = json_msgs[0]
path = f"/email/{folder}/{first_date}/{first}"
print(f"\n--- open() + read {first[:60]} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" subject: {parsed.get('subject', 'N/A')}")
print(f" from: {parsed.get('from', 'N/A')}")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
nope = f"/email/{folder}/nope.txt"
print(f" nope.txt: {vos.path.exists(nope)}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+180
View File
@@ -0,0 +1,180 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.ram import RAMResource
from mirage.resource.s3 import S3Config, S3Resource
load_dotenv(".env.development")
config = S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)
s3 = S3Resource(config)
mem = RAMResource()
ws = Workspace(
{
"/s3/": s3,
"/work/": (mem, MountMode.WRITE),
},
mode=MountMode.READ,
)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
STDIN_SCRIPT = """\
import sys, json
for line in sys.stdin:
rec = json.loads(line)
print(rec.get("type", "unknown"))
"""
async def main():
print("=== pipe S3 data into python3 -c via stdin ===\n")
print("--- cat example.jsonl | python3 -c (single line) ---")
result = await ws.execute(
'cat /s3/data/example.jsonl | python3 -c "import sys; '
"print(f'lines: {sum(1 for _ in sys.stdin)}')\"")
print(await result.stdout_str())
print(f"Exit code: {result.exit_code}")
print(f"Stats: {ops_summary()}\n")
print("--- head -n 3 example.jsonl | python3 -c (multiline) ---")
result = await ws.execute('head -n 3 /s3/data/example.jsonl | '
'python3 -c "import sys, json\n'
'for line in sys.stdin:\n'
' rec = json.loads(line)\n'
" print(rec.get('type', 'unknown'))\n"
'"')
print(await result.stdout_str())
print(f"Exit code: {result.exit_code}")
print(f"Stats: {ops_summary()}\n")
print("=== pipe S3 data into python3 -c (extract + head) ===\n")
result = await ws.execute('cat /s3/data/example.jsonl | '
'python3 -c "import sys, json\n'
'for line in sys.stdin:\n'
' rec = json.loads(line)\n'
' print(json.dumps(rec)[:80])\n'
'" | head -n 3')
print(await result.stdout_str())
print(f"Exit code: {result.exit_code}")
print(f"Stats: {ops_summary()}\n")
print("=== python3 script file via VFS ===\n")
await ws.execute("mkdir /work/scripts")
await ws.execute(f"echo '{STDIN_SCRIPT}' > /work/scripts/parse_stdin.py")
print("--- head -n 5 example.jsonl | python3 parse_stdin.py ---")
result = await ws.execute("head -n 5 /s3/data/example.jsonl"
" | python3 /work/scripts/parse_stdin.py")
print(await result.stdout_str())
if result.stderr:
print("STDERR:", await result.stderr_str())
print(f"Exit code: {result.exit_code}")
print(f"Stats: {ops_summary()}\n")
print("=== python3 -c: count + aggregate ===\n")
result = await ws.execute('cat /s3/data/example.jsonl | '
'python3 -c "import sys, json\n'
'from collections import Counter\n'
'counts = Counter()\n'
'for line in sys.stdin:\n'
' rec = json.loads(line)\n'
" counts[rec.get('type', 'unknown')] += 1\n"
'for k, v in counts.most_common(5):\n'
" print(f'{k}: {v}')\n"
'"')
print(await result.stdout_str())
print(f"Exit code: {result.exit_code}")
print(f"Stats: {ops_summary()}\n")
print("=== python3 -c with session env ===\n")
await ws.execute("export GREETING=hello_from_mirage")
result = await ws.execute(
'python3 -c "import os; print(os.environ.get(\'GREETING\', \'none\'))"'
)
print(await result.stdout_str())
print(f"Exit code: {result.exit_code}")
print(f"Stats: {ops_summary()}")
# ── heredoc patterns commonly used by AI agents ──
# These exercise the heredoc fixes: dash-strip + quoted delimiter.
print("\n=== python3 << 'PYEOF' (quoted: $X stays literal) ===")
await ws.execute("export X=shellval")
result = await ws.execute("python3 << 'PYEOF'\n"
"x = '$X' # literal, no shell expansion\n"
"print(x)\n"
"PYEOF")
print(f" stdout: {(await result.stdout_str()).strip()} "
f"(expect '$X')")
print("\n=== python3 << PYEOF (unquoted: $X expanded) ===")
result = await ws.execute("python3 << PYEOF\n"
"print('$X')\n"
"PYEOF")
print(f" stdout: {(await result.stdout_str()).strip()} "
f"(expect 'shellval')")
print("\n=== python3 <<-PYEOF (dash: tabs stripped, indented body) ===")
result = await ws.execute("python3 <<-PYEOF\n"
"\tfor i in range(3):\n"
"\t print(f'item-{i}')\n"
"\tPYEOF")
out = (await result.stdout_str()).strip()
print(f" stdout: {out!r} (expect 3 items)")
print("\n=== python3 << EOF | grep keep (heredoc + pipe) ===")
result = await ws.execute("python3 << EOF | grep keep\n"
"for i in range(5):\n"
" print('keep' if i % 2 else 'drop', i)\n"
"EOF")
out = (await result.stdout_str()).strip()
print(f" filtered: {out.splitlines()}")
print("\n=== heredoc in for-loop (body re-fires per iter) ===")
result = await ws.execute(
"for name in alice bob carol; do python3 <<-PYEOF\n"
"\tname = '$name'\n"
"\tprint(f'hello, {name}!')\n"
"\tPYEOF\n"
"done")
for line in (await result.stdout_str()).strip().splitlines():
print(f" {line}")
await ws.close()
asyncio.run(main())
+398
View File
@@ -0,0 +1,398 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import time
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gcs import GCSConfig, GCSResource
load_dotenv(".env.development")
config = GCSConfig(
bucket=os.environ["GCS_BUCKET"],
access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
)
resource = GCSResource(config)
ws = Workspace({"/gcs/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
async def main():
# ── discover structure ────────────────────────────
print("=== ls /gcs/ ===")
r = await ws.execute("ls /gcs/")
print(await r.stdout_str())
print("=== ls /gcs/data/ ===")
r = await ws.execute("ls /gcs/data/")
print(await r.stdout_str())
# ── tree ──────────────────────────────────────────
print("=== tree /gcs/ ===")
r = await ws.execute("tree /gcs/")
print(await r.stdout_str())
# ── stat (directory prefix + file) ──────────────────
print("=== stat /gcs/data (directory prefix) ===")
r = await ws.execute("stat /gcs/data")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== stat /gcs/data/example.json ===")
r = await ws.execute("stat /gcs/data/example.json")
print(f" {(await r.stdout_str()).strip()}")
# ── cat json ──────────────────────────────────────
print("\n=== cat /gcs/data/example.json | head -n 10 ===")
r = await ws.execute("cat /gcs/data/example.json | head -n 10")
print(await r.stdout_str())
# ── head / tail on jsonl ──────────────────────────
print("=== head -n 3 /gcs/data/example.jsonl ===")
r = await ws.execute("head -n 3 /gcs/data/example.jsonl")
print((await r.stdout_str())[:300])
print("\n=== tail -n 2 /gcs/data/example.jsonl ===")
r = await ws.execute("tail -n 2 /gcs/data/example.jsonl")
print((await r.stdout_str())[:300])
# ── wc ────────────────────────────────────────────
print("\n=== wc -l /gcs/data/example.jsonl ===")
r = await ws.execute("wc -l /gcs/data/example.jsonl")
print(f" {(await r.stdout_str()).strip()}")
# ── grep ──────────────────────────────────────────
print("\n=== grep -c mirage /gcs/data/example.jsonl ===")
r = await ws.execute("grep -c mirage /gcs/data/example.jsonl")
print(f" count: {(await r.stdout_str()).strip()}")
print("\n=== grep mirage /gcs/data/example.jsonl | head -n 3 ===")
r = await ws.execute("grep mirage /gcs/data/example.jsonl | head -n 3")
lines = (await r.stdout_str()).strip().splitlines()
for ln in lines:
print(f" {ln[:100]}...")
# ── find ──────────────────────────────────────────
print("\n=== find /gcs/ -name '*.json' ===")
r = await ws.execute("find /gcs/ -name '*.json'")
print(await r.stdout_str())
print("=== find /gcs/ -name '*.parquet' ===")
r = await ws.execute("find /gcs/ -name '*.parquet'")
print(await r.stdout_str())
# ── jq ────────────────────────────────────────────
print("=== jq .metadata /gcs/data/example.json ===")
r = await ws.execute("jq .metadata /gcs/data/example.json")
print(f" {(await r.stdout_str()).strip()[:200]}")
print("\n=== jq '.departments[].teams[].name'"
" /gcs/data/example.json ===")
r = await ws.execute(
'jq ".departments[].teams[].name" /gcs/data/example.json')
print(f" {(await r.stdout_str()).strip()}")
# ── pipelines ─────────────────────────────────────
print("\n=== cat example.jsonl | grep queue-operation"
" | sort | uniq | wc -l ===")
r = await ws.execute("cat /gcs/data/example.jsonl"
" | grep queue-operation | sort | uniq | wc -l")
print(f" unique lines: {(await r.stdout_str()).strip()}")
# ── cd + relative paths ───────────────────────────
print("\n=== pwd ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print('\n=== cd /gcs/data ===')
r = await ws.execute("cd /gcs/data")
print(f" exit={r.exit_code}")
print("\n=== pwd (after cd) ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== ls (relative) ===")
r = await ws.execute("ls")
print(await r.stdout_str())
print("=== head -n 3 example.json (relative) ===")
r = await ws.execute("head -n 3 example.json")
print(await r.stdout_str())
# ── streaming & barrier scenarios ─────────────────
print("\n=== cat | grep | head (streaming drain) ===")
r = await ws.execute("cat /gcs/data/example.jsonl | grep queue | head -n 3"
)
lines = (await r.stdout_str()).strip().splitlines()
print(f" got {len(lines)} lines (expected 3)")
print("\n=== grep -q && echo (barrier VALUE) ===")
r = await ws.execute(
'grep -q queue /gcs/data/example.jsonl && echo "found"')
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== grep -q || echo (barrier OR) ===")
r = await ws.execute('grep -q NONEXISTENT_STRING /gcs/data/example.jsonl'
' || echo "not found"')
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== grep ; grep (semicolon materialization) ===")
r = await ws.execute("grep -c queue /gcs/data/example.jsonl"
"; grep -c mirage /gcs/data/example.jsonl")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== grep missing ; echo $? (semicolon exit code) ===")
r = await ws.execute("grep NONEXISTENT_STRING /gcs/data/example.jsonl"
"; echo $?")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== cat nonexistent 2>&1 | head (stderr in pipe) ===")
r = await ws.execute("cat /gcs/data/nonexistent_file 2>&1 | head -n 1")
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
# ── Step 16: 2>&1 fixes — no double-emit, stream stdout ──
# Without 2>&1: error stays in stderr, stdout empty.
print("\n=== cat nonexistent | cat (no merge: error in stderr) ===")
r = await ws.execute("cat /gcs/data/nonexistent_file | cat")
print(f" stdout: '{(await r.stdout_str()).strip()}' (expect empty)")
print(f" stderr: '{(await r.stderr_str()).strip()[:80]}'")
# With 2>&1: error goes into pipe → reaches final stdout.
# Crucially, final stderr should be empty (no double-emit).
print("\n=== cat nonexistent 2>&1 | cat (no double-emit) ===")
r = await ws.execute("cat /gcs/data/nonexistent_file 2>&1 | cat")
out = (await r.stdout_str()).strip()
err = (await r.stderr_str()).strip()
print(f" stdout: '{out[:80]}'")
print(f" stderr: '{err}' (expect empty: no double-emit)")
# 2>&1 streams stdout chunk-by-chunk through merge.
# Real GCS payload (5766 lines) piped through 2>&1 to wc -l
# should match wc -l on the file directly.
print("\n=== cat large 2>&1 | wc -l (streams real payload) ===")
r = await ws.execute("wc -l /gcs/data/example.jsonl")
expected = int((await r.stdout_str()).strip().split()[0])
r = await ws.execute("cat /gcs/data/example.jsonl 2>&1 | wc -l")
got = int((await r.stdout_str()).strip())
print(f" expected: {expected} got: {got} "
f"{'OK' if got == expected else 'MISMATCH'}")
print("\n=== cat | sort | uniq | wc -l (full pipeline) ===")
r = await ws.execute("cat /gcs/data/example.jsonl"
" | sort | uniq | wc -l")
print(f" unique lines: {(await r.stdout_str()).strip()}")
# ── background job scenarios ────────────────────
print("\n=== grep -c & echo kicked off; wait (bg job) ===")
r = await ws.execute("grep -c queue /gcs/data/example.jsonl &"
" echo 'kicked off'; wait")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== sleep 0 & cat (bg doesn't consume stdin) ===")
r = await ws.execute("sleep 0 & cat /gcs/data/example.json | head -n 1")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== cat nonexistent & echo ok (bg error handled) ===")
r = await ws.execute(
"cat /gcs/data/nonexistent_file & echo ok; wait; echo done")
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== multiple bg: grep & wc & wait (parallel) ===")
r = await ws.execute("grep -c queue /gcs/data/example.jsonl &"
" wc -l /gcs/data/example.jsonl &"
" wait; echo all done")
print(f" stdout: {(await r.stdout_str()).strip()}")
# ── lazy stdin in loops (Step 15) ────────────────
# Functional checks. True laziness is measured by the unit test
# (test_while_read_break_stops_pulling) — it counts producer pulls.
# End-to-end over the network these are just correctness checks.
# Bounded loop (head limits upstream → exact iter count).
# With lazy stdin, `head -n 5` triggers EOF after 5 lines, the
# while loop sees readline()==None, exits cleanly.
print("\n=== head -n 5 | while read; do echo (bounded loop) ===")
r = await ws.execute("cat /gcs/data/example.jsonl | head -n 5"
" | while read LINE; do echo got; done | wc -l")
print(f" iterations: {(await r.stdout_str()).strip()} (expected 5)")
# Early break: only one iter, rest of upstream untouched.
# Visible via unit test `test_while_read_break_stops_pulling`;
# here we only check stdout shape.
print("\n=== while read; break (early exit) ===")
r = await ws.execute("cat /gcs/data/example.jsonl | head -n 100"
" | while read LINE; do echo first; break; done")
out = (await r.stdout_str()).strip().splitlines()
print(f" stdout lines: {len(out)} (expected 1) exit={r.exit_code}")
# for-loop over fixed value list, with stdin available to body.
# Each iter calls `read` once → consumes one line from buffer.
print("\n=== for x in a b c; do read LINE (loop reads buffer) ===")
r = await ws.execute(
"cat /gcs/data/example.jsonl | head -n 3"
" | for x in a b c; do read LINE; echo \"$x:${LINE:0:30}\"; done")
for line in (await r.stdout_str()).strip().splitlines():
print(f" {line}")
# Note: while-loop over an unbounded stream is currently capped
# at _MAX_WHILE=10000 iterations regardless of input size. This is
# a separate safety limit, not a laziness issue.
print("\n (note: while over unbounded stream caps at 10000 iters)")
# ── quoting / escaping patterns commonly used by AI agents ──
# These exercise the bash double-quote escape rules + variable
# expansion semantics the agent relies on.
print("\n=== echo \"\\$X\" (escaped dollar stays literal) ===")
await ws.execute("export X=expanded")
r = await ws.execute('echo "\\$X"')
print(f" stdout: {json.dumps((await r.stdout_str()).strip())}"
" (expect '$X')")
print("\n=== echo \"$X\" (unescaped dollar expands) ===")
r = await ws.execute('echo "$X"')
print(f" stdout: {json.dumps((await r.stdout_str()).strip())}"
" (expect 'expanded')")
print("\n=== echo '$X' (single quotes keep $X literal) ===")
r = await ws.execute("echo '$X'")
print(f" stdout: {json.dumps((await r.stdout_str()).strip())}"
" (expect '$X')")
print("\n=== cat \"$DIR/example.json\" (env var in path) ===")
await ws.execute("export DIR=/gcs/data")
r = await ws.execute('cat "$DIR/example.json" | head -n 3')
out = (await r.stdout_str()).strip().splitlines()
print(f" first lines: {json.dumps(out, separators=(',', ':'))}")
print("\n=== cat $(echo /gcs/data/example.json) | head -n 1"
" (command sub as path) ===")
r = await ws.execute("cat $(echo /gcs/data/example.json) | head -n 1")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== grep \"$(echo queue)\" /gcs/data/example.jsonl"
" | wc -l (sub as pattern) ===")
r = await ws.execute('grep "$(echo queue)" /gcs/data/example.jsonl | wc -l'
)
print(f" count: {(await r.stdout_str()).strip()}")
# ── on-the-fly max_drain_bytes (cancellable cache drain) ──────────
# Demonstrates: set a small budget → cat | head leaves the source
# only partially read → background drain trips the budget → cache
# is NOT populated. Then unset the budget → drain completes → cache
# IS populated. Drain tasks are awaited explicitly (vs sleeping)
# so we don't race the network.
target = "/gcs/data/example.jsonl"
await ws.cache.clear()
print("\n=== max_drain_bytes=1MB then cat | head (drain cancelled) ===")
ws.max_drain_bytes = 1_000_000
r = await ws.execute(f"cat {target} | head -n 3")
print(f" head returned {len((await r.stdout_str()).splitlines())} lines")
drain_keys = list(ws.cache._drain_tasks.keys())
print(f" drain tasks: {json.dumps(drain_keys, separators=(',', ':'))}")
for k in drain_keys:
await ws.cache._drain_tasks[k]
cached = None
for k in drain_keys:
cached = await ws.cache.get(k) or cached
if cached:
status = f"POPULATED ({len(cached)} bytes)"
else:
status = "EMPTY (drain cancelled, as expected)"
print(f" cache after small budget: {status}")
await ws.cache.clear()
print("\n=== max_drain_bytes=None then cat | head (drain completes) ===")
ws.max_drain_bytes = None
r = await ws.execute(f"cat {target} | head -n 3")
print(f" head returned {len((await r.stdout_str()).splitlines())} lines")
drain_keys = list(ws.cache._drain_tasks.keys())
print(f" drain tasks: {json.dumps(drain_keys, separators=(',', ':'))}")
for k in drain_keys:
await ws.cache._drain_tasks[k]
cached = None
for k in drain_keys:
cached = await ws.cache.get(k) or cached
if cached:
status = f"POPULATED ({len(cached)} bytes, as expected)"
else:
status = "EMPTY"
print(f" cache after unbounded: {status}")
# ── chunk-level streaming + multi-stage pipe backpressure ─────────
print("\n=== STREAMING (single command) ===")
target = "/gcs/data/example.jsonl"
r = await ws.execute(f"stat -c '%s' {target}")
size = int((await r.stdout_str()).strip())
print(f" object size: {size:,} bytes")
async def measure(label: str, cmd: str) -> None:
before = sum(rec.bytes for rec in ws.ops.records)
t0 = time.monotonic()
r = await ws.execute(cmd)
dt = time.monotonic() - t0
net = sum(rec.bytes for rec in ws.ops.records) - before
head = (await r.stdout_str()).strip().splitlines()
first = head[0][:48] if head else ""
print(f" {label:42s} bytes={net:>10,} t={dt:4.2f}s "
f"lines={len(head):>4} out0={json.dumps(first)}")
await ws.cache.clear()
await measure("head -n 1 (line-streamed)", f"head -n 1 {target}")
await ws.cache.clear()
await measure("head -c 100 (byte-range)", f"head -c 100 {target}")
await ws.cache.clear()
await measure("grep -m 1 (early-exit)", f"grep -m 1 mirage {target}")
print("\n=== STREAMING CHAIN (multi-stage pipe backpressure) ===")
await ws.cache.clear()
await measure("cat | head -n 1", f"cat {target} | head -n 1")
await ws.cache.clear()
await measure("cat | tr A-Z a-z | head -n 1",
f"cat {target} | tr A-Z a-z | head -n 1")
await ws.cache.clear()
await measure("cat | grep mirage | head -n 1",
f"cat {target} | grep mirage | head -n 1")
await ws.cache.clear()
await measure("4-stage: cat|tr|grep|head -n 1",
f"cat {target} | tr A-Z a-z | grep mirage | head -n 1")
await ws.cache.clear()
await measure(
"5-stage: cat|tr|grep|head|wc -l",
f"cat {target} | tr A-Z a-z | grep mirage | head -n 1 "
"| wc -l")
await ws.cache.clear()
await measure("non-cancellable: cat | wc -l", f"cat {target} | wc -l")
print(f"\nStats: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
+72
View File
@@ -0,0 +1,72 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.gcs import GCSConfig, GCSResource
load_dotenv(".env.development")
config = GCSConfig(
bucket=os.environ["GCS_BUCKET"],
access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
)
resource = GCSResource(config)
with Workspace({"/gcs/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() ---")
entries = os.listdir(f"{mp}/data")
for e in entries:
print(f" {e}")
print("\n--- open() + read example.json (first 5 lines) ---")
with open(f"{mp}/data/example.json") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" {line.rstrip()[:120]}")
print("\n--- open() + read example.jsonl (first 3 lines) ---")
with open(f"{mp}/data/example.jsonl") as f:
for i, line in enumerate(f):
if i >= 3:
break
rec = json.loads(line)
print(f" [{i}] {json.dumps(rec)[:100]}...")
print("\n--- os.path.getsize() ---")
size = os.path.getsize(f"{mp}/data/example.json")
print(f" example.json: {size} bytes")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/data/")
print(f">>> cat {mp}/data/example.json")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes")
+89
View File
@@ -0,0 +1,89 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gcs import GCSConfig, GCSResource
load_dotenv(".env.development")
config = GCSConfig(
bucket=os.environ["GCS_BUCKET"],
access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
)
resource = GCSResource(config)
async def main():
with Workspace({"/gcs/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE: open() reads from GCS transparently ===\n")
print("--- os.listdir() root ---")
root = vos.listdir("/gcs")
for e in root:
print(f" {e}")
print("\n--- os.path.isdir() on prefix ---")
print(f" /gcs/data: {vos.path.isdir('/gcs/data')}")
print("\n--- os.listdir() data ---")
entries = vos.listdir("/gcs/data")
for e in entries:
print(f" {e}")
print("\n--- open() + read example.json (first 5 lines) ---")
with open("/gcs/data/example.json") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" {line.rstrip()[:120]}")
print("\n--- open() + read example.jsonl (first 3 lines) ---")
with open("/gcs/data/example.jsonl") as f:
for i, line in enumerate(f):
if i >= 3:
break
rec = json.loads(line)
print(f" [{i}] {json.dumps(rec)[:100]}...")
print("\n--- os.path.exists() ---")
print(f" example.json: {vos.path.exists('/gcs/data/example.json')}")
print(f" nonexistent: {vos.path.exists('/gcs/data/nope.txt')}")
print("\n--- VFS commands ---")
result = await ws.execute("grep -c mirage /gcs/data/example.jsonl")
print(f" grep matches: {(await result.stdout_str()).strip()}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+146
View File
@@ -0,0 +1,146 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
load_dotenv(".env.development")
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
async def main() -> None:
ws = Workspace({"/gdocs": resource}, mode=MountMode.WRITE)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /gdocs/__nf_missing__.txt",
"head /gdocs/__nf_missing__.txt",
"stat /gdocs/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=== ls /gdocs/ ===")
r = await ws.execute("ls /gdocs/")
print(await r.stdout_str())
print("=== ls /gdocs/owned/ (first 5) ===")
r = await ws.execute("ls /gdocs/owned/ | head -n 5")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== cat ===")
r = await ws.execute(f"cat /gdocs/owned/{first}")
print((await r.stdout_str())[:300])
print("\n=== head -n 20 ===")
r = await ws.execute(f"head -n 20 /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== tail -n 10 ===")
r = await ws.execute(f"tail -n 10 /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== wc ===")
r = await ws.execute(f"wc /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== stat ===")
r = await ws.execute(f"stat /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== jq .title ===")
r = await ws.execute(f'jq ".title" /gdocs/owned/{first}')
print(await r.stdout_str())
print("=== nl ===")
r = await ws.execute(f"nl /gdocs/owned/{first} | head -n 10")
print(await r.stdout_str())
print("=== tree /gdocs/ ===")
r = await ws.execute("tree /gdocs/")
print((await r.stdout_str())[:500])
print("\n=== find /gdocs/owned/ ===")
r = await ws.execute("find /gdocs/owned/ -name '*.gdoc.json' | head -n 5")
print(await r.stdout_str())
print("=== grep textRun ===")
r = await ws.execute(f"grep textRun /gdocs/owned/{first} | head -c 200")
print(await r.stdout_str())
print("\n=== rg textRun ===")
r = await ws.execute(f"rg textRun /gdocs/owned/{first} | head -c 200")
print(await r.stdout_str())
print("\n=== basename ===")
r = await ws.execute(f"basename /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== dirname ===")
r = await ws.execute(f"dirname /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== realpath ===")
r = await ws.execute(f"realpath /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== gws-docs-documents-create ===")
r = await ws.execute('gws-docs-documents-create'
' --json \'{"title": "MIRAGE Example Doc"}\'')
doc = json.loads(await r.stdout_str())
doc_id = doc["documentId"]
print(f"Created: {doc_id}")
print("\n=== gws-docs-documents-batchUpdate ===")
body = json.dumps({
"requests": [{
"insertText": {
"location": {
"index": 1
},
"text": "Hello from MIRAGE!\n",
}
}]
})
params = json.dumps({"documentId": doc_id})
r = await ws.execute(f"gws-docs-documents-batchUpdate"
f" --params '{params}' --json '{body}'")
print(f"Updated: {(await r.stdout_str())[:80]}")
print("\n=== gws-docs-write ===")
r = await ws.execute(f'gws-docs-write'
f' --document {doc_id}'
f' --text "Appended via gws-docs-write."')
print(f"Written: {(await r.stdout_str())[:80]}")
url = f"https://docs.google.com/document/d/{doc_id}/edit"
print(f"\nOpen: {url}")
if __name__ == "__main__":
asyncio.run(main())
+72
View File
@@ -0,0 +1,72 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
load_dotenv(".env.development")
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
with Workspace({"/gdocs/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() root ---")
roots = os.listdir(mp)
for r in roots:
print(f" {r}")
for section in ("owned", "shared"):
section_path = f"{mp}/{section}"
if not os.path.isdir(section_path):
continue
docs = os.listdir(section_path)
if not docs:
continue
print(f"\n--- os.listdir() {section} (first 5) ---")
for d in docs[:5]:
print(f" {d}")
first = docs[0]
path = f"{section_path}/{first}"
print(f"\n--- open() + read {first[:60]} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" title: {parsed.get('title', 'N/A')}")
break
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> ls {mp}/owned/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+78
View File
@@ -0,0 +1,78 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
load_dotenv(".env.development")
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
async def main() -> None:
with Workspace({"/gdocs/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Docs transparently ===\n")
print("--- os.listdir() root ---")
dirs = vos.listdir("/gdocs")
for d in dirs:
print(f" {d}")
print("\n--- os.listdir() owned ---")
docs = vos.listdir("/gdocs/owned")
for doc in docs[:5]:
print(f" {doc}")
if docs:
first = docs[0]
path = f"/gdocs/owned/{first}"
print("\n--- open() + read first doc ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" title: {parsed.get('title', 'N/A')}")
print(f" content preview: {content[:200]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gdocs/owned/nope.json')}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+148
View File
@@ -0,0 +1,148 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
async def main() -> None:
ws = Workspace({"/gdrive": resource}, mode=MountMode.WRITE)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /gdrive/__nf_missing__.txt",
"head /gdrive/__nf_missing__.txt",
"stat /gdrive/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=== ls /gdrive/ ===")
result = await ws.execute("ls /gdrive/")
print(await result.stdout_str())
entries = (await result.stdout_str()).strip().splitlines()
if not entries:
print("No files")
return
first = entries[0]
print(f"=== stat /gdrive/{first} ===")
result = await ws.execute(f'stat "/gdrive/{first}"')
print(await result.stdout_str())
if first.endswith("/"):
print(f"=== ls /gdrive/{first} ===")
result = await ws.execute(f'ls "/gdrive/{first}"')
print(await result.stdout_str())
sub_entries = (await result.stdout_str()).strip().splitlines()
if sub_entries:
sub = sub_entries[0]
if not sub.endswith("/"):
print(f"=== cat /gdrive/{first}{sub} ===")
result = await ws.execute(f'cat "/gdrive/{first}{sub}"')
print((await result.stdout_str())[:500])
print("=== tree -L 1 /gdrive/ ===")
result = await ws.execute("tree -L 1 /gdrive/")
print(await result.stdout_str())
print("=== find /gdrive/ -name '*.gdoc.json' | head -n 5 ===")
result = await ws.execute("find /gdrive/ -name '*.gdoc.json' | head -n 5")
print(await result.stdout_str())
gdoc_files = (await result.stdout_str()).strip().splitlines()
if gdoc_files:
gdoc = gdoc_files[0]
print(f"=== cat {gdoc} | jq .title ===")
result = await ws.execute(f'cat "{gdoc}" | jq ".title"')
print(await result.stdout_str())
print(f"=== head -n 3 {gdoc} ===")
result = await ws.execute(f'head -n 3 "{gdoc}"')
print(await result.stdout_str())
print(f"=== wc {gdoc} ===")
result = await ws.execute(f'wc "{gdoc}"')
print(await result.stdout_str())
print(f"=== basename {gdoc} ===")
result = await ws.execute(f'basename "{gdoc}"')
print(await result.stdout_str())
print(f"=== dirname {gdoc} ===")
result = await ws.execute(f'dirname "{gdoc}"')
print(await result.stdout_str())
print(f"=== tail -n 3 {gdoc} ===")
result = await ws.execute(f'tail -n 3 "{gdoc}"')
print(await result.stdout_str())
print(f"=== nl {gdoc} ===")
result = await ws.execute(f'nl "{gdoc}"')
print((await result.stdout_str())[:300])
print(f"=== grep title {gdoc} ===")
result = await ws.execute(f'grep title "{gdoc}"')
print((await result.stdout_str())[:300])
print(f"=== rg title {gdoc} ===")
result = await ws.execute(f'rg title "{gdoc}"')
print((await result.stdout_str())[:300])
print(f"=== cut -c 1-40 {gdoc} ===")
result = await ws.execute(f'cut -c 1-40 "{gdoc}"')
print((await result.stdout_str())[:300])
print(f"=== sed -n 2p {gdoc} ===")
result = await ws.execute(f'sed -n 2p "{gdoc}"')
print((await result.stdout_str())[:300])
print(f"=== sed s/title/TITLE/g {gdoc} ===")
result = await ws.execute(f'sed "s/title/TITLE/g" "{gdoc}"')
print((await result.stdout_str())[:300])
print(f"=== realpath {gdoc} ===")
result = await ws.execute(f'realpath "{gdoc}"')
print(await result.stdout_str())
print("=== gws-docs-documents-create ===")
result = await ws.execute(
'gws-docs-documents-create'
' --json \'{"title": "Test from MIRAGE gdrive"}\'')
print((await result.stdout_str())[:300])
print("=== gws-sheets-spreadsheets-create ===")
result = await ws.execute(
'gws-sheets-spreadsheets-create'
' --json \'{"properties": {"title": "Test Sheet from gdrive"}}\'')
print((await result.stdout_str())[:300])
if __name__ == "__main__":
asyncio.run(main())
+59
View File
@@ -0,0 +1,59 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
with Workspace({"/gdrive/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() root ---")
entries = os.listdir(mp)
for e in entries[:10]:
print(f" {e}")
for e in entries:
path = f"{mp}/{e}"
if os.path.isfile(path):
print(f"\n--- reading {e} ---")
with open(path, "rb") as f:
data = f.read(1024)
print(f" {len(data)} bytes (first 1024)")
break
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+68
View File
@@ -0,0 +1,68 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
async def main():
with Workspace({"/gdrive/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE ===\n")
print("--- os.listdir() root ---")
entries = vos.listdir("/gdrive")
for e in entries[:10]:
print(f" {e}")
if entries:
first = entries[0]
path = f"/gdrive/{first}"
print(f"\n--- os.path.isdir({first}) ---")
print(f" {vos.path.isdir(path)}")
if vos.path.isfile(path):
print(f"\n--- open() + read {first} ---")
with open(path) as f:
content = f.read()
print(f" {len(content)} bytes")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+335
View File
@@ -0,0 +1,335 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.github import GitHubConfig, GitHubResource
load_dotenv(".env.development")
config = GitHubConfig(token=os.environ["GITHUB_TOKEN"])
async def _timed(ws, cmd):
start = time.perf_counter()
out = await (await ws.execute(cmd)).stdout_str()
return (time.perf_counter() - start) * 1000, out
async def main() -> None:
resource = GitHubResource(
config=config,
owner="strukto-ai",
repo="mirage",
ref="main",
)
ws = Workspace({"/github": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /github/__nf_missing__.txt",
"head /github/__nf_missing__.txt",
"stat /github/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
r = await ws.execute("ls /github")
print(await r.stdout_str())
r = await ws.execute("ls /github/python/mirage/core")
print(await r.stdout_str())
r = await ws.execute("cat /github/python/pyproject.toml")
print(await r.stdout_str())
r = await ws.execute(
"grep 'BaseResource' /github/python/mirage/resource/base.py")
print(await r.stdout_str())
r = await ws.execute("grep 'import' /github/python/mirage/*")
print(await r.stdout_str())
r = await ws.execute("grep 'import' /github/python/mirage/core/s3/*.py")
print(await r.stdout_str())
r = await ws.execute("grep -r 'async def' /github/python/mirage/core/s3/")
print(await r.stdout_str())
r = await ws.execute("find /github/mirage -name '*.py'")
print(await r.stdout_str())
r = await ws.execute("stat /github/python/mirage/types.py")
print(await r.stdout_str())
r = await ws.execute("du /github/python/mirage/core")
print(await r.stdout_str())
print("=== head -n 5 ===")
r = await ws.execute("head -n 5 /github/python/pyproject.toml")
print(await r.stdout_str())
print("=== tail -n 3 ===")
r = await ws.execute("tail -n 3 /github/python/pyproject.toml")
print(await r.stdout_str())
print("=== wc ===")
r = await ws.execute("wc /github/python/pyproject.toml")
print(await r.stdout_str())
print("=== wc -l ===")
r = await ws.execute("wc -l /github/python/pyproject.toml")
print(await r.stdout_str())
print("=== grep -n (line numbers) ===")
r = await ws.execute("grep -n 'def ' /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== grep -c (count) ===")
r = await ws.execute("grep -c 'import' /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== grep -i (case insensitive) ===")
r = await ws.execute("grep -i 'filestat' /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== grep -l (files with matches) ===")
r = await ws.execute(
"grep -rl 'BaseResource' /github/python/mirage/resource/")
print(await r.stdout_str())
# ── native search dispatch (GitHub code search narrows files) ──
s3_dir = "/github/python/mirage/core/s3/"
for label, cmd in [
(f"grep -r mirage {s3_dir} (narrows via search.code)",
f"grep -r mirage {s3_dir}"),
(f"grep -r FileType {s3_dir} (recursive scope)",
f"grep -r FileType {s3_dir}"),
(f"rg mirage {s3_dir} (rg recursive scope)", f"rg mirage {s3_dir}"),
("grep -r GitHubAccessor /github/ (repo-root search narrowing)",
"grep -r GitHubAccessor /github/ | sort"),
]:
print(f"\n=== {label} ===")
r = await ws.execute(cmd)
out = (await r.stdout_str()).strip()
err = (await r.stderr_str()).strip()
lines = out.splitlines() if out else []
print(f" exit={r.exit_code} matches: {len(lines)}")
if err:
print(f" stderr: {err[:200]}")
for line in lines[:3]:
print(f" {line[:150]}")
# ── subdir + regex narrowing + -l short-circuit (issue #404) ──
# A large subdir (>100 files) is what makes the per-file fallback slow;
# these cases narrow via GitHub code search instead of fetching each file.
big_dir = "/github/python/mirage/"
print(f"\n=== grep -rln BaseResource {big_dir} "
"(subdir narrowing, -l short-circuit) ===")
ms, out = await _timed(ws, f"grep -rln BaseResource {big_dir}")
files = out.strip().splitlines() if out.strip() else []
print(f" {ms:.0f}ms files-with-matches: {len(files)}")
for line in files[:3]:
print(f" {line}")
print(f"\n=== grep -rn 'async def .*self' {big_dir} "
"(regex narrows via required literal 'async def ') ===")
ms, out = await _timed(ws, f"grep -rn 'async def .*self' {big_dir}")
lines = out.strip().splitlines() if out.strip() else []
print(f" {ms:.0f}ms matches: {len(lines)}")
for line in lines[:3]:
print(f" {line[:150]}")
print(f"\n=== rg -l GitHubAccessor {big_dir} "
"(rg subdir narrowing, -l short-circuit) ===")
ms, out = await _timed(ws, f"rg -l GitHubAccessor {big_dir}")
files = out.strip().splitlines() if out.strip() else []
print(f" {ms:.0f}ms files-with-matches: {len(files)}")
for line in files[:3]:
print(f" {line}")
print(f"\n=== rg -l --glob '*.py' GitHubAccessor {big_dir} "
"(file filter applied to narrowed set) ===")
ms, out = await _timed(ws, f"rg -l --glob '*.py' GitHubAccessor {big_dir}")
files = out.strip().splitlines() if out.strip() else []
print(f" {ms:.0f}ms files-with-matches: {len(files)}")
for line in files[:3]:
print(f" {line}")
print(f"\n=== rg -l --type py GitHubAccessor {big_dir} "
"(--type filter applied to narrowed set) ===")
ms, out = await _timed(ws, f"rg -l --type py GitHubAccessor {big_dir}")
files = out.strip().splitlines() if out.strip() else []
print(f" {ms:.0f}ms files-with-matches: {len(files)}")
for line in files[:3]:
print(f" {line}")
print("=== find -type d ===")
r = await ws.execute("find /github/python/mirage/core -type d")
print(await r.stdout_str())
print("=== ls -l ===")
r = await ws.execute("ls -l /github/python/mirage/core/s3/")
print(await r.stdout_str())
print("=== find | sort ===")
r = await ws.execute(
"find /github/python/mirage/core/s3 -name '*.py' | sort")
print(await r.stdout_str())
print("=== diff ===")
r = await ws.execute("diff /github/python/mirage/core/s3/stat.py"
" /github/python/mirage/core/s3/read.py")
print(await r.stdout_str())
print("=== cat + pipe to wc ===")
r = await ws.execute("cat /github/python/mirage/types.py | wc -l")
print(await r.stdout_str())
print("=== grep + cut ===")
r = await ws.execute(
"grep -n 'class ' /github/python/mirage/types.py | cut -d: -f1")
print(await r.stdout_str())
print("=== grep + awk ===")
r = await ws.execute(
"grep 'class ' /github/python/mirage/types.py | awk '{print $2}'")
print(await r.stdout_str())
print("=== md5 ===")
r = await ws.execute("md5 /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== tree ===")
r = await ws.execute("tree /github/python/mirage/core/s3/")
print(await r.stdout_str())
print("=== find workspace.py ===")
r = await ws.execute("find /github -name 'workspace.py'")
print(await r.stdout_str())
print("=== wc -l (lines) ===")
r = await ws.execute("wc -l /github/python/mirage/workspace/workspace.py")
print(await r.stdout_str())
print("=== wc -w (words) ===")
r = await ws.execute("wc -w /github/python/mirage/workspace/workspace.py")
print(await r.stdout_str())
print("=== jq ===")
r = await ws.execute('jq ".name" /github/python/pyproject.toml')
print(await r.stdout_str())
print("=== nl ===")
r = await ws.execute("nl /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== tr ===")
r = await ws.execute("cat /github/python/mirage/types.py | tr 'a-z' 'A-Z'")
print(await r.stdout_str())
print("=== sort | uniq ===")
r = await ws.execute(
"grep 'import' /github/python/mirage/types.py | sort | uniq")
print(await r.stdout_str())
print("=== uniq (file path, streams via github read) ===")
r = await ws.execute("uniq /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== sha256sum ===")
r = await ws.execute("sha256sum /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== file ===")
r = await ws.execute("file /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== basename ===")
r = await ws.execute("basename /github/python/mirage/core/s3/read.py")
print(await r.stdout_str())
print("=== dirname ===")
r = await ws.execute("dirname /github/python/mirage/core/s3/read.py")
print(await r.stdout_str())
print("=== realpath ===")
r = await ws.execute("realpath /github/python/mirage/../mirage/types.py")
print(await r.stdout_str())
print("=== sed -n (line range) ===")
r = await ws.execute("sed -n '1,3p' /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== sed s/// (file) ===")
r = await ws.execute(
"sed 's/import/IMPORT/' /github/python/mirage/core/s3/read.py")
print(await r.stdout_str())
print("=== awk (file) ===")
r = await ws.execute(
"awk '{print $1}' /github/python/mirage/core/s3/read.py")
print(await r.stdout_str())
print("=== cut -c (file) ===")
r = await ws.execute("cut -c1-10 /github/python/mirage/types.py")
print(await r.stdout_str())
print("=== grep dir operands (POSIX warn) ===")
r = await ws.execute("grep 'import' /github/python/mirage/*")
out = (await r.stdout_str()).strip()
err = (await r.stderr_str()).strip()
print(
f" exit={r.exit_code} matches: {len(out.splitlines()) if out else 0}")
for line in err.splitlines()[:3]:
print(f" {line}")
print()
print("=== diff -u ===")
r = await ws.execute("diff -u /github/python/mirage/core/s3/stat.py"
" /github/python/mirage/core/s3/read.py")
print(await r.stdout_str())
print("=== tree -L ===")
r = await ws.execute("tree -L 2 /github/python/mirage/")
print(await r.stdout_str())
print("=== rg ===")
r = await ws.execute("rg 'BaseResource' /github/python/mirage/resource/")
print(await r.stdout_str())
print(
"=== caching: a warm read is served from cache (no backend fetch) ===")
cache_file = "/github/python/mirage/workspace/workspace.py"
cold_ms, body = await _timed(ws, f"cat {cache_file}")
warm_ms, _ = await _timed(ws, f"cat {cache_file}")
grep_ms, _ = await _timed(ws, f"grep 'def ' {cache_file}")
head_ms, _ = await _timed(ws, f"head -n 5 {cache_file}")
tail_ms, _ = await _timed(ws, f"tail -n 5 {cache_file}")
wc_ms, _ = await _timed(ws, f"wc -l {cache_file}")
print(f" file={cache_file} size={len(body)}B")
print(f" cold cat={cold_ms:.0f}ms warm cat={warm_ms:.0f}ms "
f"grep={grep_ms:.0f}ms head={head_ms:.0f}ms tail={tail_ms:.0f}ms "
f"wc={wc_ms:.0f}ms")
print(f" served_from_cache={warm_ms < cold_ms / 5} "
f"(warm speedup {cold_ms / max(warm_ms, 0.001):.0f}x)")
if __name__ == "__main__":
asyncio.run(main())
+81
View File
@@ -0,0 +1,81 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.github import GitHubConfig, GitHubResource
load_dotenv(".env.development")
config = GitHubConfig(token=os.environ["GITHUB_TOKEN"])
resource = GitHubResource(
config=config,
owner="strukto-ai",
repo="mirage",
ref="main",
)
with Workspace({"/github/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() root ---")
entries = os.listdir(mp)
for e in entries[:10]:
print(f" {e}")
if len(entries) > 10:
print(f" ... ({len(entries)} total)")
print("\n--- os.listdir() python/mirage/ ---")
core = os.listdir(f"{mp}/python/mirage")
for c in core[:10]:
print(f" {c}")
print("\n--- os.listdir() python/mirage/core/ ---")
core_dirs = os.listdir(f"{mp}/python/mirage/core")
for d in core_dirs[:10]:
print(f" {d}")
if len(core_dirs) > 10:
print(f" ... ({len(core_dirs)} total)")
print("\n--- open() + read python/pyproject.toml (first 5 lines) ---")
with open(f"{mp}/python/pyproject.toml") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" {line.rstrip()}")
print("\n--- open() + read python/mirage/types.py (first 5 lines) ---")
with open(f"{mp}/python/mirage/types.py") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" {line.rstrip()}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> cat {mp}/README.md")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes")
+90
View File
@@ -0,0 +1,90 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.github import GitHubConfig, GitHubResource
load_dotenv(".env.development")
config = GitHubConfig(token=os.environ["GITHUB_TOKEN"])
async def main():
resource = GitHubResource(
config=config,
owner="strukto-ai",
repo="mirage",
ref="main",
)
with Workspace({"/github/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE: open() reads from GitHub transparently ===\n")
print("--- os.listdir() root ---")
entries = vos.listdir("/github")
for e in entries[:10]:
print(f" {e}")
if len(entries) > 10:
print(f" ... ({len(entries)} total)")
print("\n--- os.listdir() mirage/ ---")
core = vos.listdir("/github/python/mirage")
for c in core[:10]:
print(f" {c}")
print("\n--- os.listdir() mirage/core/ ---")
core_dirs = vos.listdir("/github/python/mirage/core")
for d in core_dirs[:10]:
print(f" {d}")
if len(core_dirs) > 10:
print(f" ... ({len(core_dirs)} total)")
print("\n--- open() + read pyproject.toml (first 5 lines) ---")
with open("/github/python/pyproject.toml") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" {line.rstrip()}")
print("\n--- open() + read mirage/types.py (first 5 lines) ---")
with open("/github/python/mirage/types.py") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" {line.rstrip()}")
print("\n--- os.path.isdir() checks ---")
core_isdir = vos.path.isdir("/github/python/mirage/core")
print(f" /github/python/mirage/core: {core_isdir}")
is_dir = vos.path.isdir("/github/python/pyproject.toml")
print(f" /github/python/pyproject.toml: {is_dir}")
print("\n--- os.path.isfile() checks ---")
is_file = vos.path.isfile("/github/python/pyproject.toml")
print(f" /github/python/pyproject.toml: {is_file}")
core_isfile = vos.path.isfile("/github/python/mirage/core")
print(f" /github/python/mirage/core: {core_isfile}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+232
View File
@@ -0,0 +1,232 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.github_ci import GitHubCIConfig, GitHubCIResource
load_dotenv(".env.development")
config = GitHubCIConfig(
token=os.environ["GITHUB_TOKEN"],
owner="strukto-ai",
repo="mirage",
max_runs=300,
)
resource = GitHubCIResource(config=config)
async def main():
ws = Workspace({"/ci": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /ci/__nf_missing__.txt", "head /ci/__nf_missing__.txt",
"stat /ci/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── discover structure ────────────────────────────
print("=== ls /ci/ (root) ===")
r = await ws.execute("ls /ci/")
print(await r.stdout_str())
# ── list workflows ────────────────────────────────
print("=== ls /ci/workflows/ ===")
r = await ws.execute("ls /ci/workflows/")
print(await r.stdout_str())
workflows = (await r.stdout_str()).strip().splitlines()
if not workflows or not workflows[0]:
print("no workflows found")
return
wf_name = workflows[0].strip()
# ── read workflow metadata ────────────────────────
print(f"=== cat /ci/workflows/{wf_name} ===")
r = await ws.execute(f'cat "/ci/workflows/{wf_name}"')
print((await r.stdout_str())[:500])
# ── list runs ─────────────────────────────────────
print("\n=== ls /ci/runs/ ===")
r = await ws.execute("ls /ci/runs/")
print(await r.stdout_str())
print("\n=== ls -l /ci/runs/ (mtime from updated_at) ===")
long_runs = await ws.execute("ls -l /ci/runs/ | head -n 5")
print(await long_runs.stdout_str())
runs = (await r.stdout_str()).strip().splitlines()
if not runs or not runs[0]:
print("no runs found")
return
run_name = runs[0].strip()
run_path = f"/ci/runs/{run_name}"
# ── list run contents ─────────────────────────────
print(f"=== ls {run_path}/ ===")
r = await ws.execute(f'ls "{run_path}/"')
print(await r.stdout_str())
# ── read run metadata ─────────────────────────────
print(f"=== cat {run_path}/run.json | head -n 20 ===")
r = await ws.execute(f'cat "{run_path}/run.json" | head -n 20')
print(await r.stdout_str())
# ── stat on the run ───────────────────────────────
print(f"=== stat {run_path} ===")
r = await ws.execute(f'stat "{run_path}"')
print(f" {(await r.stdout_str()).strip()}")
# ── list jobs ─────────────────────────────────────
jobs_path = f"{run_path}/jobs"
print(f"\n=== ls {jobs_path}/ ===")
r = await ws.execute(f'ls "{jobs_path}/"')
print(await r.stdout_str())
jobs_out = (await r.stdout_str()).strip().splitlines()
json_jobs = [j.strip() for j in jobs_out if j.strip().endswith(".json")]
log_jobs = [j.strip() for j in jobs_out if j.strip().endswith(".log")]
# ── read a job .json ──────────────────────────────
if json_jobs:
job_name = json_jobs[0]
job_path = f"{jobs_path}/{job_name}"
print(f"=== cat {job_name} | head -n 20 ===")
r = await ws.execute(f'cat "{job_path}" | head -n 20')
print(await r.stdout_str())
print(f"=== stat {job_name} ===")
r = await ws.execute(f'stat "{job_path}"')
print(f" {(await r.stdout_str()).strip()}")
# ── read a job .log ───────────────────────────────
if log_jobs:
log_name = log_jobs[0]
log_path = f"{jobs_path}/{log_name}"
print(f"=== head -n 20 {log_name} ===")
r = await ws.execute(f'head -n 20 "{log_path}"')
print((await r.stdout_str())[:1000])
print(f"\n=== tail -n 10 {log_name} ===")
r = await ws.execute(f'tail -n 10 "{log_path}"')
print((await r.stdout_str())[:500])
print(f"\n=== wc -l {log_name} ===")
r = await ws.execute(f'wc -l "{log_path}"')
print(f" {(await r.stdout_str()).strip()}")
# ── annotations ───────────────────────────────────
print(f"\n=== cat {run_path}/annotations.jsonl ===")
r = await ws.execute(f'cat "{run_path}/annotations.jsonl"')
out = (await r.stdout_str()).strip()
if out:
for line in out.splitlines()[:5]:
print(f" {line[:120]}")
else:
print(" (no annotations)")
# ── artifacts ─────────────────────────────────────
print(f"\n=== ls {run_path}/artifacts/ ===")
r = await ws.execute(f'ls "{run_path}/artifacts/"')
out = (await r.stdout_str()).strip()
if out:
print(out)
else:
print(" (no artifacts)")
# ── tree ──────────────────────────────────────────
print("\n=== tree -L 2 /ci/ ===")
r = await ws.execute("tree -L 2 /ci/")
print(await r.stdout_str())
# ── find (scoped to a single run) ─────────────────
print(f"=== find {run_path}/ -name '*.log' | head -n 10 ===")
r = await ws.execute(f'find "{run_path}/" -name "*.log" | head -n 10')
print(await r.stdout_str())
print(f"=== find {run_path}/ -name '*.json' | head -n 10 ===")
r = await ws.execute(f'find "{run_path}/" -name "*.json" | head -n 10')
print(await r.stdout_str())
# -path matches the display path; -size counts dirs and sizeless
# rendered files as 0 (so +0c drops them, -1k keeps them).
print(f"=== find {run_path}/ -path '*jobs*' | head -n 5 ===")
r = await ws.execute(f'find "{run_path}/" -path "*jobs*" | head -n 5')
print(await r.stdout_str())
print(f"=== find {run_path}/ -maxdepth 1 -size +0c ===")
r = await ws.execute(f'find "{run_path}/" -maxdepth 1 -size +0c')
print(f" exit={r.exit_code} (sizeless entries count as 0,"
" expect no output)")
# ── cd into a run ─────────────────────────────────
print("=== pwd ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print(f'\n=== cd "{run_path}" ===')
r = await ws.execute(f'cd "{run_path}"')
print(f" exit={r.exit_code}")
print("\n=== pwd (after cd) ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== ls (relative, in run dir) ===")
r = await ws.execute("ls")
print(await r.stdout_str())
print("=== cat run.json | head -n 5 (relative) ===")
r = await ws.execute("cat run.json | head -n 5")
print(await r.stdout_str())
# ── cd into jobs ──────────────────────────────────
print('=== cd jobs ===')
r = await ws.execute("cd jobs")
print(f" exit={r.exit_code}")
print("\n=== pwd (after cd jobs) ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== ls (relative, in jobs dir) ===")
r = await ws.execute("ls")
print(await r.stdout_str())
if log_jobs:
log_name = log_jobs[0]
print(f"=== head -n 5 {log_name} (relative) ===")
r = await ws.execute(f"head -n 5 {log_name}")
print((await r.stdout_str())[:300])
# ── stat on workflows ─────────────────────────────
print("\n=== stat /ci/workflows/ ===")
r = await ws.execute('stat "/ci/workflows/"')
print(f" {(await r.stdout_str()).strip()}")
print("\n=== stat /ci/runs/ ===")
r = await ws.execute('stat "/ci/runs/"')
print(f" {(await r.stdout_str()).strip()}")
if __name__ == "__main__":
asyncio.run(main())
+128
View File
@@ -0,0 +1,128 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.github_ci import GitHubCIConfig, GitHubCIResource
load_dotenv(".env.development")
config = GitHubCIConfig(
token=os.environ["GITHUB_TOKEN"],
owner="strukto-ai",
repo="mirage",
max_runs=300,
)
resource = GitHubCIResource(config=config)
with Workspace({"/ci/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
# ── list root ────────────────────────────────
print("--- os.listdir() root ---")
entries = os.listdir(mp)
for e in entries:
print(f" {e}")
# ── list workflows ───────────────────────────
print("\n--- os.listdir() workflows ---")
workflows = os.listdir(f"{mp}/workflows")
for wf in workflows[:10]:
print(f" {wf}")
if workflows:
wf_path = f"{mp}/workflows/{workflows[0]}"
print(f"\n--- open() + read {workflows[0]} ---")
with open(wf_path) as f:
data = json.loads(f.read())
print(f" name: {data.get('name')}")
print(f" state: {data.get('state')}")
# ── list runs ────────────────────────────────
print("\n--- os.listdir() runs ---")
runs = os.listdir(f"{mp}/runs")
for r in runs[:5]:
print(f" {r}")
if len(runs) > 5:
print(f" ... ({len(runs)} total)")
if runs:
run = runs[0]
run_dir = f"{mp}/runs/{run}"
# ── list run contents ────────────────────
print(f"\n--- os.listdir() {run} ---")
contents = os.listdir(run_dir)
for c in contents:
print(f" {c}")
# ── read run.json ────────────────────────
run_json = f"{run_dir}/run.json"
if os.path.exists(run_json):
print("\n--- open() + read run.json ---")
with open(run_json) as f:
data = json.loads(f.read())
print(f" status: {data.get('status')}")
print(f" conclusion: {data.get('conclusion')}")
print(f" event: {data.get('event')}")
# ── list and read jobs ───────────────────
jobs_dir = f"{run_dir}/jobs"
if os.path.isdir(jobs_dir):
print("\n--- os.listdir() jobs ---")
jobs = os.listdir(jobs_dir)
for j in jobs:
print(f" {j}")
log_files = [j for j in jobs if j.endswith(".log")]
if log_files:
log_path = f"{jobs_dir}/{log_files[0]}"
print(
f"\n--- open() + read {log_files[0]} (first 10 lines) ---")
with open(log_path) as f:
for i, line in enumerate(f):
if i >= 10:
print(" ...")
break
print(f" {line.rstrip()[:120]}")
# ── list artifacts ───────────────────────
artifacts_dir = f"{run_dir}/artifacts"
if os.path.isdir(artifacts_dir):
print("\n--- os.listdir() artifacts ---")
artifacts = os.listdir(artifacts_dir)
for a in artifacts:
print(f" {a}")
if not artifacts:
print(" (none)")
# ── interactive ──────────────────────────────
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> ls {mp}/runs/")
print(f">>> cat {mp}/runs/<run>/run.json")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes")
+127
View File
@@ -0,0 +1,127 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.github_ci import GitHubCIConfig, GitHubCIResource
load_dotenv(".env.development")
config = GitHubCIConfig(
token=os.environ["GITHUB_TOKEN"],
owner="strukto-ai",
repo="mirage",
max_runs=300,
)
resource = GitHubCIResource(config=config)
async def main():
with Workspace({"/ci/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE: open() reads from GitHub CI transparently ===\n")
print("--- os.listdir() root ---")
entries = vos.listdir("/ci")
for e in entries:
print(f" {e}")
print("\n--- os.listdir() workflows ---")
workflows = vos.listdir("/ci/workflows")
for wf in workflows[:10]:
print(f" {wf}")
if workflows:
wf_path = f"/ci/workflows/{workflows[0]}"
print(f"\n--- open() + read {wf_path} ---")
with open(wf_path) as f:
data = json.loads(f.read())
print(f" name: {data.get('name')}")
print(f" path: {data.get('path')}")
print(f" state: {data.get('state')}")
print("\n--- os.listdir() runs ---")
runs = vos.listdir("/ci/runs")
for r in runs[:5]:
print(f" {r}")
if len(runs) > 5:
print(f" ... ({len(runs)} total)")
if runs:
run_path = f"/ci/runs/{runs[0]}"
print(f"\n--- os.listdir() {run_path} ---")
contents = vos.listdir(run_path)
for c in contents:
print(f" {c}")
if "run.json" in contents:
print("\n--- open() + read run.json ---")
with open(f"{run_path}/run.json") as f:
data = json.loads(f.read())
print(f" status: {data.get('status')}")
print(f" conclusion: {data.get('conclusion')}")
print(f" event: {data.get('event')}")
print(f" branch: {data.get('head_branch')}")
if "jobs" in contents:
jobs_path = f"{run_path}/jobs"
print("\n--- os.listdir() jobs ---")
jobs = vos.listdir(jobs_path)
for j in jobs[:10]:
print(f" {j}")
json_jobs = [j for j in jobs if j.endswith(".json")]
log_jobs = [j for j in jobs if j.endswith(".log")]
if json_jobs:
print("\n--- open() + read job .json ---")
with open(f"{jobs_path}/{json_jobs[0]}") as f:
data = json.loads(f.read())
print(f" name: {data.get('name')}")
print(f" status: {data.get('status')}")
print(f" conclusion: {data.get('conclusion')}")
steps = data.get("steps", [])
print(f" steps: {len(steps)}")
for s in steps[:3]:
print(f" {s.get('number')}. {s.get('name')}"
f" -> {s.get('conclusion')}")
if log_jobs:
print("\n--- open() + read job .log (first 10 lines) ---")
with open(f"{jobs_path}/{log_jobs[0]}") as f:
for i, line in enumerate(f):
if i >= 10:
print(" ...")
break
print(f" {line.rstrip()[:120]}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+150
View File
@@ -0,0 +1,150 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
from mirage.resource.github import GitHubConfig, GitHubResource
from mirage.resource.s3 import S3Config, S3Resource
load_dotenv(".env.development")
s3_config = S3Config(
bucket=os.environ["AWS_S3_BUCKET"],
region=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)
gdrive_config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
github_config = GitHubConfig(token=os.environ["GITHUB_TOKEN"])
ws = Workspace(
{
"/s3/": S3Resource(s3_config),
"/gdrive/": GoogleDriveResource(gdrive_config),
"/github/": GitHubResource(
github_config, owner="strukto", repo="mirage"),
},
mode=MountMode.READ,
)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
async def main():
# ── prime gdrive cache ──
await ws.execute("ls /gdrive/")
await ws.execute("ls /gdrive/mirage/")
# ── plan: directory scans across resources ──
print("=== PLAN: DIRECTORY SCANS ===\n")
dr = await ws.execute("grep mirage /s3/data/example.jsonl", provision=True)
print(f"s3 single file: network_read={dr.network_read}")
dr = await ws.execute("grep mirage /gdrive/mirage/example.jsonl",
provision=True)
print(f"gdrive single file: network_read={dr.network_read}")
dr = await ws.execute("rg import /github/mirage/commands/registry.py",
provision=True)
print(f"github single file: network_read={dr.network_read}")
print(f"\nStats after plans (should be 0): {ops_summary()}")
# ── S3: grep on single file vs directory ──
print("\n=== S3: SINGLE FILE vs DIRECTORY ===\n")
r = await ws.execute("grep mirage /s3/data/example.jsonl | wc -l")
print(f"single file: {(await r.stdout_str()).strip()} matches")
r = await ws.execute("rg -l mirage /s3/data/")
files = (await r.stdout_str()).strip().splitlines()
print(f"directory rg -l: {len(files)} files match")
print(f"Stats: {ops_summary()}")
# ── Google Drive: grep with streaming ──
print("\n=== GDRIVE: GREP WITH STREAMING ===\n")
r = await ws.execute(
"grep queue-operation /gdrive/mirage/example.jsonl | wc -l")
print(f"grep | wc: {(await r.stdout_str()).strip()} matches")
r = await ws.execute("grep queue-operation /gdrive/mirage/example.jsonl"
" | head -n 3")
lines = (await r.stdout_str()).strip().splitlines()
print(f"grep | head -n 3: {len(lines)} lines")
print(f"Stats: {ops_summary()}")
# ── GitHub: rg with search_code optimization ──
print("\n=== GITHUB: RG (search_code optimization) ===\n")
r = await ws.execute(
"rg -l workspace /github/mirage/workspace/workspace.py")
files = (await r.stdout_str()).strip().splitlines()
print(f"rg -l workspace (single file): {len(files)} files")
if files:
print(f" {files[0]}")
r = await ws.execute("rg -l import /github/mirage/")
if r.stderr:
print(f"rg on large dir: {r.stderr.decode().strip()}")
else:
files = (await r.stdout_str()).strip().splitlines()
print(f"rg -l import (large dir): {len(files)} files")
r = await ws.execute("grep -c def /github/mirage/commands/registry.py")
print(f"grep -c def registry.py: {(await r.stdout_str()).strip()}")
print(f"Stats: {ops_summary()}")
# ── cross-resource consistency ──
print("\n=== CROSS-RESOURCE: SAME DATA ===\n")
r1 = await ws.execute("wc -l /s3/data/example.jsonl")
r2 = await ws.execute("wc -l /gdrive/mirage/example.jsonl")
print(f"s3 wc -l: {(await r1.stdout_str()).strip()}")
print(f"gdrive wc -l: {(await r2.stdout_str()).strip()}")
r1 = await ws.execute("grep -c queue-operation /s3/data/example.jsonl")
r2 = await ws.execute(
"grep -c queue-operation /gdrive/mirage/example.jsonl")
print(f"s3 grep -c queue-operation: {(await r1.stdout_str()).strip()}")
print(f"gdrive grep -c queue-operation: {(await r2.stdout_str()).strip()}")
# ── bash history ──
print("\n=== BASH HISTORY ===\n")
print(f"Total ops: {ops_summary()}")
print(f"Commands recorded: {len(await ws.history())}")
r = await ws.execute("tail -n 6 /.bash_history")
for line in (await r.stdout_str()).strip().splitlines():
print(f" {line[:120]}")
asyncio.run(main())
+264
View File
@@ -0,0 +1,264 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
async def main() -> None:
ws = Workspace({"/gmail": resource}, mode=MountMode.WRITE)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /gmail/__nf_missing__.txt",
"head /gmail/__nf_missing__.txt",
"stat /gmail/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ls root labels
print("=== ls /gmail/ ===")
result = await ws.execute("ls /gmail/")
print(await result.stdout_str())
# Pick INBOX
labels = (await result.stdout_str()).strip().splitlines()
label = "INBOX"
if not any("INBOX" in lb for lb in labels):
label = labels[0] if labels else ""
if not label:
print("No labels")
return
# ls label (date directories)
print(f"=== ls /gmail/{label}/ ===")
result = await ws.execute(f"ls /gmail/{label}/")
print(await result.stdout_str())
dates = (await result.stdout_str()).strip().splitlines()
if not dates:
print("No dates")
return
first_date = dates[0]
# ls date dir (messages)
print(f"=== ls /gmail/{label}/{first_date}/ ===")
result = await ws.execute(f"ls /gmail/{label}/{first_date}/")
print(await result.stdout_str())
messages = [
m for m in (await result.stdout_str()).strip().splitlines()
if m.endswith(".gmail.json")
]
if not messages:
print("No messages")
return
first_msg = messages[0]
msg_path = f"/gmail/{label}/{first_date}/{first_msg}"
# cat message
print(f"=== cat {msg_path} ===")
result = await ws.execute(f"cat {msg_path}")
print((await result.stdout_str())[:500])
# head
print("=== head -n 5 ===")
result = await ws.execute(f"head -n 5 {msg_path}")
print(await result.stdout_str())
# tail
print("=== tail -n 3 ===")
result = await ws.execute(f"tail -n 3 {msg_path}")
print(await result.stdout_str())
# wc
print("=== wc -l ===")
result = await ws.execute(f"wc -l {msg_path}")
print(await result.stdout_str())
# stat
print("=== stat ===")
result = await ws.execute(f"stat {msg_path}")
print(await result.stdout_str())
# jq
print("=== jq .subject ===")
result = await ws.execute(f'jq ".subject" {msg_path}')
print(await result.stdout_str())
print("=== jq .from ===")
result = await ws.execute(f'jq ".from" {msg_path}')
print(await result.stdout_str())
# nl
print("=== nl ===")
result = await ws.execute(f"nl {msg_path}")
print((await result.stdout_str())[:300])
# tree
print("=== tree -L 1 /gmail/ ===")
result = await ws.execute("tree -L 1 /gmail/")
print(await result.stdout_str())
print(f"=== tree -L 1 /gmail/{label}/ ===")
result = await ws.execute(f"tree -L 1 /gmail/{label}/")
print(await result.stdout_str())
# find
print("=== find -name '*.gmail.json' ===")
result = await ws.execute(
f'find /gmail/{label}/{first_date}/ -name "*.gmail.json" | head -n 5')
print(await result.stdout_str())
# grep
print("=== grep subject ===")
result = await ws.execute(f"grep subject {msg_path}")
print(await result.stdout_str())
# rg
print("=== rg subject ===")
result = await ws.execute(f"rg subject {msg_path}")
print(await result.stdout_str())
# ── native search dispatch (Gmail q= API) ────────
print(f"\n=== grep harbor /gmail/{label}/{first_date}/*.gmail.json"
" (date scope) ===")
result = await ws.execute(
f'grep harbor /gmail/{label}/{first_date}/*.gmail.json')
out = (await result.stdout_str()).strip()
lines = out.splitlines() if out else []
print(f" exit={result.exit_code} matches: {len(lines)}")
for line in lines[:3]:
print(f" {line[:150]}")
print(f"\n=== grep harbor /gmail/{label}/ (label scope) ===")
result = await ws.execute(f'grep harbor /gmail/{label}/')
out = (await result.stdout_str()).strip()
lines = out.splitlines() if out else []
print(f" exit={result.exit_code} matches: {len(lines)}")
for line in lines[:3]:
print(f" {line[:150]}")
print("\n=== grep harbor /gmail/ (mailbox scope) ===")
result = await ws.execute('grep harbor /gmail/')
out = (await result.stdout_str()).strip()
lines = out.splitlines() if out else []
print(f" exit={result.exit_code} matches: {len(lines)}")
for line in lines[:3]:
print(f" {line[:150]}")
print("\n=== rg harbor /gmail/ ===")
result = await ws.execute('rg harbor /gmail/')
out = (await result.stdout_str()).strip()
lines = out.splitlines() if out else []
print(f" exit={result.exit_code} matches: {len(lines)}")
for line in lines[:3]:
print(f" {line[:150]}")
# basename
print("=== basename ===")
result = await ws.execute(f"basename {msg_path}")
print(await result.stdout_str())
# dirname
print("=== dirname ===")
result = await ws.execute(f"dirname {msg_path}")
print(await result.stdout_str())
# realpath
print("=== realpath ===")
result = await ws.execute(f"realpath {msg_path}")
print(await result.stdout_str())
# Resource-specific commands
# gws-gmail-triage
print("=== gws-gmail-triage ===")
result = await ws.execute('gws-gmail-triage --query "is:unread" --max 3')
print((await result.stdout_str())[:500])
# ── glob expansion (exercises resolve_glob → readdir)
print("=== echo glob: *.gmail.json ===")
result = await ws.execute(f'echo /gmail/{label}/{first_date}/*.gmail.json')
out = (await result.stdout_str()).strip()
print(f" {out[:200]}")
assert out, "glob should match at least one file"
print("=== for f in *.gmail.json (glob loop) ===")
result = await ws.execute(
f'for f in /gmail/{label}/{first_date}/*.gmail.json;'
' do echo found:$f; done | head -n 3')
out = (await result.stdout_str()).strip()
for line in out.splitlines():
print(f" {line[:120]}")
# gws-gmail-read (use message_id from filename)
msg_id = first_msg.rsplit("__", 1)[-1].replace(".gmail.json", "")
print(f"=== gws-gmail-read --id {msg_id} ===")
result = await ws.execute(f"gws-gmail-read --id {msg_id}")
print((await result.stdout_str())[:500])
# gws-gmail-send
print("=== gws-gmail-send ===")
result = await ws.execute('gws-gmail-send --to "zechengzhang97@gmail.com"'
' --subject "Test from MIRAGE"'
' --body "Sent by gmail.py example"')
out = await result.stdout_str()
print(out[:200])
sent_id = ""
if out.strip():
sent = json.loads(out)
sent_id = sent.get("id", "")
# gws-gmail-reply
if sent_id:
print("=== gws-gmail-reply ===")
result = await ws.execute(f'gws-gmail-reply --message-id {sent_id}'
' --body "Reply from MIRAGE"')
print((await result.stdout_str())[:200])
# gws-gmail-reply-all
if sent_id:
print("=== gws-gmail-reply-all ===")
result = await ws.execute(f'gws-gmail-reply-all --message-id {sent_id}'
' --body "Reply-all from MIRAGE"')
print((await result.stdout_str())[:200])
# gws-gmail-forward
if sent_id:
print("=== gws-gmail-forward ===")
result = await ws.execute(f'gws-gmail-forward --message-id {sent_id}'
' --to "zechengzhang97@gmail.com"')
print((await result.stdout_str())[:200])
if __name__ == "__main__":
asyncio.run(main())
+77
View File
@@ -0,0 +1,77 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
with Workspace({"/gmail/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() labels ---")
labels = os.listdir(mp)
for lb in labels:
print(f" {lb}")
inbox_path = f"{mp}/INBOX"
if os.path.isdir(inbox_path):
print("\n--- os.listdir() INBOX (dates) ---")
dates = os.listdir(inbox_path)
for d in dates[:5]:
print(f" {d}")
if dates:
date_path = f"{inbox_path}/{dates[0]}"
print(f"\n--- os.listdir() {dates[0]} ---")
messages = os.listdir(date_path)
for m in messages[:5]:
print(f" {m}")
json_msgs = [m for m in messages if m.endswith(".gmail.json")]
if json_msgs:
first = json_msgs[0]
path = f"{date_path}/{first}"
print(f"\n--- open() + read {first[:60]} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" subject: {parsed.get('subject', 'N/A')}")
print(f" from: {parsed.get('from', 'N/A')}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> ls {mp}/INBOX/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+83
View File
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
async def main():
with Workspace({"/gmail/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE ===\n")
print("--- os.listdir() labels ---")
labels = vos.listdir("/gmail")
for label in labels:
print(f" {label}")
print("\n--- os.listdir() INBOX dates ---")
dates = vos.listdir("/gmail/INBOX")
for d in dates[:5]:
print(f" {d}")
if dates:
first_date = dates[0]
print(f"\n--- os.listdir() {first_date} messages ---")
messages = vos.listdir(f"/gmail/INBOX/{first_date}")
for msg in messages[:5]:
print(f" {msg}")
json_msgs = [m for m in messages if m.endswith(".gmail.json")]
if json_msgs:
first = json_msgs[0]
path = f"/gmail/INBOX/{first_date}/{first}"
print(f"\n--- open() + read {first} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" subject: {parsed.get('subject', 'N/A')}")
print(f" from: {parsed.get('from', 'N/A')}")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+101
View File
@@ -0,0 +1,101 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
load_dotenv(".env.development")
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
async def main():
ws = Workspace({"/gdocs": resource}, mode=MountMode.WRITE)
r = await ws.execute("ls /gdocs/owned/ | head -n 3")
print("=== ls (first 3) ===")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== plan: cat ===")
dr = await ws.execute(f"cat /gdocs/owned/{first}", provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== plan: grep ===")
dr = await ws.execute(f"grep textRun /gdocs/owned/{first}", provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== jq .title ===")
r = await ws.execute(f'jq ".title" /gdocs/owned/{first}')
print(await r.stdout_str())
print("=== head -c 200 ===")
r = await ws.execute(f"head -c 200 /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== grep textRun ===")
r = await ws.execute(f"grep textRun /gdocs/owned/{first} | head -c 200")
print(await r.stdout_str())
print("=== tail -c 200 ===")
r = await ws.execute(f"tail -c 200 /gdocs/owned/{first}")
print(await r.stdout_str())
print("=== gws-docs-documents-create ===")
r = await ws.execute('gws-docs-documents-create'
' --json \'{"title": "MIRAGE Example Doc"}\'')
doc = json.loads(await r.stdout_str())
doc_id = doc["documentId"]
print(f"Created: {doc_id}")
print("\n=== gws-docs-documents-batchUpdate ===")
body = json.dumps({
"requests": [{
"insertText": {
"location": {
"index": 1
},
"text": "Hello from MIRAGE!\n",
}
}]
})
params = json.dumps({"documentId": doc_id})
r = await ws.execute(f"gws-docs-documents-batchUpdate"
f" --params '{params}' --json '{body}'")
print(f"Updated: {(await r.stdout_str())[:80]}")
print("\n=== gws-docs-write ===")
r = await ws.execute(f'gws-docs-write'
f' --document {doc_id}'
f' --text "Appended via gws-docs-write."')
print(f"Written: {(await r.stdout_str())[:80]}")
url = f"https://docs.google.com/document/d/{doc_id}/edit"
print(f"\nOpen: {url}")
if __name__ == "__main__":
asyncio.run(main())
+78
View File
@@ -0,0 +1,78 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdocs import GDocsConfig, GDocsResource
load_dotenv(".env.development")
config = GDocsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GDocsResource(config=config)
async def main():
with Workspace({"/gdocs/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Docs transparently ===\n")
print("--- os.listdir() root ---")
dirs = vos.listdir("/gdocs")
for d in dirs:
print(f" {d}")
print("\n--- os.listdir() owned ---")
docs = vos.listdir("/gdocs/owned")
for doc in docs[:5]:
print(f" {doc}")
if docs:
first = docs[0]
path = f"/gdocs/owned/{first}"
print("\n--- open() + read first doc ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" title: {parsed.get('title', 'N/A')}")
print(f" content preview: {content[:200]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gdocs/owned/nope.json')}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+77
View File
@@ -0,0 +1,77 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
async def main():
ws = Workspace({"/gdrive": resource}, mode=MountMode.READ)
print("=== ls /gdrive/ (first 10) ===")
r = await ws.execute("ls /gdrive/ | head -n 10")
print(await r.stdout_str())
r = await ws.execute("ls /gdrive/ | head -n 30")
listing = await r.stdout_str()
gdoc = gsheet = gslide = None
for line in listing.strip().split("\n"):
f = line.strip()
if ".gdoc.json" in f and not gdoc:
gdoc = f
if ".gsheet.json" in f and not gsheet:
gsheet = f
if ".gslide.json" in f and not gslide:
gslide = f
if gdoc:
print(f"=== jq .title on {gdoc} ===")
r = await ws.execute(f'jq ".title" "/gdrive/{gdoc}"')
print(await r.stdout_str())
print(f"=== head -c 150 on {gdoc} ===")
r = await ws.execute(f'head -c 150 "/gdrive/{gdoc}"')
print(await r.stdout_str())
if gslide:
print(f"\n=== jq .title on {gslide} ===")
r = await ws.execute(f'jq ".title" "/gdrive/{gslide}"')
print(await r.stdout_str())
print("=== jq slides length ===")
r = await ws.execute(f'jq ".slides | length" "/gdrive/{gslide}"')
print(await r.stdout_str())
if gsheet:
print(f"\n=== jq .properties.title on {gsheet} ===")
r = await ws.execute(f'jq ".properties.title" "/gdrive/{gsheet}"')
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
+487
View File
@@ -0,0 +1,487 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
backend = GoogleDriveResource(config=config)
ws = Workspace({"/gdrive/": backend}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
async def main():
# ── prime the cache: gdrive resolves paths via file IDs ──
await ws.execute("ls /gdrive/")
await ws.execute("ls /gdrive/mirage/")
# ── plan: estimate before executing ──
print("=== PLAN ESTIMATES ===\n")
dr = await ws.execute("grep mirage /gdrive/mirage/example.jsonl",
provision=True)
print("--- plan: grep mirage /gdrive/mirage/example.jsonl ---")
print(f" network_read: {dr.network_read}, cache_read: {dr.cache_read}")
print(f" read_ops: {dr.read_ops}, precision: {dr.precision}")
dr = await ws.execute(
"grep mirage /gdrive/mirage/example.jsonl | head -n 3", provision=True)
print("\n--- plan: grep mirage ... | head -n 3 ---")
print(f" op: {dr.op}, children: {len(dr.children)}")
print(f" network_read: {dr.network_read}, cache_read: {dr.cache_read}")
print(f" precision: {dr.precision}")
for c in dr.children:
net, cache = c.network_read, c.cache_read
print(f" {c.command}: net={net}, cache={cache}, {c.precision}")
dr = await ws.execute(
"grep mirage /gdrive/mirage/example.jsonl && echo found",
provision=True)
print("\n--- plan: grep ... && echo found ---")
print(f" op: {dr.op}, network_read: {dr.network_read}")
for c in dr.children:
print(f" {c.command}: net={c.network_read}, {c.precision}")
print(f"\n Stats after plans (should be 0): {ops_summary()}")
# ── cache-aware plan ──
print("\n--- caching: cat /gdrive/mirage/example.jsonl | wc -l ---")
result = await ws.execute("cat /gdrive/mirage/example.jsonl | wc -l")
print(f" lines: {(await result.stdout_str()).strip()}")
print(f" Stats after caching: {ops_summary()}")
dr = await ws.execute("grep mirage /gdrive/mirage/example.jsonl",
provision=True)
print("\n--- plan after cache: grep mirage ... ---")
print(f" network_read: {dr.network_read}, cache_read: {dr.cache_read}")
print(f" cache_hits: {dr.cache_hits}, read_ops: {dr.read_ops}")
print("\n=== ACTUAL EXECUTION ===\n")
# ── simple grep ──
print("--- grep mirage /gdrive/mirage/example.jsonl ---")
output = await (
await
ws.execute("grep mirage /gdrive/mirage/example.jsonl")).stdout_str()
lines = output.strip().splitlines() if output.strip() else []
print(f" Matches: {len(lines)}")
if lines:
print(f" First: {lines[0][:80]}...")
print(f" Stats: {ops_summary()}")
# ── grep with limit ──
print("\n--- grep -m 1 mirage /gdrive/mirage/example.jsonl ---")
output = await (await
ws.execute("grep -m 1 mirage /gdrive/mirage/example.jsonl")
).stdout_str()
lines = output.strip().splitlines() if output.strip() else []
print(f" Matches: {len(lines)}")
print(f" Stats: {ops_summary()}")
# ── pipe: grep | wc -l ──
print("\n--- grep mirage /gdrive/mirage/example.jsonl | wc -l ---")
result = await ws.execute(
"grep mirage /gdrive/mirage/example.jsonl | wc -l")
print(f" Count: {(await result.stdout_str()).strip()}")
print(f" Exit code: {result.exit_code}")
print(f" Stats: {ops_summary()}")
# ── pipe: grep | head ──
print("\n--- grep mirage /gdrive/mirage/example.jsonl | head -n 3 ---")
result = await ws.execute(
"grep mirage /gdrive/mirage/example.jsonl | head -n 3")
lines = (await result.stdout_str()).strip().splitlines()
print(f" Lines: {len(lines)}")
for ln in lines:
print(f" {ln[:80]}...")
print(f" Stats: {ops_summary()}")
# ── pipe: cat | grep | sort | uniq ──
print("\n--- cat /gdrive/mirage/example.jsonl"
" | grep queue-operation | sort | uniq ---")
result = await ws.execute("cat /gdrive/mirage/example.jsonl"
" | grep queue-operation | sort | uniq")
lines = ((await result.stdout_str()).strip().splitlines() if
(await result.stdout_str()).strip() else [])
print(f" Unique lines: {len(lines)}")
print(f" Stats: {ops_summary()}")
# ── pipe: grep | cut (extract field) ──
print("\n--- rg queue-operation /gdrive/mirage/example.jsonl"
" | head -n 5 | cut -d , -f 2 ---")
result = await ws.execute("rg queue-operation /gdrive/mirage/example.jsonl"
" | head -n 5 | cut -d , -f 2")
print(f" Fields:\n {(await result.stdout_str()).strip()}")
print(f" Stats: {ops_summary()}")
# ── && chain: grep && echo ──
print("\n--- grep -m 1 mirage /gdrive/mirage/example.jsonl"
" && echo 'found mirage' ---")
result = await ws.execute(
"grep -m 1 mirage /gdrive/mirage/example.jsonl && echo found")
print(f" Exit code: {result.exit_code}")
print(
f" Stdout ends with: ...{(await result.stdout_str()).strip()[-30:]}")
print(f" Stats: {ops_summary()}")
# ── || chain: grep nonexistent || echo fallback ──
print("\n--- grep NONEXISTENT /gdrive/mirage/example.jsonl"
" || echo 'not found' ---")
result = await ws.execute(
"grep NONEXISTENT /gdrive/mirage/example.jsonl || echo not_found")
print(f" Exit code: {result.exit_code}")
print(f" Output: {(await result.stdout_str()).strip()}")
print(f" Stats: {ops_summary()}")
# ── subshell: (grep | sort | uniq) | wc -l ──
print("\n--- (grep queue-operation /gdrive/mirage/example.jsonl"
" | sort | uniq) | wc -l ---")
result = await ws.execute(
"(grep queue-operation /gdrive/mirage/example.jsonl"
" | sort | uniq) | wc -l")
print(f" Unique queue ops: {(await result.stdout_str()).strip()}")
print(f" Stats: {ops_summary()}")
# ── semicolon: multiple independent reads ──
print("\n--- head -n 1 /gdrive/mirage/example.jsonl"
" ; wc -l /gdrive/mirage/example.jsonl ---")
result = await ws.execute("head -n 1 /gdrive/mirage/example.jsonl"
" ; wc -l /gdrive/mirage/example.jsonl")
print(f" Output: {(await result.stdout_str()).strip()}")
print(f" Stats: {ops_summary()}")
# ── lazy multi-pipe: grep | grep | head | cut ──
print("\n--- lazy multi-pipe: grep | grep -v | head | cut ---")
result = await ws.execute(
"grep queue-operation /gdrive/mirage/example.jsonl"
" | grep -v error | head -n 2 | cut -d , -f 1")
print(f" Output:\n {(await result.stdout_str()).strip()}")
result_full = await ws.execute(
"grep queue-operation /gdrive/mirage/example.jsonl"
" | grep -v error | cut -d , -f 1")
full_lines = (await result_full.stdout_str()).strip().splitlines()
print(f" Without head: {len(full_lines)} lines (full download)")
# ── recursive search ──
print("\n--- rg -l mirage /gdrive/mirage ---")
output = await (await
ws.execute("rg -l mirage /gdrive/mirage")).stdout_str()
lines = output.strip().splitlines() if output.strip() else []
print(f" Files: {lines}")
print(f" Stats: {ops_summary()}")
# ── Google native files: docs, sheets, slides ──
print("\n=== GOOGLE NATIVE FILES ===\n")
print("--- ls /gdrive (find native files) ---")
r = await ws.execute("ls /gdrive/ | head -n 20")
print(f" {(await r.stdout_str()).strip()}")
print("\n--- grep in Google Docs (.gdoc.json) ---")
r = await ws.execute("ls /gdrive/")
listing = (await r.stdout_str()).strip().splitlines()
gdoc = next((f.strip() for f in listing if ".gdoc.json" in f), None)
if gdoc:
print(f" Found doc: {gdoc}")
r = await ws.execute(f'grep -i "title" "/gdrive/{gdoc}"')
out = (await r.stdout_str()).strip()
if out:
print(f" grep title: {out[:120]}...")
print(f" Stats: {ops_summary()}")
else:
print(" No .gdoc.json found in /gdrive/")
gsheet = next((f.strip() for f in listing if ".gsheet.json" in f), None)
if gsheet:
print("\n--- grep in Google Sheets (.gsheet.json) ---")
print(f" Found sheet: {gsheet}")
r = await ws.execute(
f'grep -i "properties" "/gdrive/{gsheet}" | head -n 3')
out = (await r.stdout_str()).strip()
if out:
print(f" grep properties: {out[:120]}...")
print(f" Stats: {ops_summary()}")
gslide = next((f.strip() for f in listing if ".gslide.json" in f), None)
if gslide:
print("\n--- grep in Google Slides (.gslide.json) ---")
print(f" Found slide: {gslide}")
r = await ws.execute(f'grep -i "slide" "/gdrive/{gslide}" | head -n 3')
out = (await r.stdout_str()).strip()
if out:
print(f" grep slide: {out[:120]}...")
print(f" Stats: {ops_summary()}")
# ── mixed file type grep ──
print("\n=== MIXED FILE TYPE GREP ===\n")
print("--- grep -c title in Google Doc (eager read) ---")
if gdoc:
r = await ws.execute(f'grep -c title "/gdrive/{gdoc}"')
print(f" {gdoc}: {(await r.stdout_str()).strip()} matches")
print("\n--- grep in regular file (streaming) ---")
r = await ws.execute("grep -c queue-operation /gdrive/mirage/example.jsonl"
)
print(f" example.jsonl: {(await r.stdout_str()).strip()} matches")
print("\n--- rg across mirage folder (regular files) ---")
r = await ws.execute("rg -l queue /gdrive/mirage/")
files = (await r.stdout_str()).strip().splitlines()
print(f" Files matching 'queue': {files}")
print(f" Stats: {ops_summary()}")
# ── filetype: parquet, orc, feather, hdf5 ──
print("\n=== FILETYPE: parquet, orc, feather, hdf5 ===\n")
for label, path in [
("parquet", "/gdrive/mirage/example.parquet"),
("orc", "/gdrive/mirage/example.orc"),
("feather", "/gdrive/mirage/example.feather"),
("hdf5", "/gdrive/mirage/example.h5"),
]:
print(f"--- grep item_5 {label} ---")
r = await ws.execute(f"grep item_5 {path}")
print(f" exit={r.exit_code} {(await r.stdout_str()).strip()[:100]}")
print("\n--- cat parquet | head -n 3 ---")
r = await ws.execute("cat /gdrive/mirage/example.parquet | head -n 3")
print(f" {(await r.stdout_str()).strip()}")
print("\n--- wc -l across formats ---")
for label, path in [
("parquet", "/gdrive/mirage/example.parquet"),
("orc", "/gdrive/mirage/example.orc"),
("feather", "/gdrive/mirage/example.feather"),
("hdf5", "/gdrive/mirage/example.h5"),
]:
r = await ws.execute(f"wc -l {path}")
print(f" {label}: {(await r.stdout_str()).strip()}")
print(f" Stats: {ops_summary()}")
# ── jq: structured JSON queries ──
print("\n=== JQ QUERIES ===\n")
print("--- jq .metadata ---")
result = await ws.execute("jq .metadata /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: all team names (nested [] iterator) ---")
result = await ws.execute(
"jq \".departments[].teams[].name\" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: all employee names ---")
result = await ws.execute("jq \".departments[].teams[].members[].name\""
" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: senior engineers on platform ---")
result = await ws.execute("jq \".departments[0].teams[0].members"
" | map(select(.level == \\\"senior\\\"))"
" | map(.name)\" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: all active project names ---")
result = await ws.execute("jq \".departments[].teams[].projects"
" | map(select(.status == \\\"active\\\"))"
" | map(.name)\" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: mirage project metrics ---")
result = await ws.execute("jq .departments[0].teams[0].projects[0].metrics"
" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: total budget ---")
result = await ws.execute(
"jq .metadata.total_budget /gdrive/mirage/example.json")
budget = (await result.stdout_str()).strip()
if budget:
print(f" Total budget: ${int(budget):,}")
else:
print(" (no output)")
print("\n--- jq: vendor costs ---")
result = await ws.execute("jq \".vendor_contracts | map(.annual_cost)\""
" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: office locations ---")
result = await ws.execute(
"jq \".locations | map(.city)\" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: all incident titles ---")
result = await ws.execute("jq \".departments[].teams[].incidents[].title\""
" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- jq: OKR key results ---")
result = await ws.execute(
"jq \".okrs[0].objectives[0].key_results"
" | map(.description)\" /gdrive/mirage/example.json")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- pipe: cat | jq (from cache) ---")
result = await ws.execute(
"cat /gdrive/mirage/example.json | jq .metadata.version")
print(f" Version: {(await result.stdout_str()).strip()}")
# ── session: cd + export ──
print("\n=== SESSION: cd + export ===\n")
print("--- cd /gdrive/mirage && ls ---")
await ws.execute("cd /gdrive/mirage")
result = await ws.execute("ls")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- export + variable expansion ---")
await ws.execute("export SEARCH=mirage")
result = await ws.execute("grep $SEARCH example.jsonl | head -n 2")
print(f" {(await result.stdout_str()).strip()[:80]}...")
print("\n--- printenv ---")
result = await ws.execute("printenv")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- plan: cd + grep ---")
await ws.execute("cd /gdrive/mirage")
dr = await ws.execute("grep mirage example.jsonl", provision=True)
print(f" network_read: {dr.network_read}, cache_read: {dr.cache_read}")
# ── execution history: hidden recorder + GNU views ──
print("\n=== EXECUTION HISTORY ===\n")
events = await ws.history()
print(f" Total commands recorded: {len(events)}")
entry = events[-1]
print(f"\n Last command: {entry['command']}")
print(f" Agent: {entry['agent']}")
print(f" Exit code: {entry['exit_code']}")
print("\n --- history (shell builtin, this session) ---")
r = await ws.execute("history 5")
for line in (await r.stdout_str()).splitlines():
print(f" {line[:100]}")
print("\n --- tail -n 6 /.bash_history (all sessions, GNU file) ---")
r = await ws.execute("tail -n 6 /.bash_history")
for line in (await r.stdout_str()).splitlines():
print(f" {line[:100]}")
print("\n --- history as JSONL ---")
for e in events[-3:]:
print(json.dumps(e, separators=(",", ":")))
# ── background jobs: & operator ──
print("\n=== BACKGROUND JOBS ===\n")
print("--- launch two background greps ---")
r = await ws.execute(
"grep mirage /gdrive/mirage/example.jsonl &"
" grep queue-operation /gdrive/mirage/example.jsonl &",
agent_id="demo-agent",
)
print(f" Output: {(await r.stdout_str()).strip()}")
print("\n--- ps: show running jobs ---")
r = await ws.execute("ps u")
print(f" {(await r.stdout_str()).strip()}")
print("\n--- jobs: list all ---")
r = await ws.execute("jobs")
print(f" {(await r.stdout_str()).strip()}")
print("\n--- wait %1: get first grep result ---")
r = await ws.execute("wait %1")
lines = (await r.stdout_str()).strip().splitlines() if (
await r.stdout_str()).strip() else []
print(f" Matches: {len(lines)}, exit_code: {r.exit_code}")
if lines:
print(f" First: {lines[0][:80]}...")
print("\n--- wait %2: get second grep result ---")
r = await ws.execute("wait %2")
lines = (await r.stdout_str()).strip().splitlines() if (
await r.stdout_str()).strip() else []
print(f" Matches: {len(lines)}, exit_code: {r.exit_code}")
print("\n--- background pipe: grep | head & ---")
await ws.execute(
"grep queue-operation /gdrive/mirage/example.jsonl | head -n 3 &")
r = await ws.execute("wait %3")
print(f" Output:\n {(await r.stdout_str()).strip()}")
print("\n--- kill demo ---")
await ws.execute("grep mirage /gdrive/mirage/example.jsonl &")
await ws.execute("kill %4")
r = await ws.execute("wait %4")
print(f" Exit code after kill: {r.exit_code}")
print("\n--- wait || fallback pattern ---")
await ws.execute("grep NONEXISTENT /gdrive/mirage/example.jsonl &")
await ws.execute("grep mirage /gdrive/mirage/example.jsonl &")
r = await ws.execute("wait %5 || wait %6")
lines = (await r.stdout_str()).strip().splitlines() if (
await r.stdout_str()).strip() else []
print(f" Fallback matches: {len(lines)}")
print("\n--- jobs after all done ---")
r = await ws.execute("jobs")
print(f" {(await r.stdout_str()).strip() or '(empty)'}")
print("\n--- background job history ---")
bg_entries = [
e for e in await ws.history()
if "grep" in e["command"] and "&" not in e["command"]
]
print(f" Background job records: {len(bg_entries)}")
for e in bg_entries[-4:]:
print(f" {e['command'][:60]} exit={e['exit_code']}")
# ── bash history view ──
print("\n=== BASH HISTORY ===\n")
print("--- ls -a / (dotfile view) ---")
result = await ws.execute("ls -a /")
print(f" {(await result.stdout_str()).strip()}")
print("\n--- grep grep /.bash_history | head -n 3 ---")
result = await ws.execute("grep grep /.bash_history | head -n 3")
for line in (await result.stdout_str()).strip().splitlines():
print(f" {line[:120]}")
asyncio.run(main())
+101
View File
@@ -0,0 +1,101 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gdrive import GoogleDriveConfig, GoogleDriveResource
load_dotenv(".env.development")
config = GoogleDriveConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GoogleDriveResource(config=config)
async def main():
with Workspace({"/gdrive/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Drive transparently ===\n")
print("--- os.listdir() root ---")
entries = vos.listdir("/gdrive")
for e in entries[:10]:
print(f" {e}")
gdoc = gsheet = gslide = None
for e in entries:
if ".gdoc.json" in e and not gdoc:
gdoc = e
if ".gsheet.json" in e and not gsheet:
gsheet = e
if ".gslide.json" in e and not gslide:
gslide = e
if gdoc:
path = f"/gdrive/{gdoc}"
print(f"\n--- open() Google Doc: {gdoc} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" title: {parsed.get('title', 'N/A')}")
print(f" preview: {content[:200]}...")
if gsheet:
path = f"/gdrive/{gsheet}"
print(f"\n--- open() Google Sheet: {gsheet} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("properties", {}).get("title", "N/A")
num_sheets = len(parsed.get("sheets", []))
print(f" title: {title}")
print(f" sheets: {num_sheets}")
if gslide:
path = f"/gdrive/{gslide}"
print(f"\n--- open() Google Slides: {gslide} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" title: {parsed.get('title', 'N/A')}")
print(f" slides: {len(parsed.get('slides', []))}")
print("\n--- os.path.exists() ---")
if gdoc:
print(f" {gdoc}: {vos.path.exists(f'/gdrive/{gdoc}')}")
print(f" nonexistent: {vos.path.exists('/gdrive/nope.txt')}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+71
View File
@@ -0,0 +1,71 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
async def main():
ws = Workspace({"/gmail": resource}, mode=MountMode.WRITE)
r = await ws.execute("ls /gmail/")
print("=== labels ===")
print(await r.stdout_str())
r = await ws.execute("ls /gmail/INBOX/ | head -n 3")
print("=== INBOX (first 3) ===")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== plan: cat ===")
dr = await ws.execute(f"cat /gmail/INBOX/{first}", provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== cat message ===")
r = await ws.execute(f"cat /gmail/INBOX/{first}")
print((await r.stdout_str())[:500])
print("=== jq .subject ===")
r = await ws.execute(f'jq ".subject" /gmail/INBOX/{first}')
print(await r.stdout_str())
print("=== gws-gmail-triage ===")
r = await ws.execute('gws-gmail-triage --query "is:unread" --max 5')
print((await r.stdout_str())[:500])
print("=== gws-gmail-send ===")
r = await ws.execute(
'gws-gmail-send --to "zechengzhang97@gmail.com"'
' --subject "Hello from MIRAGE"'
' --body "This email was sent via the MIRAGE Gmail resource."')
print((await r.stdout_str())[:200])
if __name__ == "__main__":
asyncio.run(main())
+88
View File
@@ -0,0 +1,88 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
async def show(ws, cmd):
print(f"\n$ {cmd}")
r = await ws.execute(cmd)
out = await r.stdout_str()
err = await r.stderr_str()
if out:
print(f"STDOUT:\n{out}")
if err:
print(f"STDERR:\n{err}")
print(f"exit={r.exit_code}")
return out, err, r.exit_code
async def main():
ws = Workspace({"/gmail": resource}, mode=MountMode.READ)
out, _, _ = await show(ws, "ls /gmail/INBOX/ | head -5")
dates = [d for d in out.strip().split("\n") if d]
if not dates:
print("(no dates in INBOX)")
return
first_date = dates[0]
# A date dir lists message files (*.gmail.json); a message that has
# attachments also gets a sibling folder named after the message.
out, _, _ = await show(ws, f"ls /gmail/INBOX/{first_date}")
entries = [e for e in out.strip().split("\n") if e]
msg_file = next((e for e in entries if e.endswith(".gmail.json")), None)
assert msg_file, f"date dir should contain a *.gmail.json msg: {entries}"
await show(
ws, f'cat "/gmail/INBOX/{first_date}/{msg_file}" '
"| jq '{subject, from: .from.email, "
"attachments: [.attachments[].filename]}'")
# Find a message with attachments: a date-dir entry without the
# .gmail.json suffix is the attachment folder for the matching message,
# so "<name>.gmail.json" is its message file.
print("\n=== finding a message with attachments ===")
for d in dates:
r = await ws.execute(f"ls /gmail/INBOX/{d}")
items = [e for e in (await r.stdout_str()).strip().split("\n") if e]
att_dir = next((e for e in items if not e.endswith(".gmail.json")),
None)
if att_dir:
print(f"FOUND: /gmail/INBOX/{d}/{att_dir}")
await show(ws, f'ls "/gmail/INBOX/{d}/{att_dir}"')
await show(
ws, f'cat "/gmail/INBOX/{d}/{att_dir}.gmail.json" '
"| jq '.attachments'")
return
print("(no attachments found in scanned dates)")
if __name__ == "__main__":
asyncio.run(main())
+85
View File
@@ -0,0 +1,85 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gmail import GmailConfig, GmailResource
load_dotenv(".env.development")
config = GmailConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GmailResource(config=config)
async def main():
with Workspace({"/gmail/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE: open() reads from Gmail transparently ===\n")
print("--- os.listdir() labels ---")
labels = vos.listdir("/gmail")
for label in labels:
print(f" {label}")
print("\n--- os.listdir() INBOX (date folders) ---")
dates = vos.listdir("/gmail/INBOX")
for date in dates[:5]:
print(f" {date}")
first_date = dates[0] if dates else None
entries = vos.listdir(
f"/gmail/INBOX/{first_date}") if first_date else []
messages = [e for e in entries if e.endswith(".gmail.json")]
for msg in messages[:5]:
print(f" {first_date}/{msg}")
if messages:
first = messages[0]
path = f"/gmail/INBOX/{first_date}/{first}"
print("\n--- open() + read first message ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
print(f" subject: {parsed.get('subject', 'N/A')}")
print(f" from: {parsed.get('from', 'N/A')}")
print(f" snippet: {parsed.get('snippet', '')[:120]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gmail/INBOX/nope.json')}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+124
View File
@@ -0,0 +1,124 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
load_dotenv(".env.development")
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
async def main():
ws = Workspace({"/gsheets": resource}, mode=MountMode.WRITE)
r = await ws.execute("ls /gsheets/owned/ | head -n 3")
print("=== ls (first 3) ===")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== plan: cat ===")
dr = await ws.execute(f"cat /gsheets/owned/{first}", provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== plan: grep ===")
dr = await ws.execute(f"grep title /gsheets/owned/{first}", provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== jq .properties.title ===")
r = await ws.execute(f'jq ".properties.title" /gsheets/owned/{first}')
print(await r.stdout_str())
print('=== jq ".sheets | length" ===')
r = await ws.execute(f'jq ".sheets | length" /gsheets/owned/{first}')
print(await r.stdout_str())
print("=== head -c 200 ===")
r = await ws.execute(f"head -c 200 /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== grep title ===")
r = await ws.execute(f"grep title /gsheets/owned/{first} | head -c 200")
print(await r.stdout_str())
print("=== tail -c 200 ===")
r = await ws.execute(f"tail -c 200 /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== gws-sheets-spreadsheets-create ===")
body = json.dumps({"properties": {"title": "MIRAGE Sheets Test"}})
r = await ws.execute("gws-sheets-spreadsheets-create"
f" --json '{body}'")
out = await r.stdout_str()
if r.exit_code != 0 or not out.strip():
err = (await r.stderr_str()).strip() or "empty response"
print(f" create failed (exit={r.exit_code}): {err}")
return
sheet = json.loads(out)
sheet_id = sheet["spreadsheetId"]
print(f"Created: {sheet_id}")
print("\n=== gws-sheets-write ===")
params = json.dumps({
"spreadsheetId": sheet_id,
"range": "Sheet1!A1",
"valueInputOption": "USER_ENTERED",
})
values = json.dumps({
"values": [
["Name", "Age", "City"],
["Alice", "30", "NYC"],
["Bob", "25", "SF"],
]
})
r = await ws.execute(f"gws-sheets-write"
f" --params '{params}' --json '{values}'")
print(f"Written: {(await r.stdout_str())[:80]}")
print("\n=== gws-sheets-read ===")
r = await ws.execute(f'gws-sheets-read'
f' --spreadsheet {sheet_id}'
f' --range "Sheet1!A1:C3"')
print(f"Values: {await r.stdout_str()}")
print("=== gws-sheets-append ===")
r = await ws.execute(f"gws-sheets-append"
f" --spreadsheet {sheet_id}"
f" --values Diana,28,Chicago")
print(f"Appended: {(await r.stdout_str())[:80]}")
print("\n=== gws-sheets-read (all) ===")
r = await ws.execute(f'gws-sheets-read'
f' --spreadsheet {sheet_id}'
f' --range Sheet1')
print(f"All: {await r.stdout_str()}")
url = f"https://docs.google.com/spreadsheets/d/{sheet_id}"
print(f"\nOpen: {url}")
if __name__ == "__main__":
asyncio.run(main())
+83
View File
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
load_dotenv(".env.development")
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
async def main():
with Workspace({"/gsheets/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Sheets transparently ===\n"
)
print("--- os.listdir() root ---")
dirs = vos.listdir("/gsheets")
for d in dirs:
print(f" {d}")
print("\n--- os.listdir() owned ---")
sheets = vos.listdir("/gsheets/owned")
for s in sheets[:5]:
print(f" {s}")
if sheets:
first = sheets[0]
path = f"/gsheets/owned/{first}"
print("\n--- open() + read first spreadsheet ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("properties", {}).get("title", "N/A")
num_sheets = len(parsed.get("sheets", []))
print(f" title: {title}")
print(f" sheets: {num_sheets}")
print(f" preview: {content[:200]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gsheets/owned/nope.json')}"
)
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+103
View File
@@ -0,0 +1,103 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
load_dotenv(".env.development")
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
async def main():
ws = Workspace({"/gslides": resource}, mode=MountMode.WRITE)
r = await ws.execute("ls /gslides/owned/ | head -n 3")
print("=== ls (first 3) ===")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== plan: cat ===")
dr = await ws.execute(f"cat /gslides/owned/{first}", provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== plan: grep ===")
dr = await ws.execute(f"grep textRun /gslides/owned/{first}",
provision=True)
print(f" network_read={dr.network_read}, precision={dr.precision}")
print("=== jq .title ===")
r = await ws.execute(f'jq ".title" /gslides/owned/{first}')
print(await r.stdout_str())
print('=== jq ".slides | length" ===')
r = await ws.execute(f'jq ".slides | length" /gslides/owned/{first}')
print(await r.stdout_str())
print("=== head -c 200 ===")
r = await ws.execute(f"head -c 200 /gslides/owned/{first}")
print(await r.stdout_str())
print("=== grep textRun ===")
r = await ws.execute(f"grep textRun /gslides/owned/{first} | head -c 200")
print(await r.stdout_str())
print("=== tail -c 200 ===")
r = await ws.execute(f"tail -c 200 /gslides/owned/{first}")
print(await r.stdout_str())
print("=== gws-slides-presentations-create ===")
r = await ws.execute('gws-slides-presentations-create'
' --json \'{"title": "MIRAGE Slides Test"}\'')
pres = json.loads(await r.stdout_str())
pres_id = pres["presentationId"]
print(f"Created: {pres_id}")
print("\n=== gws-slides-presentations-batchUpdate ===")
body = json.dumps({
"requests": [{
"createSlide": {
"insertionIndex": 1,
"slideLayoutReference": {
"predefinedLayout": "BLANK"
},
}
}]
})
params = json.dumps({"presentationId": pres_id})
r = await ws.execute("gws-slides-presentations-batchUpdate"
f" --params '{params}' --json '{body}'")
update = json.loads(await r.stdout_str())
slide_id = update["replies"][0]["createSlide"]["objectId"]
print(f"Added slide: {slide_id}")
url = (f"https://docs.google.com/presentation/"
f"d/{pres_id}/edit")
print(f"\nOpen: {url}")
if __name__ == "__main__":
asyncio.run(main())
+83
View File
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
load_dotenv(".env.development")
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
async def main():
with Workspace({"/gslides/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Slides transparently ===\n"
)
print("--- os.listdir() root ---")
dirs = vos.listdir("/gslides")
for d in dirs:
print(f" {d}")
print("\n--- os.listdir() owned ---")
presentations = vos.listdir("/gslides/owned")
for p in presentations[:5]:
print(f" {p}")
if presentations:
first = presentations[0]
path = f"/gslides/owned/{first}"
print("\n--- open() + read first presentation ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("title", "N/A")
num_slides = len(parsed.get("slides", []))
print(f" title: {title}")
print(f" slides: {num_slides}")
print(f" preview: {content[:200]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gslides/owned/nope.json')}"
)
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+181
View File
@@ -0,0 +1,181 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
load_dotenv(".env.development")
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
async def main() -> None:
ws = Workspace({"/gsheets": resource}, mode=MountMode.WRITE)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /gsheets/__nf_missing__.txt",
"head /gsheets/__nf_missing__.txt",
"stat /gsheets/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=== ls /gsheets/ ===")
r = await ws.execute("ls /gsheets/")
print(await r.stdout_str())
print("=== ls /gsheets/owned/ (first 5) ===")
r = await ws.execute("ls /gsheets/owned/ | head -n 5")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== cat ===")
r = await ws.execute(f"cat /gsheets/owned/{first}")
print((await r.stdout_str())[:300])
print("\n=== head -n 20 ===")
r = await ws.execute(f"head -n 20 /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== tail -n 10 ===")
r = await ws.execute(f"tail -n 10 /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== wc ===")
r = await ws.execute(f"wc /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== stat ===")
r = await ws.execute(f"stat /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== jq .properties.title ===")
r = await ws.execute(f'jq ".properties.title" /gsheets/owned/{first}')
print(await r.stdout_str())
print('=== jq ".sheets | length" ===')
r = await ws.execute(f'jq ".sheets | length" /gsheets/owned/{first}')
print(await r.stdout_str())
print("=== nl ===")
r = await ws.execute(f"nl /gsheets/owned/{first} | head -n 10")
print(await r.stdout_str())
print("=== tree /gsheets/ ===")
r = await ws.execute("tree /gsheets/")
print((await r.stdout_str())[:500])
print("\n=== find /gsheets/owned/ ===")
r = await ws.execute(
"find /gsheets/owned/ -name '*.gsheet.json' | head -n 5")
print(await r.stdout_str())
print("=== grep title ===")
r = await ws.execute(f"grep title /gsheets/owned/{first} | head -c 200")
print(await r.stdout_str())
print("\n=== rg title ===")
r = await ws.execute(f"rg title /gsheets/owned/{first} | head -c 200")
print(await r.stdout_str())
print("\n=== basename ===")
r = await ws.execute(f"basename /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== dirname ===")
r = await ws.execute(f"dirname /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== realpath ===")
r = await ws.execute(f"realpath /gsheets/owned/{first}")
print(await r.stdout_str())
print("=== gws-sheets-spreadsheets-create ===")
body = json.dumps({"properties": {"title": "MIRAGE Sheets Test"}})
r = await ws.execute("gws-sheets-spreadsheets-create"
f" --json '{body}'")
sheet = json.loads(await r.stdout_str())
sheet_id = sheet["spreadsheetId"]
print(f"Created: {sheet_id}")
print("\n=== gws-sheets-write ===")
params = json.dumps({
"spreadsheetId": sheet_id,
"range": "Sheet1!A1",
"valueInputOption": "USER_ENTERED",
})
values = json.dumps({
"values": [
["Name", "Age", "City"],
["Alice", "30", "NYC"],
["Bob", "25", "SF"],
]
})
r = await ws.execute(f"gws-sheets-write"
f" --params '{params}' --json '{values}'")
print(f"Written: {(await r.stdout_str())[:80]}")
print("\n=== gws-sheets-read ===")
r = await ws.execute(f'gws-sheets-read'
f' --spreadsheet {sheet_id}'
f' --range "Sheet1!A1:C3"')
print(f"Values: {await r.stdout_str()}")
print("=== gws-sheets-append ===")
r = await ws.execute(f"gws-sheets-append"
f" --spreadsheet {sheet_id}"
f" --values Diana,28,Chicago")
print(f"Appended: {(await r.stdout_str())[:80]}")
print("\n=== gws-sheets-read (all) ===")
r = await ws.execute(f'gws-sheets-read'
f' --spreadsheet {sheet_id}'
f' --range Sheet1')
print(f"All: {await r.stdout_str()}")
print("\n=== gws-sheets-spreadsheets-batchUpdate ===")
batch_body = json.dumps({
"requests": [{
"updateSpreadsheetProperties": {
"properties": {
"title": "MIRAGE Sheets Test (Updated)"
},
"fields": "title",
}
}]
})
batch_params = json.dumps({"spreadsheetId": sheet_id})
r = await ws.execute("gws-sheets-spreadsheets-batchUpdate"
f" --params '{batch_params}' --json '{batch_body}'")
print(f"BatchUpdate: {(await r.stdout_str())[:80]}")
url = f"https://docs.google.com/spreadsheets/d/{sheet_id}"
print(f"\nOpen: {url}")
if __name__ == "__main__":
asyncio.run(main())
+73
View File
@@ -0,0 +1,73 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
load_dotenv(".env.development")
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
with Workspace({"/gsheets/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() root ---")
roots = os.listdir(mp)
for r in roots:
print(f" {r}")
for section in ("owned", "shared"):
section_path = f"{mp}/{section}"
if not os.path.isdir(section_path):
continue
sheets = os.listdir(section_path)
if not sheets:
continue
print(f"\n--- os.listdir() {section} (first 5) ---")
for s in sheets[:5]:
print(f" {s}")
first = sheets[0]
path = f"{section_path}/{first}"
print(f"\n--- open() + read {first[:60]} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("properties", {}).get("title", "N/A")
print(f" title: {title}")
break
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> ls {mp}/owned/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+83
View File
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gsheets import GSheetsConfig, GSheetsResource
load_dotenv(".env.development")
config = GSheetsConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSheetsResource(config=config)
async def main() -> None:
with Workspace({"/gsheets/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Sheets transparently ===\n"
)
print("--- os.listdir() root ---")
dirs = vos.listdir("/gsheets")
for d in dirs:
print(f" {d}")
print("\n--- os.listdir() owned ---")
sheets = vos.listdir("/gsheets/owned")
for s in sheets[:5]:
print(f" {s}")
if sheets:
first = sheets[0]
path = f"/gsheets/owned/{first}"
print("\n--- open() + read first spreadsheet ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("properties", {}).get("title", "N/A")
num_sheets = len(parsed.get("sheets", []))
print(f" title: {title}")
print(f" sheets: {num_sheets}")
print(f" preview: {content[:200]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gsheets/owned/nope.json')}"
)
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+148
View File
@@ -0,0 +1,148 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
load_dotenv(".env.development")
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
async def main() -> None:
ws = Workspace({"/gslides": resource}, mode=MountMode.WRITE)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /gslides/__nf_missing__.txt",
"head /gslides/__nf_missing__.txt",
"stat /gslides/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=== ls /gslides/ ===")
r = await ws.execute("ls /gslides/")
print(await r.stdout_str())
print("=== ls /gslides/owned/ (first 5) ===")
r = await ws.execute("ls /gslides/owned/ | head -n 5")
print(await r.stdout_str())
first = (await r.stdout_str()).strip().split("\n")[0]
print("=== cat ===")
r = await ws.execute(f"cat /gslides/owned/{first}")
print((await r.stdout_str())[:300])
print("\n=== head -n 20 ===")
r = await ws.execute(f"head -n 20 /gslides/owned/{first}")
print(await r.stdout_str())
print("=== tail -n 10 ===")
r = await ws.execute(f"tail -n 10 /gslides/owned/{first}")
print(await r.stdout_str())
print("=== wc ===")
r = await ws.execute(f"wc /gslides/owned/{first}")
print(await r.stdout_str())
print("=== stat ===")
r = await ws.execute(f"stat /gslides/owned/{first}")
print(await r.stdout_str())
print("=== jq .title ===")
r = await ws.execute(f'jq ".title" /gslides/owned/{first}')
print(await r.stdout_str())
print('=== jq ".slides | length" ===')
r = await ws.execute(f'jq ".slides | length" /gslides/owned/{first}')
print(await r.stdout_str())
print("=== nl ===")
r = await ws.execute(f"nl /gslides/owned/{first} | head -n 10")
print(await r.stdout_str())
print("=== tree /gslides/ ===")
r = await ws.execute("tree /gslides/")
print((await r.stdout_str())[:500])
print("\n=== find /gslides/owned/ ===")
r = await ws.execute(
"find /gslides/owned/ -name '*.gslide.json' | head -n 5")
print(await r.stdout_str())
print("=== grep textRun ===")
r = await ws.execute(f"grep textRun /gslides/owned/{first} | head -c 200")
print(await r.stdout_str())
print("\n=== rg textRun ===")
r = await ws.execute(f"rg textRun /gslides/owned/{first} | head -c 200")
print(await r.stdout_str())
print("\n=== basename ===")
r = await ws.execute(f"basename /gslides/owned/{first}")
print(await r.stdout_str())
print("=== dirname ===")
r = await ws.execute(f"dirname /gslides/owned/{first}")
print(await r.stdout_str())
print("=== realpath ===")
r = await ws.execute(f"realpath /gslides/owned/{first}")
print(await r.stdout_str())
print("=== gws-slides-presentations-create ===")
r = await ws.execute('gws-slides-presentations-create'
' --json \'{"title": "MIRAGE Slides Test"}\'')
pres = json.loads(await r.stdout_str())
pres_id = pres["presentationId"]
print(f"Created: {pres_id}")
print("\n=== gws-slides-presentations-batchUpdate ===")
body = json.dumps({
"requests": [{
"createSlide": {
"insertionIndex": 1,
"slideLayoutReference": {
"predefinedLayout": "BLANK"
},
}
}]
})
params = json.dumps({"presentationId": pres_id})
r = await ws.execute("gws-slides-presentations-batchUpdate"
f" --params '{params}' --json '{body}'")
update = json.loads(await r.stdout_str())
slide_id = update["replies"][0]["createSlide"]["objectId"]
print(f"Added slide: {slide_id}")
url = (f"https://docs.google.com/presentation/"
f"d/{pres_id}/edit")
print(f"\nOpen: {url}")
if __name__ == "__main__":
asyncio.run(main())
+75
View File
@@ -0,0 +1,75 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
load_dotenv(".env.development")
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
with Workspace({"/gslides/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() root ---")
roots = os.listdir(mp)
for r in roots:
print(f" {r}")
for section in ("owned", "shared"):
section_path = f"{mp}/{section}"
if not os.path.isdir(section_path):
continue
slides = os.listdir(section_path)
if not slides:
continue
print(f"\n--- os.listdir() {section} (first 5) ---")
for s in slides[:5]:
print(f" {s}")
first = slides[0]
path = f"{section_path}/{first}"
print(f"\n--- open() + read {first[:60]} ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("title", "N/A")
num_slides = len(parsed.get("slides", []))
print(f" title: {title}")
print(f" slides: {num_slides}")
break
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> ls {mp}/owned/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
+83
View File
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.gslides import GSlidesConfig, GSlidesResource
load_dotenv(".env.development")
config = GSlidesConfig(
client_id=os.environ["GOOGLE_CLIENT_ID"],
client_secret=os.environ["GOOGLE_CLIENT_SECRET"],
refresh_token=os.environ["GOOGLE_REFRESH_TOKEN"],
)
resource = GSlidesResource(config=config)
async def main() -> None:
with Workspace({"/gslides/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(
"=== VFS MODE: open() reads from Google Slides transparently ===\n"
)
print("--- os.listdir() root ---")
dirs = vos.listdir("/gslides")
for d in dirs:
print(f" {d}")
print("\n--- os.listdir() owned ---")
presentations = vos.listdir("/gslides/owned")
for p in presentations[:5]:
print(f" {p}")
if presentations:
first = presentations[0]
path = f"/gslides/owned/{first}"
print("\n--- open() + read first presentation ---")
with open(path) as f:
content = f.read()
parsed = json.loads(content)
title = parsed.get("title", "N/A")
num_slides = len(parsed.get("slides", []))
print(f" title: {title}")
print(f" slides: {num_slides}")
print(f" preview: {content[:200]}...")
print("\n--- os.path.exists() ---")
print(f" {first}: {vos.path.exists(path)}")
print(
f" nonexistent: {vos.path.exists('/gslides/owned/nope.json')}"
)
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
asyncio.run(main())
+436
View File
@@ -0,0 +1,436 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
import uuid
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_buckets import HfBucketsConfig, HfBucketsResource
load_dotenv(".env.development")
config = HfBucketsConfig(
bucket=os.environ["HF_BUCKET_NAME"],
token=os.environ["HF_TOKEN"],
)
resource = HfBucketsResource(config)
ws = Workspace({"/hf/": resource}, mode=MountMode.WRITE)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
def show_plan(label: str, dr) -> None:
print(f"\n--- plan: {label} ---")
print(f" network_read: {dr.network_read} cache_read: {dr.cache_read}")
print(f" read_ops: {dr.read_ops} cache_hits: {dr.cache_hits} "
f"precision: {dr.precision}")
async def main():
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /hf/__nf_missing__.txt", "head /hf/__nf_missing__.txt",
"stat /hf/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── discover structure ────────────────────────────
print("=== ls /hf/ ===")
r = await ws.execute("ls /hf/")
print(await r.stdout_str())
# ── tree ──────────────────────────────────────────
print("=== tree /hf/ ===")
r = await ws.execute("tree /hf/")
print(await r.stdout_str())
# ── stat (root + file) ──────────────────────────────
print("=== stat /hf (root) ===")
r = await ws.execute("stat /hf")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== stat /hf/example.json ===")
r = await ws.execute("stat /hf/example.json")
print(f" {(await r.stdout_str()).strip()}")
# ── cat json ──────────────────────────────────────
print("\n=== cat /hf/example.json | head -n 10 ===")
r = await ws.execute("cat /hf/example.json | head -n 10")
print(await r.stdout_str())
# ── head / tail on jsonl ──────────────────────────
print("=== head -n 3 /hf/example.jsonl ===")
r = await ws.execute("head -n 3 /hf/example.jsonl")
print((await r.stdout_str())[:300])
print("\n=== tail -n 2 /hf/example.jsonl ===")
r = await ws.execute("tail -n 2 /hf/example.jsonl")
print((await r.stdout_str())[:300])
# ── wc ────────────────────────────────────────────
print("\n=== wc -l /hf/example.jsonl ===")
r = await ws.execute("wc -l /hf/example.jsonl")
print(f" {(await r.stdout_str()).strip()}")
# ── grep ──────────────────────────────────────────
print("\n=== grep -c mirage /hf/example.jsonl ===")
r = await ws.execute("grep -c mirage /hf/example.jsonl")
print(f" count: {(await r.stdout_str()).strip()}")
print("\n=== grep mirage /hf/example.jsonl | head -n 3 ===")
r = await ws.execute("grep mirage /hf/example.jsonl | head -n 3")
lines = (await r.stdout_str()).strip().splitlines()
for ln in lines:
print(f" {ln[:100]}...")
# ── find ──────────────────────────────────────────
print("\n=== find /hf/ -name '*.json' ===")
r = await ws.execute("find /hf/ -name '*.json'")
print(await r.stdout_str())
print("=== find /hf/ -name '*.parquet' ===")
r = await ws.execute("find /hf/ -name '*.parquet'")
print(await r.stdout_str())
# ── jq ────────────────────────────────────────────
print("=== jq .metadata /hf/example.json ===")
r = await ws.execute("jq .metadata /hf/example.json")
print(f" {(await r.stdout_str()).strip()[:200]}")
print("\n=== jq -r '.departments[].teams[].name'"
" /hf/example.json ===")
r = await ws.execute('jq ".departments[].teams[].name" /hf/example.json')
print(f" {(await r.stdout_str()).strip()}")
# ── pipelines ─────────────────────────────────────
print("\n=== cat example.jsonl | grep queue-operation"
" | sort | uniq | wc -l ===")
r = await ws.execute("cat /hf/example.jsonl"
" | grep queue-operation | sort | uniq | wc -l")
print(f" unique lines: {(await r.stdout_str()).strip()}")
# ── cd + relative paths ───────────────────────────
print("\n=== pwd ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print('\n=== cd /hf ===')
r = await ws.execute("cd /hf")
print(f" exit={r.exit_code}")
print("\n=== pwd (after cd) ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== ls (relative) ===")
r = await ws.execute("ls")
print(await r.stdout_str())
print("=== head -n 3 example.json (relative) ===")
r = await ws.execute("head -n 3 example.json")
print(await r.stdout_str())
# ── streaming & barrier scenarios ─────────────────
print("\n=== cat | grep | head (streaming drain) ===")
r = await ws.execute("cat /hf/example.jsonl | grep queue | head -n 3")
lines = (await r.stdout_str()).strip().splitlines()
print(f" got {len(lines)} lines (expected 3)")
print("\n=== grep -q && echo (barrier VALUE) ===")
r = await ws.execute('grep -q queue /hf/example.jsonl && echo "found"')
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== grep -q || echo (barrier OR) ===")
r = await ws.execute('grep -q NONEXISTENT_STRING /hf/example.jsonl'
' || echo "not found"')
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== grep ; grep (semicolon materialization) ===")
r = await ws.execute("grep -c queue /hf/example.jsonl"
"; grep -c mirage /hf/example.jsonl")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== grep missing ; echo $? (semicolon exit code) ===")
r = await ws.execute("grep NONEXISTENT_STRING /hf/example.jsonl"
"; echo $?")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== cat nonexistent 2>&1 | head (stderr in pipe) ===")
r = await ws.execute("cat /hf/nonexistent_file 2>&1 | head -n 1")
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== cat nonexistent | cat (no merge: error in stderr) ===")
r = await ws.execute("cat /hf/nonexistent_file | cat")
print(f" stdout: '{(await r.stdout_str()).strip()}' (expect empty)")
print(f" stderr: '{(await r.stderr_str()).strip()[:80]}'")
print("\n=== cat nonexistent 2>&1 | cat (no double-emit) ===")
r = await ws.execute("cat /hf/nonexistent_file 2>&1 | cat")
out = (await r.stdout_str()).strip()
err = (await r.stderr_str()).strip()
print(f" stdout: '{out[:80]}'")
print(f" stderr: '{err}' (expect empty: no double-emit)")
print("\n=== cat large 2>&1 | wc -l (streams real payload) ===")
r = await ws.execute("wc -l /hf/example.jsonl")
expected = int((await r.stdout_str()).strip().split()[0])
r = await ws.execute("cat /hf/example.jsonl 2>&1 | wc -l")
got = int((await r.stdout_str()).strip())
print(f" expected: {expected} got: {got} "
f"{'OK' if got == expected else 'MISMATCH'}")
print("\n=== cat | sort | uniq | wc -l (full pipeline) ===")
r = await ws.execute("cat /hf/example.jsonl | sort | uniq | wc -l")
print(f" unique lines: {(await r.stdout_str()).strip()}")
# ── background job scenarios ────────────────────
print("\n=== grep -c & echo kicked off; wait (bg job) ===")
r = await ws.execute("grep -c queue /hf/example.jsonl &"
" echo 'kicked off'; wait")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== sleep 0 & cat (bg doesn't consume stdin) ===")
r = await ws.execute("sleep 0 & cat /hf/example.json | head -n 1")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== cat nonexistent & echo ok (bg error handled) ===")
r = await ws.execute("cat /hf/nonexistent_file & echo ok; wait; echo done")
print(f" stdout: {(await r.stdout_str()).strip()}")
print(f" exit: {r.exit_code}")
print("\n=== multiple bg: grep & wc & wait (parallel) ===")
r = await ws.execute("grep -c queue /hf/example.jsonl &"
" wc -l /hf/example.jsonl &"
" wait; echo all done")
print(f" stdout: {(await r.stdout_str()).strip()}")
# ── lazy stdin in loops ───────────────────────────
print("\n=== head -n 5 | while read; do echo (bounded loop) ===")
r = await ws.execute("cat /hf/example.jsonl | head -n 5"
" | while read LINE; do echo got; done | wc -l")
print(f" iterations: {(await r.stdout_str()).strip()} (expected 5)")
print("\n=== while read; break (early exit) ===")
r = await ws.execute("cat /hf/example.jsonl | head -n 100"
" | while read LINE; do echo first; break; done")
out = (await r.stdout_str()).strip().splitlines()
print(f" stdout lines: {len(out)} (expected 1) exit={r.exit_code}")
print("\n=== for x in a b c; do read LINE (loop reads buffer) ===")
r = await ws.execute(
"cat /hf/example.jsonl | head -n 3"
" | for x in a b c; do read LINE; echo \"$x:${LINE:0:30}\"; done")
for line in (await r.stdout_str()).strip().splitlines():
print(f" {line}")
# ── quoting / escaping ────────────────────────────
print("\n=== echo \"\\$X\" (escaped dollar stays literal) ===")
await ws.execute("export X=expanded")
r = await ws.execute('echo "\\$X"')
print(f" stdout: {(await r.stdout_str()).strip()!r} (expect '$X')")
print("\n=== echo \"$X\" (unescaped dollar expands) ===")
r = await ws.execute('echo "$X"')
print(f" stdout: {(await r.stdout_str()).strip()!r} (expect 'expanded')")
print("\n=== echo '$X' (single quotes keep $X literal) ===")
r = await ws.execute("echo '$X'")
print(f" stdout: {(await r.stdout_str()).strip()!r} (expect '$X')")
print("\n=== cat \"$DIR/example.json\" (env var in path) ===")
await ws.execute("export DIR=/hf")
r = await ws.execute('cat "$DIR/example.json" | head -n 3')
out = (await r.stdout_str()).strip().splitlines()
print(f" first lines: {out}")
print("\n=== cat $(echo /hf/example.json) | head -n 1"
" (command sub as path) ===")
r = await ws.execute("cat $(echo /hf/example.json) | head -n 1")
print(f" stdout: {(await r.stdout_str()).strip()}")
print("\n=== grep \"$(echo queue)\" /hf/example.jsonl"
" | wc -l (sub as pattern) ===")
r = await ws.execute('grep "$(echo queue)" /hf/example.jsonl | wc -l')
print(f" count: {(await r.stdout_str()).strip()}")
# ── PROVISION: estimate cost without executing ────────────────────
# Returns a ProvisionResult with network_read / cache_read / precision
# instead of running the command. Provisioned commands:
# file_read_provision: cat, wc (full-file read, EXACT)
# head_tail_provision: head, tail (EXACT for -c, RANGE for -n)
# metadata_provision : find, ls, stat (0 bytes, EXACT)
# grep / jq : inline provisioners in their command files
# When a file is already in the workspace cache, the workspace
# automatically moves network_read into cache_read and zeros
# network_read. That's why we clear the cache first to see the
# "first-time" estimate, then re-plan after caching to see the diff.
print("\n=== PROVISION (plan without executing) ===")
await ws.cache.clear()
before_plan = ops_summary()
# ── single-command plans (cache cleared, so network_read shows up) ──
dr = await ws.execute("cat /hf/example.json", provision=True)
show_plan("cat /hf/example.json (file_read_provision)", dr)
dr = await ws.execute("wc -l /hf/example.jsonl", provision=True)
show_plan("wc -l /hf/example.jsonl (file_read_provision)", dr)
dr = await ws.execute("head -c 100 /hf/example.jsonl", provision=True)
show_plan("head -c 100 /hf/example.jsonl (byte budget, EXACT)", dr)
dr = await ws.execute("head -n 5 /hf/example.jsonl", provision=True)
show_plan("head -n 5 /hf/example.jsonl (line budget, RANGE)", dr)
print(f" range: [{dr.network_read_low}, {dr.network_read_high}]")
dr = await ws.execute("ls /hf/", provision=True)
show_plan("ls /hf/ (metadata_provision, 0 bytes)", dr)
dr = await ws.execute("stat /hf/example.json", provision=True)
show_plan("stat /hf/example.json (metadata_provision)", dr)
dr = await ws.execute("grep mirage /hf/example.jsonl", provision=True)
show_plan("grep mirage /hf/example.jsonl (grep_provision)", dr)
dr = await ws.execute("jq .metadata /hf/example.json", provision=True)
show_plan("jq .metadata /hf/example.json (jq_provision)", dr)
# ── compound plans ──
dr = await ws.execute("cat /hf/example.jsonl | head -n 3", provision=True)
print("\n--- plan: cat | head -n 3 (pipeline with children) ---")
print(f" op: {dr.op} children: {len(dr.children)} "
f"precision: {dr.precision}")
print(f" network_read: {dr.network_read} cache_read: {dr.cache_read}")
for c in dr.children:
print(f" {c.command}: net={c.network_read} "
f"cache={c.cache_read} {c.precision}")
dr = await ws.execute("grep mirage /hf/example.jsonl && echo found",
provision=True)
print("\n--- plan: grep ... && echo found (compound) ---")
print(f" op: {dr.op} network_read: {dr.network_read}")
for c in dr.children:
print(f" {c.command}: net={c.network_read} {c.precision}")
print(f"\n Stats before plans: {before_plan}")
print(f" Stats after plans: {ops_summary()} (planning is read-free)")
# ── cache-aware plan: same command after caching the file ──
print("\n--- caching: cat /hf/example.json | wc -c ---")
r = await ws.execute("cat /hf/example.json | wc -c")
print(f" bytes: {(await r.stdout_str()).strip()}")
dr = await ws.execute("cat /hf/example.json", provision=True)
print("\n--- plan after cache: cat /hf/example.json ---")
print(f" network_read: {dr.network_read} "
f"cache_read: {dr.cache_read} cache_hits: {dr.cache_hits}")
print(" (network shifted to cache: file is hot)")
# ── chunk-level streaming + multi-stage pipe backpressure ─────────
print("\n=== STREAMING (single command) ===")
target = "/hf/example.jsonl"
r = await ws.execute(f"stat -c '%s' {target}")
size = int((await r.stdout_str()).strip())
print(f" object size: {size:,} bytes")
async def measure(label: str, cmd: str) -> None:
before = sum(rec.bytes for rec in ws.ops.records)
t0 = time.monotonic()
r = await ws.execute(cmd)
dt = time.monotonic() - t0
net = sum(rec.bytes for rec in ws.ops.records) - before
head = (await r.stdout_str()).strip().splitlines()
first = head[0][:48] if head else ""
print(f" {label:42s} bytes={net:>10,} t={dt:4.2f}s "
f"lines={len(head):>4} out0={first!r}")
await ws.cache.clear()
await measure("head -n 1 (line-streamed)", f"head -n 1 {target}")
await ws.cache.clear()
await measure("head -c 100 (byte-range)", f"head -c 100 {target}")
await ws.cache.clear()
await measure("grep -m 1 (early-exit)", f"grep -m 1 mirage {target}")
print("\n=== STREAMING CHAIN (multi-stage pipe backpressure) ===")
await ws.cache.clear()
await measure("cat | head -n 1", f"cat {target} | head -n 1")
await ws.cache.clear()
await measure("cat | tr A-Z a-z | head -n 1",
f"cat {target} | tr A-Z a-z | head -n 1")
await ws.cache.clear()
await measure("cat | grep mirage | head -n 1",
f"cat {target} | grep mirage | head -n 1")
await ws.cache.clear()
await measure("4-stage: cat|tr|grep|head -n 1",
f"cat {target} | tr A-Z a-z | grep mirage | head -n 1")
await ws.cache.clear()
await measure(
"5-stage: cat|tr|grep|head|wc -l",
f"cat {target} | tr A-Z a-z | grep mirage | head -n 1 "
"| wc -l")
await ws.cache.clear()
await measure("non-cancellable: cat | wc -l", f"cat {target} | wc -l")
# WRITE + REMOVE flow. HF Buckets silently drops zero-byte uploads,
# so we use ws.ops.write to push non-empty bytes (touch would no-op).
# Demonstrates parent-dir index cache invalidation: without it, `ls`
# after a write would return stale entries.
test_file = f"/hf/test-{uuid.uuid4().hex[:8]}.txt"
test_file.rsplit("/", 1)[-1]
print(f"\n=== WRITE + REMOVE FLOW (using {test_file}) ===")
print(f"\n--- write '{test_file}' (14 bytes) ---")
await ws.ops.write(test_file, b"Hello, Mirage!")
print(" written")
print(f"\n--- stat {test_file} (should succeed) ---")
r = await ws.execute(f"stat {test_file}")
print(f" {(await r.stdout_str()).strip()}")
print(f"\n--- cat {test_file} ---")
r = await ws.execute(f"cat {test_file}")
print(f" {(await r.stdout_str()).strip()!r}")
print(f"\n--- rm {test_file} ---")
r = await ws.execute(f"rm {test_file}")
print(f" exit: {r.exit_code}")
print(f"\n--- stat {test_file} (should fail: not found) ---")
r = await ws.execute(f"stat {test_file}")
err = (await r.stderr_str()).strip()[:80]
print(f" exit: {r.exit_code} stderr: {err}")
# touch on an existing file is still tested (no-op when c=False but
# file exists, since exists() short-circuits).
print("\n--- touch on existing file (no-op) ---")
r = await ws.execute("touch /hf/example.json")
print(f" exit: {r.exit_code}")
print(f"\nStats: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,80 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.hf_buckets import HfBucketsConfig, HfBucketsResource
load_dotenv(".env.development")
config = HfBucketsConfig(
bucket=os.environ["HF_BUCKET_NAME"],
token=os.environ["HF_TOKEN"],
)
resource = HfBucketsResource(config)
with Workspace({"/hf/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print(f"--- os.listdir({mp}) ---")
root_entries = os.listdir(mp)
for e in root_entries:
print(f" {e}")
data_dir = mp
if "data" in root_entries and os.path.isdir(f"{mp}/data"):
data_dir = f"{mp}/data"
print(f"\n--- os.listdir({data_dir}) ---")
for e in os.listdir(data_dir):
print(f" {e}")
data_entries = os.listdir(data_dir)
target = None
for entry in data_entries:
if entry.endswith(".jsonl") or entry.endswith(".json"):
target = f"{data_dir}/{entry}"
break
if target:
print(f"\n--- open({target}) + read 3 lines ---")
with open(target) as f:
for i, line in enumerate(f):
if i >= 3:
break
print(f" [{i}] {line.strip()[:100]}")
print(f"\n--- os.path.getsize({target}) ---")
size = os.path.getsize(target)
print(f" size: {size} bytes")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/data/")
if target:
print(f">>> cat {target}")
print(">>> Press Enter to unmount and exit...")
try:
input()
except EOFError:
pass
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
@@ -0,0 +1,75 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_buckets import HfBucketsConfig, HfBucketsResource
load_dotenv(".env.development")
config = HfBucketsConfig(
bucket=os.environ["HF_BUCKET_NAME"],
token=os.environ["HF_TOKEN"],
)
resource = HfBucketsResource(config)
async def main():
with Workspace({"/hf/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS: open() reads from HF Bucket transparently ===")
print("\n--- os.listdir('/hf') ---")
root_entries = vos.listdir("/hf")
for e in root_entries:
print(f" {e}")
data_dir = "/hf"
if "data" in root_entries and vos.path.isdir("/hf/data"):
data_dir = "/hf/data"
print(f"\n--- os.listdir('{data_dir}') ---")
for e in vos.listdir(data_dir):
print(f" {e}")
entries = vos.listdir(data_dir)
target = None
for entry in entries:
if entry.endswith(".jsonl") or entry.endswith(".json"):
target = f"{data_dir}/{entry}"
break
if target:
print(f"\n--- read first 3 lines of {target} ---")
with open(target) as f:
for i, line in enumerate(f):
if i >= 3:
break
print(f" [{i}] {line.strip()[:100]}")
print("\n--- VFS commands ---")
r = await ws.execute(f"ls {data_dir}")
print(f" ls {data_dir}: {(await r.stdout_str()).strip()}")
if target:
r = await ws.execute(f"head -n 3 {target}")
print(f" head -n 3:\n{(await r.stdout_str()).rstrip()}")
r = await ws.execute(f"wc -l {target}")
print(f" wc -l: {(await r.stdout_str()).strip()}")
asyncio.run(main())
+235
View File
@@ -0,0 +1,235 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_datasets import HfDatasetsConfig, HfDatasetsResource
load_dotenv(".env.development")
config = HfDatasetsConfig(
repo_id=os.environ.get("HF_DATASET_REPO",
"AlienKevin/SWE-ZERO-12M-trajectories"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfDatasetsResource(config)
ws = Workspace({"/ds/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
def show_plan(label: str, dr) -> None:
print(f"\n--- plan: {label} ---")
print(f" network_read: {dr.network_read} cache_read: {dr.cache_read}")
print(f" read_ops: {dr.read_ops} cache_hits: {dr.cache_hits} "
f"precision: {dr.precision}")
async def main():
print(f"=== mounted {resource.accessor.bucket_uri} at /ds/ ===")
print("\n=== not-found errors show the full virtual path ===")
for cmd in ("cat /ds/__nf_missing__.txt", "head /ds/__nf_missing__.txt",
"stat /ds/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── discover structure ──────────────────────────────
print("\n=== ls /ds/ ===")
r = await ws.execute("ls /ds/")
print(await r.stdout_str())
print("=== ls -lh /ds/ ===")
r = await ws.execute("ls -lh /ds/")
print(await r.stdout_str())
print("=== tree -L 2 /ds/ ===")
r = await ws.execute("tree -L 2 /ds/")
print((await r.stdout_str())[:400])
# ── stat ────────────────────────────────────────────
print("\n=== stat /ds (root) ===")
r = await ws.execute("stat /ds")
print(f" {(await r.stdout_str()).strip()}")
print("\n=== stat /ds/README.md ===")
r = await ws.execute("stat /ds/README.md")
print(f" {(await r.stdout_str()).strip()}")
# ── find ────────────────────────────────────────────
print("\n=== find /ds/ -name '*.md' ===")
r = await ws.execute("find /ds/ -name '*.md'")
print(await r.stdout_str())
print("=== find /ds/ -type d ===")
r = await ws.execute("find /ds/ -type d")
print(await r.stdout_str())
print("=== find /ds/ -name '*.parquet' | head -n 5 ===")
r = await ws.execute("find /ds/ -name '*.parquet' | head -n 5")
print(await r.stdout_str())
print("=== find /ds/ -name '*.parquet' | wc -l ===")
r = await ws.execute("find /ds/ -name '*.parquet' | wc -l")
print(f" parquet shard count: {(await r.stdout_str()).strip()}")
# ── cat / head / tail / wc ──────────────────────────
print("\n=== cat /ds/README.md | head -n 20 ===")
r = await ws.execute("cat /ds/README.md | head -n 20")
print(await r.stdout_str())
print("=== wc -l /ds/README.md ===")
r = await ws.execute("wc -l /ds/README.md")
print(f" {(await r.stdout_str()).strip()}")
print("=== head -c 200 /ds/README.md (byte range) ===")
r = await ws.execute("head -c 200 /ds/README.md")
print(await r.stdout_str())
# ── grep ────────────────────────────────────────────
print("\n=== grep -c parquet /ds/README.md ===")
r = await ws.execute("grep -c parquet /ds/README.md")
print(f" count: {(await r.stdout_str()).strip()}")
print("=== grep -i license /ds/README.md ===")
r = await ws.execute("grep -i license /ds/README.md")
print(f" {(await r.stdout_str()).strip()}")
# ── pipelines ───────────────────────────────────────
print("\n=== find /ds/ -name '*.parquet' | sort | head -n 3 ===")
r = await ws.execute("find /ds/ -name '*.parquet' | sort | head -n 3")
print(await r.stdout_str())
print("=== cat /ds/README.md | grep -i tag | head -n 5 ===")
r = await ws.execute("cat /ds/README.md | grep -i tag | head -n 5")
print(await r.stdout_str())
# ── cd + relative paths ─────────────────────────────
print("=== pwd ===")
r = await ws.execute("pwd")
print(f" {(await r.stdout_str()).strip()}")
print("=== cd /ds; pwd; ls | head ===")
r = await ws.execute("cd /ds")
r = await ws.execute("pwd")
print(f" pwd: {(await r.stdout_str()).strip()}")
r = await ws.execute("ls | head -n 5")
print(f" ls (relative):\n{(await r.stdout_str()).rstrip()}")
# ── barriers + semicolons ───────────────────────────
print("\n=== grep -q parquet /ds/README.md && echo 'found' ===")
r = await ws.execute("grep -q parquet /ds/README.md && echo 'found'")
print(f" stdout: {(await r.stdout_str()).strip()} exit: {r.exit_code}")
print("=== grep -q nope /ds/README.md || echo 'absent' ===")
r = await ws.execute("grep -q nope /ds/README.md || echo 'absent'")
print(f" stdout: {(await r.stdout_str()).strip()} exit: {r.exit_code}")
print("=== grep -c parquet README ; wc -l README (semicolon) ===")
r = await ws.execute("grep -c parquet /ds/README.md"
"; wc -l /ds/README.md")
print(f" stdout: {(await r.stdout_str()).strip()}")
# ── quoting / escaping / command substitution ───────
print("\n=== quoting / $(...) / env vars ===")
await ws.execute("export TGT=/ds/README.md")
r = await ws.execute('cat "$TGT" | head -n 2')
print(f' cat "$TGT" | head -n 2:\n{(await r.stdout_str()).rstrip()}')
r = await ws.execute("head -n 1 $(echo /ds/README.md)")
print(f" head -n 1 $(echo /ds/README.md): "
f"{(await r.stdout_str()).strip()}")
# ── background jobs ─────────────────────────────────
print("\n=== background jobs ===")
r = await ws.execute("grep -c parquet /ds/README.md &"
" echo 'kicked off'; wait")
print(f" stdout: {(await r.stdout_str()).strip()}")
r = await ws.execute("grep -c parquet /ds/README.md &"
" wc -l /ds/README.md &"
" wait; echo all done")
print(f" parallel: {(await r.stdout_str()).strip()}")
# ── PROVISION (dry-run cost plans) ──────────────────
print("\n=== PROVISION (plan without executing) ===")
await ws.cache.clear()
before = ops_summary()
dr = await ws.execute("cat /ds/README.md", provision=True)
show_plan("cat /ds/README.md", dr)
dr = await ws.execute("head -c 100 /ds/README.md", provision=True)
show_plan("head -c 100 /ds/README.md (byte budget, EXACT)", dr)
dr = await ws.execute("head -n 5 /ds/README.md", provision=True)
show_plan("head -n 5 /ds/README.md (line budget, RANGE)", dr)
print(f" range: [{dr.network_read_low}, {dr.network_read_high}]")
dr = await ws.execute("ls /ds/", provision=True)
show_plan("ls /ds/ (metadata only, 0 bytes)", dr)
dr = await ws.execute("find /ds/ -name '*.parquet'", provision=True)
show_plan("find /ds/ -name '*.parquet' (metadata only)", dr)
print(f"\n before plans: {before}")
print(f" after plans: {ops_summary()} (planning is read-free)")
# ── streaming chain backpressure ────────────────────
print("\n=== STREAMING (chain backpressure) ===")
target = "/ds/README.md"
r = await ws.execute(f"stat -c '%s' {target}")
print(f" size: {(await r.stdout_str()).strip()} bytes")
async def measure(label: str, cmd: str) -> None:
before_bytes = sum(rec.bytes for rec in ws.ops.records)
t0 = time.monotonic()
r = await ws.execute(cmd)
dt = time.monotonic() - t0
net = sum(rec.bytes for rec in ws.ops.records) - before_bytes
out = (await r.stdout_str()).rstrip().splitlines()
print(f" {label:38s} bytes={net:>6,} t={dt:4.2f}s "
f"lines={len(out):>3}")
await ws.cache.clear()
await measure("head -n 1 (line streamed)", f"head -n 1 {target}")
await ws.cache.clear()
await measure("head -c 100 (byte range)", f"head -c 100 {target}")
await ws.cache.clear()
await measure("grep -m 1 (early exit)", f"grep -m 1 parquet {target}")
await ws.cache.clear()
await measure("cat | head -n 1", f"cat {target} | head -n 1")
await ws.cache.clear()
await measure("cat | tr A-Z a-z | head -n 1",
f"cat {target} | tr A-Z a-z | head -n 1")
await ws.cache.clear()
await measure("4-stage: cat|tr|grep|head -n 1",
f"cat {target} | tr A-Z a-z | grep parquet | head -n 1")
print(f"\nFinal: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,62 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.hf_datasets import HfDatasetsConfig, HfDatasetsResource
load_dotenv(".env.development")
config = HfDatasetsConfig(
repo_id=os.environ.get("HF_DATASET_REPO",
"AlienKevin/SWE-ZERO-12M-trajectories"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfDatasetsResource(config)
with Workspace({"/ds/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE: mounted at {mp} ===\n")
print(f"--- os.listdir({mp}) ---")
root_entries = os.listdir(mp)
for e in root_entries:
print(f" {e}")
target = f"{mp}/README.md"
if os.path.exists(target):
print(f"\n--- read first 5 lines of {target} ---")
with open(target) as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" [{i}] {line.rstrip()[:100]}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> In another terminal:")
print(f">>> ls {mp}/")
print(f">>> find {mp}/ -name '*.parquet' | head")
print(">>> Press Enter to unmount...")
try:
input()
except EOFError:
pass
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
@@ -0,0 +1,64 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_datasets import HfDatasetsConfig, HfDatasetsResource
load_dotenv(".env.development")
config = HfDatasetsConfig(
repo_id=os.environ.get("HF_DATASET_REPO",
"AlienKevin/SWE-ZERO-12M-trajectories"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfDatasetsResource(config)
async def main():
with Workspace({"/ds/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(f"=== VFS: {resource.accessor.bucket_uri} ===")
print("\n--- os.listdir('/ds') ---")
root_entries = vos.listdir("/ds")
for e in root_entries:
print(f" {e}")
if "README.md" in root_entries:
print("\n--- read first 5 lines of /ds/README.md ---")
with open("/ds/README.md") as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" [{i}] {line.rstrip()[:100]}")
if "data" in root_entries and vos.path.isdir("/ds/data"):
print("\n--- os.listdir('/ds/data') (first 5) ---")
for e in vos.listdir("/ds/data")[:5]:
print(f" {e}")
print("\n--- shell view ---")
r = await ws.execute("ls /ds/")
print(f" ls /ds/: {(await r.stdout_str()).strip()}")
r = await ws.execute("find /ds/ -name '*.parquet' | head -n 3")
print(f" parquet shards (first 3): {(await r.stdout_str()).strip()}")
asyncio.run(main())
+235
View File
@@ -0,0 +1,235 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_models import HfModelsConfig, HfModelsResource
load_dotenv(".env.development")
config = HfModelsConfig(
repo_id=os.environ.get("HF_MODEL_REPO", "sapientinc/HRM-Text-1B"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfModelsResource(config)
ws = Workspace({"/m/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
def show_plan(label: str, dr) -> None:
print(f"\n--- plan: {label} ---")
print(f" network_read: {dr.network_read} cache_read: {dr.cache_read}")
print(f" read_ops: {dr.read_ops} cache_hits: {dr.cache_hits} "
f"precision: {dr.precision}")
async def main():
print(f"=== mounted {resource.accessor.bucket_uri} at /m/ ===")
print("\n=== not-found errors show the full virtual path ===")
for cmd in ("cat /m/__nf_missing__.txt", "head /m/__nf_missing__.txt",
"stat /m/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── discover structure ──────────────────────────────
print("\n=== ls /m/ ===")
r = await ws.execute("ls /m/")
print(await r.stdout_str())
print("=== ls -lh /m/ (sizes; weights stay remote) ===")
r = await ws.execute("ls -lh /m/")
print(await r.stdout_str())
print("=== tree /m/ ===")
r = await ws.execute("tree /m/")
print(await r.stdout_str())
# ── stat (root + small + huge file) ─────────────────
print("\n=== stat /m/config.json ===")
r = await ws.execute("stat /m/config.json")
print(f" {(await r.stdout_str()).strip()}")
print("=== stat /m/model.safetensors (no download) ===")
r = await ws.execute("stat /m/model.safetensors")
print(f" {(await r.stdout_str()).strip()}")
print("=== stat -c '%s' /m/model.safetensors ===")
r = await ws.execute("stat -c '%s' /m/model.safetensors")
print(f" weights size (bytes): {(await r.stdout_str()).strip()}")
# ── cat the config (small + fast) ───────────────────
print("\n=== cat /m/config.json ===")
r = await ws.execute("cat /m/config.json")
print(await r.stdout_str())
print("=== wc -l /m/config.json ===")
r = await ws.execute("wc -l /m/config.json")
print(f" {(await r.stdout_str()).strip()}")
# ── jq: introspect the architecture ─────────────────
print("\n=== jq .model_type /m/config.json ===")
r = await ws.execute("jq .model_type /m/config.json")
print(f" {(await r.stdout_str()).strip()}")
print("=== jq .architectures /m/config.json ===")
r = await ws.execute("jq .architectures /m/config.json")
print(f" {(await r.stdout_str()).strip()}")
print("=== jq '{hidden_size, num_hidden_layers, num_attention_heads,"
" vocab_size}' /m/config.json ===")
r = await ws.execute("jq '{hidden_size, num_hidden_layers,"
" num_attention_heads, vocab_size}'"
" /m/config.json")
print(await r.stdout_str())
# ── tokenizer config ────────────────────────────────
print("=== cat /m/tokenizer_config.json | jq .pad_token_id ===")
r = await ws.execute("cat /m/tokenizer_config.json"
" | jq .pad_token_id 2>/dev/null"
" || echo '(no pad_token_id)'")
print(f" {(await r.stdout_str()).strip()}")
# ── README inspection ───────────────────────────────
print("\n=== head -n 30 /m/README.md ===")
r = await ws.execute("head -n 30 /m/README.md")
print(await r.stdout_str())
print("=== grep -ic license /m/README.md ===")
r = await ws.execute("grep -ic license /m/README.md")
print(f" matches: {(await r.stdout_str()).strip()}")
# ── sed: read-transform on the read-only HF mount ───
print("\n=== sed -n '1,3p' /m/config.json (print first 3 lines) ===")
r = await ws.execute("sed -n '1,3p' /m/config.json")
print(await r.stdout_str())
print("=== cat /m/config.json | sed 's/\"//g' | head -n 3"
" (strip quotes) ===")
r = await ws.execute("cat /m/config.json | sed 's/\"//g' | head -n 3")
print(await r.stdout_str())
# ── find variants ───────────────────────────────────
print("\n=== find /m/ -name '*.json' ===")
r = await ws.execute("find /m/ -name '*.json'")
print(await r.stdout_str())
print("=== find /m/ -name '*.safetensors' ===")
r = await ws.execute("find /m/ -name '*.safetensors'")
print(await r.stdout_str())
print("=== find /m/ -type f -size +1M ===")
r = await ws.execute("find /m/ -type f -size +1M")
print(await r.stdout_str())
# ── pipelines ───────────────────────────────────────
print("=== find /m/ -name '*.json' | sort | head ===")
r = await ws.execute("find /m/ -name '*.json' | sort | head")
print(await r.stdout_str())
print("=== cat /m/config.json | grep -c ':' ===")
r = await ws.execute("cat /m/config.json | grep -c ':'")
print(f" config keys: {(await r.stdout_str()).strip()}")
# ── barriers ────────────────────────────────────────
print("\n=== test for safetensors via grep -q ===")
r = await ws.execute("ls /m/ | grep -q safetensors"
" && echo 'has weights' || echo 'no weights'")
print(f" {(await r.stdout_str()).strip()}")
# ── quoting + command substitution ──────────────────
print("\n=== quoting + $(...) ===")
await ws.execute("export CFG=/m/config.json")
r = await ws.execute('jq .hidden_size "$CFG"')
print(f' jq .hidden_size "$CFG": {(await r.stdout_str()).strip()}')
r = await ws.execute("cat $(echo /m/config.json) | jq .num_hidden_layers")
print(f" via $(): {(await r.stdout_str()).strip()}")
# ── background jobs ─────────────────────────────────
print("\n=== background: jq fields in parallel ===")
r = await ws.execute("jq .hidden_size /m/config.json &"
" jq .num_hidden_layers /m/config.json &"
" jq .vocab_size /m/config.json &"
" wait; echo done")
print(f" stdout: {(await r.stdout_str()).strip()}")
# ── PROVISION ───────────────────────────────────────
print("\n=== PROVISION (plan without executing) ===")
await ws.cache.clear()
before = ops_summary()
dr = await ws.execute("cat /m/config.json", provision=True)
show_plan("cat /m/config.json (tiny)", dr)
dr = await ws.execute("cat /m/model.safetensors", provision=True)
show_plan("cat /m/model.safetensors (2GB! plan only, no read)", dr)
dr = await ws.execute("head -c 256 /m/model.safetensors", provision=True)
show_plan("head -c 256 /m/model.safetensors (byte range, 256B)", dr)
dr = await ws.execute("stat /m/model.safetensors", provision=True)
show_plan("stat /m/model.safetensors (metadata, 0 bytes)", dr)
dr = await ws.execute("jq .hidden_size /m/config.json", provision=True)
show_plan("jq .hidden_size /m/config.json", dr)
print(f"\n before plans: {before}")
print(f" after plans: {ops_summary()} (planning is read-free)")
# ── byte-range read of the safetensors header ───────
print("\n=== STREAMING (range reads on 2GB weights) ===")
async def measure(label: str, cmd: str) -> None:
before_bytes = sum(rec.bytes for rec in ws.ops.records)
t0 = time.monotonic()
r = await ws.execute(cmd)
dt = time.monotonic() - t0
net = sum(rec.bytes for rec in ws.ops.records) - before_bytes
out = (await r.stdout_str()).rstrip().splitlines()
first = (out[0][:40] + "..." if out else "")
print(f" {label:42s} bytes={net:>9,} t={dt:4.2f}s "
f"lines={len(out):>3} out0={first!r}")
await ws.cache.clear()
await measure("head -c 128 model.safetensors (range)",
"head -c 128 /m/model.safetensors | xxd | head -n 4")
await ws.cache.clear()
await measure("stat -c '%s' model.safetensors",
"stat -c '%s' /m/model.safetensors")
await ws.cache.clear()
await measure("cat config.json | jq .vocab_size",
"cat /m/config.json | jq .vocab_size")
await ws.cache.clear()
await measure("wc -c config.json", "wc -c /m/config.json")
print(f"\nFinal: {ops_summary()}")
print("(2GB safetensors never fully downloaded; only header range "
"+ configs read)")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,58 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.hf_models import HfModelsConfig, HfModelsResource
load_dotenv(".env.development")
config = HfModelsConfig(
repo_id=os.environ.get("HF_MODEL_REPO", "sapientinc/HRM-Text-1B"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfModelsResource(config)
with Workspace({"/m/": Mount(resource, mode=MountMode.READ, fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE: mounted at {mp} ===\n")
print(f"--- os.listdir({mp}) ---")
root_entries = os.listdir(mp)
for e in root_entries:
print(f" {e}")
if "config.json" in root_entries:
cfg_path = f"{mp}/config.json"
size = os.path.getsize(cfg_path)
print(f"\n--- {cfg_path} ({size} bytes) ---")
with open(cfg_path) as f:
print(f.read()[:400])
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> In another terminal:")
print(f">>> ls -lh {mp}/")
print(f">>> cat {mp}/config.json | jq .")
print(">>> Press Enter to unmount...")
try:
input()
except EOFError:
pass
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
@@ -0,0 +1,65 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_models import HfModelsConfig, HfModelsResource
load_dotenv(".env.development")
config = HfModelsConfig(
repo_id=os.environ.get("HF_MODEL_REPO", "sapientinc/HRM-Text-1B"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfModelsResource(config)
async def main():
with Workspace({"/m/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(f"=== VFS: {resource.accessor.bucket_uri} ===")
print("\n--- os.listdir('/m') ---")
root_entries = vos.listdir("/m")
for e in root_entries:
print(f" {e}")
if "config.json" in root_entries:
print("\n--- read /m/config.json + parse ---")
with open("/m/config.json") as f:
cfg = json.load(f)
for k in ("model_type", "architectures", "hidden_size",
"num_hidden_layers"):
if k in cfg:
print(f" {k}: {cfg[k]}")
weight_files = [e for e in root_entries if e.endswith(".safetensors")]
if weight_files:
print("\n--- weights present (sizes only, not downloaded) ---")
for wf in weight_files[:5]:
size = vos.path.getsize(f"/m/{wf}")
print(f" {wf}: {size:>12,} bytes")
print("\n--- shell view ---")
r = await ws.execute("ls -lh /m/")
print(await r.stdout_str())
asyncio.run(main())
+219
View File
@@ -0,0 +1,219 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_spaces import HfSpacesConfig, HfSpacesResource
load_dotenv(".env.development")
config = HfSpacesConfig(
repo_id=os.environ.get("HF_SPACE_REPO", "HuggingFaceBio/carbon-demo"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfSpacesResource(config)
ws = Workspace({"/s/": resource}, mode=MountMode.READ)
def ops_summary() -> str:
records = ws.ops.records
total = sum(r.bytes for r in records)
return f"{len(records)} ops, {total} bytes transferred"
def show_plan(label: str, dr) -> None:
print(f"\n--- plan: {label} ---")
print(f" network_read: {dr.network_read} cache_read: {dr.cache_read}")
print(f" read_ops: {dr.read_ops} cache_hits: {dr.cache_hits} "
f"precision: {dr.precision}")
async def main():
print(f"=== mounted {resource.accessor.bucket_uri} at /s/ ===")
print("\n=== not-found errors show the full virtual path ===")
for cmd in ("cat /s/__nf_missing__.txt", "head /s/__nf_missing__.txt",
"stat /s/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
# ── discover structure ──────────────────────────────
print("\n=== ls /s/ ===")
r = await ws.execute("ls /s/")
print(await r.stdout_str())
print("=== ls -lh /s/ ===")
r = await ws.execute("ls -lh /s/")
print(await r.stdout_str())
print("=== tree -L 2 /s/ ===")
r = await ws.execute("tree -L 2 /s/")
print((await r.stdout_str())[:600])
# ── stat ────────────────────────────────────────────
print("\n=== stat /s/README.md ===")
r = await ws.execute("stat /s/README.md")
print(f" {(await r.stdout_str()).strip()}")
# ── find variants ───────────────────────────────────
print("\n=== find /s/ -type d ===")
r = await ws.execute("find /s/ -type d")
print(await r.stdout_str())
print("=== find /s/ -name '*.py' ===")
r = await ws.execute("find /s/ -name '*.py'")
print(await r.stdout_str())
print("=== find /s/ -name '*.html' | wc -l ===")
r = await ws.execute("find /s/ -name '*.html' | wc -l")
print(f" html count: {(await r.stdout_str()).strip()}")
print("=== find /s/ -maxdepth 1 -type f ===")
r = await ws.execute("find /s/ -maxdepth 1 -type f")
print(await r.stdout_str())
# ── cat / head / tail / wc ──────────────────────────
print("\n=== cat /s/README.md | head -n 15 ===")
r = await ws.execute("cat /s/README.md | head -n 15")
print(await r.stdout_str())
print("=== wc -l /s/README.md ===")
r = await ws.execute("wc -l /s/README.md")
print(f" {(await r.stdout_str()).strip()}")
print("=== cat /s/requirements.txt ===")
r = await ws.execute("cat /s/requirements.txt 2>/dev/null"
" || echo '(no requirements.txt)'")
print((await r.stdout_str()).rstrip())
# ── grep across app code ────────────────────────────
print("\n=== grep -l import /s/*.py ===")
r = await ws.execute("grep -l import /s/*.py 2>/dev/null")
print(await r.stdout_str())
print("=== grep -c '^import\\|^from' /s/app.py ===")
r = await ws.execute("grep -c '^import\\|^from' /s/app.py"
" 2>/dev/null || echo 0")
print(f" import lines: {(await r.stdout_str()).strip()}")
print("=== grep -ic flask /s/app.py ===")
r = await ws.execute("grep -ic flask /s/app.py 2>/dev/null || echo 0")
print(f" flask matches: {(await r.stdout_str()).strip()}")
# ── pipelines ───────────────────────────────────────
print("\n=== find /s/ -name '*.py' | sort | head -n 5 ===")
r = await ws.execute("find /s/ -name '*.py' | sort | head -n 5")
print(await r.stdout_str())
print("=== cat /s/README.md | grep -i '^#' | head -n 10 ===")
r = await ws.execute("cat /s/README.md | grep -i '^#' | head -n 10")
print(await r.stdout_str())
# ── cd + relative paths ─────────────────────────────
print("=== cd /s; pwd; ls | head ===")
await ws.execute("cd /s")
r = await ws.execute("pwd")
print(f" pwd: {(await r.stdout_str()).strip()}")
r = await ws.execute("ls | head -n 5")
print(f" ls (relative):\n{(await r.stdout_str()).rstrip()}")
# ── barriers + semicolons ───────────────────────────
print("\n=== grep -q app /s/README.md && echo 'mentions app' ===")
r = await ws.execute("grep -q app /s/README.md && echo 'mentions app'")
print(f" stdout: {(await r.stdout_str()).strip()} exit: {r.exit_code}")
print("=== grep -q nonexistent /s/README.md || echo 'absent' ===")
r = await ws.execute("grep -q nonexistent /s/README.md"
" || echo 'absent'")
print(f" stdout: {(await r.stdout_str()).strip()} exit: {r.exit_code}")
# ── quoting + command substitution ──────────────────
print("\n=== quoting + $() ===")
await ws.execute("export README=/s/README.md")
r = await ws.execute('wc -l "$README"')
print(f' wc -l "$README": {(await r.stdout_str()).strip()}')
r = await ws.execute("head -n 1 $(echo /s/README.md)")
print(f" head -n 1 $(echo /s/README.md): "
f"{(await r.stdout_str()).strip()}")
# ── background jobs ─────────────────────────────────
print("\n=== background: wc + grep in parallel ===")
r = await ws.execute("wc -l /s/README.md &"
" grep -c '^#' /s/README.md &"
" wait; echo done")
print(f" stdout: {(await r.stdout_str()).strip()}")
# ── PROVISION ───────────────────────────────────────
print("\n=== PROVISION (plan without executing) ===")
await ws.cache.clear()
before = ops_summary()
dr = await ws.execute("cat /s/README.md", provision=True)
show_plan("cat /s/README.md", dr)
dr = await ws.execute("head -c 100 /s/README.md", provision=True)
show_plan("head -c 100 /s/README.md (byte budget, EXACT)", dr)
dr = await ws.execute("ls /s/", provision=True)
show_plan("ls /s/ (metadata only)", dr)
dr = await ws.execute("find /s/ -name '*.py'", provision=True)
show_plan("find /s/ -name '*.py' (metadata only)", dr)
dr = await ws.execute("grep -l import /s/*.py", provision=True)
show_plan("grep -l import /s/*.py", dr)
print(f"\n before plans: {before}")
print(f" after plans: {ops_summary()} (planning is read-free)")
# ── streaming chains ────────────────────────────────
print("\n=== STREAMING (chain backpressure) ===")
target = "/s/README.md"
async def measure(label: str, cmd: str) -> None:
before_bytes = sum(rec.bytes for rec in ws.ops.records)
t0 = time.monotonic()
r = await ws.execute(cmd)
dt = time.monotonic() - t0
net = sum(rec.bytes for rec in ws.ops.records) - before_bytes
out = (await r.stdout_str()).rstrip().splitlines()
print(f" {label:38s} bytes={net:>6,} t={dt:4.2f}s "
f"lines={len(out):>3}")
await ws.cache.clear()
await measure("head -n 1 (line streamed)", f"head -n 1 {target}")
await ws.cache.clear()
await measure("head -c 100 (byte range)", f"head -c 100 {target}")
await ws.cache.clear()
await measure("cat | head -n 1", f"cat {target} | head -n 1")
await ws.cache.clear()
await measure("cat | grep '^#' | head -n 1",
f"cat {target} | grep '^#' | head -n 1")
await ws.cache.clear()
await measure("4-stage: cat|tr|grep|head -n 1",
f"cat {target} | tr A-Z a-z | grep app | head -n 1")
print(f"\nFinal: {ops_summary()}")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,61 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.hf_spaces import HfSpacesConfig, HfSpacesResource
load_dotenv(".env.development")
config = HfSpacesConfig(
repo_id=os.environ.get("HF_SPACE_REPO", "HuggingFaceBio/carbon-demo"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfSpacesResource(config)
with Workspace({"/s/": Mount(resource, mode=MountMode.READ, fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE: mounted at {mp} ===\n")
print(f"--- os.listdir({mp}) ---")
root_entries = os.listdir(mp)
for e in root_entries:
print(f" {e}")
readme = f"{mp}/README.md"
if os.path.exists(readme):
size = os.path.getsize(readme)
print(f"\n--- {readme} ({size} bytes), first 5 lines ---")
with open(readme) as f:
for i, line in enumerate(f):
if i >= 5:
break
print(f" [{i}] {line.rstrip()[:120]}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> In another terminal:")
print(f">>> ls {mp}/")
print(f">>> cat {mp}/README.md")
print(">>> Press Enter to unmount...")
try:
input()
except EOFError:
pass
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes transferred")
@@ -0,0 +1,61 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.hf_spaces import HfSpacesConfig, HfSpacesResource
load_dotenv(".env.development")
config = HfSpacesConfig(
repo_id=os.environ.get("HF_SPACE_REPO", "HuggingFaceBio/carbon-demo"),
token=os.environ.get("HF_TOKEN"),
)
resource = HfSpacesResource(config)
async def main():
with Workspace({"/s/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print(f"=== VFS: {resource.accessor.bucket_uri} ===")
print("\n--- os.listdir('/s') ---")
root_entries = vos.listdir("/s")
for e in root_entries:
print(f" {e}")
if "README.md" in root_entries:
print("\n--- read first 8 lines of /s/README.md ---")
with open("/s/README.md") as f:
for i, line in enumerate(f):
if i >= 8:
break
print(f" [{i}] {line.rstrip()[:120]}")
if "requirements.txt" in root_entries:
print("\n--- /s/requirements.txt ---")
with open("/s/requirements.txt") as f:
print(f.read().rstrip())
print("\n--- shell view ---")
r = await ws.execute("find /s/ -name '*.py' | head -n 5")
print(f" python files: {(await r.stdout_str()).strip()}")
asyncio.run(main())
+98
View File
@@ -0,0 +1,98 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import lancedb
import pyarrow as pa
from lancedb.embeddings import EmbeddingFunction, get_registry
from lancedb.pydantic import LanceModel, Vector
FASHION_VOCAB = [
"men", "women", "tshirt", "shirt", "jeans", "shoes", "sneakers", "heels",
"jacket", "dress", "blue", "red", "black", "white", "green", "running",
"casual", "formal", "sports", "summer", "winter"
]
_INDEX = {token: i for i, token in enumerate(FASHION_VOCAB)}
_PRODUCTS = [
("Men", "Tshirts", "Blue", "Roadster Men Blue Casual Tshirt"),
("Men", "Tshirts", "Black", "HRX Men Black Sports Tshirt"),
("Men", "Shoes", "White", "Nike Men White Running Sneakers"),
("Men", "Shoes", "Black", "Puma Men Black Formal Shoes"),
("Men", "Jeans", "Blue", "Levis Men Blue Casual Jeans"),
("Women", "Tshirts", "Red", "Roadster Women Red Casual Tshirt"),
("Women", "Shoes", "Red", "Steve Madden Women Red Heels"),
("Women", "Shoes", "White", "Adidas Women White Running Sneakers"),
("Women", "Dress", "Black", "Zara Women Black Formal Dress"),
("Women", "Jeans", "Blue", "H&M Women Blue Summer Jeans"),
]
def _tokens(text: str) -> list[str]:
return [word.strip(".,").lower() for word in text.split()]
def _embed(text: str) -> list[float]:
vector = [0.0] * len(FASHION_VOCAB)
for token in _tokens(text):
idx = _INDEX.get(token)
if idx is not None:
vector[idx] = 1.0
norm = sum(value * value for value in vector)**0.5 or 1.0
return [value / norm for value in vector]
@get_registry().register("fashion-keyword")
class KeywordEmbedding(EmbeddingFunction):
def ndims(self) -> int:
return len(FASHION_VOCAB)
def compute_query_embeddings(self, query, *args, **kwargs):
if isinstance(query, str):
return [_embed(query)]
return [_embed(str(item)) for item in query]
def compute_source_embeddings(self, texts, *args, **kwargs):
items = texts.to_pylist() if isinstance(texts,
pa.Array) else list(texts)
return [_embed(str(item)) for item in items]
def build_table(uri: str, table_name: str = "fashion") -> None:
func = get_registry().get("fashion-keyword").create()
class Product(LanceModel):
id: int
gender: str
articleType: str
baseColour: str
productDisplayName: str = func.SourceField()
image_bytes: bytes
vector: Vector(func.ndims()) = func.VectorField()
db = lancedb.connect(uri)
if table_name in db.table_names():
db.drop_table(table_name)
table = db.create_table(table_name, schema=Product)
rows = []
for idx, (gender, article, colour, name) in enumerate(_PRODUCTS, start=1):
rows.append({
"id": idx,
"gender": gender,
"articleType": article,
"baseColour": colour,
"productDisplayName": name,
"image_bytes": b"\xff\xd8\xff" + name.encode(),
})
table.add(rows)
@@ -0,0 +1,72 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import tempfile
from build_table import build_table
from mirage import MountMode, Workspace
from mirage.resource.lancedb import LanceDBConfig, LanceDBResource
async def show(ws: Workspace, cmd: str) -> None:
print(f"\n=== {cmd} ===")
result = await ws.execute(cmd)
print((await result.stdout_str()).rstrip())
async def main() -> None:
uri = tempfile.mkdtemp(prefix="mirage-fashion-")
build_table(uri, "fashion")
config = LanceDBConfig(
uri=uri,
table="fashion",
group_by=["gender", "articleType", "baseColour"],
id_column="id",
title_column="productDisplayName",
blob_column="image_bytes",
blob_ext="jpg",
text_column="productDisplayName",
vector_column="vector",
search_limit=4,
)
ws = Workspace({"/fashion/": LanceDBResource(config)}, mode=MountMode.READ)
print(f"=== mounted LanceDB table 'fashion' ({uri}) at /fashion/ ===")
await show(ws, "ls /fashion/")
await show(ws, "tree -L 2 /fashion/")
await show(ws, "ls /fashion/Men/Shoes")
await show(ws, "cat /fashion/Men/Shoes/White/3.md")
await show(ws, "head -n 3 /fashion/Men/Shoes/White/3.md")
await show(ws, "tail -n 2 /fashion/Men/Shoes/White/3.md")
print("\n=== stat /fashion/Men/Shoes/White/3.jpg (raw image bytes) ===")
r = await ws.execute("stat -c '%s' /fashion/Men/Shoes/White/3.jpg")
print(f" image size: {(await r.stdout_str()).strip()} bytes")
await show(ws, 'search "white running sneakers" /fashion')
await show(ws, "grep -ril blue /fashion/Women")
await show(ws, "rg -li running /fashion/Men")
print("\n=== find /fashion -name '*.md' | wc -l ===")
r = await ws.execute("find /fashion -name '*.md' | wc -l")
print(f" product cards: {(await r.stdout_str()).strip()}")
if __name__ == "__main__":
asyncio.run(main())
+227
View File
@@ -0,0 +1,227 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.langfuse import LangfuseConfig, LangfuseResource
load_dotenv(".env.development")
config = LangfuseConfig(
public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
secret_key=os.environ["LANGFUSE_SECRET_KEY"],
host=os.environ["LANGFUSE_HOST"],
default_trace_limit=20,
)
resource = LangfuseResource(config=config)
async def _run(ws, cmd):
print(f"\n>>> {cmd}")
r = await ws.execute(cmd)
out = (await r.stdout_str()).strip()
err = await r.stderr_str()
if out:
for line in out.splitlines()[:10]:
print(f" {line[:120]}")
total = len(out.splitlines())
if total > 10:
print(f" ... ({total} lines total)")
if err:
print(f" [stderr] {err.strip()[:120]}")
if not out and not err:
print(f" (empty, exit={r.exit_code})")
return r
async def _timed(ws, cmd):
start = time.perf_counter()
out = await (await ws.execute(cmd)).stdout_str()
return (time.perf_counter() - start) * 1000, out
async def main():
ws = Workspace({"/langfuse": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /langfuse/__nf_missing__.txt",
"head /langfuse/__nf_missing__.txt",
"stat /langfuse/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=" * 60)
print("LS across all directories")
print("=" * 60)
await _run(ws, "ls /langfuse/")
await _run(ws, "ls /langfuse/traces/")
await _run(ws, "ls /langfuse/sessions/")
await _run(ws, "ls /langfuse/prompts/")
await _run(ws, "ls /langfuse/datasets/")
print("\n" + "=" * 60)
print("CAT across different resource types")
print("=" * 60)
r = await ws.execute("ls /langfuse/traces/")
traces_out = (await r.stdout_str()).strip()
trace_files = [f for f in traces_out.splitlines() if f.strip()]
if trace_files:
tp = f"/langfuse/traces/{trace_files[0].strip()}"
await _run(ws, f'cat "{tp}" | head -n 5')
else:
print(" no traces found, skipping cat trace")
await _run(ws, 'cat "/langfuse/prompts/summarize/1.json"')
await _run(ws, 'cat "/langfuse/datasets/qa-eval/items.jsonl"')
print("\n" + "=" * 60)
print("GREP across different scopes")
print("=" * 60)
await _run(ws, 'grep "chat" "/langfuse/traces/"')
await _run(ws, 'grep "support" "/langfuse/traces/"')
await _run(ws, 'grep "chat-session" "/langfuse/sessions/"')
await _run(ws, 'grep "summarize" "/langfuse/prompts/"')
await _run(ws, 'grep "qa" "/langfuse/datasets/"')
if trace_files:
tp = f"/langfuse/traces/{trace_files[0].strip()}"
await _run(ws, f'grep "name" "{tp}"')
print("\n" + "=" * 60)
print("JQ across different resource types")
print("=" * 60)
if trace_files:
tp = f"/langfuse/traces/{trace_files[0].strip()}"
await _run(ws, f'jq ".name" "{tp}"')
await _run(ws, f'jq ".session_id" "{tp}"')
await _run(ws, f'jq ".input" "{tp}"')
await _run(
ws,
'jq ".prompt" "/langfuse/prompts/summarize/1.json"',
)
await _run(
ws,
'jq ".config" "/langfuse/prompts/classify/1.json"',
)
await _run(
ws,
'jq ".[] | .input" '
'"/langfuse/datasets/qa-eval/items.jsonl"',
)
await _run(
ws,
'jq -r ".[] | .expected_output.answer" '
'"/langfuse/datasets/qa-eval/items.jsonl"',
)
print("\n" + "=" * 60)
print("SESSION browsing")
print("=" * 60)
await _run(ws, "ls /langfuse/sessions/chat-session-001/")
r = await ws.execute("ls /langfuse/sessions/chat-session-001/", )
session_traces = (await r.stdout_str()).strip().splitlines()
if session_traces:
st = session_traces[0].strip()
sp = f"/langfuse/sessions/chat-session-001/{st}"
await _run(ws, f'cat "{sp}" | head -n 3')
print("\n" + "=" * 60)
print("PROMPTS and DATASETS detail")
print("=" * 60)
await _run(ws, "ls /langfuse/prompts/summarize/")
await _run(ws, "ls /langfuse/datasets/qa-eval/")
await _run(ws, "ls /langfuse/datasets/qa-eval/runs/")
await _run(ws, 'stat "/langfuse/prompts/summarize"')
await _run(ws, 'stat "/langfuse/datasets/qa-eval"')
print("\n" + "=" * 60)
print("TREE, FIND, NAVIGATION")
print("=" * 60)
await _run(ws, "tree -L 2 /langfuse/")
await _run(
ws,
'find "/langfuse/prompts/" -name "*.json"',
)
# -path matches the display path; -size counts dirs and sizeless
# rendered files as 0 (so +0c drops them, -1k keeps them).
await _run(ws, 'find "/langfuse/" -maxdepth 1 -path "*prompts*"')
await _run(ws, 'find "/langfuse/" -maxdepth 1 -size +0c')
await _run(ws, 'find "/langfuse/" -maxdepth 1 -size -1k')
await ws.execute('cd "/langfuse/prompts"')
await _run(ws, "pwd")
await _run(ws, "ls")
print("\n" + "=" * 60)
print("CACHING: warm reads served from cache (no backend fetch)")
print("=" * 60)
r = await ws.execute('find "/langfuse/prompts/" -name "*.json"')
prompt_files = (await r.stdout_str()).strip().splitlines()
if prompt_files:
cache_file = prompt_files[0].strip()
cold_ms, body = await _timed(ws, f'cat "{cache_file}"')
warm_ms, _ = await _timed(ws, f'cat "{cache_file}"')
grep_ms, _ = await _timed(ws, f'grep . "{cache_file}"')
head_ms, _ = await _timed(ws, f'head -n 1 "{cache_file}"')
tail_ms, _ = await _timed(ws, f'tail -n 1 "{cache_file}"')
wc_ms, _ = await _timed(ws, f'wc -l "{cache_file}"')
print(f" file={cache_file} size={len(body)}B")
print(
f" cold cat={cold_ms:.0f}ms warm cat={warm_ms:.0f}ms "
f"grep={grep_ms:.0f}ms head={head_ms:.0f}ms tail={tail_ms:.0f}ms "
f"wc={wc_ms:.0f}ms")
print(f" served_from_cache={warm_ms < cold_ms / 5} "
f"(warm speedup {cold_ms / max(warm_ms, 0.001):.0f}x)")
print("\n" + "=" * 60)
print("GLOB: mid-path patterns walk segment by segment")
print("=" * 60)
r = await ws.execute("echo /langfuse/prom*/*")
out = (await r.stdout_str()).strip()
print(f" echo /langfuse/prom*/* -> {out[:200]}")
assert "/langfuse/prompts/" in out, "mid-path glob did not expand"
# A glob that matches nothing stays the literal word, so the
# command reports it like GNU coreutils.
r = await ws.execute("cat /langfuse/zz-none-*/x.json")
err = (await r.stderr_str()).strip()
print(f" cat /langfuse/zz-none-*/x.json -> exit={r.exit_code} "
f"{err[:120]}")
assert r.exit_code == 1 and "zz-none-*" in err
if __name__ == "__main__":
asyncio.run(main())
+90
View File
@@ -0,0 +1,90 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.langfuse import LangfuseConfig, LangfuseResource
load_dotenv(".env.development")
config = LangfuseConfig(
public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
secret_key=os.environ["LANGFUSE_SECRET_KEY"],
host=os.environ["LANGFUSE_HOST"],
)
resource = LangfuseResource(config=config)
with Workspace({"/langfuse/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() top-level ---")
top_level = os.listdir(mp)
for entry in top_level:
print(f" {entry}")
if not top_level:
print(" no entries found")
else:
print("\n--- os.listdir() traces ---")
traces = os.listdir(f"{mp}/traces")
for t in traces[:5]:
print(f" {t}")
if len(traces) > 5:
print(f" ... ({len(traces)} total)")
if traces:
first_trace = traces[0]
path = f"{mp}/traces/{first_trace}"
print(f"\n--- open() + read {first_trace} ---")
with open(path) as f:
content = f.read().strip()
if content:
try:
doc = json.loads(content)
print(f" name: {doc.get('name', '?')}")
print(f" id: {doc.get('id', '?')}")
except json.JSONDecodeError:
for line in content.splitlines()[:5]:
print(f" {line[:120]}")
else:
print(" (empty)")
print("\n--- os.listdir() prompts ---")
prompts = os.listdir(f"{mp}/prompts")
for p in prompts:
print(f" {p}")
print("\n--- os.listdir() datasets ---")
datasets = os.listdir(f"{mp}/datasets")
for d in datasets:
print(f" {d}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/")
print(f">>> cat {mp}/traces/<trace-id>.json")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, "
f"{total} bytes transferred")
+106
View File
@@ -0,0 +1,106 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.langfuse import LangfuseConfig, LangfuseResource
load_dotenv(".env.development")
config = LangfuseConfig(
public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
secret_key=os.environ["LANGFUSE_SECRET_KEY"],
host=os.environ["LANGFUSE_HOST"],
)
resource = LangfuseResource(config=config)
async def main():
with Workspace({"/langfuse/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE: open() reads from Langfuse ===\n")
print("--- os.listdir() top-level ---")
top_level = vos.listdir("/langfuse")
for entry in top_level:
print(f" {entry}")
print("\n--- os.listdir() traces ---")
traces = vos.listdir("/langfuse/traces")
for t in traces[:5]:
print(f" {t}")
if len(traces) > 5:
print(f" ... ({len(traces)} total)")
if not traces:
print(" no traces found")
return
first_trace = traces[0]
path = f"/langfuse/traces/{first_trace}"
print(f"\n--- open() + read {first_trace} ---")
with open(path) as f:
content = f.read()
try:
doc = json.loads(content)
print(f" name: {doc.get('name', '?')}")
print(f" id: {doc.get('id', '?')}")
sid = doc.get("sessionId", doc.get("session_id", "?"))
print(f" session_id: {sid}")
except json.JSONDecodeError:
for line in content.splitlines()[:5]:
print(f" {line[:120]}")
print("\n--- os.listdir() sessions ---")
sessions = vos.listdir("/langfuse/sessions")
for s in sessions:
print(f" {s}")
print("\n--- os.listdir() prompts ---")
prompts = vos.listdir("/langfuse/prompts")
for p in prompts:
print(f" {p}")
print("\n--- os.listdir() datasets ---")
datasets = vos.listdir("/langfuse/datasets")
for d in datasets:
print(f" {d}")
if datasets:
ds = datasets[0]
print(f"\n--- os.listdir() datasets/{ds} ---")
items = vos.listdir(f"/langfuse/datasets/{ds}")
for item in items:
print(f" {item}")
print("\n--- bash history ---")
with open("/.bash_history") as f:
for i, line in enumerate(f):
if i >= 6:
break
print(f" {line.rstrip()[:120]}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, "
f"{total} bytes transferred")
asyncio.run(main())
+318
View File
@@ -0,0 +1,318 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import os
import time
from dotenv import load_dotenv
from langfuse import Langfuse, propagate_attributes
load_dotenv("/Users/zecheng/strukto/mirage/.env.development")
langfuse = Langfuse(
public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
secret_key=os.environ["LANGFUSE_SECRET_KEY"],
host=os.environ["LANGFUSE_HOST"],
)
print("Auth check:", langfuse.auth_check())
print("\n=== Creating prompts ===")
for name, prompt_text, cfg, label in [
("summarize",
"Summarize the following text in {{style}} style:\n\n{{text}}", {
"model": "gpt-4o",
"temperature": 0.3
}, "production"),
("summarize",
"You are an expert summarizer. Summarize in {{style}} style:\n\n{{text}}",
{
"model": "gpt-4o",
"temperature": 0.2
}, "staging"),
("classify",
"Classify the sentiment as positive, negative, or neutral:\n\n{{text}}", {
"model": "gpt-4o-mini",
"temperature": 0
}, "production"),
("extract-entities",
"Extract all named entities. Return as JSON.\n\n{{text}}", {
"model": "gpt-4o",
"temperature": 0
}, "production"),
]:
langfuse.create_prompt(
name=name,
type="text",
prompt=prompt_text,
config=cfg,
labels=[label],
)
print(f" created: {name} [{label}]")
print("\n=== Creating traces ===")
traces_data = [
{
"trace_name": "chat-completion",
"session_id": "chat-session-001",
"user_id": "user-alice",
"tags": ["chat", "geography"],
"input": {
"messages": [{
"role": "user",
"content": "What is the capital of France?"
}]
},
"output": {
"response": "The capital of France is Paris."
},
"model": "gpt-4o",
},
{
"trace_name": "chat-completion",
"session_id": "chat-session-001",
"user_id": "user-alice",
"tags": ["chat", "geography"],
"input": {
"messages": [{
"role": "user",
"content": "Tell me more about Paris"
}]
},
"output": {
"response":
"Paris is known for the Eiffel Tower and Louvre Museum."
},
"model": "gpt-4o",
},
{
"trace_name": "chat-completion",
"session_id": "chat-session-002",
"user_id": "user-bob",
"tags": ["chat", "science"],
"input": {
"messages": [{
"role": "user",
"content": "Explain quantum computing"
}]
},
"output": {
"response":
"Quantum computing uses qubits that can be in superposition..."
},
"model": "gpt-4o",
},
{
"trace_name": "summarize-document",
"session_id": "chat-session-002",
"user_id": "user-bob",
"tags": ["summarization", "science"],
"input": {
"text": "A long research paper about quantum computing..."
},
"output": {
"summary":
"This paper introduces a novel approach to error correction..."
},
"model": "gpt-4o",
},
{
"trace_name": "support-classify",
"session_id": "support-ticket-101",
"user_id": "user-charlie",
"tags": ["support", "classification"],
"input": {
"text": "I can't log in, keeps showing error 403"
},
"output": {
"category": "authentication",
"priority": "high"
},
"model": "gpt-4o-mini",
},
{
"trace_name": "support-respond",
"session_id": "support-ticket-101",
"user_id": "user-charlie",
"tags": ["support", "response"],
"input": {
"ticket": "Can't log in, error 403"
},
"output": {
"response": "Please try clearing your browser cookies..."
},
"model": "gpt-4o",
},
{
"trace_name": "entity-extraction",
"user_id": "user-alice",
"tags": ["extraction", "ner"],
"input": {
"text": "Apple CEO Tim Cook announced new products in Cupertino."
},
"output": {
"entities": [{
"name": "Apple",
"type": "ORG"
}, {
"name": "Tim Cook",
"type": "PERSON"
}]
},
"model": "gpt-4o",
},
{
"trace_name": "chat-completion",
"user_id": "user-dave",
"tags": ["chat", "creative"],
"input": {
"messages": [{
"role": "user",
"content": "Write a haiku about programming"
}]
},
"output": {
"response":
"Code flows like water\n"
"Bugs swim in the logic stream\n"
"Debug, compile, run",
},
"model": "gpt-4o",
},
]
for td in traces_data:
with propagate_attributes(
trace_name=td["trace_name"],
session_id=td.get("session_id"),
user_id=td.get("user_id"),
tags=td.get("tags"),
metadata={"env": "production"},
):
with langfuse.start_as_current_observation(
name=td["trace_name"],
as_type="span",
input=td["input"],
output=td["output"],
):
with langfuse.start_as_current_observation(
name=f"{td['trace_name']}-llm",
as_type="generation",
model=td["model"],
input=td["input"],
output=td["output"],
usage_details={
"input_tokens": 50 + len(str(td["input"])),
"output_tokens": 30 + len(str(td["output"])),
},
):
pass
print(f" created: {td['trace_name']} "
f"(session={td.get('session_id', '-')})")
print(f"\n total: {len(traces_data)} traces")
print("\n=== Creating datasets ===")
langfuse.create_dataset(name="qa-eval", description="QA evaluation")
qa_items = [
({
"question": "What is the capital of France?"
}, {
"answer": "Paris"
}),
({
"question": "Who wrote Romeo and Juliet?"
}, {
"answer": "Shakespeare"
}),
({
"question": "What is the speed of light?"
}, {
"answer": "299,792,458 m/s"
}),
({
"question": "What is the largest planet?"
}, {
"answer": "Jupiter"
}),
({
"question": "Who painted the Mona Lisa?"
}, {
"answer": "da Vinci"
}),
]
for inp, exp in qa_items:
langfuse.create_dataset_item(
dataset_name="qa-eval",
input=inp,
expected_output=exp,
)
print(f" qa-eval: {len(qa_items)} items")
langfuse.create_dataset(name="sentiment-eval",
description="Sentiment classification")
sent_items = [
({
"text": "I love this product!"
}, {
"sentiment": "positive"
}),
({
"text": "Terrible experience"
}, {
"sentiment": "negative"
}),
({
"text": "It was okay"
}, {
"sentiment": "neutral"
}),
({
"text": "Best purchase ever"
}, {
"sentiment": "positive"
}),
({
"text": "Completely broken"
}, {
"sentiment": "negative"
}),
]
for inp, exp in sent_items:
langfuse.create_dataset_item(
dataset_name="sentiment-eval",
input=inp,
expected_output=exp,
)
print(f" sentiment-eval: {len(sent_items)} items")
print("\n=== Flushing ===")
langfuse.flush()
time.sleep(3)
print("\n=== Verifying ===")
traces = langfuse.fetch_traces(limit=5)
print(f" traces: {len(traces.data)} found")
for t in traces.data[:3]:
print(f" {t.name} (session={t.session_id})")
prompt = langfuse.get_prompt("summarize")
print(f" summarize prompt: v{prompt.version}")
print("\nDone!")
+213
View File
@@ -0,0 +1,213 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.linear import LinearConfig, LinearResource
load_dotenv(".env.development")
config = LinearConfig(api_key=os.environ["LINEAR_API_KEY"])
resource = LinearResource(config=config)
async def main() -> None:
ws = Workspace({"/linear": resource}, mode=MountMode.READ)
print("=== not-found errors show the full virtual path ===")
for cmd in ("cat /linear/__nf_missing__.txt",
"head /linear/__nf_missing__.txt",
"stat /linear/__nf_missing__.txt"):
result = await ws.execute(cmd)
print(f"$ {cmd}")
print(f" exit={result.exit_code} "
f"{(await result.stderr_str()).strip()}")
print("=== ls /linear/teams/ ===")
result = await ws.execute("ls /linear/teams/")
print(await result.stdout_str())
print("=== ls -l /linear/teams/ (mtime from updatedAt) ===")
long_result = await ws.execute("ls -l /linear/teams/")
print(await long_result.stdout_str())
first_team = (await result.stdout_str()).strip().splitlines()[0] if (
await result.stdout_str()).strip() else ""
if not first_team:
print("No teams available")
return
print(f"=== cat /linear/teams/{first_team}/team.json ===")
result = await ws.execute(f"cat /linear/teams/{first_team}/team.json")
print(await result.stdout_str())
print(f"=== ls /linear/teams/{first_team}/issues/ ===")
issue_result = await ws.execute(f"ls /linear/teams/{first_team}/issues/")
print(await issue_result.stdout_str())
print(f"=== ls /linear/teams/{first_team}/projects/ ===")
project_result = await ws.execute(
f"ls /linear/teams/{first_team}/projects/")
print(await project_result.stdout_str())
project_names = (await project_result.stdout_str()).strip().splitlines()
mirage_project = next(
(name for name in project_names if name.startswith("Mirage__")),
"",
)
if mirage_project:
print(
f"=== cat /linear/teams/{first_team}/projects/{mirage_project} ==="
)
result = await ws.execute(
f"cat /linear/teams/{first_team}/projects/{mirage_project}")
print(await result.stdout_str())
project_payload = json.loads(await result.stdout_str())
print("=== issues in Mirage project ===")
for issue in project_payload.get("issues", []):
print(f"{issue['issue_key']}: "
f"{issue['title']} "
f"[{issue['state_name']}]")
else:
print("Mirage project not found in first team")
first_issue = (await
issue_result.stdout_str()).strip().splitlines()[0] if (
await issue_result.stdout_str()).strip() else ""
if not first_issue:
print("No issues available in first team")
return
issue_path = f"/linear/teams/{first_team}/issues/{first_issue}"
print("=== cat issue.json ===")
result = await ws.execute(f"cat {issue_path}/issue.json")
print(await result.stdout_str())
print("=== head -n 3 comments.jsonl ===")
result = await ws.execute(f"head -n 3 {issue_path}/comments.jsonl")
print(await result.stdout_str())
print("=== tail -n 1 comments.jsonl ===")
result = await ws.execute(f"tail -n 1 {issue_path}/comments.jsonl")
print(await result.stdout_str())
print("=== wc -l comments.jsonl ===")
result = await ws.execute(f"wc -l {issue_path}/comments.jsonl")
print(await result.stdout_str())
print("=== stat issue.json ===")
result = await ws.execute(f"stat {issue_path}/issue.json")
print(await result.stdout_str())
print("=== jq .title issue.json ===")
result = await ws.execute(f'jq ".title" {issue_path}/issue.json')
print(await result.stdout_str())
print("=== jq .state_name issue.json ===")
result = await ws.execute(f'jq ".state_name" {issue_path}/issue.json')
print(await result.stdout_str())
print("=== tree -L 1 /linear/ ===")
result = await ws.execute("tree -L 1 /linear/")
print(await result.stdout_str())
print(f"=== tree -L 1 teams/{first_team} ===")
result = await ws.execute(f"tree -L 1 /linear/teams/{first_team}/")
print(await result.stdout_str())
print("=== find issues -name '*.json' ===")
result = await ws.execute(
f'find /linear/teams/{first_team}/issues/ -name "*.json"'
" | head -n 5")
print(await result.stdout_str())
print("=== find teams -type d (directory filter) ===")
result = await ws.execute(f"find /linear/teams/{first_team}/ -type d"
" | head -n 5")
print(await result.stdout_str())
# -path matches the display path; -size counts dirs and sizeless
# rendered files as 0 (so +0c drops them, -1k keeps them).
print("=== find team -path '*issues*' ===")
result = await ws.execute(f'find /linear/teams/{first_team}/'
f' -path "*issues*" | head -n 5')
print(await result.stdout_str())
print("=== find team -maxdepth 1 -size +0c (dirs drop out) ===")
result = await ws.execute(
f"find /linear/teams/{first_team}/ -maxdepth 1 -size +0c")
print(f" exit={result.exit_code}")
print(await result.stdout_str())
print(f"=== du -s teams/{first_team} (walk fallback) ===")
result = await ws.execute(f"du -s /linear/teams/{first_team}/")
print(await result.stdout_str())
print("=== grep Mirage issue.json ===")
result = await ws.execute(f'grep Mirage {issue_path}/issue.json')
print(await result.stdout_str())
print("=== rg Backlog team.json ===")
result = await ws.execute(
f"rg Backlog /linear/teams/{first_team}/team.json")
print(await result.stdout_str())
print("=== basename ===")
result = await ws.execute(f"basename {issue_path}/issue.json")
print(await result.stdout_str())
print("=== dirname ===")
result = await ws.execute(f"dirname {issue_path}/issue.json")
print(await result.stdout_str())
# ── glob expansion (exercises resolve_glob → readdir) ──
issues_dir = f"/linear/teams/{first_team}/issues"
print(f"=== echo {issues_dir}/* (glob) ===")
r = await ws.execute(f"echo {issues_dir}/*")
out = (await r.stdout_str()).strip()
print(f" {out[:200]}")
print(f"\n=== for f in {issues_dir}/* (glob loop) ===")
r = await ws.execute(f"for f in {issues_dir}/*; do echo found:$f; done"
" | head -n 3")
out = (await r.stdout_str()).strip()
for line in out.splitlines():
print(f" {line[:120]}")
# ── mid-path glob: the team segment is the pattern, the literal
# tail keeps walking (lists teams once, then filters).
print("\n=== echo /linear/teams/*/issues (mid-path glob) ===")
r = await ws.execute("echo /linear/teams/*/issues")
out = (await r.stdout_str()).strip()
print(f" {out[:200]}")
assert out.endswith("/issues"), "mid-path glob did not expand"
# A glob that matches nothing stays the literal word, so the
# command reports it like GNU coreutils.
print("\n=== cat /linear/teams/zz-none-*/team.json (no match) ===")
r = await ws.execute("cat /linear/teams/zz-none-*/team.json")
err = (await r.stderr_str()).strip()
print(f" exit={r.exit_code} {err[:120]}")
assert r.exit_code == 1 and "zz-none-*" in err
if __name__ == "__main__":
asyncio.run(main())
+76
View File
@@ -0,0 +1,76 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
import os
import subprocess
from dotenv import load_dotenv
from mirage import Mount, MountMode, Workspace
from mirage.resource.linear import LinearConfig, LinearResource
load_dotenv(".env.development")
config = LinearConfig(api_key=os.environ["LINEAR_API_KEY"])
resource = LinearResource(config=config)
with Workspace({"/linear/": Mount(resource, mode=MountMode.READ,
fuse=True)}) as ws:
mp = ws.fuse_mountpoint
print(f"=== FUSE MODE: mounted at {mp} ===\n")
print("--- os.listdir() teams ---")
teams = os.listdir(f"{mp}/teams")
for t in teams[:5]:
print(f" {t}")
if teams:
team = teams[0]
team_path = f"{mp}/teams/{team}"
print(f"\n--- os.listdir() {team} ---")
contents = os.listdir(team_path)
for c in contents:
print(f" {c}")
# Linear cannot report a file size before fetching, so an unopened
# team.json stats as 0 bytes; any open (cat/wc/cp) hydrates it, and
# stat then reports the real size (see docs/python/setup/fuse.mdx).
team_json = f"{team_path}/team.json"
print("\n--- size-unknown semantics on team.json ---")
print(f" stat before open: {os.stat(team_json).st_size} bytes")
wc = subprocess.run(["wc", "-c", team_json],
capture_output=True,
text=True)
print(f" wc -c : {wc.stdout.split()[0]} bytes")
print(f" stat after read : {os.stat(team_json).st_size} bytes")
print("\n--- open() team.json ---")
with open(f"{team_path}/team.json") as f:
data = json.loads(f.read())
name = data.get("team_name", "?")
key = data.get("team_key", "?")
print(f" {key}: {name}")
print(f"\n>>> FUSE mounted at: {mp}")
print(">>> Open another terminal and run:")
print(f">>> ls {mp}/teams/")
print(">>> Press Enter to unmount and exit...")
input()
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes")
+82
View File
@@ -0,0 +1,82 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import json
import os
import sys
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.linear import LinearConfig, LinearResource
load_dotenv(".env.development")
config = LinearConfig(api_key=os.environ["LINEAR_API_KEY"])
resource = LinearResource(config=config)
async def main():
with Workspace({"/linear/": resource}, mode=MountMode.READ) as ws:
vos = sys.modules["os"]
print("=== VFS MODE ===\n")
print("--- os.listdir() root ---")
entries = vos.listdir("/linear")
for e in entries:
print(f" {e}")
print("\n--- os.listdir() teams ---")
teams = vos.listdir("/linear/teams")
for t in teams[:5]:
print(f" {t}")
if teams:
team = teams[0]
team_path = f"/linear/teams/{team}"
print(f"\n--- os.listdir() {team} ---")
contents = vos.listdir(team_path)
for c in contents:
print(f" {c}")
print("\n--- open() team.json ---")
with open(f"{team_path}/team.json") as f:
data = json.loads(f.read())
print(f" name: {data.get('team_name')}")
print(f" key: {data.get('team_key')}")
issues_path = f"{team_path}/issues"
if vos.path.isdir(issues_path):
issues = vos.listdir(issues_path)
print(f"\n--- os.listdir() issues ({len(issues)}) ---")
for i in issues[:5]:
print(f" {i}")
if issues:
issue_dir = f"{issues_path}/{issues[0]}"
print("\n--- open() issue.json ---")
with open(f"{issue_dir}/issue.json") as f:
data = json.loads(f.read())
print(f" key: {data.get('issue_key')}")
print(f" title: {data.get('title')}")
print(f" state: {data.get('state_name')}")
records = ws.ops.records
total = sum(r.bytes for r in records)
print(f"\nStats: {len(records)} ops, {total} bytes")
asyncio.run(main())

Some files were not shown because too many files have changed in this diff Show More