chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,353 @@
|
||||
# Cloud Sandbox Extension Examples
|
||||
|
||||
These examples are for manual verification of the cloud sandbox backends that live under `agents.extensions.sandbox`.
|
||||
|
||||
They intentionally keep the flow simple:
|
||||
|
||||
1. Build a tiny manifest in memory.
|
||||
2. Create a `SandboxAgent` that inspects that workspace through one shell tool.
|
||||
3. Run the agent against E2B, Modal, Daytona, Cloudflare, Runloop, Blaxel, or Vercel.
|
||||
|
||||
All of these examples require `OPENAI_API_KEY`, because they call the model through the normal `Runner` path. Each cloud backend also needs its own provider credentials.
|
||||
|
||||
## E2B
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra e2b
|
||||
```
|
||||
|
||||
Create an E2B account, create an API key, and export it as `E2B_API_KEY`.
|
||||
The official setup docs are:
|
||||
|
||||
- <https://e2b.dev/docs/api-key>
|
||||
- <https://e2b.dev/docs/quickstart>
|
||||
|
||||
Export the required environment variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export E2B_API_KEY=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/e2b_runner.py --stream
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--sandbox-type e2b_code_interpreter`
|
||||
- `--template <template-name>`
|
||||
- `--timeout 300`
|
||||
- `--pause-on-exit`
|
||||
|
||||
The example defaults to `e2b`, which provides a bash-style interface. Use `e2b_code_interpreter` for a Jupyter-style interface.
|
||||
|
||||
## Modal
|
||||
|
||||
If you want the same explicit session lifecycle shown in `examples/sandbox/basic.py`, that example now accepts
|
||||
`--backend modal` and reuses the same streamed tool-output flow:
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/basic.py \
|
||||
--backend modal
|
||||
```
|
||||
|
||||
The dedicated script below stays as the smaller extension-specific example.
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra modal
|
||||
```
|
||||
|
||||
Authenticate Modal with either CLI token setup or environment variables. The
|
||||
official references are:
|
||||
|
||||
- <https://modal.com/docs/reference/cli/token>
|
||||
- <https://modal.com/docs/reference/modal.config>
|
||||
- <https://modal.com/docs/guide/sandbox>
|
||||
|
||||
If you want to configure credentials directly from the CLI:
|
||||
|
||||
```bash
|
||||
uv run modal token set --token-id <token-id> --token-secret <token-secret>
|
||||
```
|
||||
|
||||
Or export environment variables for the current shell:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export MODAL_TOKEN_ID=...
|
||||
export MODAL_TOKEN_SECRET=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/modal_runner.py \
|
||||
--app-name openai-agents-python-sandbox-example \
|
||||
--stream
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--workspace-persistence tar`
|
||||
- `--workspace-persistence snapshot_filesystem`
|
||||
- `--workspace-persistence snapshot_directory`
|
||||
- `--sandbox-create-timeout-s 60`
|
||||
- `--native-cloud-bucket-secret-name my-modal-secret`
|
||||
|
||||
`app_name` is required by `ModalSandboxClientOptions`, so the example makes it an explicit CLI flag instead of hiding it.
|
||||
|
||||
Modal sandboxes also support native cloud bucket mounts through `ModalCloudBucketMountStrategy` on `S3Mount`, `R2Mount`, and HMAC-authenticated `GCSMount`.
|
||||
|
||||
For native cloud bucket testing, you can either export raw credential environment variables or pass `--native-cloud-bucket-secret-name` to reuse an existing named Modal Secret instead.
|
||||
|
||||
## Cloudflare
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra cloudflare
|
||||
```
|
||||
|
||||
Export the required environment variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export CLOUDFLARE_SANDBOX_WORKER_URL=...
|
||||
```
|
||||
|
||||
If your Cloudflare Sandbox Service worker requires bearer auth, also export:
|
||||
|
||||
```bash
|
||||
export CLOUDFLARE_SANDBOX_API_KEY=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/cloudflare_runner.py --stream
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--stream` -- stream model output to the terminal.
|
||||
- `--demo pty` -- run a PTY demo (interactive Python session with `tty=true`).
|
||||
- `--skip-snapshot-check` -- skip the stop/resume snapshot round-trip verification.
|
||||
- `--native-cloud-bucket-name <bucket>` -- mount an R2/S3 bucket via `CloudflareBucketMountStrategy`.
|
||||
- `--native-cloud-bucket-endpoint-url <url>` -- optional S3 endpoint URL.
|
||||
- `--api-key <key>` -- bearer token for the worker (or set `CLOUDFLARE_SANDBOX_API_KEY`).
|
||||
|
||||
|
||||
Cloudflare sandboxes support native cloud bucket mounts through `CloudflareBucketMountStrategy` on `S3Mount`, `R2Mount`, and HMAC-authenticated `GCSMount`.
|
||||
|
||||
## What to expect
|
||||
|
||||
Each script asks the model to inspect a small workspace and summarize it. A
|
||||
successful run should:
|
||||
|
||||
1. Start the chosen cloud sandbox backend.
|
||||
2. Materialize the manifest into the sandbox workspace.
|
||||
3. Call the shell tool at least once.
|
||||
4. Print either streamed text or a final short answer about the workspace.
|
||||
|
||||
These examples are not live-validated in CI because they depend on external cloud credentials, but they are shaped so contributors can verify backend behavior locally with one command per provider.
|
||||
|
||||
## Vercel
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra vercel
|
||||
```
|
||||
|
||||
Export the required environment variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export VERCEL_OIDC_TOKEN=...
|
||||
```
|
||||
|
||||
Or use explicit token and scope variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export VERCEL_TOKEN=...
|
||||
export VERCEL_PROJECT_ID=...
|
||||
export VERCEL_TEAM_ID=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/vercel_runner.py --stream
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--workspace-persistence tar`
|
||||
- `--workspace-persistence snapshot`
|
||||
- `--runtime node22`
|
||||
- `--timeout-ms 120000`
|
||||
|
||||
The Vercel example stays on the non-PTY path on purpose. It covers command execution, workspace materialization, and persistence verification without depending on interactive websocket support.
|
||||
|
||||
## Daytona
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra daytona
|
||||
```
|
||||
|
||||
Export the required environment variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export DAYTONA_API_KEY=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/daytona/daytona_runner.py --stream
|
||||
```
|
||||
|
||||
## Runloop
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra runloop
|
||||
```
|
||||
|
||||
Sign up for Runloop, no credit card required and $50 in credits @ [platform.runloop.ai](https://platform.runloop.ai/).
|
||||
Export the required environment variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export RUNLOOP_API_KEY=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/runloop/runner.py --stream
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--blueprint-name <name>`
|
||||
- `--pause-on-exit`
|
||||
- `--root`
|
||||
|
||||
Runloop-specific SDK features are also available directly on
|
||||
`RunloopSandboxClientOptions` and `RunloopSandboxClient.platform`. Example:
|
||||
|
||||
```python
|
||||
from agents.extensions.sandbox.runloop import (
|
||||
RunloopAfterIdle,
|
||||
RunloopGatewaySpec,
|
||||
RunloopLaunchParameters,
|
||||
RunloopMcpSpec,
|
||||
RunloopSandboxClient,
|
||||
RunloopSandboxClientOptions,
|
||||
RunloopTunnelConfig,
|
||||
)
|
||||
|
||||
client = RunloopSandboxClient()
|
||||
sandbox = await client.create(
|
||||
options=RunloopSandboxClientOptions(
|
||||
blueprint_name="python-3-12",
|
||||
launch_parameters=RunloopLaunchParameters(
|
||||
network_policy_id="np_123",
|
||||
resource_size_request="MEDIUM",
|
||||
after_idle=RunloopAfterIdle(idle_time_seconds=300, on_idle="suspend"),
|
||||
),
|
||||
tunnel=RunloopTunnelConfig(auth_mode="authenticated"),
|
||||
gateways={
|
||||
"OPENAI_GATEWAY": RunloopGatewaySpec(
|
||||
gateway="openai",
|
||||
secret="OPENAI_GATEWAY_SECRET",
|
||||
)
|
||||
},
|
||||
mcp={
|
||||
"GITHUB_MCP": RunloopMcpSpec(
|
||||
mcp_config="github-readonly",
|
||||
secret="GITHUB_MCP_SECRET",
|
||||
)
|
||||
},
|
||||
managed_secrets={"OPENAI_API_KEY": "..."},
|
||||
metadata={"team": "agents"},
|
||||
)
|
||||
)
|
||||
|
||||
public_blueprints = await client.platform.blueprints.list_public()
|
||||
public_benchmarks = await client.platform.benchmarks.list_public()
|
||||
```
|
||||
|
||||
`managed_secrets` are stored as Runloop account secrets and only secret references are persisted in session state. The platform facade also exposes Runloop-native helpers for blueprints, benchmarks, secrets, network policies, and axons.
|
||||
|
||||
If you enable `--root`, Runloop launches the devbox with `launch_parameters.user_parameters={"username":"root","uid":0}`. In that mode, the default home and working directory become `/root`, so the example also uses `/root` as its manifest workspace root. If you configure root launch in your own code, either rely on that root-mode default or explicitly choose a `manifest.root` under `/root`.
|
||||
## Blaxel
|
||||
|
||||
### Setup
|
||||
|
||||
Install the repo extra:
|
||||
|
||||
```bash
|
||||
uv sync --extra blaxel
|
||||
```
|
||||
|
||||
Create a Blaxel account and get an API key. The official docs are:
|
||||
|
||||
- <https://docs.blaxel.ai>
|
||||
- <https://app.blaxel.ai>
|
||||
|
||||
Export the required environment variables:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
export BL_API_KEY=...
|
||||
export BL_WORKSPACE=...
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
uv run python examples/sandbox/extensions/blaxel_runner.py --stream
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--image blaxel/py-app`
|
||||
- `--region us-pdx-1`
|
||||
- `--memory 4096`
|
||||
- `--ttl 1h`
|
||||
- `--pause-on-exit`
|
||||
- `--skip-snapshot-check`
|
||||
|
||||
The runner also includes standalone demos for individual features. Pass
|
||||
`--demo <name>` to run one:
|
||||
|
||||
- `pty` -- agent-driven interactive Python session via PTY
|
||||
- `drive` -- [Blaxel Drive mount](https://docs.blaxel.ai/Agent-drive/Overview) (persistent storage, requires `--drive-name`)
|
||||
|
||||
Blaxel sandboxes support cloud bucket mounts (S3, R2, GCS) through `BlaxelCloudBucketMountStrategy` and persistent drive mounts through `BlaxelDriveMountStrategy`. See the [Blaxel Drive docs](https://docs.blaxel.ai/Agent-drive/Overview) for details.
|
||||
@@ -0,0 +1 @@
|
||||
"""Manual validation examples for cloud sandbox extensions."""
|
||||
@@ -0,0 +1,466 @@
|
||||
"""
|
||||
Blaxel-backed sandbox example for manual validation.
|
||||
|
||||
This example mirrors the other cloud extension runners. It supports:
|
||||
- Standard agent run (non-streaming and streaming).
|
||||
- PTY interactive session demo (agent-driven).
|
||||
- Blaxel Drive mount demo (persistent storage).
|
||||
|
||||
Prerequisites:
|
||||
uv sync --extra blaxel
|
||||
export OPENAI_API_KEY=...
|
||||
export BL_API_KEY=...
|
||||
export BL_WORKSPACE=...
|
||||
|
||||
Run:
|
||||
# Basic agent run
|
||||
uv run python examples/sandbox/extensions/blaxel_runner.py --stream
|
||||
|
||||
# With a specific image and region
|
||||
uv run python examples/sandbox/extensions/blaxel_runner.py \\
|
||||
--image blaxel/py-app --region us-pdx-1 --stream
|
||||
|
||||
# PTY terminal demo (agent-driven interactive Python session)
|
||||
uv run python examples/sandbox/extensions/blaxel_runner.py --demo pty
|
||||
|
||||
# Drive mount demo (requires an existing drive, defaults region to us-was-1)
|
||||
uv run python examples/sandbox/extensions/blaxel_runner.py \\
|
||||
--demo drive --drive-name my-drive
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner, set_tracing_disabled
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.capabilities import Shell
|
||||
from agents.sandbox.entries import File
|
||||
from agents.sandbox.manifest import Environment
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest, tool_call_name
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
DEFAULT_BLAXEL_WORKSPACE_ROOT,
|
||||
BlaxelDriveMountStrategy,
|
||||
BlaxelSandboxClient,
|
||||
BlaxelSandboxClientOptions,
|
||||
)
|
||||
from agents.extensions.sandbox.blaxel import BlaxelDriveMount
|
||||
except Exception as exc:
|
||||
raise SystemExit(
|
||||
"Blaxel sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra blaxel"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_MODEL = "gpt-5.6-sol"
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
DEFAULT_PTY_QUESTION = (
|
||||
"Start an interactive Python session with `tty=true`. In that same session, compute "
|
||||
"`5 + 5`, then add 5 more to the previous result. Briefly report the outputs and "
|
||||
"confirm that you stayed in one Python process."
|
||||
)
|
||||
|
||||
|
||||
def _build_manifest() -> Manifest:
|
||||
"""Build a small demo manifest for the default agent run."""
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Blaxel Demo Workspace\n\nThis workspace validates the Blaxel sandbox backend.\n"
|
||||
),
|
||||
"project/status.md": (
|
||||
"# Project Status\n\n"
|
||||
"- Backend: Blaxel cloud sandbox\n"
|
||||
"- Region: auto-selected\n"
|
||||
"- Features: exec, file I/O, PTY, drives, preview URLs\n"
|
||||
),
|
||||
"project/tasks.md": (
|
||||
"# Tasks\n\n"
|
||||
"1. Inspect the workspace files.\n"
|
||||
"2. List all features mentioned in status.md.\n"
|
||||
"3. Summarize in 2-3 sentences.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
return Manifest(
|
||||
root=DEFAULT_BLAXEL_WORKSPACE_ROOT,
|
||||
entries=manifest.entries,
|
||||
environment=Environment(
|
||||
value={"DEMO_ENV": "blaxel-agent-demo"},
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _require_env(name: str) -> str:
|
||||
value = os.environ.get(name)
|
||||
if value:
|
||||
return value
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
def _stream_event_banner(event_name: str, raw_item: object) -> str | None:
|
||||
_ = raw_item
|
||||
if event_name == "tool_called":
|
||||
return "[tool call]"
|
||||
if event_name == "tool_output":
|
||||
return "[tool output]"
|
||||
return None
|
||||
|
||||
|
||||
def _raw_item_call_id(raw_item: object) -> str | None:
|
||||
if isinstance(raw_item, dict):
|
||||
call_id = raw_item.get("call_id") or raw_item.get("id")
|
||||
else:
|
||||
call_id = getattr(raw_item, "call_id", None) or getattr(raw_item, "id", None)
|
||||
return call_id if isinstance(call_id, str) and call_id else None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PTY demo (agent-driven)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _run_pty_demo(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
image: str | None,
|
||||
region: str | None,
|
||||
) -> None:
|
||||
"""Demonstrate PTY interaction: start an interactive Python process and continue it."""
|
||||
agent = SandboxAgent(
|
||||
name="Blaxel PTY Demo",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Complete the task by interacting with the sandbox through the shell capability. "
|
||||
"Keep the final answer concise. "
|
||||
"Preserve process state when the task depends on it. If you start an interactive "
|
||||
"program, continue using that same process instead of launching a second one."
|
||||
),
|
||||
default_manifest=Manifest(
|
||||
root=DEFAULT_BLAXEL_WORKSPACE_ROOT,
|
||||
entries=text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Blaxel PTY Agent Example\n\n"
|
||||
"This workspace is used by the Blaxel PTY demo.\n"
|
||||
),
|
||||
}
|
||||
).entries,
|
||||
),
|
||||
capabilities=[Shell()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
client = BlaxelSandboxClient()
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=client,
|
||||
options=BlaxelSandboxClientOptions(
|
||||
name=f"blaxel-demo-pty-{uuid.uuid4().hex[:8]}",
|
||||
image=image,
|
||||
region=region,
|
||||
),
|
||||
),
|
||||
workflow_name="Blaxel PTY sandbox example",
|
||||
)
|
||||
|
||||
try:
|
||||
result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
|
||||
saw_text_delta = False
|
||||
saw_any_text = False
|
||||
tool_names_by_call_id: dict[str, str] = {}
|
||||
|
||||
async for event in result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
saw_any_text = True
|
||||
continue
|
||||
|
||||
if event.type != "run_item_stream_event":
|
||||
continue
|
||||
|
||||
raw_item = event.item.raw_item
|
||||
banner = _stream_event_banner(event.name, raw_item)
|
||||
if banner is None:
|
||||
continue
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
saw_text_delta = False
|
||||
|
||||
if event.name == "tool_called":
|
||||
t_name = tool_call_name(raw_item)
|
||||
call_id = _raw_item_call_id(raw_item)
|
||||
if call_id is not None and t_name:
|
||||
tool_names_by_call_id[call_id] = t_name
|
||||
if t_name:
|
||||
banner = f"{banner} {t_name}"
|
||||
elif event.name == "tool_output":
|
||||
call_id = _raw_item_call_id(raw_item)
|
||||
output_tool_name = tool_names_by_call_id.get(call_id or "")
|
||||
if output_tool_name:
|
||||
banner = f"{banner} {output_tool_name}"
|
||||
|
||||
print(banner)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
if not saw_any_text:
|
||||
print(result.final_output)
|
||||
finally:
|
||||
await client.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Drive demo
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _run_drive_demo(
|
||||
*,
|
||||
model: str,
|
||||
question: str | None,
|
||||
image: str | None,
|
||||
region: str | None,
|
||||
drive_name: str | None,
|
||||
stream: bool,
|
||||
) -> None:
|
||||
"""Mount a Blaxel Drive and write a file to it."""
|
||||
if not drive_name:
|
||||
print("Usage: --demo drive --drive-name <name>")
|
||||
print()
|
||||
print("You need an existing Blaxel Drive. Create one at:")
|
||||
print(" https://app.blaxel.ai or via the Blaxel CLI.")
|
||||
return
|
||||
|
||||
# Blaxel drives must be in the same region as the sandbox.
|
||||
effective_region = region or os.environ.get("BL_REGION") or "us-was-1"
|
||||
mount_path = "/mnt/demo-drive"
|
||||
|
||||
manifest = Manifest(
|
||||
root=DEFAULT_BLAXEL_WORKSPACE_ROOT,
|
||||
entries={
|
||||
"README.md": File(
|
||||
content=(b"# Blaxel Drive Demo\n\nThe drive is mounted at /mnt/demo-drive.\n")
|
||||
),
|
||||
"drive": BlaxelDriveMount(
|
||||
drive_name=drive_name,
|
||||
drive_mount_path=mount_path,
|
||||
mount_strategy=BlaxelDriveMountStrategy(),
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
marker = f"demo-{uuid.uuid4().hex[:8]}"
|
||||
agent = SandboxAgent(
|
||||
name="Blaxel Drive Demo",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Execute the exact shell commands the user gives you. "
|
||||
"Do not explore, do not run any other commands. "
|
||||
"Report the stdout and stderr of each command you ran. "
|
||||
"You must run the exact commands from the user message using the shell tool. "
|
||||
"Do not substitute, rewrite, or add any commands. Just execute and report output."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[Shell()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
client = BlaxelSandboxClient()
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=client,
|
||||
options=BlaxelSandboxClientOptions(
|
||||
name=f"blaxel-demo-drive-{uuid.uuid4().hex[:8]}",
|
||||
image=image,
|
||||
region=effective_region,
|
||||
),
|
||||
),
|
||||
workflow_name="Blaxel drive demo",
|
||||
)
|
||||
|
||||
effective_question = question or (
|
||||
f"Run: echo 'drive persistence ok ({marker})' > {mount_path}/{marker}.txt && "
|
||||
f"cat {mount_path}/{marker}.txt && ls {mount_path}"
|
||||
)
|
||||
|
||||
if not stream:
|
||||
result = await Runner.run(agent, effective_question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
else:
|
||||
stream_result = Runner.run_streamed(agent, effective_question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
if saw_text_delta:
|
||||
print()
|
||||
|
||||
await client.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Standard agent run (streaming / non-streaming)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str | None,
|
||||
image: str | None,
|
||||
region: str | None,
|
||||
memory: int | None,
|
||||
ttl: str | None,
|
||||
pause_on_exit: bool,
|
||||
stream: bool,
|
||||
demo: str | None,
|
||||
drive_name: str | None,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
|
||||
# Handle dedicated demos.
|
||||
if demo == "pty":
|
||||
await _run_pty_demo(
|
||||
model=model,
|
||||
question=question or DEFAULT_PTY_QUESTION,
|
||||
image=image,
|
||||
region=region,
|
||||
)
|
||||
return
|
||||
|
||||
if demo == "drive":
|
||||
await _run_drive_demo(
|
||||
model=model,
|
||||
question=question,
|
||||
image=image,
|
||||
region=region,
|
||||
drive_name=drive_name,
|
||||
stream=stream,
|
||||
)
|
||||
return
|
||||
|
||||
manifest = _build_manifest()
|
||||
agent = SandboxAgent(
|
||||
name="Blaxel Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected. Also run `echo $DEMO_ENV` to confirm environment "
|
||||
"variables are set."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=BlaxelSandboxClient(),
|
||||
options=BlaxelSandboxClientOptions(
|
||||
name=f"blaxel-demo-agent-{uuid.uuid4().hex[:8]}",
|
||||
image=image,
|
||||
region=region,
|
||||
memory=memory,
|
||||
ttl=ttl,
|
||||
labels={"purpose": "agent-demo", "source": "blaxel-runner"},
|
||||
pause_on_exit=pause_on_exit,
|
||||
),
|
||||
),
|
||||
workflow_name="Blaxel sandbox example",
|
||||
)
|
||||
|
||||
effective_question = question or DEFAULT_QUESTION
|
||||
|
||||
if not stream:
|
||||
result = await Runner.run(agent, effective_question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
stream_result = Runner.run_streamed(agent, effective_question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
set_tracing_disabled(True)
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Blaxel sandbox demo -- showcases sandbox features.",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=(
|
||||
"demos:\n"
|
||||
" agent Run a sandboxed agent (default)\n"
|
||||
" pty Agent-driven PTY interactive terminal\n"
|
||||
" drive Mount a Blaxel Drive (requires --drive-name)\n"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--demo",
|
||||
choices=["agent", "pty", "drive"],
|
||||
default="agent",
|
||||
help="Which demo to run (default: agent).",
|
||||
)
|
||||
parser.add_argument("--model", default=DEFAULT_MODEL, help="Model name.")
|
||||
parser.add_argument("--question", default=None, help="Override the default prompt.")
|
||||
parser.add_argument("--stream", action="store_true", help="Stream response.")
|
||||
parser.add_argument("--image", default=None, help="Sandbox image.")
|
||||
parser.add_argument("--region", default=None, help="Sandbox region.")
|
||||
parser.add_argument("--memory", type=int, default=None, help="Memory in MB.")
|
||||
parser.add_argument("--ttl", default=None, help="Sandbox TTL (e.g. '1h').")
|
||||
parser.add_argument("--pause-on-exit", action="store_true", help="Pause on exit.")
|
||||
parser.add_argument("--drive-name", default=None, help="Drive name for drive demo.")
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
image=args.image,
|
||||
region=args.region,
|
||||
memory=args.memory,
|
||||
ttl=args.ttl,
|
||||
pause_on_exit=args.pause_on_exit,
|
||||
stream=args.stream,
|
||||
demo=args.demo,
|
||||
drive_name=args.drive_name,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,446 @@
|
||||
"""
|
||||
Cloudflare-backed sandbox example for manual validation.
|
||||
|
||||
This example mirrors the Modal and E2B extension runners. It supports:
|
||||
- Standard agent run (non-streaming and streaming).
|
||||
- Snapshot stop/resume round-trip verification.
|
||||
- PTY interactive session demo.
|
||||
- Cloud bucket mount demo (R2/S3/GCS via CloudflareBucketMountStrategy).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner, set_tracing_disabled
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import LocalSnapshotSpec, Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.capabilities import Shell
|
||||
from agents.sandbox.entries import File, R2Mount, S3Mount
|
||||
from agents.sandbox.session import BaseSandboxSession
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest, tool_call_name
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
CloudflareBucketMountStrategy,
|
||||
CloudflareSandboxClient,
|
||||
CloudflareSandboxClientOptions,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"Cloudflare sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra cloudflare"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_MODEL = "gpt-5.6-sol"
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
DEFAULT_PTY_QUESTION = (
|
||||
"Start an interactive Python session with `tty=true`. In that same session, compute "
|
||||
"`5 + 5`, then add 5 more to the previous result. Briefly report the outputs and "
|
||||
"confirm that you stayed in one Python process."
|
||||
)
|
||||
SNAPSHOT_CHECK_PATH = Path("snapshot-check.txt")
|
||||
SNAPSHOT_CHECK_CONTENT = "cloudflare snapshot round-trip ok\n"
|
||||
|
||||
|
||||
def _build_manifest(
|
||||
*,
|
||||
native_cloud_bucket_name: str | None = None,
|
||||
native_cloud_bucket_mount_path: str | None = None,
|
||||
native_cloud_bucket_endpoint_url: str | None = None,
|
||||
) -> Manifest:
|
||||
"""Build a small demo manifest, optionally including a cloud bucket mount."""
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Cloudflare Demo Workspace\n\n"
|
||||
"This workspace exists to validate the Cloudflare sandbox backend manually.\n"
|
||||
),
|
||||
"incident.md": (
|
||||
"# Incident\n\n"
|
||||
"- Customer: Fabrikam Retail.\n"
|
||||
"- Issue: delayed reporting rollout.\n"
|
||||
"- Primary blocker: incomplete security questionnaire.\n"
|
||||
),
|
||||
"plan.md": (
|
||||
"# Plan\n\n"
|
||||
"1. Close the questionnaire.\n"
|
||||
"2. Reconfirm the rollout date with the customer.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
if native_cloud_bucket_name is None:
|
||||
return manifest
|
||||
|
||||
# Determine whether this looks like an R2 bucket (has account ID) or S3.
|
||||
account_id = os.environ.get("CLOUDFLARE_ACCOUNT_ID")
|
||||
if account_id:
|
||||
manifest.entries["cloud-bucket"] = R2Mount(
|
||||
bucket=native_cloud_bucket_name,
|
||||
account_id=account_id,
|
||||
access_key_id=os.environ.get("R2_ACCESS_KEY_ID"),
|
||||
secret_access_key=os.environ.get("R2_SECRET_ACCESS_KEY"),
|
||||
mount_path=Path(native_cloud_bucket_mount_path)
|
||||
if native_cloud_bucket_mount_path is not None
|
||||
else None,
|
||||
read_only=False,
|
||||
mount_strategy=CloudflareBucketMountStrategy(),
|
||||
)
|
||||
else:
|
||||
manifest.entries["cloud-bucket"] = S3Mount(
|
||||
bucket=native_cloud_bucket_name,
|
||||
access_key_id=os.environ.get("AWS_ACCESS_KEY_ID"),
|
||||
secret_access_key=os.environ.get("AWS_SECRET_ACCESS_KEY"),
|
||||
endpoint_url=native_cloud_bucket_endpoint_url,
|
||||
mount_path=Path(native_cloud_bucket_mount_path)
|
||||
if native_cloud_bucket_mount_path is not None
|
||||
else None,
|
||||
read_only=False,
|
||||
mount_strategy=CloudflareBucketMountStrategy(),
|
||||
)
|
||||
return manifest
|
||||
|
||||
|
||||
def _build_pty_manifest() -> Manifest:
|
||||
"""Build a tiny manifest for the PTY demo."""
|
||||
return Manifest(
|
||||
entries={
|
||||
"README.md": File(
|
||||
content=(
|
||||
b"# Cloudflare PTY Agent Example\n\n"
|
||||
b"This workspace is used by the Cloudflare PTY demo.\n"
|
||||
)
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _require_env(name: str) -> str:
|
||||
value = os.environ.get(name)
|
||||
if value:
|
||||
return value
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
async def _read_text(session: BaseSandboxSession, path: Path) -> str:
|
||||
data = await session.read(path)
|
||||
text = cast(str | bytes, data.read())
|
||||
if isinstance(text, bytes):
|
||||
return text.decode("utf-8")
|
||||
return text
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stop/resume snapshot round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _verify_stop_resume(*, worker_url: str, api_key: str | None) -> None:
|
||||
"""Create a sandbox, write a file, stop, resume, and verify the file persisted."""
|
||||
client = CloudflareSandboxClient()
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": "# Snapshot test\n",
|
||||
}
|
||||
)
|
||||
options = CloudflareSandboxClientOptions(worker_url=worker_url, api_key=api_key)
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="cf-snapshot-example-") as snapshot_dir:
|
||||
sandbox = await client.create(
|
||||
manifest=manifest,
|
||||
snapshot=LocalSnapshotSpec(base_path=Path(snapshot_dir)),
|
||||
options=options,
|
||||
)
|
||||
|
||||
try:
|
||||
await sandbox.start()
|
||||
await sandbox.write(
|
||||
SNAPSHOT_CHECK_PATH,
|
||||
io.BytesIO(SNAPSHOT_CHECK_CONTENT.encode("utf-8")),
|
||||
)
|
||||
await sandbox.stop()
|
||||
finally:
|
||||
await sandbox.shutdown()
|
||||
|
||||
resumed_sandbox = await client.resume(sandbox.state)
|
||||
try:
|
||||
await resumed_sandbox.start()
|
||||
restored_text = await _read_text(resumed_sandbox, SNAPSHOT_CHECK_PATH)
|
||||
if restored_text != SNAPSHOT_CHECK_CONTENT:
|
||||
raise RuntimeError(
|
||||
f"Snapshot resume verification failed: "
|
||||
f"expected {SNAPSHOT_CHECK_CONTENT!r}, got {restored_text!r}"
|
||||
)
|
||||
finally:
|
||||
await resumed_sandbox.aclose()
|
||||
|
||||
print("snapshot round-trip ok")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PTY demo
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _stream_event_banner(event_name: str, raw_item: object) -> str | None:
|
||||
_ = raw_item
|
||||
if event_name == "tool_called":
|
||||
return "[tool call]"
|
||||
if event_name == "tool_output":
|
||||
return "[tool output]"
|
||||
return None
|
||||
|
||||
|
||||
def _raw_item_call_id(raw_item: object) -> str | None:
|
||||
if isinstance(raw_item, dict):
|
||||
call_id = raw_item.get("call_id") or raw_item.get("id")
|
||||
else:
|
||||
call_id = getattr(raw_item, "call_id", None) or getattr(raw_item, "id", None)
|
||||
return call_id if isinstance(call_id, str) and call_id else None
|
||||
|
||||
|
||||
async def _run_pty_demo(*, model: str, worker_url: str, api_key: str | None) -> None:
|
||||
"""Demonstrate PTY interaction: start an interactive Python process and continue it."""
|
||||
agent = SandboxAgent(
|
||||
name="Cloudflare PTY Demo",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Complete the task by interacting with the sandbox through the shell capability. "
|
||||
"Keep the final answer concise. "
|
||||
"Preserve process state when the task depends on it. If you start an interactive "
|
||||
"program, continue using that same process instead of launching a second one."
|
||||
),
|
||||
default_manifest=_build_pty_manifest(),
|
||||
capabilities=[Shell()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
client = CloudflareSandboxClient()
|
||||
sandbox = await client.create(
|
||||
manifest=agent.default_manifest,
|
||||
options=CloudflareSandboxClientOptions(worker_url=worker_url, api_key=api_key),
|
||||
)
|
||||
|
||||
try:
|
||||
async with sandbox:
|
||||
result = Runner.run_streamed(
|
||||
agent,
|
||||
DEFAULT_PTY_QUESTION,
|
||||
run_config=RunConfig(
|
||||
sandbox=SandboxRunConfig(session=sandbox),
|
||||
workflow_name="Cloudflare PTY sandbox example",
|
||||
),
|
||||
)
|
||||
|
||||
saw_text_delta = False
|
||||
saw_any_text = False
|
||||
tool_names_by_call_id: dict[str, str] = {}
|
||||
|
||||
async for event in result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
saw_any_text = True
|
||||
continue
|
||||
|
||||
if event.type != "run_item_stream_event":
|
||||
continue
|
||||
|
||||
raw_item = event.item.raw_item
|
||||
banner = _stream_event_banner(event.name, raw_item)
|
||||
if banner is None:
|
||||
continue
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
saw_text_delta = False
|
||||
|
||||
if event.name == "tool_called":
|
||||
t_name = tool_call_name(raw_item)
|
||||
call_id = _raw_item_call_id(raw_item)
|
||||
if call_id is not None and t_name:
|
||||
tool_names_by_call_id[call_id] = t_name
|
||||
if t_name:
|
||||
banner = f"{banner} {t_name}"
|
||||
elif event.name == "tool_output":
|
||||
call_id = _raw_item_call_id(raw_item)
|
||||
output_tool_name = tool_names_by_call_id.get(call_id or "")
|
||||
if output_tool_name:
|
||||
banner = f"{banner} {output_tool_name}"
|
||||
|
||||
print(banner)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
if not saw_any_text:
|
||||
print(result.final_output)
|
||||
finally:
|
||||
await client.delete(sandbox)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Standard agent run (streaming / non-streaming)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
worker_url: str,
|
||||
api_key: str | None,
|
||||
stream: bool,
|
||||
demo: str | None,
|
||||
skip_snapshot_check: bool,
|
||||
native_cloud_bucket_name: str | None,
|
||||
native_cloud_bucket_mount_path: str,
|
||||
native_cloud_bucket_endpoint_url: str | None,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
|
||||
# Handle dedicated demos.
|
||||
if demo == "pty":
|
||||
await _run_pty_demo(model=model, worker_url=worker_url, api_key=api_key)
|
||||
return
|
||||
|
||||
# Snapshot stop/resume round-trip.
|
||||
if not skip_snapshot_check:
|
||||
await _verify_stop_resume(worker_url=worker_url, api_key=api_key)
|
||||
|
||||
manifest = _build_manifest(
|
||||
native_cloud_bucket_name=native_cloud_bucket_name,
|
||||
native_cloud_bucket_mount_path=native_cloud_bucket_mount_path,
|
||||
native_cloud_bucket_endpoint_url=native_cloud_bucket_endpoint_url,
|
||||
)
|
||||
agent = SandboxAgent(
|
||||
name="Cloudflare Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[Shell()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=CloudflareSandboxClient(),
|
||||
options=CloudflareSandboxClientOptions(worker_url=worker_url, api_key=api_key),
|
||||
),
|
||||
workflow_name="Cloudflare sandbox example",
|
||||
)
|
||||
|
||||
if not stream:
|
||||
result = await Runner.run(agent, question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
stream_result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
set_tracing_disabled(True)
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Run a Cloudflare sandbox agent with optional PTY, streaming, and snapshot demos."
|
||||
)
|
||||
parser.add_argument("--model", default=DEFAULT_MODEL, help="Model name to use.")
|
||||
parser.add_argument(
|
||||
"--question",
|
||||
default=DEFAULT_QUESTION,
|
||||
help="Prompt to send to the agent.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--worker-url",
|
||||
default=os.environ.get("CLOUDFLARE_SANDBOX_WORKER_URL"),
|
||||
help="Cloudflare Worker base URL. Defaults to CLOUDFLARE_SANDBOX_WORKER_URL.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--api-key",
|
||||
default=os.environ.get("CLOUDFLARE_SANDBOX_API_KEY"),
|
||||
help="Optional bearer token for the worker. Defaults to CLOUDFLARE_SANDBOX_API_KEY.",
|
||||
)
|
||||
parser.add_argument("--stream", action="store_true", default=False, help="Stream the response.")
|
||||
parser.add_argument(
|
||||
"--demo",
|
||||
default=None,
|
||||
choices=["pty"],
|
||||
help="Run a standalone demo instead of the standard agent flow.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-snapshot-check",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Skip the snapshot stop/resume round-trip verification.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-name",
|
||||
default=None,
|
||||
help="Optional R2/S3 bucket name to mount with CloudflareBucketMountStrategy.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-mount-path",
|
||||
default="cloud-bucket",
|
||||
help=(
|
||||
"Mount path for --native-cloud-bucket-name. Relative paths are resolved under the "
|
||||
"workspace root."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-endpoint-url",
|
||||
default=None,
|
||||
help="Optional endpoint URL for --native-cloud-bucket-name (S3 only).",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.worker_url:
|
||||
raise SystemExit(
|
||||
"A Cloudflare Worker URL is required. Pass --worker-url or set CLOUDFLARE_SANDBOX_WORKER_URL."
|
||||
)
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
worker_url=args.worker_url,
|
||||
api_key=args.api_key,
|
||||
stream=args.stream,
|
||||
demo=args.demo,
|
||||
skip_snapshot_check=args.skip_snapshot_check,
|
||||
native_cloud_bucket_name=args.native_cloud_bucket_name,
|
||||
native_cloud_bucket_mount_path=args.native_cloud_bucket_mount_path,
|
||||
native_cloud_bucket_endpoint_url=args.native_cloud_bucket_endpoint_url,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1 @@
|
||||
"""Daytona sandbox extension examples."""
|
||||
@@ -0,0 +1,208 @@
|
||||
"""
|
||||
Minimal Daytona-backed sandbox example for manual validation.
|
||||
|
||||
This mirrors the E2B and Modal extension examples: it creates a tiny workspace,
|
||||
asks a sandboxed agent to inspect it through one shell tool, and prints a short
|
||||
answer.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.entries import S3Mount
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[4]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
DEFAULT_DAYTONA_WORKSPACE_ROOT,
|
||||
DaytonaCloudBucketMountStrategy,
|
||||
DaytonaSandboxClient,
|
||||
DaytonaSandboxClientOptions,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"Daytona sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra daytona"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
|
||||
|
||||
def _build_manifest(
|
||||
*,
|
||||
cloud_bucket_name: str | None = None,
|
||||
cloud_bucket_mount_path: str | None = None,
|
||||
cloud_bucket_endpoint_url: str | None = None,
|
||||
cloud_bucket_key_prefix: str | None = None,
|
||||
) -> Manifest:
|
||||
"""Build a small demo manifest, optionally including a cloud bucket mount."""
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Daytona Demo Workspace\n\n"
|
||||
"This workspace exists to validate the Daytona sandbox backend manually.\n"
|
||||
),
|
||||
"launch.md": (
|
||||
"# Launch\n\n"
|
||||
"- Customer: Contoso Logistics.\n"
|
||||
"- Goal: validate the remote sandbox agent path.\n"
|
||||
"- Current status: Daytona backend smoke and app-server connectivity are passing.\n"
|
||||
),
|
||||
"tasks.md": (
|
||||
"# Tasks\n\n"
|
||||
"1. Inspect the workspace files.\n"
|
||||
"2. Summarize the setup and any notable status in two sentences.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
if cloud_bucket_name is None:
|
||||
return Manifest(root=DEFAULT_DAYTONA_WORKSPACE_ROOT, entries=manifest.entries)
|
||||
|
||||
manifest.entries["cloud-bucket"] = S3Mount(
|
||||
bucket=cloud_bucket_name,
|
||||
access_key_id=os.environ.get("AWS_ACCESS_KEY_ID"),
|
||||
secret_access_key=os.environ.get("AWS_SECRET_ACCESS_KEY"),
|
||||
session_token=os.environ.get("AWS_SESSION_TOKEN"),
|
||||
endpoint_url=cloud_bucket_endpoint_url,
|
||||
prefix=cloud_bucket_key_prefix,
|
||||
mount_path=Path(cloud_bucket_mount_path) if cloud_bucket_mount_path is not None else None,
|
||||
read_only=False,
|
||||
mount_strategy=DaytonaCloudBucketMountStrategy(),
|
||||
)
|
||||
return Manifest(root=DEFAULT_DAYTONA_WORKSPACE_ROOT, entries=manifest.entries)
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
pause_on_exit: bool,
|
||||
stream: bool,
|
||||
cloud_bucket_name: str | None = None,
|
||||
cloud_bucket_mount_path: str | None = None,
|
||||
cloud_bucket_endpoint_url: str | None = None,
|
||||
cloud_bucket_key_prefix: str | None = None,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
_require_env("DAYTONA_API_KEY")
|
||||
|
||||
manifest = _build_manifest(
|
||||
cloud_bucket_name=cloud_bucket_name,
|
||||
cloud_bucket_mount_path=cloud_bucket_mount_path,
|
||||
cloud_bucket_endpoint_url=cloud_bucket_endpoint_url,
|
||||
cloud_bucket_key_prefix=cloud_bucket_key_prefix,
|
||||
)
|
||||
agent = SandboxAgent(
|
||||
name="Daytona Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
client = DaytonaSandboxClient()
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=client,
|
||||
options=DaytonaSandboxClientOptions(pause_on_exit=pause_on_exit),
|
||||
),
|
||||
workflow_name="Daytona sandbox example",
|
||||
)
|
||||
|
||||
try:
|
||||
if not stream:
|
||||
result = await Runner.run(agent, question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
stream_result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
finally:
|
||||
await client.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", default="gpt-5.6-sol", help="Model name to use.")
|
||||
parser.add_argument("--question", default=DEFAULT_QUESTION, help="Prompt to send to the agent.")
|
||||
parser.add_argument(
|
||||
"--pause-on-exit",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Pause the Daytona sandbox on shutdown instead of deleting it.",
|
||||
)
|
||||
parser.add_argument("--stream", action="store_true", default=False, help="Stream the response.")
|
||||
parser.add_argument(
|
||||
"--cloud-bucket-name",
|
||||
default=None,
|
||||
help="S3 bucket name to mount into the sandbox.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cloud-bucket-mount-path",
|
||||
default=None,
|
||||
help=(
|
||||
"Mount path for --cloud-bucket-name. Relative paths are resolved under the "
|
||||
"workspace root. Defaults to the mount class default."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cloud-bucket-endpoint-url",
|
||||
default=None,
|
||||
help="Optional endpoint URL for --cloud-bucket-name (S3 only, e.g. MinIO).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cloud-bucket-key-prefix",
|
||||
default=None,
|
||||
help="Optional key prefix for --cloud-bucket-name.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
pause_on_exit=args.pause_on_exit,
|
||||
stream=args.stream,
|
||||
cloud_bucket_name=args.cloud_bucket_name,
|
||||
cloud_bucket_mount_path=args.cloud_bucket_mount_path,
|
||||
cloud_bucket_endpoint_url=args.cloud_bucket_endpoint_url,
|
||||
cloud_bucket_key_prefix=args.cloud_bucket_key_prefix,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,79 @@
|
||||
# NASA Spending Text-to-SQL Agent
|
||||
|
||||
Multi-turn conversational agent that translates natural-language questions about NASA federal spending into SQL queries, executes them against a local SQLite database, and returns structured tabular results.
|
||||
|
||||
## How it works
|
||||
|
||||
1. **Schema knowledge**: The agent receives a compact schema summary in its system prompt and can read detailed per-table documentation from workspace files on demand.
|
||||
2. **SQL execution**: A custom `SqlCapability` provides a `run_sql` tool with guardrails — read-only mode, statement validation, row limits, and query timeouts. The agent is instructed to use `run_sql` for all queries; the tool enforces read-only access at the SQLite level.
|
||||
3. **Multi-turn conversation**: The agent retains context across turns, so you can ask follow-up questions like "break that down by year" or "just the top 5".
|
||||
4. **Compaction**: Uses the `Compaction` capability to automatically summarize older conversation context, keeping long sessions within the model's context window.
|
||||
5. **Pause/resume**: Type `exit` to pause the sandbox and quit. Run the script again to reconnect to the same paused sandbox — no re-download needed. If the sandbox can't be reconnected (e.g. it was deleted or expired), a fresh one is created and the database is rebuilt automatically.
|
||||
6. **Memory**: Uses the `Memory` capability to extract learnings from each conversation and consolidate them into structured files. On subsequent sessions, the agent starts with context from previous conversations (useful query patterns, data caveats, etc.).
|
||||
|
||||
## Data
|
||||
|
||||
The database contains NASA federal spending data from [USAspending.gov](https://usaspending.gov), defaulting to FY2021-FY2025 (configurable via `--start-fy`/`--end-fy` flags on `setup_db.py`).
|
||||
|
||||
It uses a single `spending` table where each row is one transaction (obligation, modification, or de-obligation) on a federal award. The agent aggregates as needed via SQL.
|
||||
|
||||
The database is built automatically on first run (requires internet access in the sandbox). Subsequent runs reuse the existing database.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.12+
|
||||
- `openai-agents` installed with Daytona support (`uv sync --extra daytona` from repo root)
|
||||
- `OPENAI_API_KEY` environment variable set (for the LLM)
|
||||
- `DAYTONA_API_KEY` environment variable set (for the sandbox — get one at [daytona.io](https://daytona.io))
|
||||
- Internet access (for first-run database setup inside the sandbox)
|
||||
|
||||
## Run
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY="sk-..."
|
||||
export DAYTONA_API_KEY="..."
|
||||
uv run python -m examples.sandbox.extensions.daytona.usaspending_text2sql.agent
|
||||
```
|
||||
|
||||
## Example questions
|
||||
|
||||
```
|
||||
> What are NASA's top 10 contractors by total spending?
|
||||
> Break that down by fiscal year
|
||||
> Which NASA centers award the most contracts?
|
||||
> Show me grants to universities in California
|
||||
> How has NASA spending changed over time?
|
||||
> What are the largest individual awards in the last 3 years?
|
||||
> Compare contract vs grant spending by year
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
daytona/usaspending_text2sql/
|
||||
├── agent.py — SandboxAgent definition + interactive REPL
|
||||
├── sql_capability.py — SqlCapability (Capability) with run_sql tool and guardrails
|
||||
├── setup_db.py — Runs inside sandbox; fetches data from USAspending API, builds SQLite DB
|
||||
├── schema/
|
||||
│ ├── overview.md — Compact schema summary (injected into instructions)
|
||||
│ └── tables/ — Per-table column documentation (read on demand via Shell capability)
|
||||
└── README.md
|
||||
```
|
||||
|
||||
### SQL guardrails (defense in depth)
|
||||
|
||||
1. **Connection-level**: SQLite opened with `?mode=ro` URI (read-only)
|
||||
2. **PRAGMA**: `query_only = ON` prevents writes even if validation is bypassed
|
||||
3. **Statement validation**: Only `SELECT`, `WITH`, `EXPLAIN`, `PRAGMA` are allowed
|
||||
4. **Row limit**: Hard cap (default 100 rows) with truncation detection
|
||||
5. **Timeout**: Queries killed after 30 seconds
|
||||
|
||||
### Audit log
|
||||
|
||||
All sandbox operations (exec calls, start/stop, SQL queries and their results) are logged to `.audit_log.jsonl` as structured JSONL events via the SDK's `Instrumentation` and `JsonlOutboxSink`. This is useful for debugging, replaying sessions, or inspecting exactly what SQL the agent ran.
|
||||
|
||||
### Sandbox
|
||||
|
||||
This example uses Daytona as its sandbox backend. The agent and capability definitions are backend-agnostic, but the entrypoint (`agent.py`) hardcodes `DaytonaSandboxClient` and Daytona-specific features like pause/resume.
|
||||
@@ -0,0 +1 @@
|
||||
"""USAspending text-to-SQL Daytona sandbox example."""
|
||||
@@ -0,0 +1,540 @@
|
||||
"""NASA spending text-to-SQL agent.
|
||||
|
||||
Multi-turn conversational agent that translates natural-language questions
|
||||
about NASA federal spending into SQL queries, executes them against a
|
||||
USAspending SQLite database, and returns structured results.
|
||||
|
||||
Usage:
|
||||
uv run python -m examples.sandbox.extensions.daytona.usaspending_text2sql.agent
|
||||
|
||||
The database is built automatically inside the sandbox on first run by
|
||||
executing setup_db.py (requires internet access). Subsequent runs reuse the
|
||||
existing database.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.capabilities.compaction import Compaction
|
||||
from agents.sandbox.capabilities.memory import Memory
|
||||
from agents.sandbox.capabilities.shell import Shell
|
||||
from agents.sandbox.config import MemoryGenerateConfig, MemoryReadConfig
|
||||
from agents.sandbox.entries import Dir, File, LocalDir, LocalFile
|
||||
from agents.sandbox.session import (
|
||||
EventPayloadPolicy,
|
||||
Instrumentation,
|
||||
JsonlOutboxSink,
|
||||
)
|
||||
from examples.auto_mode import input_with_fallback, is_auto_mode
|
||||
from examples.sandbox.extensions.daytona.usaspending_text2sql.sql_capability import (
|
||||
SqlCapability,
|
||||
)
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
DEFAULT_DAYTONA_WORKSPACE_ROOT,
|
||||
DaytonaSandboxClient,
|
||||
DaytonaSandboxClientOptions,
|
||||
DaytonaSandboxSessionState,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover
|
||||
raise SystemExit(
|
||||
"Daytona sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra daytona"
|
||||
) from exc
|
||||
|
||||
EXAMPLE_DIR = Path(__file__).parent
|
||||
SCHEMA_DIR = EXAMPLE_DIR / "schema"
|
||||
SETUP_DB_PATH = EXAMPLE_DIR / "setup_db.py"
|
||||
SESSION_STATE_PATH = EXAMPLE_DIR / ".session_state.json"
|
||||
AUDIT_LOG_PATH = EXAMPLE_DIR / ".audit_log.jsonl"
|
||||
|
||||
# Set at runtime once the exposed port is resolved.
|
||||
_downloads_base_url: str = ""
|
||||
|
||||
DEVELOPER_INSTRUCTIONS = (
|
||||
(SCHEMA_DIR / "overview.md").read_text()
|
||||
+ """
|
||||
|
||||
## Instructions
|
||||
|
||||
- Always use the `run_sql` tool to query the database. Never attempt to run sqlite3 directly.
|
||||
- Read schema documentation from schema/tables/ if you need detailed column information.
|
||||
- Read schema/glossary.md for official USAspending term definitions (e.g., what "obligation" vs "outlay" means).
|
||||
- Prefer aggregations (GROUP BY, SUM, COUNT, AVG) over returning many raw rows.
|
||||
- Format monetary values with dollar signs and commas in your final answers (e.g., $1,234,567).
|
||||
- When the user asks a follow-up question, use conversation context to understand references
|
||||
like "break that down by year" or "just the top 5".
|
||||
- If a query fails, read the error message and try to fix the SQL.
|
||||
- Explain your query logic briefly so the user can verify correctness.
|
||||
|
||||
## Data caveats
|
||||
|
||||
- The database contains **obligations** (money legally committed), not outlays (money actually paid).
|
||||
When the user asks about "spending", clarify that these are obligation amounts.
|
||||
- Amounts are tied to the **action_date** (when the obligation was signed), not when the work happens.
|
||||
A multi-year contract may appear entirely in the fiscal year it was obligated.
|
||||
- Some recipients are masked as "MULTIPLE RECIPIENTS" or "REDACTED DUE TO PII" for privacy reasons.
|
||||
Mention this if recipient-level analysis looks incomplete.
|
||||
"""
|
||||
)
|
||||
|
||||
DB_PATH = "data/usaspending.db"
|
||||
DEFAULT_AUTO_QUESTION = "What are NASA's top 5 contractors by total obligations?"
|
||||
|
||||
WORKSPACE_ROOT = DEFAULT_DAYTONA_WORKSPACE_ROOT
|
||||
|
||||
|
||||
def build_agent() -> SandboxAgent:
|
||||
"""Build the agent blueprint."""
|
||||
generate_memory = not is_auto_mode()
|
||||
manifest = Manifest(
|
||||
root=WORKSPACE_ROOT,
|
||||
entries={
|
||||
"setup_db.py": LocalFile(src=SETUP_DB_PATH),
|
||||
"schema": LocalDir(src=SCHEMA_DIR),
|
||||
"data": Dir(ephemeral=True),
|
||||
"memories/MEMORY.md": File(content=b""),
|
||||
"memories/memory_summary.md": File(content=b""),
|
||||
"memories/phase_two_selection.json": File(content=b""),
|
||||
},
|
||||
)
|
||||
|
||||
return SandboxAgent(
|
||||
name="NASA Spending Q&A",
|
||||
default_manifest=manifest,
|
||||
model="gpt-5.6-sol",
|
||||
instructions=(
|
||||
"You are a helpful data analyst that answers questions about NASA federal spending "
|
||||
"by writing and executing SQL queries.\n\n" + DEVELOPER_INSTRUCTIONS
|
||||
),
|
||||
capabilities=[
|
||||
SqlCapability(db_path=DB_PATH),
|
||||
Shell(),
|
||||
Compaction(),
|
||||
Memory(
|
||||
read=MemoryReadConfig(live_update=False),
|
||||
generate=(
|
||||
MemoryGenerateConfig(
|
||||
extra_prompt=(
|
||||
"Pay attention to which SQL patterns work best for the USAspending "
|
||||
"data, column quirks (e.g. recipient_parent_name vs recipient_name "
|
||||
"for grouping), and data caveats the user discovers (e.g. negative "
|
||||
"obligations, masked recipients)."
|
||||
),
|
||||
)
|
||||
if generate_memory
|
||||
else None
|
||||
),
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Terminal formatting helpers (unchanged from universal_computer version)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DIM = "\033[2;39m"
|
||||
DIM_CYAN = "\033[2;36m"
|
||||
DIM_BLUE = "\033[2;34m"
|
||||
DIM_YELLOW = "\033[2;33m"
|
||||
DIM_GREEN = "\033[2;32m"
|
||||
RESET = "\033[0m"
|
||||
|
||||
_SQL_KEYWORDS = (
|
||||
r"\b(?:SELECT|FROM|WHERE|JOIN|LEFT|RIGHT|INNER|OUTER|CROSS|FULL|NATURAL|ON|AND|OR"
|
||||
r"|NOT|IN|IS|NULL|AS|WITH|GROUP\s+BY|ORDER\s+BY|HAVING|LIMIT|OFFSET|UNION"
|
||||
r"|ALL|DISTINCT|CASE|WHEN|THEN|ELSE|END|EXISTS|BETWEEN|LIKE|INSERT|UPDATE"
|
||||
r"|DELETE|CREATE|DROP|ALTER|SET|VALUES|INTO|TABLE|INDEX|VIEW|ASC|DESC|BY"
|
||||
r"|OVER|PARTITION\s+BY)\b"
|
||||
)
|
||||
|
||||
_SQL_FUNCTIONS = (
|
||||
r"\b(?:COUNT|SUM|AVG|MIN|MAX|COALESCE|CAST|SUBSTR|LENGTH|ROUND|ABS|IFNULL"
|
||||
r"|NULLIF|REPLACE|TRIM|UPPER|LOWER|DATE|DATETIME|STRFTIME|TYPEOF|TOTAL"
|
||||
r"|GROUP_CONCAT|PRINTF|ROW_NUMBER|RANK|DENSE_RANK)(?=\s*\()"
|
||||
)
|
||||
|
||||
_SQL_STRING = r"'(?:''|[^'])*'"
|
||||
|
||||
|
||||
def _highlight_sql(sql: str) -> str:
|
||||
"""Apply ANSI syntax highlighting to a SQL string."""
|
||||
placeholders: list[str] = []
|
||||
|
||||
def _stash_string(m: re.Match[str]) -> str:
|
||||
placeholders.append(m.group(0))
|
||||
return f"\x00STR{len(placeholders) - 1}\x00"
|
||||
|
||||
result = re.sub(_SQL_STRING, _stash_string, sql)
|
||||
|
||||
result = re.sub(
|
||||
_SQL_KEYWORDS,
|
||||
lambda m: f"{DIM_BLUE}{m.group(0)}{DIM}",
|
||||
result,
|
||||
flags=re.IGNORECASE,
|
||||
)
|
||||
result = re.sub(
|
||||
_SQL_FUNCTIONS,
|
||||
lambda m: f"{DIM_YELLOW}{m.group(0)}{DIM}",
|
||||
result,
|
||||
flags=re.IGNORECASE,
|
||||
)
|
||||
|
||||
def _restore_string(m: re.Match[str]) -> str:
|
||||
idx = int(m.group(1))
|
||||
return f"{DIM_GREEN}{placeholders[idx]}{DIM}"
|
||||
|
||||
result = re.sub(r"\x00STR(\d+)\x00", _restore_string, result)
|
||||
return result
|
||||
|
||||
|
||||
def _format_tool_args(name: str, arguments: str) -> str:
|
||||
"""Format a tool call for display, pretty-printing SQL queries."""
|
||||
if name == "run_sql":
|
||||
try:
|
||||
args = json.loads(arguments)
|
||||
query = args.get("query", "")
|
||||
limit = args.get("limit")
|
||||
header = f" {DIM}[SQL]"
|
||||
if limit is not None:
|
||||
header += f" (limit {limit})"
|
||||
header += RESET
|
||||
highlighted = _highlight_sql(query)
|
||||
sql = textwrap.indent(highlighted, " ")
|
||||
return f"{header}\n{DIM}{sql}{RESET}"
|
||||
except Exception:
|
||||
pass
|
||||
return f" {DIM}[tool] {name}({arguments}){RESET}"
|
||||
|
||||
|
||||
def _format_tool_result(output: str) -> str | None:
|
||||
"""Format a tool result for display. Returns None for non-SQL results."""
|
||||
try:
|
||||
data = json.loads(output)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
if output.strip():
|
||||
return f" {DIM}{output.strip()}{RESET}"
|
||||
return None
|
||||
|
||||
columns = data.get("columns")
|
||||
rows = data.get("rows")
|
||||
if not isinstance(columns, list) or not isinstance(rows, list):
|
||||
return None
|
||||
|
||||
row_count = data.get("row_count", len(rows))
|
||||
display_count = data.get("display_count", len(rows))
|
||||
truncated = data.get("truncated", False)
|
||||
|
||||
if not columns:
|
||||
return f" {DIM_CYAN}\u2192 Result (0 rows){RESET}"
|
||||
|
||||
# Build the summary line.
|
||||
parts = []
|
||||
if display_count < row_count:
|
||||
parts.append(f"showing {display_count} of {row_count}")
|
||||
else:
|
||||
parts.append(f"{row_count} rows")
|
||||
if truncated:
|
||||
parts.append("CSV truncated at limit")
|
||||
|
||||
csv_file = data.get("csv_file")
|
||||
download_line = ""
|
||||
if csv_file and _downloads_base_url:
|
||||
download_line = f"\n {DIM}\u2193 {_downloads_base_url}{csv_file}{RESET}"
|
||||
|
||||
# Try to fit the table in the terminal. If too wide, skip it —
|
||||
# the model's prose summary + download link are enough.
|
||||
try:
|
||||
term_width = os.get_terminal_size().columns
|
||||
except OSError:
|
||||
term_width = 120
|
||||
|
||||
widths = [len(str(c)) for c in columns]
|
||||
for row in rows:
|
||||
for i, val in enumerate(row):
|
||||
widths[i] = max(widths[i], len(str(val) if val is not None else "NULL"))
|
||||
|
||||
# 4 leading spaces + "| " between each col + trailing " |"
|
||||
table_width = 4 + sum(widths) + 3 * len(widths) + 1
|
||||
|
||||
if table_width > term_width:
|
||||
header = f" {DIM_CYAN}\u2192 Result ({row_count} rows) \u2014 too wide to print in terminal, download below{RESET}"
|
||||
return f"{header}{download_line}"
|
||||
|
||||
def fmt_row(vals: list[Any]) -> str:
|
||||
cells = []
|
||||
for v, w in zip(vals, widths, strict=False):
|
||||
cells.append(str(v if v is not None else "NULL").ljust(w))
|
||||
return " | " + " | ".join(cells) + " |"
|
||||
|
||||
lines = [fmt_row(columns)]
|
||||
lines.append(" |" + "|".join("-" * (w + 2) for w in widths) + "|")
|
||||
for row in rows:
|
||||
lines.append(fmt_row(row))
|
||||
|
||||
header = f" {DIM_CYAN}\u2192 Result ({', '.join(parts)})"
|
||||
table = "\n".join(lines)
|
||||
return f"{header}\n{table}{RESET}{download_line}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Multi-turn REPL using Runner.run_streamed()
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def run_turn(
|
||||
agent: SandboxAgent,
|
||||
conversation: list[Any],
|
||||
question: str,
|
||||
run_config: RunConfig,
|
||||
) -> list[Any]:
|
||||
"""Run one conversational turn and return the updated conversation history."""
|
||||
input_items = conversation + [{"role": "user", "content": question}]
|
||||
|
||||
result = Runner.run_streamed(agent, input_items, run_config=run_config)
|
||||
|
||||
async for event in result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
print(event.data.delta, end="", flush=True)
|
||||
continue
|
||||
|
||||
if event.type != "run_item_stream_event":
|
||||
continue
|
||||
|
||||
if event.name == "tool_called":
|
||||
item = event.item
|
||||
raw = getattr(item, "raw_item", None)
|
||||
if raw is not None:
|
||||
name = getattr(raw, "name", "")
|
||||
arguments = getattr(raw, "arguments", "")
|
||||
print()
|
||||
print(_format_tool_args(name, arguments))
|
||||
continue
|
||||
|
||||
if event.name == "tool_output":
|
||||
item = event.item
|
||||
output = getattr(item, "output", "")
|
||||
if isinstance(output, str):
|
||||
formatted = _format_tool_result(output)
|
||||
if formatted is not None:
|
||||
print(formatted)
|
||||
print()
|
||||
continue
|
||||
|
||||
print()
|
||||
|
||||
# Build the full conversation history for the next turn using the SDK's
|
||||
# built-in conversion, which correctly serializes all item types.
|
||||
return result.to_input_list()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Session state persistence for pause/resume
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _load_session_state() -> DaytonaSandboxSessionState | None:
|
||||
"""Load saved session state from disk, or return None."""
|
||||
if not SESSION_STATE_PATH.exists():
|
||||
return None
|
||||
try:
|
||||
return DaytonaSandboxSessionState.model_validate_json(SESSION_STATE_PATH.read_text())
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _save_session_state(state: DaytonaSandboxSessionState) -> None:
|
||||
"""Persist session state to disk so the sandbox can be reused next run."""
|
||||
SESSION_STATE_PATH.write_text(state.model_dump_json(indent=2))
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
"""Exit early with a clear message when a required environment variable is missing."""
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
def _status(message: str) -> None:
|
||||
"""Print progress immediately so automation logs show where startup is blocked."""
|
||||
print(message, flush=True)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main entrypoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
_status("Starting Daytona NASA spending text-to-SQL example...")
|
||||
_require_env("OPENAI_API_KEY")
|
||||
_require_env("DAYTONA_API_KEY")
|
||||
|
||||
agent = build_agent()
|
||||
|
||||
instrumentation = Instrumentation(
|
||||
sinks=[JsonlOutboxSink(AUDIT_LOG_PATH)],
|
||||
payload_policy=EventPayloadPolicy(include_exec_output=True),
|
||||
)
|
||||
RESULTS_PORT = 8080
|
||||
|
||||
_status("Creating Daytona sandbox client...")
|
||||
client = DaytonaSandboxClient(instrumentation=instrumentation)
|
||||
client_options = DaytonaSandboxClientOptions(
|
||||
pause_on_exit=True,
|
||||
exposed_ports=(RESULTS_PORT,),
|
||||
)
|
||||
|
||||
# Try to resume a previously paused sandbox.
|
||||
saved_state = _load_session_state()
|
||||
sandbox = None
|
||||
destroy = False
|
||||
|
||||
try:
|
||||
if saved_state is not None:
|
||||
old_sandbox_id = saved_state.sandbox_id
|
||||
try:
|
||||
_status(f"Resuming Daytona sandbox {old_sandbox_id}...")
|
||||
sandbox = await client.resume(saved_state)
|
||||
assert isinstance(sandbox.state, DaytonaSandboxSessionState)
|
||||
if sandbox.state.sandbox_id == old_sandbox_id:
|
||||
_status("Reconnected to existing sandbox.")
|
||||
else:
|
||||
_status("Previous sandbox no longer exists. Created a new one.")
|
||||
except Exception as e:
|
||||
_status(f"Could not resume previous sandbox: {e}")
|
||||
saved_state = None
|
||||
sandbox = None
|
||||
|
||||
if sandbox is None:
|
||||
_status("Creating Daytona sandbox...")
|
||||
sandbox = await client.create(manifest=agent.default_manifest, options=client_options)
|
||||
|
||||
_status("Starting Daytona sandbox...")
|
||||
await sandbox.start()
|
||||
|
||||
# Persist state immediately so crashes don't orphan the sandbox.
|
||||
assert isinstance(sandbox.state, DaytonaSandboxSessionState)
|
||||
_save_session_state(sandbox.state)
|
||||
|
||||
# Build database inside sandbox (idempotent — skips if DB already exists).
|
||||
_status("Setting up database (may take a few minutes on first run)...")
|
||||
result = await sandbox.exec("python3", "setup_db.py", timeout=1800.0)
|
||||
stdout = result.stdout.decode("utf-8", errors="replace")
|
||||
if stdout.strip():
|
||||
print(stdout)
|
||||
if not result.ok():
|
||||
stderr = result.stderr.decode("utf-8", errors="replace")
|
||||
print(f"Database setup failed:\n{stderr}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Start a file server in the sandbox so query results can be downloaded.
|
||||
_status("Starting results file server...")
|
||||
await sandbox.exec("mkdir -p results", timeout=5.0)
|
||||
await sandbox.exec(
|
||||
f"nohup python3 -m http.server {RESULTS_PORT} --directory results > /dev/null 2>&1 &",
|
||||
timeout=5.0,
|
||||
)
|
||||
|
||||
# Resolve the Daytona signed URL for the file server.
|
||||
global _downloads_base_url
|
||||
try:
|
||||
endpoint = await sandbox.resolve_exposed_port(RESULTS_PORT)
|
||||
_downloads_base_url = endpoint.url_for("http")
|
||||
except Exception as e:
|
||||
print(f" Warning: could not resolve download URL: {e}")
|
||||
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(session=sandbox),
|
||||
workflow_name="NASA Spending Q&A",
|
||||
)
|
||||
|
||||
downloads_line = ""
|
||||
if _downloads_base_url:
|
||||
downloads_line = f"\n Browse results: {DIM_CYAN}{_downloads_base_url}{RESET}"
|
||||
|
||||
print(f"""
|
||||
{DIM}{"=" * 60}{RESET}
|
||||
NASA Spending Q&A (FY2021\u2013FY2025)
|
||||
|
||||
Data from USAspending.gov \u2014 contracts, grants, and IDVs
|
||||
awarded by NASA. Each row is a transaction (obligation).
|
||||
|
||||
Includes: amounts, award descriptions, recipients, recipient
|
||||
locations, places of performance, industry and product
|
||||
categories, sub-agencies, and fiscal years.
|
||||
{downloads_line}
|
||||
Type {DIM_CYAN}'exit'{RESET} to pause sandbox, {DIM_CYAN}'destroy'{RESET} to delete it.
|
||||
{DIM}{"=" * 60}{RESET}
|
||||
""")
|
||||
|
||||
conversation: list[Any] = []
|
||||
auto_mode = is_auto_mode()
|
||||
|
||||
while True:
|
||||
try:
|
||||
if auto_mode:
|
||||
question = input_with_fallback("> ", DEFAULT_AUTO_QUESTION)
|
||||
else:
|
||||
question = input("> ")
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print()
|
||||
break
|
||||
|
||||
cmd = question.strip().lower()
|
||||
if cmd == "exit":
|
||||
break
|
||||
if cmd == "destroy":
|
||||
destroy = True
|
||||
break
|
||||
|
||||
if not question.strip():
|
||||
continue
|
||||
|
||||
try:
|
||||
conversation = await run_turn(agent, conversation, question, run_config)
|
||||
except Exception as e:
|
||||
print(f"\nError: {e}")
|
||||
print()
|
||||
|
||||
if auto_mode:
|
||||
break
|
||||
|
||||
if destroy:
|
||||
assert isinstance(sandbox.state, DaytonaSandboxSessionState)
|
||||
sandbox.state.pause_on_exit = False
|
||||
SESSION_STATE_PATH.unlink(missing_ok=True)
|
||||
_status("Deleting sandbox...")
|
||||
else:
|
||||
assert isinstance(sandbox.state, DaytonaSandboxSessionState)
|
||||
_save_session_state(sandbox.state)
|
||||
_status("Saving memory and pausing sandbox (can take a couple of minutes)...")
|
||||
|
||||
finally:
|
||||
if sandbox is not None:
|
||||
if destroy:
|
||||
# Skip memory flush — sandbox is being deleted.
|
||||
await sandbox.stop()
|
||||
await sandbox.shutdown()
|
||||
else:
|
||||
await sandbox.aclose()
|
||||
await client.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,60 @@
|
||||
## Database: usaspending.db
|
||||
|
||||
NASA federal spending data from USAspending.gov. Each row is a single spending transaction (obligation or de-obligation) on a federal award.
|
||||
|
||||
### Table: spending
|
||||
|
||||
One row per transaction. Multiple transactions can share the same `award_id` (an award's initial obligation plus subsequent modifications, amendments, and de-obligations).
|
||||
|
||||
**Key columns:**
|
||||
- `award_id` — unique award identifier (many transactions share one award_id)
|
||||
- `award_piid_fain` — human-readable contract number (PIID) or assistance award number (FAIN)
|
||||
- `parent_award_piid` — parent IDV contract number (links task orders to their contract vehicle; contracts only)
|
||||
- `award_type` — 'contract', 'grant', 'idv', or 'other'
|
||||
- `action_date` — date of this transaction (YYYY-MM-DD)
|
||||
- `fiscal_year` — federal fiscal year (Oct-Sep; FY2024 = Oct 2023 - Sep 2024)
|
||||
- `federal_action_obligation` — dollar amount of this transaction (can be negative for de-obligations)
|
||||
- `total_obligation` — cumulative obligation for the entire award at time of this transaction
|
||||
- `base_and_all_options_value` — total potential ceiling value including unexercised options (contracts only)
|
||||
- `recipient_name` — who received the funds
|
||||
- `recipient_parent_name` — parent company (e.g., subsidiaries roll up; contracts only)
|
||||
- `recipient_state`, `recipient_city`, `recipient_country` — recipient location
|
||||
- `awarding_office` — NASA center/office that made the award (e.g., 'GODDARD SPACE FLIGHT CENTER', 'JET PROPULSION LABORATORY')
|
||||
- `funding_office` — NASA center/office providing funding (often same as awarding)
|
||||
- `naics_code`, `naics_description` — industry classification (primarily for contracts)
|
||||
- `psc_code`, `psc_description` — product/service classification
|
||||
- `place_of_performance_state`, `place_of_performance_city` — where work is performed
|
||||
- `period_of_perf_start`, `period_of_perf_end` — award period of performance dates (YYYY-MM-DD)
|
||||
- `extent_competed` — competition level: 'Full and Open Competition', 'Not Competed', etc. (contracts only)
|
||||
- `type_of_set_aside` — small business set-aside type: '8(a)', 'HUBZone', 'SDVOSB', etc. (contracts only)
|
||||
- `number_of_offers` — number of offers received (contracts only)
|
||||
- `contract_pricing_type` — pricing structure: 'Firm Fixed Price', 'Cost Plus', etc. (contracts only)
|
||||
- `business_types` — recipient type for assistance: nonprofit, university, state govt, etc. (grants only)
|
||||
- `description` — free-text description of the transaction
|
||||
|
||||
### Common query patterns
|
||||
|
||||
```sql
|
||||
-- Total spending by fiscal year
|
||||
SELECT fiscal_year, SUM(federal_action_obligation) AS total
|
||||
FROM spending GROUP BY fiscal_year ORDER BY fiscal_year;
|
||||
|
||||
-- Top recipients (roll up by parent company)
|
||||
SELECT COALESCE(NULLIF(recipient_parent_name, ''), recipient_name) AS entity,
|
||||
SUM(federal_action_obligation) AS total
|
||||
FROM spending GROUP BY entity ORDER BY total DESC LIMIT 10;
|
||||
|
||||
-- Spending by award type
|
||||
SELECT award_type, COUNT(*), SUM(federal_action_obligation) AS total
|
||||
FROM spending GROUP BY award_type;
|
||||
|
||||
-- Competitive vs sole-source contracts
|
||||
SELECT extent_competed, COUNT(DISTINCT award_id) AS awards,
|
||||
SUM(federal_action_obligation) AS total
|
||||
FROM spending WHERE award_type = 'contract'
|
||||
GROUP BY extent_competed ORDER BY total DESC;
|
||||
|
||||
-- Spending by NASA center
|
||||
SELECT awarding_office, SUM(federal_action_obligation) AS total
|
||||
FROM spending GROUP BY awarding_office ORDER BY total DESC;
|
||||
```
|
||||
@@ -0,0 +1,52 @@
|
||||
# spending
|
||||
|
||||
One row per prime award transaction from NASA. Each row represents a financial action — an initial obligation, modification, amendment, or de-obligation on a federal award.
|
||||
|
||||
## Columns
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| rowid | INTEGER PK | Auto-increment row identifier |
|
||||
| award_id | TEXT | Unique award identifier. Multiple rows share the same award_id when an award has multiple transactions |
|
||||
| award_piid_fain | TEXT | Human-readable award number: PIID for contracts (e.g., 'NNJ13ZBG001'), FAIN for assistance |
|
||||
| parent_award_piid | TEXT | Parent IDV contract number. Links task/delivery orders to their parent contract vehicle (contracts only) |
|
||||
| award_type | TEXT | Category: 'contract', 'grant', 'idv', or 'other' |
|
||||
| description | TEXT | Free-text description of the transaction or award purpose |
|
||||
| action_date | TEXT | Date of this transaction (ISO 8601: YYYY-MM-DD) |
|
||||
| fiscal_year | INTEGER | Federal fiscal year (Oct-Sep; FY2024 = Oct 2023 - Sep 2024) |
|
||||
| federal_action_obligation | REAL | Dollar amount of this specific transaction. Can be negative for de-obligations |
|
||||
| total_obligation | REAL | Cumulative obligation for the entire award at the time of this transaction |
|
||||
| base_and_all_options_value | REAL | Total potential ceiling value of the contract including all unexercised options. Contracts only; NULL for grants |
|
||||
| recipient_name | TEXT | Legal name of the recipient organization |
|
||||
| recipient_parent_name | TEXT | Parent company name (e.g., subsidiaries like 'Lockheed Martin Space' roll up to 'Lockheed Martin Corporation'). Contracts only; empty for grants |
|
||||
| recipient_state | TEXT | Two-letter US state code of recipient's address. Empty for foreign recipients |
|
||||
| recipient_city | TEXT | City of recipient's address |
|
||||
| recipient_country | TEXT | Country name (e.g., 'UNITED STATES', 'UNITED KINGDOM') |
|
||||
| awarding_office | TEXT | NASA center/office that made the award (e.g., 'GODDARD SPACE FLIGHT CENTER', 'JET PROPULSION LABORATORY'). Values are uppercase |
|
||||
| funding_office | TEXT | NASA center/office providing funding (often same as awarding). Values are uppercase |
|
||||
| naics_code | TEXT | North American Industry Classification System code. Primarily for contracts; may be empty for grants |
|
||||
| naics_description | TEXT | Human-readable NAICS description |
|
||||
| psc_code | TEXT | Product/Service Code for contracts, CFDA number for assistance. Different classification systems in the same column |
|
||||
| psc_description | TEXT | Human-readable description of the PSC (contracts) or CFDA program (assistance) |
|
||||
| place_of_performance_state | TEXT | State where work is performed. Two-letter codes for contracts, full names for assistance. May differ from recipient_state |
|
||||
| place_of_performance_city | TEXT | City where work is performed |
|
||||
| period_of_perf_start | TEXT | Award period of performance start date (YYYY-MM-DD) |
|
||||
| period_of_perf_end | TEXT | Award period of performance end date (YYYY-MM-DD). This is the current end date and may reflect extensions |
|
||||
| extent_competed | TEXT | Competition level. Values include 'Full and Open Competition', 'Not Available for Competition', 'Not Competed', etc. Contracts only; empty for grants |
|
||||
| type_of_set_aside | TEXT | Small business set-aside type. Values include 'Small Business Set-Aside', '8(a) Set-Aside', 'HUBZone Set-Aside', 'Service-Disabled Veteran-Owned Small Business Set-Aside', 'Women-Owned Small Business', etc. Contracts only |
|
||||
| number_of_offers | INTEGER | Number of offers/bids received. 1 = effectively sole-source even if technically competed. Contracts only; NULL for grants |
|
||||
| contract_pricing_type | TEXT | Pricing structure: 'Firm Fixed Price', 'Cost Plus Fixed Fee', 'Cost No Fee', 'Time and Materials', etc. Contracts only |
|
||||
| business_types | TEXT | Recipient organization type for assistance awards: nonprofit, university, state government, tribal, etc. Grants only; empty for contracts |
|
||||
|
||||
## Notes
|
||||
|
||||
- **Aggregating to award level**: use `GROUP BY award_id` with `SUM(federal_action_obligation)` to get total spending per award. The `total_obligation` column is a snapshot at each transaction and may not reflect the final total.
|
||||
- **Contract ceiling vs obligation**: `base_and_all_options_value` is the potential maximum; `total_obligation` is what's actually committed. A contract may have $10M obligated against a $500M ceiling.
|
||||
- **Parent company roll-up**: Use `COALESCE(NULLIF(recipient_parent_name, ''), recipient_name)` to group subsidiaries under their parent. Only populated for contracts.
|
||||
- **recipient_name** may vary slightly for the same entity across rows (e.g., 'BOEING CO' vs 'THE BOEING COMPANY'). Use `LIKE` or `UPPER()` for fuzzy matching.
|
||||
- **award_type** is derived from USAspending type codes: A/B/C/D -> 'contract', 02-05 -> 'grant', IDV_* -> 'idv'.
|
||||
- **federal_action_obligation** can be negative (de-obligations, corrections). Sum them to get net spending.
|
||||
- **naics_code** and **naics_description** are only populated for contracts; empty for grants/assistance.
|
||||
- **psc_code** contains Product/Service Codes for contracts and CFDA numbers for assistance awards. **psc_description** contains the corresponding description. These are different classification systems stored in the same column.
|
||||
- **Contracts-only columns**: `base_and_all_options_value`, `recipient_parent_name`, `parent_award_piid`, `extent_competed`, `type_of_set_aside`, `number_of_offers`, `contract_pricing_type` are only populated for contracts/IDVs.
|
||||
- **Grants-only columns**: `business_types` is only populated for assistance awards.
|
||||
@@ -0,0 +1,718 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Download NASA spending data from USAspending.gov and build a SQLite database.
|
||||
|
||||
This script is designed to run inside a sandbox environment with only Python
|
||||
stdlib available. It fetches data via the USAspending bulk download API,
|
||||
parses the resulting CSVs, and creates a local SQLite database.
|
||||
|
||||
Usage:
|
||||
python setup_db.py [--force] [--start-fy 2021] [--end-fy 2025]
|
||||
|
||||
The script is idempotent: it skips the download/build if the database already
|
||||
exists unless --force is passed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import csv
|
||||
import functools
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import ssl
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
ARTIFACT_ROOT = Path(os.environ.get("EXAMPLES_ARTIFACTS_DIR", "."))
|
||||
DB_DIR = ARTIFACT_ROOT / "data"
|
||||
DB_PATH = DB_DIR / "usaspending.db"
|
||||
GLOSSARY_PATH = ARTIFACT_ROOT / "schema" / "glossary.md"
|
||||
|
||||
USASPENDING_API = "https://api.usaspending.gov"
|
||||
BULK_DOWNLOAD_ENDPOINT = f"{USASPENDING_API}/api/v2/bulk_download/awards/"
|
||||
DOWNLOAD_STATUS_ENDPOINT = f"{USASPENDING_API}/api/v2/download/status"
|
||||
GLOSSARY_ENDPOINT = f"{USASPENDING_API}/api/v2/references/glossary/"
|
||||
|
||||
NASA_AGENCY = {
|
||||
"type": "awarding",
|
||||
"tier": "toptier",
|
||||
"name": "National Aeronautics and Space Administration",
|
||||
}
|
||||
|
||||
# Award type codes per the USAspending API contract.
|
||||
CONTRACT_CODES = ["A", "B", "C", "D"]
|
||||
GRANT_CODES = ["02", "03", "04", "05"]
|
||||
IDV_CODES = ["IDV_A", "IDV_B", "IDV_B_A", "IDV_B_B", "IDV_B_C", "IDV_C", "IDV_D", "IDV_E"]
|
||||
ALL_AWARD_CODES = CONTRACT_CODES + GRANT_CODES + IDV_CODES
|
||||
|
||||
AWARD_TYPE_MAP: dict[str, str] = {}
|
||||
for _code in CONTRACT_CODES:
|
||||
AWARD_TYPE_MAP[_code] = "contract"
|
||||
for _code in GRANT_CODES:
|
||||
AWARD_TYPE_MAP[_code] = "grant"
|
||||
for _code in IDV_CODES:
|
||||
AWARD_TYPE_MAP[_code] = "idv"
|
||||
|
||||
# Common headers — the USAspending WAF rejects requests without a User-Agent.
|
||||
_HEADERS = {
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": "USAspending-setup/1.0 (universal_computer example)",
|
||||
"Accept": "application/json",
|
||||
}
|
||||
|
||||
SCHEMA_SQL = """
|
||||
CREATE TABLE IF NOT EXISTS spending (
|
||||
rowid INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
award_id TEXT,
|
||||
award_piid_fain TEXT,
|
||||
parent_award_piid TEXT,
|
||||
award_type TEXT,
|
||||
description TEXT,
|
||||
action_date TEXT,
|
||||
fiscal_year INTEGER,
|
||||
federal_action_obligation REAL,
|
||||
total_obligation REAL,
|
||||
base_and_all_options_value REAL,
|
||||
recipient_name TEXT,
|
||||
recipient_parent_name TEXT,
|
||||
recipient_state TEXT,
|
||||
recipient_city TEXT,
|
||||
recipient_country TEXT,
|
||||
awarding_office TEXT,
|
||||
funding_office TEXT,
|
||||
naics_code TEXT,
|
||||
naics_description TEXT,
|
||||
psc_code TEXT,
|
||||
psc_description TEXT,
|
||||
place_of_performance_state TEXT,
|
||||
place_of_performance_city TEXT,
|
||||
period_of_perf_start TEXT,
|
||||
period_of_perf_end TEXT,
|
||||
extent_competed TEXT,
|
||||
type_of_set_aside TEXT,
|
||||
number_of_offers INTEGER,
|
||||
contract_pricing_type TEXT,
|
||||
business_types TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_award_id ON spending(award_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_fiscal_year ON spending(fiscal_year);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_award_type ON spending(award_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_recipient ON spending(recipient_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_recipient_parent ON spending(recipient_parent_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_state ON spending(recipient_state);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_action_date ON spending(action_date);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_naics ON spending(naics_code);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_obligation ON spending(federal_action_obligation);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_extent_competed ON spending(extent_competed);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_perf_start ON spending(period_of_perf_start);
|
||||
CREATE INDEX IF NOT EXISTS idx_spending_awarding_office ON spending(awarding_office);
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTTP helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@functools.cache
|
||||
def _urlopen_ssl_context() -> ssl.SSLContext | None:
|
||||
"""Use certifi's CA bundle when available, otherwise keep stdlib defaults."""
|
||||
try:
|
||||
import certifi
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
return ssl.create_default_context(cafile=certifi.where())
|
||||
|
||||
|
||||
def _urlopen_with_retry(
|
||||
req: urllib.request.Request, *, timeout: int = 60, retries: int = 3
|
||||
) -> bytes:
|
||||
"""urlopen with retries for the flaky USAspending endpoints."""
|
||||
last_exc: Exception | None = None
|
||||
ssl_context = _urlopen_ssl_context()
|
||||
for attempt in range(1, retries + 1):
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout, context=ssl_context) as resp:
|
||||
return bytes(resp.read())
|
||||
except (urllib.error.URLError, ConnectionError, OSError) as e:
|
||||
last_exc = e
|
||||
if attempt < retries:
|
||||
wait = 2**attempt
|
||||
print(f" Retry {attempt}/{retries} after error: {e} (waiting {wait}s)")
|
||||
time.sleep(wait)
|
||||
raise RuntimeError(f"Request failed after {retries} attempts: {last_exc}") from last_exc
|
||||
|
||||
|
||||
def api_post(url: str, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
"""POST JSON to a USAspending API endpoint and return the parsed response."""
|
||||
data = json.dumps(payload).encode("utf-8")
|
||||
req = urllib.request.Request(url, data=data, headers=_HEADERS, method="POST")
|
||||
body = _urlopen_with_retry(req)
|
||||
return json.loads(body.decode("utf-8")) # type: ignore[no-any-return]
|
||||
|
||||
|
||||
def api_get(url: str) -> dict[str, Any]:
|
||||
"""GET a USAspending API endpoint and return the parsed response."""
|
||||
req = urllib.request.Request(url, headers=_HEADERS)
|
||||
body = _urlopen_with_retry(req)
|
||||
return json.loads(body.decode("utf-8")) # type: ignore[no-any-return]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bulk download
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def submit_bulk_download(
|
||||
award_types: list[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Submit a bulk download request and return (status_url, file_url).
|
||||
|
||||
The USAspending bulk download API requires:
|
||||
- filters.agencies: list of agency objects (name/tier/type)
|
||||
- filters.prime_award_types: list of award type codes
|
||||
- filters.date_type: "action_date" or "last_modified_date"
|
||||
- filters.date_range: {start_date, end_date} (max 1 year span)
|
||||
|
||||
This only submits the request — call poll_download_status() to wait for completion.
|
||||
"""
|
||||
payload = {
|
||||
"filters": {
|
||||
"agencies": [NASA_AGENCY],
|
||||
"prime_award_types": award_types,
|
||||
"date_type": "action_date",
|
||||
"date_range": {
|
||||
"start_date": start_date,
|
||||
"end_date": end_date,
|
||||
},
|
||||
},
|
||||
"file_format": "csv",
|
||||
}
|
||||
|
||||
resp = api_post(BULK_DOWNLOAD_ENDPOINT, payload)
|
||||
file_url = resp.get("file_url")
|
||||
status_url = resp.get("status_url")
|
||||
|
||||
if not status_url and not file_url:
|
||||
raise RuntimeError(f"Unexpected API response: {resp}")
|
||||
|
||||
return status_url, file_url
|
||||
|
||||
|
||||
def poll_download_status(status_url: str | None, file_url: str | None) -> str:
|
||||
"""Poll the download status endpoint until the file is ready."""
|
||||
if not status_url:
|
||||
if file_url:
|
||||
return file_url
|
||||
raise RuntimeError("No status_url or file_url to poll")
|
||||
|
||||
for attempt in range(120):
|
||||
try:
|
||||
status = api_get(status_url)
|
||||
except Exception:
|
||||
time.sleep(5)
|
||||
continue
|
||||
|
||||
state = status.get("status", "unknown")
|
||||
if state == "finished":
|
||||
return status.get("file_url") or file_url or ""
|
||||
elif state == "failed":
|
||||
raise RuntimeError(f"Download generation failed: {status.get('message', 'unknown')}")
|
||||
|
||||
if attempt % 6 == 0:
|
||||
print(f" Generating... (status: {state})")
|
||||
time.sleep(5)
|
||||
|
||||
raise RuntimeError("Timed out waiting for download (10 minutes)")
|
||||
|
||||
|
||||
def download_and_extract(file_url: str, extract_dir: Path) -> list[Path]:
|
||||
"""Download a zip file and extract CSVs to extract_dir."""
|
||||
extract_dir.mkdir(parents=True, exist_ok=True)
|
||||
zip_path = extract_dir / "download.zip"
|
||||
|
||||
print(" Downloading...")
|
||||
req = urllib.request.Request(file_url, headers={"User-Agent": _HEADERS["User-Agent"]})
|
||||
data = _urlopen_with_retry(req, timeout=300, retries=3)
|
||||
zip_path.write_bytes(data)
|
||||
file_size_mb = len(data) / (1024 * 1024)
|
||||
print(f" Downloaded {file_size_mb:.1f} MB")
|
||||
|
||||
print(" Extracting CSV files...")
|
||||
csv_files = []
|
||||
with zipfile.ZipFile(zip_path, "r") as zf:
|
||||
for name in zf.namelist():
|
||||
if name.endswith(".csv"):
|
||||
zf.extract(name, extract_dir)
|
||||
csv_files.append(extract_dir / name)
|
||||
print(f" {name}")
|
||||
|
||||
zip_path.unlink()
|
||||
return csv_files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CSV ingestion
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def safe_float(val: str) -> float | None:
|
||||
if not val or val.strip() == "":
|
||||
return None
|
||||
try:
|
||||
return float(val.replace(",", ""))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def safe_int(val: str) -> int | None:
|
||||
if not val or val.strip() == "":
|
||||
return None
|
||||
try:
|
||||
return int(val.strip())
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def classify_award_type(type_code: str, award_id: str) -> str:
|
||||
mapped = AWARD_TYPE_MAP.get(type_code)
|
||||
if mapped:
|
||||
return mapped
|
||||
# Fallback: detect IDVs from the award_id prefix when the type code
|
||||
# doesn't match our expected IDV codes.
|
||||
if award_id.startswith("CONT_IDV_"):
|
||||
return "idv"
|
||||
return "other"
|
||||
|
||||
|
||||
def _detect_csv_type(headers: set[str]) -> str:
|
||||
"""Detect whether a CSV is contracts or assistance based on its headers.
|
||||
|
||||
Per the USAspending data dictionary, PrimeAwardUniqueKey is stored as
|
||||
'contract_award_unique_key' in contracts and 'assistance_award_unique_key'
|
||||
in assistance.
|
||||
"""
|
||||
if "contract_award_unique_key" in headers:
|
||||
return "contracts"
|
||||
if "assistance_award_unique_key" in headers:
|
||||
return "assistance"
|
||||
raise ValueError(
|
||||
"Cannot detect CSV type: neither 'contract_award_unique_key' nor "
|
||||
"'assistance_award_unique_key' found in headers"
|
||||
)
|
||||
|
||||
|
||||
# Column mappings per CSV type, derived from the USAspending data dictionary
|
||||
# (https://api.usaspending.gov/api/v2/references/data_dictionary/).
|
||||
#
|
||||
# "shared" columns have the same name in both contracts and assistance CSVs.
|
||||
# Type-specific columns are listed under "contracts" and "assistance".
|
||||
|
||||
# Column mappings verified against actual CSV headers downloaded from USAspending
|
||||
# on 2026-03-26, and cross-referenced with the data dictionary API at
|
||||
# https://api.usaspending.gov/api/v2/references/data_dictionary/.
|
||||
#
|
||||
# "shared" columns have the same name in both contracts and assistance CSVs.
|
||||
# Type-specific columns differ between the two and are listed separately.
|
||||
|
||||
_SHARED_COLUMNS = {
|
||||
# db_column -> csv_column
|
||||
"action_date": "action_date",
|
||||
"fiscal_year": "action_date_fiscal_year",
|
||||
"federal_action_obligation": "federal_action_obligation",
|
||||
"recipient_name": "recipient_name",
|
||||
"recipient_state": "recipient_state_code",
|
||||
"recipient_city": "recipient_city_name",
|
||||
"recipient_country": "recipient_country_name",
|
||||
"awarding_office": "awarding_office_name",
|
||||
"funding_office": "funding_office_name",
|
||||
"description": "transaction_description",
|
||||
"place_of_performance_city": "primary_place_of_performance_city_name",
|
||||
"period_of_perf_start": "period_of_performance_start_date",
|
||||
"period_of_perf_end": "period_of_performance_current_end_date",
|
||||
}
|
||||
|
||||
_TYPE_COLUMNS: dict[str, dict[str, str]] = {
|
||||
"contracts": {
|
||||
"award_id": "contract_award_unique_key",
|
||||
"award_piid_fain": "award_id_piid",
|
||||
"parent_award_piid": "parent_award_id_piid",
|
||||
"award_type_code": "award_type_code",
|
||||
"total_obligation": "total_dollars_obligated",
|
||||
"base_and_all_options_value": "base_and_all_options_value",
|
||||
"recipient_parent_name": "recipient_parent_name",
|
||||
"place_of_performance_state": "primary_place_of_performance_state_code",
|
||||
"naics_code": "naics_code",
|
||||
"naics_description": "naics_description",
|
||||
"psc_code": "product_or_service_code",
|
||||
"psc_description": "product_or_service_code_description",
|
||||
"extent_competed": "extent_competed",
|
||||
"type_of_set_aside": "type_of_set_aside",
|
||||
"number_of_offers": "number_of_offers_received",
|
||||
"contract_pricing_type": "type_of_contract_pricing",
|
||||
"business_types": "", # not present in contracts CSVs
|
||||
},
|
||||
"assistance": {
|
||||
"award_id": "assistance_award_unique_key",
|
||||
"award_piid_fain": "award_id_fain",
|
||||
"parent_award_piid": "", # not applicable to assistance
|
||||
"award_type_code": "assistance_type_code",
|
||||
"total_obligation": "total_obligated_amount",
|
||||
"base_and_all_options_value": "", # contracts only
|
||||
"recipient_parent_name": "", # contracts only
|
||||
"place_of_performance_state": "primary_place_of_performance_state_name",
|
||||
"naics_code": "", # not present in assistance CSVs
|
||||
"naics_description": "",
|
||||
"psc_code": "cfda_number",
|
||||
"psc_description": "cfda_title",
|
||||
"extent_competed": "", # contracts only
|
||||
"type_of_set_aside": "", # contracts only
|
||||
"number_of_offers": "", # contracts only
|
||||
"contract_pricing_type": "", # contracts only
|
||||
"business_types": "business_types_description",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def ingest_csv(db: sqlite3.Connection, csv_path: Path) -> int:
|
||||
"""Ingest a USAspending prime transactions CSV into the spending table."""
|
||||
count = 0
|
||||
|
||||
with open(csv_path, encoding="utf-8", errors="replace") as f:
|
||||
reader = csv.DictReader(f)
|
||||
if reader.fieldnames is None:
|
||||
return 0
|
||||
|
||||
headers = set(reader.fieldnames)
|
||||
csv_type = _detect_csv_type(headers)
|
||||
type_cols = _TYPE_COLUMNS[csv_type]
|
||||
|
||||
# Verify expected columns exist
|
||||
all_expected = dict(_SHARED_COLUMNS)
|
||||
all_expected.update(type_cols)
|
||||
missing = [
|
||||
db_col for db_col, csv_col in all_expected.items() if csv_col and csv_col not in headers
|
||||
]
|
||||
if missing:
|
||||
print(f" Warning: missing expected columns: {missing}")
|
||||
|
||||
award_id_col = type_cols["award_id"]
|
||||
award_type_col = type_cols["award_type_code"]
|
||||
|
||||
for row in reader:
|
||||
award_id = row.get(award_id_col, "")
|
||||
if not award_id:
|
||||
continue
|
||||
|
||||
type_code = row.get(award_type_col, "")
|
||||
award_type = classify_award_type(type_code, award_id)
|
||||
|
||||
def col(db_name: str, _row: dict[str, str] = row) -> str:
|
||||
"""Look up a value: type-specific columns first, then shared."""
|
||||
csv_col = type_cols.get(db_name) or _SHARED_COLUMNS.get(db_name, "")
|
||||
return _row.get(csv_col, "") if csv_col else ""
|
||||
|
||||
db.execute(
|
||||
"""INSERT INTO spending
|
||||
(award_id, award_piid_fain, parent_award_piid,
|
||||
award_type, description, action_date, fiscal_year,
|
||||
federal_action_obligation, total_obligation, base_and_all_options_value,
|
||||
recipient_name, recipient_parent_name,
|
||||
recipient_state, recipient_city, recipient_country,
|
||||
awarding_office, funding_office,
|
||||
naics_code, naics_description, psc_code, psc_description,
|
||||
place_of_performance_state, place_of_performance_city,
|
||||
period_of_perf_start, period_of_perf_end,
|
||||
extent_competed, type_of_set_aside, number_of_offers,
|
||||
contract_pricing_type, business_types)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
|
||||
(
|
||||
award_id,
|
||||
col("award_piid_fain"),
|
||||
col("parent_award_piid"),
|
||||
award_type,
|
||||
col("description"),
|
||||
col("action_date"),
|
||||
safe_int(col("fiscal_year")),
|
||||
safe_float(col("federal_action_obligation")),
|
||||
safe_float(col("total_obligation")),
|
||||
safe_float(col("base_and_all_options_value")),
|
||||
col("recipient_name"),
|
||||
col("recipient_parent_name"),
|
||||
col("recipient_state"),
|
||||
col("recipient_city"),
|
||||
col("recipient_country"),
|
||||
col("awarding_office"),
|
||||
col("funding_office"),
|
||||
col("naics_code"),
|
||||
col("naics_description"),
|
||||
col("psc_code"),
|
||||
col("psc_description"),
|
||||
col("place_of_performance_state"),
|
||||
col("place_of_performance_city"),
|
||||
col("period_of_perf_start"),
|
||||
col("period_of_perf_end"),
|
||||
col("extent_competed"),
|
||||
col("type_of_set_aside"),
|
||||
safe_int(col("number_of_offers")),
|
||||
col("contract_pricing_type"),
|
||||
col("business_types"),
|
||||
),
|
||||
)
|
||||
count += 1
|
||||
|
||||
return count
|
||||
|
||||
|
||||
def build_database(csv_files: list[Path]) -> None:
|
||||
"""Build the SQLite database from extracted CSV files."""
|
||||
DB_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f"Creating database at {DB_PATH}...")
|
||||
db = sqlite3.connect(str(DB_PATH))
|
||||
db.executescript(SCHEMA_SQL)
|
||||
|
||||
total = 0
|
||||
for csv_path in csv_files:
|
||||
print(f" Ingesting {csv_path.name}...")
|
||||
count = ingest_csv(db, csv_path)
|
||||
total += count
|
||||
print(f" {count:,} rows")
|
||||
|
||||
db.commit()
|
||||
|
||||
cursor = db.execute("SELECT COUNT(*) FROM spending")
|
||||
rows_stored = cursor.fetchone()[0]
|
||||
cursor = db.execute("SELECT COUNT(DISTINCT award_id) FROM spending")
|
||||
unique_awards = cursor.fetchone()[0]
|
||||
db.close()
|
||||
|
||||
db_size_mb = DB_PATH.stat().st_size / (1024 * 1024)
|
||||
print(f"\nDatabase built: {DB_PATH}")
|
||||
print(f" Rows: {rows_stored:,}")
|
||||
print(f" Unique awards: {unique_awards:,}")
|
||||
print(f" Size: {db_size_mb:.1f} MB")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Glossary
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def fetch_glossary() -> None:
|
||||
"""Fetch the official USAspending glossary and write it to schema/glossary.md."""
|
||||
if GLOSSARY_PATH.exists():
|
||||
print(f"Glossary already exists at {GLOSSARY_PATH}, skipping.")
|
||||
return
|
||||
|
||||
GLOSSARY_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print("Fetching USAspending glossary...")
|
||||
try:
|
||||
resp = api_get(f"{GLOSSARY_ENDPOINT}?limit=500")
|
||||
except Exception as e:
|
||||
print(f" Warning: failed to fetch glossary: {e}")
|
||||
return
|
||||
|
||||
results = resp.get("results", [])
|
||||
if not results:
|
||||
print(" Warning: glossary API returned no results.")
|
||||
return
|
||||
|
||||
results.sort(key=lambda t: t.get("term", "").lower())
|
||||
|
||||
lines = [
|
||||
"# USAspending Glossary",
|
||||
"",
|
||||
"Official definitions from [USAspending.gov](https://www.usaspending.gov).",
|
||||
f"Retrieved automatically by setup_db.py ({len(results)} terms).",
|
||||
"",
|
||||
]
|
||||
|
||||
for entry in results:
|
||||
term = entry.get("term", "").strip()
|
||||
plain = (entry.get("plain") or "").strip()
|
||||
official = (entry.get("official") or "").strip()
|
||||
|
||||
if not term:
|
||||
continue
|
||||
|
||||
lines.append(f"## {term}")
|
||||
lines.append("")
|
||||
if plain:
|
||||
lines.append(plain)
|
||||
lines.append("")
|
||||
if official and official != plain:
|
||||
lines.append(f"**Official definition:** {official}")
|
||||
lines.append("")
|
||||
|
||||
GLOSSARY_PATH.write_text("\n".join(lines), encoding="utf-8")
|
||||
print(f" Wrote {len(results)} glossary terms to {GLOSSARY_PATH}")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def fiscal_year_dates(fy: int) -> tuple[str, str]:
|
||||
"""Return (start_date, end_date) for a federal fiscal year.
|
||||
|
||||
Federal FY runs Oct 1 of the prior calendar year through Sep 30.
|
||||
Example: FY2024 = 2023-10-01 to 2024-09-30.
|
||||
"""
|
||||
return f"{fy - 1}-10-01", f"{fy}-09-30"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build NASA USAspending SQLite database")
|
||||
parser.add_argument("--force", action="store_true", help="Rebuild even if database exists")
|
||||
parser.add_argument(
|
||||
"--start-fy", type=int, default=2021, help="First fiscal year to download (default: 2021)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--end-fy", type=int, default=2025, help="Last fiscal year to download (default: 2025)"
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.start_fy > args.end_fy:
|
||||
parser.error(f"--start-fy ({args.start_fy}) must be <= --end-fy ({args.end_fy})")
|
||||
|
||||
requested_fys = set(range(args.start_fy, args.end_fy + 1))
|
||||
|
||||
if DB_PATH.exists() and not args.force:
|
||||
# Verify the existing DB covers all requested fiscal years.
|
||||
try:
|
||||
conn = sqlite3.connect(f"file:{DB_PATH}?mode=ro", uri=True)
|
||||
rows = conn.execute("SELECT DISTINCT fiscal_year FROM spending").fetchall()
|
||||
conn.close()
|
||||
present_fys = {int(r[0]) for r in rows if r[0] is not None}
|
||||
missing_fys = requested_fys - present_fys
|
||||
if not missing_fys:
|
||||
db_size_mb = DB_PATH.stat().st_size / (1024 * 1024)
|
||||
print(
|
||||
f"Database already exists at {DB_PATH} ({db_size_mb:.1f} MB) "
|
||||
f"with all requested FYs. Use --force to rebuild."
|
||||
)
|
||||
return
|
||||
print(
|
||||
f"Database exists but is missing FY data for: "
|
||||
f"{', '.join(str(fy) for fy in sorted(missing_fys))}. Rebuilding..."
|
||||
)
|
||||
except Exception:
|
||||
print("Database exists but could not be verified. Rebuilding...")
|
||||
DB_PATH.unlink()
|
||||
elif DB_PATH.exists():
|
||||
DB_PATH.unlink()
|
||||
|
||||
tmp_dir = DB_DIR / "tmp_download"
|
||||
|
||||
print("=== NASA USAspending Database Builder ===")
|
||||
print(f"Fiscal years: {args.start_fy} - {args.end_fy}\n")
|
||||
|
||||
# The bulk download API limits date_range to 1 year, so we request
|
||||
# one fiscal year at a time. We submit all requests upfront so the
|
||||
# server-side assembly (the slow part) runs concurrently, then poll
|
||||
# and download the results.
|
||||
all_csv_files: list[Path] = []
|
||||
failed_fys: list[int] = []
|
||||
fiscal_years = list(range(args.start_fy, args.end_fy + 1))
|
||||
|
||||
# Phase 1: Submit all bulk download requests concurrently.
|
||||
print("Submitting download requests...")
|
||||
pending: dict[int, tuple[str | None, str | None]] = {}
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=len(fiscal_years)) as pool:
|
||||
|
||||
def _submit(fy: int) -> tuple[int, str | None, str | None]:
|
||||
start_date, end_date = fiscal_year_dates(fy)
|
||||
status_url, file_url = submit_bulk_download(
|
||||
ALL_AWARD_CODES,
|
||||
start_date,
|
||||
end_date,
|
||||
)
|
||||
return fy, status_url, file_url
|
||||
|
||||
futures = {pool.submit(_submit, fy): fy for fy in fiscal_years}
|
||||
for future in concurrent.futures.as_completed(futures):
|
||||
fy = futures[future]
|
||||
try:
|
||||
_, status_url, file_url = future.result()
|
||||
pending[fy] = (status_url, file_url)
|
||||
print(f" FY{fy}: submitted")
|
||||
except Exception as e:
|
||||
print(f" FY{fy}: submit failed: {e}")
|
||||
failed_fys.append(fy)
|
||||
|
||||
# Phase 2: Poll all pending requests until ready, then download.
|
||||
for fy in sorted(pending):
|
||||
print(f"\n--- FY{fy} ---")
|
||||
status_url, file_url = pending[fy]
|
||||
try:
|
||||
file_url = poll_download_status(status_url, file_url)
|
||||
print(f" Ready: {file_url}")
|
||||
fy_dir = tmp_dir / f"fy{fy}"
|
||||
csv_files = download_and_extract(file_url, fy_dir)
|
||||
all_csv_files.extend(csv_files)
|
||||
except Exception as e:
|
||||
print(f" Error: failed FY{fy}: {e}")
|
||||
failed_fys.append(fy)
|
||||
|
||||
if not all_csv_files:
|
||||
print("\nError: no data downloaded. Check internet connectivity.")
|
||||
sys.exit(1)
|
||||
|
||||
if failed_fys:
|
||||
print(
|
||||
f"\nError: failed to download data for: "
|
||||
f"{', '.join(f'FY{fy}' for fy in failed_fys)}. "
|
||||
f"Cannot build a complete database."
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
print("\n--- Fetching glossary ---")
|
||||
fetch_glossary()
|
||||
|
||||
print("\n--- Building database ---")
|
||||
build_database(all_csv_files)
|
||||
|
||||
# Verify the built DB covers all requested fiscal years.
|
||||
conn = sqlite3.connect(f"file:{DB_PATH}?mode=ro", uri=True)
|
||||
rows = conn.execute("SELECT DISTINCT fiscal_year FROM spending").fetchall()
|
||||
conn.close()
|
||||
present_fys = {int(r[0]) for r in rows if r[0] is not None}
|
||||
missing_fys = requested_fys - present_fys
|
||||
if missing_fys:
|
||||
print(
|
||||
f"\nError: database built but missing data for: "
|
||||
f"{', '.join(f'FY{fy}' for fy in sorted(missing_fys))}. "
|
||||
f"Downloaded files may have been empty."
|
||||
)
|
||||
DB_PATH.unlink()
|
||||
sys.exit(1)
|
||||
|
||||
# Clean up temp files
|
||||
for f in tmp_dir.rglob("*"):
|
||||
if f.is_file():
|
||||
f.unlink()
|
||||
for d in sorted(tmp_dir.rglob("*"), reverse=True):
|
||||
if d.is_dir():
|
||||
d.rmdir()
|
||||
if tmp_dir.exists():
|
||||
tmp_dir.rmdir()
|
||||
|
||||
print("\nDone!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,175 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import textwrap
|
||||
from typing import Any, Literal
|
||||
|
||||
from agents.sandbox import Capability, ExecTimeoutError, Manifest
|
||||
from agents.sandbox.session.base_sandbox_session import BaseSandboxSession
|
||||
from agents.tool import FunctionTool
|
||||
|
||||
# Python script executed inside the sandbox to run SQL queries safely.
|
||||
# Receives the query on stdin, enforces read-only mode and row limits.
|
||||
_QUERY_RUNNER_SCRIPT = r"""
|
||||
import csv, json, os, sqlite3, sys, time
|
||||
|
||||
db_path = sys.argv[1]
|
||||
display_limit = int(sys.argv[2])
|
||||
csv_limit = int(sys.argv[3])
|
||||
results_dir = sys.argv[4] if len(sys.argv) > 4 else ""
|
||||
|
||||
query = sys.stdin.read().strip()
|
||||
if not query:
|
||||
print("Error: empty query")
|
||||
sys.exit(0)
|
||||
|
||||
# Statement-level validation: only allow read-only operations
|
||||
first_token = query.lstrip().split()[0].upper() if query.strip() else ""
|
||||
if first_token not in ("SELECT", "WITH", "EXPLAIN", "PRAGMA"):
|
||||
print(f"Error: only SELECT, WITH, EXPLAIN, and PRAGMA statements are allowed (got {first_token})")
|
||||
sys.exit(0)
|
||||
|
||||
try:
|
||||
conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)
|
||||
conn.execute("PRAGMA query_only = ON")
|
||||
cursor = conn.execute(query)
|
||||
columns = [desc[0] for desc in cursor.description] if cursor.description else []
|
||||
rows = cursor.fetchmany(csv_limit + 1)
|
||||
conn.close()
|
||||
except sqlite3.Error as e:
|
||||
print(f"SQL error: {e}")
|
||||
sys.exit(0)
|
||||
|
||||
if not columns:
|
||||
print(json.dumps({"columns": [], "rows": [], "row_count": 0, "truncated": False}))
|
||||
sys.exit(0)
|
||||
|
||||
csv_truncated = len(rows) > csv_limit
|
||||
if csv_truncated:
|
||||
rows = rows[:csv_limit]
|
||||
|
||||
# Save full result as CSV for download
|
||||
csv_file = ""
|
||||
if results_dir:
|
||||
os.makedirs(results_dir, exist_ok=True)
|
||||
csv_file = f"query_{int(time.time())}_{os.getpid()}.csv"
|
||||
with open(os.path.join(results_dir, csv_file), "w", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(columns)
|
||||
writer.writerows(rows)
|
||||
|
||||
# Return only display_limit rows to the model, but report total counts
|
||||
total_rows = len(rows)
|
||||
display_rows = rows[:display_limit]
|
||||
|
||||
result = {
|
||||
"columns": columns,
|
||||
"rows": display_rows,
|
||||
"row_count": total_rows,
|
||||
"display_count": len(display_rows),
|
||||
"truncated": csv_truncated,
|
||||
}
|
||||
if csv_file:
|
||||
result["csv_file"] = csv_file
|
||||
if total_rows > len(display_rows):
|
||||
result["note"] = f"Showing {len(display_rows)} of {total_rows} rows. Full result saved to CSV."
|
||||
|
||||
print(json.dumps(result))
|
||||
"""
|
||||
|
||||
|
||||
def _shell_quote(s: str) -> str:
|
||||
"""Single-quote a string for safe shell interpolation."""
|
||||
return "'" + s.replace("'", "'\\''") + "'"
|
||||
|
||||
|
||||
_SQL_CAPABILITY_INSTRUCTIONS = textwrap.dedent(
|
||||
"""\
|
||||
When querying the database:
|
||||
- Always use `run_sql` to execute SQL. Never run sqlite3 directly via a shell.
|
||||
- Write standard SQLite-compatible SQL.
|
||||
- Prefer aggregations (GROUP BY, SUM, COUNT, AVG) over returning many raw rows.
|
||||
- The display shows up to 100 rows, but up to 10,000 rows are saved to a downloadable CSV.
|
||||
If the user needs a large export, let them know the full result is available via the download link.
|
||||
- Use the schema documentation files in schema/tables/ if you need column details.
|
||||
- Read schema/glossary.md for official definitions of USAspending terms.
|
||||
- For monetary values, the database stores amounts in dollars as REAL values.
|
||||
"""
|
||||
).strip()
|
||||
|
||||
|
||||
def _make_run_sql_tool(
|
||||
session: BaseSandboxSession,
|
||||
db_path: str,
|
||||
max_display_rows: int,
|
||||
max_csv_rows: int,
|
||||
timeout_seconds: float,
|
||||
results_dir: str,
|
||||
) -> FunctionTool:
|
||||
"""Build a FunctionTool that executes read-only SQL inside the sandbox."""
|
||||
|
||||
async def run_sql(query: str, limit: int | None = None) -> str:
|
||||
"""Execute a read-only SQL query against the NASA USAspending SQLite database.
|
||||
|
||||
Returns results as JSON with columns, rows, row_count, and truncated fields.
|
||||
Results are also saved as a downloadable CSV. The display is limited to a
|
||||
small number of rows, but the CSV may contain many more.
|
||||
|
||||
Args:
|
||||
query: SQL SELECT query to execute against the USAspending database.
|
||||
Only read-only queries are allowed.
|
||||
limit: Optional display row limit override.
|
||||
"""
|
||||
display_limit = max(1, min(limit or max_display_rows, max_display_rows))
|
||||
|
||||
command = (
|
||||
f"printf '%s' {_shell_quote(query)} "
|
||||
f"| python3 -c {_shell_quote(_QUERY_RUNNER_SCRIPT)} "
|
||||
f"{_shell_quote(db_path)} {display_limit} {max_csv_rows}"
|
||||
f" {_shell_quote(results_dir)}"
|
||||
)
|
||||
|
||||
try:
|
||||
result = await session.exec(command, timeout=timeout_seconds)
|
||||
except (ExecTimeoutError, TimeoutError):
|
||||
return f"Query timed out after {timeout_seconds}s. Try a simpler query or add a LIMIT."
|
||||
|
||||
output = result.stdout.decode("utf-8", errors="replace")
|
||||
stderr = result.stderr.decode("utf-8", errors="replace")
|
||||
|
||||
if not result.ok():
|
||||
return f"Execution error (exit {result.exit_code}):\n{stderr or output}"
|
||||
|
||||
return output.strip() if output.strip() else "Query returned no results."
|
||||
|
||||
from agents.tool import function_tool as _function_tool
|
||||
|
||||
return _function_tool(run_sql, name_override="run_sql")
|
||||
|
||||
|
||||
class SqlCapability(Capability):
|
||||
type: Literal["sql"] = "sql"
|
||||
db_path: str = "data/usaspending.db"
|
||||
max_display_rows: int = 100
|
||||
max_csv_rows: int = 10_000
|
||||
timeout_seconds: float = 30.0
|
||||
results_dir: str = "results"
|
||||
|
||||
def bind(self, session: BaseSandboxSession) -> None:
|
||||
self.session = session
|
||||
|
||||
def tools(self) -> list[Any]:
|
||||
if self.session is None:
|
||||
raise ValueError("SqlCapability is not bound to a SandboxSession")
|
||||
return [
|
||||
_make_run_sql_tool(
|
||||
session=self.session,
|
||||
db_path=self.db_path,
|
||||
max_display_rows=self.max_display_rows,
|
||||
max_csv_rows=self.max_csv_rows,
|
||||
timeout_seconds=self.timeout_seconds,
|
||||
results_dir=self.results_dir,
|
||||
)
|
||||
]
|
||||
|
||||
async def instructions(self, manifest: Manifest) -> str | None:
|
||||
return _SQL_CAPABILITY_INSTRUCTIONS
|
||||
@@ -0,0 +1,273 @@
|
||||
"""
|
||||
Minimal E2B-backed sandbox example for manual validation.
|
||||
|
||||
This example is intentionally small: it creates a tiny workspace, lets the
|
||||
agent inspect it through one shell tool, and prints a short answer.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import LocalSnapshotSpec, Manifest, SandboxAgent, SandboxRunConfig
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
E2BSandboxClient,
|
||||
E2BSandboxClientOptions,
|
||||
E2BSandboxType,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"E2B sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra e2b"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
DEFAULT_SANDBOX_TYPE = E2BSandboxType.E2B.value
|
||||
SNAPSHOT_CHECK_PATH = Path("snapshot-check.txt")
|
||||
SNAPSHOT_CHECK_CONTENT = "e2b snapshot round-trip ok\n"
|
||||
|
||||
|
||||
def _build_manifest() -> Manifest:
|
||||
return text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Renewal Notes\n\n"
|
||||
"This workspace contains a tiny account review packet for manual sandbox testing.\n"
|
||||
),
|
||||
"customer.md": (
|
||||
"# Customer\n\n"
|
||||
"- Name: Northwind Health.\n"
|
||||
"- Renewal date: 2026-04-15.\n"
|
||||
"- Risk: unresolved SSO setup.\n"
|
||||
),
|
||||
"next_steps.md": (
|
||||
"# Next steps\n\n"
|
||||
"1. Finish the SSO fix.\n"
|
||||
"2. Confirm legal language before procurement review.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
def _rewrite_template_resolution_error(exc: Exception) -> None:
|
||||
message = str(exc)
|
||||
marker = "error resolving template '"
|
||||
if marker not in message:
|
||||
return
|
||||
template = message.split(marker, 1)[1].split("'", 1)[0]
|
||||
raise SystemExit(
|
||||
f"E2B could not resolve template `{template}`.\n"
|
||||
"Pass `--template <your-template>` with a template that exists for this E2B account/team. "
|
||||
"If you were relying on the example default, the SDK default template for this backend is "
|
||||
"not available in your current E2B environment."
|
||||
) from exc
|
||||
|
||||
|
||||
async def _verify_stop_resume(
|
||||
*,
|
||||
sandbox_type: Literal["e2b_code_interpreter", "e2b"],
|
||||
template: str | None,
|
||||
timeout: int | None,
|
||||
pause_on_exit: bool,
|
||||
workspace_persistence: Literal["tar", "snapshot"],
|
||||
) -> None:
|
||||
client = E2BSandboxClient()
|
||||
with tempfile.TemporaryDirectory(prefix="e2b-snapshot-example-") as snapshot_dir:
|
||||
sandbox = await client.create(
|
||||
manifest=_build_manifest(),
|
||||
snapshot=LocalSnapshotSpec(base_path=Path(snapshot_dir)),
|
||||
options=E2BSandboxClientOptions(
|
||||
sandbox_type=E2BSandboxType(sandbox_type),
|
||||
template=template,
|
||||
timeout=timeout,
|
||||
pause_on_exit=pause_on_exit,
|
||||
workspace_persistence=workspace_persistence,
|
||||
),
|
||||
)
|
||||
|
||||
try:
|
||||
await sandbox.start()
|
||||
await sandbox.write(
|
||||
SNAPSHOT_CHECK_PATH,
|
||||
io.BytesIO(SNAPSHOT_CHECK_CONTENT.encode("utf-8")),
|
||||
)
|
||||
await sandbox.stop()
|
||||
finally:
|
||||
await sandbox.shutdown()
|
||||
|
||||
resumed_sandbox = await client.resume(sandbox.state)
|
||||
try:
|
||||
await resumed_sandbox.start()
|
||||
restored = await resumed_sandbox.read(SNAPSHOT_CHECK_PATH)
|
||||
restored_text = restored.read()
|
||||
if isinstance(restored_text, bytes):
|
||||
restored_text = restored_text.decode("utf-8")
|
||||
if restored_text != SNAPSHOT_CHECK_CONTENT:
|
||||
raise RuntimeError(
|
||||
"Snapshot resume verification failed for "
|
||||
f"{sandbox_type!r}: expected {SNAPSHOT_CHECK_CONTENT!r}, got {restored_text!r}"
|
||||
)
|
||||
finally:
|
||||
await resumed_sandbox.shutdown()
|
||||
|
||||
print(f"snapshot round-trip ok ({sandbox_type}, {workspace_persistence})")
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
sandbox_type: Literal["e2b_code_interpreter", "e2b"],
|
||||
template: str | None,
|
||||
timeout: int | None,
|
||||
pause_on_exit: bool,
|
||||
workspace_persistence: Literal["tar", "snapshot"],
|
||||
stream: bool,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
_require_env("E2B_API_KEY")
|
||||
|
||||
try:
|
||||
await _verify_stop_resume(
|
||||
sandbox_type=sandbox_type,
|
||||
template=template,
|
||||
timeout=timeout,
|
||||
pause_on_exit=pause_on_exit,
|
||||
workspace_persistence=workspace_persistence,
|
||||
)
|
||||
except Exception as exc:
|
||||
_rewrite_template_resolution_error(exc)
|
||||
raise
|
||||
|
||||
manifest = _build_manifest()
|
||||
agent = SandboxAgent(
|
||||
name="E2B Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=E2BSandboxClient(),
|
||||
options=E2BSandboxClientOptions(
|
||||
sandbox_type=E2BSandboxType(sandbox_type),
|
||||
template=template,
|
||||
timeout=timeout,
|
||||
pause_on_exit=pause_on_exit,
|
||||
workspace_persistence=workspace_persistence,
|
||||
),
|
||||
),
|
||||
workflow_name="E2B sandbox example",
|
||||
)
|
||||
|
||||
if not stream:
|
||||
try:
|
||||
result = await Runner.run(agent, question, run_config=run_config)
|
||||
except Exception as exc:
|
||||
_rewrite_template_resolution_error(exc)
|
||||
raise
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
try:
|
||||
stream_result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
except Exception as exc:
|
||||
_rewrite_template_resolution_error(exc)
|
||||
raise
|
||||
saw_text_delta = False
|
||||
try:
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
except Exception as exc:
|
||||
_rewrite_template_resolution_error(exc)
|
||||
raise
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", default="gpt-5.6-sol", help="Model name to use.")
|
||||
parser.add_argument("--question", default=DEFAULT_QUESTION, help="Prompt to send to the agent.")
|
||||
parser.add_argument(
|
||||
"--sandbox-type",
|
||||
default=DEFAULT_SANDBOX_TYPE,
|
||||
choices=[member.value for member in E2BSandboxType],
|
||||
help=(
|
||||
"E2B sandbox interface to create. `e2b` provides a bash-style interface; "
|
||||
"`e2b_code_interpreter` provides a Jupyter-style interface."
|
||||
),
|
||||
)
|
||||
parser.add_argument("--template", default=None, help="Optional E2B template name.")
|
||||
parser.add_argument(
|
||||
"--timeout",
|
||||
type=int,
|
||||
default=300,
|
||||
help="Optional E2B sandbox timeout in seconds.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--pause-on-exit",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Pause the sandbox on shutdown instead of killing it.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--workspace-persistence",
|
||||
default="tar",
|
||||
choices=["tar", "snapshot"],
|
||||
help="Workspace persistence mode for the E2B sandbox.",
|
||||
)
|
||||
parser.add_argument("--stream", action="store_true", default=False, help="Stream the response.")
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
sandbox_type=args.sandbox_type,
|
||||
template=args.template,
|
||||
timeout=args.timeout,
|
||||
pause_on_exit=args.pause_on_exit,
|
||||
workspace_persistence=args.workspace_persistence,
|
||||
stream=args.stream,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,366 @@
|
||||
"""
|
||||
Minimal Modal-backed sandbox example for manual validation.
|
||||
|
||||
This example mirrors the local and Docker sandbox demos, but it sends the
|
||||
workspace to a Modal sandbox.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import LocalSnapshotSpec, Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.entries import GCSMount, Mount, S3Mount
|
||||
from agents.sandbox.session import BaseSandboxSession
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
ModalCloudBucketMountStrategy,
|
||||
ModalSandboxClient,
|
||||
ModalSandboxClientOptions,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"Modal sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra modal"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
SNAPSHOT_CHECK_PATH = Path("snapshot-check.txt")
|
||||
SNAPSHOT_CHECK_CONTENT = "modal snapshot round-trip ok\n"
|
||||
MOUNT_CHECK_FILENAME = "native-cloud-bucket-check.txt"
|
||||
MOUNT_CHECK_CONTENT = "modal native cloud bucket read/write ok\n"
|
||||
MOUNT_CHECK_UPDATED_CONTENT = "modal native cloud bucket read/write ok after resume\n"
|
||||
|
||||
|
||||
def _build_manifest(
|
||||
*,
|
||||
native_cloud_bucket_name: str | None = None,
|
||||
native_cloud_bucket_provider: Literal["s3", "gcs-hmac"] = "s3",
|
||||
native_cloud_bucket_mount_path: str | None = None,
|
||||
native_cloud_bucket_endpoint_url: str | None = None,
|
||||
native_cloud_bucket_key_prefix: str | None = None,
|
||||
native_cloud_bucket_secret_name: str | None = None,
|
||||
) -> Manifest:
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Modal Demo Workspace\n\n"
|
||||
"This workspace exists to validate the Modal sandbox backend manually.\n"
|
||||
),
|
||||
"incident.md": (
|
||||
"# Incident\n\n"
|
||||
"- Customer: Fabrikam Retail.\n"
|
||||
"- Issue: delayed reporting rollout.\n"
|
||||
"- Primary blocker: incomplete security questionnaire.\n"
|
||||
),
|
||||
"plan.md": (
|
||||
"# Plan\n\n"
|
||||
"1. Close the questionnaire.\n"
|
||||
"2. Reconfirm the rollout date with the customer.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
if native_cloud_bucket_name is None:
|
||||
return manifest
|
||||
|
||||
mount_path = (
|
||||
Path(native_cloud_bucket_mount_path) if native_cloud_bucket_mount_path is not None else None
|
||||
)
|
||||
mount_strategy = ModalCloudBucketMountStrategy(
|
||||
secret_name=native_cloud_bucket_secret_name,
|
||||
)
|
||||
if native_cloud_bucket_provider == "gcs-hmac":
|
||||
manifest.entries["cloud-bucket"] = GCSMount(
|
||||
bucket=native_cloud_bucket_name,
|
||||
access_id=(
|
||||
None
|
||||
if native_cloud_bucket_secret_name is not None
|
||||
else (
|
||||
os.environ.get("GCS_HMAC_ACCESS_KEY_ID")
|
||||
or os.environ.get("GOOGLE_ACCESS_KEY_ID")
|
||||
)
|
||||
),
|
||||
secret_access_key=(
|
||||
None
|
||||
if native_cloud_bucket_secret_name is not None
|
||||
else (
|
||||
os.environ.get("GCS_HMAC_SECRET_ACCESS_KEY")
|
||||
or os.environ.get("GOOGLE_ACCESS_KEY_SECRET")
|
||||
)
|
||||
),
|
||||
endpoint_url=native_cloud_bucket_endpoint_url,
|
||||
prefix=native_cloud_bucket_key_prefix,
|
||||
mount_path=mount_path,
|
||||
read_only=False,
|
||||
mount_strategy=mount_strategy,
|
||||
)
|
||||
else:
|
||||
manifest.entries["cloud-bucket"] = S3Mount(
|
||||
bucket=native_cloud_bucket_name,
|
||||
access_key_id=(
|
||||
None
|
||||
if native_cloud_bucket_secret_name is not None
|
||||
else os.environ.get("AWS_ACCESS_KEY_ID")
|
||||
),
|
||||
secret_access_key=(
|
||||
None
|
||||
if native_cloud_bucket_secret_name is not None
|
||||
else os.environ.get("AWS_SECRET_ACCESS_KEY")
|
||||
),
|
||||
session_token=(
|
||||
None
|
||||
if native_cloud_bucket_secret_name is not None
|
||||
else os.environ.get("AWS_SESSION_TOKEN")
|
||||
),
|
||||
endpoint_url=native_cloud_bucket_endpoint_url,
|
||||
prefix=native_cloud_bucket_key_prefix,
|
||||
mount_path=mount_path,
|
||||
read_only=False,
|
||||
mount_strategy=mount_strategy,
|
||||
)
|
||||
return manifest
|
||||
|
||||
|
||||
def _native_cloud_bucket_mount_path(manifest: Manifest) -> Path | None:
|
||||
entry = manifest.entries.get("cloud-bucket")
|
||||
if not isinstance(entry, Mount):
|
||||
return None
|
||||
if entry.mount_path is None:
|
||||
return Path(manifest.root) / "cloud-bucket"
|
||||
if entry.mount_path.is_absolute():
|
||||
return entry.mount_path
|
||||
return Path(manifest.root) / entry.mount_path
|
||||
|
||||
|
||||
async def _read_text(session: BaseSandboxSession, path: Path) -> str:
|
||||
data = await session.read(path)
|
||||
text = cast(str | bytes, data.read())
|
||||
if isinstance(text, bytes):
|
||||
return text.decode("utf-8")
|
||||
return text
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
async def _verify_stop_resume(
|
||||
*,
|
||||
manifest: Manifest,
|
||||
app_name: str,
|
||||
workspace_persistence: Literal["tar", "snapshot_filesystem", "snapshot_directory"],
|
||||
sandbox_create_timeout_s: float | None,
|
||||
) -> None:
|
||||
client = ModalSandboxClient()
|
||||
mount_path = _native_cloud_bucket_mount_path(manifest)
|
||||
mount_check_path = mount_path / MOUNT_CHECK_FILENAME if mount_path is not None else None
|
||||
options = ModalSandboxClientOptions(
|
||||
app_name=app_name,
|
||||
workspace_persistence=workspace_persistence,
|
||||
sandbox_create_timeout_s=sandbox_create_timeout_s,
|
||||
)
|
||||
with tempfile.TemporaryDirectory(prefix="modal-snapshot-example-") as snapshot_dir:
|
||||
sandbox = await client.create(
|
||||
manifest=manifest,
|
||||
snapshot=LocalSnapshotSpec(base_path=Path(snapshot_dir)),
|
||||
options=options,
|
||||
)
|
||||
|
||||
try:
|
||||
await sandbox.start()
|
||||
await sandbox.write(
|
||||
SNAPSHOT_CHECK_PATH,
|
||||
io.BytesIO(SNAPSHOT_CHECK_CONTENT.encode("utf-8")),
|
||||
)
|
||||
await sandbox.stop()
|
||||
finally:
|
||||
await sandbox.shutdown()
|
||||
|
||||
resumed_sandbox = await client.resume(sandbox.state)
|
||||
try:
|
||||
await resumed_sandbox.start()
|
||||
restored_text = await _read_text(resumed_sandbox, SNAPSHOT_CHECK_PATH)
|
||||
if restored_text != SNAPSHOT_CHECK_CONTENT:
|
||||
raise RuntimeError(
|
||||
f"Snapshot resume verification failed for {workspace_persistence!r}: "
|
||||
f"expected {SNAPSHOT_CHECK_CONTENT!r}, got {restored_text!r}"
|
||||
)
|
||||
finally:
|
||||
await resumed_sandbox.aclose()
|
||||
|
||||
print(f"native cloud bucket read/write ok ({mount_check_path})")
|
||||
print(f"snapshot round-trip ok ({workspace_persistence})")
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
app_name: str,
|
||||
workspace_persistence: Literal["tar", "snapshot_filesystem", "snapshot_directory"],
|
||||
sandbox_create_timeout_s: float | None,
|
||||
native_cloud_bucket_name: str | None,
|
||||
native_cloud_bucket_provider: Literal["s3", "gcs-hmac"],
|
||||
native_cloud_bucket_mount_path: str,
|
||||
native_cloud_bucket_endpoint_url: str | None,
|
||||
native_cloud_bucket_key_prefix: str | None,
|
||||
native_cloud_bucket_secret_name: str | None,
|
||||
stream: bool,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
manifest = _build_manifest(
|
||||
native_cloud_bucket_name=native_cloud_bucket_name,
|
||||
native_cloud_bucket_provider=native_cloud_bucket_provider,
|
||||
native_cloud_bucket_mount_path=native_cloud_bucket_mount_path,
|
||||
native_cloud_bucket_endpoint_url=native_cloud_bucket_endpoint_url,
|
||||
native_cloud_bucket_key_prefix=native_cloud_bucket_key_prefix,
|
||||
native_cloud_bucket_secret_name=native_cloud_bucket_secret_name,
|
||||
)
|
||||
|
||||
await _verify_stop_resume(
|
||||
manifest=manifest,
|
||||
app_name=app_name,
|
||||
workspace_persistence=workspace_persistence,
|
||||
sandbox_create_timeout_s=sandbox_create_timeout_s,
|
||||
)
|
||||
|
||||
agent = SandboxAgent(
|
||||
name="Modal Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=ModalSandboxClient(),
|
||||
options=ModalSandboxClientOptions(
|
||||
app_name=app_name,
|
||||
workspace_persistence=workspace_persistence,
|
||||
sandbox_create_timeout_s=sandbox_create_timeout_s,
|
||||
),
|
||||
),
|
||||
workflow_name="Modal sandbox example",
|
||||
)
|
||||
|
||||
if not stream:
|
||||
result = await Runner.run(agent, question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
stream_result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", default="gpt-5.6-sol", help="Model name to use.")
|
||||
parser.add_argument("--question", default=DEFAULT_QUESTION, help="Prompt to send to the agent.")
|
||||
parser.add_argument(
|
||||
"--app-name",
|
||||
default="openai-agents-python-sandbox-example",
|
||||
help="Modal app name to create or reuse for the sandbox.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--workspace-persistence",
|
||||
default="tar",
|
||||
choices=["tar", "snapshot_filesystem", "snapshot_directory"],
|
||||
help="Workspace persistence mode for the Modal sandbox.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--sandbox-create-timeout-s",
|
||||
type=float,
|
||||
default=None,
|
||||
help="Optional timeout for creating the Modal sandbox.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-name",
|
||||
default=None,
|
||||
help="Optional cloud bucket name to mount with ModalCloudBucketMountStrategy.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-provider",
|
||||
default="s3",
|
||||
choices=["s3", "gcs-hmac"],
|
||||
help="Provider type for --native-cloud-bucket-name.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-mount-path",
|
||||
default="cloud-bucket",
|
||||
help=(
|
||||
"Mount path for --native-cloud-bucket-name. Relative paths are resolved under the "
|
||||
"workspace root."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-endpoint-url",
|
||||
default=None,
|
||||
help="Optional endpoint URL for --native-cloud-bucket-name.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-key-prefix",
|
||||
default=None,
|
||||
help="Optional key prefix for --native-cloud-bucket-name.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--native-cloud-bucket-secret-name",
|
||||
default=None,
|
||||
help=(
|
||||
"Optional named Modal Secret to use for --native-cloud-bucket-name instead of "
|
||||
"reading raw credentials from environment variables."
|
||||
),
|
||||
)
|
||||
parser.add_argument("--stream", action="store_true", default=False, help="Stream the response.")
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
app_name=args.app_name,
|
||||
workspace_persistence=args.workspace_persistence,
|
||||
sandbox_create_timeout_s=args.sandbox_create_timeout_s,
|
||||
native_cloud_bucket_name=args.native_cloud_bucket_name,
|
||||
native_cloud_bucket_provider=args.native_cloud_bucket_provider,
|
||||
native_cloud_bucket_mount_path=args.native_cloud_bucket_mount_path,
|
||||
native_cloud_bucket_endpoint_url=args.native_cloud_bucket_endpoint_url,
|
||||
native_cloud_bucket_key_prefix=args.native_cloud_bucket_key_prefix,
|
||||
native_cloud_bucket_secret_name=args.native_cloud_bucket_secret_name,
|
||||
stream=args.stream,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,995 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal, cast
|
||||
from urllib.parse import urljoin
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
from pydantic import BaseModel
|
||||
|
||||
from agents import Agent, ModelSettings, Runner, function_tool
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[4]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest, tool_call_name
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
DEFAULT_RUNLOOP_ROOT_WORKSPACE_ROOT,
|
||||
DEFAULT_RUNLOOP_WORKSPACE_ROOT,
|
||||
RunloopAfterIdle,
|
||||
RunloopGatewaySpec,
|
||||
RunloopLaunchParameters,
|
||||
RunloopMcpSpec,
|
||||
RunloopSandboxClient,
|
||||
RunloopSandboxClientOptions,
|
||||
RunloopSandboxSessionState,
|
||||
RunloopTunnelConfig,
|
||||
RunloopUserParameters,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"Runloop sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra runloop"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_MODEL = "gpt-5.6-sol"
|
||||
DEFAULT_HTTP_PORT = 8123
|
||||
DEFAULT_AGENT_PROMPT = (
|
||||
"Inspect this Runloop sandbox workspace, verify the configuration using the shell tool, "
|
||||
"and summarize which Runloop-specific capabilities were exercised."
|
||||
)
|
||||
EXAMPLE_RESOURCE_SLUG = "runloop-capabilities-example"
|
||||
PERSISTENT_SECRET_NAME = "RUNLOOP_CAPABILITIES_EXAMPLE_TOKEN"
|
||||
PERSISTENT_SECRET_VALUE = "runloop-capabilities-example-token"
|
||||
PERSISTENT_NETWORK_POLICY_NAME = "runloop-capabilities-example-policy"
|
||||
HTTP_LOG_PATH = Path(".runloop-http.log")
|
||||
RUNTIME_CONTEXT_PATH = Path("runtime_context.json")
|
||||
AGENT_PROOF_PATH = Path("verification/agent-proof.txt")
|
||||
|
||||
|
||||
class RunloopResourceQueryResult(BaseModel):
|
||||
resource_type: Literal["secret", "network_policy"]
|
||||
name: str
|
||||
found: bool
|
||||
id: str | None = None
|
||||
description: str | None = None
|
||||
|
||||
|
||||
class RunloopResourceBootstrapResult(BaseModel):
|
||||
resource_type: Literal["secret", "network_policy"]
|
||||
name: str
|
||||
action: Literal["created", "reused", "override"]
|
||||
id: str | None = None
|
||||
found_before_bootstrap: bool
|
||||
|
||||
|
||||
def _phase(title: str) -> None:
|
||||
print(f"\n=== {title} ===", flush=True)
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
def _run_id() -> str:
|
||||
return uuid.uuid4().hex[:8]
|
||||
|
||||
|
||||
def _summarize_resource(item: object, fields: tuple[str, ...]) -> dict[str, object]:
|
||||
summary: dict[str, object] = {}
|
||||
for field in fields:
|
||||
value = getattr(item, field, None)
|
||||
if value is not None:
|
||||
summary[field] = value
|
||||
return summary
|
||||
|
||||
|
||||
async def _collect_async_items(items: Any, *, limit: int) -> list[Any]:
|
||||
collected: list[Any] = []
|
||||
async for item in items:
|
||||
collected.append(item)
|
||||
if len(collected) >= limit:
|
||||
break
|
||||
return collected
|
||||
|
||||
|
||||
def _status_code(exc: BaseException) -> int | None:
|
||||
status_code = getattr(exc, "status_code", None)
|
||||
if isinstance(status_code, int):
|
||||
return status_code
|
||||
response = getattr(exc, "response", None)
|
||||
response_status = getattr(response, "status_code", None)
|
||||
return response_status if isinstance(response_status, int) else None
|
||||
|
||||
|
||||
def _is_not_found(exc: BaseException) -> bool:
|
||||
return _status_code(exc) == 404
|
||||
|
||||
|
||||
def _error_message(exc: BaseException) -> str | None:
|
||||
message = getattr(exc, "message", None)
|
||||
if isinstance(message, str):
|
||||
return message
|
||||
body = getattr(exc, "body", None)
|
||||
if isinstance(body, dict):
|
||||
body_message = body.get("message")
|
||||
if isinstance(body_message, str):
|
||||
return body_message
|
||||
return None
|
||||
|
||||
|
||||
def _is_conflict(exc: BaseException) -> bool:
|
||||
status_code = _status_code(exc)
|
||||
if status_code == 409:
|
||||
return True
|
||||
if status_code == 400:
|
||||
message = _error_message(exc)
|
||||
return isinstance(message, str) and "already exists" in message.lower()
|
||||
return False
|
||||
|
||||
|
||||
async def _collect_maybe_async_items(items: Any, *, limit: int) -> list[Any]:
|
||||
if hasattr(items, "__aiter__"):
|
||||
return await _collect_async_items(items, limit=limit)
|
||||
return list(items)[:limit]
|
||||
|
||||
|
||||
async def _read_text(session: Any, path: Path) -> str:
|
||||
data = await session.read(path)
|
||||
try:
|
||||
payload = data.read()
|
||||
finally:
|
||||
data.close()
|
||||
if isinstance(payload, bytes):
|
||||
return payload.decode("utf-8")
|
||||
return str(payload)
|
||||
|
||||
|
||||
async def _write_json(session: Any, path: Path, payload: dict[str, object]) -> None:
|
||||
await session.write(
|
||||
path, io.BytesIO(json.dumps(payload, indent=2, sort_keys=True).encode("utf-8"))
|
||||
)
|
||||
|
||||
|
||||
def _build_manifest(*, workspace_root: str, context: dict[str, object]) -> Manifest:
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Runloop Capabilities Example\n\n"
|
||||
"This workspace is used to validate the Runloop-specific sandbox integration end "
|
||||
"to end.\n"
|
||||
),
|
||||
"checklist.md": (
|
||||
"# Checklist\n\n"
|
||||
"1. Inspect the workspace.\n"
|
||||
"2. Verify the resource discovery results in the context files.\n"
|
||||
"3. Confirm the managed secret is available without printing its full value.\n"
|
||||
"4. Confirm the HTTP preview server and verification file.\n"
|
||||
"5. Summarize what Runloop-native features were exercised and whether persistent "
|
||||
"resources were reused or created.\n"
|
||||
),
|
||||
"platform_context.json": json.dumps(context, indent=2, sort_keys=True) + "\n",
|
||||
}
|
||||
)
|
||||
return Manifest(root=workspace_root, entries=manifest.entries)
|
||||
|
||||
|
||||
def _build_sandbox_agent(
|
||||
*, model: str, manifest: Manifest, managed_secret_name: str
|
||||
) -> SandboxAgent:
|
||||
return SandboxAgent(
|
||||
name="Runloop Capabilities Guide",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Inspect the Runloop sandbox workspace carefully before answering. Use the shell tool "
|
||||
"to verify what happened in the environment and keep the final response concise. "
|
||||
"Follow this sequence:\n"
|
||||
"1. Run `pwd` and `find . -maxdepth 3 -type f | sort`.\n"
|
||||
"2. Read `README.md`, `checklist.md`, `platform_context.json`, and `runtime_context.json`.\n"
|
||||
"3. Report whether the managed secret and network policy existed before bootstrap by "
|
||||
"reading the query/bootstrap summaries from the context files.\n"
|
||||
f"4. Confirm whether `${managed_secret_name}` is set, but never print the full value. "
|
||||
"Only report whether it exists and its character length.\n"
|
||||
f"5. Read `{HTTP_LOG_PATH.as_posix()}` and confirm the HTTP server started.\n"
|
||||
f"6. Create `{AGENT_PROOF_PATH.as_posix()}` with these exact lines:\n"
|
||||
" runloop_capabilities_verified=true\n"
|
||||
" managed_secret_checked=true\n"
|
||||
" tunnel_verified=true\n"
|
||||
"7. Print that verification file from the shell.\n"
|
||||
"8. Final answer: 2 short sentences naming the specific Runloop features exercised, "
|
||||
"including whether the persistent secret and policy were reused or created.\n"
|
||||
"Only mention facts you verified from files, environment inspection, or shell output."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
|
||||
def _build_query_agent(
|
||||
*,
|
||||
model: str,
|
||||
query_secret_tool: Any,
|
||||
query_policy_tool: Any,
|
||||
managed_secret_name: str,
|
||||
network_policy_name: str,
|
||||
) -> Agent:
|
||||
return Agent(
|
||||
name="Runloop Resource Discovery Guide",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Use the provided Runloop query tools to check whether the persistent example "
|
||||
"resources already exist before any create step. Keep the final answer concise."
|
||||
),
|
||||
tools=[query_secret_tool, query_policy_tool],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
).clone(
|
||||
instructions=(
|
||||
"Use the provided Runloop query tools to check whether the persistent example "
|
||||
"resources already exist before any create step. Keep the final answer concise."
|
||||
),
|
||||
handoff_description=None,
|
||||
output_type=None,
|
||||
)
|
||||
|
||||
|
||||
def _stream_event_banner(event_name: str) -> str | None:
|
||||
if event_name == "tool_called":
|
||||
return "[tool call]"
|
||||
if event_name == "tool_output":
|
||||
return "[tool output]"
|
||||
return None
|
||||
|
||||
|
||||
def _runloop_state(session: Any) -> RunloopSandboxSessionState:
|
||||
return cast(RunloopSandboxSessionState, session.state)
|
||||
|
||||
|
||||
async def _run_plain_agent(
|
||||
*,
|
||||
agent: Agent,
|
||||
prompt: str,
|
||||
workflow_name: str,
|
||||
stream: bool,
|
||||
) -> str:
|
||||
if not stream:
|
||||
result = await Runner.run(agent, prompt, run_config=RunConfig(workflow_name=workflow_name))
|
||||
print(result.final_output)
|
||||
return str(result.final_output)
|
||||
|
||||
stream_result = Runner.run_streamed(
|
||||
agent,
|
||||
prompt,
|
||||
run_config=RunConfig(workflow_name=workflow_name),
|
||||
)
|
||||
saw_text_delta = False
|
||||
saw_any_text = False
|
||||
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
saw_any_text = True
|
||||
continue
|
||||
|
||||
if event.type != "run_item_stream_event":
|
||||
continue
|
||||
|
||||
banner = _stream_event_banner(event.name)
|
||||
if banner is None:
|
||||
continue
|
||||
if saw_text_delta:
|
||||
print()
|
||||
saw_text_delta = False
|
||||
print(f"{banner}: {tool_call_name(event.item.raw_item) or 'tool'}", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
if not saw_any_text:
|
||||
print(stream_result.final_output)
|
||||
return str(stream_result.final_output)
|
||||
|
||||
|
||||
async def _run_sandbox_agent(
|
||||
*,
|
||||
agent: SandboxAgent,
|
||||
prompt: str,
|
||||
session: Any,
|
||||
workflow_name: str,
|
||||
stream: bool,
|
||||
) -> str:
|
||||
if not stream:
|
||||
result = await Runner.run(
|
||||
agent,
|
||||
prompt,
|
||||
run_config=RunConfig(
|
||||
sandbox=SandboxRunConfig(session=session),
|
||||
workflow_name=workflow_name,
|
||||
),
|
||||
)
|
||||
print(result.final_output)
|
||||
return str(result.final_output)
|
||||
|
||||
stream_result = Runner.run_streamed(
|
||||
agent,
|
||||
prompt,
|
||||
run_config=RunConfig(
|
||||
sandbox=SandboxRunConfig(session=session),
|
||||
workflow_name=workflow_name,
|
||||
),
|
||||
)
|
||||
saw_text_delta = False
|
||||
saw_any_text = False
|
||||
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
saw_any_text = True
|
||||
continue
|
||||
|
||||
if event.type != "run_item_stream_event":
|
||||
continue
|
||||
|
||||
banner = _stream_event_banner(event.name)
|
||||
if banner is None:
|
||||
continue
|
||||
if saw_text_delta:
|
||||
print()
|
||||
saw_text_delta = False
|
||||
print(f"{banner}: {tool_call_name(event.item.raw_item) or 'tool'}", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
if not saw_any_text:
|
||||
print(stream_result.final_output)
|
||||
return str(stream_result.final_output)
|
||||
|
||||
|
||||
async def _start_http_server(session: Any, *, port: int, workspace_root: str) -> None:
|
||||
command = (
|
||||
"python -m http.server "
|
||||
f"{port} --bind 0.0.0.0 --directory {workspace_root} "
|
||||
f"> {HTTP_LOG_PATH.as_posix()} 2>&1 &"
|
||||
)
|
||||
result = await session.exec(command, shell=True, timeout=10)
|
||||
if not result.ok():
|
||||
raise RuntimeError(result.stderr.decode("utf-8", errors="replace"))
|
||||
|
||||
|
||||
def _build_endpoint_url(endpoint: Any) -> str:
|
||||
scheme = "https" if endpoint.tls else "http"
|
||||
port = endpoint.port
|
||||
host = endpoint.host
|
||||
if (scheme == "https" and port == 443) or (scheme == "http" and port == 80):
|
||||
return f"{scheme}://{host}/"
|
||||
return f"{scheme}://{host}:{port}/"
|
||||
|
||||
|
||||
async def _fetch_text(url: str, *, timeout_s: float) -> str:
|
||||
def _fetch() -> str:
|
||||
with urllib.request.urlopen(url, timeout=timeout_s) as response:
|
||||
payload = response.read()
|
||||
if isinstance(payload, bytes):
|
||||
return payload.decode("utf-8", errors="replace")
|
||||
return str(payload)
|
||||
|
||||
return await asyncio.to_thread(_fetch)
|
||||
|
||||
|
||||
async def _poll_http_preview(url: str, *, expected_substring: str, timeout_s: float) -> str:
|
||||
deadline = time.monotonic() + timeout_s
|
||||
last_error: Exception | None = None
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
body = await _fetch_text(url, timeout_s=5.0)
|
||||
if expected_substring in body:
|
||||
return body
|
||||
except (urllib.error.URLError, TimeoutError) as exc:
|
||||
last_error = exc
|
||||
await asyncio.sleep(2)
|
||||
if last_error is not None:
|
||||
raise RuntimeError(f"HTTP preview never became ready: {last_error}") from last_error
|
||||
raise RuntimeError("HTTP preview never returned the expected content.")
|
||||
|
||||
|
||||
async def _preflight_public_resources(client: RunloopSandboxClient) -> dict[str, object]:
|
||||
blueprints = await _collect_async_items(
|
||||
await client.platform.blueprints.list_public(limit=3),
|
||||
limit=3,
|
||||
)
|
||||
benchmarks = await _collect_async_items(
|
||||
await client.platform.benchmarks.list_public(limit=3),
|
||||
limit=3,
|
||||
)
|
||||
|
||||
blueprint_summaries = [
|
||||
_summarize_resource(item, ("id", "name", "status")) for item in blueprints
|
||||
]
|
||||
benchmark_summaries = [
|
||||
_summarize_resource(item, ("id", "name", "description")) for item in benchmarks
|
||||
]
|
||||
|
||||
if blueprint_summaries:
|
||||
print("public blueprints:")
|
||||
for summary in blueprint_summaries:
|
||||
print(f" - {summary}")
|
||||
else:
|
||||
print("public blueprints: none returned")
|
||||
|
||||
if benchmark_summaries:
|
||||
print("public benchmarks:")
|
||||
for summary in benchmark_summaries:
|
||||
print(f" - {summary}")
|
||||
else:
|
||||
print("public benchmarks: none returned")
|
||||
|
||||
return {
|
||||
"public_blueprints": blueprint_summaries,
|
||||
"public_benchmarks": benchmark_summaries,
|
||||
}
|
||||
|
||||
|
||||
async def _query_runloop_secret(
|
||||
client: RunloopSandboxClient,
|
||||
*,
|
||||
name: str,
|
||||
) -> RunloopResourceQueryResult:
|
||||
try:
|
||||
secret = cast(Any, await client.platform.secrets.get(name))
|
||||
except Exception as exc:
|
||||
if _is_not_found(exc):
|
||||
return RunloopResourceQueryResult(resource_type="secret", name=name, found=False)
|
||||
raise
|
||||
|
||||
return RunloopResourceQueryResult(
|
||||
resource_type="secret",
|
||||
name=name,
|
||||
found=True,
|
||||
id=cast(str | None, getattr(secret, "id", None)),
|
||||
)
|
||||
|
||||
|
||||
async def _query_runloop_network_policy(
|
||||
client: RunloopSandboxClient,
|
||||
*,
|
||||
name: str,
|
||||
) -> RunloopResourceQueryResult:
|
||||
policies = await _collect_maybe_async_items(
|
||||
await client.platform.network_policies.list(name=name, limit=10),
|
||||
limit=10,
|
||||
)
|
||||
for policy in policies:
|
||||
if getattr(policy, "name", None) != name:
|
||||
continue
|
||||
info = cast(
|
||||
Any, await client.platform.network_policies.get(cast(str, policy.id)).get_info()
|
||||
)
|
||||
return RunloopResourceQueryResult(
|
||||
resource_type="network_policy",
|
||||
name=name,
|
||||
found=True,
|
||||
id=cast(str | None, getattr(policy, "id", None)),
|
||||
description=cast(str | None, getattr(info, "description", None)),
|
||||
)
|
||||
|
||||
return RunloopResourceQueryResult(resource_type="network_policy", name=name, found=False)
|
||||
|
||||
|
||||
def _build_resource_query_tools(
|
||||
client: RunloopSandboxClient,
|
||||
*,
|
||||
managed_secret_name: str,
|
||||
network_policy_name: str,
|
||||
) -> tuple[list[Any], dict[str, RunloopResourceQueryResult]]:
|
||||
query_results: dict[str, RunloopResourceQueryResult] = {}
|
||||
|
||||
@function_tool
|
||||
async def query_runloop_secret(name: str) -> RunloopResourceQueryResult:
|
||||
"""Query whether a Runloop secret exists by name and return non-sensitive metadata."""
|
||||
|
||||
result = await _query_runloop_secret(client, name=name)
|
||||
query_results["secret"] = result
|
||||
return result
|
||||
|
||||
@function_tool
|
||||
async def query_runloop_network_policy(name: str) -> RunloopResourceQueryResult:
|
||||
"""Query whether a Runloop network policy exists by name and return basic metadata."""
|
||||
|
||||
result = await _query_runloop_network_policy(client, name=name)
|
||||
query_results["network_policy"] = result
|
||||
return result
|
||||
|
||||
tools = [query_runloop_secret, query_runloop_network_policy]
|
||||
_ = (managed_secret_name, network_policy_name)
|
||||
return tools, query_results
|
||||
|
||||
|
||||
async def _run_resource_query_phase(
|
||||
client: RunloopSandboxClient,
|
||||
*,
|
||||
model: str,
|
||||
stream: bool,
|
||||
managed_secret_name: str,
|
||||
network_policy_name: str,
|
||||
) -> tuple[dict[str, RunloopResourceQueryResult], str]:
|
||||
tools, query_results = _build_resource_query_tools(
|
||||
client,
|
||||
managed_secret_name=managed_secret_name,
|
||||
network_policy_name=network_policy_name,
|
||||
)
|
||||
query_agent = Agent(
|
||||
name="Runloop Resource Discovery Guide",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Use both query tools before answering. You are checking whether the persistent "
|
||||
"Runloop example resources already exist before any create step.\n\n"
|
||||
f"1. Call `query_runloop_secret` with `{managed_secret_name}`.\n"
|
||||
f"2. Call `query_runloop_network_policy` with `{network_policy_name}`.\n"
|
||||
"3. Final answer in 2 short sentences stating whether each resource already exists."
|
||||
),
|
||||
tools=tools,
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
prompt = (
|
||||
"Check whether the persistent Runloop secret and network policy for this example already "
|
||||
"exist before the script attempts any create or reuse step."
|
||||
)
|
||||
output = await _run_plain_agent(
|
||||
agent=query_agent,
|
||||
prompt=prompt,
|
||||
workflow_name="Runloop resource query example",
|
||||
stream=stream,
|
||||
)
|
||||
if "secret" not in query_results or "network_policy" not in query_results:
|
||||
raise RuntimeError("The query agent did not call both Runloop resource query tools.")
|
||||
return query_results, output
|
||||
|
||||
|
||||
async def _bootstrap_persistent_resources(
|
||||
client: RunloopSandboxClient,
|
||||
*,
|
||||
managed_secret_name: str,
|
||||
managed_secret_value: str,
|
||||
network_policy_name: str,
|
||||
network_policy_id_override: str | None,
|
||||
query_results: dict[str, RunloopResourceQueryResult],
|
||||
axon_name: str | None,
|
||||
) -> dict[str, object]:
|
||||
secret_query = query_results["secret"]
|
||||
policy_query = query_results["network_policy"]
|
||||
|
||||
bootstrap: dict[str, object] = {
|
||||
"managed_secret_value": managed_secret_value,
|
||||
"secret": RunloopResourceBootstrapResult(
|
||||
resource_type="secret",
|
||||
name=managed_secret_name,
|
||||
action="reused" if secret_query.found else "created",
|
||||
id=secret_query.id,
|
||||
found_before_bootstrap=secret_query.found,
|
||||
),
|
||||
"network_policy": RunloopResourceBootstrapResult(
|
||||
resource_type="network_policy",
|
||||
name=network_policy_name,
|
||||
action="override"
|
||||
if network_policy_id_override
|
||||
else ("reused" if policy_query.found else "created"),
|
||||
id=network_policy_id_override or policy_query.id,
|
||||
found_before_bootstrap=policy_query.found,
|
||||
),
|
||||
"axon_id": None,
|
||||
"axon_name": axon_name,
|
||||
}
|
||||
|
||||
secret_result = cast(RunloopResourceBootstrapResult, bootstrap["secret"])
|
||||
if not secret_query.found:
|
||||
created_secret = cast(
|
||||
Any,
|
||||
await client.platform.secrets.create(
|
||||
name=managed_secret_name, value=managed_secret_value
|
||||
),
|
||||
)
|
||||
secret_result.id = cast(str | None, getattr(created_secret, "id", None))
|
||||
print(
|
||||
"persistent secret bootstrap:",
|
||||
secret_result.model_dump(mode="json"),
|
||||
)
|
||||
|
||||
policy_result = cast(RunloopResourceBootstrapResult, bootstrap["network_policy"])
|
||||
if network_policy_id_override is None and not policy_query.found:
|
||||
try:
|
||||
created_policy = cast(
|
||||
Any,
|
||||
await client.platform.network_policies.create(
|
||||
name=network_policy_name,
|
||||
allow_all=True,
|
||||
description="Persistent network policy for the Runloop capabilities example.",
|
||||
),
|
||||
)
|
||||
except Exception as exc:
|
||||
if not _is_conflict(exc):
|
||||
raise
|
||||
policy_result.action = "reused"
|
||||
policy_result.found_before_bootstrap = True
|
||||
refreshed_policy = await _query_runloop_network_policy(client, name=network_policy_name)
|
||||
policy_result.id = refreshed_policy.id
|
||||
else:
|
||||
policy_result.id = cast(str | None, getattr(created_policy, "id", None))
|
||||
print(
|
||||
"persistent network policy bootstrap:",
|
||||
policy_result.model_dump(mode="json"),
|
||||
)
|
||||
|
||||
if axon_name is not None:
|
||||
axon = cast(Any, await client.platform.axons.create(name=axon_name))
|
||||
await client.platform.axons.query_sql(
|
||||
cast(str, axon.id),
|
||||
sql="CREATE TABLE IF NOT EXISTS events (id INTEGER PRIMARY KEY AUTOINCREMENT, kind TEXT NOT NULL)",
|
||||
)
|
||||
await client.platform.axons.batch_sql(
|
||||
cast(str, axon.id),
|
||||
statements=[
|
||||
{"sql": "INSERT INTO events (kind) VALUES (?)", "params": ["capabilities"]},
|
||||
{"sql": "INSERT INTO events (kind) VALUES (?)", "params": ["agent_guided"]},
|
||||
],
|
||||
)
|
||||
query_result = cast(
|
||||
Any,
|
||||
await client.platform.axons.query_sql(
|
||||
cast(str, axon.id),
|
||||
sql="SELECT COUNT(*) AS total_events FROM events",
|
||||
),
|
||||
)
|
||||
publish_result = cast(
|
||||
Any,
|
||||
await client.platform.axons.publish(
|
||||
cast(str, axon.id),
|
||||
event_type="capabilities_example",
|
||||
origin="AGENT_EVENT",
|
||||
payload=json.dumps({"axon_name": axon_name}),
|
||||
source="openai-agents-python",
|
||||
),
|
||||
)
|
||||
bootstrap["axon_id"] = cast(str, axon.id)
|
||||
print(
|
||||
"axon demo created:",
|
||||
{
|
||||
"id": cast(str, axon.id),
|
||||
"name": axon_name,
|
||||
"rows": query_result.rows,
|
||||
"published": getattr(publish_result, "published", None),
|
||||
},
|
||||
)
|
||||
|
||||
return bootstrap
|
||||
|
||||
|
||||
def _optional_gateways(args: argparse.Namespace) -> dict[str, RunloopGatewaySpec]:
|
||||
if not (args.gateway_env_var and args.gateway_name and args.gateway_secret_name):
|
||||
return {}
|
||||
return {
|
||||
args.gateway_env_var: RunloopGatewaySpec(
|
||||
gateway=args.gateway_name,
|
||||
secret=args.gateway_secret_name,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
def _optional_mcp(args: argparse.Namespace) -> dict[str, RunloopMcpSpec]:
|
||||
if not (args.mcp_env_var and args.mcp_config and args.mcp_secret_name):
|
||||
return {}
|
||||
return {
|
||||
args.mcp_env_var: RunloopMcpSpec(
|
||||
mcp_config=args.mcp_config,
|
||||
secret=args.mcp_secret_name,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
async def main(args: argparse.Namespace) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
_require_env("RUNLOOP_API_KEY")
|
||||
|
||||
workspace_root = (
|
||||
DEFAULT_RUNLOOP_ROOT_WORKSPACE_ROOT if args.root else DEFAULT_RUNLOOP_WORKSPACE_ROOT
|
||||
)
|
||||
run_id = _run_id()
|
||||
metadata = {
|
||||
"example": "runloop-capabilities",
|
||||
"run_id": run_id,
|
||||
}
|
||||
|
||||
client = RunloopSandboxClient()
|
||||
session = None
|
||||
resumed = None
|
||||
session_closed = False
|
||||
resumed_closed = False
|
||||
|
||||
try:
|
||||
_phase("Public Resource Discovery")
|
||||
public_context = await _preflight_public_resources(client)
|
||||
|
||||
_phase("Agent Resource Discovery")
|
||||
query_results, query_agent_output = await _run_resource_query_phase(
|
||||
client,
|
||||
model=args.model,
|
||||
stream=args.stream,
|
||||
managed_secret_name=PERSISTENT_SECRET_NAME,
|
||||
network_policy_name=PERSISTENT_NETWORK_POLICY_NAME,
|
||||
)
|
||||
print(
|
||||
"resource query results:",
|
||||
{key: value.model_dump(mode="json") for key, value in query_results.items()},
|
||||
)
|
||||
|
||||
_phase("Persistent Resource Bootstrap")
|
||||
axon_name = f"{EXAMPLE_RESOURCE_SLUG}-axon-{run_id}" if args.with_axon_demo else None
|
||||
bootstrap = await _bootstrap_persistent_resources(
|
||||
client,
|
||||
managed_secret_name=PERSISTENT_SECRET_NAME,
|
||||
managed_secret_value=PERSISTENT_SECRET_VALUE,
|
||||
network_policy_name=PERSISTENT_NETWORK_POLICY_NAME,
|
||||
network_policy_id_override=args.network_policy_id,
|
||||
query_results=query_results,
|
||||
axon_name=axon_name,
|
||||
)
|
||||
secret_bootstrap = cast(RunloopResourceBootstrapResult, bootstrap["secret"])
|
||||
network_policy_bootstrap = cast(RunloopResourceBootstrapResult, bootstrap["network_policy"])
|
||||
network_policy_id = network_policy_bootstrap.id
|
||||
|
||||
context = {
|
||||
"example_slug": EXAMPLE_RESOURCE_SLUG,
|
||||
"workspace_root": workspace_root,
|
||||
"requested_blueprint_name": args.blueprint_name,
|
||||
"public_resources": public_context,
|
||||
"resource_query_agent_output": query_agent_output,
|
||||
"resource_queries": {
|
||||
key: value.model_dump(mode="json") for key, value in query_results.items()
|
||||
},
|
||||
"resource_bootstrap": {
|
||||
"secret": secret_bootstrap.model_dump(mode="json"),
|
||||
"network_policy": network_policy_bootstrap.model_dump(mode="json"),
|
||||
"axon_id": bootstrap["axon_id"],
|
||||
"axon_name": bootstrap["axon_name"],
|
||||
},
|
||||
"managed_secret_env_var": PERSISTENT_SECRET_NAME,
|
||||
"network_policy_id": network_policy_id,
|
||||
"metadata": metadata,
|
||||
"gateway_bindings": sorted(_optional_gateways(args)),
|
||||
"mcp_bindings": sorted(_optional_mcp(args)),
|
||||
}
|
||||
|
||||
manifest = _build_manifest(workspace_root=workspace_root, context=context)
|
||||
agent = _build_sandbox_agent(
|
||||
model=args.model,
|
||||
manifest=manifest,
|
||||
managed_secret_name=PERSISTENT_SECRET_NAME,
|
||||
)
|
||||
options = RunloopSandboxClientOptions(
|
||||
blueprint_name=args.blueprint_name,
|
||||
pause_on_exit=True,
|
||||
exposed_ports=(args.http_port,),
|
||||
user_parameters=(RunloopUserParameters(username="root", uid=0) if args.root else None),
|
||||
launch_parameters=RunloopLaunchParameters(
|
||||
network_policy_id=network_policy_id,
|
||||
resource_size_request=args.resource_size,
|
||||
after_idle=RunloopAfterIdle(idle_time_seconds=300, on_idle="suspend"),
|
||||
launch_commands=["echo runloop-capabilities-example"],
|
||||
),
|
||||
tunnel=RunloopTunnelConfig(
|
||||
auth_mode="open",
|
||||
http_keep_alive=True,
|
||||
wake_on_http=True,
|
||||
),
|
||||
gateways=_optional_gateways(args),
|
||||
mcp=_optional_mcp(args),
|
||||
metadata=metadata,
|
||||
managed_secrets={PERSISTENT_SECRET_NAME: PERSISTENT_SECRET_VALUE},
|
||||
)
|
||||
|
||||
_phase("Sandbox Create")
|
||||
session = await client.create(manifest=manifest, options=options)
|
||||
await session.start()
|
||||
session_state = _runloop_state(session)
|
||||
print(
|
||||
"session started:",
|
||||
{
|
||||
"devbox_id": session_state.devbox_id,
|
||||
"secret_refs": session_state.secret_refs,
|
||||
"metadata": session_state.metadata,
|
||||
},
|
||||
)
|
||||
|
||||
_phase("Tunnel Check")
|
||||
await _write_json(
|
||||
session,
|
||||
RUNTIME_CONTEXT_PATH,
|
||||
{
|
||||
**context,
|
||||
"devbox_id": session_state.devbox_id,
|
||||
"secret_refs": session_state.secret_refs,
|
||||
"runtime_phase": "before_tunnel_check",
|
||||
},
|
||||
)
|
||||
await _start_http_server(session, port=args.http_port, workspace_root=workspace_root)
|
||||
endpoint = await session.resolve_exposed_port(args.http_port)
|
||||
preview_url = urljoin(_build_endpoint_url(endpoint), "README.md")
|
||||
preview_body = await _poll_http_preview(
|
||||
preview_url,
|
||||
expected_substring="Runloop Capabilities Example",
|
||||
timeout_s=45.0,
|
||||
)
|
||||
print("resolved tunnel:", preview_url)
|
||||
await _write_json(
|
||||
session,
|
||||
RUNTIME_CONTEXT_PATH,
|
||||
{
|
||||
**context,
|
||||
"devbox_id": session_state.devbox_id,
|
||||
"secret_refs": session_state.secret_refs,
|
||||
"tunnel_url": preview_url,
|
||||
"http_preview_contains_readme": "Runloop Capabilities Example" in preview_body,
|
||||
"runtime_phase": "before_agent_run",
|
||||
},
|
||||
)
|
||||
|
||||
_phase("Agent Verification")
|
||||
await _run_sandbox_agent(
|
||||
agent=agent,
|
||||
prompt=args.prompt,
|
||||
session=session,
|
||||
workflow_name="Runloop capabilities example",
|
||||
stream=args.stream,
|
||||
)
|
||||
proof_text = await _read_text(session, AGENT_PROOF_PATH)
|
||||
print("agent proof:")
|
||||
print(proof_text.rstrip())
|
||||
|
||||
_phase("Suspend")
|
||||
await session.aclose()
|
||||
session_closed = True
|
||||
print("session persisted and suspended")
|
||||
|
||||
_phase("Resume Check")
|
||||
resumed = await client.resume(session.state)
|
||||
await resumed.start()
|
||||
resumed_state = _runloop_state(resumed)
|
||||
resumed_runtime_context = await _read_text(resumed, RUNTIME_CONTEXT_PATH)
|
||||
resumed_proof_text = await _read_text(resumed, AGENT_PROOF_PATH)
|
||||
print("resumed runtime context bytes:", len(resumed_runtime_context.encode("utf-8")))
|
||||
print("resumed proof:")
|
||||
print(resumed_proof_text.rstrip())
|
||||
resumed_state.pause_on_exit = False
|
||||
await resumed.aclose()
|
||||
resumed_closed = True
|
||||
print("resumed session cleaned up with delete semantics")
|
||||
|
||||
_phase("Persistent Resource Summary")
|
||||
print(
|
||||
"persistent resources retained:",
|
||||
{
|
||||
"secret": secret_bootstrap.model_dump(mode="json"),
|
||||
"network_policy": network_policy_bootstrap.model_dump(mode="json"),
|
||||
},
|
||||
)
|
||||
if bootstrap["axon_id"] is not None:
|
||||
print(
|
||||
"axon retained for manual cleanup:",
|
||||
{
|
||||
"axon_id": bootstrap["axon_id"],
|
||||
"axon_name": bootstrap["axon_name"],
|
||||
},
|
||||
)
|
||||
finally:
|
||||
if resumed is not None and not resumed_closed:
|
||||
try:
|
||||
_runloop_state(resumed).pause_on_exit = False
|
||||
await resumed.aclose()
|
||||
except Exception as exc:
|
||||
print(f"warning: failed to close resumed session cleanly: {exc}")
|
||||
elif session is not None and not session_closed:
|
||||
try:
|
||||
_runloop_state(session).pause_on_exit = False
|
||||
await session.aclose()
|
||||
except Exception as exc:
|
||||
print(f"warning: failed to close initial session cleanly: {exc}")
|
||||
elif session is not None and session_closed and resumed is None:
|
||||
try:
|
||||
cleanup_session = await client.resume(session.state)
|
||||
_runloop_state(cleanup_session).pause_on_exit = False
|
||||
await cleanup_session.aclose()
|
||||
except Exception as exc:
|
||||
print(f"warning: failed to resume suspended session for cleanup: {exc}")
|
||||
|
||||
await client.close()
|
||||
|
||||
|
||||
def _build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", default=DEFAULT_MODEL, help="Model name to use.")
|
||||
parser.add_argument(
|
||||
"--prompt", default=DEFAULT_AGENT_PROMPT, help="Prompt to send to the agent."
|
||||
)
|
||||
parser.add_argument("--blueprint-name", default=None, help="Optional Runloop blueprint name.")
|
||||
parser.add_argument(
|
||||
"--resource-size",
|
||||
default="MEDIUM",
|
||||
choices=["X_SMALL", "SMALL", "MEDIUM", "LARGE", "X_LARGE", "XX_LARGE", "CUSTOM_SIZE"],
|
||||
help="Runloop resource size request for the devbox.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--network-policy-id",
|
||||
default=None,
|
||||
help="Optional Runloop network policy id override. Without this flag, the example reuses or creates the persistent example policy by name.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--http-port",
|
||||
type=int,
|
||||
default=DEFAULT_HTTP_PORT,
|
||||
help="Port used by the preview HTTP server.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--root",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Launch the Runloop devbox as root. The workspace root becomes /root.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--stream",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Stream the agent response and tool activity.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--with-axon-demo",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Also create and use a temporary Axon. This leaves the Axon behind for manual cleanup.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--gateway-env-var", default=None, help="Env var name for a gateway binding."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--gateway-name", default=None, help="Runloop gateway name for the binding."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--gateway-secret-name",
|
||||
default=None,
|
||||
help="Runloop secret name used by the gateway binding.",
|
||||
)
|
||||
parser.add_argument("--mcp-env-var", default=None, help="Env var name for an MCP binding.")
|
||||
parser.add_argument(
|
||||
"--mcp-config", default=None, help="Runloop MCP config name for the binding."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--mcp-secret-name",
|
||||
default=None,
|
||||
help="Runloop secret name used by the MCP binding.",
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main(_build_parser().parse_args()))
|
||||
@@ -0,0 +1,170 @@
|
||||
"""
|
||||
Minimal Runloop-backed sandbox example for manual validation.
|
||||
|
||||
This mirrors the other cloud extension examples: it creates a tiny workspace, asks a sandboxed
|
||||
agent to inspect it through one shell tool, and prints a short answer.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[4]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import (
|
||||
DEFAULT_RUNLOOP_ROOT_WORKSPACE_ROOT,
|
||||
DEFAULT_RUNLOOP_WORKSPACE_ROOT,
|
||||
RunloopSandboxClient,
|
||||
RunloopSandboxClientOptions,
|
||||
RunloopUserParameters,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"Runloop sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra runloop"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
|
||||
|
||||
def _build_manifest(*, workspace_root: str) -> Manifest:
|
||||
manifest = text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Runloop Demo Workspace\n\n"
|
||||
"This workspace exists to validate the Runloop sandbox backend manually.\n"
|
||||
),
|
||||
"launch.md": (
|
||||
"# Launch\n\n"
|
||||
"- Customer: Contoso Logistics.\n"
|
||||
"- Goal: validate the remote sandbox agent path.\n"
|
||||
"- Current status: Runloop backend smoke and app-server connectivity are passing.\n"
|
||||
),
|
||||
"tasks.md": (
|
||||
"# Tasks\n\n"
|
||||
"1. Inspect the workspace files.\n"
|
||||
"2. Summarize the setup and any notable status in two sentences.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
return Manifest(root=workspace_root, entries=manifest.entries)
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
pause_on_exit: bool,
|
||||
blueprint_name: str | None,
|
||||
root: bool,
|
||||
stream: bool,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
_require_env("RUNLOOP_API_KEY")
|
||||
|
||||
workspace_root = DEFAULT_RUNLOOP_ROOT_WORKSPACE_ROOT if root else DEFAULT_RUNLOOP_WORKSPACE_ROOT
|
||||
manifest = _build_manifest(workspace_root=workspace_root)
|
||||
agent = SandboxAgent(
|
||||
name="Runloop Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
client = RunloopSandboxClient()
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=client,
|
||||
options=RunloopSandboxClientOptions(
|
||||
blueprint_name=blueprint_name,
|
||||
pause_on_exit=pause_on_exit,
|
||||
user_parameters=(RunloopUserParameters(username="root", uid=0) if root else None),
|
||||
),
|
||||
),
|
||||
workflow_name="Runloop sandbox example",
|
||||
)
|
||||
|
||||
try:
|
||||
if not stream:
|
||||
result = await Runner.run(agent, question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
stream_result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
finally:
|
||||
await client.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", default="gpt-5.6-sol", help="Model name to use.")
|
||||
parser.add_argument("--question", default=DEFAULT_QUESTION, help="Prompt to send to the agent.")
|
||||
parser.add_argument(
|
||||
"--pause-on-exit",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Suspend the Runloop devbox on shutdown instead of deleting it.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--blueprint-name",
|
||||
default=None,
|
||||
help="Optional Runloop blueprint name to use when creating the devbox.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--root",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Launch the Runloop devbox as root. The default home/workspace root becomes /root.",
|
||||
)
|
||||
parser.add_argument("--stream", action="store_true", default=False, help="Stream the response.")
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
pause_on_exit=args.pause_on_exit,
|
||||
blueprint_name=args.blueprint_name,
|
||||
root=args.root,
|
||||
stream=args.stream,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,86 @@
|
||||
# Temporal Sandbox Agent
|
||||
|
||||
A conversational coding agent that runs as a durable Temporal workflow with support for multiple sandbox backends (Daytona, Docker, E2B, local unix).
|
||||
|
||||
## Quickstart
|
||||
|
||||
**Prerequisites:** Docker (for the Docker backend) and API keys for any cloud backends you want to use. The local and Docker sandboxes work without any cloud provider API keys.
|
||||
|
||||
## Local smoke test
|
||||
|
||||
If you only want to confirm that Temporal workflows run locally, use the minimal
|
||||
example first:
|
||||
|
||||
```
|
||||
export OPENAI_API_KEY="sk-..."
|
||||
# Optional: export EXAMPLES_TEMPORAL_MODEL="gpt-5.4-mini"
|
||||
# Optional: export EXAMPLES_TEMPORAL_TRACE="openai"
|
||||
uv run --extra temporal python -m examples.sandbox.extensions.temporal.local_hello_workflow
|
||||
```
|
||||
|
||||
This starts the Temporal Python SDK test server, runs one workflow and one model activity, connects the workflow to a local Unix sandbox, and then shuts down. It does not require the Temporal CLI, an already running Temporal dev server, or sandbox backend credentials.
|
||||
|
||||
The local smoke test enables OpenAI Agents tracing by default. Set `EXAMPLES_TEMPORAL_TRACE=none` to disable tracing, or `EXAMPLES_TEMPORAL_TRACE=openai_with_temporal_spans` to also ask the Temporal plugin to add Temporal spans. The Temporal span mode depends on Temporal plugin behavior and may omit regular Agents spans with some plugin versions; use the default `openai` mode when you want standard OpenAI trace spans.
|
||||
|
||||
1. Install [just](https://just.systems/man/en/packages.html) and the [Temporal CLI](https://docs.temporal.io/cli/setup-cli#install-the-cli) if you don't have them already.
|
||||
|
||||
2. Change into the example directory:
|
||||
|
||||
```
|
||||
cd examples/sandbox/extensions/temporal
|
||||
```
|
||||
|
||||
3. Create a `.env` file in this directory with your API keys:
|
||||
|
||||
```
|
||||
OPENAI_API_KEY="sk-..."
|
||||
DAYTONA_API_KEY="dtn_..." # optional, for Daytona backend
|
||||
E2B_API_KEY="e2b_..." # optional, for E2B backend
|
||||
```
|
||||
|
||||
4. Start the Temporal dev server:
|
||||
|
||||
```
|
||||
just temporal
|
||||
```
|
||||
|
||||
5. In a second terminal, start the worker:
|
||||
|
||||
```
|
||||
just worker
|
||||
```
|
||||
|
||||
6. In a third terminal, start the TUI:
|
||||
|
||||
```
|
||||
just tui
|
||||
```
|
||||
|
||||
The `just worker` and `just tui` commands automatically install dependencies before starting.
|
||||
|
||||
## TUI commands
|
||||
|
||||
| Command | Description |
|
||||
|--------------------|--------------------------------------------------------|
|
||||
| `/switch` | Switch the current session to a different sandbox backend |
|
||||
| `/fork [title]` | Fork the session onto a (possibly different) backend |
|
||||
| `/title <name>` | Rename the current session |
|
||||
| `/done` | Exit the TUI |
|
||||
|
||||
Both `/switch` and `/fork` open an interactive backend picker. When switching to the local backend you can specify the workspace root directory.
|
||||
|
||||
## How it works
|
||||
|
||||
A single Temporal worker registers all sandbox backends via `SandboxClientProvider`, so every backend's activities are available on one task queue. The workflow picks which backend to target each turn by calling `temporal_sandbox_client(name)` in its `RunConfig`.
|
||||
|
||||
**Files:**
|
||||
|
||||
- `temporal_sandbox_agent.py` -- The `AgentWorkflow` definition and worker entrypoint. Each conversation turn calls `Runner.run()` with a `SandboxRunConfig` that targets the active backend. The workflow is
|
||||
long-lived: it idles between turns and persists indefinitely in Temporal.
|
||||
- `temporal_session_manager.py` -- A singleton `SessionManagerWorkflow` that tracks active sessions and handles create, fork, switch, and destroy operations.
|
||||
- `temporal_sandbox_tui.py` -- A [Textual](https://textual.textualize.io/) TUI that connects to the session manager and drives conversations via signals, updates, and queries.
|
||||
- `examples/sandbox/misc/workspace_shell.py` -- A shared `Capability` that gives the agent a shell tool for running commands in the sandbox workspace.
|
||||
|
||||
**Switching backends** is an in-place operation: the workflow receives a `switch_backend` update, changes its backend and manifest, clears the backend-specific session state, and the next turn creates a fresh session on the new backend. The portable snapshot is preserved so workspace files carry over.
|
||||
|
||||
**Forking** pauses the source workflow, snapshots its state and conversation history, and starts a new child workflow on the chosen backend. The fork gets an independent copy of the workspace and conversation.
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Worker startup diagnostics."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
YELLOW = "\033[1;33m"
|
||||
RESET = "\033[0m"
|
||||
|
||||
|
||||
def print_backend_warnings(registered_names: set[str]) -> None:
|
||||
"""Print a prominent warning banner for any unconfigured sandbox backends."""
|
||||
import docker # type: ignore[import-untyped]
|
||||
|
||||
backend_env = {
|
||||
"daytona": "DAYTONA_API_KEY",
|
||||
"e2b": "E2B_API_KEY",
|
||||
}
|
||||
missing = {name: var for name, var in backend_env.items() if name not in registered_names}
|
||||
try:
|
||||
docker.from_env().ping()
|
||||
except Exception:
|
||||
missing["docker"] = "Docker daemon"
|
||||
|
||||
if not missing:
|
||||
return
|
||||
|
||||
lines = [
|
||||
"WARNING: Some sandbox backends are NOT available.",
|
||||
"Missing:",
|
||||
]
|
||||
for name, var in sorted(missing.items()):
|
||||
lines.append(f" - {name} ({var})")
|
||||
lines.append("The TUI will fail if you select an unconfigured backend.")
|
||||
lines.append("To use them, set the missing env vars and restart the worker.")
|
||||
width = max(len(line) for line in lines) + 4
|
||||
border = "!" * (width + 2)
|
||||
print(f"{YELLOW}{border}{RESET}")
|
||||
for line in lines:
|
||||
print(f"{YELLOW}! {line:<{width - 2}} !{RESET}")
|
||||
print(f"{YELLOW}{border}{RESET}")
|
||||
@@ -0,0 +1,21 @@
|
||||
# Temporal Sandbox Agent
|
||||
|
||||
set dotenv-load
|
||||
set dotenv-path := ".env"
|
||||
|
||||
# Ensure extras are installed
|
||||
[private]
|
||||
sync:
|
||||
@uv sync --extra temporal --extra daytona --extra e2b --extra docker 2>&1 | grep -v "^Audited\|^Resolved" || true
|
||||
|
||||
# Start the local Temporal dev server
|
||||
temporal:
|
||||
temporal server start-dev
|
||||
|
||||
# Start the Temporal worker
|
||||
worker: sync
|
||||
uv run --extra temporal --extra daytona --extra e2b --extra docker python temporal_sandbox_agent.py worker
|
||||
|
||||
# Start the TUI client
|
||||
tui: sync
|
||||
uv run --extra temporal --extra daytona --extra e2b --extra docker python temporal_sandbox_agent.py run
|
||||
@@ -0,0 +1,150 @@
|
||||
"""Minimal local Temporal SandboxAgent workflow example.
|
||||
|
||||
This example is intentionally smaller than ``temporal_sandbox_agent.py``. It starts a local
|
||||
Temporal test server through the Temporal Python SDK, runs a ``SandboxAgent`` workflow against
|
||||
the local Unix sandbox backend, and then shuts everything down.
|
||||
|
||||
It does not require the Temporal CLI, a long-running Temporal server, or cloud sandbox backend
|
||||
credentials. It does require ``OPENAI_API_KEY`` because the model call runs through the Temporal
|
||||
OpenAI Agents plugin as an activity.
|
||||
|
||||
Usage:
|
||||
uv run --extra temporal python -m examples.sandbox.extensions.temporal.local_hello_workflow
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from datetime import timedelta
|
||||
|
||||
from temporalio import workflow
|
||||
from temporalio.client import Client
|
||||
from temporalio.contrib.openai_agents import (
|
||||
ModelActivityParameters,
|
||||
OpenAIAgentsPlugin,
|
||||
SandboxClientProvider,
|
||||
)
|
||||
from temporalio.contrib.openai_agents.workflow import temporal_sandbox_client
|
||||
from temporalio.testing import WorkflowEnvironment
|
||||
from temporalio.worker import Worker
|
||||
from temporalio.worker.workflow_sandbox import SandboxedWorkflowRunner, SandboxRestrictions
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.capabilities import Shell
|
||||
from agents.sandbox.entries import File
|
||||
from agents.sandbox.sandboxes import UnixLocalSandboxClient, UnixLocalSandboxClientOptions
|
||||
|
||||
TASK_QUEUE = "local-temporal-sandbox-agent"
|
||||
WORKFLOW_ID = "local-temporal-sandbox-agent-workflow"
|
||||
DEFAULT_MODEL = "gpt-5.4-mini"
|
||||
EXPECTED_GREETING = "Temporal sandbox says hello from a local file"
|
||||
TRACE_MODE_NONE = "none"
|
||||
TRACE_MODE_OPENAI = "openai"
|
||||
TRACE_MODE_OPENAI_WITH_TEMPORAL_SPANS = "openai_with_temporal_spans"
|
||||
TRACE_MODES = {
|
||||
TRACE_MODE_NONE,
|
||||
TRACE_MODE_OPENAI,
|
||||
TRACE_MODE_OPENAI_WITH_TEMPORAL_SPANS,
|
||||
}
|
||||
|
||||
|
||||
@workflow.defn
|
||||
class LocalSandboxAgentWorkflow:
|
||||
@workflow.run
|
||||
async def run(self, model: str, trace_mode: str) -> str:
|
||||
agent = SandboxAgent(
|
||||
name="Local Temporal Sandbox Agent",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Inspect the sandbox workspace with the shell tool before answering. "
|
||||
"Report the greeting from README.md exactly."
|
||||
),
|
||||
default_manifest=Manifest(
|
||||
entries={
|
||||
"README.md": File(content=b"Temporal sandbox says hello from a local file.\n"),
|
||||
}
|
||||
),
|
||||
capabilities=[Shell()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
result = await Runner.run(
|
||||
agent,
|
||||
"Read README.md and report its greeting.",
|
||||
run_config=RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=temporal_sandbox_client("local"),
|
||||
options=UnixLocalSandboxClientOptions(),
|
||||
),
|
||||
workflow_name="Local Temporal SandboxAgent workflow",
|
||||
tracing_disabled=trace_mode == TRACE_MODE_NONE,
|
||||
),
|
||||
)
|
||||
return str(result.final_output)
|
||||
|
||||
|
||||
def _client_with_plugin(client: Client, trace_mode: str) -> Client:
|
||||
plugin = OpenAIAgentsPlugin(
|
||||
model_params=ModelActivityParameters(start_to_close_timeout=timedelta(seconds=120)),
|
||||
sandbox_clients=[SandboxClientProvider("local", UnixLocalSandboxClient())],
|
||||
add_temporal_spans=trace_mode == TRACE_MODE_OPENAI_WITH_TEMPORAL_SPANS,
|
||||
)
|
||||
config = client.config()
|
||||
config["plugins"] = [*config.get("plugins", []), plugin]
|
||||
return Client(**config)
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if not os.environ.get(name):
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
def _trace_mode_from_env() -> str:
|
||||
trace_mode = os.getenv("EXAMPLES_TEMPORAL_TRACE", TRACE_MODE_OPENAI).strip().lower()
|
||||
if trace_mode not in TRACE_MODES:
|
||||
supported = ", ".join(sorted(TRACE_MODES))
|
||||
raise SystemExit(
|
||||
f"EXAMPLES_TEMPORAL_TRACE must be one of: {supported}. Got {trace_mode!r}."
|
||||
)
|
||||
return trace_mode
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
model = os.getenv("EXAMPLES_TEMPORAL_MODEL", DEFAULT_MODEL)
|
||||
trace_mode = _trace_mode_from_env()
|
||||
print(f"Using model: {model}")
|
||||
print(f"Using trace mode: {trace_mode}")
|
||||
print("Starting local Temporal test server...")
|
||||
async with await WorkflowEnvironment.start_time_skipping() as env:
|
||||
client = _client_with_plugin(env.client, trace_mode)
|
||||
print("Starting local Temporal worker...")
|
||||
async with Worker(
|
||||
client,
|
||||
task_queue=TASK_QUEUE,
|
||||
workflows=[LocalSandboxAgentWorkflow],
|
||||
workflow_runner=SandboxedWorkflowRunner(
|
||||
restrictions=SandboxRestrictions.default.with_passthrough_modules(
|
||||
"annotated_types",
|
||||
"pydantic_core",
|
||||
),
|
||||
),
|
||||
):
|
||||
result = await client.execute_workflow(
|
||||
LocalSandboxAgentWorkflow.run,
|
||||
args=[model, trace_mode],
|
||||
id=WORKFLOW_ID,
|
||||
task_queue=TASK_QUEUE,
|
||||
)
|
||||
|
||||
print(f"Workflow result: {result}")
|
||||
if EXPECTED_GREETING not in result:
|
||||
raise RuntimeError(f"Expected workflow result to contain {EXPECTED_GREETING!r}.")
|
||||
print("Local Temporal SandboxAgent workflow completed successfully.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,722 @@
|
||||
"""Temporal Sandbox agent example.
|
||||
|
||||
Runs a SandboxAgent as a durable Temporal workflow. The workflow is long-lived
|
||||
and conversational: after processing each turn it idles waiting for the next
|
||||
user message. Workflows persist indefinitely in Temporal. A separate session
|
||||
manager workflow (``temporal_session_manager.py``) orchestrates session
|
||||
creation, destruction, and discovery.
|
||||
|
||||
Usage
|
||||
-----
|
||||
Install the Temporal extra first::
|
||||
|
||||
uv sync --extra temporal --extra daytona
|
||||
|
||||
Start a local Temporal server (requires the Temporal CLI)::
|
||||
|
||||
temporal server start-dev
|
||||
|
||||
In one terminal, start the worker::
|
||||
|
||||
python examples/sandbox/extensions/temporal_sandbox_agent.py worker
|
||||
|
||||
In another terminal, start the TUI::
|
||||
|
||||
python examples/sandbox/extensions/temporal_sandbox_agent.py run
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import os as _os
|
||||
import sys
|
||||
from datetime import timedelta
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal, cast
|
||||
|
||||
from pydantic import BaseModel, SerializeAsAny, field_validator, model_serializer
|
||||
from temporalio import workflow
|
||||
from temporalio.client import Client
|
||||
from temporalio.contrib.openai_agents.workflow import temporal_sandbox_client
|
||||
from temporalio.worker import Worker
|
||||
from temporalio.worker.workflow_sandbox import (
|
||||
SandboxedWorkflowRunner,
|
||||
SandboxRestrictions,
|
||||
)
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.agent import Agent
|
||||
from agents.extensions.sandbox import (
|
||||
DaytonaSandboxClientOptions,
|
||||
DaytonaSandboxSessionState,
|
||||
E2BSandboxClientOptions,
|
||||
E2BSandboxSessionState,
|
||||
)
|
||||
from agents.items import (
|
||||
MessageOutputItem,
|
||||
RunItem,
|
||||
ToolApprovalItem,
|
||||
ToolCallItem,
|
||||
TResponseInputItem,
|
||||
)
|
||||
from agents.lifecycle import RunHooksBase
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.sandboxes import (
|
||||
DockerSandboxClientOptions,
|
||||
DockerSandboxSessionState,
|
||||
UnixLocalSandboxClientOptions,
|
||||
UnixLocalSandboxSessionState,
|
||||
)
|
||||
from agents.sandbox.session.sandbox_session_state import SandboxSessionState
|
||||
from agents.sandbox.snapshot import SnapshotBase
|
||||
|
||||
# Allow sibling and repo-root imports.
|
||||
_THIS_DIR = _os.path.dirname(_os.path.abspath(__file__))
|
||||
_REPO_ROOT = _os.path.abspath(_os.path.join(_THIS_DIR, "..", "..", "..", ".."))
|
||||
for _p in (_THIS_DIR, _REPO_ROOT):
|
||||
if _p not in sys.path:
|
||||
sys.path.insert(0, _p)
|
||||
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability # noqa: E402
|
||||
|
||||
|
||||
class SandboxBackend(str, Enum):
|
||||
DAYTONA = "daytona"
|
||||
DOCKER = "docker"
|
||||
E2B = "e2b"
|
||||
LOCAL = "local"
|
||||
|
||||
|
||||
DEFAULT_BACKEND = SandboxBackend.DAYTONA
|
||||
TASK_QUEUE = "sandbox-agent-queue"
|
||||
|
||||
|
||||
class _AlwaysSerializeType(BaseModel):
|
||||
"""Base that ensures the ``type`` discriminator survives ``exclude_unset`` round-trips."""
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_always_include_type(self, handler: Any) -> dict[str, Any]:
|
||||
data: dict[str, Any] = handler(self)
|
||||
data["type"] = self.type # type: ignore[attr-defined]
|
||||
return data
|
||||
|
||||
|
||||
class SwitchToLocalBackend(_AlwaysSerializeType):
|
||||
"""Switch target for the local unix sandbox backend."""
|
||||
|
||||
type: Literal["local"] = "local"
|
||||
workspace_root: str = "/workspace"
|
||||
|
||||
|
||||
class SwitchBackendSignal(BaseModel):
|
||||
"""Payload for the ``switch_backend`` signal."""
|
||||
|
||||
target: Literal["daytona", "docker", "e2b"] | SwitchToLocalBackend
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Workflow input / output types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _HasSnapshot(BaseModel):
|
||||
@field_validator("snapshot", mode="before", check_fields=False)
|
||||
@classmethod
|
||||
def _parse_snapshot(cls, v: object) -> SnapshotBase | None:
|
||||
if v is None or isinstance(v, SnapshotBase):
|
||||
return v
|
||||
return SnapshotBase.parse(v)
|
||||
|
||||
|
||||
class WorkflowSnapshot(_HasSnapshot):
|
||||
"""Atomic snapshot of an agent workflow's forkable state."""
|
||||
|
||||
sandbox_session_state: (
|
||||
DaytonaSandboxSessionState
|
||||
| DockerSandboxSessionState
|
||||
| E2BSandboxSessionState
|
||||
| UnixLocalSandboxSessionState
|
||||
| None
|
||||
) = None
|
||||
snapshot: SerializeAsAny[SnapshotBase] | None = (
|
||||
None # serialized SnapshotBase for cross-backend creation
|
||||
)
|
||||
previous_response_id: str | None = None
|
||||
history: list[dict[str, Any]] = []
|
||||
|
||||
|
||||
class AgentRequest(_HasSnapshot):
|
||||
messages: list[dict[str, Any]]
|
||||
cwd: str = ""
|
||||
backend: str = "daytona" # SandboxBackend value — determines client options
|
||||
sandbox_session_state: (
|
||||
DaytonaSandboxSessionState
|
||||
| DockerSandboxSessionState
|
||||
| E2BSandboxSessionState
|
||||
| UnixLocalSandboxSessionState
|
||||
| None
|
||||
) = None
|
||||
snapshot: SerializeAsAny[SnapshotBase] | None = (
|
||||
None # serialized SnapshotBase for cross-backend creation
|
||||
)
|
||||
previous_response_id: str | None = None
|
||||
history: list[dict[str, Any]] = [] # conversation history to seed (e.g. when forking)
|
||||
manifest: Manifest | None = None # per-session manifest override
|
||||
|
||||
|
||||
class AgentResponse(BaseModel):
|
||||
"""Returned when the workflow is destroyed."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class ToolCallRecord(BaseModel):
|
||||
"""A single tool call with its input and output for TUI display."""
|
||||
|
||||
tool_name: str
|
||||
description: str
|
||||
arguments_json: str
|
||||
output: str | None = None
|
||||
requires_approval: bool = False
|
||||
approved: bool | None = None
|
||||
|
||||
|
||||
class ChatResponse(BaseModel):
|
||||
"""Structured response from chat() replacing the plain string."""
|
||||
|
||||
text: str | None = None
|
||||
tool_calls: list[ToolCallRecord] = []
|
||||
approval_request: ToolCallRecord | None = None
|
||||
|
||||
|
||||
class LiveToolCall(BaseModel):
|
||||
"""A tool call visible to the TUI during an active turn."""
|
||||
|
||||
call_id: str
|
||||
tool_name: str
|
||||
arguments: str
|
||||
status: str = "pending" # pending | running | completed
|
||||
output: str | None = None
|
||||
|
||||
|
||||
class TurnState(BaseModel):
|
||||
"""Everything the TUI needs — returned by a single query during polling."""
|
||||
|
||||
# idle | thinking | awaiting_approval | complete
|
||||
status: str = "idle"
|
||||
tool_calls: list[LiveToolCall] = []
|
||||
response_text: str | None = None
|
||||
approval_request: ToolCallRecord | None = None
|
||||
turn_id: int = 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _format_approval_item(item: ToolApprovalItem) -> str:
|
||||
"""Return a human-readable summary of a tool approval request."""
|
||||
raw = item.raw_item
|
||||
name = getattr(raw, "name", None) or item.tool_name or "unknown"
|
||||
|
||||
# Try to extract arguments for shell commands
|
||||
args_str = getattr(raw, "arguments", None)
|
||||
if args_str and isinstance(args_str, str):
|
||||
try:
|
||||
parsed = json.loads(args_str)
|
||||
if name == "shell" and "commands" in parsed:
|
||||
cmds = parsed["commands"]
|
||||
return f"shell: {'; '.join(cmds)}"
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
|
||||
return f"{name}: {args_str or '(no args)'}"
|
||||
|
||||
|
||||
def _extract_text_from_items(items: list[RunItem]) -> str | None:
|
||||
"""Pull the last assistant text from generated run items."""
|
||||
for item in reversed(items):
|
||||
if isinstance(item, MessageOutputItem):
|
||||
raw = item.raw_item
|
||||
content = getattr(raw, "content", [])
|
||||
if isinstance(content, list):
|
||||
for block in content:
|
||||
text = getattr(block, "text", None)
|
||||
if isinstance(text, str):
|
||||
return text
|
||||
return None
|
||||
|
||||
|
||||
def _tool_call_records_from_items(items: list[RunItem]) -> list[ToolCallRecord]:
|
||||
"""Build ToolCallRecord list from generated RunItems."""
|
||||
records: list[ToolCallRecord] = []
|
||||
for item in items:
|
||||
if isinstance(item, ToolCallItem):
|
||||
raw = item.raw_item
|
||||
name = getattr(raw, "name", None) or "unknown"
|
||||
args = getattr(raw, "arguments", "{}")
|
||||
records.append(
|
||||
ToolCallRecord(
|
||||
tool_name=name,
|
||||
description=f"{name}: {args}",
|
||||
arguments_json=args if isinstance(args, str) else json.dumps(args),
|
||||
)
|
||||
)
|
||||
return records
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Workflow definition
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _LiveStateHooks(RunHooksBase[Any, Agent[Any]]):
|
||||
"""RunHooks that update workflow-queryable state for live TUI polling."""
|
||||
|
||||
def __init__(self, wf: AgentWorkflow) -> None:
|
||||
self._wf = wf
|
||||
|
||||
async def on_llm_end(self, context, agent, response):
|
||||
"""Extract tool calls from the model response and register them."""
|
||||
for item in response.output:
|
||||
call_id = getattr(item, "call_id", None)
|
||||
if not call_id:
|
||||
continue
|
||||
# Standard function calls have name + arguments
|
||||
name = getattr(item, "name", None)
|
||||
if name:
|
||||
self._wf._live_tool_calls.append(
|
||||
LiveToolCall(
|
||||
call_id=call_id,
|
||||
tool_name=name,
|
||||
arguments=getattr(item, "arguments", None) or "{}",
|
||||
status="pending",
|
||||
)
|
||||
)
|
||||
continue
|
||||
# Shell tool calls have action.commands / action.command
|
||||
action = getattr(item, "action", None)
|
||||
if action:
|
||||
cmds = getattr(action, "commands", None) or getattr(action, "command", None)
|
||||
if isinstance(cmds, list):
|
||||
args = json.dumps({"commands": cmds})
|
||||
elif isinstance(cmds, str):
|
||||
args = json.dumps({"command": cmds})
|
||||
else:
|
||||
args = "{}"
|
||||
tool_name = getattr(item, "type", None) or "shell"
|
||||
self._wf._live_tool_calls.append(
|
||||
LiveToolCall(
|
||||
call_id=call_id,
|
||||
tool_name=tool_name,
|
||||
arguments=args,
|
||||
status="pending",
|
||||
)
|
||||
)
|
||||
|
||||
async def on_tool_start(self, context, agent, tool):
|
||||
# Match first pending tool call (tools execute in order)
|
||||
for tc in self._wf._live_tool_calls:
|
||||
if tc.status == "pending":
|
||||
tc.status = "running"
|
||||
break
|
||||
|
||||
async def on_tool_end(self, context, agent, tool, result):
|
||||
# Match first running tool call
|
||||
for tc in self._wf._live_tool_calls:
|
||||
if tc.status == "running":
|
||||
tc.status = "completed"
|
||||
tc.output = result[:4000] if result else None
|
||||
break
|
||||
|
||||
|
||||
@workflow.defn
|
||||
class AgentWorkflow:
|
||||
"""A long-lived conversational agent workflow.
|
||||
|
||||
The workflow persists indefinitely in Temporal, idling between TUI
|
||||
sessions. It only terminates when explicitly destroyed via the
|
||||
``destroy`` signal (sent by the session manager).
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._pending_messages: list[str] = []
|
||||
self._done = False
|
||||
self._conversation_history: list[dict[str, Any]] = []
|
||||
self._sandbox_session_state: (
|
||||
DaytonaSandboxSessionState
|
||||
| DockerSandboxSessionState
|
||||
| E2BSandboxSessionState
|
||||
| UnixLocalSandboxSessionState
|
||||
| None
|
||||
) = None
|
||||
self._previous_response_id: str | None = None
|
||||
self._paused: bool = False
|
||||
self._pause_requested = False
|
||||
self._turn_tool_calls: list[ToolCallRecord] = []
|
||||
self._manifest_override: Manifest | None = None
|
||||
self._backend: SandboxBackend = DEFAULT_BACKEND
|
||||
self._snapshot: SnapshotBase | None = None
|
||||
self._live_tool_calls: list[LiveToolCall] = []
|
||||
# Turn state — queried by the TUI polling loop
|
||||
self._turn_status: str = "idle"
|
||||
self._turn_id: int = 0
|
||||
self._last_response_text: str | None = None
|
||||
self._pending_approval: ToolCallRecord | None = None
|
||||
|
||||
@workflow.query
|
||||
def is_paused(self) -> bool:
|
||||
return self._paused
|
||||
|
||||
@workflow.signal
|
||||
async def send_message(self, msg: str) -> None:
|
||||
"""Enqueue a user message. The TUI drives everything via get_turn_state polling."""
|
||||
self._pending_messages.append(msg)
|
||||
self._conversation_history.append({"role": "user", "content": msg})
|
||||
|
||||
@workflow.query
|
||||
def get_history(self) -> list[dict[str, Any]]:
|
||||
"""Return conversation history for TUI replay on reconnect."""
|
||||
return self._conversation_history
|
||||
|
||||
@workflow.query
|
||||
def get_snapshot_id(self) -> str | None:
|
||||
"""Return just the current snapshot ID (lightweight)."""
|
||||
if self._sandbox_session_state:
|
||||
return self._sandbox_session_state.snapshot.id
|
||||
return None
|
||||
|
||||
@workflow.query
|
||||
def get_snapshot(self) -> WorkflowSnapshot:
|
||||
"""Return an atomic snapshot of run state and conversation history."""
|
||||
# Prefer the live session snapshot, but fall back to self._snapshot
|
||||
# so workspace state survives a backend switch (which clears
|
||||
# _sandbox_session_state) until the next turn recreates a session.
|
||||
snapshot = self._snapshot
|
||||
if self._sandbox_session_state:
|
||||
snapshot = self._sandbox_session_state.snapshot
|
||||
return WorkflowSnapshot(
|
||||
sandbox_session_state=self._sandbox_session_state,
|
||||
snapshot=snapshot,
|
||||
previous_response_id=self._previous_response_id,
|
||||
history=self._conversation_history,
|
||||
)
|
||||
|
||||
@workflow.query
|
||||
def get_turn_state(self) -> TurnState:
|
||||
"""Single query that returns everything the TUI needs."""
|
||||
return TurnState(
|
||||
status=self._turn_status,
|
||||
tool_calls=list(self._live_tool_calls),
|
||||
response_text=self._last_response_text,
|
||||
approval_request=self._pending_approval,
|
||||
turn_id=self._turn_id,
|
||||
)
|
||||
|
||||
@workflow.update
|
||||
async def pause(self) -> None:
|
||||
"""Request the workflow to pause."""
|
||||
if self._paused:
|
||||
return
|
||||
self._pause_requested = True
|
||||
await workflow.wait_condition(lambda: self._paused)
|
||||
|
||||
@workflow.update
|
||||
async def switch_backend(self, args: SwitchBackendSignal) -> None:
|
||||
"""Switch to a different sandbox backend for subsequent turns.
|
||||
|
||||
Clears the backend-specific session state so the next turn creates a
|
||||
fresh session on the new backend. The portable snapshot is preserved
|
||||
so the workspace filesystem can be carried over.
|
||||
"""
|
||||
match args.target:
|
||||
case "daytona":
|
||||
self._backend = SandboxBackend.DAYTONA
|
||||
self._manifest_override = Manifest(root="/home/daytona/workspace")
|
||||
case "docker":
|
||||
self._backend = SandboxBackend.DOCKER
|
||||
self._manifest_override = Manifest(root="/workspace")
|
||||
case "e2b":
|
||||
self._backend = SandboxBackend.E2B
|
||||
self._manifest_override = Manifest() # E2B resolves relative to sandbox home
|
||||
case SwitchToLocalBackend(workspace_root=root):
|
||||
self._backend = SandboxBackend.LOCAL
|
||||
self._manifest_override = Manifest(root=root)
|
||||
self._sandbox_session_state = None
|
||||
|
||||
@workflow.signal
|
||||
async def destroy(self) -> None:
|
||||
"""Terminate the workflow permanently."""
|
||||
self._done = True
|
||||
|
||||
def _resolve_sandbox_options(
|
||||
self,
|
||||
) -> (
|
||||
DaytonaSandboxClientOptions
|
||||
| DockerSandboxClientOptions
|
||||
| E2BSandboxClientOptions
|
||||
| UnixLocalSandboxClientOptions
|
||||
):
|
||||
match self._backend:
|
||||
case SandboxBackend.DAYTONA:
|
||||
return DaytonaSandboxClientOptions(pause_on_exit=False)
|
||||
case SandboxBackend.DOCKER:
|
||||
return DockerSandboxClientOptions(image="python:3.14")
|
||||
case SandboxBackend.E2B:
|
||||
return E2BSandboxClientOptions(sandbox_type="e2b")
|
||||
case SandboxBackend.LOCAL:
|
||||
return UnixLocalSandboxClientOptions()
|
||||
|
||||
def _resolve_manifest(self) -> Manifest:
|
||||
match self._backend:
|
||||
case SandboxBackend.DAYTONA:
|
||||
return Manifest(root="/home/daytona/workspace")
|
||||
case SandboxBackend.DOCKER:
|
||||
return Manifest(root="/workspace")
|
||||
case SandboxBackend.E2B:
|
||||
return Manifest() # E2B resolves workspace root relative to the sandbox home
|
||||
case SandboxBackend.LOCAL:
|
||||
return Manifest(root="/workspace")
|
||||
|
||||
@workflow.run
|
||||
async def run(self, request: AgentRequest) -> AgentResponse:
|
||||
self._backend = SandboxBackend(request.backend)
|
||||
self._snapshot = request.snapshot
|
||||
if request.history:
|
||||
self._conversation_history = list(request.history)
|
||||
if request.sandbox_session_state:
|
||||
self._sandbox_session_state = request.sandbox_session_state
|
||||
if request.previous_response_id:
|
||||
self._previous_response_id = request.previous_response_id
|
||||
|
||||
self._manifest_override = request.manifest
|
||||
|
||||
while not self._done:
|
||||
await workflow.wait_condition(
|
||||
lambda: (len(self._pending_messages) > 0 or self._pause_requested or self._done),
|
||||
)
|
||||
|
||||
if self._pause_requested:
|
||||
# Let the caller (e.g. SessionManagerWorkflow.fork_session) know
|
||||
# no turn is in progress so it can safely snapshot state.
|
||||
self._paused = True
|
||||
self._pause_requested = False
|
||||
await workflow.wait_condition(lambda: len(self._pending_messages) > 0 or self._done)
|
||||
self._paused = False
|
||||
|
||||
if self._done:
|
||||
break
|
||||
|
||||
user_messages = list(self._pending_messages)
|
||||
self._pending_messages.clear()
|
||||
|
||||
self._turn_id += 1
|
||||
self._turn_status = "thinking"
|
||||
self._live_tool_calls = []
|
||||
self._pending_approval = None
|
||||
self._last_response_text = None
|
||||
|
||||
try:
|
||||
manifest = self._manifest_override or self._resolve_manifest()
|
||||
agent = self._build_agent(manifest)
|
||||
await self._run_turn(agent, user_messages)
|
||||
self._last_response_text = self._last_text
|
||||
if self._last_text:
|
||||
self._conversation_history.append(
|
||||
{"role": "assistant", "content": self._last_text}
|
||||
)
|
||||
except Exception as e:
|
||||
self._last_response_text = f"Error: {e}"
|
||||
finally:
|
||||
self._turn_status = "complete"
|
||||
|
||||
return AgentResponse()
|
||||
|
||||
def _build_agent(self, manifest: Manifest, model: str = "gpt-5.6-sol") -> SandboxAgent:
|
||||
"""Construct the SandboxAgent used by the workflow."""
|
||||
return SandboxAgent(
|
||||
name="Temporal Sandbox Agent",
|
||||
model=model,
|
||||
instructions=(
|
||||
"You are a helpful coding assistant. Inspect the workspace and answer "
|
||||
"questions. Use the shell tool to run commands. "
|
||||
"Do not invent files or statuses that are not present in the workspace. "
|
||||
"Cite the file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="auto"),
|
||||
)
|
||||
|
||||
async def _run_turn(
|
||||
self,
|
||||
agent: SandboxAgent,
|
||||
user_messages: list[str],
|
||||
) -> None:
|
||||
self._turn_tool_calls = []
|
||||
self._last_text: str | None = None
|
||||
|
||||
hooks = _LiveStateHooks(self)
|
||||
|
||||
# Always pass fresh input — previous_response_id gives the API
|
||||
# conversation context. Sandbox session state is carried via
|
||||
# run_config.sandbox.session_state to preserve the sandbox across turns.
|
||||
if len(user_messages) == 1:
|
||||
input_arg: str | list[TResponseInputItem] = user_messages[0]
|
||||
else:
|
||||
input_arg = [{"role": "user", "content": m} for m in user_messages]
|
||||
|
||||
run_config = RunConfig(
|
||||
sandbox=SandboxRunConfig(
|
||||
client=temporal_sandbox_client(self._backend.value),
|
||||
options=self._resolve_sandbox_options(),
|
||||
# Restore sandbox session state from the previous turn if available.
|
||||
session_state=self._sandbox_session_state,
|
||||
snapshot=self._snapshot,
|
||||
),
|
||||
workflow_name="Temporal Sandbox workflow",
|
||||
)
|
||||
|
||||
# Run the agent -- loops internally handling tool calls
|
||||
result = await Runner.run(
|
||||
agent,
|
||||
input_arg,
|
||||
run_config=run_config,
|
||||
hooks=hooks,
|
||||
previous_response_id=self._previous_response_id,
|
||||
)
|
||||
|
||||
# Extract results
|
||||
self._turn_tool_calls.extend(_tool_call_records_from_items(result.new_items))
|
||||
self._last_text = _extract_text_from_items(result.new_items)
|
||||
|
||||
# Track response ID for conversation continuity and save state
|
||||
# to preserve sandbox session across turns.
|
||||
self._previous_response_id = result.last_response_id
|
||||
|
||||
# Persist sandbox session state for the next turn.
|
||||
try:
|
||||
state = result.to_state()
|
||||
sandbox_data = state.to_json().get("sandbox", {})
|
||||
session_state_data = sandbox_data.get("session_state")
|
||||
if session_state_data:
|
||||
self._sandbox_session_state = cast(
|
||||
DaytonaSandboxSessionState | UnixLocalSandboxSessionState,
|
||||
SandboxSessionState.parse(session_state_data),
|
||||
)
|
||||
# Keep the portable snapshot up to date so it can seed a
|
||||
# fresh session after a backend switch.
|
||||
self._snapshot = self._sandbox_session_state.snapshot
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Worker entrypoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def run_worker() -> None:
|
||||
# Imported here to avoid unnecessary passthroughs in the workflow sandbox.
|
||||
import docker # type: ignore[import-untyped]
|
||||
from _worker_setup import print_backend_warnings # type: ignore[import-not-found]
|
||||
from temporal_session_manager import ( # type: ignore[import-not-found]
|
||||
SessionManagerWorkflow,
|
||||
pause_workflow,
|
||||
query_workflow_snapshot,
|
||||
switch_workflow_backend,
|
||||
)
|
||||
from temporalio.contrib.openai_agents import (
|
||||
ModelActivityParameters,
|
||||
OpenAIAgentsPlugin,
|
||||
SandboxClientProvider,
|
||||
)
|
||||
|
||||
from agents.extensions.sandbox import DaytonaSandboxClient, E2BSandboxClient
|
||||
from agents.sandbox.sandboxes import DockerSandboxClient, UnixLocalSandboxClient
|
||||
|
||||
sandbox_clients: list[SandboxClientProvider] = [
|
||||
SandboxClientProvider("local", UnixLocalSandboxClient()),
|
||||
]
|
||||
if _os.environ.get("DAYTONA_API_KEY"):
|
||||
sandbox_clients.append(SandboxClientProvider("daytona", DaytonaSandboxClient()))
|
||||
if _os.environ.get("E2B_API_KEY"):
|
||||
sandbox_clients.append(SandboxClientProvider("e2b", E2BSandboxClient()))
|
||||
try:
|
||||
sandbox_clients.append(
|
||||
SandboxClientProvider("docker", DockerSandboxClient(docker.from_env()))
|
||||
)
|
||||
except docker.errors.DockerException:
|
||||
pass
|
||||
|
||||
plugin = OpenAIAgentsPlugin(
|
||||
model_params=ModelActivityParameters(
|
||||
start_to_close_timeout=timedelta(seconds=120),
|
||||
),
|
||||
sandbox_clients=sandbox_clients,
|
||||
)
|
||||
|
||||
temporal_client = await Client.connect("localhost:7233", plugins=[plugin])
|
||||
|
||||
worker = Worker(
|
||||
temporal_client,
|
||||
task_queue=TASK_QUEUE,
|
||||
workflows=[AgentWorkflow, SessionManagerWorkflow],
|
||||
activities=[pause_workflow, query_workflow_snapshot, switch_workflow_backend],
|
||||
workflow_runner=SandboxedWorkflowRunner(
|
||||
restrictions=SandboxRestrictions.default.with_passthrough_modules(
|
||||
"pydantic_core",
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
print_backend_warnings({p.name for p in sandbox_clients})
|
||||
print(f"Worker started on task queue '{TASK_QUEUE}'. Press Ctrl-C to stop.")
|
||||
await worker.run()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI entrypoints
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def run_conversation() -> None:
|
||||
"""Start the TUI -- sessions are managed entirely via Temporal."""
|
||||
from temporal_sandbox_tui import ConversationApp # type: ignore[import-not-found]
|
||||
|
||||
app = ConversationApp(
|
||||
workflow_cls=AgentWorkflow,
|
||||
task_queue=TASK_QUEUE,
|
||||
cwd=str(Path.cwd()),
|
||||
)
|
||||
await app.run_async()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Argument parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Run the Sandbox agent as a multi-turn Temporal workflow."
|
||||
)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
sub.add_parser("worker", help="Start the Temporal worker process.")
|
||||
sub.add_parser("run", help="Start an interactive agent conversation.")
|
||||
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
args = parse_args()
|
||||
if args.command == "worker":
|
||||
asyncio.run(run_worker())
|
||||
else:
|
||||
asyncio.run(run_conversation())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,406 @@
|
||||
# mypy: ignore-errors
|
||||
# standalone example with sys.path sibling imports that mypy cannot follow
|
||||
"""Temporal session manager workflow.
|
||||
|
||||
A long-lived singleton workflow that acts as the sole orchestrator for agent
|
||||
session lifecycles. It starts and stops agent workflows, and maintains a
|
||||
registry of active sessions so that TUI clients can list, resume, rename,
|
||||
and destroy sessions without any filesystem persistence.
|
||||
|
||||
The manager is started once (well-known workflow ID ``session-manager``) and
|
||||
lives forever. All lifecycle operations — create, destroy, rename, fork — go
|
||||
through the manager so the registry is always consistent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
from temporalio import activity, workflow
|
||||
from temporalio.exceptions import ApplicationError
|
||||
from temporalio.workflow import ParentClosePolicy
|
||||
|
||||
with workflow.unsafe.imports_passed_through():
|
||||
from pydantic import BaseModel, field_validator, model_serializer
|
||||
from temporal_sandbox_agent import ( # type: ignore[import-not-found]
|
||||
TASK_QUEUE,
|
||||
AgentRequest,
|
||||
AgentWorkflow,
|
||||
SwitchBackendSignal,
|
||||
SwitchToLocalBackend,
|
||||
WorkflowSnapshot,
|
||||
)
|
||||
from temporalio.client import Client
|
||||
from temporalio.contrib.openai_agents import OpenAIAgentsPlugin
|
||||
from temporalio.contrib.pydantic import pydantic_data_converter
|
||||
|
||||
from agents import trace
|
||||
from agents.sandbox import Manifest
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MANAGER_WORKFLOW_ID = "session-manager"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class DaytonaBackendConfig(BaseModel):
|
||||
type: Literal["daytona"] = "daytona"
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_always_include_type(self, handler: Any) -> dict[str, Any]:
|
||||
data: dict[str, Any] = handler(self)
|
||||
data["type"] = self.type
|
||||
return data
|
||||
|
||||
|
||||
class DockerBackendConfig(BaseModel):
|
||||
type: Literal["docker"] = "docker"
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_always_include_type(self, handler: Any) -> dict[str, Any]:
|
||||
data: dict[str, Any] = handler(self)
|
||||
data["type"] = self.type
|
||||
return data
|
||||
|
||||
|
||||
class E2BBackendConfig(BaseModel):
|
||||
type: Literal["e2b"] = "e2b"
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_always_include_type(self, handler: Any) -> dict[str, Any]:
|
||||
data: dict[str, Any] = handler(self)
|
||||
data["type"] = self.type
|
||||
return data
|
||||
|
||||
|
||||
class LocalBackendConfig(BaseModel):
|
||||
type: Literal["local"] = "local"
|
||||
workspace_root: Path | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_always_include_type(self, handler: Any) -> dict[str, Any]:
|
||||
data: dict[str, Any] = handler(self)
|
||||
data["type"] = self.type
|
||||
return data
|
||||
|
||||
@field_validator("workspace_root")
|
||||
@classmethod
|
||||
def _must_be_absolute(cls, v: Path | None) -> Path | None:
|
||||
if v is not None and not v.is_absolute():
|
||||
raise ValueError("workspace_root must be an absolute path")
|
||||
return v
|
||||
|
||||
|
||||
BackendConfig = DaytonaBackendConfig | DockerBackendConfig | E2BBackendConfig | LocalBackendConfig
|
||||
|
||||
|
||||
class SessionInfo(BaseModel):
|
||||
workflow_id: str
|
||||
title: str
|
||||
created_at: datetime
|
||||
cwd: str = ""
|
||||
backend: BackendConfig = DaytonaBackendConfig()
|
||||
parent_workflow_id: str | None = None
|
||||
fork_count: int = 0
|
||||
snapshot_id: str | None = None
|
||||
|
||||
|
||||
class CreateSessionRequest(BaseModel):
|
||||
cwd: str
|
||||
manifest: Manifest | None = None
|
||||
backend: BackendConfig = DaytonaBackendConfig()
|
||||
|
||||
|
||||
class RenameRequest(BaseModel):
|
||||
workflow_id: str
|
||||
title: str
|
||||
|
||||
|
||||
class ForkSessionRequest(BaseModel):
|
||||
source_workflow_id: str
|
||||
title: str | None = None # defaults to "{original title} (fork #N)"
|
||||
target_backend: BackendConfig | None = None
|
||||
|
||||
|
||||
class SwitchBackendRequest(BaseModel):
|
||||
source_workflow_id: str
|
||||
target_backend: BackendConfig
|
||||
|
||||
|
||||
class _SwitchWorkflowBackendArgs(BaseModel):
|
||||
"""Activity args for switch_workflow_backend."""
|
||||
|
||||
workflow_id: str
|
||||
signal: SwitchBackendSignal
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _default_manifest(
|
||||
backend: BackendConfig,
|
||||
) -> Manifest:
|
||||
"""Return the default workspace manifest for the given backend config."""
|
||||
if isinstance(backend, DaytonaBackendConfig):
|
||||
return Manifest(root="/home/daytona/workspace")
|
||||
if isinstance(backend, DockerBackendConfig):
|
||||
return Manifest(root="/workspace")
|
||||
if isinstance(backend, E2BBackendConfig):
|
||||
return Manifest() # E2B resolves workspace root relative to the sandbox home
|
||||
root = str(backend.workspace_root) if backend.workspace_root else "/workspace"
|
||||
return Manifest(root=root)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Activities
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@activity.defn
|
||||
async def pause_workflow(workflow_id: str) -> None:
|
||||
"""Pause the agent workflow and wait for its session to fully stop."""
|
||||
client = await Client.connect("localhost:7233", data_converter=pydantic_data_converter)
|
||||
handle = client.get_workflow_handle(workflow_id)
|
||||
await handle.execute_update(AgentWorkflow.pause)
|
||||
|
||||
|
||||
@activity.defn
|
||||
async def switch_workflow_backend(args: _SwitchWorkflowBackendArgs) -> None:
|
||||
"""Switch the agent workflow's backend and wait for it to take effect."""
|
||||
client = await Client.connect("localhost:7233", data_converter=pydantic_data_converter)
|
||||
handle = client.get_workflow_handle(args.workflow_id)
|
||||
await handle.execute_update(AgentWorkflow.switch_backend, args.signal)
|
||||
|
||||
|
||||
@activity.defn
|
||||
async def query_workflow_snapshot(workflow_id: str) -> WorkflowSnapshot:
|
||||
"""Query the target workflow for its run state and conversation history."""
|
||||
client = await Client.connect("localhost:7233", data_converter=pydantic_data_converter)
|
||||
handle = client.get_workflow_handle(workflow_id)
|
||||
return await handle.query(AgentWorkflow.get_snapshot)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Workflow
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@workflow.defn
|
||||
class SessionManagerWorkflow:
|
||||
"""Registry and orchestrator for agent sessions.
|
||||
|
||||
* ``create_session`` — starts a new agent child workflow and registers it.
|
||||
* ``destroy_session`` — signals the agent workflow to terminate and
|
||||
removes it from the registry.
|
||||
* ``list_sessions`` — query returning all active sessions.
|
||||
* ``rename_session`` — signal to update a session title.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._sessions: dict[str, SessionInfo] = {}
|
||||
self._shutdown = False
|
||||
|
||||
# -- Main loop (lives forever) -----------------------------------------
|
||||
|
||||
@workflow.run
|
||||
async def run(self) -> None:
|
||||
await workflow.wait_condition(lambda: self._shutdown)
|
||||
|
||||
# -- Lifecycle: create & destroy (updates for request-response) ---------
|
||||
|
||||
@workflow.update
|
||||
async def create_session(self, request: CreateSessionRequest) -> str:
|
||||
"""Start a new agent workflow and register it. Returns the workflow ID."""
|
||||
workflow_id = f"sandbox-agent-{workflow.uuid4()}"
|
||||
|
||||
manifest = request.manifest
|
||||
if manifest is None:
|
||||
manifest = _default_manifest(request.backend)
|
||||
|
||||
with OpenAIAgentsPlugin().tracing_context():
|
||||
with trace("Temporal Sandbox Sandbox Agent"):
|
||||
await workflow.start_child_workflow(
|
||||
AgentWorkflow.run,
|
||||
AgentRequest(
|
||||
messages=[],
|
||||
cwd=request.cwd,
|
||||
backend=request.backend.type,
|
||||
history=[],
|
||||
manifest=manifest,
|
||||
),
|
||||
id=workflow_id,
|
||||
task_queue=TASK_QUEUE,
|
||||
parent_close_policy=ParentClosePolicy.ABANDON,
|
||||
)
|
||||
self._sessions[workflow_id] = SessionInfo(
|
||||
workflow_id=workflow_id,
|
||||
title=f"Session {workflow_id[-8:]}",
|
||||
created_at=workflow.now(),
|
||||
cwd=request.cwd,
|
||||
backend=request.backend,
|
||||
)
|
||||
return workflow_id
|
||||
|
||||
@workflow.update
|
||||
async def fork_session(self, request: ForkSessionRequest) -> str:
|
||||
"""Fork an existing session into a new workflow with identical state.
|
||||
|
||||
Pauses the source workflow, queries its RunState and conversation
|
||||
history, then starts a new child workflow seeded with that state.
|
||||
When ``target_backend`` differs from the source, the sandbox session
|
||||
state is not carried over (it is backend-specific), but the portable
|
||||
snapshot is extracted so the new backend can create a fresh session
|
||||
from the same workspace filesystem state.
|
||||
"""
|
||||
source = self._sessions.get(request.source_workflow_id)
|
||||
if source is None:
|
||||
raise ApplicationError(f"Source session {request.source_workflow_id} not found")
|
||||
|
||||
# Pause the source workflow so its session stops naturally
|
||||
await workflow.execute_activity(
|
||||
pause_workflow,
|
||||
request.source_workflow_id,
|
||||
start_to_close_timeout=timedelta(minutes=11),
|
||||
)
|
||||
|
||||
# Fetch the source workflow's state via activity
|
||||
workflow_snapshot: WorkflowSnapshot = await workflow.execute_activity(
|
||||
query_workflow_snapshot,
|
||||
request.source_workflow_id,
|
||||
start_to_close_timeout=timedelta(seconds=30),
|
||||
)
|
||||
|
||||
target_config = (
|
||||
request.target_backend if request.target_backend is not None else source.backend
|
||||
)
|
||||
cross_backend = target_config.type != source.backend.type
|
||||
|
||||
# Determine fork title
|
||||
source.fork_count += 1
|
||||
if cross_backend:
|
||||
title = request.title or f"{source.title} [{target_config.type}]"
|
||||
else:
|
||||
title = request.title or f"{source.title} (fork #{source.fork_count})"
|
||||
|
||||
# Always pass the portable snapshot so the forked session can seed
|
||||
# its workspace. Never carry session_state — a fork creates an
|
||||
# independent session seeded from the snapshot, not a resume of the
|
||||
# source session.
|
||||
snapshot = workflow_snapshot.snapshot
|
||||
|
||||
manifest = _default_manifest(target_config)
|
||||
|
||||
# Start the forked workflow with the source's run state and history
|
||||
workflow_id = f"sandbox-agent-{workflow.uuid4()}"
|
||||
await workflow.start_child_workflow(
|
||||
AgentWorkflow.run,
|
||||
AgentRequest(
|
||||
messages=[],
|
||||
cwd=source.cwd,
|
||||
backend=target_config.type,
|
||||
sandbox_session_state=None,
|
||||
snapshot=snapshot,
|
||||
previous_response_id=workflow_snapshot.previous_response_id,
|
||||
history=workflow_snapshot.history,
|
||||
manifest=manifest,
|
||||
),
|
||||
id=workflow_id,
|
||||
task_queue=TASK_QUEUE,
|
||||
parent_close_policy=ParentClosePolicy.ABANDON,
|
||||
)
|
||||
|
||||
self._sessions[workflow_id] = SessionInfo(
|
||||
workflow_id=workflow_id,
|
||||
title=title,
|
||||
created_at=workflow.now(),
|
||||
cwd=source.cwd,
|
||||
backend=target_config,
|
||||
parent_workflow_id=request.source_workflow_id,
|
||||
snapshot_id=workflow_snapshot.sandbox_session_state.snapshot.id
|
||||
if workflow_snapshot.sandbox_session_state
|
||||
else None,
|
||||
)
|
||||
return workflow_id
|
||||
|
||||
@workflow.update
|
||||
async def switch_backend(self, request: SwitchBackendRequest) -> str:
|
||||
"""Switch a session to a different sandbox backend in-place.
|
||||
|
||||
Signals the agent workflow to change its backend for subsequent turns.
|
||||
The workflow stays the same — no fork, no new child workflow. The
|
||||
portable snapshot is preserved so the workspace can be carried over;
|
||||
the backend-specific session state is cleared by the agent workflow.
|
||||
"""
|
||||
source = self._sessions.get(request.source_workflow_id)
|
||||
if source is None:
|
||||
raise ApplicationError(f"Session {request.source_workflow_id} not found")
|
||||
|
||||
if isinstance(request.target_backend, LocalBackendConfig):
|
||||
target: Literal["daytona", "docker", "e2b"] | SwitchToLocalBackend = (
|
||||
SwitchToLocalBackend(
|
||||
workspace_root=str(request.target_backend.workspace_root)
|
||||
if request.target_backend.workspace_root
|
||||
else "/workspace",
|
||||
)
|
||||
)
|
||||
else:
|
||||
target = request.target_backend.type
|
||||
await workflow.execute_activity(
|
||||
switch_workflow_backend,
|
||||
_SwitchWorkflowBackendArgs(
|
||||
workflow_id=request.source_workflow_id,
|
||||
signal=SwitchBackendSignal(target=target),
|
||||
),
|
||||
start_to_close_timeout=timedelta(seconds=30),
|
||||
)
|
||||
|
||||
source.backend = request.target_backend
|
||||
return request.source_workflow_id
|
||||
|
||||
@workflow.update
|
||||
async def destroy_session(self, workflow_id: str) -> None:
|
||||
"""Signal the agent workflow to destroy and remove it from the registry."""
|
||||
handle = workflow.get_external_workflow_handle(workflow_id)
|
||||
await handle.signal(AgentWorkflow.destroy)
|
||||
self._sessions.pop(workflow_id, None)
|
||||
|
||||
# -- Metadata: queries and signals --------------------------------------
|
||||
|
||||
@workflow.query
|
||||
def list_sessions(self) -> list[SessionInfo]:
|
||||
"""Return all active sessions, newest first."""
|
||||
return sorted(
|
||||
self._sessions.values(),
|
||||
key=lambda s: s.created_at,
|
||||
reverse=True,
|
||||
)
|
||||
|
||||
@workflow.signal
|
||||
async def rename_session(self, request: RenameRequest) -> None:
|
||||
"""Update the title of an existing session."""
|
||||
if request.workflow_id in self._sessions:
|
||||
self._sessions[request.workflow_id].title = request.title
|
||||
|
||||
@workflow.signal
|
||||
async def update_snapshot_id(self, request: RenameRequest) -> None:
|
||||
"""Update the cached snapshot_id for a session.
|
||||
|
||||
Reuses RenameRequest where ``title`` carries the snapshot ID.
|
||||
"""
|
||||
if request.workflow_id in self._sessions:
|
||||
self._sessions[request.workflow_id].snapshot_id = request.title
|
||||
|
||||
@workflow.signal
|
||||
async def shutdown(self) -> None:
|
||||
"""Terminate the manager workflow (rarely needed)."""
|
||||
self._shutdown = True
|
||||
@@ -0,0 +1,424 @@
|
||||
"""
|
||||
Minimal Vercel-backed sandbox example for manual validation.
|
||||
|
||||
This mirrors the other cloud extension examples: it creates a tiny workspace,
|
||||
verifies stop/resume persistence, then asks a sandboxed agent to inspect the
|
||||
workspace through one shell tool.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
|
||||
from agents import ModelSettings, Runner
|
||||
from agents.models.openai_provider import OpenAIProvider
|
||||
from agents.run import RunConfig
|
||||
from agents.sandbox import LocalSnapshotSpec, Manifest, SandboxAgent, SandboxRunConfig
|
||||
from agents.sandbox.session import BaseSandboxSession
|
||||
|
||||
if __package__ is None or __package__ == "":
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
||||
|
||||
from examples.sandbox.misc.example_support import text_manifest
|
||||
from examples.sandbox.misc.workspace_shell import WorkspaceShellCapability
|
||||
|
||||
try:
|
||||
from agents.extensions.sandbox import VercelSandboxClient, VercelSandboxClientOptions
|
||||
except Exception as exc: # pragma: no cover - import path depends on optional extras
|
||||
raise SystemExit(
|
||||
"Vercel sandbox examples require the optional repo extra.\n"
|
||||
"Install it with: uv sync --extra vercel"
|
||||
) from exc
|
||||
|
||||
|
||||
DEFAULT_QUESTION = "Summarize this cloud sandbox workspace in 2 sentences."
|
||||
SNAPSHOT_CHECK_PATH = Path("snapshot-check.txt")
|
||||
SNAPSHOT_CHECK_CONTENT = "vercel snapshot round-trip ok\n"
|
||||
LIVE_RESUME_CHECK_PATH = Path("live-resume-check.txt")
|
||||
LIVE_RESUME_CHECK_CONTENT = "vercel live resume ok\n"
|
||||
EXPOSED_PORT = 3000
|
||||
PORT_CHECK_CONTENT = "<h1>vercel exposed port ok</h1>\n"
|
||||
PORT_CHECK_NODE_SERVER_PATH = Path(".port-check-server.js")
|
||||
PORT_CHECK_NODE_SERVER_CONTENT = f"""\
|
||||
const http = require("node:http");
|
||||
|
||||
http
|
||||
.createServer((_request, response) => {{
|
||||
response.writeHead(200, {{"Content-Type": "text/html; charset=utf-8"}});
|
||||
response.end({json.dumps(PORT_CHECK_CONTENT)});
|
||||
}})
|
||||
.listen({EXPOSED_PORT}, "0.0.0.0");
|
||||
"""
|
||||
PORT_CHECK_PYTHON_SERVER_PATH = Path(".port-check-server.py")
|
||||
PORT_CHECK_PYTHON_SERVER_CONTENT = f"""\
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_GET(self) -> None:
|
||||
body = {PORT_CHECK_CONTENT!r}.encode("utf-8")
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, format: str, *args: object) -> None:
|
||||
return
|
||||
|
||||
|
||||
ThreadingHTTPServer(("0.0.0.0", {EXPOSED_PORT}), Handler).serve_forever()
|
||||
"""
|
||||
|
||||
|
||||
def _build_manifest() -> Manifest:
|
||||
return text_manifest(
|
||||
{
|
||||
"README.md": (
|
||||
"# Vercel Demo Workspace\n\n"
|
||||
"This workspace exists to validate the Vercel sandbox backend manually.\n"
|
||||
),
|
||||
"handoff.md": (
|
||||
"# Handoff\n\n"
|
||||
"- Customer: Northwind Traders.\n"
|
||||
"- Goal: validate Vercel sandbox exec and persistence flows.\n"
|
||||
"- Current status: non-PTY backend slice is wired and under test.\n"
|
||||
),
|
||||
"todo.md": (
|
||||
"# Todo\n\n"
|
||||
"1. Inspect the workspace files.\n"
|
||||
"2. Summarize the current status in two sentences.\n"
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
async def _read_text(session: BaseSandboxSession, path: Path) -> str:
|
||||
data = await session.read(path)
|
||||
text = cast(str | bytes, data.read())
|
||||
if isinstance(text, bytes):
|
||||
return text.decode("utf-8")
|
||||
return text
|
||||
|
||||
|
||||
def _require_env(name: str) -> None:
|
||||
if os.environ.get(name):
|
||||
return
|
||||
raise SystemExit(f"{name} must be set before running this example.")
|
||||
|
||||
|
||||
def _require_vercel_credentials() -> None:
|
||||
if os.environ.get("VERCEL_OIDC_TOKEN"):
|
||||
return
|
||||
if (
|
||||
os.environ.get("VERCEL_TOKEN")
|
||||
and os.environ.get("VERCEL_PROJECT_ID")
|
||||
and os.environ.get("VERCEL_TEAM_ID")
|
||||
):
|
||||
return
|
||||
raise SystemExit(
|
||||
"Vercel credentials are required. Set VERCEL_OIDC_TOKEN, or set "
|
||||
"VERCEL_TOKEN together with VERCEL_PROJECT_ID and VERCEL_TEAM_ID."
|
||||
)
|
||||
|
||||
|
||||
async def _verify_stop_resume(
|
||||
*,
|
||||
manifest: Manifest,
|
||||
runtime: str | None,
|
||||
timeout_ms: int | None,
|
||||
workspace_persistence: Literal["tar", "snapshot"],
|
||||
) -> None:
|
||||
client = VercelSandboxClient()
|
||||
options = VercelSandboxClientOptions(
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
)
|
||||
with tempfile.TemporaryDirectory(prefix="vercel-snapshot-example-") as snapshot_dir:
|
||||
sandbox = await client.create(
|
||||
manifest=manifest,
|
||||
snapshot=LocalSnapshotSpec(base_path=Path(snapshot_dir)),
|
||||
options=options,
|
||||
)
|
||||
|
||||
try:
|
||||
await sandbox.start()
|
||||
await sandbox.write(
|
||||
SNAPSHOT_CHECK_PATH,
|
||||
io.BytesIO(SNAPSHOT_CHECK_CONTENT.encode("utf-8")),
|
||||
)
|
||||
await sandbox.stop()
|
||||
finally:
|
||||
await sandbox.shutdown()
|
||||
|
||||
resumed_sandbox = await client.resume(sandbox.state)
|
||||
try:
|
||||
await resumed_sandbox.start()
|
||||
restored_text = await _read_text(resumed_sandbox, SNAPSHOT_CHECK_PATH)
|
||||
if restored_text != SNAPSHOT_CHECK_CONTENT:
|
||||
raise RuntimeError(
|
||||
f"Snapshot resume verification failed for {workspace_persistence!r}: "
|
||||
f"expected {SNAPSHOT_CHECK_CONTENT!r}, got {restored_text!r}"
|
||||
)
|
||||
finally:
|
||||
await resumed_sandbox.aclose()
|
||||
|
||||
print(f"snapshot round-trip ok ({workspace_persistence})")
|
||||
|
||||
|
||||
async def _verify_resume_running_sandbox(
|
||||
*,
|
||||
manifest: Manifest,
|
||||
runtime: str | None,
|
||||
timeout_ms: int | None,
|
||||
workspace_persistence: Literal["tar", "snapshot"],
|
||||
) -> None:
|
||||
client = VercelSandboxClient()
|
||||
sandbox = await client.create(
|
||||
manifest=manifest,
|
||||
options=VercelSandboxClientOptions(
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
),
|
||||
)
|
||||
|
||||
try:
|
||||
await sandbox.start()
|
||||
await sandbox.write(
|
||||
LIVE_RESUME_CHECK_PATH,
|
||||
io.BytesIO(LIVE_RESUME_CHECK_CONTENT.encode("utf-8")),
|
||||
)
|
||||
serialized = client.serialize_session_state(sandbox.state)
|
||||
resumed_sandbox = await client.resume(client.deserialize_session_state(serialized))
|
||||
try:
|
||||
restored_text = await _read_text(resumed_sandbox, LIVE_RESUME_CHECK_PATH)
|
||||
if restored_text != LIVE_RESUME_CHECK_CONTENT:
|
||||
raise RuntimeError(
|
||||
"Running sandbox resume verification failed: "
|
||||
f"expected {LIVE_RESUME_CHECK_CONTENT!r}, got {restored_text!r}"
|
||||
)
|
||||
finally:
|
||||
await resumed_sandbox.aclose()
|
||||
finally:
|
||||
await sandbox.shutdown()
|
||||
|
||||
print(f"running sandbox resume ok ({workspace_persistence})")
|
||||
|
||||
|
||||
def _fetch_url(url: str) -> str:
|
||||
with urllib.request.urlopen(url, timeout=10) as response:
|
||||
return cast(str, response.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _port_check_server_command() -> str:
|
||||
node_path = PORT_CHECK_NODE_SERVER_PATH.as_posix()
|
||||
python_path = PORT_CHECK_PYTHON_SERVER_PATH.as_posix()
|
||||
return (
|
||||
"if command -v node >/dev/null 2>&1; then "
|
||||
f"node {node_path}; "
|
||||
"elif command -v python3 >/dev/null 2>&1; then "
|
||||
f"python3 {python_path}; "
|
||||
"else "
|
||||
"echo 'Neither node nor python3 is available for exposed port verification.' >&2; "
|
||||
"exit 127; "
|
||||
"fi >/tmp/vercel-http.log 2>&1 &"
|
||||
)
|
||||
|
||||
|
||||
async def _verify_exposed_port(
|
||||
*,
|
||||
manifest: Manifest,
|
||||
runtime: str | None,
|
||||
timeout_ms: int | None,
|
||||
workspace_persistence: Literal["tar", "snapshot"],
|
||||
) -> None:
|
||||
client = VercelSandboxClient()
|
||||
sandbox = await client.create(
|
||||
manifest=manifest,
|
||||
options=VercelSandboxClientOptions(
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
exposed_ports=(EXPOSED_PORT,),
|
||||
),
|
||||
)
|
||||
|
||||
try:
|
||||
await sandbox.start()
|
||||
await sandbox.write(
|
||||
PORT_CHECK_NODE_SERVER_PATH,
|
||||
io.BytesIO(PORT_CHECK_NODE_SERVER_CONTENT.encode("utf-8")),
|
||||
)
|
||||
await sandbox.write(
|
||||
PORT_CHECK_PYTHON_SERVER_PATH,
|
||||
io.BytesIO(PORT_CHECK_PYTHON_SERVER_CONTENT.encode("utf-8")),
|
||||
)
|
||||
result = await sandbox.exec(
|
||||
_port_check_server_command(),
|
||||
shell=True,
|
||||
)
|
||||
if not result.ok():
|
||||
raise RuntimeError(
|
||||
f"Failed to start HTTP server for exposed port check: {result.stderr!r}"
|
||||
)
|
||||
|
||||
endpoint = await sandbox.resolve_exposed_port(EXPOSED_PORT)
|
||||
url = f"{'https' if endpoint.tls else 'http'}://{endpoint.host}:{endpoint.port}/"
|
||||
|
||||
last_error: Exception | None = None
|
||||
for _ in range(20):
|
||||
try:
|
||||
body = await asyncio.to_thread(_fetch_url, url)
|
||||
except (TimeoutError, urllib.error.URLError, ValueError) as exc:
|
||||
last_error = exc
|
||||
await asyncio.sleep(0.5)
|
||||
continue
|
||||
|
||||
if PORT_CHECK_CONTENT.strip() not in body:
|
||||
raise RuntimeError(f"Exposed port returned unexpected body from {url!r}: {body!r}")
|
||||
print(f"exposed port ok ({workspace_persistence}) -> {url}")
|
||||
return
|
||||
|
||||
raise RuntimeError(f"Exposed port verification failed for {url!r}") from last_error
|
||||
finally:
|
||||
await sandbox.shutdown()
|
||||
|
||||
|
||||
async def main(
|
||||
*,
|
||||
model: str,
|
||||
question: str,
|
||||
runtime: str | None,
|
||||
timeout_ms: int | None,
|
||||
workspace_persistence: Literal["tar", "snapshot"],
|
||||
stream: bool,
|
||||
) -> None:
|
||||
_require_env("OPENAI_API_KEY")
|
||||
_require_vercel_credentials()
|
||||
|
||||
manifest = _build_manifest()
|
||||
|
||||
await _verify_stop_resume(
|
||||
manifest=manifest,
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
)
|
||||
await _verify_resume_running_sandbox(
|
||||
manifest=manifest,
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
)
|
||||
await _verify_exposed_port(
|
||||
manifest=manifest,
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
)
|
||||
|
||||
agent = SandboxAgent(
|
||||
name="Vercel Sandbox Assistant",
|
||||
model=model,
|
||||
instructions=(
|
||||
"Answer questions about the sandbox workspace. Inspect the files before answering "
|
||||
"and keep the response concise. "
|
||||
"Do not invent files or statuses that are not present in the workspace. Cite the "
|
||||
"file names you inspected."
|
||||
),
|
||||
default_manifest=manifest,
|
||||
capabilities=[WorkspaceShellCapability()],
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
)
|
||||
|
||||
client = VercelSandboxClient()
|
||||
sandbox = await client.create(
|
||||
manifest=manifest,
|
||||
options=VercelSandboxClientOptions(
|
||||
runtime=runtime,
|
||||
timeout_ms=timeout_ms,
|
||||
workspace_persistence=workspace_persistence,
|
||||
),
|
||||
)
|
||||
|
||||
run_config = RunConfig(
|
||||
model_provider=OpenAIProvider(),
|
||||
sandbox=SandboxRunConfig(session=sandbox),
|
||||
# Disable tracing because it does not currently work reliably with alternate
|
||||
# upstreams such as AI Gateway, and provider config already comes from env.
|
||||
tracing_disabled=True,
|
||||
workflow_name="Vercel sandbox example",
|
||||
)
|
||||
|
||||
try:
|
||||
async with sandbox:
|
||||
if not stream:
|
||||
result = await Runner.run(agent, question, run_config=run_config)
|
||||
print(result.final_output)
|
||||
return
|
||||
|
||||
stream_result = Runner.run_streamed(agent, question, run_config=run_config)
|
||||
saw_text_delta = False
|
||||
async for event in stream_result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(
|
||||
event.data, ResponseTextDeltaEvent
|
||||
):
|
||||
if not saw_text_delta:
|
||||
print("assistant> ", end="", flush=True)
|
||||
saw_text_delta = True
|
||||
print(event.data.delta, end="", flush=True)
|
||||
|
||||
if saw_text_delta:
|
||||
print()
|
||||
finally:
|
||||
await client.delete(sandbox)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", default="gpt-5.6-sol", help="Model name to use.")
|
||||
parser.add_argument("--question", default=DEFAULT_QUESTION, help="Prompt to send to the agent.")
|
||||
parser.add_argument(
|
||||
"--runtime",
|
||||
default=None,
|
||||
help="Optional Vercel runtime, for example `node22` or `python3.14`.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout-ms",
|
||||
type=int,
|
||||
default=120_000,
|
||||
help="Optional Vercel sandbox timeout in milliseconds.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--workspace-persistence",
|
||||
choices=("tar", "snapshot"),
|
||||
default="tar",
|
||||
help="Workspace persistence mode to verify before the agent run.",
|
||||
)
|
||||
parser.add_argument("--stream", action="store_true", default=False, help="Stream the response.")
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(
|
||||
main(
|
||||
model=args.model,
|
||||
question=args.question,
|
||||
runtime=args.runtime,
|
||||
timeout_ms=args.timeout_ms,
|
||||
workspace_persistence=cast(Literal["tar", "snapshot"], args.workspace_persistence),
|
||||
stream=args.stream,
|
||||
)
|
||||
)
|
||||
Reference in New Issue
Block a user