chore: import upstream snapshot with attribution

examples/sandbox/extensions/daytona/usaspending_text2sql/README.md

@@ -0,0 +1,79 @@
+# NASA Spending Text-to-SQL Agent
+
+Multi-turn conversational agent that translates natural-language questions about NASA federal spending into SQL queries, executes them against a local SQLite database, and returns structured tabular results.
+
+## How it works
+
+1. **Schema knowledge**: The agent receives a compact schema summary in its system prompt and can read detailed per-table documentation from workspace files on demand.
+2. **SQL execution**: A custom `SqlCapability` provides a `run_sql` tool with guardrails — read-only mode, statement validation, row limits, and query timeouts. The agent is instructed to use `run_sql` for all queries; the tool enforces read-only access at the SQLite level.
+3. **Multi-turn conversation**: The agent retains context across turns, so you can ask follow-up questions like "break that down by year" or "just the top 5".
+4. **Compaction**: Uses the `Compaction` capability to automatically summarize older conversation context, keeping long sessions within the model's context window.
+5. **Pause/resume**: Type `exit` to pause the sandbox and quit. Run the script again to reconnect to the same paused sandbox — no re-download needed. If the sandbox can't be reconnected (e.g. it was deleted or expired), a fresh one is created and the database is rebuilt automatically.
+6. **Memory**: Uses the `Memory` capability to extract learnings from each conversation and consolidate them into structured files. On subsequent sessions, the agent starts with context from previous conversations (useful query patterns, data caveats, etc.).
+
+## Data
+
+The database contains NASA federal spending data from [USAspending.gov](https://usaspending.gov), defaulting to FY2021-FY2025 (configurable via `--start-fy`/`--end-fy` flags on `setup_db.py`).
+
+It uses a single `spending` table where each row is one transaction (obligation, modification, or de-obligation) on a federal award. The agent aggregates as needed via SQL.
+
+The database is built automatically on first run (requires internet access in the sandbox). Subsequent runs reuse the existing database.
+
+## Prerequisites
+
+- Python 3.12+
+- `openai-agents` installed with Daytona support (`uv sync --extra daytona` from repo root)
+- `OPENAI_API_KEY` environment variable set (for the LLM)
+- `DAYTONA_API_KEY` environment variable set (for the sandbox — get one at [daytona.io](https://daytona.io))
+- Internet access (for first-run database setup inside the sandbox)
+
+## Run
+
+From the repository root:
+
+```bash
+export OPENAI_API_KEY="sk-..."
+export DAYTONA_API_KEY="..."
+uv run python -m examples.sandbox.extensions.daytona.usaspending_text2sql.agent
+```
+
+## Example questions
+
+```
+> What are NASA's top 10 contractors by total spending?
+> Break that down by fiscal year
+> Which NASA centers award the most contracts?
+> Show me grants to universities in California
+> How has NASA spending changed over time?
+> What are the largest individual awards in the last 3 years?
+> Compare contract vs grant spending by year
+```
+
+## Architecture
+
+```
+daytona/usaspending_text2sql/
+├── agent.py            — SandboxAgent definition + interactive REPL
+├── sql_capability.py   — SqlCapability (Capability) with run_sql tool and guardrails
+├── setup_db.py         — Runs inside sandbox; fetches data from USAspending API, builds SQLite DB
+├── schema/
+│   ├── overview.md     — Compact schema summary (injected into instructions)
+│   └── tables/         — Per-table column documentation (read on demand via Shell capability)
+└── README.md
+```
+
+### SQL guardrails (defense in depth)
+
+1. **Connection-level**: SQLite opened with `?mode=ro` URI (read-only)
+2. **PRAGMA**: `query_only = ON` prevents writes even if validation is bypassed
+3. **Statement validation**: Only `SELECT`, `WITH`, `EXPLAIN`, `PRAGMA` are allowed
+4. **Row limit**: Hard cap (default 100 rows) with truncation detection
+5. **Timeout**: Queries killed after 30 seconds
+
+### Audit log
+
+All sandbox operations (exec calls, start/stop, SQL queries and their results) are logged to `.audit_log.jsonl` as structured JSONL events via the SDK's `Instrumentation` and `JsonlOutboxSink`. This is useful for debugging, replaying sessions, or inspecting exactly what SQL the agent ran.
+
+### Sandbox
+
+This example uses Daytona as its sandbox backend. The agent and capability definitions are backend-agnostic, but the entrypoint (`agent.py`) hardcodes `DaytonaSandboxClient` and Daytona-specific features like pause/resume.

examples/sandbox/extensions/daytona/usaspending_text2sql/__init__.py

@@ -0,0 +1 @@
+"""USAspending text-to-SQL Daytona sandbox example."""

examples/sandbox/extensions/daytona/usaspending_text2sql/agent.py

@@ -0,0 +1,540 @@
+"""NASA spending text-to-SQL agent.
+
+Multi-turn conversational agent that translates natural-language questions
+about NASA federal spending into SQL queries, executes them against a
+USAspending SQLite database, and returns structured results.
+
+Usage:
+    uv run python -m examples.sandbox.extensions.daytona.usaspending_text2sql.agent
+
+The database is built automatically inside the sandbox on first run by
+executing setup_db.py (requires internet access). Subsequent runs reuse the
+existing database.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+import re
+import sys
+import textwrap
+from pathlib import Path
+from typing import Any
+
+from openai.types.responses import ResponseTextDeltaEvent
+
+from agents import Runner
+from agents.run import RunConfig
+from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
+from agents.sandbox.capabilities.compaction import Compaction
+from agents.sandbox.capabilities.memory import Memory
+from agents.sandbox.capabilities.shell import Shell
+from agents.sandbox.config import MemoryGenerateConfig, MemoryReadConfig
+from agents.sandbox.entries import Dir, File, LocalDir, LocalFile
+from agents.sandbox.session import (
+    EventPayloadPolicy,
+    Instrumentation,
+    JsonlOutboxSink,
+)
+from examples.auto_mode import input_with_fallback, is_auto_mode
+from examples.sandbox.extensions.daytona.usaspending_text2sql.sql_capability import (
+    SqlCapability,
+)
+
+try:
+    from agents.extensions.sandbox import (
+        DEFAULT_DAYTONA_WORKSPACE_ROOT,
+        DaytonaSandboxClient,
+        DaytonaSandboxClientOptions,
+        DaytonaSandboxSessionState,
+    )
+except Exception as exc:  # pragma: no cover
+    raise SystemExit(
+        "Daytona sandbox examples require the optional repo extra.\n"
+        "Install it with: uv sync --extra daytona"
+    ) from exc
+
+EXAMPLE_DIR = Path(__file__).parent
+SCHEMA_DIR = EXAMPLE_DIR / "schema"
+SETUP_DB_PATH = EXAMPLE_DIR / "setup_db.py"
+SESSION_STATE_PATH = EXAMPLE_DIR / ".session_state.json"
+AUDIT_LOG_PATH = EXAMPLE_DIR / ".audit_log.jsonl"
+
+# Set at runtime once the exposed port is resolved.
+_downloads_base_url: str = ""
+
+DEVELOPER_INSTRUCTIONS = (
+    (SCHEMA_DIR / "overview.md").read_text()
+    + """
+
+## Instructions
+
+- Always use the `run_sql` tool to query the database. Never attempt to run sqlite3 directly.
+- Read schema documentation from schema/tables/ if you need detailed column information.
+- Read schema/glossary.md for official USAspending term definitions (e.g., what "obligation" vs "outlay" means).
+- Prefer aggregations (GROUP BY, SUM, COUNT, AVG) over returning many raw rows.
+- Format monetary values with dollar signs and commas in your final answers (e.g., $1,234,567).
+- When the user asks a follow-up question, use conversation context to understand references
+  like "break that down by year" or "just the top 5".
+- If a query fails, read the error message and try to fix the SQL.
+- Explain your query logic briefly so the user can verify correctness.
+
+## Data caveats
+
+- The database contains **obligations** (money legally committed), not outlays (money actually paid).
+  When the user asks about "spending", clarify that these are obligation amounts.
+- Amounts are tied to the **action_date** (when the obligation was signed), not when the work happens.
+  A multi-year contract may appear entirely in the fiscal year it was obligated.
+- Some recipients are masked as "MULTIPLE RECIPIENTS" or "REDACTED DUE TO PII" for privacy reasons.
+  Mention this if recipient-level analysis looks incomplete.
+"""
+)
+
+DB_PATH = "data/usaspending.db"
+DEFAULT_AUTO_QUESTION = "What are NASA's top 5 contractors by total obligations?"
+
+WORKSPACE_ROOT = DEFAULT_DAYTONA_WORKSPACE_ROOT
+
+
+def build_agent() -> SandboxAgent:
+    """Build the agent blueprint."""
+    generate_memory = not is_auto_mode()
+    manifest = Manifest(
+        root=WORKSPACE_ROOT,
+        entries={
+            "setup_db.py": LocalFile(src=SETUP_DB_PATH),
+            "schema": LocalDir(src=SCHEMA_DIR),
+            "data": Dir(ephemeral=True),
+            "memories/MEMORY.md": File(content=b""),
+            "memories/memory_summary.md": File(content=b""),
+            "memories/phase_two_selection.json": File(content=b""),
+        },
+    )
+
+    return SandboxAgent(
+        name="NASA Spending Q&A",
+        default_manifest=manifest,
+        model="gpt-5.6-sol",
+        instructions=(
+            "You are a helpful data analyst that answers questions about NASA federal spending "
+            "by writing and executing SQL queries.\n\n" + DEVELOPER_INSTRUCTIONS
+        ),
+        capabilities=[
+            SqlCapability(db_path=DB_PATH),
+            Shell(),
+            Compaction(),
+            Memory(
+                read=MemoryReadConfig(live_update=False),
+                generate=(
+                    MemoryGenerateConfig(
+                        extra_prompt=(
+                            "Pay attention to which SQL patterns work best for the USAspending "
+                            "data, column quirks (e.g. recipient_parent_name vs recipient_name "
+                            "for grouping), and data caveats the user discovers (e.g. negative "
+                            "obligations, masked recipients)."
+                        ),
+                    )
+                    if generate_memory
+                    else None
+                ),
+            ),
+        ],
+    )
+
+
+# ---------------------------------------------------------------------------
+# Terminal formatting helpers (unchanged from universal_computer version)
+# ---------------------------------------------------------------------------
+
+DIM = "\033[2;39m"
+DIM_CYAN = "\033[2;36m"
+DIM_BLUE = "\033[2;34m"
+DIM_YELLOW = "\033[2;33m"
+DIM_GREEN = "\033[2;32m"
+RESET = "\033[0m"
+
+_SQL_KEYWORDS = (
+    r"\b(?:SELECT|FROM|WHERE|JOIN|LEFT|RIGHT|INNER|OUTER|CROSS|FULL|NATURAL|ON|AND|OR"
+    r"|NOT|IN|IS|NULL|AS|WITH|GROUP\s+BY|ORDER\s+BY|HAVING|LIMIT|OFFSET|UNION"
+    r"|ALL|DISTINCT|CASE|WHEN|THEN|ELSE|END|EXISTS|BETWEEN|LIKE|INSERT|UPDATE"
+    r"|DELETE|CREATE|DROP|ALTER|SET|VALUES|INTO|TABLE|INDEX|VIEW|ASC|DESC|BY"
+    r"|OVER|PARTITION\s+BY)\b"
+)
+
+_SQL_FUNCTIONS = (
+    r"\b(?:COUNT|SUM|AVG|MIN|MAX|COALESCE|CAST|SUBSTR|LENGTH|ROUND|ABS|IFNULL"
+    r"|NULLIF|REPLACE|TRIM|UPPER|LOWER|DATE|DATETIME|STRFTIME|TYPEOF|TOTAL"
+    r"|GROUP_CONCAT|PRINTF|ROW_NUMBER|RANK|DENSE_RANK)(?=\s*\()"
+)
+
+_SQL_STRING = r"'(?:''|[^'])*'"
+
+
+def _highlight_sql(sql: str) -> str:
+    """Apply ANSI syntax highlighting to a SQL string."""
+    placeholders: list[str] = []
+
+    def _stash_string(m: re.Match[str]) -> str:
+        placeholders.append(m.group(0))
+        return f"\x00STR{len(placeholders) - 1}\x00"
+
+    result = re.sub(_SQL_STRING, _stash_string, sql)
+
+    result = re.sub(
+        _SQL_KEYWORDS,
+        lambda m: f"{DIM_BLUE}{m.group(0)}{DIM}",
+        result,
+        flags=re.IGNORECASE,
+    )
+    result = re.sub(
+        _SQL_FUNCTIONS,
+        lambda m: f"{DIM_YELLOW}{m.group(0)}{DIM}",
+        result,
+        flags=re.IGNORECASE,
+    )
+
+    def _restore_string(m: re.Match[str]) -> str:
+        idx = int(m.group(1))
+        return f"{DIM_GREEN}{placeholders[idx]}{DIM}"
+
+    result = re.sub(r"\x00STR(\d+)\x00", _restore_string, result)
+    return result
+
+
+def _format_tool_args(name: str, arguments: str) -> str:
+    """Format a tool call for display, pretty-printing SQL queries."""
+    if name == "run_sql":
+        try:
+            args = json.loads(arguments)
+            query = args.get("query", "")
+            limit = args.get("limit")
+            header = f"  {DIM}[SQL]"
+            if limit is not None:
+                header += f"  (limit {limit})"
+            header += RESET
+            highlighted = _highlight_sql(query)
+            sql = textwrap.indent(highlighted, "    ")
+            return f"{header}\n{DIM}{sql}{RESET}"
+        except Exception:
+            pass
+    return f"  {DIM}[tool] {name}({arguments}){RESET}"
+
+
+def _format_tool_result(output: str) -> str | None:
+    """Format a tool result for display. Returns None for non-SQL results."""
+    try:
+        data = json.loads(output)
+    except (json.JSONDecodeError, TypeError):
+        if output.strip():
+            return f"  {DIM}{output.strip()}{RESET}"
+        return None
+
+    columns = data.get("columns")
+    rows = data.get("rows")
+    if not isinstance(columns, list) or not isinstance(rows, list):
+        return None
+
+    row_count = data.get("row_count", len(rows))
+    display_count = data.get("display_count", len(rows))
+    truncated = data.get("truncated", False)
+
+    if not columns:
+        return f"  {DIM_CYAN}\u2192 Result (0 rows){RESET}"
+
+    # Build the summary line.
+    parts = []
+    if display_count < row_count:
+        parts.append(f"showing {display_count} of {row_count}")
+    else:
+        parts.append(f"{row_count} rows")
+    if truncated:
+        parts.append("CSV truncated at limit")
+
+    csv_file = data.get("csv_file")
+    download_line = ""
+    if csv_file and _downloads_base_url:
+        download_line = f"\n  {DIM}\u2193 {_downloads_base_url}{csv_file}{RESET}"
+
+    # Try to fit the table in the terminal. If too wide, skip it —
+    # the model's prose summary + download link are enough.
+    try:
+        term_width = os.get_terminal_size().columns
+    except OSError:
+        term_width = 120
+
+    widths = [len(str(c)) for c in columns]
+    for row in rows:
+        for i, val in enumerate(row):
+            widths[i] = max(widths[i], len(str(val) if val is not None else "NULL"))
+
+    # 4 leading spaces + "| " between each col + trailing " |"
+    table_width = 4 + sum(widths) + 3 * len(widths) + 1
+
+    if table_width > term_width:
+        header = f"  {DIM_CYAN}\u2192 Result ({row_count} rows) \u2014 too wide to print in terminal, download below{RESET}"
+        return f"{header}{download_line}"
+
+    def fmt_row(vals: list[Any]) -> str:
+        cells = []
+        for v, w in zip(vals, widths, strict=False):
+            cells.append(str(v if v is not None else "NULL").ljust(w))
+        return "    | " + " | ".join(cells) + " |"
+
+    lines = [fmt_row(columns)]
+    lines.append("    |" + "|".join("-" * (w + 2) for w in widths) + "|")
+    for row in rows:
+        lines.append(fmt_row(row))
+
+    header = f"  {DIM_CYAN}\u2192 Result ({', '.join(parts)})"
+    table = "\n".join(lines)
+    return f"{header}\n{table}{RESET}{download_line}"
+
+
+# ---------------------------------------------------------------------------
+# Multi-turn REPL using Runner.run_streamed()
+# ---------------------------------------------------------------------------
+
+
+async def run_turn(
+    agent: SandboxAgent,
+    conversation: list[Any],
+    question: str,
+    run_config: RunConfig,
+) -> list[Any]:
+    """Run one conversational turn and return the updated conversation history."""
+    input_items = conversation + [{"role": "user", "content": question}]
+
+    result = Runner.run_streamed(agent, input_items, run_config=run_config)
+
+    async for event in result.stream_events():
+        if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
+            print(event.data.delta, end="", flush=True)
+            continue
+
+        if event.type != "run_item_stream_event":
+            continue
+
+        if event.name == "tool_called":
+            item = event.item
+            raw = getattr(item, "raw_item", None)
+            if raw is not None:
+                name = getattr(raw, "name", "")
+                arguments = getattr(raw, "arguments", "")
+                print()
+                print(_format_tool_args(name, arguments))
+            continue
+
+        if event.name == "tool_output":
+            item = event.item
+            output = getattr(item, "output", "")
+            if isinstance(output, str):
+                formatted = _format_tool_result(output)
+                if formatted is not None:
+                    print(formatted)
+            print()
+            continue
+
+    print()
+
+    # Build the full conversation history for the next turn using the SDK's
+    # built-in conversion, which correctly serializes all item types.
+    return result.to_input_list()
+
+
+# ---------------------------------------------------------------------------
+# Session state persistence for pause/resume
+# ---------------------------------------------------------------------------
+
+
+def _load_session_state() -> DaytonaSandboxSessionState | None:
+    """Load saved session state from disk, or return None."""
+    if not SESSION_STATE_PATH.exists():
+        return None
+    try:
+        return DaytonaSandboxSessionState.model_validate_json(SESSION_STATE_PATH.read_text())
+    except Exception:
+        return None
+
+
+def _save_session_state(state: DaytonaSandboxSessionState) -> None:
+    """Persist session state to disk so the sandbox can be reused next run."""
+    SESSION_STATE_PATH.write_text(state.model_dump_json(indent=2))
+
+
+def _require_env(name: str) -> None:
+    """Exit early with a clear message when a required environment variable is missing."""
+    if os.environ.get(name):
+        return
+    raise SystemExit(f"{name} must be set before running this example.")
+
+
+def _status(message: str) -> None:
+    """Print progress immediately so automation logs show where startup is blocked."""
+    print(message, flush=True)
+
+
+# ---------------------------------------------------------------------------
+# Main entrypoint
+# ---------------------------------------------------------------------------
+
+
+async def main() -> None:
+    _status("Starting Daytona NASA spending text-to-SQL example...")
+    _require_env("OPENAI_API_KEY")
+    _require_env("DAYTONA_API_KEY")
+
+    agent = build_agent()
+
+    instrumentation = Instrumentation(
+        sinks=[JsonlOutboxSink(AUDIT_LOG_PATH)],
+        payload_policy=EventPayloadPolicy(include_exec_output=True),
+    )
+    RESULTS_PORT = 8080
+
+    _status("Creating Daytona sandbox client...")
+    client = DaytonaSandboxClient(instrumentation=instrumentation)
+    client_options = DaytonaSandboxClientOptions(
+        pause_on_exit=True,
+        exposed_ports=(RESULTS_PORT,),
+    )
+
+    # Try to resume a previously paused sandbox.
+    saved_state = _load_session_state()
+    sandbox = None
+    destroy = False
+
+    try:
+        if saved_state is not None:
+            old_sandbox_id = saved_state.sandbox_id
+            try:
+                _status(f"Resuming Daytona sandbox {old_sandbox_id}...")
+                sandbox = await client.resume(saved_state)
+                assert isinstance(sandbox.state, DaytonaSandboxSessionState)
+                if sandbox.state.sandbox_id == old_sandbox_id:
+                    _status("Reconnected to existing sandbox.")
+                else:
+                    _status("Previous sandbox no longer exists. Created a new one.")
+            except Exception as e:
+                _status(f"Could not resume previous sandbox: {e}")
+                saved_state = None
+                sandbox = None
+
+        if sandbox is None:
+            _status("Creating Daytona sandbox...")
+            sandbox = await client.create(manifest=agent.default_manifest, options=client_options)
+
+        _status("Starting Daytona sandbox...")
+        await sandbox.start()
+
+        # Persist state immediately so crashes don't orphan the sandbox.
+        assert isinstance(sandbox.state, DaytonaSandboxSessionState)
+        _save_session_state(sandbox.state)
+
+        # Build database inside sandbox (idempotent — skips if DB already exists).
+        _status("Setting up database (may take a few minutes on first run)...")
+        result = await sandbox.exec("python3", "setup_db.py", timeout=1800.0)
+        stdout = result.stdout.decode("utf-8", errors="replace")
+        if stdout.strip():
+            print(stdout)
+        if not result.ok():
+            stderr = result.stderr.decode("utf-8", errors="replace")
+            print(f"Database setup failed:\n{stderr}", file=sys.stderr)
+            sys.exit(1)
+
+        # Start a file server in the sandbox so query results can be downloaded.
+        _status("Starting results file server...")
+        await sandbox.exec("mkdir -p results", timeout=5.0)
+        await sandbox.exec(
+            f"nohup python3 -m http.server {RESULTS_PORT} --directory results > /dev/null 2>&1 &",
+            timeout=5.0,
+        )
+
+        # Resolve the Daytona signed URL for the file server.
+        global _downloads_base_url
+        try:
+            endpoint = await sandbox.resolve_exposed_port(RESULTS_PORT)
+            _downloads_base_url = endpoint.url_for("http")
+        except Exception as e:
+            print(f"  Warning: could not resolve download URL: {e}")
+
+        run_config = RunConfig(
+            sandbox=SandboxRunConfig(session=sandbox),
+            workflow_name="NASA Spending Q&A",
+        )
+
+        downloads_line = ""
+        if _downloads_base_url:
+            downloads_line = f"\n  Browse results: {DIM_CYAN}{_downloads_base_url}{RESET}"
+
+        print(f"""
+{DIM}{"=" * 60}{RESET}
+  NASA Spending Q&A (FY2021\u2013FY2025)
+
+  Data from USAspending.gov \u2014 contracts, grants, and IDVs
+  awarded by NASA. Each row is a transaction (obligation).
+
+  Includes: amounts, award descriptions, recipients, recipient
+  locations, places of performance, industry and product
+  categories, sub-agencies, and fiscal years.
+{downloads_line}
+  Type {DIM_CYAN}'exit'{RESET} to pause sandbox, {DIM_CYAN}'destroy'{RESET} to delete it.
+{DIM}{"=" * 60}{RESET}
+""")
+
+        conversation: list[Any] = []
+        auto_mode = is_auto_mode()
+
+        while True:
+            try:
+                if auto_mode:
+                    question = input_with_fallback("> ", DEFAULT_AUTO_QUESTION)
+                else:
+                    question = input("> ")
+            except (EOFError, KeyboardInterrupt):
+                print()
+                break
+
+            cmd = question.strip().lower()
+            if cmd == "exit":
+                break
+            if cmd == "destroy":
+                destroy = True
+                break
+
+            if not question.strip():
+                continue
+
+            try:
+                conversation = await run_turn(agent, conversation, question, run_config)
+            except Exception as e:
+                print(f"\nError: {e}")
+            print()
+
+            if auto_mode:
+                break
+
+        if destroy:
+            assert isinstance(sandbox.state, DaytonaSandboxSessionState)
+            sandbox.state.pause_on_exit = False
+            SESSION_STATE_PATH.unlink(missing_ok=True)
+            _status("Deleting sandbox...")
+        else:
+            assert isinstance(sandbox.state, DaytonaSandboxSessionState)
+            _save_session_state(sandbox.state)
+            _status("Saving memory and pausing sandbox (can take a couple of minutes)...")
+
+    finally:
+        if sandbox is not None:
+            if destroy:
+                # Skip memory flush — sandbox is being deleted.
+                await sandbox.stop()
+                await sandbox.shutdown()
+            else:
+                await sandbox.aclose()
+        await client.close()
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

examples/sandbox/extensions/daytona/usaspending_text2sql/schema/overview.md

@@ -0,0 +1,60 @@
+## Database: usaspending.db
+
+NASA federal spending data from USAspending.gov. Each row is a single spending transaction (obligation or de-obligation) on a federal award.
+
+### Table: spending
+
+One row per transaction. Multiple transactions can share the same `award_id` (an award's initial obligation plus subsequent modifications, amendments, and de-obligations).
+
+**Key columns:**
+- `award_id` — unique award identifier (many transactions share one award_id)
+- `award_piid_fain` — human-readable contract number (PIID) or assistance award number (FAIN)
+- `parent_award_piid` — parent IDV contract number (links task orders to their contract vehicle; contracts only)
+- `award_type` — 'contract', 'grant', 'idv', or 'other'
+- `action_date` — date of this transaction (YYYY-MM-DD)
+- `fiscal_year` — federal fiscal year (Oct-Sep; FY2024 = Oct 2023 - Sep 2024)
+- `federal_action_obligation` — dollar amount of this transaction (can be negative for de-obligations)
+- `total_obligation` — cumulative obligation for the entire award at time of this transaction
+- `base_and_all_options_value` — total potential ceiling value including unexercised options (contracts only)
+- `recipient_name` — who received the funds
+- `recipient_parent_name` — parent company (e.g., subsidiaries roll up; contracts only)
+- `recipient_state`, `recipient_city`, `recipient_country` — recipient location
+- `awarding_office` — NASA center/office that made the award (e.g., 'GODDARD SPACE FLIGHT CENTER', 'JET PROPULSION LABORATORY')
+- `funding_office` — NASA center/office providing funding (often same as awarding)
+- `naics_code`, `naics_description` — industry classification (primarily for contracts)
+- `psc_code`, `psc_description` — product/service classification
+- `place_of_performance_state`, `place_of_performance_city` — where work is performed
+- `period_of_perf_start`, `period_of_perf_end` — award period of performance dates (YYYY-MM-DD)
+- `extent_competed` — competition level: 'Full and Open Competition', 'Not Competed', etc. (contracts only)
+- `type_of_set_aside` — small business set-aside type: '8(a)', 'HUBZone', 'SDVOSB', etc. (contracts only)
+- `number_of_offers` — number of offers received (contracts only)
+- `contract_pricing_type` — pricing structure: 'Firm Fixed Price', 'Cost Plus', etc. (contracts only)
+- `business_types` — recipient type for assistance: nonprofit, university, state govt, etc. (grants only)
+- `description` — free-text description of the transaction
+
+### Common query patterns
+
+```sql
+-- Total spending by fiscal year
+SELECT fiscal_year, SUM(federal_action_obligation) AS total
+FROM spending GROUP BY fiscal_year ORDER BY fiscal_year;
+
+-- Top recipients (roll up by parent company)
+SELECT COALESCE(NULLIF(recipient_parent_name, ''), recipient_name) AS entity,
+       SUM(federal_action_obligation) AS total
+FROM spending GROUP BY entity ORDER BY total DESC LIMIT 10;
+
+-- Spending by award type
+SELECT award_type, COUNT(*), SUM(federal_action_obligation) AS total
+FROM spending GROUP BY award_type;
+
+-- Competitive vs sole-source contracts
+SELECT extent_competed, COUNT(DISTINCT award_id) AS awards,
+       SUM(federal_action_obligation) AS total
+FROM spending WHERE award_type = 'contract'
+GROUP BY extent_competed ORDER BY total DESC;
+
+-- Spending by NASA center
+SELECT awarding_office, SUM(federal_action_obligation) AS total
+FROM spending GROUP BY awarding_office ORDER BY total DESC;
+```

examples/sandbox/extensions/daytona/usaspending_text2sql/schema/tables/spending.md

@@ -0,0 +1,52 @@
+# spending
+
+One row per prime award transaction from NASA. Each row represents a financial action — an initial obligation, modification, amendment, or de-obligation on a federal award.
+
+## Columns
+
+| Column | Type | Description |
+|--------|------|-------------|
+| rowid | INTEGER PK | Auto-increment row identifier |
+| award_id | TEXT | Unique award identifier. Multiple rows share the same award_id when an award has multiple transactions |
+| award_piid_fain | TEXT | Human-readable award number: PIID for contracts (e.g., 'NNJ13ZBG001'), FAIN for assistance |
+| parent_award_piid | TEXT | Parent IDV contract number. Links task/delivery orders to their parent contract vehicle (contracts only) |
+| award_type | TEXT | Category: 'contract', 'grant', 'idv', or 'other' |
+| description | TEXT | Free-text description of the transaction or award purpose |
+| action_date | TEXT | Date of this transaction (ISO 8601: YYYY-MM-DD) |
+| fiscal_year | INTEGER | Federal fiscal year (Oct-Sep; FY2024 = Oct 2023 - Sep 2024) |
+| federal_action_obligation | REAL | Dollar amount of this specific transaction. Can be negative for de-obligations |
+| total_obligation | REAL | Cumulative obligation for the entire award at the time of this transaction |
+| base_and_all_options_value | REAL | Total potential ceiling value of the contract including all unexercised options. Contracts only; NULL for grants |
+| recipient_name | TEXT | Legal name of the recipient organization |
+| recipient_parent_name | TEXT | Parent company name (e.g., subsidiaries like 'Lockheed Martin Space' roll up to 'Lockheed Martin Corporation'). Contracts only; empty for grants |
+| recipient_state | TEXT | Two-letter US state code of recipient's address. Empty for foreign recipients |
+| recipient_city | TEXT | City of recipient's address |
+| recipient_country | TEXT | Country name (e.g., 'UNITED STATES', 'UNITED KINGDOM') |
+| awarding_office | TEXT | NASA center/office that made the award (e.g., 'GODDARD SPACE FLIGHT CENTER', 'JET PROPULSION LABORATORY'). Values are uppercase |
+| funding_office | TEXT | NASA center/office providing funding (often same as awarding). Values are uppercase |
+| naics_code | TEXT | North American Industry Classification System code. Primarily for contracts; may be empty for grants |
+| naics_description | TEXT | Human-readable NAICS description |
+| psc_code | TEXT | Product/Service Code for contracts, CFDA number for assistance. Different classification systems in the same column |
+| psc_description | TEXT | Human-readable description of the PSC (contracts) or CFDA program (assistance) |
+| place_of_performance_state | TEXT | State where work is performed. Two-letter codes for contracts, full names for assistance. May differ from recipient_state |
+| place_of_performance_city | TEXT | City where work is performed |
+| period_of_perf_start | TEXT | Award period of performance start date (YYYY-MM-DD) |
+| period_of_perf_end | TEXT | Award period of performance end date (YYYY-MM-DD). This is the current end date and may reflect extensions |
+| extent_competed | TEXT | Competition level. Values include 'Full and Open Competition', 'Not Available for Competition', 'Not Competed', etc. Contracts only; empty for grants |
+| type_of_set_aside | TEXT | Small business set-aside type. Values include 'Small Business Set-Aside', '8(a) Set-Aside', 'HUBZone Set-Aside', 'Service-Disabled Veteran-Owned Small Business Set-Aside', 'Women-Owned Small Business', etc. Contracts only |
+| number_of_offers | INTEGER | Number of offers/bids received. 1 = effectively sole-source even if technically competed. Contracts only; NULL for grants |
+| contract_pricing_type | TEXT | Pricing structure: 'Firm Fixed Price', 'Cost Plus Fixed Fee', 'Cost No Fee', 'Time and Materials', etc. Contracts only |
+| business_types | TEXT | Recipient organization type for assistance awards: nonprofit, university, state government, tribal, etc. Grants only; empty for contracts |
+
+## Notes
+
+- **Aggregating to award level**: use `GROUP BY award_id` with `SUM(federal_action_obligation)` to get total spending per award. The `total_obligation` column is a snapshot at each transaction and may not reflect the final total.
+- **Contract ceiling vs obligation**: `base_and_all_options_value` is the potential maximum; `total_obligation` is what's actually committed. A contract may have $10M obligated against a $500M ceiling.
+- **Parent company roll-up**: Use `COALESCE(NULLIF(recipient_parent_name, ''), recipient_name)` to group subsidiaries under their parent. Only populated for contracts.
+- **recipient_name** may vary slightly for the same entity across rows (e.g., 'BOEING CO' vs 'THE BOEING COMPANY'). Use `LIKE` or `UPPER()` for fuzzy matching.
+- **award_type** is derived from USAspending type codes: A/B/C/D -> 'contract', 02-05 -> 'grant', IDV_* -> 'idv'.
+- **federal_action_obligation** can be negative (de-obligations, corrections). Sum them to get net spending.
+- **naics_code** and **naics_description** are only populated for contracts; empty for grants/assistance.
+- **psc_code** contains Product/Service Codes for contracts and CFDA numbers for assistance awards. **psc_description** contains the corresponding description. These are different classification systems stored in the same column.
+- **Contracts-only columns**: `base_and_all_options_value`, `recipient_parent_name`, `parent_award_piid`, `extent_competed`, `type_of_set_aside`, `number_of_offers`, `contract_pricing_type` are only populated for contracts/IDVs.
+- **Grants-only columns**: `business_types` is only populated for assistance awards.

examples/sandbox/extensions/daytona/usaspending_text2sql/setup_db.py

@@ -0,0 +1,718 @@
+#!/usr/bin/env python3
+"""Download NASA spending data from USAspending.gov and build a SQLite database.
+
+This script is designed to run inside a sandbox environment with only Python
+stdlib available. It fetches data via the USAspending bulk download API,
+parses the resulting CSVs, and creates a local SQLite database.
+
+Usage:
+    python setup_db.py [--force] [--start-fy 2021] [--end-fy 2025]
+
+The script is idempotent: it skips the download/build if the database already
+exists unless --force is passed.
+"""
+
+from __future__ import annotations
+
+import argparse
+import concurrent.futures
+import csv
+import functools
+import json
+import os
+import sqlite3
+import ssl
+import sys
+import time
+import urllib.error
+import urllib.request
+import zipfile
+from pathlib import Path
+from typing import Any
+
+ARTIFACT_ROOT = Path(os.environ.get("EXAMPLES_ARTIFACTS_DIR", "."))
+DB_DIR = ARTIFACT_ROOT / "data"
+DB_PATH = DB_DIR / "usaspending.db"
+GLOSSARY_PATH = ARTIFACT_ROOT / "schema" / "glossary.md"
+
+USASPENDING_API = "https://api.usaspending.gov"
+BULK_DOWNLOAD_ENDPOINT = f"{USASPENDING_API}/api/v2/bulk_download/awards/"
+DOWNLOAD_STATUS_ENDPOINT = f"{USASPENDING_API}/api/v2/download/status"
+GLOSSARY_ENDPOINT = f"{USASPENDING_API}/api/v2/references/glossary/"
+
+NASA_AGENCY = {
+    "type": "awarding",
+    "tier": "toptier",
+    "name": "National Aeronautics and Space Administration",
+}
+
+# Award type codes per the USAspending API contract.
+CONTRACT_CODES = ["A", "B", "C", "D"]
+GRANT_CODES = ["02", "03", "04", "05"]
+IDV_CODES = ["IDV_A", "IDV_B", "IDV_B_A", "IDV_B_B", "IDV_B_C", "IDV_C", "IDV_D", "IDV_E"]
+ALL_AWARD_CODES = CONTRACT_CODES + GRANT_CODES + IDV_CODES
+
+AWARD_TYPE_MAP: dict[str, str] = {}
+for _code in CONTRACT_CODES:
+    AWARD_TYPE_MAP[_code] = "contract"
+for _code in GRANT_CODES:
+    AWARD_TYPE_MAP[_code] = "grant"
+for _code in IDV_CODES:
+    AWARD_TYPE_MAP[_code] = "idv"
+
+# Common headers — the USAspending WAF rejects requests without a User-Agent.
+_HEADERS = {
+    "Content-Type": "application/json",
+    "User-Agent": "USAspending-setup/1.0 (universal_computer example)",
+    "Accept": "application/json",
+}
+
+SCHEMA_SQL = """
+CREATE TABLE IF NOT EXISTS spending (
+    rowid INTEGER PRIMARY KEY AUTOINCREMENT,
+    award_id TEXT,
+    award_piid_fain TEXT,
+    parent_award_piid TEXT,
+    award_type TEXT,
+    description TEXT,
+    action_date TEXT,
+    fiscal_year INTEGER,
+    federal_action_obligation REAL,
+    total_obligation REAL,
+    base_and_all_options_value REAL,
+    recipient_name TEXT,
+    recipient_parent_name TEXT,
+    recipient_state TEXT,
+    recipient_city TEXT,
+    recipient_country TEXT,
+    awarding_office TEXT,
+    funding_office TEXT,
+    naics_code TEXT,
+    naics_description TEXT,
+    psc_code TEXT,
+    psc_description TEXT,
+    place_of_performance_state TEXT,
+    place_of_performance_city TEXT,
+    period_of_perf_start TEXT,
+    period_of_perf_end TEXT,
+    extent_competed TEXT,
+    type_of_set_aside TEXT,
+    number_of_offers INTEGER,
+    contract_pricing_type TEXT,
+    business_types TEXT
+);
+
+CREATE INDEX IF NOT EXISTS idx_spending_award_id ON spending(award_id);
+CREATE INDEX IF NOT EXISTS idx_spending_fiscal_year ON spending(fiscal_year);
+CREATE INDEX IF NOT EXISTS idx_spending_award_type ON spending(award_type);
+CREATE INDEX IF NOT EXISTS idx_spending_recipient ON spending(recipient_name);
+CREATE INDEX IF NOT EXISTS idx_spending_recipient_parent ON spending(recipient_parent_name);
+CREATE INDEX IF NOT EXISTS idx_spending_state ON spending(recipient_state);
+CREATE INDEX IF NOT EXISTS idx_spending_action_date ON spending(action_date);
+CREATE INDEX IF NOT EXISTS idx_spending_naics ON spending(naics_code);
+CREATE INDEX IF NOT EXISTS idx_spending_obligation ON spending(federal_action_obligation);
+CREATE INDEX IF NOT EXISTS idx_spending_extent_competed ON spending(extent_competed);
+CREATE INDEX IF NOT EXISTS idx_spending_perf_start ON spending(period_of_perf_start);
+CREATE INDEX IF NOT EXISTS idx_spending_awarding_office ON spending(awarding_office);
+"""
+
+
+# ---------------------------------------------------------------------------
+# HTTP helpers
+# ---------------------------------------------------------------------------
+
+
+@functools.cache
+def _urlopen_ssl_context() -> ssl.SSLContext | None:
+    """Use certifi's CA bundle when available, otherwise keep stdlib defaults."""
+    try:
+        import certifi
+    except ImportError:
+        return None
+
+    return ssl.create_default_context(cafile=certifi.where())
+
+
+def _urlopen_with_retry(
+    req: urllib.request.Request, *, timeout: int = 60, retries: int = 3
+) -> bytes:
+    """urlopen with retries for the flaky USAspending endpoints."""
+    last_exc: Exception | None = None
+    ssl_context = _urlopen_ssl_context()
+    for attempt in range(1, retries + 1):
+        try:
+            with urllib.request.urlopen(req, timeout=timeout, context=ssl_context) as resp:
+                return bytes(resp.read())
+        except (urllib.error.URLError, ConnectionError, OSError) as e:
+            last_exc = e
+            if attempt < retries:
+                wait = 2**attempt
+                print(f"    Retry {attempt}/{retries} after error: {e} (waiting {wait}s)")
+                time.sleep(wait)
+    raise RuntimeError(f"Request failed after {retries} attempts: {last_exc}") from last_exc
+
+
+def api_post(url: str, payload: dict[str, Any]) -> dict[str, Any]:
+    """POST JSON to a USAspending API endpoint and return the parsed response."""
+    data = json.dumps(payload).encode("utf-8")
+    req = urllib.request.Request(url, data=data, headers=_HEADERS, method="POST")
+    body = _urlopen_with_retry(req)
+    return json.loads(body.decode("utf-8"))  # type: ignore[no-any-return]
+
+
+def api_get(url: str) -> dict[str, Any]:
+    """GET a USAspending API endpoint and return the parsed response."""
+    req = urllib.request.Request(url, headers=_HEADERS)
+    body = _urlopen_with_retry(req)
+    return json.loads(body.decode("utf-8"))  # type: ignore[no-any-return]
+
+
+# ---------------------------------------------------------------------------
+# Bulk download
+# ---------------------------------------------------------------------------
+
+
+def submit_bulk_download(
+    award_types: list[str],
+    start_date: str,
+    end_date: str,
+) -> tuple[str | None, str | None]:
+    """Submit a bulk download request and return (status_url, file_url).
+
+    The USAspending bulk download API requires:
+    - filters.agencies: list of agency objects (name/tier/type)
+    - filters.prime_award_types: list of award type codes
+    - filters.date_type: "action_date" or "last_modified_date"
+    - filters.date_range: {start_date, end_date} (max 1 year span)
+
+    This only submits the request — call poll_download_status() to wait for completion.
+    """
+    payload = {
+        "filters": {
+            "agencies": [NASA_AGENCY],
+            "prime_award_types": award_types,
+            "date_type": "action_date",
+            "date_range": {
+                "start_date": start_date,
+                "end_date": end_date,
+            },
+        },
+        "file_format": "csv",
+    }
+
+    resp = api_post(BULK_DOWNLOAD_ENDPOINT, payload)
+    file_url = resp.get("file_url")
+    status_url = resp.get("status_url")
+
+    if not status_url and not file_url:
+        raise RuntimeError(f"Unexpected API response: {resp}")
+
+    return status_url, file_url
+
+
+def poll_download_status(status_url: str | None, file_url: str | None) -> str:
+    """Poll the download status endpoint until the file is ready."""
+    if not status_url:
+        if file_url:
+            return file_url
+        raise RuntimeError("No status_url or file_url to poll")
+
+    for attempt in range(120):
+        try:
+            status = api_get(status_url)
+        except Exception:
+            time.sleep(5)
+            continue
+
+        state = status.get("status", "unknown")
+        if state == "finished":
+            return status.get("file_url") or file_url or ""
+        elif state == "failed":
+            raise RuntimeError(f"Download generation failed: {status.get('message', 'unknown')}")
+
+        if attempt % 6 == 0:
+            print(f"    Generating... (status: {state})")
+        time.sleep(5)
+
+    raise RuntimeError("Timed out waiting for download (10 minutes)")
+
+
+def download_and_extract(file_url: str, extract_dir: Path) -> list[Path]:
+    """Download a zip file and extract CSVs to extract_dir."""
+    extract_dir.mkdir(parents=True, exist_ok=True)
+    zip_path = extract_dir / "download.zip"
+
+    print("  Downloading...")
+    req = urllib.request.Request(file_url, headers={"User-Agent": _HEADERS["User-Agent"]})
+    data = _urlopen_with_retry(req, timeout=300, retries=3)
+    zip_path.write_bytes(data)
+    file_size_mb = len(data) / (1024 * 1024)
+    print(f"  Downloaded {file_size_mb:.1f} MB")
+
+    print("  Extracting CSV files...")
+    csv_files = []
+    with zipfile.ZipFile(zip_path, "r") as zf:
+        for name in zf.namelist():
+            if name.endswith(".csv"):
+                zf.extract(name, extract_dir)
+                csv_files.append(extract_dir / name)
+                print(f"    {name}")
+
+    zip_path.unlink()
+    return csv_files
+
+
+# ---------------------------------------------------------------------------
+# CSV ingestion
+# ---------------------------------------------------------------------------
+
+
+def safe_float(val: str) -> float | None:
+    if not val or val.strip() == "":
+        return None
+    try:
+        return float(val.replace(",", ""))
+    except ValueError:
+        return None
+
+
+def safe_int(val: str) -> int | None:
+    if not val or val.strip() == "":
+        return None
+    try:
+        return int(val.strip())
+    except ValueError:
+        return None
+
+
+def classify_award_type(type_code: str, award_id: str) -> str:
+    mapped = AWARD_TYPE_MAP.get(type_code)
+    if mapped:
+        return mapped
+    # Fallback: detect IDVs from the award_id prefix when the type code
+    # doesn't match our expected IDV codes.
+    if award_id.startswith("CONT_IDV_"):
+        return "idv"
+    return "other"
+
+
+def _detect_csv_type(headers: set[str]) -> str:
+    """Detect whether a CSV is contracts or assistance based on its headers.
+
+    Per the USAspending data dictionary, PrimeAwardUniqueKey is stored as
+    'contract_award_unique_key' in contracts and 'assistance_award_unique_key'
+    in assistance.
+    """
+    if "contract_award_unique_key" in headers:
+        return "contracts"
+    if "assistance_award_unique_key" in headers:
+        return "assistance"
+    raise ValueError(
+        "Cannot detect CSV type: neither 'contract_award_unique_key' nor "
+        "'assistance_award_unique_key' found in headers"
+    )
+
+
+# Column mappings per CSV type, derived from the USAspending data dictionary
+# (https://api.usaspending.gov/api/v2/references/data_dictionary/).
+#
+# "shared" columns have the same name in both contracts and assistance CSVs.
+# Type-specific columns are listed under "contracts" and "assistance".
+
+# Column mappings verified against actual CSV headers downloaded from USAspending
+# on 2026-03-26, and cross-referenced with the data dictionary API at
+# https://api.usaspending.gov/api/v2/references/data_dictionary/.
+#
+# "shared" columns have the same name in both contracts and assistance CSVs.
+# Type-specific columns differ between the two and are listed separately.
+
+_SHARED_COLUMNS = {
+    # db_column                    -> csv_column
+    "action_date": "action_date",
+    "fiscal_year": "action_date_fiscal_year",
+    "federal_action_obligation": "federal_action_obligation",
+    "recipient_name": "recipient_name",
+    "recipient_state": "recipient_state_code",
+    "recipient_city": "recipient_city_name",
+    "recipient_country": "recipient_country_name",
+    "awarding_office": "awarding_office_name",
+    "funding_office": "funding_office_name",
+    "description": "transaction_description",
+    "place_of_performance_city": "primary_place_of_performance_city_name",
+    "period_of_perf_start": "period_of_performance_start_date",
+    "period_of_perf_end": "period_of_performance_current_end_date",
+}
+
+_TYPE_COLUMNS: dict[str, dict[str, str]] = {
+    "contracts": {
+        "award_id": "contract_award_unique_key",
+        "award_piid_fain": "award_id_piid",
+        "parent_award_piid": "parent_award_id_piid",
+        "award_type_code": "award_type_code",
+        "total_obligation": "total_dollars_obligated",
+        "base_and_all_options_value": "base_and_all_options_value",
+        "recipient_parent_name": "recipient_parent_name",
+        "place_of_performance_state": "primary_place_of_performance_state_code",
+        "naics_code": "naics_code",
+        "naics_description": "naics_description",
+        "psc_code": "product_or_service_code",
+        "psc_description": "product_or_service_code_description",
+        "extent_competed": "extent_competed",
+        "type_of_set_aside": "type_of_set_aside",
+        "number_of_offers": "number_of_offers_received",
+        "contract_pricing_type": "type_of_contract_pricing",
+        "business_types": "",  # not present in contracts CSVs
+    },
+    "assistance": {
+        "award_id": "assistance_award_unique_key",
+        "award_piid_fain": "award_id_fain",
+        "parent_award_piid": "",  # not applicable to assistance
+        "award_type_code": "assistance_type_code",
+        "total_obligation": "total_obligated_amount",
+        "base_and_all_options_value": "",  # contracts only
+        "recipient_parent_name": "",  # contracts only
+        "place_of_performance_state": "primary_place_of_performance_state_name",
+        "naics_code": "",  # not present in assistance CSVs
+        "naics_description": "",
+        "psc_code": "cfda_number",
+        "psc_description": "cfda_title",
+        "extent_competed": "",  # contracts only
+        "type_of_set_aside": "",  # contracts only
+        "number_of_offers": "",  # contracts only
+        "contract_pricing_type": "",  # contracts only
+        "business_types": "business_types_description",
+    },
+}
+
+
+def ingest_csv(db: sqlite3.Connection, csv_path: Path) -> int:
+    """Ingest a USAspending prime transactions CSV into the spending table."""
+    count = 0
+
+    with open(csv_path, encoding="utf-8", errors="replace") as f:
+        reader = csv.DictReader(f)
+        if reader.fieldnames is None:
+            return 0
+
+        headers = set(reader.fieldnames)
+        csv_type = _detect_csv_type(headers)
+        type_cols = _TYPE_COLUMNS[csv_type]
+
+        # Verify expected columns exist
+        all_expected = dict(_SHARED_COLUMNS)
+        all_expected.update(type_cols)
+        missing = [
+            db_col for db_col, csv_col in all_expected.items() if csv_col and csv_col not in headers
+        ]
+        if missing:
+            print(f"    Warning: missing expected columns: {missing}")
+
+        award_id_col = type_cols["award_id"]
+        award_type_col = type_cols["award_type_code"]
+
+        for row in reader:
+            award_id = row.get(award_id_col, "")
+            if not award_id:
+                continue
+
+            type_code = row.get(award_type_col, "")
+            award_type = classify_award_type(type_code, award_id)
+
+            def col(db_name: str, _row: dict[str, str] = row) -> str:
+                """Look up a value: type-specific columns first, then shared."""
+                csv_col = type_cols.get(db_name) or _SHARED_COLUMNS.get(db_name, "")
+                return _row.get(csv_col, "") if csv_col else ""
+
+            db.execute(
+                """INSERT INTO spending
+                   (award_id, award_piid_fain, parent_award_piid,
+                    award_type, description, action_date, fiscal_year,
+                    federal_action_obligation, total_obligation, base_and_all_options_value,
+                    recipient_name, recipient_parent_name,
+                    recipient_state, recipient_city, recipient_country,
+                    awarding_office, funding_office,
+                    naics_code, naics_description, psc_code, psc_description,
+                    place_of_performance_state, place_of_performance_city,
+                    period_of_perf_start, period_of_perf_end,
+                    extent_competed, type_of_set_aside, number_of_offers,
+                    contract_pricing_type, business_types)
+                   VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
+                (
+                    award_id,
+                    col("award_piid_fain"),
+                    col("parent_award_piid"),
+                    award_type,
+                    col("description"),
+                    col("action_date"),
+                    safe_int(col("fiscal_year")),
+                    safe_float(col("federal_action_obligation")),
+                    safe_float(col("total_obligation")),
+                    safe_float(col("base_and_all_options_value")),
+                    col("recipient_name"),
+                    col("recipient_parent_name"),
+                    col("recipient_state"),
+                    col("recipient_city"),
+                    col("recipient_country"),
+                    col("awarding_office"),
+                    col("funding_office"),
+                    col("naics_code"),
+                    col("naics_description"),
+                    col("psc_code"),
+                    col("psc_description"),
+                    col("place_of_performance_state"),
+                    col("place_of_performance_city"),
+                    col("period_of_perf_start"),
+                    col("period_of_perf_end"),
+                    col("extent_competed"),
+                    col("type_of_set_aside"),
+                    safe_int(col("number_of_offers")),
+                    col("contract_pricing_type"),
+                    col("business_types"),
+                ),
+            )
+            count += 1
+
+    return count
+
+
+def build_database(csv_files: list[Path]) -> None:
+    """Build the SQLite database from extracted CSV files."""
+    DB_DIR.mkdir(parents=True, exist_ok=True)
+
+    print(f"Creating database at {DB_PATH}...")
+    db = sqlite3.connect(str(DB_PATH))
+    db.executescript(SCHEMA_SQL)
+
+    total = 0
+    for csv_path in csv_files:
+        print(f"  Ingesting {csv_path.name}...")
+        count = ingest_csv(db, csv_path)
+        total += count
+        print(f"    {count:,} rows")
+
+    db.commit()
+
+    cursor = db.execute("SELECT COUNT(*) FROM spending")
+    rows_stored = cursor.fetchone()[0]
+    cursor = db.execute("SELECT COUNT(DISTINCT award_id) FROM spending")
+    unique_awards = cursor.fetchone()[0]
+    db.close()
+
+    db_size_mb = DB_PATH.stat().st_size / (1024 * 1024)
+    print(f"\nDatabase built: {DB_PATH}")
+    print(f"  Rows:           {rows_stored:,}")
+    print(f"  Unique awards:  {unique_awards:,}")
+    print(f"  Size:           {db_size_mb:.1f} MB")
+
+
+# ---------------------------------------------------------------------------
+# Glossary
+# ---------------------------------------------------------------------------
+
+
+def fetch_glossary() -> None:
+    """Fetch the official USAspending glossary and write it to schema/glossary.md."""
+    if GLOSSARY_PATH.exists():
+        print(f"Glossary already exists at {GLOSSARY_PATH}, skipping.")
+        return
+
+    GLOSSARY_PATH.parent.mkdir(parents=True, exist_ok=True)
+
+    print("Fetching USAspending glossary...")
+    try:
+        resp = api_get(f"{GLOSSARY_ENDPOINT}?limit=500")
+    except Exception as e:
+        print(f"  Warning: failed to fetch glossary: {e}")
+        return
+
+    results = resp.get("results", [])
+    if not results:
+        print("  Warning: glossary API returned no results.")
+        return
+
+    results.sort(key=lambda t: t.get("term", "").lower())
+
+    lines = [
+        "# USAspending Glossary",
+        "",
+        "Official definitions from [USAspending.gov](https://www.usaspending.gov).",
+        f"Retrieved automatically by setup_db.py ({len(results)} terms).",
+        "",
+    ]
+
+    for entry in results:
+        term = entry.get("term", "").strip()
+        plain = (entry.get("plain") or "").strip()
+        official = (entry.get("official") or "").strip()
+
+        if not term:
+            continue
+
+        lines.append(f"## {term}")
+        lines.append("")
+        if plain:
+            lines.append(plain)
+            lines.append("")
+        if official and official != plain:
+            lines.append(f"**Official definition:** {official}")
+            lines.append("")
+
+    GLOSSARY_PATH.write_text("\n".join(lines), encoding="utf-8")
+    print(f"  Wrote {len(results)} glossary terms to {GLOSSARY_PATH}")
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+
+
+def fiscal_year_dates(fy: int) -> tuple[str, str]:
+    """Return (start_date, end_date) for a federal fiscal year.
+
+    Federal FY runs Oct 1 of the prior calendar year through Sep 30.
+    Example: FY2024 = 2023-10-01 to 2024-09-30.
+    """
+    return f"{fy - 1}-10-01", f"{fy}-09-30"
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Build NASA USAspending SQLite database")
+    parser.add_argument("--force", action="store_true", help="Rebuild even if database exists")
+    parser.add_argument(
+        "--start-fy", type=int, default=2021, help="First fiscal year to download (default: 2021)"
+    )
+    parser.add_argument(
+        "--end-fy", type=int, default=2025, help="Last fiscal year to download (default: 2025)"
+    )
+    args = parser.parse_args()
+
+    if args.start_fy > args.end_fy:
+        parser.error(f"--start-fy ({args.start_fy}) must be <= --end-fy ({args.end_fy})")
+
+    requested_fys = set(range(args.start_fy, args.end_fy + 1))
+
+    if DB_PATH.exists() and not args.force:
+        # Verify the existing DB covers all requested fiscal years.
+        try:
+            conn = sqlite3.connect(f"file:{DB_PATH}?mode=ro", uri=True)
+            rows = conn.execute("SELECT DISTINCT fiscal_year FROM spending").fetchall()
+            conn.close()
+            present_fys = {int(r[0]) for r in rows if r[0] is not None}
+            missing_fys = requested_fys - present_fys
+            if not missing_fys:
+                db_size_mb = DB_PATH.stat().st_size / (1024 * 1024)
+                print(
+                    f"Database already exists at {DB_PATH} ({db_size_mb:.1f} MB) "
+                    f"with all requested FYs. Use --force to rebuild."
+                )
+                return
+            print(
+                f"Database exists but is missing FY data for: "
+                f"{', '.join(str(fy) for fy in sorted(missing_fys))}. Rebuilding..."
+            )
+        except Exception:
+            print("Database exists but could not be verified. Rebuilding...")
+        DB_PATH.unlink()
+    elif DB_PATH.exists():
+        DB_PATH.unlink()
+
+    tmp_dir = DB_DIR / "tmp_download"
+
+    print("=== NASA USAspending Database Builder ===")
+    print(f"Fiscal years: {args.start_fy} - {args.end_fy}\n")
+
+    # The bulk download API limits date_range to 1 year, so we request
+    # one fiscal year at a time. We submit all requests upfront so the
+    # server-side assembly (the slow part) runs concurrently, then poll
+    # and download the results.
+    all_csv_files: list[Path] = []
+    failed_fys: list[int] = []
+    fiscal_years = list(range(args.start_fy, args.end_fy + 1))
+
+    # Phase 1: Submit all bulk download requests concurrently.
+    print("Submitting download requests...")
+    pending: dict[int, tuple[str | None, str | None]] = {}
+    with concurrent.futures.ThreadPoolExecutor(max_workers=len(fiscal_years)) as pool:
+
+        def _submit(fy: int) -> tuple[int, str | None, str | None]:
+            start_date, end_date = fiscal_year_dates(fy)
+            status_url, file_url = submit_bulk_download(
+                ALL_AWARD_CODES,
+                start_date,
+                end_date,
+            )
+            return fy, status_url, file_url
+
+        futures = {pool.submit(_submit, fy): fy for fy in fiscal_years}
+        for future in concurrent.futures.as_completed(futures):
+            fy = futures[future]
+            try:
+                _, status_url, file_url = future.result()
+                pending[fy] = (status_url, file_url)
+                print(f"  FY{fy}: submitted")
+            except Exception as e:
+                print(f"  FY{fy}: submit failed: {e}")
+                failed_fys.append(fy)
+
+    # Phase 2: Poll all pending requests until ready, then download.
+    for fy in sorted(pending):
+        print(f"\n--- FY{fy} ---")
+        status_url, file_url = pending[fy]
+        try:
+            file_url = poll_download_status(status_url, file_url)
+            print(f"  Ready: {file_url}")
+            fy_dir = tmp_dir / f"fy{fy}"
+            csv_files = download_and_extract(file_url, fy_dir)
+            all_csv_files.extend(csv_files)
+        except Exception as e:
+            print(f"  Error: failed FY{fy}: {e}")
+            failed_fys.append(fy)
+
+    if not all_csv_files:
+        print("\nError: no data downloaded. Check internet connectivity.")
+        sys.exit(1)
+
+    if failed_fys:
+        print(
+            f"\nError: failed to download data for: "
+            f"{', '.join(f'FY{fy}' for fy in failed_fys)}. "
+            f"Cannot build a complete database."
+        )
+        sys.exit(1)
+
+    print("\n--- Fetching glossary ---")
+    fetch_glossary()
+
+    print("\n--- Building database ---")
+    build_database(all_csv_files)
+
+    # Verify the built DB covers all requested fiscal years.
+    conn = sqlite3.connect(f"file:{DB_PATH}?mode=ro", uri=True)
+    rows = conn.execute("SELECT DISTINCT fiscal_year FROM spending").fetchall()
+    conn.close()
+    present_fys = {int(r[0]) for r in rows if r[0] is not None}
+    missing_fys = requested_fys - present_fys
+    if missing_fys:
+        print(
+            f"\nError: database built but missing data for: "
+            f"{', '.join(f'FY{fy}' for fy in sorted(missing_fys))}. "
+            f"Downloaded files may have been empty."
+        )
+        DB_PATH.unlink()
+        sys.exit(1)
+
+    # Clean up temp files
+    for f in tmp_dir.rglob("*"):
+        if f.is_file():
+            f.unlink()
+    for d in sorted(tmp_dir.rglob("*"), reverse=True):
+        if d.is_dir():
+            d.rmdir()
+    if tmp_dir.exists():
+        tmp_dir.rmdir()
+
+    print("\nDone!")
+
+
+if __name__ == "__main__":
+    main()

examples/sandbox/extensions/daytona/usaspending_text2sql/sql_capability.py

@@ -0,0 +1,175 @@
+from __future__ import annotations
+
+import textwrap
+from typing import Any, Literal
+
+from agents.sandbox import Capability, ExecTimeoutError, Manifest
+from agents.sandbox.session.base_sandbox_session import BaseSandboxSession
+from agents.tool import FunctionTool
+
+# Python script executed inside the sandbox to run SQL queries safely.
+# Receives the query on stdin, enforces read-only mode and row limits.
+_QUERY_RUNNER_SCRIPT = r"""
+import csv, json, os, sqlite3, sys, time
+
+db_path = sys.argv[1]
+display_limit = int(sys.argv[2])
+csv_limit = int(sys.argv[3])
+results_dir = sys.argv[4] if len(sys.argv) > 4 else ""
+
+query = sys.stdin.read().strip()
+if not query:
+    print("Error: empty query")
+    sys.exit(0)
+
+# Statement-level validation: only allow read-only operations
+first_token = query.lstrip().split()[0].upper() if query.strip() else ""
+if first_token not in ("SELECT", "WITH", "EXPLAIN", "PRAGMA"):
+    print(f"Error: only SELECT, WITH, EXPLAIN, and PRAGMA statements are allowed (got {first_token})")
+    sys.exit(0)
+
+try:
+    conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)
+    conn.execute("PRAGMA query_only = ON")
+    cursor = conn.execute(query)
+    columns = [desc[0] for desc in cursor.description] if cursor.description else []
+    rows = cursor.fetchmany(csv_limit + 1)
+    conn.close()
+except sqlite3.Error as e:
+    print(f"SQL error: {e}")
+    sys.exit(0)
+
+if not columns:
+    print(json.dumps({"columns": [], "rows": [], "row_count": 0, "truncated": False}))
+    sys.exit(0)
+
+csv_truncated = len(rows) > csv_limit
+if csv_truncated:
+    rows = rows[:csv_limit]
+
+# Save full result as CSV for download
+csv_file = ""
+if results_dir:
+    os.makedirs(results_dir, exist_ok=True)
+    csv_file = f"query_{int(time.time())}_{os.getpid()}.csv"
+    with open(os.path.join(results_dir, csv_file), "w", newline="") as f:
+        writer = csv.writer(f)
+        writer.writerow(columns)
+        writer.writerows(rows)
+
+# Return only display_limit rows to the model, but report total counts
+total_rows = len(rows)
+display_rows = rows[:display_limit]
+
+result = {
+    "columns": columns,
+    "rows": display_rows,
+    "row_count": total_rows,
+    "display_count": len(display_rows),
+    "truncated": csv_truncated,
+}
+if csv_file:
+    result["csv_file"] = csv_file
+    if total_rows > len(display_rows):
+        result["note"] = f"Showing {len(display_rows)} of {total_rows} rows. Full result saved to CSV."
+
+print(json.dumps(result))
+"""
+
+
+def _shell_quote(s: str) -> str:
+    """Single-quote a string for safe shell interpolation."""
+    return "'" + s.replace("'", "'\\''") + "'"
+
+
+_SQL_CAPABILITY_INSTRUCTIONS = textwrap.dedent(
+    """\
+    When querying the database:
+    - Always use `run_sql` to execute SQL. Never run sqlite3 directly via a shell.
+    - Write standard SQLite-compatible SQL.
+    - Prefer aggregations (GROUP BY, SUM, COUNT, AVG) over returning many raw rows.
+    - The display shows up to 100 rows, but up to 10,000 rows are saved to a downloadable CSV.
+      If the user needs a large export, let them know the full result is available via the download link.
+    - Use the schema documentation files in schema/tables/ if you need column details.
+    - Read schema/glossary.md for official definitions of USAspending terms.
+    - For monetary values, the database stores amounts in dollars as REAL values.
+    """
+).strip()
+
+
+def _make_run_sql_tool(
+    session: BaseSandboxSession,
+    db_path: str,
+    max_display_rows: int,
+    max_csv_rows: int,
+    timeout_seconds: float,
+    results_dir: str,
+) -> FunctionTool:
+    """Build a FunctionTool that executes read-only SQL inside the sandbox."""
+
+    async def run_sql(query: str, limit: int | None = None) -> str:
+        """Execute a read-only SQL query against the NASA USAspending SQLite database.
+
+        Returns results as JSON with columns, rows, row_count, and truncated fields.
+        Results are also saved as a downloadable CSV. The display is limited to a
+        small number of rows, but the CSV may contain many more.
+
+        Args:
+            query: SQL SELECT query to execute against the USAspending database.
+                Only read-only queries are allowed.
+            limit: Optional display row limit override.
+        """
+        display_limit = max(1, min(limit or max_display_rows, max_display_rows))
+
+        command = (
+            f"printf '%s' {_shell_quote(query)} "
+            f"| python3 -c {_shell_quote(_QUERY_RUNNER_SCRIPT)} "
+            f"{_shell_quote(db_path)} {display_limit} {max_csv_rows}"
+            f" {_shell_quote(results_dir)}"
+        )
+
+        try:
+            result = await session.exec(command, timeout=timeout_seconds)
+        except (ExecTimeoutError, TimeoutError):
+            return f"Query timed out after {timeout_seconds}s. Try a simpler query or add a LIMIT."
+
+        output = result.stdout.decode("utf-8", errors="replace")
+        stderr = result.stderr.decode("utf-8", errors="replace")
+
+        if not result.ok():
+            return f"Execution error (exit {result.exit_code}):\n{stderr or output}"
+
+        return output.strip() if output.strip() else "Query returned no results."
+
+    from agents.tool import function_tool as _function_tool
+
+    return _function_tool(run_sql, name_override="run_sql")
+
+
+class SqlCapability(Capability):
+    type: Literal["sql"] = "sql"
+    db_path: str = "data/usaspending.db"
+    max_display_rows: int = 100
+    max_csv_rows: int = 10_000
+    timeout_seconds: float = 30.0
+    results_dir: str = "results"
+
+    def bind(self, session: BaseSandboxSession) -> None:
+        self.session = session
+
+    def tools(self) -> list[Any]:
+        if self.session is None:
+            raise ValueError("SqlCapability is not bound to a SandboxSession")
+        return [
+            _make_run_sql_tool(
+                session=self.session,
+                db_path=self.db_path,
+                max_display_rows=self.max_display_rows,
+                max_csv_rows=self.max_csv_rows,
+                timeout_seconds=self.timeout_seconds,
+                results_dir=self.results_dir,
+            )
+        ]
+
+    async def instructions(self, manifest: Manifest) -> str | None:
+        return _SQL_CAPABILITY_INSTRUCTIONS