chore: import upstream snapshot with attribution

This commit is contained in:
wehub-resource-sync
2026-07-13 12:39:17 +08:00
commit 4ed4e9ff99
1368 changed files with 334957 additions and 0 deletions
+40
View File
@@ -0,0 +1,40 @@
# SDK Maintainer References
This directory captures long-lived implementation contracts of the OpenAI Agents Python SDK that are not replaceable by OpenAI API or platform facts from the Developer Docs MCP. The repo's `docs/` remain an SDK-specific behavioral contract; these references distill the ownership, compatibility, ordering, and failure semantics that maintainers need to preserve that contract.
## Usage
Read the reference map before changing or reviewing an affected runtime boundary, then open only the files relevant to that boundary. During issue and PR review, treat this directory as read-only background: use it to identify expected invariants, adjacent surfaces, and regression risks, but verify the current claim against the remote issue or PR, current code, tests, docs, release boundary, and focused runtime evidence. Do not edit references as a side effect of a review or treat them as proof of current issue status, PR behavior, or repository readiness.
When implementation or dedicated repository-maintenance work establishes a reusable invariant that remains valid beyond one issue or PR, update the narrowest owning reference separately. Preserve the generalized contract, not the case history or decision outcome that revealed it.
## Inclusion Criteria
Add or retain a reference when the knowledge is SDK-specific, stable across multiple releases, easy to violate from one local code path, and expensive to reconstruct from source, tests, and repo docs during every review. Treat `docs/` as the SDK's user-facing behavioral contract; use these references to preserve the implementation constraints behind that contract. Prefer invariants and ownership rules over summaries of individual issues, PRs, or recent fixes.
Do not store current issue or PR status, generic maintainer-review workflow, release notes, OpenAI API or platform behavior available through `$openai-knowledge`, or one-off implementation details in this directory. Put review methodology under `.agents/skills/`, released migration notes in `docs/release.md`, and API or platform facts behind `$openai-knowledge`.
## Reference Map
| Reference | Read before changing or reviewing |
|---|---|
| [Agent definition and run context](agent-definition-and-run-context.md) | Agent fields, cloning, dynamic instructions, enabled tools or handoffs, context wrappers, usage, or public agent identity |
| [Runner lifecycle](runner-lifecycle.md) | Turn accounting, guardrails, handoffs, interruptions, cancellation, or streaming parity |
| [Run item lifecycle](run-item-lifecycle.md) | Model output processing, new item types, stream events, replay conversion, session persistence, or RunState serialization |
| [Function and output schema](function-and-output-schema.md) | Function-tool signatures and metadata, strict JSON schema conversion, or structured output types |
| [Conversation state ownership](conversation-state-ownership.md) | Sessions versus server-managed continuation, input deltas, retries, compaction, or conversation resume |
| [Session persistence](session-persistence.md) | Session input callbacks, per-turn saves, retry rewind, atomicity, or compaction replacement |
| [RunState schema and resume boundary](runstate-schema.md) | Serialized state, schema versions, approvals, agent identity, or durable resume data |
| [Tool identity and routing](tool-identity.md) | Tool names, namespaces, lookup, approvals, MCP naming, handoffs, or call IDs |
| [Tool execution lifecycle](tool-execution-lifecycle.md) | Function-tool planning, approvals, guardrails, concurrency, cancellation, timeouts, or failure conversion |
| [Local MCP server lifecycle](local-mcp-server-lifecycle.md) | Local MCP connection ownership, manager state, request serialization, caching, filtering, retries, or cleanup |
| [Model and provider boundaries](model-provider-boundaries.md) | Model resolution, provider adapters, feature capability, request conversion, terminal events, or retries |
| [Tracing lifecycle](tracing-lifecycle.md) | Trace and span context, processors, export, flush, shutdown, resume, or sensitive data |
| [Realtime session lifecycle](realtime-session-lifecycle.md) | Realtime listeners, connections, background tasks, handoffs, event iteration, or cleanup |
| [Realtime tracing architecture](realtime-tracing.md) | Realtime API server traces versus Agents SDK client traces |
| [Voice pipeline lifecycle](voice-pipeline-lifecycle.md) | VoicePipeline STT/workflow/TTS ownership, event and audio ordering, stream cleanup, PCM framing, or tracing |
| [Sandbox runtime boundary](sandbox-runtime-boundary.md) | Sandbox session ownership, preparation, resume state, manifests, materialization, or cleanup |
## Maintenance Rules
Keep each rule in the narrowest reference that owns it. Cross-link instead of copying detailed rules between files. Describe current architecture and compatibility boundaries, not the chronology of how a bug was found. Use source paths and durable public contracts as anchors, and remove or rewrite guidance when ownership moves.
@@ -0,0 +1,57 @@
# Agent Definition and Run Context
Use this reference for changes to public `Agent` fields, cloning, dynamic instructions, enabled tools or handoffs, output schemas, `RunContextWrapper`, `ToolContext`, usage aggregation, or the distinction between a public agent and an internal prepared clone.
## Public Definition and Cloning
- Exported `Agent` and `AgentBase` dataclass field order is a positional compatibility boundary. Append optional fields where possible and test old positional construction when the order changes.
- `Agent.__post_init__()` is the eager boundary for invalid field categories such as names, tools, handoffs, hooks, model settings, output types, and tool-use behavior. Dynamic callbacks are validated when invoked because their result depends on the current run.
- `Agent.clone()` uses `dataclasses.replace()` and is shallow. Mutable fields and contained tool, handoff, hook, and provider objects remain shared unless the caller explicitly supplies replacements.
- When `clone(model=...)` replaces a model whose settings still equal the old model's implicit defaults, recompute the new model's implicit defaults. Preserve explicitly customized `model_settings` instead of silently resetting them.
## Per-Turn Resolution
- Resolve dynamic instructions with the current context and public agent for each model turn. Enforce the documented two-argument callable shape and await async results.
- Evaluate callable `FunctionTool.is_enabled` and `Handoff.is_enabled` against the current run context. Do not cache a prior run's enabled set on the reusable agent.
- Use one resolved tool and handoff view for model exposure, reserved-name and collision checks, local dispatch, tracing, and Realtime session updates. Re-resolving independently at those surfaces can expose one set and execute another.
- An internal prepared agent may add bound tools, instructions, or sampling settings, but hooks, `ToolContext.agent`, handoff callbacks, and public results should identify the public agent unless an internal identity is explicitly part of the contract.
- The effective output schema belongs to the agent and model call that produced the candidate output. A handoff can change the final output type, so do not assume the starting agent's schema when parsing or typing the final result.
## Context Ownership
- Every agent, tool, handoff, guardrail, and lifecycle hook in one run must agree on the same application context type. The context object is local runtime state and is never added to model input automatically.
- A normal `ToolContext.from_agent_context()` shares the underlying application object, usage accumulator, and approval mapping with its parent while adding call-scoped fields such as call ID, namespace, arguments, and conversation history.
- Nested `Agent.as_tool()` execution has a separate run loop, approval scope, and resumable tool state. On the normal function-tool path it still shares the application object and usage accumulator, while `tool_input` belongs to the nested wrapper and must not overwrite the parent's scoped value.
- Sharing the application object is not the same as sharing every wrapper field. Add explicit application-level isolation when nested mutation is unsafe, and do not reuse parent approval decisions for nested calls merely because the tool name or call ID looks similar.
- Context serialization is a separate durability decision. Read [RunState schema and resume boundary](runstate-schema.md) before persisting custom context objects, approvals, usage, or nested tool input.
## Usage Accounting
- `RunContextWrapper.usage` is the run-wide mutable accumulator. Add each model response exactly once across streaming, non-streaming, retries, nested runs, handoffs, and resume paths.
- Preserve authoritative `request_usage_entries` when combining usage. Do not synthesize a second per-request entry from aggregate totals when the provider or retry layer already supplied request-level records.
- Retry accounting may include failed attempts with no token totals. Keep request count, aggregate tokens, request-level entries, and trace span usage internally consistent without inventing provider token data.
- Streamed usage remains incomplete until terminal chunks and the stream driver finish. Do not finalize billing, result summaries, or usage-bearing spans from the last visible text delta alone.
## Review Checklist
1. Test direct construction and clone behavior without mutating shared caller-owned objects.
2. Resolve dynamic instructions, tools, and handoffs through the same public agent and current context used for dispatch.
3. Verify handoff and internal prepared-agent paths expose the intended public identity and effective output schema.
4. Test nested agent tools for shared application state and isolated scoped metadata.
5. Compare aggregate and per-request usage after streaming, retries, handoffs, interruption resume, and nested runs.
## Sources
- `docs/agents.md`
- `docs/context.md`
- `docs/results.md`
- `src/agents/agent.py`
- `src/agents/run_context.py`
- `src/agents/tool_context.py`
- `src/agents/usage.py`
- `src/agents/run_internal/turn_preparation.py`
- `src/agents/run_internal/run_loop.py`
- `tests/test_agent_config.py`
- `tests/test_agent_clone_shallow_copy.py`
- `tests/test_agent_as_tool.py`
- `tests/test_usage.py`
@@ -0,0 +1,67 @@
# Conversation State Ownership
Use this reference for changes involving multi-turn input, sessions, `conversation_id`, `previous_response_id`, `auto_previous_response_id`, compaction, retries, `call_model_input_filter`, or `RunState` resume.
## Choose One Conversation Strategy
The state owner determines what the next model request should contain.
| Strategy | State owner | Next-turn input |
|---|---|---|
| Explicit replay with `result.to_input_list()` | Application | Replay-ready history plus the new turn |
| SDK session | Application storage plus the SDK | The same session plus the new turn |
| `conversation_id` | OpenAI Conversations API | The same conversation ID plus only the new turn |
| `previous_response_id` or `auto_previous_response_id` | OpenAI Responses API | The previous response ID plus only the new turn |
| `RunState` resume | Serialized Agents SDK run | Resume the same interrupted run; this is not a new conversation strategy |
In normal use, select one conversation strategy. Mixing client-managed replay or sessions with server-managed continuation can duplicate context unless the implementation explicitly reconciles both owners. Read [Session persistence](session-persistence.md) for the client-managed storage contract.
## Server-Managed Continuation
- `OpenAIServerConversationTracker` in `src/agents/run_internal/oai_conversation.py` owns delta calculation for `conversation_id`, `previous_response_id`, and `auto_previous_response_id`.
- Send only items that the server has not already acknowledged. Object identity is useful only within one process; resume and retry paths also require stable item IDs, tool call IDs, and content fingerprints.
- Update `previous_response_id` from the most recent response that actually has an ID. Do not erase a valid chain because an adjacent provider response lacks one.
- Session persistence cannot be combined with server-managed continuation. `validate_session_conversation_settings()` rejects a session with `conversation_id`, `previous_response_id`, or `auto_previous_response_id`; do not add a second history writer without defining reconciliation and dedupe semantics.
- Treat `conversation_id` and `previous_response_id` / `auto_previous_response_id` chaining as mutually exclusive state owners.
## Filters, Retries, and Resume
- `call_model_input_filter` runs on the prepared model payload. With server-managed continuation, that payload may already be a new-turn delta rather than full history.
- The filter must return `ModelInputData` with list input. Mark exactly the returned list as sent immediately before the request so nested preparation cannot add unsent items, rewind that tracking before retrying a failed request, and preserve it after success.
- Keep streaming and non-streaming tracker updates aligned. Both paths must preserve the same delta, retry, and response-ID semantics.
- Stateful retries require replay-safety evidence. Do not blindly resend a request that may already have advanced server state.
- `RunState` persists conversation identifiers and reconstructs tracker knowledge for resumed runs. Resume must not replay acknowledged input, lose unsent tool outputs, or increment the turn count without a model call.
- Conversation continuation carries context into a new turn. `RunState` resume continues a paused run. Do not substitute one mechanism for the other.
## Compaction
- `compaction_mode="previous_response_id"` depends on a usable stored response chain.
- `compaction_mode="input"` rebuilds from client-held items and is the fallback when the server chain is unavailable or `store=False` prevents later response lookup.
- Compaction must preserve the chosen state owner. Do not compact from local history and then also replay that history through server-managed continuation.
## Handoffs
- Server-managed conversations send deltas, so handoff input filters are not supported. `Handoff.input_filter` and `RunConfig.handoff_input_filter` should raise instead of rewriting a history the server already owns.
- `nest_handoff_history` is a client-history transformation. When server-managed continuation is active, disable it with a warning and continue with delta-only input.
- Keep generated items and session items distinct during handoff processing. The next model input may be filtered, but session history needs the full unfiltered item sequence when client-managed sessions are active.
## Review Checklist
1. Name the state owner before changing request construction.
2. Specify whether the model receives full history or a delta on every affected path.
3. Verify first turn, follow-up turn, retry, interruption, serialized resume, and streaming behavior.
4. Test tool calls and outputs separately; call IDs and output fingerprints have different dedupe roles.
5. Confirm that filtering, compaction, and session persistence do not introduce a second source of truth.
## Sources
- [OpenAI conversation state guide](https://developers.openai.com/api/docs/guides/conversation-state)
- [OpenAI running agents guide](https://developers.openai.com/api/docs/guides/agents/running-agents#choose-one-conversation-strategy)
- `src/agents/run_internal/oai_conversation.py`
- `src/agents/run_internal/run_loop.py`
- `src/agents/run_internal/session_persistence.py`
- `src/agents/run_state.py`
- `docs/running_agents.md`
- `docs/sessions/index.md`
Recheck the official API reference with `$openai-knowledge` before changing server-managed continuation behavior.
@@ -0,0 +1,50 @@
# Function and Output Schema
Use this reference for changes to function-tool signature inspection, parameter metadata, strict JSON schema conversion, tool argument reconstruction, or structured agent output schemas.
Schema behavior is a compatibility boundary shared by Python callables, Pydantic, model providers, and runtime validation. Keep schema generation and invocation aligned rather than fixing one representation in isolation.
## Function Schema Ownership
- Explicit decorator arguments for a function name or description override values inferred from the callable and its docstring.
- Parameter descriptions from parsed docstrings take precedence over description strings carried by `Annotated`. Preserve `Field` constraints, aliases, and defaults when merging `Annotated` metadata.
- A run context parameter is special only in the first parameter position. Exclude it from the model-visible schema while still supplying it during invocation; do not silently treat later context-typed parameters as injected context.
- Keep the inspected signature, generated Pydantic model, JSON schema, and `to_call_args()` reconstruction consistent. Cover positional-only parameters, keyword-only parameters, `*args`, and `**kwargs` when changing this path.
- Reject unsupported callable shapes or invalid schemas when the tool is constructed so failures do not depend on whether a particular model later selects the tool.
## Strict JSON Schema Conversion
- Strict conversion closes object schemas with `additionalProperties: false` and marks their declared properties required. Reject an explicit `additionalProperties: true` instead of silently changing its meaning.
- Preserve the meaning of unions, intersections, definitions, and references. Normalize `oneOf` where required, process `allOf`, retain chained references, and merge a referenced schema with sibling keys without discarding the siblings.
- Remove defaults that only encode Python `None`; a nullable type must remain represented by its type schema rather than by an unsupported default.
- `ensure_strict_json_schema()` may mutate a non-empty input dictionary. Copy caller-owned schemas at public boundaries before conversion. Empty-schema conversion must return a fresh object rather than shared mutable state.
- Keep strictness explicit. If a tool or output schema opts out of strict mode, preserve that choice through provider conversion instead of partially applying strict normalization.
## Structured Output Schemas
- Plain `str` output and no declared output type use the plain-text path. Pydantic models and dictionary-shaped outputs expose their object schema directly; other Python types use the SDK's wrapper object with the `response` key.
- Keep generated output names stable and descriptive for nested generics, unions, and `Literal` types. These names are observable in provider requests and diagnostics.
- Parse model output as JSON and validate it through the output type adapter. Convert JSON or validation failures to the SDK's model-behavior error boundary rather than leaking provider- or Pydantic-specific exceptions.
- Streaming and non-streaming adapters must carry the same schema, strictness flag, wrapper behavior, and validation result.
## Review Checklist
1. Test precedence among explicit metadata, docstrings, `Annotated`, and `Field` values.
2. Test invocation reconstruction for positional-only, keyword-only, variadic, and context-bearing callables.
3. Test nested objects, unions, intersections, sibling and chained references, nullable fields, and caller-owned schema mutation.
4. Test plain text, direct object output, wrapped scalar or generic output, invalid JSON, and validation failure.
5. Verify every provider adapter receives the same normalized schema and strictness decision.
## Sources
- `src/agents/function_schema.py`
- `src/agents/strict_schema.py`
- `src/agents/tool.py`
- `src/agents/agent_output.py`
- `src/agents/models/`
- `tests/test_function_schema.py`
- `tests/test_function_tool_decorator.py`
- `tests/test_strict_schema.py`
- `tests/test_strict_schema_oneof.py`
- `tests/test_output_tool.py`
- `tests/models/`
@@ -0,0 +1,56 @@
# Local MCP Server Lifecycle
Use this reference for changes to Python-managed MCP servers, `MCPServerManager`, client-session request ordering, tool caching or filtering, local MCP retries, cancellation, or cleanup. Hosted MCP is a provider tool and follows the OpenAI API contract; use `$openai-knowledge` for that protocol surface. Read [Tool identity and routing](tool-identity.md) for server-prefixed names and [Tool execution lifecycle](tool-execution-lifecycle.md) for approval and invocation behavior after an MCP tool is converted to a `FunctionTool`.
## Connection Ownership and Task Affinity
- A local `MCPServer` owns its transport, `ClientSession`, and `AsyncExitStack` from `connect()` through `cleanup()`. Partial connection failure still requires closing every context already entered.
- Some MCP transports use AnyIO cancel scopes that require connection and cleanup in the same task. Do not wrap either operation in a helper that silently creates another task.
- `MCPServerManager` preserves task affinity in sequential mode and uses one long-lived worker task per server in parallel mode. Timeouts must run inside that owning task; on Python versions without `asyncio.timeout()`, cancel the current worker task and translate only timer-originated cancellation to `TimeoutError`.
- Cleanup runs servers in reverse order and continues across ordinary cleanup failures. Cancellation suppression is an explicit manager policy; do not accidentally convert unrelated `BaseException` failures into recoverable connection errors.
- Server cleanup must clear session and transport-visible state even when exit-stack cleanup raises, so the same server object can reconnect without exposing stale session handles or workers.
## Manager State
- Keep configured servers, connected servers, failed servers, active servers, and per-server errors as distinct views. `active_servers` is the agent-facing list; with `drop_failed_servers=True` it excludes failed connections while preserving configured order.
- Non-strict connection records failures and continues with the connected subset. Strict connection cleans up work started by the failed attempt and restores the previous coherent active state before raising.
- `reconnect(failed_only=True)` retries the deduplicated failed set without disturbing healthy connections. A full reconnect cleans up all servers first and rebuilds manager state.
- Parallel connection still needs deterministic per-server state and complete cleanup after cancellation or one hard failure. Do not let completion order decide `active_servers`, `failed_servers`, or which workers remain registered.
## Shared Session Requests and Retries
- Streamable HTTP can require requests on one shared MCP session to be serialized. The same lock must cover tool calls, tool listing, prompts, and resource operations that share that session; serializing only `call_tool()` still permits sibling cancellation and protocol races.
- Preserve outer cancellation. A cancelled shared request may qualify for an isolated-session retry only when the transport identifies it as an inner or transient session failure and retry budget remains.
- Isolated-session retries are transport-specific recovery. Count isolated session setup and execution against the same retry budget, retry only the supported transient failure shapes, and never replay mixed exception groups or ordinary 4xx failures as if they were safe.
- Generic `list_tools()` and `call_tool()` retries use the configured attempt count and backoff. Validate required arguments locally before starting retries so deterministic input errors never reach the server or consume retry budget.
- MCP tool failure conversion follows the effective server or agent `failure_error_function`. Explicit `None` means propagate; cancellation of the parent run must not become model-visible tool failure output.
## Tool Discovery, Cache, and Filtering
- The unfiltered server tool list is the cacheable value. Apply static or dynamic filters to a copy for each requesting agent and run context; never let one request's filtered or merged metadata mutate the shared cache.
- `cache_tools_list=True` assumes server schemas are stable until `invalidate_tools_cache()` marks them dirty. Connection or filter changes must not accidentally make a stale filtered list authoritative.
- Dynamic filters require both `run_context` and agent. A filter exception excludes that tool and logs the failure rather than exposing it by default.
- Schema conversion to strict form is best effort and must not mutate the MCP server's original input schema. If strict conversion fails, preserve the original schema and keep metadata isolated per converted `FunctionTool`.
- Tool list collision errors, prefixed-name generation, and approval policy validation must be deterministic regardless of server response or connection completion order.
## Review Checklist
1. Identify the task that owns connect, every request, timeout cancellation, and cleanup for each transport.
2. Test partial connect failure, strict and non-strict manager modes, reconnect, repeated cleanup, and manager cancellation.
3. Test overlapping tool, prompt, and resource requests when shared-session serialization is enabled.
4. Prove retries preserve outer cancellation, consume one budget, and do not replay deterministic or unsupported failures.
5. Test cache invalidation, context-dependent filters, schema immutability, duplicate names, and reconnect with the public runner path.
## Sources
- `docs/mcp.md`
- `src/agents/mcp/server.py`
- `src/agents/mcp/manager.py`
- `src/agents/mcp/util.py`
- `tests/mcp/test_mcp_server_manager.py`
- `tests/mcp/test_connect_disconnect.py`
- `tests/mcp/test_client_session_retries.py`
- `tests/mcp/test_caching.py`
- `tests/mcp/test_tool_filtering.py`
- `tests/mcp/test_server_errors.py`
- `tests/mcp/test_runner_calls_mcp.py`
@@ -0,0 +1,75 @@
# Model and Provider Boundaries
Use this reference for changes to model resolution, `ModelSettings`, provider adapters, Responses versus Chat Completions behavior, request conversion, streaming terminal events, transport reuse, or model retries.
## Core Boundary
The run loop depends on the `Model` interface, not on one provider's request or response schema.
- `Model.get_response()` returns a normalized `ModelResponse`.
- `Model.stream_response()` yields normalized response stream events while preserving provider payloads needed by public raw-event consumers.
- `ModelProvider.get_model()` resolves names to model implementations and owns provider-level caches or connections.
- `Model.close()` and `ModelProvider.aclose()` release persistent transport resources when an implementation owns them.
Provider adapters own request construction, provider feature validation, terminal event interpretation, usage conversion, and translation into SDK item shapes. Keep provider-specific branching out of the core run loop unless it represents a shared SDK contract.
## Model and Settings Resolution
- An explicit `RunConfig.model` overrides the agent model. A model instance is used directly; a model name is resolved through the configured `ModelProvider`.
- Implicit default settings must follow the resolved model name, including when a run-level model name replaces the agent default.
- Resolve agent settings with run-level settings by overlaying non-`None` values. Preserve the documented merge behavior for structured fields such as `extra_args` and retry settings.
- Do not pass provider request extras into tracing by default. `ModelSettings.to_traceable_dict()` is the boundary for settings considered safe and meaningful in traces.
## Capability Ownership
Do not infer that a feature available in one adapter is supported by every `Model` implementation.
- Responses-specific features include server-managed response chaining, conversation-aware request fields, tool namespaces, deferred tool loading, tool search, response includes, compaction, and Responses websocket transport.
- Chat Completions generally requires client-managed replay and adapter conversion of Responses-compatible SDK items. Unsupported server-state or tool features should be rejected or explicitly ignored according to the adapter's documented validation mode.
- Realtime has its own session protocol, event model, and server tracing. Do not route Realtime behavior through the standard Responses or Chat Completions assumptions.
- Third-party model adapters may preserve only the shared `Model` contract. New provider-specific fields need an explicit conversion and fallback policy.
Validate capabilities at the adapter boundary where the resolved model and complete request are known. Avoid public flags that appear accepted by the SDK but are silently dropped before the provider request.
## Provider Data and Terminal Semantics
- Preserve provider-supplied string IDs, request IDs, usage, and opaque provider data when the public SDK contract exposes them.
- Normalize provider objects and mapping payloads without relying on truthiness for valid empty or zero values.
- A transport stream ending is not automatically a successful model response. Responses `failed` and `incomplete` terminals, explicit error events, and a missing terminal payload must produce the documented failure behavior in both HTTP and websocket paths.
- Keep semantically equivalent HTTP, websocket, streaming, and non-streaming paths aligned on final `ModelResponse`, errors, request IDs, and usage.
## Transport Resource Ownership
- Persistent Responses websocket models are loop-bound resources. Cache reusable websocket model instances by running event loop and model name; do not share one connection or `asyncio.Lock` across loops.
- Use weak loop ownership so an unused cache does not keep a closed event loop alive. When a live connection itself pins a closed loop, prune it with synchronous abort and state clearing rather than awaiting work on that closed loop.
- A provider that caches persistent models must make `aclose()` close every unique cached model and clear its caches. Close on a still-running owner loop when possible; do not drive an inactive foreign loop inside `asyncio.to_thread()`.
- A model used without a running loop cannot safely join the loop-scoped websocket cache. Preserve the non-reuse fallback rather than attaching it to an arbitrary global loop.
- Connection reuse ends after protocol errors, pre-terminal disconnects, cancellation that invalidates framing, or explicit close. Clear connection and loop-bound lock state together so a later request cannot reuse half-closed transport state.
## Retry and Replay Safety
- Provider retry advice can describe retryability, delay, and replay safety; the runner must not replace provider-specific evidence with a generic status-code assumption.
- Requests that use server-managed conversation state or may have produced side effects are not automatically replay-safe. A retry policy must account for whether the provider could have accepted the previous attempt.
- Retry conversion and error handlers must preserve the original exception semantics and avoid leaking sensitive request payloads through chaining, logs, traces, or provider error objects.
## Review Checklist
1. Identify which adapter owns the feature and how unsupported adapters behave.
2. Verify model and implicit-settings resolution when run config overrides the agent.
3. Compare HTTP/websocket and streaming/non-streaming terminal behavior when applicable.
4. Preserve request IDs, usage, provider data, and error semantics through normalization.
5. Prove retries are safe for the request's state ownership and side effects.
6. Test transport reuse, cross-loop access, closed-loop pruning, and provider shutdown when persistent connections are involved.
## Sources
- `src/agents/models/interface.py`
- `src/agents/model_settings.py`
- `src/agents/run_internal/turn_preparation.py`
- `src/agents/models/openai_responses.py`
- `src/agents/models/openai_chatcompletions.py`
- `src/agents/models/multi_provider.py`
- `src/agents/models/_response_terminal.py`
- `src/agents/run_internal/model_retry.py`
- `tests/models/`
- `tests/test_config.py`
@@ -0,0 +1,73 @@
# Realtime Session Lifecycle
Use this reference for `RealtimeSession` changes involving entry, exit, listeners, connections, background tasks, approvals, handoffs, event iteration, tracing context, or cleanup.
## Resource Ownership
Treat the session as the owner of these resources once they are acquired:
| Resource | Acquisition | Required release or terminal state |
|---|---|---|
| Model listener | `add_listener()` during entry | `remove_listener()` |
| Model connection | `model.connect()` | `model.close()` |
| Event iterators | Waiting on the event queue | Wake or terminate every waiter on close |
| Guardrail tasks | Created during output processing | Complete, or cancel and account for completion |
| Tool-call tasks | Created when `async_tool_calls=True` | Complete, or cancel and account for completion |
| Pending approvals and outputs | Added during tool execution | Resolve, retain for retry, or clear during terminal cleanup |
| Agent and model settings | Updated on handoff or `update_agent()` | Keep runtime state and model configuration aligned |
Do not add a new side effect before a failure point without defining who releases it.
## Entry and Exit
- Python does not call `__aexit__` when `__aenter__` raises. Any listener, connection, task, tracing scope, or other resource acquired before the exception needs explicit failure cleanup.
- Keep construction free of external side effects. Acquire listeners and connections during entry where failures can be handled coherently.
- `close()` and internal cleanup must be idempotent. Repeated close paths should still wake event iterators without closing the model twice.
- Mark the session closed only after the cleanup state is coherent. If model close fails, decide deliberately whether retry is possible and which resources remain owned.
## Async Task and Context Rules
- `asyncio` tasks inherit a snapshot of the creator's context. A background task cannot update the caller task's `ContextVar` state.
- A `ContextVar` token must be reset in the same context that created it. Never pass a token to a different task and assume cleanup can reset it safely.
- Shared session fields can be mutated by the listener path, tool-call tasks, `close()`, `update_agent()`, and handoff handling. Review ordering and races whenever one of those paths changes.
- Calling `task.cancel()` requests cancellation; it does not prove the task has finished its `finally` blocks or released resources. Await cancelled tasks when completion matters, or document and test why dropping them is safe.
- Background-task exceptions must reach a deterministic owner. They must not silently disappear or leave event consumers blocked.
## Agent Transitions
- Handoffs and the public `update_agent()` API are equivalent agent-transition surfaces. Keep their model settings, tool and handoff resolution, emitted events, and tracing metadata aligned unless a difference is intentional and documented.
- Resolve dynamic tools and enabled handoffs once per transition when possible, then reuse the exact resolved values for model settings and metadata.
- With concurrent tool calls, capture the agent snapshot associated with each call. Do not route a call through whichever agent happens to be current when the task eventually runs.
## Guardrails and Response Ordering
- Realtime output guardrails inspect accumulated transcript text at configured debounce thresholds, not each token and not a final `Runner` output object. They emit `guardrail_tripped` instead of raising a normal Runner tripwire exception.
- A tripped output guardrail marks the response interrupted before awaiting transport work, emits one trip event per response, forces response cancellation, and sends safe follow-up input naming the guardrail. Concurrent guardrail tasks must not interrupt or message the same response twice.
- Guardrail callbacks can run after audio has already been buffered or played. Consumers must treat `audio_interrupted` as the signal to stop local playback; text rejection alone cannot retract audio already delivered.
- An exception from one output guardrail is logged and skipped so it does not silently terminate the live session. Exceptions that escape the background guardrail task must become a `RealtimeError` event rather than disappearing.
- Realtime function-tool input guardrails follow the same optional pre-approval and mandatory post-approval ordering as standard function tools, but their rejection is returned through Realtime tool output and events.
- Follow-up `response.create` work triggered by tools, handoffs, or guardrails must respect the active response lifecycle. Wait for `response.done` or the model layer's equivalent gate before starting a conflicting response.
## Failure-Path Tests
Add focused tests for affected phases:
1. Instruction, tool, or handoff resolution fails during entry.
2. Model connection fails after listener registration.
3. A background tool or guardrail task raises or is cancelled.
4. Cleanup runs while event iterators are waiting.
5. `close()` is called repeatedly or from another task.
6. A handoff or `update_agent()` fails partway through model-settings application.
7. Tool output sending fails after local execution and must be retried without running the tool twice.
8. Concurrent guardrail tasks trip once, cancel playback, and do not overlap follow-up responses.
Verify lifecycle changes with the real public path where feasible; helper-only tests are insufficient when task ownership or context propagation determines the result.
## Sources
- `src/agents/realtime/session.py`
- `src/agents/realtime/model.py`
- `src/agents/realtime/openai_realtime.py`
- `tests/realtime/test_session.py`
- `tests/realtime/test_session_exceptions.py`
- `docs/realtime/guide.md`
+42
View File
@@ -0,0 +1,42 @@
# Realtime Tracing Architecture
Use this reference when reviewing or implementing Realtime tracing behavior, especially claims that `RealtimeSession` should emit the same trace hierarchy as `Runner`.
## Two Separate Tracing Systems
Realtime integrations involve two independent tracing paths:
| Path | Owner | Configuration | Result |
|---|---|---|---|
| Realtime API server tracing | Realtime API | `"auto"`, `workflow_name`, `group_id`, and `metadata` | The server creates a Realtime session trace in the Traces Dashboard. |
| Agents SDK client tracing | Agents SDK tracing provider | `trace()`, `agent_span()`, and other SDK span factories | The SDK exports locally created traces and spans through its tracing processor. |
The current Python SDK has no mapping from an Agents SDK client `trace_id`, `span_id`, or parent context into `RealtimeModelTracingConfig` or the model's `session.update`. A server-created Realtime trace is therefore not attached as a child of an SDK-created trace or span by this implementation. Likewise, adding an SDK `agent_span()` around `RealtimeSession` does not make server-side trace contents children of that span.
If both paths are enabled, the dashboard can contain two separate traces. A shared `group_id` can make them easier to filter and correlate, but it does not merge them or create a parent-child relationship.
## Current Python SDK Behavior
- `RealtimeModelTracingConfig` exposes only `workflow_name`, `group_id`, and `metadata` in `src/agents/realtime/config.py`.
- `OpenAIRealtimeWebSocketModel` defaults the Realtime tracing configuration to `"auto"` when the caller does not provide one.
- After receiving `session.created`, the model sends the tracing configuration through a `session.update` event.
- `RealtimeRunConfig.tracing_disabled` prevents the SDK from enabling Realtime tracing for that session.
Verify these paths in `src/agents/realtime/openai_realtime.py` and `src/agents/realtime/session.py`; do not rely on old issue descriptions because Realtime tracing support has changed over time.
## Maintainer Constraints
1. Identify whether the behavior belongs to the Realtime API's server trace or an Agents SDK client trace created with `trace()`.
2. A client-side agent span does not repair missing server tracing and does not create the unified hierarchy produced by `Runner`.
3. The current Python SDK cannot place server-created Realtime spans under an SDK-created trace or span because it does not carry client trace parentage through the Realtime tracing configuration. Recheck the live protocol with `$openai-knowledge` before treating that implementation gap as permanent.
4. Use the server trace for Realtime model activity. Use a shared `group_id` or metadata when correlation with a client trace is required.
5. Parallel SDK spans need an explicit product and maintenance contract covering the dual-trace user experience, async task context, handoff parenting, failure cleanup, and the client-only operations represented by those spans.
6. A client trace becoming non-empty is not evidence that Realtime server activity has been captured or parented correctly.
## Sources
- `src/agents/realtime/config.py`
- `src/agents/realtime/openai_realtime.py`
- `src/agents/realtime/session.py`
Recheck the official API reference with `$openai-knowledge` before changing this guidance or implementing new protocol behavior.
+79
View File
@@ -0,0 +1,79 @@
# Run Item Lifecycle
Use this reference for changes to model output processing, `RunItem` types, tool call and output items, stream events, replay conversion, session history, or serialized run state.
## Item Flow
The runtime carries one semantic item through several representations:
1. A model adapter returns provider output in `ModelResponse.output`.
2. `process_model_response()` converts recognized output into public `RunItem` objects and internal executable tool-run records in `ProcessedResponse`.
3. Tool execution and handoffs add output items and choose a `SingleStepResult.next_step`.
4. The resulting items feed `RunResult`, semantic stream events, session persistence, tracing, and `RunState` serialization.
5. Replayable items convert back to model input through `RunItem.to_input_item()` or `run_item_to_input_item()` after SDK-only metadata is handled.
Keep provider payloads, public run items, and internal execution records distinct. A provider item may be observable without requiring local execution, while a local tool-run record may need to preserve the selected SDK tool object and routing identity.
## Generated, Session, and Model Input Views
- `new_step_items` describes items generated by the current step.
- `session_step_items` preserves the full unfiltered sequence when session history must retain items that a handoff or input filter omitted from the next model request.
- `generated_items` is the public observability view and prefers `session_step_items` when present.
- Model input is a replay view, not the canonical storage view. Approval placeholders, SDK-only metadata, unsupported IDs, and orphaned calls may need filtering or normalization before an API request.
Do not force these views into one list. History persistence, user-visible results, and the next provider request have different correctness requirements.
## Adding or Changing an Item Type
Update every applicable surface together:
- `src/agents/items.py` for the public `RunItem` type, accessors, and replay conversion.
- `src/agents/run_internal/run_steps.py` for processed response and executable tool-run records.
- `src/agents/run_internal/turn_resolution.py` for provider output recognition, item creation, side effects, and next-step selection.
- `src/agents/run_internal/tool_execution.py`, `tool_actions.py`, or `tool_planning.py` for execution, dedupe, approvals, and outputs.
- `src/agents/run_internal/items.py` for normalization, replay conversion, fingerprints, dedupe, and provider-boundary metadata stripping.
- `src/agents/stream_events.py` and streaming queue helpers for public semantic events.
- `src/agents/run_state.py` for serialization and deserialization when the item can survive interruption.
- `src/agents/run_internal/session_persistence.py` for session conversion, sanitization, and retry accounting.
- Tracing and usage conversion when the item contributes observable tool or model work.
## Compatibility Rules
- Public stream event names are compatibility-sensitive. Do not rename an existing event, even to fix spelling, without an explicit breaking-change plan.
- Preserve provider-supplied IDs and opaque provider data until the owning boundary deliberately removes them. Do not invent IDs or coerce malformed values to make replay appear valid.
- Preserve SDK-only metadata needed for display, routing, approvals, tool origin, or resume, but strip it before sending payloads to a provider that does not accept it.
- Tool call and output pairs must retain the same string call ID across execution, replay, session persistence, and resume.
- Empty, falsey, structured, image, file, and custom tool outputs are valid values unless the public tool contract explicitly rejects them; do not use broad truthiness checks to decide whether output exists.
## Replay Integrity
- Prune orphan calls only from runner-generated or resumed history where the SDK owns call/output pairing. Preserve caller-supplied initial input unless an explicit public normalization contract says otherwise.
- When dropping an orphan tool call, also drop reasoning items tied to that removed call so the provider does not receive a reasoning item without its required following item. Do not drop a lone reasoning item merely because its following item is absent locally; server-managed conversation state may own that item.
- `reasoning_item_id_policy="omit"` strips IDs only from SDK-generated follow-up reasoning items. It does not rewrite initial caller input, must survive `RunState` resume, and can be superseded by a later `call_model_input_filter` that deliberately returns IDs.
- Pair anonymous tool-search outputs with the latest compatible anonymous call and never pair a named call with an anonymous output. A missing call ID does not justify inventing a persistent provider identity.
- `provider_data` and provider IDs have boundary-specific ownership. Preserve them for raw results and provider requests that accept them, but strip private or replay-unsafe metadata from session and server-conversation history where the SDK contract requires sanitized items.
## Review Checklist
1. Follow the item from provider response through result, stream, session, replay, and `RunState`.
2. Test both typed provider objects and mapping payloads when adapters support both.
3. Verify IDs, metadata, and output values survive every required round-trip.
4. Test filtering and dedupe without losing the latest valid call/output pair.
5. Compare streaming event order with the non-streaming item sequence.
6. Test orphan pruning and reasoning pairing with client-managed replay and server-managed continuation separately.
## Sources
- `src/agents/items.py`
- `src/agents/stream_events.py`
- `src/agents/run_internal/items.py`
- `src/agents/run_internal/run_steps.py`
- `src/agents/run_internal/turn_resolution.py`
- `src/agents/run_internal/session_persistence.py`
- `src/agents/run_state.py`
- `tests/test_items_helpers.py`
- `tests/test_run_internal_items.py`
- `tests/test_stream_events.py`
- `tests/test_run_state.py`
- `docs/running_agents.md`
- `docs/results.md`
+67
View File
@@ -0,0 +1,67 @@
# Runner Lifecycle
Use this reference for changes to `Runner`, turn accounting, guardrails, hooks, handoffs, interruptions, cancellation, or streaming and non-streaming behavior.
## Turn Boundary
A turn is one logical model invocation plus processing of that response. Tool execution, handoff resolution, session persistence, interruption resume, and retries inside that logical invocation do not independently consume turns.
- Increment the turn counter exactly once when the run loop starts a logical model turn. Transport or provider retries inside `get_new_response()` remain part of that turn.
- A handoff changes the current agent, but the next turn begins only when the new agent invokes a model.
- Resuming `NextStepInterruption` continues the paused turn. Resolve stored approvals and tool work before deciding whether another model call is needed.
- Preserve `max_turns` and the current turn in `RunState`; resume must not reset the budget or charge a turn twice.
## Guardrail Ordering
- Input guardrails belong to the starting agent and run only for the initial user input. Do not rerun them after handoffs or when resuming an interruption.
- Sequential input guardrails must finish before model-side effects begin. Parallel input guardrails may overlap the model call, so a tripwire or exception must cancel and await the in-flight model task and sibling guardrail tasks.
- Tool input guardrails run before the approved tool side effect. Tool output guardrails run after local execution and before the output is accepted into the next step.
- Output guardrails run only after a candidate final output exists. Streaming must await them and preserve the same tripwire and exception behavior as non-streaming execution before declaring completion.
- Guardrail results are observable run state. Preserve them across handoffs, error handlers, streamed completion, and `RunState` round-trips.
## Step State Machine
`SingleStepResult.next_step` is the control boundary after one model response and its local side effects:
| Step | Meaning |
|---|---|
| `NextStepRunAgain` | Continue with the current agent and make another model call |
| `NextStepHandoff` | Switch the current agent, emit the transition, then continue |
| `NextStepFinalOutput` | A final candidate exists; finish terminal hooks, output guardrails, persistence, and result construction |
| `NextStepInterruption` | Persist enough processed state to resume pending approvals without rerunning completed work |
Do not bypass this state machine with path-local completion logic. New terminal or pausable behavior must define non-streaming, streaming, session, tracing, and serialized-resume semantics.
## Streaming Parity and Cancellation
- Streaming and non-streaming paths must produce equivalent final output, generated items, current agent, usage, guardrail results, session history, and interruption state for the same model behavior.
- Raw transport events may differ, but semantic `RunItemStreamEvent` and `AgentUpdatedStreamEvent` emission must follow the same processed items and agent transitions used by the non-streaming result.
- `stream_events()` is the stream driver's cleanup boundary. Keep consuming it until exhaustion after normal completion or `cancel()`, or explicitly close the async iterator; merely breaking after the last visible token does not prove session writes, guardrails, compaction, sandbox cleanup, usage, or terminal errors have settled.
- Immediate cancellation marks the result complete and requests task cancellation. `after_turn` cancellation leaves the current model/tool turn running so it can persist state and usage before the next turn. Preserve this distinction instead of treating both modes as queue shutdown.
- Terminal run-loop, guardrail, and max-turn errors must be surfaced from `stream_events()` after the required queued events are handled. Preserve `run_loop_exception` as a diagnostic view of the background task, not as a replacement completion primitive.
- `task.cancel()` is a request, not cleanup completion. Await cancelled tasks when their `finally` blocks, exceptions, or owned resources affect run correctness.
- Keep lifecycle hooks aligned across both paths, especially model start/end, handoff, tool start/end, and final-output hooks.
## Review Checklist
1. Identify which turn and which agent own the behavior.
2. Trace every `NextStep` outcome, including interruption resume.
3. Compare streaming and non-streaming side effects and terminal ordering.
4. Test guardrail tripwires and exceptions in sequential and parallel modes when relevant.
5. Verify normal exhaustion, explicit iterator close, immediate cancellation, and after-turn cancellation leave the documented result and owned resources in a coherent state.
## Sources
- `src/agents/run.py`
- `src/agents/run_internal/run_loop.py`
- `src/agents/run_internal/run_steps.py`
- `src/agents/run_internal/turn_preparation.py`
- `src/agents/run_internal/turn_resolution.py`
- `src/agents/run_internal/guardrails.py`
- `tests/test_agent_runner.py`
- `tests/test_agent_runner_streamed.py`
- `tests/test_cancel_streaming.py`
- `tests/test_guardrails.py`
- `tests/test_run_state.py`
- `docs/streaming.md`
- `docs/results.md`
+64
View File
@@ -0,0 +1,64 @@
# RunState Schema and Resume Boundary
Use this reference for changes involving `RunState` serialization, deserialization, approvals, trace state, sandbox state, agent identity, tool output payloads, or any persisted resume data.
## Compatibility Boundary
`RunState` is the durable SDK pause/resume boundary. Treat the serialized JSON shape as compatibility-sensitive once a schema version has shipped in a release.
- `to_json()` always emits `CURRENT_SCHEMA_VERSION`.
- `from_json()` must continue reading every version in `SUPPORTED_SCHEMA_VERSIONS`.
- Older SDKs intentionally reject newer or unsupported versions rather than attempting forward compatibility.
- Unreleased schema versions may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported.
- Every supported version must have a non-empty one-line entry in `SCHEMA_VERSION_SUMMARIES`.
## When to Bump the Schema
Bump `CURRENT_SCHEMA_VERSION` when a serialized `RunState` snapshot changes in a way that affects resume correctness or would silently lose data when read under an older schema label.
Examples include:
- New persisted fields on `RunState`, `ModelResponse`, `ProcessedResponse`, interruptions, approvals, tool outputs, sandbox state, trace state, or agent-owned state.
- New run item, tool call, approval, or output item variants that can appear in serialized state.
- New SDK-only metadata needed to route, dedupe, approve, retry, or resume a tool call.
- A changed meaning for an existing serialized field.
Do not rely on current-reader tests alone. Add a regression that rewrites `$schemaVersion` to an older supported label when appropriate and proves the old label is accepted, rejected, or migrated deliberately.
## Identity and Routing State
Serialized state must preserve enough identity to resume without changing behavior:
- Agent identity must distinguish duplicate agent names in the same graph.
- Function tools should persist canonical lookup keys, including `bare`, `namespaced`, and `deferred_top_level`.
- Tool call IDs must remain provider-supplied strings; do not coerce arbitrary values into IDs.
- Approval decisions and rejection messages must restore against the same tool identity and call ID they originally targeted.
- The per-agent tool-use tracker must preserve stable duplicate-agent identity so tool-choice reset behaves the same after resume.
- Server-managed conversation identifiers must restore into `OpenAIServerConversationTracker` without replaying acknowledged input.
## Context and Secrets
Context serialization is intentionally conservative.
- Mapping contexts can round-trip directly.
- Custom contexts need explicit serializers and deserializers when exact restoration matters.
- Without a safe serializer, snapshots may record metadata and warnings rather than the raw object.
- Do not persist secrets in `RunContextWrapper.context`, trace data, tool outputs, or custom data unless the caller explicitly chose that durability boundary.
## Review Checklist
1. Identify every serialized field whose shape or meaning changes.
2. Decide whether the affected schema version is released or unreleased.
3. Update `CURRENT_SCHEMA_VERSION` and `SCHEMA_VERSION_SUMMARIES` when resume compatibility requires it.
4. Keep released schema versions readable, or fail with an explicit compatibility error if the old label cannot safely represent the new data.
5. Test `to_json()` output, `from_json()` restoration, string round-trips, and resumed execution through the public `Runner.run(...)` or `Runner.run_streamed(...)` path.
## Sources
- `src/agents/run_state.py`
- `src/agents/result.py`
- `src/agents/run_internal/agent_runner_helpers.py`
- `src/agents/run_internal/oai_conversation.py`
- `src/agents/run_internal/run_steps.py`
- `src/agents/run_internal/tool_execution.py`
- `tests/test_run_state.py`
@@ -0,0 +1,70 @@
# Sandbox Runtime Boundary
Use this reference for changes to sandbox session ownership, `SandboxAgent` preparation, manifests, capabilities, host-path materialization, snapshots, resume state, agent transitions, or cleanup.
## Runtime Ownership
The outer `Runner` owns agent turns, approvals, handoffs, tracing, session history, and `RunState`. A sandbox session owns the execution environment, workspace, processes, mounts, and provider-specific connection state. Do not move one layer's lifecycle into the other without defining resume and cleanup behavior for both.
- A live `SandboxRunConfig.session` is caller-owned. The runner may configure and use it but must not delete or fully tear it down.
- A session created or resumed through `SandboxRunConfig.client` is runner-owned. Cleanup runs pre-stop hooks, persists snapshot-backed workspace state, stops and shuts down the session, deletes provider resources when required, and closes dependencies.
- Session cleanup must be idempotent and release acquired `SandboxAgent` concurrency guards even when persistence or provider cleanup fails.
- A `SandboxAgent` instance cannot be reused concurrently across runs because prepared capability tools and session state are bound to one live run. Clone or construct separate agents for concurrent work.
## Session Source and Saved State
Resolve the session source in this order: injected live session, resumable sandbox state carried by `RunState`, explicit `SandboxRunConfig.session_state`, then a newly created session. Manifest and snapshot inputs seed only a fresh session; they do not overwrite an injected or resumed workspace.
- `RunState` sandbox data and explicit `session_state` represent provider connection or session state used to reconnect to existing work.
- A snapshot represents saved workspace contents used to seed a new session. It is not interchangeable with provider session state.
- Preserve stable per-agent resume identity across handoffs, including graphs with duplicate agent names. Object identity is process-local, so serialized state needs stable keys and explicit current-agent selection.
- Serialize runner-owned sessions after stop-time persistence has completed so a later resume can reattach when the backend survives or reconstruct the workspace from the saved snapshot when it does not.
## Agent Preparation
- Clone capability instances per run before binding them to a live session. Reusing mutable capability objects can leak tools, sampling settings, or session references across runs.
- Validate capability dependencies before exposing tools. Capability tool construction, instruction fragments, input processing, and sampling adjustments must use the same effective capability set.
- Build instructions in the documented order: SDK sandbox base prompt or explicit replacement, agent instructions, capability instructions, remote-mount policy, then the rendered filesystem description.
- Bind capability tools to the live session and preserve a link from the prepared clone to the public `SandboxAgent`. Dynamic instructions and hooks should observe the public agent rather than an internal clone with implementation-only state.
- Handoffs stay in the outer run loop and select another agent-bound sandbox session. A nested `Agent.as_tool()` run owns its own nested runner and sandbox lifecycle.
## Filesystem Trust Boundary
- Manifest entry destinations are workspace-relative and must not escape the workspace. The workspace root itself must be absolute where the backend requires an absolute runtime root.
- `LocalFile` and `LocalDir` sources are host-side inputs. Resolve them against a trusted base directory, require explicit application-controlled `extra_path_grants` outside that base, and reject untrusted manifests that try to authorize their own host access.
- Validate local sources at use time, not only when parsing the manifest. Defend against symlinked sources, parent-directory swaps, platform path aliases, and archive members that change meaning between validation and extraction.
- Archive extraction must reject traversal, unsafe links, and unsupported member types before writing, and enforce entry, byte, and expansion limits without materializing an unbounded member list.
- Extra path grants are runtime access, not durable workspace content. Snapshots and `persist_workspace()` include the workspace root, not arbitrary granted paths.
- Credentials for mounts or providers must remain in the owning adapter and must not appear in generated shell commands, model-visible errors, logs, or serialized sandbox state.
## Provider and Error Boundary
- Normalize backend failures to sandbox errors without discarding provider details needed for diagnosis. Preserve explicit retryability instead of inferring it later from a message string.
- Keep portable sandbox paths separate from host filesystem paths and provider identifiers. Conversion belongs in the backend or materialization boundary, not in agent-facing tools.
- Temporary clones, mounts, sinks, and dependency resources need failure cleanup during partial startup as well as normal shutdown.
- Capability tools should report bounded output and preserve provider exit status or structured error data without exposing private runtime metadata to the model.
## Review Checklist
1. Name the owner of every live session, provider client, mount, process, capability, and temporary resource.
2. Test injected, resumed, explicit-state, snapshot-seeded, and fresh-session paths separately.
3. Verify handoffs, duplicate agent names, interruption resume, and cleanup failure preserve the intended session mapping.
4. Test host-path, symlink, traversal, archive-limit, and credential-redaction boundaries on applicable platforms.
5. Exercise the public `Runner` path so agent preparation, capability binding, persistence, and cleanup run together.
## Sources
- `docs/sandbox/guide.md`
- `docs/sandbox/clients.md`
- `src/agents/sandbox/runtime.py`
- `src/agents/sandbox/runtime_session_manager.py`
- `src/agents/sandbox/runtime_agent_preparation.py`
- `src/agents/sandbox/manifest.py`
- `src/agents/sandbox/materialization.py`
- `src/agents/sandbox/workspace_paths.py`
- `src/agents/sandbox/session/archive_extraction.py`
- `tests/sandbox/test_runtime.py`
- `tests/sandbox/test_runtime_agent_preparation.py`
- `tests/sandbox/test_session_state_roundtrip.py`
- `tests/sandbox/test_materialization.py`
- `tests/sandbox/test_extract.py`
+80
View File
@@ -0,0 +1,80 @@
# Session Persistence
Use this reference for changes to client-managed sessions, session input callbacks, per-turn persistence, retries, rewind, compaction replacement, or session backend implementations.
Read [Conversation state ownership](conversation-state-ownership.md) first when server-managed continuation is also involved. A client-managed session is a history store; it is not a second owner for a server-managed conversation.
## Session Contract
- `get_items(limit=N)` returns the latest `N` items in chronological order.
- `add_items()` appends one logical batch. Backends should make the batch atomic so partial turns are not visible after failure.
- `pop_item()` removes the current tail item and is used only for guarded rollback of items the current run can prove it owns.
- `clear_session()` clears the session boundary; compaction decorators that replace history must provide stronger restore behavior around destructive replacement.
Third-party implementations target the `Session` protocol. Internal base classes and backend-specific metadata are not the compatibility contract unless explicitly documented.
## Backend Consistency
- An explicit `get_items(limit=N)` argument overrides the backend's default session limit. Return the latest `N` items in chronological order, with a deterministic tie-breaker when timestamps can collide.
- Preserve caller batch order. Persist the items and any indexes or structural metadata required to read them as one atomic operation. A failed batch must leave earlier history unchanged, and any backend-internal retry must not create duplicates.
- Serialize initialization and conflicting writes at the backend's actual consistency boundary. Concurrent first writers must not race, and cancellation or failure must not strand locks or transactions.
- Apply configured table or collection names and session settings consistently across reads, writes, deletes, metadata updates, and wrapper operations.
- For backends that deserialize stored records, a corrupt record must not hide valid history or cause unrelated records to be deleted. Define consistent `get_items()` and `pop_item()` behavior that isolates the bad record and continues safely.
- Preserve creation timestamps and advance update timestamps deliberately. Backend-only identifiers and metadata must not leak into model-facing session items.
- Close only resources the backend owns. An injected engine, client, or connection remains caller-owned unless the public contract explicitly transfers ownership.
## Preparing Input Versus Persisting Input
`prepare_input_with_session()` returns two different values: the normalized input for the next model request and the subset of new-turn items that should be appended to the session.
- Existing history must not be re-appended as new input, even when `session_input_callback` deep-copies, reorders, filters, duplicates, or reconstructs items.
- A callback may change the model view without rewriting already stored history.
- Handoff and model-input filters may omit items from the next request while `session_step_items` retains the complete unfiltered sequence for history and observability.
- Normalize and deduplicate the model request and persistence candidates through the same canonical item helpers, then apply boundary-specific sanitization.
## Per-Turn Save and Resume
- Persist each completed turn, not only the final run result. Tool outputs and handoff items must survive a later error or interruption.
- `_current_turn_persisted_item_count` tracks which generated items have already been saved during streaming, retry, or resume. Count items after conversion and persistence filtering, not from the unsanitized source list.
- Resuming an interruption must save newly produced approval and tool output items without duplicating inputs or previously persisted outputs.
- Preserve full session items separately from filtered model input when updating `RunState` after resume.
- A guardrail trip must preserve the accepted user input while excluding speculative assistant or tool work that the tripwire invalidated. Test sequential and parallel guardrails in streaming and non-streaming modes because their persistence timing differs even though the resulting history must remain coherent.
## Retry Rewind
Retry cleanup is ownership-sensitive and best effort.
- Rewind only an exact serialized suffix that belongs to the failed attempt. Never scan backward and delete merely similar historical items.
- Verify the complete suffix before popping. If a pop fails or returns an unexpected item, restore already popped items in chronological order.
- Wait for backends with asynchronous cleanup semantics before starting the next retry when stale tail items could be observed.
- Do not forward live `RunContextWrapper` objects through retry rewind or compaction storage paths unless the session API explicitly owns that runtime context.
## Compaction Replacement
- Treat history replacement as a transaction: capture the prior state, apply the compacted state, and restore the prior state if clear or replacement fails.
- Defer response-based compaction while local tool outputs still need to be associated with the response chain.
- Choose input-based or previous-response-based compaction according to the actual state owner and `store` behavior; do not combine a local replay with a server-owned history chain.
- Compaction output is a run item and must follow the item lifecycle, session sanitization, and `RunState` rules rather than bypassing them as backend-only data.
## Review Checklist
1. Distinguish model input, new-turn persistence candidates, and full session history.
2. Test atomic failure, duplicate content, reordered callbacks, and filtered handoff input.
3. Test save behavior after tool execution, handoff, guardrail trip, interruption, and resume.
4. Prove retry rewind removes only the attempt-owned suffix and restores on partial failure.
5. Test compaction replacement failures without losing the previous history.
6. Test backend ordering, atomic batches, concurrent first writes, configured names and limits, corrupt records, and resource ownership.
## Sources
- `src/agents/memory/session.py`
- `src/agents/memory/session_settings.py`
- `src/agents/memory/sqlite_session.py`
- `src/agents/extensions/memory/`
- `src/agents/run_internal/session_persistence.py`
- `src/agents/run_internal/items.py`
- `src/agents/run_internal/run_steps.py`
- `tests/memory/`
- `tests/extensions/memory/`
- `tests/test_agent_runner.py`
- `tests/test_agent_runner_streamed.py`
@@ -0,0 +1,64 @@
# Tool Execution Lifecycle
Use this reference for changes to function-tool planning, approvals, tool guardrails, concurrency, cancellation, timeouts, hooks, error conversion, or resumed execution. Read [Tool identity and routing](tool-identity.md) when names, namespaces, lookup keys, or call IDs also change.
## Plan Before Side Effects
`process_model_response()` discovers executable work, but `tool_planning.py` decides which work may run now. Keep discovery, approval partitioning, and invocation as separate phases.
- Fresh and resumed turns need different plans. A resumed interruption must execute unresolved or newly approved work without rediscovering or rerunning completed calls.
- Approval state is authoritative once resolved. Do not call a dynamic `needs_approval` checker again for a call whose status is already approved or rejected.
- Deduplicate by invocation identity before execution while preserving model order for public call and output items. A repeated tool definition is not a repeated call, and a repeated call ID must not execute twice.
- Validate enabled tools and canonical lookup before side effects. A tool disabled after model output or absent from the resolved tool set must follow the configured missing-tool behavior rather than reaching a stale callable.
## Approval and Guardrail Ordering
- Pre-approval input guardrails are an early rejection optimization. They may run before an approval interruption, but input guardrails must run again immediately before invocation because state, policy, or arguments may have changed while approval was pending.
- Rechecking guardrails does not mean rechecking approval. Persisted approval decisions and rejection messages must remain attached to the same tool identity and call ID across `RunState` resume.
- Tool input guardrails finish before the local side effect. Tool output guardrails finish before output becomes accepted run state, model input, or persisted session history.
- The tool guardrail pipeline applies to `FunctionTool` invocation. Handoffs, hosted tools, built-in provider tools, and nested `Agent.as_tool()` runs have separate execution boundaries unless they explicitly opt into equivalent checks.
## Concurrency and Failure Semantics
SDK-side function-tool concurrency is independent of provider-side parallel tool-call generation. The provider controls how many calls appear in one response; `RunConfig.tool_execution.max_function_tool_concurrency` controls how many local function handlers run at once.
- Preserve model order in emitted outputs even when handlers complete out of order.
- Isolate sibling results. A cancelled or failed call must not discard outputs already produced by successful siblings.
- Distinguish cancellation of one tool handler from cancellation of the parent run. Tool-local cancellation can follow the configured tool failure policy; parent cancellation must propagate promptly instead of becoming model-visible tool output.
- `task.cancel()` is not terminal cleanup. On sibling failure, drain cancelled handlers and wait for post-invocation work within the bounded cleanup policy. On parent cancellation, cancel remaining tasks and attach result callbacks so late exceptions are observed without delaying cancellation indefinitely.
- Select and raise failures deterministically when several tasks fail, while still observing secondary failures. Do not let task-set iteration order or eager task execution change the public result.
## Invocation Boundary
- Decorated synchronous Python functions run through `asyncio.to_thread()` so they do not block the event loop. Async function tools run in the event loop and are the only decorated handlers that support SDK timeouts.
- Timeout handling and ordinary exception handling are distinct policies. `timeout_behavior` and `timeout_error_function` own timeout conversion; `failure_error_function=None` means ordinary exceptions propagate instead of becoming model-visible output.
- Tool start/end hooks and function spans surround the actual invocation once per call, including failure and cancellation paths. Do not emit a successful end state before output guardrails complete.
- Per-run resources such as resolved `Computer` implementations must be initialized and disposed by the run that acquired them.
- Nested `Agent.as_tool()` execution owns a nested run loop and nested resumable state. Scope cached nested state by the parent `RunState` and call identity, not only by the reusable agent or tool object.
- `AgentToolUseTracker` records tool use per agent identity. When `reset_tool_choice=True`, reset the effective next-turn tool choice after that agent uses a tool so `required` or a named choice cannot force an accidental loop; do not mutate the agent's declared settings across independent runs.
- Persist and restore the tool-use tracker across interruption and sandbox resume, including graphs with duplicate agent names, so resumed tool-choice behavior matches uninterrupted execution.
## Review Checklist
1. Trace fresh execution, approval interruption, approval rejection, and serialized resume separately.
2. Verify guardrail, approval, hook, trace, invocation, output, and persistence order.
3. Test sequential, bounded-concurrency, sibling failure, tool-local cancellation, and parent cancellation paths.
4. Test default, custom, and disabled failure conversion plus timeout behavior where applicable.
5. Confirm every started task and per-run resource reaches a deterministic terminal state.
## Sources
- `docs/running_agents.md`
- `docs/tools.md`
- `docs/guardrails.md`
- `docs/human_in_the_loop.md`
- `src/agents/run_internal/tool_planning.py`
- `src/agents/run_internal/tool_execution.py`
- `src/agents/tool.py`
- `tests/test_agent_runner.py`
- `tests/test_agent_runner_streamed.py`
- `tests/test_function_tool.py`
- `tests/test_tool_guardrails.py`
- `tests/test_tool_choice_reset.py`
- `tests/test_tool_use_tracker.py`
- `tests/test_run_state.py`
+71
View File
@@ -0,0 +1,71 @@
# Tool Identity and Routing
Use this reference for changes involving function-tool names, namespaces, provider wire names, lookup, approvals, tracing, MCP exposure, handoffs, or tool call IDs.
## Identity Layers
One tool can have several related identifiers. They are not interchangeable.
| Layer | Purpose | Canonical source |
|---|---|---|
| Public name | User- and model-facing tool name | `tool.name` |
| Explicit namespace | Distinguishes tools with the same public name | Tool namespace metadata |
| Qualified or dispatch name | Routes a model call to the intended tool | `namespace.name` when a namespace exists |
| Lookup key | Collision-free internal identity | `bare`, `namespaced`, or `deferred_top_level` tuple |
| Approval keys | Matches approval decisions to the intended tool | Canonical qualified and permitted alias keys |
| Trace name | Human-readable tracing label | Explicit trace name or public name |
| Call ID | Identifies one invocation, not the tool definition | Provider-supplied string |
Do not collapse these layers into one string or introduce local rules that only one caller uses.
## Canonical Helpers
Use `src/agents/_tool_identity.py` as the single implementation layer. Important helpers include:
- `get_function_tool_lookup_key_for_tool()` and `get_function_tool_lookup_key_for_call()` for canonical lookup identity.
- `get_function_tool_dispatch_name()` and `get_function_tool_qualified_name()` for routing and display surfaces that require qualification.
- `get_function_tool_approval_keys()` for approval matching.
- `get_function_tool_trace_name()` and `get_tool_call_trace_name()` for trace labels.
- `validate_function_tool_lookup_configuration()` and `build_function_tool_lookup_map()` for collision detection and dispatch maps.
- `normalize_tool_call_for_function_tool()` when provider payloads must be normalized for a selected tool.
If a proposed change bypasses these helpers, first prove that the target surface has intentionally different semantics.
## MCP and Handoff Rules
- `include_server_in_tool_names` is opt-in. Server-prefixed MCP names affect the model-exposed collision-safe name; they do not rename the original tool on the MCP server.
- Reserved names and enabled handoff names participate in collision avoidance only on the paths that expose generated model-facing names.
- `Handoff.default_tool_name()` is the source of default handoff tool names. Keep Realtime and non-Realtime handoff conversion aligned with it.
- Do not forward a naming option through a path where the downstream helper does not consult it and then describe the change as runtime behavior. Trace the complete caller-to-dispatch path first.
## Deferred Tool Search Rules
- A top-level `FunctionTool` with `defer_loading=True` and no explicit namespace uses the synthetic lookup key `("deferred_top_level", tool.name)`.
- The Responses wire shape for a loaded deferred top-level tool can look like `namespace == name`. Treat that namespace as reserved for the synthetic deferred tool-search path, not as a normal explicit namespace.
- `tool_namespace()` must reject an explicit namespace that equals the inner tool name. Otherwise a normal namespaced tool and a deferred top-level tool would have the same wire shape.
- Preserve the synthetic namespace on approval, interruption, tracing, and `ToolContext` surfaces when it identifies the model call, but dispatch the actual local tool through the deferred lookup key and strip the synthetic namespace before invoking the tool.
- Permanent approvals for deferred top-level tools should key by `deferred_top_level:<name>`. A bare-name approval alias is allowed only when no visible bare sibling can make that alias ambiguous.
## Tool Call ID Rules
- Preserve provider-supplied string call IDs across call items, approvals, outputs, retries, and serialized state.
- Do not coerce arbitrary values with `str(...)`. Canonical extractors return a call ID only when the source value is already a string.
- Do not use a call ID as a tool-definition identity or a tool name as an invocation identity.
- When a provider omits a stable identifier, use an existing fingerprint or dedupe policy for that item type instead of inventing a cross-provider ID contract.
## Review Checklist
1. Identify every identifier layer affected by the change.
2. Trace the actual runtime path from model-visible name to lookup, approval, invocation, output, and trace metadata.
3. Compare adjacent canonical helpers before adding conversion or fallback behavior.
4. Test collisions between bare, namespaced, deferred, MCP, local function, and handoff tools when applicable.
5. Require a regression test that fails on the base and proves the model-visible or dispatch behavior, not only an intermediate argument value.
## Sources
- `src/agents/_tool_identity.py`
- `src/agents/agent.py`
- `src/agents/mcp/`
- `src/agents/handoffs/__init__.py`
- `src/agents/run_internal/tool_execution.py`
- `src/agents/run_state.py`
+58
View File
@@ -0,0 +1,58 @@
# Tracing Lifecycle
Use this reference for changes to SDK trace or span context, processors, export, flush, shutdown, resumed trace state, or sensitive-data handling. Read [Realtime tracing architecture](realtime-tracing.md) before applying these client-side rules to Realtime server traces.
## Context and Parenting
- The current trace and span are held in `ContextVar` state. Async tasks inherit a snapshot when created; later changes in a child task do not rewrite the parent task's context.
- A context token must be reset in the context that created it. Start and finish ownership cannot be transferred between tasks without an explicit context boundary.
- A no-op trace or span cannot be a real parent. Propagate no-op behavior instead of exporting children with the sentinel `no-op` trace or span ID.
- Span factories should inherit trace metadata needed by processors, but they must not mutate the trace's caller-owned metadata mapping.
## Run and Resume Ownership
- A runner-created trace encloses run-loop-owned guardrails, model calls, tool execution, handoffs, session persistence, and error handling. Do not assume every completion callback or resource cleanup runs before trace finish; place newly traced cleanup explicitly inside the trace lifetime or create a deliberate separate trace/span context.
- An existing caller trace remains caller-owned. `Runner` may create child spans but must not finish or flush the caller's trace.
- `RunState` stores enough trace metadata to continue an interrupted run. Resume may reattach only when the trace ID was previously started in the process and the effective workflow name, group ID, metadata, and tracing key identity still match.
- Reattachment must not emit a duplicate trace-start event. If the saved state cannot prove a compatible live trace, create a normal trace according to the current run configuration instead of pretending to resume the old context.
- Tracing API keys are omitted from serialized `RunState` by default. A hash can verify that the caller supplied the same explicit key without persisting the secret; raw key persistence is opt-in.
## Processor and Export Isolation
- Trace processors are observability extensions and must not change application success. Catch processor callback, exporter, flush, and shutdown failures and report them as non-fatal.
- The default batch worker starts lazily on first queued item to avoid import-time thread and fork hazards. Keep top-level imports free of worker creation and shutdown-handler duplication.
- An exporter exception must not kill the batch worker and strand future traces. Drop or report the failed batch according to policy, then keep the worker usable.
- `flush_traces()` waits for queued and in-flight export work, so callers should invoke it after the trace closes when they require immediate delivery. It is not a substitute for finishing a partially built trace.
- Shutdown is best effort and deadline-aware. It should request exporter shutdown, interrupt retry backoff, drain within the remaining deadline, and return without changing the process exit code when an exporter blocks or a backend remains unavailable.
- Keep `TraceProvider.force_flush()` and `shutdown()` defaulting to no-ops for compatibility with custom providers that predate these lifecycle methods.
## Data Boundaries
- `trace_include_sensitive_data=False` controls captured span payload fields; it does not automatically sanitize exception objects, chaining, tracebacks, logs, or telemetry created elsewhere.
- Redaction must cover `__cause__`, `__context__`, formatter failures, and model-visible error conversion when an original exception carries tool arguments or provider payloads. `raise ... from None` changes display, not object retention.
- The OpenAI trace exporter owns ingest-specific payload sanitization such as field-size limits and supported usage keys. Custom processors should continue receiving the SDK's normal trace data unless their contract says otherwise.
- Per-run tracing keys, organization, and project routing must stay attached to the trace or exported item that selected them; do not let mutable global exporter state reroute an already-created trace.
## Review Checklist
1. Identify which task and context own each trace and span start, finish, and token reset.
2. Test success, exception, cancellation, interruption, serialized resume, full stream exhaustion, and explicit stream close.
3. Verify processor and exporter failures remain non-fatal and do not kill later export work.
4. Test flush and shutdown with queued work, in-flight export, retry backoff, and a blocking exporter.
5. Audit sensitive data through span payloads, exception chains, logs, and serialized state.
## Sources
- `docs/tracing.md`
- `src/agents/tracing/context.py`
- `src/agents/tracing/scope.py`
- `src/agents/tracing/traces.py`
- `src/agents/tracing/spans.py`
- `src/agents/tracing/provider.py`
- `src/agents/tracing/processors.py`
- `src/agents/tracing/setup.py`
- `src/agents/run_state.py`
- `tests/test_trace_processor.py`
- `tests/test_tracing.py`
- `tests/test_run_state.py`
- `tests/tracing/test_import_side_effects.py`
@@ -0,0 +1,56 @@
# Voice Pipeline Lifecycle
Use this reference for changes to `VoicePipeline`, `AudioInput`, `StreamedAudioInput`, STT sessions, TTS task ordering, voice lifecycle events, PCM framing, result streaming, or voice tracing. Realtime agents use a different live-session architecture; read [Realtime session lifecycle](realtime-session-lifecycle.md) for that path.
## Pipeline Ownership
`VoicePipeline` owns an STT-to-workflow-to-TTS producer task and returns a `StreamedAudioResult` that drives its observable completion.
- Static `AudioInput` produces one transcription and one workflow turn. `StreamedAudioInput` creates a long-lived transcription session and runs one workflow turn for each emitted transcript until the input or session ends.
- The multi-turn pipeline owns the transcription session and closes it in `finally` before marking output complete. Partial setup and workflow failure must not strand the STT connection or producer task.
- `workflow.on_start()` applies only to the streamed multi-turn path. Its failure is logged and skipped so the transcription session can still start; normal per-turn workflow failures are terminal and surface through the result stream.
- The SDK does not provide application-level interruption handling for `StreamedAudioInput`. Lifecycle events expose turn boundaries, but microphone muting, playback interruption, and barge-in policy remain application-owned.
## Text, Audio, and Event Ordering
- A workflow can yield multiple text fragments. The text splitter returns ready-to-synthesize text plus a remainder; synthesize non-empty ready text even when it is shorter than a default sentence threshold, and retain the remainder for the turn's final flush.
- TTS segment tasks may run concurrently, but `_ordered_tasks` and the dispatcher must emit their audio and lifecycle events in workflow text order rather than completion order.
- `turn_started` precedes audio for that turn. `turn_ended` is emitted only after the turn's final text remainder has been synthesized and its audio dispatched. `session_ended` follows all ordered segment queues and all turns.
- A `VoiceStreamEventError` terminates result streaming and the stored exception is raised after task cleanup. `session_ended` is a lifecycle marker, not proof of success; consumers must still observe the terminal exception from `stream()`.
- Consuming `StreamedAudioResult.stream()` is the public completion and error boundary. On normal `session_ended`, let the producer finish before cleanup so session close and trace end are not cancelled by result teardown.
## PCM and Caller Data
- PCM16 samples span two bytes. Preserve a trailing half-sample across TTS chunks, combine it with the next chunk, and pad only the final unmatched byte at end of segment.
- Apply `buffer_size` to TTS source chunks without changing sample order. Convert to float32 only after PCM16 framing is complete, then apply caller-provided `transform_data` to each emitted array.
- `AudioInput.to_base64()` and audio-file conversion must not mutate the caller's NumPy buffer when converting float input to PCM16.
- Empty input and empty text-splitter output are valid boundaries. They must not cause NumPy reduction errors, phantom TTS calls, or missing turn/session lifecycle events.
## Trace Lifetime and Data
- The pipeline trace stays active for the full asynchronous producer lifecycle, not only until `VoicePipeline.run()` returns its result object.
- Each output turn owns a speech-group span and each synthesized segment owns a child speech span. Finish the turn span after ordered audio dispatch and finish the pipeline trace after STT session close and output completion.
- Text and audio sensitivity are independent controls. `trace_include_sensitive_data` governs transcript and TTS text, while `trace_include_sensitive_audio_data` governs encoded audio payloads.
- Error paths must finish active speech spans and the enclosing trace without replacing the original pipeline exception.
## Review Checklist
1. Test static and streamed input, including STT setup failure, workflow failure, TTS failure, and transcription-session close.
2. Verify fragment concurrency never changes audio, turn, or session event order.
3. Test short splitter output, empty output, odd-byte chunks, cross-chunk sample boundaries, int16, and float32 conversion.
4. Consume the public result stream and verify terminal errors, task cleanup, session close, and trace-end order.
5. Confirm sensitive text and audio are independently omitted from trace payloads.
## Sources
- `docs/voice/pipeline.md`
- `docs/voice/tracing.md`
- `src/agents/voice/pipeline.py`
- `src/agents/voice/result.py`
- `src/agents/voice/input.py`
- `src/agents/voice/model.py`
- `src/agents/voice/models/openai_stt.py`
- `tests/voice/test_pipeline.py`
- `tests/voice/test_input.py`
- `tests/voice/test_openai_stt.py`
- `tests/voice/test_openai_tts.py`
@@ -0,0 +1,44 @@
---
name: code-change-verification
description: Run the mandatory verification stack when changes affect runtime code, tests, or build/test behavior in the OpenAI Agents Python repository.
---
# Code Change Verification
## Overview
Ensure work is only marked complete after formatting, linting, type checking, and tests pass. Use this skill when changes affect runtime code, tests, or build/test configuration. You can skip it for docs-only or repository metadata unless a user asks for the full stack.
## Quick start
1. Keep this skill at `./.agents/skills/code-change-verification` so it loads automatically for the repository.
2. macOS/Linux: `bash .agents/skills/code-change-verification/scripts/run.sh`.
3. Windows: `powershell -ExecutionPolicy Bypass -File .agents/skills/code-change-verification/scripts/run.ps1`.
4. The scripts run `make format` first, then run `make lint`, `make typecheck`, and `make tests` in parallel with fail-fast semantics.
5. While the parallel steps are still running, the scripts emit periodic heartbeat updates so you can tell that work is still in progress.
6. If any command fails, fix the issue, rerun the script, and report the failing output.
7. Confirm completion only when all commands succeed with no remaining issues.
## Environment setup
The verification scripts assume repository dependencies are already installed. Do not run `make sync` as part of every verification pass; use it for a fresh checkout, after dependency files change, or when dependency resolution fails before the checks start.
On Linux, some Python packages with native extensions may require system packages such as `libffi-dev`, Python development headers, or build tools. If verification cannot start because one of these packages is missing, treat it as a local environment setup issue. Install the missing dependency when possible, or report the failing command and missing dependency in the PR test plan before rerunning verification in a prepared environment.
## Manual workflow
- For a fresh checkout, or if dependencies are not installed or have changed, run `make sync` first to install dev requirements via `uv`.
- Run from the repository root with `make format` first, then `make lint`, `make typecheck`, and `make tests`.
- Do not skip steps; stop and fix issues immediately when a command fails.
- If you run the steps manually, you may parallelize `make lint`, `make typecheck`, and `make tests` after `make format` completes, but you must stop the remaining steps as soon as one fails.
- Re-run the full stack after applying fixes so the commands execute in the required order.
## Resources
### scripts/run.sh
- Executes `make format` first, then runs `make lint`, `make typecheck`, and `make tests` in parallel with fail-fast semantics from the repository root. It also emits periodic heartbeat updates while the parallel steps are still running. Prefer this entry point to preserve the required ordering while reducing total runtime.
### scripts/run.ps1
- Windows-friendly wrapper that runs the same sequence with `make format` first and the remaining steps in parallel with fail-fast semantics, plus periodic heartbeat updates while work is still running. Use from PowerShell with execution policy bypass if required by your environment.
@@ -0,0 +1,4 @@
interface:
display_name: "Code Change Verification"
short_description: "Run the required local verification stack"
default_prompt: "Use $code-change-verification to run the required local verification stack and report any failures."
@@ -0,0 +1,208 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Definition
$repoRoot = $null
try {
$repoRoot = (& git -C $scriptDir rev-parse --show-toplevel 2>$null)
} catch {
$repoRoot = $null
}
if (-not $repoRoot) {
$repoRoot = (Resolve-Path (Join-Path $scriptDir "..\\..\\..\\..")).Path
} else {
$repoRoot = ([string]$repoRoot).Trim()
}
Set-Location $repoRoot
$logDir = Join-Path ([System.IO.Path]::GetTempPath()) ("code-change-verification-" + [System.Guid]::NewGuid().ToString("N"))
New-Item -ItemType Directory -Path $logDir | Out-Null
$steps = New-Object System.Collections.Generic.List[object]
$heartbeatIntervalSeconds = 10
if ($env:CODE_CHANGE_VERIFICATION_HEARTBEAT_SECONDS) {
$heartbeatIntervalSeconds = [int]$env:CODE_CHANGE_VERIFICATION_HEARTBEAT_SECONDS
}
function Resolve-MakeInvocation {
$command = Get-Command make -ErrorAction Stop
while ($command.CommandType -eq [System.Management.Automation.CommandTypes]::Alias) {
$command = $command.ResolvedCommand
}
if ($command.CommandType -in @(
[System.Management.Automation.CommandTypes]::Application,
[System.Management.Automation.CommandTypes]::ExternalScript
)) {
$commandPath = if ($command.Path) { $command.Path } else { $command.Source }
return [PSCustomObject]@{
FilePath = $commandPath
ArgumentList = @()
}
}
if ($command.CommandType -eq [System.Management.Automation.CommandTypes]::Function) {
$shellPath = (Get-Process -Id $PID).Path
if (-not $shellPath) {
throw "Unable to resolve the current PowerShell executable for make wrapper launches."
}
$wrapperPath = Join-Path $logDir "invoke-make.ps1"
$escapedRepoRoot = $repoRoot -replace "'", "''"
$wrapperTemplate = @'
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
Set-Location -LiteralPath '{0}'
function global:make {{
{1}
}}
& make @args
exit $LASTEXITCODE
'@
$wrapperScript = $wrapperTemplate -f $escapedRepoRoot, $command.Definition.TrimEnd()
Set-Content -Path $wrapperPath -Value $wrapperScript -Encoding UTF8
return [PSCustomObject]@{
FilePath = $shellPath
ArgumentList = @("-NoLogo", "-NoProfile", "-File", $wrapperPath)
}
}
throw "code-change-verification: make must resolve to an application, script, alias, or function."
}
$script:MakeInvocation = Resolve-MakeInvocation
function Invoke-MakeStep {
param(
[Parameter(Mandatory = $true)][string]$Step
)
Write-Host "Running make $Step..."
& $script:MakeInvocation.FilePath @($script:MakeInvocation.ArgumentList + $Step)
if ($LASTEXITCODE -ne 0) {
Write-Host "code-change-verification: make $Step failed with exit code $LASTEXITCODE."
return $LASTEXITCODE
}
return 0
}
function Start-MakeStep {
param(
[Parameter(Mandatory = $true)][string]$Step
)
$stdoutLogPath = Join-Path $logDir "$Step.stdout.log"
$stderrLogPath = Join-Path $logDir "$Step.stderr.log"
Write-Host "Running make $Step..."
$process = Start-Process -FilePath $script:MakeInvocation.FilePath -ArgumentList @($script:MakeInvocation.ArgumentList + $Step) -RedirectStandardOutput $stdoutLogPath -RedirectStandardError $stderrLogPath -PassThru
$steps.Add([PSCustomObject]@{
Name = $Step
Process = $process
StdoutLogPath = $stdoutLogPath
StderrLogPath = $stderrLogPath
StartTime = Get-Date
})
}
function Stop-RunningSteps {
foreach ($step in $steps) {
if ($null -eq $step.Process) {
continue
}
& taskkill /PID $step.Process.Id /T /F *> $null
}
foreach ($step in $steps) {
if ($null -eq $step.Process) {
continue
}
try {
$step.Process.WaitForExit()
} catch {
}
}
}
function Wait-ForParallelSteps {
$pending = New-Object System.Collections.Generic.List[object]
foreach ($step in $steps) {
$pending.Add($step)
}
$nextHeartbeatAt = (Get-Date).AddSeconds($heartbeatIntervalSeconds)
while ($pending.Count -gt 0) {
foreach ($step in @($pending)) {
$step.Process.Refresh()
if (-not $step.Process.HasExited) {
continue
}
$duration = [int]((Get-Date) - $step.StartTime).TotalSeconds
if ($step.Process.ExitCode -eq 0) {
Write-Host "make $($step.Name) passed in ${duration}s."
[void]$pending.Remove($step)
continue
}
Write-Host "code-change-verification: make $($step.Name) failed with exit code $($step.Process.ExitCode) after ${duration}s."
if (Test-Path $step.StderrLogPath) {
Write-Host "--- $($step.Name) stderr log (last 80 lines) ---"
Get-Content $step.StderrLogPath -Tail 80
}
if (Test-Path $step.StdoutLogPath) {
Write-Host "--- $($step.Name) stdout log (last 80 lines) ---"
Get-Content $step.StdoutLogPath -Tail 80
}
Stop-RunningSteps
return $step.Process.ExitCode
}
if ($pending.Count -gt 0) {
if ((Get-Date) -ge $nextHeartbeatAt) {
$running = @()
foreach ($step in $pending) {
$elapsed = [int]((Get-Date) - $step.StartTime).TotalSeconds
$running += "$($step.Name) (${elapsed}s)"
}
Write-Host ("code-change-verification: still running: " + ($running -join ", ") + ".")
$nextHeartbeatAt = (Get-Date).AddSeconds($heartbeatIntervalSeconds)
}
Start-Sleep -Seconds 1
}
}
return 0
}
$exitCode = 0
try {
$exitCode = Invoke-MakeStep -Step "format"
if ($exitCode -eq 0) {
Write-Host "Running make lint, make typecheck, and make tests in parallel..."
Start-MakeStep -Step "lint"
Start-MakeStep -Step "typecheck"
Start-MakeStep -Step "tests"
$exitCode = Wait-ForParallelSteps
}
} finally {
Stop-RunningSteps
Remove-Item $logDir -Recurse -Force -ErrorAction SilentlyContinue
}
if ($exitCode -ne 0) {
exit $exitCode
}
Write-Host "code-change-verification: all commands passed."
+398
View File
@@ -0,0 +1,398 @@
#!/usr/bin/env bash
# Fail fast on any error or undefined variable.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if command -v git >/dev/null 2>&1; then
REPO_ROOT="$(git -C "${SCRIPT_DIR}" rev-parse --show-toplevel 2>/dev/null || true)"
fi
REPO_ROOT="${REPO_ROOT:-$(cd "${SCRIPT_DIR}/../../../.." && pwd)}"
cd "${REPO_ROOT}"
LOG_DIR="$(mktemp -d "${TMPDIR:-/tmp}/code-change-verification.XXXXXX")"
STATUS_PIPE="${LOG_DIR}/status.fifo"
HEARTBEAT_INTERVAL_SECONDS="${CODE_CHANGE_VERIFICATION_HEARTBEAT_SECONDS:-10}"
declare -a STEP_LAUNCHER=()
declare -a STEP_PIDS=()
declare -a STEP_NAMES=()
declare -a STEP_LOGS=()
declare -a STEP_STARTS=()
RUNNING_STEPS=0
EXIT_STATUS=0
resolve_executable_path() {
local name="$1"
type -P "${name}" 2>/dev/null || true
}
configure_step_launcher() {
local perl_path=""
local python_path=""
local uv_path=""
perl_path="$(resolve_executable_path perl)"
if [ -n "${perl_path}" ]; then
STEP_LAUNCHER=("${perl_path}" -MPOSIX=setsid -e 'setsid() or die $!; exec @ARGV')
return 0
fi
python_path="$(resolve_executable_path python3)"
if [ -z "${python_path}" ]; then
python_path="$(resolve_executable_path python)"
fi
if [ -n "${python_path}" ]; then
STEP_LAUNCHER=("${python_path}" -c 'import os, sys; os.setsid(); os.execvp(sys.argv[1], sys.argv[1:])')
return 0
fi
uv_path="$(resolve_executable_path uv)"
if [ -n "${uv_path}" ]; then
STEP_LAUNCHER=("${uv_path}" run --no-sync python -c 'import os, sys; os.setsid(); os.execvp(sys.argv[1], sys.argv[1:])')
return 0
fi
echo "code-change-verification: perl, python3, python, or uv is required to manage parallel step process groups." >&2
exit 1
}
configure_step_launcher
mkfifo "${STATUS_PIPE}"
exec 3<> "${STATUS_PIPE}"
cleanup() {
local trap_status="$?"
local status="${EXIT_STATUS}"
if [ "${status}" -eq 0 ]; then
status="${trap_status}"
fi
if [ "${#STEP_PIDS[@]}" -gt 0 ]; then
stop_running_steps
fi
exec 3>&- 3<&- || true
rm -rf "${LOG_DIR}"
exit "${status}"
}
on_interrupt() {
EXIT_STATUS=130
exit 130
}
on_terminate() {
EXIT_STATUS=143
exit 143
}
stop_running_steps() {
local pid=""
if [ "${#STEP_PIDS[@]}" -eq 0 ]; then
return
fi
for pid in "${STEP_PIDS[@]}"; do
if [ -n "${pid}" ]; then
kill -TERM -- "-${pid}" 2>/dev/null || true
fi
done
sleep 1
for pid in "${STEP_PIDS[@]}"; do
if [ -n "${pid}" ]; then
# A process group can remain alive after its leader exits, so escalate by group id unconditionally.
kill -KILL -- "-${pid}" 2>/dev/null || true
fi
done
for pid in "${STEP_PIDS[@]}"; do
if [ -n "${pid}" ]; then
wait "${pid}" 2>/dev/null || true
fi
done
STEP_PIDS=()
STEP_NAMES=()
STEP_LOGS=()
STEP_STARTS=()
RUNNING_STEPS=0
}
find_step_index() {
local target_name="$1"
local idx=""
for idx in "${!STEP_NAMES[@]}"; do
if [ "${STEP_NAMES[$idx]}" = "${target_name}" ]; then
echo "${idx}"
return 0
fi
done
return 1
}
clear_step() {
local idx="$1"
STEP_PIDS[$idx]=""
STEP_NAMES[$idx]=""
STEP_LOGS[$idx]=""
STEP_STARTS[$idx]=""
RUNNING_STEPS=$((RUNNING_STEPS - 1))
}
step_pid_is_alive() {
local pid="$1"
local state=""
if ! kill -0 "${pid}" 2>/dev/null; then
return 1
fi
state="$(ps -o stat= -p "${pid}" 2>/dev/null | tr -d '[:space:]')"
case "${state}" in
Z*|z*|"")
return 1
;;
esac
return 0
}
print_heartbeat() {
local now
local idx=""
local name=""
local start_time=""
local elapsed=""
local running=""
now=$(date +%s)
for idx in "${!STEP_NAMES[@]}"; do
name="${STEP_NAMES[$idx]}"
start_time="${STEP_STARTS[$idx]}"
if [ -z "${name}" ]; then
continue
fi
elapsed=$((now - start_time))
if [ -n "${running}" ]; then
running="${running}, "
fi
running="${running}${name} (${elapsed}s)"
done
if [ -n "${running}" ]; then
echo "code-change-verification: still running: ${running}."
fi
}
start_step() {
local name="$1"
shift
local log_file="${LOG_DIR}/${name}.log"
echo "Running make ${name}..."
: > "${log_file}"
# Start each step in its own process group so fail-fast cleanup can stop pytest worker trees too.
"${STEP_LAUNCHER[@]}" \
bash -c '
step_name="$1"
log_file="$2"
status_pipe="$3"
shift 3
if "$@" >"$log_file" 2>&1; then
status=0
else
status=$?
fi
printf "%s\t%s\n" "$step_name" "$status" >"$status_pipe"
exit "$status"
' \
bash "${name}" "${log_file}" "${STATUS_PIPE}" "$@" &
STEP_PIDS+=("$!")
STEP_NAMES+=("${name}")
STEP_LOGS+=("${log_file}")
STEP_STARTS+=("$(date +%s)")
RUNNING_STEPS=$((RUNNING_STEPS + 1))
}
finish_step() {
local name="$1"
local status="$2"
local idx=""
local pid=""
local log_file=""
local start_time=""
local now
idx="$(find_step_index "${name}")"
pid="${STEP_PIDS[$idx]}"
log_file="${STEP_LOGS[$idx]}"
start_time="${STEP_STARTS[$idx]}"
now=$(date +%s)
wait "${pid}" 2>/dev/null || true
if [ "${status}" -eq 0 ]; then
clear_step "${idx}"
echo "make ${name} passed in $((now - start_time))s."
return 0
fi
echo "code-change-verification: make ${name} failed with exit code ${status} after $((now - start_time))s." >&2
echo "--- ${name} log (last 80 lines) ---" >&2
tail -n 80 "${log_file}" >&2 || true
stop_running_steps
return "${status}"
}
check_for_missing_reporters() {
local idx=""
local pid=""
local name=""
local log_file=""
local start_time=""
local now
local step_status=0
for idx in "${!STEP_PIDS[@]}"; do
pid="${STEP_PIDS[$idx]}"
if [ -z "${pid}" ] || step_pid_is_alive "${pid}"; then
continue
fi
if try_finish_step_from_status_pipe 1; then
if [ "${STATUS_PIPE_DRAINED}" -eq 1 ]; then
return 0
fi
else
step_status=$?
return "${step_status}"
fi
name="${STEP_NAMES[$idx]}"
log_file="${STEP_LOGS[$idx]}"
start_time="${STEP_STARTS[$idx]}"
now=$(date +%s)
set +e
wait "${pid}" 2>/dev/null
step_status=$?
set -e
if [ "${step_status}" -eq 0 ]; then
finish_step "${name}" 0
return 0
fi
echo "code-change-verification: make ${name} exited before reporting completion status after $((now - start_time))s." >&2
echo "--- ${name} log (last 80 lines) ---" >&2
tail -n 80 "${log_file}" >&2 || true
stop_running_steps
return "${step_status}"
done
return 0
}
STATUS_PIPE_DRAINED=0
try_finish_step_from_status_pipe() {
local timeout="$1"
local name=""
local status=""
local step_status=0
STATUS_PIPE_DRAINED=0
if ! IFS=$'\t' read -r -t "${timeout}" name status <&3; then
return 0
fi
STATUS_PIPE_DRAINED=1
finish_step "${name}" "${status}"
step_status=$?
if [ "${step_status}" -ne 0 ]; then
return "${step_status}"
fi
return 0
}
wait_for_parallel_steps() {
local name=""
local status=""
local step_status=""
local next_heartbeat_at
local now
next_heartbeat_at=$(( $(date +%s) + HEARTBEAT_INTERVAL_SECONDS ))
while [ "${RUNNING_STEPS}" -gt 0 ]; do
if try_finish_step_from_status_pipe 1; then
if [ "${STATUS_PIPE_DRAINED}" -eq 1 ]; then
continue
fi
else
step_status=$?
if [ "${step_status}" -ne 0 ]; then
return "${step_status}"
fi
continue
fi
check_for_missing_reporters
step_status=$?
if [ "${step_status}" -ne 0 ]; then
return "${step_status}"
fi
now=$(date +%s)
if [ "${now}" -ge "${next_heartbeat_at}" ]; then
print_heartbeat
next_heartbeat_at=$((now + HEARTBEAT_INTERVAL_SECONDS))
fi
done
}
trap cleanup EXIT
trap on_interrupt INT
trap on_terminate TERM
echo "Running make format..."
set +e
make format
EXIT_STATUS=$?
set -e
if [ "${EXIT_STATUS}" -ne 0 ]; then
exit "${EXIT_STATUS}"
fi
echo "Running make lint, make typecheck, and make tests in parallel..."
start_step "lint" make lint
start_step "typecheck" make typecheck
start_step "tests" make tests
set +e
wait_for_parallel_steps
EXIT_STATUS=$?
set -e
if [ "${EXIT_STATUS}" -ne 0 ]; then
exit "${EXIT_STATUS}"
fi
trap - EXIT INT TERM
exec 3>&- 3<&-
rm -rf "${LOG_DIR}"
echo "code-change-verification: all commands passed."
+76
View File
@@ -0,0 +1,76 @@
---
name: docs-sync
description: Analyze main branch implementation and configuration to find missing, incorrect, or outdated documentation in docs/. Use when asked to audit doc coverage, sync docs with code, or propose doc updates/structure changes. Only update English docs under docs/** and never touch translated docs under docs/ja, docs/ko, or docs/zh. Provide a report and ask for approval before editing docs.
---
# Docs Sync
## Overview
Identify doc coverage gaps and inaccuracies by comparing main branch features and configuration options against the current docs structure, then propose targeted improvements.
## Workflow
1. Confirm scope and base branch
- Identify the current branch and default branch (usually `main`).
- Prefer analyzing the current branch to keep work aligned with in-flight changes.
- If the current branch is not `main`, analyze only the diff vs `main` to scope doc updates.
- Avoid switching branches if it would disrupt local changes; use `git show main:<path>` or `git worktree add` when needed.
2. Build a feature inventory from the selected scope
- If on `main`: inventory the full surface area and review docs comprehensively.
- If not on `main`: inventory only changes vs `main` (feature additions/changes/removals).
- Focus on user-facing behavior: public exports, configuration options, environment variables, CLI commands, default values, and documented runtime behaviors.
- Capture evidence for each item (file path + symbol/setting).
- Use targeted search to find option types and feature flags (for example: `rg "Settings"`, `rg "Config"`, `rg "os.environ"`, `rg "OPENAI_"`).
- When the topic involves OpenAI platform features, invoke `$openai-knowledge` to pull current details from the OpenAI Developer Docs MCP server instead of guessing, while treating the SDK source code as the source of truth when discrepancies appear.
3. Doc-first pass: review existing pages
- Walk each relevant page under `docs/` (excluding `docs/ja`, `docs/ko`, and `docs/zh`).
- Identify missing mentions of important, supported options (opt-in flags, env vars), customization points, or new features from `src/agents/` and `examples/`.
- Propose additions where users would reasonably expect to find them on that page.
4. Code-first pass: map features to docs
- Review the current docs information architecture under `docs/` and `mkdocs.yml`.
- Determine the best page/section for each feature based on existing patterns and the API reference structure under `docs/ref`.
- Identify features that lack any doc page or have a page but no corresponding content.
- Note when a structural adjustment would improve discoverability.
- When improving `docs/ref/*` pages, treat the corresponding docstrings/comments in `src/` as the source of truth. Prefer updating those code comments so regenerated reference docs stay correct, instead of hand-editing the generated pages.
5. Detect gaps and inaccuracies
- **Missing**: features/configs present in main but absent in docs.
- **Incorrect/outdated**: names, defaults, or behaviors that diverge from main.
- **Structural issues** (optional): pages overloaded, missing overviews, or mis-grouped topics.
6. Produce a Docs Sync Report and ask for approval
- Provide a clear report with evidence, suggested doc locations, and proposed edits.
- Ask the user whether to proceed with doc updates.
7. If approved, apply changes (English only)
- Edit only English docs in `docs/**`.
- Do **not** edit `docs/ja`, `docs/ko`, or `docs/zh`.
- Keep changes aligned with the existing docs style and navigation.
- Update `mkdocs.yml` when adding or renaming pages.
- Build docs with `make build-docs` after edits to verify the docs site still builds.
## Output format
Use this template when reporting findings:
Docs Sync Report
- Doc-first findings
- Page + missing content -> evidence + suggested insertion point
- Code-first gaps
- Feature + evidence -> suggested doc page/section (or missing page)
- Incorrect or outdated docs
- Doc file + issue + correct info + evidence
- Structural suggestions (optional)
- Proposed change + rationale
- Proposed edits
- Doc file -> concise change summary
- Questions for the user
## References
- `references/doc-coverage-checklist.md`
@@ -0,0 +1,4 @@
interface:
display_name: "Docs Sync"
short_description: "Audit docs coverage and propose targeted updates"
default_prompt: "Use $docs-sync to audit the current branch against docs/ and propose targeted documentation updates."
@@ -0,0 +1,56 @@
# Doc Coverage Checklist
Use this checklist to scan the selected scope (main = comprehensive, or current-branch diff) and validate documentation coverage.
## Feature inventory targets
- Public exports: classes, functions, types, and module entry points.
- Configuration options: `*Settings` types, default config objects, and builder patterns.
- Environment variables or runtime flags.
- CLI commands, scripts, and example entry points that define supported usage.
- User-facing behaviors: retry, timeouts, streaming, errors, logging, telemetry, and data handling.
- Deprecations, removals, or renamed settings.
## Doc-first pass (page-by-page)
- Review each relevant English page (excluding `docs/ja`, `docs/ko`, and `docs/zh`).
- Look for missing opt-in flags, env vars, or customization options that the page implies.
- Add new features that belong on that page based on user intent and navigation.
## Code-first pass (feature inventory)
- Map features to the closest existing page based on the docs navigation in `mkdocs.yml`.
- Prefer updating existing pages over creating new ones unless the topic is clearly new.
- Use conceptual pages for cross-cutting concerns (auth, errors, streaming, tracing, tools).
- Keep quick-start flows minimal; move advanced details into deeper pages.
## Evidence capture
- Record the main-branch file path and symbol/setting name.
- Note defaults or behavior-critical details for accuracy checks.
- Avoid large code dumps; a short identifier is enough.
## Red flags for outdated or incorrect docs
- Option names/types no longer exist or differ from code.
- Default values or allowed ranges do not match implementation.
- Features removed in code but still documented.
- New behaviors introduced without corresponding docs updates.
## When to propose structural changes
- A page mixes unrelated audiences (quick-start + deep reference) without clear separation.
- Multiple pages duplicate the same concept without cross-links.
- New feature areas have no obvious home in the nav structure.
## Diff mode guidance (current branch vs main)
- Focus only on changed behavior: new exports/options, modified defaults, removed features, or renamed settings.
- Use `git diff main...HEAD` (or equivalent) to constrain analysis.
- Document removals explicitly so docs can be pruned if needed.
## Patch guidance
- Keep edits scoped and aligned with existing tone and format.
- Update cross-links when moving or renaming sections.
- Leave translated docs untouched; English-only updates.
+86
View File
@@ -0,0 +1,86 @@
---
name: examples-auto-run
description: Run python examples in auto mode with logging, rerun helpers, and background control.
---
# examples-auto-run
## What it does
- Runs `uv run examples/run_examples.py` with:
- Optional dependency extras enabled by default:
`litellm`, `any-llm`, `sqlalchemy`, `redis`, `blaxel`, `modal`, `runloop`, and `temporal`.
- `EXAMPLES_INTERACTIVE_MODE=auto` (auto-input/auto-approve).
- Per-example logs under `.tmp/examples-start-logs/`.
- Main summary log path passed via `--main-log` (also under `.tmp/examples-start-logs/`).
- Generates a rerun list of failures at `.tmp/examples-rerun.txt` when `--write-rerun` is set.
- Provides start/stop/status/logs/tail/collect/rerun helpers via `run.sh`.
- Background option keeps the process running with a pidfile; `stop` cleans it up.
## Usage
```bash
# Start (auto mode; interactive included by default)
.agents/skills/examples-auto-run/scripts/run.sh start [extra args to run_examples.py]
# Examples:
.agents/skills/examples-auto-run/scripts/run.sh start --filter basic
.agents/skills/examples-auto-run/scripts/run.sh start --include-server --include-audio
# Check status
.agents/skills/examples-auto-run/scripts/run.sh status
# Stop running job
.agents/skills/examples-auto-run/scripts/run.sh stop
# List logs
.agents/skills/examples-auto-run/scripts/run.sh logs
# Tail latest log (or specify one)
.agents/skills/examples-auto-run/scripts/run.sh tail
.agents/skills/examples-auto-run/scripts/run.sh tail main_20260113-123000.log
# Collect rerun list from a main log (defaults to latest main_*.log)
.agents/skills/examples-auto-run/scripts/run.sh collect
# Rerun only failed entries from rerun file (auto mode)
.agents/skills/examples-auto-run/scripts/run.sh rerun
```
## Codex execution environment
- When Codex invokes `start` or `rerun`, run the command outside the Codex sandbox by default (`sandbox_permissions=require_escalated`). Many examples start nested sandboxes, browsers, npm helpers, or cloud/local service processes; running from inside the Codex sandbox can produce environment-only failures such as `sandbox-exec: sandbox_apply: Operation not permitted`, Playwright cache permission errors, or npm cache permission errors.
- Use sandboxed execution only when the user explicitly asks for it or when running a narrow dry-run / log inspection command that does not execute examples.
## Defaults (overridable via env)
- `EXAMPLES_INTERACTIVE_MODE=auto`
- `EXAMPLES_INCLUDE_INTERACTIVE=1`
- `EXAMPLES_INCLUDE_SERVER=0`
- `EXAMPLES_INCLUDE_AUDIO=0`
- `EXAMPLES_INCLUDE_EXTERNAL=0`
- `EXAMPLES_UV_EXTRAS="litellm any-llm sqlalchemy redis blaxel modal runloop temporal"` (set to an empty string to disable extras)
- Auto-approvals in auto mode: `APPLY_PATCH_AUTO_APPROVE=1`, `SHELL_AUTO_APPROVE=1`, `AUTO_APPROVE_MCP=1`
## Log locations
- Main logs: `.tmp/examples-start-logs/main_*.log`
- Per-example logs (from `run_examples.py`): `.tmp/examples-start-logs/<module_path>.log`
- Rerun list: `.tmp/examples-rerun.txt`
- Stdout logs: `.tmp/examples-start-logs/stdout_*.log`
## Notes
- The runner delegates to `uv run --extra ... examples/run_examples.py`, which already writes per-example logs and supports `--collect`, `--rerun-file`, and `--print-auto-skip`.
- `examples/sandbox/extensions/vercel_runner.py` is temporarily excluded from auto runs due to credential issues. Do not force-run it until the credential setup is fixed.
- `start` uses `--write-rerun` so failures are captured automatically.
- If `.tmp/examples-rerun.txt` exists and is non-empty, invoking the skill with no args runs `rerun` by default.
## Behavioral validation (Codex/LLM responsibility)
The runner does not perform any automated behavioral validation. After every foreground `start` or `rerun`, **Codex must manually validate** all exit-0 entries:
1. Read the example source (and comments) to infer intended flow, tools used, and expected key outputs.
2. Open the matching per-example log under `.tmp/examples-start-logs/`.
3. Confirm the intended actions/results occurred; flag omissions or divergences.
4. Do this for **all passed examples**, not just a sample.
5. Report immediately after the run with concise citations to the exact log lines that justify the validation.
@@ -0,0 +1,4 @@
interface:
display_name: "Examples Auto Run"
short_description: "Run examples in auto mode with logs and rerun helpers"
default_prompt: "Use $examples-auto-run to run the repo examples in auto mode, collect logs, and summarize any failures."
+236
View File
@@ -0,0 +1,236 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"
PID_FILE="$ROOT/.tmp/examples-auto-run.pid"
LOG_DIR="$ROOT/.tmp/examples-start-logs"
RERUN_FILE="$ROOT/.tmp/examples-rerun.txt"
DEFAULT_UV_EXTRAS="litellm any-llm sqlalchemy redis blaxel modal runloop temporal"
build_uv_prefix() {
UV_RUN=(uv run)
local extras_value
if [[ -n "${EXAMPLES_UV_EXTRAS+x}" ]]; then
extras_value="$EXAMPLES_UV_EXTRAS"
else
extras_value="$DEFAULT_UV_EXTRAS"
fi
local extra
for extra in $extras_value; do
UV_RUN+=(--extra "$extra")
done
export EXAMPLES_UV_EXTRAS="$extras_value"
}
ensure_dirs() {
mkdir -p "$LOG_DIR" "$ROOT/.tmp"
}
is_running() {
local pid="$1"
[[ -n "$pid" ]] && ps -p "$pid" >/dev/null 2>&1
}
cmd_start() {
ensure_dirs
local background=0
if [[ "${1:-}" == "--background" ]]; then
background=1
shift
fi
local ts main_log stdout_log
ts="$(date +%Y%m%d-%H%M%S)"
main_log="$LOG_DIR/main_${ts}.log"
stdout_log="$LOG_DIR/stdout_${ts}.log"
build_uv_prefix
local run_cmd=(
"${UV_RUN[@]}" examples/run_examples.py
--auto-mode
--write-rerun
--main-log "$main_log"
--logs-dir "$LOG_DIR"
)
if [[ "$background" -eq 1 ]]; then
if [[ -f "$PID_FILE" ]]; then
local pid
pid="$(cat "$PID_FILE" 2>/dev/null || true)"
if is_running "$pid"; then
echo "examples/run_examples.py already running (pid=$pid)."
exit 1
fi
fi
(
trap '' HUP
export EXAMPLES_INTERACTIVE_MODE="${EXAMPLES_INTERACTIVE_MODE:-auto}"
export APPLY_PATCH_AUTO_APPROVE="${APPLY_PATCH_AUTO_APPROVE:-1}"
export SHELL_AUTO_APPROVE="${SHELL_AUTO_APPROVE:-1}"
export AUTO_APPROVE_MCP="${AUTO_APPROVE_MCP:-1}"
export EXAMPLES_INCLUDE_INTERACTIVE="${EXAMPLES_INCLUDE_INTERACTIVE:-1}"
export EXAMPLES_INCLUDE_SERVER="${EXAMPLES_INCLUDE_SERVER:-0}"
export EXAMPLES_INCLUDE_AUDIO="${EXAMPLES_INCLUDE_AUDIO:-0}"
export EXAMPLES_INCLUDE_EXTERNAL="${EXAMPLES_INCLUDE_EXTERNAL:-0}"
cd "$ROOT"
exec "${run_cmd[@]}" "$@" > >(tee "$stdout_log") 2>&1
) &
local pid=$!
echo "$pid" >"$PID_FILE"
echo "Started run_examples.py (pid=$pid)"
echo "Main log: $main_log"
echo "Stdout log: $stdout_log"
echo "Run '.agents/skills/examples-auto-run/scripts/run.sh validate \"$main_log\"' after it finishes."
return 0
fi
export EXAMPLES_INTERACTIVE_MODE="${EXAMPLES_INTERACTIVE_MODE:-auto}"
export APPLY_PATCH_AUTO_APPROVE="${APPLY_PATCH_AUTO_APPROVE:-1}"
export SHELL_AUTO_APPROVE="${SHELL_AUTO_APPROVE:-1}"
export AUTO_APPROVE_MCP="${AUTO_APPROVE_MCP:-1}"
export EXAMPLES_INCLUDE_INTERACTIVE="${EXAMPLES_INCLUDE_INTERACTIVE:-1}"
export EXAMPLES_INCLUDE_SERVER="${EXAMPLES_INCLUDE_SERVER:-0}"
export EXAMPLES_INCLUDE_AUDIO="${EXAMPLES_INCLUDE_AUDIO:-0}"
export EXAMPLES_INCLUDE_EXTERNAL="${EXAMPLES_INCLUDE_EXTERNAL:-0}"
cd "$ROOT"
set +e
"${run_cmd[@]}" "$@" 2>&1 | tee "$stdout_log"
local run_status=${PIPESTATUS[0]}
set -e
return "$run_status"
}
cmd_stop() {
if [[ ! -f "$PID_FILE" ]]; then
echo "No pid file; nothing to stop."
return 0
fi
local pid
pid="$(cat "$PID_FILE" 2>/dev/null || true)"
if [[ -z "$pid" ]]; then
rm -f "$PID_FILE"
echo "Pid file empty; cleaned."
return 0
fi
if ! is_running "$pid"; then
rm -f "$PID_FILE"
echo "Process $pid not running; cleaned pid file."
return 0
fi
echo "Stopping pid $pid ..."
kill "$pid" 2>/dev/null || true
sleep 1
if is_running "$pid"; then
echo "Sending SIGKILL to $pid ..."
kill -9 "$pid" 2>/dev/null || true
fi
rm -f "$PID_FILE"
echo "Stopped."
}
cmd_status() {
if [[ -f "$PID_FILE" ]]; then
local pid
pid="$(cat "$PID_FILE" 2>/dev/null || true)"
if is_running "$pid"; then
echo "Running (pid=$pid)"
return 0
fi
fi
echo "Not running."
}
cmd_logs() {
ensure_dirs
ls -1t "$LOG_DIR"
}
cmd_tail() {
ensure_dirs
local file="${1:-}"
if [[ -z "$file" ]]; then
file="$(ls -1t "$LOG_DIR" | head -n1)"
fi
if [[ -z "$file" ]]; then
echo "No log files yet."
exit 1
fi
tail -f "$LOG_DIR/$file"
}
collect_rerun() {
ensure_dirs
local log_file="${1:-}"
if [[ -z "$log_file" ]]; then
log_file="$(ls -1t "$LOG_DIR"/main_*.log 2>/dev/null | head -n1)"
fi
if [[ -z "$log_file" ]] || [[ ! -f "$log_file" ]]; then
echo "No main log file found."
exit 1
fi
cd "$ROOT"
build_uv_prefix
"${UV_RUN[@]}" examples/run_examples.py --collect "$log_file" --output "$RERUN_FILE"
}
cmd_rerun() {
ensure_dirs
local file="${1:-$RERUN_FILE}"
if [[ ! -s "$file" ]]; then
echo "Rerun list is empty: $file"
exit 0
fi
local ts main_log stdout_log
ts="$(date +%Y%m%d-%H%M%S)"
main_log="$LOG_DIR/main_${ts}.log"
stdout_log="$LOG_DIR/stdout_${ts}.log"
cd "$ROOT"
export EXAMPLES_INTERACTIVE_MODE="${EXAMPLES_INTERACTIVE_MODE:-auto}"
export APPLY_PATCH_AUTO_APPROVE="${APPLY_PATCH_AUTO_APPROVE:-1}"
export SHELL_AUTO_APPROVE="${SHELL_AUTO_APPROVE:-1}"
export AUTO_APPROVE_MCP="${AUTO_APPROVE_MCP:-1}"
build_uv_prefix
set +e
"${UV_RUN[@]}" examples/run_examples.py --auto-mode --rerun-file "$file" --write-rerun --main-log "$main_log" --logs-dir "$LOG_DIR" 2>&1 | tee "$stdout_log"
local run_status=${PIPESTATUS[0]}
set -e
return "$run_status"
}
usage() {
cat <<'EOF'
Usage: run.sh <start|stop|status|logs|tail|collect|rerun> [args...]
Commands:
start [--filter ... | other args] Run examples in auto mode (foreground). Pass --background to run detached.
stop Kill the running auto-run (if any).
status Show whether it is running.
logs List log files (.tmp/examples-start-logs).
tail [logfile] Tail the latest (or specified) log.
collect [main_log] Parse a main log and write failed examples to .tmp/examples-rerun.txt.
rerun [rerun_file] Run only the examples listed in .tmp/examples-rerun.txt.
Environment overrides:
EXAMPLES_INTERACTIVE_MODE (default auto)
EXAMPLES_INCLUDE_SERVER/INTERACTIVE/AUDIO/EXTERNAL (defaults: 0/1/0/0)
EXAMPLES_UV_EXTRAS (default: litellm any-llm sqlalchemy redis blaxel modal runloop; set empty to disable)
APPLY_PATCH_AUTO_APPROVE, SHELL_AUTO_APPROVE, AUTO_APPROVE_MCP (default 1 in auto mode)
EOF
}
default_cmd="start"
if [[ $# -eq 0 && -s "$RERUN_FILE" ]]; then
default_cmd="rerun"
fi
case "${1:-$default_cmd}" in
start) shift || true; cmd_start "$@" ;;
stop) shift || true; cmd_stop ;;
status) shift || true; cmd_status ;;
logs) shift || true; cmd_logs ;;
tail) shift; cmd_tail "${1:-}" ;;
collect) shift || true; collect_rerun "${1:-}" ;;
rerun) shift || true; cmd_rerun "${1:-}" ;;
*) usage; exit 1 ;;
esac
@@ -0,0 +1,125 @@
---
name: final-release-review
description: Perform a release-readiness review by locating the previous release tag from remote tags and auditing the diff (e.g., v1.2.3...<commit>) for breaking changes, regressions, improvement opportunities, and risks before releasing openai-agents-python.
---
# Final Release Review
## Purpose
Use this skill when validating the latest release candidate commit (default tip of `origin/main`) for release. It guides you to fetch remote tags, pick the previous release tag, and thoroughly inspect the `BASE_TAG...TARGET` diff for breaking changes, introduced bugs/regressions, improvement opportunities, and release risks.
The review must be stable and actionable: avoid variance between runs by using explicit gate rules, and never produce a `BLOCKED` call without concrete evidence and clear unblock actions.
## Quick start
1. Ensure repository root: `pwd``path-to-workspace/openai-agents-python`.
2. Sync tags and pick base (default `v*`):
```bash
BASE_TAG="$(.agents/skills/final-release-review/scripts/find_latest_release_tag.sh origin 'v*')"
```
3. Choose target commit (default tip of `origin/main`, ensure fresh): `git fetch origin main --prune` then `TARGET="$(git rev-parse origin/main)"`.
4. Snapshot scope:
```bash
git diff --stat "${BASE_TAG}"..."${TARGET}"
git diff --dirstat=files,0 "${BASE_TAG}"..."${TARGET}"
git log --oneline --reverse "${BASE_TAG}".."${TARGET}"
git diff --name-status "${BASE_TAG}"..."${TARGET}"
```
5. Deep review using `references/review-checklist.md` to spot breaking changes, regressions, and improvement chances.
6. Capture findings and call the release gate: ship/block with conditions; propose focused tests for risky areas.
## Deterministic gate policy
- Default to **🟢 GREEN LIGHT TO SHIP** unless at least one blocking trigger below is satisfied.
- Use **🔴 BLOCKED** only when you can cite concrete release-blocking evidence and provide actionable unblock steps.
- Blocking triggers (at least one required for `BLOCKED`):
- A confirmed regression or bug introduced in `BASE...TARGET` (for example, failing targeted test, incompatible behavior in diff, or removed behavior without fallback).
- A confirmed breaking public API/protocol/config change with missing or mismatched versioning and no migration path (for example, patch release for a breaking change).
- A concrete data-loss, corruption, or security-impacting change with unresolved mitigation.
- A release-critical packaging/build/runtime path is broken by the diff (not speculative).
- Non-blocking by itself:
- Large diff size, broad refactor, or many touched files.
- "Could regress" risk statements without concrete evidence.
- Not running tests locally.
- If evidence is incomplete, issue **🟢 GREEN LIGHT TO SHIP** with targeted validation follow-ups instead of `BLOCKED`.
## Workflow
- **Prepare**
- Run the quick-start tag command to ensure you use the latest remote tag. If the tag pattern differs, override the pattern argument (e.g., `'*.*.*'`).
- If the user specifies a base tag, prefer it but still fetch remote tags first.
- Keep the working tree clean to avoid diff noise.
- **Assumptions**
- Assume the target commit (default `origin/main` tip) has already passed `$code-change-verification` in CI unless the user says otherwise.
- Do not block a release solely because you did not run tests locally; focus on concrete behavioral or API risks.
- Release policy: routine releases use patch versions; use minor only for breaking changes or major feature additions. Major versions are reserved until the 1.0 release.
- **Map the diff**
- Use `--stat`, `--dirstat`, and `--name-status` outputs to spot hot directories and file types.
- For suspicious files, prefer `git diff --word-diff BASE...TARGET -- <path>`.
- Note any deleted or newly added tests, config, migrations, or scripts.
- **Analyze risk**
- Walk through the categories in `references/review-checklist.md` (breaking changes, regression clues, improvement opportunities).
- When you suspect a risk, cite the specific file/commit and explain the behavioral impact.
- For every finding, include all of: `Evidence`, `Impact`, and `Action`.
- Severity calibration:
- **🟢 LOW**: low blast radius or clearly covered behavior; no release gate impact.
- **🟡 MODERATE**: plausible user-facing regression signal; needs validation but not a confirmed blocker.
- **🔴 HIGH**: confirmed or strongly evidenced release-blocking issue.
- Suggest minimal, high-signal validation commands (targeted tests or linters) instead of generic reruns when time is tight.
- Breaking changes do not automatically require a BLOCKED release call when they are already covered by an appropriate version bump and migration/upgrade notes; only block when the bump is missing/mismatched (e.g., patch bump) or when the breaking change introduces unresolved risk.
- **Form a recommendation**
- State BASE_TAG and TARGET explicitly.
- Provide a concise diff summary (key directories/files and counts).
- List: breaking-change candidates, probable regressions/bugs, improvement opportunities, missing release notes/migrations.
- Recommend ship/block and the exact checks needed to unblock if blocking. If a breaking change is properly versioned (minor/major), you may still recommend a GREEN LIGHT TO SHIP while calling out the change. Use emoji and boldface in the release call to make the gate obvious.
- If you cannot provide a concrete unblock checklist item, do not use `BLOCKED`.
## Output format (required)
All output must be in English.
Use the following report structure in every response produced by this skill. Be proactive and decisive: make a clear ship/block call near the top, and assign an explicit risk level (LOW/MODERATE/HIGH) to each finding with a short impact statement. Avoid overly cautious hedging when the risk is low and tests passed.
Always use the fixed repository URL in the Diff section (`https://github.com/openai/openai-agents-python/compare/...`). Do not use `${GITHUB_REPOSITORY}` or any other template variable. Format risk levels as bold emoji labels: **🟢 LOW**, **🟡 MODERATE**, **🔴 HIGH**.
Every risk finding must contain an actionable next step. If the report uses `**🔴 BLOCKED**`, include an `Unblock checklist` section with at least one concrete command/task and a pass condition.
```
### Release readiness review (<tag> -> TARGET <ref>)
This is a release readiness report done by `$final-release-review` skill.
### Diff
https://github.com/openai/openai-agents-python/compare/<tag>...<target-commit>
### Release call:
**<🟢 GREEN LIGHT TO SHIP | 🔴 BLOCKED>** <one-line rationale>
### Scope summary:
- <N files changed (+A/-D); key areas touched: ...>
### Risk assessment (ordered by impact):
1) **<Finding title>**
- Risk: **<🟢 LOW | 🟡 MODERATE | 🔴 HIGH>**. <Impact statement in one sentence.>
- Evidence: <specific diff/test/commit signal; avoid generic statements>
- Files: <path(s)>
- Action: <concrete next step command/task with pass criteria>
2) ...
### Unblock checklist (required when Release call is BLOCKED):
1. [ ] <concrete check/fix>
- Exit criteria: <what must be true to unblock>
2. ...
### Notes:
- <working tree status, tag/target assumptions, or re-run guidance>
```
If no risks are found, include a "No material risks identified" line under Risk assessment and still provide a ship call. If you did not run local verification, do not add a verification status section or use it as a release blocker; note any assumptions briefly in Notes. If the report is not blocked, omit the `Unblock checklist` section.
### Resources
- `scripts/find_latest_release_tag.sh`: Fetches remote tags and returns the newest tag matching a pattern (default `v*`).
- `references/review-checklist.md`: Detailed signals and commands for spotting breaking changes, regressions, and release polish gaps.
@@ -0,0 +1,4 @@
interface:
display_name: "Final Release Review"
short_description: "Audit a release candidate against the previous tag"
default_prompt: "Use $final-release-review to audit the release candidate diff against the previous release tag and call the ship/block gate."
@@ -0,0 +1,65 @@
# Release Diff Review Checklist
## Quick commands
- Sync tags: `git fetch origin --tags --prune`.
- Identify latest release tag (default pattern `v*`): `git tag -l 'v*' --sort=-v:refname | head -n1` or use `.agents/skills/final-release-review/scripts/find_latest_release_tag.sh`.
- Generate overview: `git diff --stat BASE...TARGET`, `git diff --dirstat=files,0 BASE...TARGET`, `git log --oneline --reverse BASE..TARGET`.
- Inspect risky files quickly: `git diff --name-status BASE...TARGET`, `git diff --word-diff BASE...TARGET -- <path>`.
## Gate decision matrix
- Choose `🟢 GREEN LIGHT TO SHIP` when no concrete blocking trigger is found.
- Choose `🔴 BLOCKED` only when at least one blocking trigger has concrete evidence and a defined unblock action.
- Blocking triggers:
- Confirmed regression/bug introduced in the diff.
- Confirmed breaking public API/protocol/config change with missing or mismatched versioning/migration path.
- Concrete data-loss/corruption/security-impacting issue with unresolved mitigation.
- Release-critical build/package/runtime break introduced by the diff.
- Non-blocking by itself:
- Large refactor or high file count.
- Speculative risk without evidence.
- Not running tests locally.
- If uncertain, keep gate green and provide focused follow-up checks.
## Actionability contract
- Every risk finding should include:
- `Evidence`: specific file/commit/diff/test signal.
- `Impact`: one-sentence user or runtime effect.
- `Action`: concrete command/task with pass criteria.
- A `BLOCKED` report must contain an `Unblock checklist` with at least one executable item.
- If no executable unblock item exists, do not block; downgrade to green with follow-up checks.
## Breaking change signals
- Public API surface: removed/renamed modules, classes, functions, or re-exports; changed parameters/return types, default values changed, new required options, stricter validation.
- Protocol/schema: request/response fields added/removed/renamed, enum changes, JSON shape changes, ID formats, pagination defaults.
- Config/CLI/env: renamed flags, default behavior flips, removed fallbacks, environment variable changes, logging levels tightened.
- Dependencies/platform: Python version requirement changes, dependency major bumps, `pyproject.toml`/`uv.lock` changes, removed or renamed extras.
- Persistence/data: migration scripts missing, data model changes, stored file formats, cache keys altered without invalidation.
- Docs/examples drift: examples still reflect old behavior or lack migration note.
## Regression risk clues
- Large refactors with light test deltas or deleted tests; new `skip`/`todo` markers.
- Concurrency/timing: new async flows, asyncio event-loop changes, retries, timeouts, debounce/caching changes, race-prone patterns.
- Error handling: catch blocks removed, swallowed errors, broader catch-all added without logging, stricter throws without caller updates.
- Stateful components: mutable shared state, global singletons, lifecycle changes (init/teardown), resource cleanup removal.
- Third-party changes: swapped core libraries, feature flags toggled, observability removed or gated.
## Improvement opportunities
- Missing coverage for new code paths; add focused tests.
- Performance: obvious N+1 loops, repeated I/O without caching, excessive serialization.
- Developer ergonomics: unclear naming, missing inline docs for public APIs, missing examples for new features.
- Release hygiene: add migration/upgrade note when behavior changes; ensure changelog/notes capture user-facing shifts.
## Evidence to capture in the review output
- BASE tag and TARGET ref used for the diff; confirm tags fetched.
- High-level diff stats and key directories touched.
- Concrete files/commits that indicate breaking changes or risk, with brief rationale.
- Tests or commands suggested to validate suspected risks (include pass criteria).
- Explicit release gate call (ship/block) with conditions to unblock.
- `Unblock checklist` section when (and only when) gate is `BLOCKED`.
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
remote="${1:-origin}"
pattern="${2:-v*}"
# Sync tags from the remote to ensure the latest release tag is available locally.
git fetch "$remote" --tags --prune --quiet
latest_tag=$(git tag -l "$pattern" --sort=-v:refname | head -n1)
if [[ -z "$latest_tag" ]]; then
echo "No tags found matching pattern '$pattern' after fetching from $remote." >&2
exit 1
fi
echo "$latest_tag"
@@ -0,0 +1,61 @@
---
name: implementation-strategy
description: Decide how to implement runtime and API changes in openai-agents-python before editing code. Use when a task changes exported APIs, runtime behavior, serialized state, tests, or docs and you need to choose the compatibility boundary, whether shims or migrations are warranted, and when unreleased interfaces can be rewritten directly.
---
# Implementation Strategy
## Overview
Use this skill before editing code when the task changes runtime behavior or anything that might look like a compatibility concern. The goal is to keep implementations simple while protecting real released contracts.
## Quick start
1. Identify the surface you are changing: released public API, unreleased branch-local API, internal helper, persisted schema, wire protocol, CLI/config/env surface, or docs/examples only.
2. Determine the latest release boundary from `origin` first, and only fall back to local tags when remote tags are unavailable:
```bash
BASE_TAG="$(.agents/skills/final-release-review/scripts/find_latest_release_tag.sh origin 'v*' 2>/dev/null || git tag -l 'v*' --sort=-v:refname | head -n1)"
echo "$BASE_TAG"
```
3. Judge breaking-change risk against that latest release tag, not against unreleased branch churn or post-tag changes already on `main`. If the command fell back to local tags, treat the result as potentially stale and say so.
4. Prefer the simplest implementation that satisfies the current task. Update callers, tests, docs, and examples directly instead of preserving superseded unreleased interfaces.
5. Add a compatibility layer only when there is a concrete released consumer, an otherwise supported durable external state boundary that requires it, or when the user explicitly asks for a migration path.
## Compatibility boundary rules
- Released public API or documented external behavior: preserve compatibility or provide an explicit migration path.
- Persisted schema, serialized state, wire protocol, CLI flags, environment variables, and externally consumed config: treat as compatibility-sensitive when they are part of the latest release or when the repo explicitly intends to preserve them across commits, processes, or machines.
- Python-specific durable surfaces such as `RunState`, session persistence, exported dataclass constructor order, and documented model/provider configuration should be treated as compatibility-sensitive when they were part of the latest release tag or are explicitly supported as a shared durability boundary.
- Interface changes introduced only on the current branch: not a compatibility target. Rewrite them directly.
- Interface changes present on `main` but added after the latest release tag: not a semver breaking change by themselves. Rewrite them directly unless they already define a released or explicitly supported durable external state boundary.
- Internal helpers, private types, same-branch tests, fixtures, and examples: update them directly instead of adding adapters.
- Unreleased persisted schema versions on `main` may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported. When you do that, update the support set and tests together so the boundary is explicit.
## Default implementation stance
- Prefer deletion or replacement over aliases, overloads, shims, feature flags, and dual-write logic when the old shape is unreleased.
- Do not preserve a confusing abstraction just because it exists in the current branch diff.
- If review feedback claims a change is breaking, verify it against the latest release tag and actual external impact before accepting the feedback.
- If a change truly crosses the latest released contract boundary, call that out explicitly in the ExecPlan, release notes context, and user-facing summary.
## SDK-specific decision rules
- When unsupported OpenAI API or provider-adapter behavior already has a released default path, avoid turning it into a default hard error unless the latest release boundary justifies that break. Prefer an opt-in strict mode such as `strict_feature_validation=True`, while keeping the default path compatible through warning, ignoring unsupported data, or a clearly non-empty placeholder.
- For OpenAI API feature gaps, evaluate streaming and non-streaming paths together. Custom tool calls, multi-choice Chat Completions chunks, non-text tool outputs, and similar provider payload differences must not be strict in one path and permissive or malformed in the other.
- When a change creates new public SDK behavior, do not expose it only through hard-coded module globals. Prefer an explicit public configuration object or parameter, preserve the existing default behavior when compatibility-sensitive, and make opt-in SDK defaults explicit.
- Append new optional fields or constructor parameters to public dataclasses and constructors. Do not insert them before existing public fields unless you also provide a compatibility layer and regression coverage for the old positional call shape.
- Treat threshold and quota values as part of the API design when they affect runtime behavior. Distinguish OpenAI platform quota-derived values from defensive SDK defaults; if the value is not anchored in a documented platform limit, avoid making it an unconditional default-on behavior.
- Define `None` semantics deliberately for public configuration. For example, use separate meanings for "feature disabled or no SDK limit", "use SDK default limits", and "disable only this specific limit" rather than relying on implicit truthiness checks.
## When to stop and confirm
- The change would alter behavior shipped in the latest release tag.
- The change would modify durable external data, protocol formats, or serialized state.
- The user explicitly asked for backward compatibility, deprecation, or migration support.
## Output expectations
When this skill materially affects the implementation approach, state the decision briefly in your reasoning or handoff, for example:
- `Compatibility boundary: latest release tag v0.x.y; branch-local interface rewrite, no shim needed.`
- `Compatibility boundary: released RunState schema; preserve compatibility and add migration coverage.`
@@ -0,0 +1,4 @@
interface:
display_name: "Implementation Strategy"
short_description: "Choose a compatibility-aware implementation plan"
default_prompt: "Use $implementation-strategy to choose the implementation approach and compatibility boundary before editing runtime code."
+211
View File
@@ -0,0 +1,211 @@
---
name: maintainer-review
description: Review a GitHub issue or pull request URL as an openai-agents-python maintainer, with a staged assessment of whether the claim is real, practically important, already solvable with supported functionality, correctly scoped, better served by another design, and worth maintainer and contributor effort. Use when assessing issue validity or severity, deciding whether an issue should be prioritized or closed, determining whether a requested feature represents an unmet need rather than a discoverability or usage gap, judging whether a PR is worth bringing to mergeable quality, comparing open PRs or alternative designs, separating code quality from repository readiness, or drafting a concise maintainer assessment. When closure, additional evidence, or code changes should be requested, also produce a polite, concise, complete, copy-paste-ready maintainer comment.
---
# Maintainer Review
## Objective
Make a maintainer decision, not a generic code-review summary. Separate these questions:
1. Is the claimed behavior real?
2. What user outcome or constraint exists independently of the reporter's proposed API or fix?
3. Can supported functionality already achieve that outcome with reasonable composition or configuration?
4. If a gap remains, is the proposed solution the best design and implementation layer?
5. Can normal users plausibly reach the gap, and what happens when they do?
6. Is it important enough to act on now?
7. If this PR did not already exist, would maintainers choose to open and implement the same work?
8. For a PR, is this solution worth merging and maintaining?
9. Can overlapping or stale operations corrupt shared state or clean up resources owned by surviving work?
10. If competing PRs exist, which single implementation path should maintainers pursue?
11. Which ambiguous scope or semantic choices are maintainer-owned product/API decisions, and what concrete direction should the contributor implement?
12. What concise maintainer message should communicate a closure or change request clearly and politely?
Treat an issue's requested field, callback, flag, class, or implementation strategy as a proposed mechanism, not as the accepted requirement. Do not begin by asking how to implement it. First prove that a concrete user outcome is not already supported and that the proposed mechanism is better than the available alternatives.
Lead with the current review state. Use `Preliminary assessment` while runtime approval or evidence is pending, and `Maintainer decision` only when the review can be concluded. Use the diff, issue narrative, or contributor effort as evidence, not as a proxy for impact.
## Workflow
### 1. Establish the exact target
- Accept a GitHub issue or PR URL as the primary input. Resolve its owner, repository, item type, and number before reviewing it.
- For an issue, read the full report, comments, reproduction, environment, linked material, and maintainer responses.
- For a PR, inspect the current remote base and head, full patch, commit history when relevant, tests, linked issue, and review discussion. Do not substitute the current local checkout for the remote change under review.
- State the claim in one falsifiable sentence. Distinguish the reported symptom from the reporter's proposed cause or fix.
- Identify the released behavior boundary when compatibility or regression claims matter.
- Verify whether linked evidence matches the PR's exact runtime variant, provider or tool type, triggering condition, and user outcome. A generic issue title, conceptual similarity, or wording such as `Related to` does not transfer evidence of need to an adjacent extension. If the reported scenario has already been fixed, treat additional variants as new needs requiring their own evidence.
Respect repository instructions for remote access and mutation. A review does not authorize comments, labels, branch changes, pushes, or other remote writes.
### 2. Establish the unmet need and challenge the proposed solution
Complete this pass before deeply evaluating a proposed implementation and before any positive issue or PR assessment.
First assign one `Need evidence` status:
- **Demonstrated**: The exact scope has a concrete supported scenario, a real-path reproduction, a released compatibility requirement, repeated demand, or a broad invariant with a meaningful consequence.
- **Plausible but unproven**: The path can exist, but realistic provider behavior, user reach, frequency, consequence, or demand is not established.
- **Already covered**: A reasonable supported workflow already satisfies the outcome.
- **Unsupported**: The outcome belongs outside the SDK contract or at a provider, adapter, or caller-owned layer.
Only `Demonstrated` need may receive `Merge-worthy as-is` or `Merge-worthy after focused changes`. For `Plausible but unproven`, prefer `Needs evidence` or `Not worth completing`; for `Already covered` or `Unsupported`, prefer closure or the relevant simpler alternative.
1. Restate the desired user outcome without naming the requested API, class, file, option, or implementation. Separate the actual constraint from the reporter's preferred mechanism.
2. Trace the closest supported ways to achieve that outcome in the current release and current target. Inspect the owning code path, public API, tests, and relevant docs rather than assuming that an unfamiliar capability is missing. Consider configuration, composition, cloning, callbacks, extension points, provider adapters, and doing the work at a caller-owned layer.
3. Determine whether the report shows a capability gap, an ergonomics or discoverability problem, an unsupported use case, or no demonstrated problem. A more convenient spelling is not automatically a missing capability.
4. Compare the proposed solution against the strongest existing approach and at least one better-design candidate: no code change, clearer documentation or validation, a narrower fix, reuse of an existing abstraction, or enforcement at a more coherent shared boundary.
5. For each viable approach, compare whether it satisfies the concrete scenario, what new public or internal contract it creates, cross-path consistency, compatibility, and permanent maintenance cost.
Do not treat a test proving that new code can work as evidence that the feature is needed. A `FakeModel` response, manually constructed provider item, mock, or new regression test can establish code-path reachability and implementation correctness; it does not by itself establish realistic provider behavior, user reach, frequency, practical consequence, or demand.
API symmetry, naming consistency, and parity with an adjacent tool, provider, or output type are design arguments, not evidence of need. Parity may justify work when it removes existing complexity or enforces a broad demonstrated invariant, but adding branches, tests, documentation, or public behavior requires independent practical justification.
If the need is not `Demonstrated`, inspect the patch only far enough to understand its contract, risk, and maintenance cost. Do not turn implementation defects, missing tests, or documentation gaps into a request-changes recommendation, because those questions become merge-blocking only after the need gate passes. If the report provides no concrete scenario, the existing functionality appears sufficient, or the requested mechanism solves only a hypothetical convenience problem, prefer `Needs evidence`, `Close`, `Supersede with a simpler alternative`, or `Not worth completing` over designing the requested feature on the reporter's behalf.
### 3. Discover competing open PRs proportionally
Do this before deeply evaluating a specified PR. A PR URL selects the starting point, not necessarily the entire comparison set.
- Determine the primary issue from explicit closing keywords, linked issues, issue timeline or development links, PR body and comments, and the reproduced symptom. If the association is inferred rather than explicit, state the evidence.
- When an issue is explicitly linked, enumerate all open PRs that address it through the issue timeline, development links, cross-references, closing keywords, and ordinary references. Include draft PRs but label them as drafts.
- When no issue is linked, run a bounded duplicate search using the strongest two or three signals from the title, reproduction, violated invariant, and runtime path. Stop when additional queries are unlikely to produce a credible competing implementation.
- Exclude closed or merged PRs from the active comparison set, while using them as history when relevant.
- Do not group PRs merely because they mention the same subsystem. Require a shared issue, symptom, violated invariant, or materially overlapping fix.
- Record the search methods and candidate set internally. If repository access cannot establish completeness, say so instead of claiming that every open PR was found. Do not list unrelated search hits in the final report.
When multiple candidates exist, compare them on need coverage, runtime correctness, scope, implementation layer, tests, compatibility, complexity, readiness, remaining maintainer work, and whether useful parts can be combined. Prefer the best maintainable solution, not the first submission or the smallest diff by default.
### 4. Use a two-stage evidence flow
Always begin with a desk review. Inspect the concrete runtime path before judging a small change as either trivial or meaningful. Check callers, adjacent helpers, validation layers, fallback paths, and existing tests. Search history or documentation only when it changes the decision. Inspecting test code is part of the desk review; executing tests, imports, examples, reproductions, benchmarks, or service calls is a runtime probe.
For repository-specific runtime invariants, start with `.agents/references/README.md` and open only the references that match the affected boundary. Treat `.agents/references/` as read-only during issue and PR review: use it to identify expected invariants, adjacent surfaces, and regression risks, then verify the current claim against the remote change, current code, tests, docs, release boundary, and focused runtime evidence. Do not edit references as a side effect of the review, infer current issue or PR status from them, or treat old issue or PR outcomes as current evidence. If the review reveals a reusable invariant that should be captured, recommend a separate repository-maintenance update unless the user explicitly asks to update references in the same task.
Use this evidence order across the two stages:
1. Trace the closest existing supported capabilities and determine whether they already satisfy the underlying user outcome.
2. Inspect existing tests and complete the code-path trace, including the mandatory interleaving and ownership pass when triggered, without executing code.
3. With explicit user approval, run a focused local reproduction of the exact claim when the desk-review rules below require it.
4. A comparison with the released version, base branch, or known-good control.
5. A broader runtime matrix only when the maintainer decision remains uncertain and the user approves it.
#### Stage 1: desk review
Produce an initial result from static evidence before running code:
##### Mandatory unmet-need and design pass
Before a positive assessment, complete the pass in step 2 and be able to state all of the following from concrete evidence:
1. The user outcome that current supported behavior cannot achieve.
2. The closest existing API or composition path and the exact reason it is insufficient.
3. Why the proposed behavior belongs at the chosen abstraction layer instead of a caller, adapter, validation, documentation, or existing extension point.
4. Why the proposed permanent contract is better than no code change and the strongest narrower alternative.
5. What real scenario, compatibility requirement, or repeated demand justifies the new maintenance surface.
6. Whether maintainers would choose to pursue the same work if no contributor had already supplied a patch.
If any answer is missing and could change whether code should exist at all, do not call the issue actionable or the PR merge-worthy. Request only the evidence needed to distinguish a genuine capability gap from a usage, discoverability, or solution-design problem. This is a product and architecture evidence gap, not a runtime-probe trigger by itself.
##### Mandatory interleaving and ownership pass
Run this pass before any positive PR assessment when a patch adds, removes, or reorders cleanup, retry, reconnect, cancellation, listeners, shared futures or tasks, connections or streams, state flags, or mutable state across an `await`, callback, event, or deferred completion.
1. Name each shared resource or state value and the operation that owns it. Include listeners, futures, tasks, connections, streams, locks, caches, state flags, persistence, and telemetry.
2. Trace at least two overlapping operations, `A` and `B`, across every suspension or re-entry point. Check `A pending -> B starts -> A fails -> B succeeds`, `A pending -> B starts -> B fails -> A succeeds`, close or cancellation between setup and completion, and a stale completion arriving after newer work.
3. For every cleanup or rollback, identify the exact attempt and resource generation it is allowed to dispose. Treat unconditional cleanup after a suspension point as a regression candidate until the code proves it cannot tear down newer or surviving work.
4. Compare base and head for the survivor invariant. Replacing duplicated work with missing handlers, a closed shared resource, reverted state, or a failed surviving task is a regression, not successful cleanup.
5. Inspect tests for controlled interleavings using deferred futures, callbacks, or events. Require assertions about the surviving operation's observable behavior and final resource state, not only listener counts or individual exception results.
Do not mark a concurrency-sensitive patch `Merge-worthy as-is` merely because sequential reconnect, retry, failure, and close tests pass. If the code trace proves an unsafe interleaving, conclude from static evidence and request a focused fix and regression test. If ownership remains ambiguous, keep the result preliminary and request approval for the smallest decisive runtime probe.
- If the claim or PR is decisively negative from a complete reachable code-path trace, conclude the review without a runtime probe. Examples include an impossible or unsupported path, duplicated existing handling, a demonstrated no-op, a direct compatibility break, or a clearly wrong abstraction. Do not call an ambiguous result negative merely to avoid a probe.
- If the initial result is positive and there is no unresolved runtime concern, and any triggered interleaving and ownership pass is complete, the desk review may be sufficient for a final maintainer decision. Do not run a probe only to restate evidence that cannot plausibly change the decision.
- If the initial result is positive but there is any unresolved runtime concern that could plausibly change claim validity, severity, merge-worthiness, required changes, or the preferred competing PR, stop before executing code. Report a `Preliminary assessment`, name the concern, propose the smallest decisive probe and control, and ask the user for approval to run it.
- A purely stylistic, documentation, CI-status, or repository-readiness concern does not trigger a runtime probe unless it masks a runtime question.
Do not issue a definitive positive maintainer decision while a decision-relevant runtime concern remains unresolved. If the user declines the probe, keep the result preliminary and state the exact confidence limitation.
#### Stage 2: approved runtime probe
After explicit approval, run only the smallest probe needed to resolve the stated concern. Exercise the real public or internal path and include a base, release, or known-good control when relevant. Do not stop at a happy-path smoke check when failure behavior determines the decision. Return to the user for separate approval before expanding materially beyond the approved probe.
For latency, timeout, buffering, backpressure, or cleanup claims, measure at least one observable elapsed-time or state-transition path when feasible. Do not assume that a mocked unit test exercises real scheduling or provider behavior. Prefer a local probe first; use an approval-gated live-service probe only when local evidence cannot settle the decision.
Use `$runtime-behavior-probe` only when the user explicitly invokes it and the skill is available, or when the user explicitly approves using it for the proposed runtime work. Preserve its environment-variable approval, live-service, cost, cleanup, and reporting gates. Do not make ordinary maintainer review depend on that skill being available.
For changes involving validation, fail-fast behavior, cleanup, retries, interruption, or concurrency, trace lifecycle ordering in addition to the main behavior:
- Identify listeners, tasks, connections, files, locks, state mutations, and other resources acquired before the new check or failure point.
- Verify cleanup when construction, context-manager entry, validation, connection, or execution raises before normal teardown runs.
- Require a negative-path test when a failure can leave observable state or resources behind.
Do not over-investigate. Stop when additional evidence is unlikely to change validity, severity, or the maintainer recommendation.
### 5. Calibrate validity and impact
Use `references/evaluation-framework.md` to assess claim validity, realistic reach, consequence, breadth, frequency, recoverability, compatibility, and severity. Keep observed facts separate from inference and state any missing evidence that could change the decision.
Report the `Need evidence` status before classifying the need as a capability gap, ergonomics or discoverability gap, unsupported use case, or no demonstrated gap. Do not assign practical impact to the absence of the requested mechanism when an existing supported workflow already produces the requested outcome. Do not infer practical importance merely from reachability, API asymmetry, or a technically successful patch.
For a PR, make `Severity` describe the underlying issue or user need only. Do not combine it with the risk created by the proposed patch. Report a meaningful patch-induced regression, compatibility, lifecycle, or maintenance risk separately as `Patch risk`.
Do not infer that a report is low-value merely because an AI may have found or written it. Do not speculate about authorship or motive. Identify contribution-shaped reports through objective signals: no reproducible behavior, unrealistic inputs, an impossible call path, duplicated existing handling, tests that do not exercise the claim, or a fix whose runtime result is a no-op.
### 6. Apply the maintainer-effort test
Use the framework's issue dispositions and PR checks to decide whether the outcome justifies permanent code, tests, documentation, and maintainer attention. Classify code quality separately from repository readiness.
Use one code recommendation:
- **Merge-worthy as-is**: real need, sound implementation, proportionate scope, adequate tests.
- **Merge-worthy after focused changes**: real need and viable direction, with bounded corrections.
- **Supersede with a simpler alternative**: real need, but a smaller or more coherent fix is preferable.
- **Not worth completing**: negligible or unsupported impact, no-op behavior, wrong abstraction, or excessive completion cost.
`Merge-worthy as-is` and `Merge-worthy after focused changes` are invalid unless `Need evidence` is `Demonstrated`. A bounded set of implementation fixes cannot promote a `Plausible but unproven` need into a merge-worthy recommendation.
For `Merge-worthy as-is` and `Merge-worthy after focused changes`, use one repository-readiness status when it helps communicate the integration state:
- **Ready**: current head is reviewable and required checks are green.
- **CI or review pending**: code recommendation is stable, but required external gates are incomplete.
- **Rebase or conflict resolution required**: the head cannot merge cleanly or is materially stale.
- **Blocked**: a concrete external or repository condition prevents a reliable merge decision.
Omit repository readiness for `Supersede with a simpler alternative` and `Not worth completing`; CI, review, mergeability, or branch freshness does not change those dispositions. Put any validation limitation that materially affects confidence in the evidence instead. When readiness is included, use exactly one of the four statuses above and do not invent variants such as `ready mechanically` or use rebase status for semantic staleness.
Do not downgrade an otherwise sound code recommendation solely because CI is pending. Do not call a PR ready when semantic conflict resolution or material code changes remain.
When multiple open PRs address the same issue, make one portfolio-level recommendation: select the strongest PR, request focused changes in one candidate, combine specific ideas into one PR, supersede all candidates with a simpler approach, or close duplicates. Explain why the recommended path is better than each alternative without turning the report into line-by-line review.
Always compare the proposed patch with the strongest existing supported approach and at least one alternative: no code change, validation or documentation, a narrower fix, reuse of an existing helper, or a different layer that enforces the invariant consistently. A review is incomplete if it establishes only that the patch works without establishing why the current product cannot meet the underlying need and why this design is preferable.
When multiple plausible semantic scopes, compatibility boundaries, or public API contracts remain, do not ask the contributor to choose among maintainer-owned options. Decide the preferred scope from the evidence, compatibility contract, and product/API design principles, then request that specific change. If the evidence is insufficient to choose, mark the review preliminary or request maintainer input; do not present an open-ended implementation fork as the contributor's decision.
### 7. Report findings and maintainer action
Choose the assessment language using this precedence:
1. Follow an explicit language request in the current conversation.
2. Follow an applicable language instruction from `~/.codex/AGENTS.md`, the repository's `AGENTS.md`, or another governing instruction file.
3. If recent conversation turns are consistently in one language, use that language.
4. Otherwise, default to English.
Do not infer the assessment language from the GitHub URL, contributor, code, or browser locale. Maintainer comment drafts remain English regardless of the assessment language. Keep the report decision-oriented and compact. Use no more than five evidence bullets by default; add more only when the decision genuinely depends on them.
Use the matching compact report variant in `references/evaluation-framework.md`. While runtime approval is pending, use its preliminary-assessment variant and end with the approval request instead of presenting a final recommendation. Collapse sections for simple cases rather than padding the answer. Put unexpected or negative runtime findings first, and name the preferred PR or approach explicitly when candidates compete.
For PRs, put `Need evidence` before code recommendation. When the need is not `Demonstrated`, lead with that result, omit repository readiness, and avoid presenting patch fixes as the primary maintainer action.
When existing functionality or a better alternative materially affects the decision, state it explicitly in the evidence and recommendation. Name the exact supported path, what it does and does not cover, and why it is preferable. Do not bury a `Not worth completing` or `Supersede with a simpler alternative` conclusion beneath praise for implementation quality.
When recommending closure, requesting more evidence, requesting code changes, or superseding a PR, append the English, copy-paste-ready maintainer comment defined by the framework. If multiple PRs need different actions, label one draft for each affected PR. Include only merge-blocking requests in the main action paragraph; keep optional documentation or polish clearly non-blocking or omit it.
For request-changes comments, phrase maintainer-owned semantic decisions as a directive, not as a menu. It is fine to mention the rejected alternative briefly in the rationale, but the requested action must identify the chosen behavior, scope, or compatibility boundary. Use "please do X because..." instead of "either do X or Y" when X versus Y changes the SDK contract or user-visible semantics.
Do not produce a line-by-line review unless requested. Do not equate passing tests with merge-worthiness, or a logically correct patch with practical value.
## Resource
- `references/evaluation-framework.md` contains the severity rubric, evidence checks, lifecycle review, issue dispositions, PR quality checks, maintainer-comment guidance, and report variants.
@@ -0,0 +1,4 @@
interface:
display_name: "Maintainer Review"
short_description: "Gate PR value on demonstrated user need"
default_prompt: "Use $maintainer-review with this GitHub issue or PR URL. Before evaluating implementation quality, verify that linked evidence matches the exact runtime variant and assign Need evidence as Demonstrated, Plausible but unproven, Already covered, or Unsupported. Only a Demonstrated need may receive a merge-worthy recommendation; synthetic tests, API parity, and contributor effort do not establish need. Then compare existing and alternative approaches, complete the desk review and required lifecycle ownership checks, request approval before any decision-relevant runtime probe, compare credible competing PRs, recommend the best maintainer action, and include an English comment draft when closure or changes are needed."
@@ -0,0 +1,366 @@
# Maintainer Evaluation Framework
Use this reference when a claim is ambiguous, severity is disputed, or a PR is technically correct but may not justify merge effort.
## Contents
- [Decision model](#decision-model)
- [Severity rubric](#severity-rubric)
- [Evidence-strength checks](#evidence-strength-checks)
- [Unmet need and alternative design gate](#unmet-need-and-alternative-design-gate)
- [Issue disposition](#issue-disposition)
- [PR quality and value](#pr-quality-and-value)
- [Documentation threshold](#documentation-threshold)
- [Lifecycle and failure-path review](#lifecycle-and-failure-path-review)
- [Concurrency and cleanup ownership](#concurrency-and-cleanup-ownership)
- [Better-alternative prompts](#better-alternative-prompts)
- [Competing PR comparison](#competing-pr-comparison)
- [Maintainer comment drafts](#maintainer-comment-drafts)
- [Compact report variants](#compact-report-variants)
## Decision model
Treat validity, severity, and merge-worthiness as separate results. Also distinguish a `Preliminary assessment`, which may still require approved runtime evidence, from a final `Maintainer decision`. Do not label a provisional positive result as a verdict or final decision.
| Dimension | Questions | Strong evidence |
|---|---|---|
| Claim validity | Does the exact reported behavior occur? Is the proposed cause correct? | Reproduction, failing focused test, or complete reachable code path |
| Reachability | Can supported, realistic inputs reach it? | Public API trace, real configuration, linked user report, or release comparison |
| Consequence | What fails, and is the result silent or recoverable? | Observed output/error/state plus downstream effect |
| Breadth | Who is affected? | Supported providers, platforms, versions, and configurations identified precisely |
| Frequency | Is this normal, intermittent, or pathological? | Repeat runs, telemetry or reports when available, deterministic preconditions |
| Need evidence | Is the exact scope demonstrated, merely plausible, already covered, or unsupported? | Same-scope user scenario, real-path reproduction, released compatibility requirement, repeated demand, or broad consequential invariant |
| Unmet need | What user outcome cannot be achieved through supported behavior today? | Concrete scenario plus a trace showing why the closest existing path is insufficient |
| Existing capability | Can configuration, composition, cloning, callbacks, extension points, or a caller-owned layer already satisfy the outcome? | Current release code, tests, docs, and an exact supported workflow |
| Compatibility | Is released behavior or durable state changed? | Latest release comparison and explicit contract inspection |
| Solution fit | Is the requested mechanism the best design and implementation layer? | Proposed solution compared with the strongest existing path and at least one narrower or more coherent alternative |
| Maintainer-owned scope | When several plausible semantics remain, which behavior should the SDK own? | A concrete maintainer decision grounded in compatibility, user outcome, and API design, not an open-ended contributor choice |
| Resource ownership | Can stale, failed, cancelled, or overlapping work mutate or clean up resources owned by surviving work? | Interleaving trace, attempt or generation ownership, and survivor assertions |
| Maintenance cost | What permanent complexity and review burden does it add? | Changed surface, new branches/configuration, test burden, remaining work |
## Severity rubric
- **Negligible**: No runtime difference, unreachable or unsupported input, cosmetic inconsistency, or a fully harmless edge case. Usually close, document, or decline code complexity.
- **Low**: Real but narrow and recoverable behavior with a simple workaround and no data, security, or compatibility risk. Merge only when the fix is small and clearly improves an invariant.
- **Moderate**: Plausible supported use fails or produces incorrect behavior for a meaningful subset of users. Prioritize a bounded fix and regression test.
- **High**: Common or important supported use is broken, causes serious compatibility problems, leaks sensitive data, or risks persistent corruption. Treat as urgent and require strong validation.
- **Critical**: Broadly exploitable security impact, severe data loss, or systemic failure requiring immediate coordinated action. Use only with concrete evidence.
Severity is approximately consequence multiplied by realistic reach and frequency, reduced by recoverability. Do not raise severity because a report sounds alarming or lower it because a patch is small.
## Evidence-strength checks
Before calling a claim confirmed, answer:
- Does the reproduction exercise the same public or internal path named in the report?
- Does the failure still occur on the relevant base, release, or current target?
- Does the test fail without the patch and pass with it?
- Are setup failures, stale builds, environment leakage, proxies, caches, or unsupported options excluded?
- Does an adjacent helper or equivalent path follow different semantics?
- Is the observed behavior prohibited by an actual contract, or merely surprising?
- For latency, timeout, buffering, backpressure, or cleanup claims, was observable elapsed time or a real state transition measured when feasible rather than inferred only from mocks?
- For shared asynchronous state, do tests control completion order and prove that stale failure or cleanup cannot affect the surviving operation?
Use `partially confirmed` when the symptom is real but the cause, reach, or claimed scope is wrong. Use `unproven` when decisive evidence is missing. Use `contradicted` only when evidence directly disproves the claim.
## Unmet need and alternative design gate
Issue reports often combine a desired outcome with a proposed API or implementation. Treat the proposed mechanism as a hypothesis. Confirm the unmet outcome before evaluating how well the patch implements that mechanism.
### Linked-evidence scope
Evidence from a linked issue applies only when the issue and PR share the same runtime variant, provider or tool type, trigger, supported configuration, and user outcome. A broad title, ordinary reference, `Related to` statement, or conceptual similarity is not enough. If an earlier change already resolved the concrete reported scenario, an adjacent extension starts with no inherited evidence of need.
### Need evidence status
Assign one status before deep implementation review:
- **Demonstrated**: The exact scope has a concrete supported scenario, real-path reproduction, released compatibility requirement, repeated demand, or broad invariant with meaningful consequence.
- **Plausible but unproven**: The code path is possible, but realistic reach, frequency, consequence, provider behavior, or demand is missing.
- **Already covered**: A reasonable supported workflow already satisfies the outcome.
- **Unsupported**: The outcome is outside the SDK contract or belongs at a provider, adapter, or caller-owned layer.
Only `Demonstrated` need can support a merge-worthy code recommendation. `Plausible but unproven` maps to `Needs evidence` or `Not worth completing`, even when the patch is technically correct and its remaining fixes are bounded. `Already covered` and `Unsupported` normally map to closure or a simpler non-core alternative.
Before accepting an issue or recommending a PR, record:
| Question | Required evidence |
|---|---|
| What outcome is needed? | A concrete supported scenario stated without the proposed API or fix |
| What exists today? | The closest current-release API, configuration, composition, extension point, or caller-owned solution |
| Why is it insufficient? | An exact behavioral, compatibility, lifecycle, or operational constraint, not preference alone |
| What are the alternatives? | The proposed patch, the strongest existing path, and at least one no-code, narrower, or better-layer design |
| Why add a contract? | Practical benefit sufficient to justify public surface, runtime branches, cross-path tests, documentation, and long-term maintenance |
Classify the result:
- **Capability gap**: a supported, realistic outcome cannot be achieved with current functionality. Code may be warranted.
- **Ergonomics or discoverability gap**: the outcome is already possible, but the supported route is confusing or unnecessarily difficult. Prefer documentation, validation, or a narrowly justified convenience improvement.
- **Unsupported use case**: the desired outcome lies outside the SDK contract or belongs at a provider, adapter, application, or other caller-owned layer. Do not expand the core API merely to make it possible.
- **No demonstrated gap**: no concrete scenario proves that existing functionality is insufficient. Request evidence or close rather than designing from the proposed mechanism.
Passing tests for a new implementation establish feasibility and correctness, not need. A `FakeModel` response, manually constructed provider item, mock, or synthetic fixture does not establish realistic provider behavior, user reach, frequency, consequence, or demand. API symmetry and parity with an adjacent runtime are design arguments, not need evidence. A technically coherent patch can still be `Not worth completing` when the motivating scenario is hypothetical, already supported, or better solved elsewhere.
Use the counterfactual maintainer test: if the PR did not already exist, would maintainers choose to file and implement the same work from the available evidence? Contributor effort lowers implementation cost but does not create product need or remove permanent maintenance cost.
When the need is not `Demonstrated`, inspect implementation only far enough to estimate contract, risk, and maintenance cost. Do not convert patch defects, missing tests, or documentation gaps into a request-changes disposition; those become merge blockers only after the need gate passes.
## Issue disposition
Choose one primary action:
- **Prioritize**: confirmed moderate-or-higher impact or an important invariant with no safe workaround.
- **Accept, low priority**: confirmed low impact, existing supported functionality is insufficient for the demonstrated scenario, and a proportionate fix appears possible.
- **Narrow scope**: a valid core exists, but the report overstates affected paths or expected behavior.
- **Needs evidence**: plausible claim, but no minimal reproduction, supported setup, contract basis, or concrete scenario showing why existing functionality is insufficient.
- **Close**: duplicate, unsupported, unreachable, contradicted, no-op, already addressed by a reasonable supported path, or not worth permanent complexity.
When requesting evidence, ask only for information that could change the disposition.
## PR quality and value
Assess these independently:
1. **Need**: Same-scope issue or runtime evidence demonstrates a concrete unmet user outcome that the closest supported capability cannot reasonably satisfy. Do not inherit evidence from an adjacent variant or already-fixed scenario.
2. **Correctness**: The fix works for the reported case and meaningful boundaries.
3. **Placement**: The invariant is enforced once at the right layer instead of duplicating existing functionality, patching locally, or moving caller- or provider-owned policy into the core SDK.
4. **Consistency**: Equivalent sync/async, streaming/non-streaming, provider, serialization, and resume paths remain aligned where applicable.
5. **Tests**: A regression test fails on the base, passes on the head, and tests the exact non-happy-path value or state. When shared state crosses an asynchronous boundary, tests control relevant completion orders and assert the surviving operation's behavior and final resource state.
6. **Compatibility**: Released positional APIs, wire formats, persisted schemas, and established error behavior are preserved or intentionally migrated.
7. **Proportionality**: Complexity and public surface are justified by impact.
8. **Completion cost**: Remaining fixes, docs, tests, and design work are bounded enough to justify maintainer attention.
A PR can be correct but not merge-worthy. Typical reasons include a nonexistent or negligible need, an outcome already supported through a reasonable existing mechanism, a no-op on the actual runtime path, incomplete cross-path semantics, an abstraction cost larger than the benefit, or a simpler design at another layer.
Do not use implementation correctness, bounded remaining work, CI status, or contributor effort to upgrade a need that is only `Plausible but unproven`. Merge-worthiness is gated by demonstrated need, not by how close the patch is to completion.
Keep issue impact and patch risk separate. `Severity` describes the underlying issue or user need. A regression, compatibility break, lifecycle leak, or maintenance hazard introduced by the proposed patch belongs under `Patch risk` and must not inflate or obscure the issue severity.
When a PR exposes an ambiguous semantic boundary, decide whether that boundary belongs to maintainers before drafting requests. If the choice affects SDK contract, compatibility, persistence, error semantics, public API meaning, or cross-path behavior, the review should pick one direction or explicitly block on maintainer input. Do not delegate that choice to the contributor as "either X or Y"; ask for the chosen behavior and the tests or docs needed to lock it down.
## Documentation threshold
Do not treat documentation as automatically required for every public option, constructor parameter, provider setting, or behavior change. Make docs merge-blocking only when at least one of these is true:
- Existing user-facing docs become materially false, unsafe, or misleading.
- Correct or safe use depends on a non-obvious constraint, migration step, compatibility boundary, or operational warning.
- Repository policy, the accepted issue scope, or an explicit maintainer decision requires documentation in the same PR.
- The intended feature would be practically unusable or undiscoverable by its target users without a documented entry point, and generated API reference or clear code-level discovery is insufficient.
If docs would merely improve discoverability or completeness, keep them non-blocking. Do not change `Merge-worthy as-is` to `Merge-worthy after focused changes` solely for optional docs, and do not include optional docs in the maintainer comment's required-action paragraph. Respect an explicit maintainer choice to omit docs or defer them to a separate follow-up.
## Lifecycle and failure-path review
Apply this section when a change adds validation, fail-fast behavior, cleanup, retries, interruption, background work, or concurrency.
- Identify the earliest point where all dynamic inputs needed for a correct decision are available.
- List side effects before and after that point: listeners, tasks, connections, files, locks, caches, state mutations, and telemetry.
- Exercise failure during construction, context-manager entry, validation, connection, and execution when those phases exist.
- Confirm that normal teardown is actually entered. If an enter or constructor fails, verify cleanup explicitly rather than assuming an exit hook runs.
- Prefer validation after dynamic configuration is resolved but before avoidable side effects begin.
- Require a regression test for any listener, task, connection, or state that could remain after failure.
## Concurrency and cleanup ownership
Apply this section before a positive assessment whenever lifecycle work crosses an `await`, callback, event, deferred completion, retry, reconnect, cancellation, or shared resource boundary. Sequential correctness is insufficient because a patch can improve isolated cleanup while introducing cross-attempt teardown.
Use a two-operation interleaving matrix during desk review:
| Ordering | Required question |
|---|---|
| `A pending -> B starts -> A fails -> B succeeds` | Can A's cleanup remove or revert anything B needs? |
| `A pending -> B starts -> B fails -> A succeeds` | Can B's cleanup leave A successful but non-functional? |
| `A succeeds -> B starts -> stale A completion` | Can stale A overwrite B's newer state or generation? |
| setup -> close/cancel -> late completion | Can late work resurrect listeners, state, tasks, or connections after teardown? |
For each ordering:
- Identify the resource owner before and after every suspension point.
- Distinguish per-attempt resources from shared runner, session, transport, cache, or listener state.
- Require cleanup to carry an ownership token, generation, identity check, serialization guarantee, or another invariant that prevents cross-attempt disposal.
- Compare base and head on the survivor invariant. Fewer duplicates do not justify losing the only active handler, connection, task, or state update.
- Require a controlled interleaving test when the ordering is reachable. The test must assert both the failing operation and the surviving operation's observable behavior after all completions settle.
An unscoped `finally`, `except`, close handler, cancellation callback, or rollback that mutates shared state after a suspension point is merge-blocking when another operation can still own or use that state.
## Better-alternative prompts
Start with the strongest existing supported path, then test at least one additional alternative against the proposed patch. Do not complete a positive review without this comparison.
- Can the requested outcome already be achieved through configuration, composition, cloning, callbacks, extension points, a custom provider or adapter, or caller-owned code?
- If the existing route is awkward, is the problem discoverability or ergonomics rather than missing capability?
- What happens if maintainers make no code change?
- Can input validation or an existing helper enforce the invariant earlier?
- Can the fix be limited to the one supported path that fails?
- Would documentation or a clearer error prevent misuse without runtime complexity?
- Can the test be added first to reveal the smallest correct change?
- Is the proposed public option compensating for an internal design issue?
- Is the proposed core behavior actually provider- or application-specific policy that belongs at another layer?
## Competing PR comparison
When two or more open PRs address the same issue, first verify that they belong in one comparison set. Accept an explicit issue link, the same minimal reproduction, the same violated invariant, or materially overlapping runtime paths as association evidence. Do not treat a shared label or subsystem as sufficient.
Compare each candidate on the same evidence basis:
| Criterion | Question |
|---|---|
| Need | Does a concrete user outcome remain unmet after tracing existing supported functionality? |
| Existing capability | Could every candidate be avoided by configuration, composition, an extension point, or a better caller- or provider-owned solution? |
| Coverage | Does it solve the whole confirmed issue, a useful subset, or an adjacent problem? |
| Correctness | Does the fix work on the real path and meaningful boundaries? |
| Placement | Does it enforce the invariant at the correct shared layer? |
| Tests | Does it reproduce the base failure and distinguish the candidate approaches? |
| Compatibility | Does it preserve released APIs, state, protocol, providers, and established behavior? |
| Complexity | What permanent branches, abstractions, configuration, or coupling does it add? |
| Readiness | Is it mergeable now, or how much focused work remains? |
| Reuse | Are there valuable tests or implementation pieces that should be combined into another candidate? |
Choose one portfolio-level disposition:
- **Prefer one PR**: identify the strongest candidate and close or supersede duplicates.
- **Prefer one after focused changes**: keep one candidate active and state bounded changes required before merge.
- **Combine selectively**: identify the destination PR and the exact ideas or tests worth transferring; avoid asking maintainers to reconcile entire competing implementations.
- **Replace all**: explain the simpler or more coherent implementation that should supersede every candidate.
- **Merge none**: the issue is invalid, negligible, unsupported, or none of the approaches justify completion cost.
Do not split the decision into independent approvals. Competing PRs consume overlapping review and maintenance budgets, so recommend one path for the issue as a whole.
## Maintainer comment drafts
Always write maintainer comments in English, regardless of the assessment language. Produce a draft when the recommendation is to close, request evidence, request focused code changes, supersede a PR, or choose one competing PR over another.
Keep each draft polite, direct, and copy-paste-ready. Usually use 60-160 words in one to three short paragraphs:
1. Acknowledge the contribution or report.
2. Explain the decision with the smallest amount of decisive technical evidence.
3. Give the exact next action or the condition for reconsideration.
Do not include internal labels such as `severity: low`, speculate about AI authorship or contributor intent, repeat the full review, or soften the message until the requested action becomes unclear.
Do not ask contributors to choose maintainer-owned semantics. If two implementations are technically possible but one changes the SDK contract, decide the contract in the review and make the comment actionable. Use a short rationale such as "This keeps the new handler scoped to the existing raise site" or "This makes the handler name match all invalid final messages", then request the exact code and tests for that decision.
### Close
```text
Thanks for taking the time to investigate this. I traced the reported case through <path or behavior>, and <decisive finding>. In the supported path, <practical result>, so the added complexity is not justified by the demonstrated impact.
I am going to close this <issue/PR>. If you can provide <specific reproduction or evidence that would change the decision>, we can revisit the underlying problem with that narrower scope.
```
### Request changes
```text
Thanks for the contribution. The underlying issue is valid, and this approach is directionally reasonable. Before we can merge it, please address the following points: <bounded list of required changes>.
These changes are needed because <concise contract, lifecycle, compatibility, or test reason>. Once they are covered with a regression test that fails on the base and passes on the updated branch, the PR should be ready for another review.
```
Adapt the wording to the actual evidence. Do not use these templates as generic filler.
### Existing capability or better alternative
```text
Thanks for the contribution. I traced the underlying use case through <existing API or workflow>, which already supports <desired outcome and relevant limits>. The proposed change adds <new contract or complexity>, but the issue does not demonstrate a concrete supported case that the existing approach cannot handle.
I am going to close this <issue/PR> for now. If you can provide <specific scenario showing the existing approach is insufficient>, we can revisit the unmet need and choose the narrowest appropriate design from that evidence.
```
## Compact report variants
Use `Maintainer decision` for a concluded review. Use `Preliminary assessment` when a desk review is tentatively positive but a decision-relevant runtime concern remains. `Verdict` is intentionally avoided in the report headings because it does not communicate whether the result is provisional or final.
### Runtime approval gate
```markdown
## Preliminary assessment
<Tentative issue or PR assessment based on desk review only.>
## Static evidence
- <decisive code-path or test-inspection evidence>
- <what remains uncertain at runtime>
## Proposed runtime probe
- Concern: <the uncertainty that could change the decision>
- Probe: <smallest exact execution path>
- Control: <base, release, or known-good comparison when relevant>
- Scope: <local-only or any live-service, cost, mutation, or cleanup implications>
## Approval request
<Ask whether to run this exact probe. Do not present a final positive recommendation yet.>
```
### Issue
```markdown
## Maintainer decision
<Real/partial/unproven/contradicted, severity, and disposition.>
## Evidence
- <decisive evidence>
- <scope or uncertainty>
## Existing capability and alternatives
<Closest supported path, why it is or is not sufficient, and the preferred design alternative.>
## Recommendation
<Prioritize, accept low priority, narrow, request evidence, or close.>
## Maintainer comment draft
<Include when closure or additional evidence should be requested.>
```
### Pull request
```markdown
## Maintainer decision
<Need, practical impact, and merge-worthiness.>
- Need evidence: <Demonstrated / Plausible but unproven / Already covered / Unsupported>
- Code recommendation: <code disposition>
- Repository readiness: <integration status; include only for a merge-worthy recommendation when material>
## Evidence
- <runtime or code-path result>
- <test and compatibility result>
## Existing capability and alternatives
<Closest supported path, why the demonstrated scenario cannot use it, and why this patch is preferable to no code change or a narrower design.>
## Issue impact
- Validity: <claim validity>
- Severity: <severity of the underlying issue or need>
- Reach: <realistic reach>
## Patch risk
<Include only when the proposed patch introduces a meaningful regression, compatibility, lifecycle, or maintenance risk.>
## PR quality
- Solution fit: <assessment>
- Tests: <assessment>
- Remaining effort: <bounded/unbounded and why>
## Recommendation
<Merge, focused changes, simpler replacement, or close.>
## Maintainer comment draft
<Include only when closure, evidence, or changes should be requested.>
```
### Competing pull requests
```markdown
## Maintainer decision
<Issue validity, practical severity, and preferred implementation path.>
## Open PR comparison
| PR | Approach | Correctness | Tests | Compatibility/complexity | Readiness |
|---|---|---|---|---|---|
| #... | ... | ... | ... | ... | ... |
## Recommendation
<Select one, request focused changes, combine specific parts, replace all, or merge none.>
<State what should happen to every other open candidate.>
## Maintainer comment drafts
<One copy-paste-ready draft for each PR that should be closed, changed, or superseded.>
```
+44
View File
@@ -0,0 +1,44 @@
---
name: openai-knowledge
description: Use when working with the OpenAI API (Responses API) or OpenAI platform features (tools, streaming, Realtime API, auth, models, rate limits, MCP) and you need authoritative, up-to-date documentation (schemas, examples, limits, edge cases). Prefer the OpenAI Developer Documentation MCP server tools when available; otherwise guide the user to enable `openaiDeveloperDocs`.
---
# OpenAI Knowledge
## Overview
Use the OpenAI Developer Documentation MCP server to search and fetch exact docs (markdown), then base your answer on that text instead of guessing.
## Workflow
### 1) Check whether the Docs MCP server is available
If the `mcp__openaiDeveloperDocs__*` tools are available, use them.
If you are unsure, run `codex mcp list` and check for `openaiDeveloperDocs`.
### 2) Use MCP tools to pull exact docs
- Search first, then fetch the specific page or pages.
- `mcp__openaiDeveloperDocs__search_openai_docs` → pick the best URL.
- `mcp__openaiDeveloperDocs__fetch_openai_doc` → retrieve the exact markdown (optionally with an `anchor`).
- When you need endpoint schemas or parameters, use:
- `mcp__openaiDeveloperDocs__get_openapi_spec`
- `mcp__openaiDeveloperDocs__list_api_endpoints`
Base your answer on the fetched text and quote or paraphrase it precisely. Do not invent flags, field names, defaults, or limits.
### 3) If MCP is not configured, guide setup (do not change config unless asked)
Provide one of these setup options, then ask the user to restart the Codex session so the tools load:
- CLI:
- `codex mcp add openaiDeveloperDocs --url https://developers.openai.com/mcp`
- Config file (`~/.codex/config.toml`):
- Add:
```toml
[mcp_servers.openaiDeveloperDocs]
url = "https://developers.openai.com/mcp"
```
Also point to: https://developers.openai.com/resources/docs-mcp#quickstart
@@ -0,0 +1,4 @@
interface:
display_name: "OpenAI Knowledge"
short_description: "Pull authoritative OpenAI platform documentation"
default_prompt: "Use $openai-knowledge to fetch the exact OpenAI docs needed for this API or platform question."
+59
View File
@@ -0,0 +1,59 @@
---
name: pr-draft-summary
description: Create the required PR-ready summary block, branch suggestion, title, and draft description for openai-agents-python. Use before the final response whenever the current task changed runtime code, tests, examples, build/test configuration, or docs with behavior impact, regardless of perceived change size and including local-only or uncommitted work. Skip only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, or when the user explicitly says not to include the PR draft block.
---
# PR Draft Summary
## Purpose
Produce the PR-ready summary required in this repository after eligible code work is complete: a concise summary plus a PR-ready title and draft description that begins with "This pull request <verb> ...". The block should be ready to paste into a PR for openai-agents-python.
## When to Trigger
- Before every final response, check whether the current task changed runtime code (`src/agents/`), tests (`tests/`), examples (`examples/`), build/test configuration, or docs with behavior impact.
- If it did, run this skill after required verification and before sending the final response. Do not use perceived change size to decide whether to run it.
- Run it for eligible local-only and uncommitted work even when the user did not ask to create a pull request. Producing this text does not authorize creating a branch, committing, pushing, or opening a pull request.
- Skip only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, or when the user explicitly says not to include the PR draft block.
## Inputs to Collect Automatically (do not ask the user)
- Current branch: `git rev-parse --abbrev-ref HEAD`.
- Working tree: `git status -sb`.
- Untracked files: `git ls-files --others --exclude-standard` (use with `git status -sb` to ensure they are surfaced; `--stat` does not include them).
- Changed files: `git diff --name-only` (unstaged) and `git diff --name-only --cached` (staged); sizes via `git diff --stat` and `git diff --stat --cached`.
- Latest release tag (prefer remote-aware lookup): `LATEST_RELEASE_TAG=$(.agents/skills/final-release-review/scripts/find_latest_release_tag.sh origin 'v*' 2>/dev/null || git tag -l 'v*' --sort=-v:refname | head -n1)`.
- Base reference (use the branch's upstream, fallback to `origin/main`):
- `BASE_REF=$(git rev-parse --abbrev-ref --symbolic-full-name @{upstream} 2>/dev/null || echo origin/main)`.
- `BASE_COMMIT=$(git merge-base --fork-point "$BASE_REF" HEAD || git merge-base "$BASE_REF" HEAD || echo "$BASE_REF")`.
- Commits ahead of the base fork point: `git log --oneline --no-merges ${BASE_COMMIT}..HEAD`.
- Category signals for this repo: runtime (`src/agents/`), tests (`tests/`), examples (`examples/`), docs (`docs/`, `mkdocs.yml`), build/test config (`pyproject.toml`, `uv.lock`, `Makefile`, `.github/`).
## Workflow
1) Run the commands above without asking the user; compute `BASE_REF`/`BASE_COMMIT` first so later commands reuse them.
2) If there are no staged/unstaged/untracked changes and no commits ahead of `${BASE_COMMIT}`, reply briefly that no code changes were detected and skip emitting the PR block.
3) Infer change type from the touched paths listed under "Category signals"; classify as feature, fix, refactor, or docs-with-impact, and flag backward-compatibility risk only when the diff changes released public APIs, external config, persisted data, serialized state, or wire protocols. Judge that risk against `LATEST_RELEASE_TAG`, not unreleased branch-only churn.
4) Summarize changes in 13 short sentences using the key paths (top 5) and `git diff --stat` output; explicitly call out untracked files from `git status -sb`/`git ls-files --others --exclude-standard` because `--stat` does not include them. If the working tree is clean but there are commits ahead of `${BASE_COMMIT}`, summarize using those commit messages.
5) Choose the lead verb for the description: feature → `adds`, bug fix → `fixes`, refactor/perf → `improves` or `updates`, docs-only → `updates`.
6) Suggest a branch name. If already off main, keep it; otherwise propose `feat/<slug>`, `fix/<slug>`, or `docs/<slug>` based on the primary area (e.g., `docs/pr-draft-summary-guidance`).
7) If the current branch matches `issue-<number>` (digits only), keep that branch suggestion. Optionally pull light issue context (for example via the GitHub API) when available, but do not block or retry if it is not. When an issue number is present, reference `https://github.com/openai/openai-agents-python/issues/<number>` and include an auto-closing line such as `This pull request resolves #<number>.`.
8) Draft the PR title and description using the template below.
9) Output only the block in "Output Format". Keep any surrounding status note minimal and in English.
## Output Format
When closing out a task, add this concise Markdown block (English only) after any brief status note unless the task falls under the documented skip cases or the user says they do not want it.
```
# Pull Request Draft
## Branch name suggestion
git checkout -b <kebab-case suggestion, e.g., feat/pr-draft-summary-skill>
## Title
<single-line imperative title, which can be a commit message; if a common prefix like chore: and feat: etc., having them is preferred>
## Description
<include what you changed plus a draft pull request title and description for your local changes; start the description with prose such as "This pull request resolves/updates/adds ..." using a verb that matches the change (you can use bullets later), explain the change background (for bugs, clearly describe the bug, symptoms, or repro; for features, what is needed and why), any behavior changes or considerations to be aware of, and you do not need to mention tests you ran.>
```
Keep it tight—no redundant prose around the block, and avoid repeating details between `Changes` and the description. Tests do not need to be listed unless specifically requested.
@@ -0,0 +1,4 @@
interface:
display_name: "PR Draft Summary"
short_description: "Draft the repo-ready PR title and description"
default_prompt: "Use $pr-draft-summary to generate the PR-ready summary block, title, and draft description for the current changes."
@@ -0,0 +1,161 @@
---
name: runtime-behavior-probe
description: Plan and execute runtime-behavior investigations with temporary probe scripts, validation matrices, state controls, and findings-first reports. Use only when the user explicitly invokes this skill to verify actual runtime behavior beyond normal code-level checks, especially to uncover edge cases, undocumented behavior, or common failure modes in local or live integrations. A baseline smoke check is fine as an entry point, but do not stop at happy-path confirmation.
---
# Runtime Behavior Probe
## Overview
Use this skill to investigate real runtime behavior, not to restate code or documentation. Start by planning the investigation, then execute a case matrix, record observed behavior, and report both the findings and the method used to obtain them.
## Core Rules
- Treat this skill as manual-only. Do not rely on implicit invocation.
- A baseline success or smoke case is often the right entry point, but do not stop there when the real question involves edge cases, drift, or failure behavior.
- Plan before running anything. Write the case matrix first, then fill it in with observed results. The matrix can live in a scratch note, a temporary file, or the probe script header.
- Default to local or read-only probes. Consider a live service only when it is clearly relevant, then apply the lightweight gates below before you run it.
- Size the probe to the decision. Start with the smallest matrix that can disqualify or validate the current hypothesis, then expand only when uncertainty remains.
- Before a live probe, apply three lightweight gates:
- Destination gate. Use only a live destination that is clearly allowed for the task.
- Intent gate. Run the live probe only when the user explicitly wants runtime verification on that integration, or explicitly approves it after you propose the probe.
- Data gate. If the probe will read environment variables, mutate remote state, incur material cost, or exercise non-public or user data, name the exact variable names or data class and get explicit approval first.
- Classify each case as read-only, mutating, or costly before execution. For mutating or costly cases, or for any live case that will read environment variables, define cleanup or rollback before running the probe.
- Use temporary files or a temporary directory for one-off probe scripts.
- Keep temporary artifacts until the final response is drafted. Then delete them by default unless the user asked to keep them or they are needed for follow-up. Even when artifacts are deleted, keep a short run summary of the command shape, runtime context, and artifact status in the report.
- Before executing a live probe that will read environment variables, tell the user the exact variable names you plan to use and why, then wait for explicit approval. Examples include `OPENAI_API_KEY` and other expected default names for the system under test.
- When the environment-variable approval gate is required and the `request_user_input` tool is available, use that tool instead of a plain-text approval question. Ask one concise question with mutually exclusive choices such as `Allow once (Recommended)` and `Do not allow`, omit `autoResolutionMs`, and make the approval single-probe and limited to the exact named variables and destination. If the tool is unavailable, fall back to a concise plain-text approval question and do not proceed until the user explicitly approves.
- Never print secrets, even when they come from standard environment variables that this skill may use.
- For OpenAI API or OpenAI platform probes in this repository, use [$openai-knowledge](../openai-knowledge/SKILL.md) early to confirm contract-sensitive details such as supported parameters, field names, and limits. Use runtime probing to validate or challenge the documented behavior, not to skip the documentation pass entirely. If the docs MCP is unavailable, fall back to the official OpenAI docs and say that you used the fallback in the report.
- For benchmark or comparison probes, make parity explicit before execution. Record what is held constant, what variable is under test, which response-shape constraints keep the comparison fair, and any usage or token counters that matter for interpreting latency or cost.
- For OpenAI hosted tool probes, remove setup ambiguity before attributing a negative result to runtime behavior:
- Force the tool path with the matching `tool_choice` when the question depends on tool invocation.
- Treat `container_auto` and `container_reference` as separate cases, not interchangeable setup details.
- Clear unsupported model or tool options first so they do not invalidate the probe.
## Workflow
1. Restate the investigation target in operational terms. Name the runtime surface, the key uncertainty, and the highest-risk behaviors to test.
2. Do a short preflight. Check the relevant code or docs first, decide whether the question needs local or live validation, and note any repo, baseline, or release boundary that matters.
3. Create a validation matrix before executing probes. Cover both baseline behavior and the most relevant failure or drift cases. The matrix can live in a scratch note, a temporary file, or a structured header inside the probe script.
4. For each case, choose an execution mode up front:
- `single-shot` for deterministic one-run checks.
- `repeat-N` for cache, retry, streaming, interruption, rate-limit, concurrency, or other run-to-run-sensitive behavior.
- `warm-up + repeat-N` when first-run cold-start effects could distort the result.
Use these defaults unless the task clearly needs something else:
- Quick screen of a repeat-sensitive question: `repeat-3`.
- Decision-grade latency or release recommendation: `warm-up + repeat-10`.
- Costly live cases: start at `repeat-3`, then expand only if the answer remains unclear.
If it is genuinely unclear whether extra runs are worth the time or cost, ask the user before expanding the probe.
5. When the question is benchmark-like or comparative, run in phases. Start with a high-signal pilot matrix against a control, then expand only the surviving candidates or unresolved cases.
6. If the question is about a suspected regression or behavior change, add at least one known-good control case such as `origin/main`, the latest release, or the same request without the suspected option.
7. For comparative probes, define parity before execution. Record prompt or input shape, tool-choice setup, model-settings parity, state reuse rules, and any response-shape constraint that keeps the comparison fair. If materially different output length could bias the result, record usage or token notes too.
8. If the question asks whether one option has the same intelligence or quality as another, decide whether the matrix supports only example-pattern parity or a broader quality claim. For broader claims, add at least one harder or more open-ended case. Otherwise say explicitly that the result is limited to the covered patterns.
9. Plan state controls before execution when hidden state could affect the result. Record whether each case uses fresh or reused state, how cache reuse or cache busting is handled, what unique IDs isolate repeated runs, and how cleanup is verified.
10. If any live case will read environment variables, list the exact variable names and purpose for each case, then ask the user for approval before execution. Prefer `request_user_input` for this gate when it is available, with no auto-resolution and choices that grant or deny only this specific probe. Keep the approval ask short and include destination, read-only versus mutating or costly risk, exact variable names, and cleanup or rollback if relevant.
11. Build task-specific probe scripts in a temporary location. Keep the script small, observable, and easy to discard.
12. In `openai-agents-python`, make the runtime context explicit:
- Run Python probes from the repository root with `uv run python` when practical.
- Record the current commit, working directory, Python executable, and Python version.
- Avoid accidental imports from a different checkout or site-packages location. If you must deviate from `uv run python`, say exactly why and what interpreter or environment was used instead.
13. Execute the matrix and capture evidence. Record request shape, setup, observation summary, unexpected or negative result, error details, timing, runtime context, approved environment-variable names, repeat counts, warm-up handling, variance when relevant, cleanup behavior, and for comparisons note what was held constant plus any response-shape or usage notes that affect interpretation.
14. Update the matrix with actual outcomes, not guesses.
15. Keep temporary artifacts until the final response is drafted. Then delete them unless the user asked to keep them or they are needed for follow-up. Benchmark and repeat-heavy probes often need follow-up, so keeping artifacts is normal when the result may be revisited. If deleted, retain and report a short run summary.
16. Report findings first, with unexpected or negative findings first. Then summarize how the validation was performed and which cases were covered.
17. If the probe isolates one clear defect, you may include a short implementation hypothesis or minimal repro direction. Do not expand into a larger next-step plan unless the user asked for it.
## Validation Matrix
Use a matrix that makes the news easy to scan. Start from the runtime question and the observation summary, not just from `expected` and `pass` or `fail`.
Use a matrix with at least these columns:
- `case_id`
- `scenario`
- `mode`
- `question`
- `setup`
- `observation_summary`
- `result_flag`
- `evidence`
Add these columns when they materially improve the investigation:
- `comparison_basis`
- `variable_under_test`
- `held_constant`
- `output_constraint`
- `status`
- `confidence`
- `state_setup`
- `repeats`
- `warm_up`
- `variance`
- `usage_note`
- `risk_profile`
- `env_vars`
- `approval`
- `control`
Treat `result_flag` as a fast scan field such as `unexpected`, `negative`, `expected`, or `blocked`. Use `status` only when there is a credible comparison basis, baseline, or documented contract to compare against.
Always consider whether the matrix should include these categories:
- Baseline success.
- Control or baseline comparison when a regression is suspected.
- Boundary input or parameter variation.
- Invalid or unsupported input.
- Missing or incorrect configuration.
- Transient external failure such as timeout, network interruption, or rate limiting.
- Retry, idempotence, or cleanup behavior.
- Concurrency or overlapping operations when shared state or ordering may matter.
- Open-ended quality or intelligence samples when the question is broader than pattern parity.
Open [validation-matrix.md](./references/validation-matrix.md) when you need a stronger prioritization model or a reusable case template.
## Temporary Probe Scripts
Write one-off scripts in a temporary file or temporary directory such as one created by `mktemp -d` or Python `tempfile`. Keep the script outside the repository by default, even when it imports code from the repository.
If the probe needs repository code:
- Run it with the repository as the working directory, or
- Set `PYTHONPATH` or the equivalent import path explicitly.
- In `openai-agents-python`, prefer `uv run python /tmp/probe.py` from the repository root.
Design the probe to maximize observability:
- Print or log the exact scenario being exercised.
- Capture runtime context such as git SHA, working directory, Python executable and version, relevant package versions, model or deployment name, endpoint or base URL alias, and any retry or tool options that materially affect behavior.
- For live probes, record only the names of environment variables that were approved for use. Never print their values.
- Capture structured outputs when possible.
- Preserve raw error type, message, and status code.
- For repeat-sensitive cases, capture the attempt index, warm-up status, and any stable identifiers that help compare runs.
- For repeated or benchmark-style probes, write both raw results and a compact summary artifact when practical.
- Keep branching minimal so each script answers a narrow question.
Before deleting the temporary script or directory, keep a short run summary of the script path, command used, runtime context, and whether the evidence was kept or deleted.
Open [python_probe.py](./templates/python_probe.py) when you want a lightweight disposable Python probe scaffold.
## Reporting
Report in this order:
1. Findings. Put unexpected or negative findings first. If there was no real news, say that explicitly.
2. Validation approach. Summarize the code used, the runtime surface exercised, the execution modes, and the case matrix coverage.
3. Case results. Include the matrix or a condensed version of it when the case count is large.
4. Artifact status and brief run summary. State whether temporary artifacts were deleted or kept, and provide kept paths or the retained summary.
5. Optional implementation note. Include this only when one clear defect was isolated and a short implementation direction would help.
For comparative probes, the report should also say what was held constant, what variable was under test, and whether the result supports only pattern parity or a broader quality claim.
Open [reporting-format.md](./references/reporting-format.md) for the recommended response template.
## Resources
- Open [validation-matrix.md](./references/validation-matrix.md) to design and prioritize the case matrix.
- Open [error-cases.md](./references/error-cases.md) to expand common failure scenarios.
- Open [openai-runtime-patterns.md](./references/openai-runtime-patterns.md) for recurring OpenAI and Responses API probe patterns.
- Open [reporting-format.md](./references/reporting-format.md) for the final report structure.
- Open [python_probe.py](./templates/python_probe.py) for a minimal disposable Python probe scaffold.
@@ -0,0 +1,6 @@
interface:
display_name: "Runtime Behavior Probe"
short_description: "Plan and run runtime behavior probes"
default_prompt: "Use $runtime-behavior-probe to investigate actual runtime behavior with a validation matrix, explicit state controls, and a findings-first report."
policy:
allow_implicit_invocation: false
@@ -0,0 +1,80 @@
# Common Error Cases
Use this reference to expand beyond the happy path. Favor error cases that a real user or operator is likely to hit.
## Configuration Errors
Check whether the runtime behaves differently for:
- Missing required environment variables.
- Present but malformed secrets or identifiers.
- Wrong endpoint or base URL.
- Wrong model or deployment name.
- Incompatible local dependency versions.
Look for:
- Error type and status code.
- Whether the failure is immediate or delayed.
- Whether the message is actionable.
- Whether retrying without fixing configuration changes anything.
## Input Errors
Probe common bad-input patterns such as:
- Missing required fields.
- Wrong data type.
- Unsupported enum or option value.
- Empty but syntactically valid input.
- Oversized input or too many items.
- Mutually incompatible options.
Prefer realistic invalid inputs over artificial nonsense. The point is to learn how the runtime fails in practice.
## Transport and Availability Errors
When networked services are involved, consider:
- Connection failure.
- Read timeout.
- Server timeout or upstream gateway error.
- Rate limit response.
- Partial stream interruption.
- Reusing a connection after a failure.
Capture whether the client library retries automatically, whether it surfaces retry metadata, and whether the final exception preserves the original cause.
## State and Repetition Errors
Many surprising bugs appear only when an operation is repeated or interrupted:
- Re-submit the same request.
- Repeat after a timeout.
- Retry after a partial tool call or partial stream.
- Resume after local cleanup or process restart.
- Repeat with slightly changed inputs while reusing shared state.
Observe whether the operation is idempotent, duplicated, silently ignored, or left in a partial state.
## Concurrency Errors
When shared state, ordering, or isolation may matter, consider:
- Two overlapping requests with the same logical input.
- Parallel runs that reuse the same cache key, session, container, or temporary resource.
- Concurrent retries, cancellation, or cleanup racing with active work.
- Output or event streams from one run leaking into another.
Capture whether the runtime serializes, rejects, duplicates, corrupts, or cross-contaminates the work.
## Investigation Heuristics
Use these heuristics to pick error cases quickly:
- Ask which failure a real engineer would debug first in production.
- Ask which failure is most expensive if it is misunderstood.
- Ask which failure would be invisible from code review alone.
- Ask which failure path is likely to differ across environments.
If the error behavior is already perfectly obvious from a local validator or type system, it is usually low priority for this skill.
@@ -0,0 +1,137 @@
# OpenAI Runtime Patterns
Use this reference for recurring OpenAI investigations so you do not have to rediscover the probe strategy each time. In this repository, use [$openai-knowledge](../../openai-knowledge/SKILL.md) up front for contract-sensitive details, then use this reference to design the runtime validation. If the docs MCP is unavailable, fall back to the official OpenAI docs and say so in the report.
## General Rules
- Prefer small live probes over large harnesses.
- Keep one script focused on one uncertainty.
- For comparative or benchmark-like questions, start with a pilot and expand only when the answer is still unclear.
- Capture both the request shape and the returned item types.
- Preserve raw error payloads and status codes.
- Record whether behavior differs between the first call and a repeated call.
- When the question is about regression or contract drift, add a known-good control run before attributing the result to the change under investigation.
- Keep comparison parity explicit. Record what was held constant, what variable changed, and whether output-shape or usage differences could bias the conclusion.
- When the question depends on tool invocation, force the target path with the matching `tool_choice`.
- Treat `container_auto` and `container_reference` as distinct setup modes, not interchangeable details.
- Clear unsupported model or tool options before diagnosing runtime behavior.
## Standard Environment Variables
Do not read these variables automatically. Before a live probe uses any of them, tell the user the exact variable names you plan to read and why each one is needed, then wait for explicit approval. Never print their values:
- `OPENAI_API_KEY`
- `OPENAI_BASE_URL`
- `OPENAI_ORG_ID`
- `OPENAI_PROJECT_ID`
If the task targets another standard integration, use that integration's expected default variable names under the same rule.
## Environment False Signals
Before attributing a failure to the patch under review, exclude environment and source-selection problems with a control run.
- Confirm the commit and worktree under test. When editable installs, shared environments, `PYTHONPATH`, or generated artifacts can select stale code, verify the imported package path and rebuild before probing.
- Run base and head controls with the same interpreter, dependencies, environment variables, and command shape.
- Treat proxy initialization, sandbox denials, unavailable containers, expired snapshots, authentication, quotas, rate limits, service outages, and stale caches as environment conditions until a controlled rerun ties them to the patch.
- Never print proxy URLs or credentials. Change only the minimum in-scope environment or disposable state needed for the control run, and record which variable names or constraints changed.
In the final report, distinguish code failures, unsupported configurations, environment blockers, and inconclusive probes. Do not combine them into one failed-test count.
## Responses API Probe Patterns
For Responses API work, start from the uncertainty instead of from the full feature surface.
### Benchmark or model-switch comparisons
Use when you need to compare models, settings, transports, or providers with enough rigor to support a product or release decision.
Probe suggestions:
- Start with a pilot that includes one control and two or three highest-signal scenarios.
- Keep prompt shape, tool choice, state setup, and non-tested settings aligned across candidates.
- If the question is about speed, capture medians and, when relevant, first-token latency plus any usage note that could explain the difference.
- If the question is about "same intelligence" or "same quality," add at least one harder or more open-ended case. Otherwise report the result as pattern parity only.
- Expand to a larger matrix only when the pilot survives, the candidates are close, or a major runtime surface is still uncovered.
### Plain response behavior
Use when you need to confirm:
- The shape of returned output items.
- Whether text appears in one item or multiple items.
- How metadata appears in the final object.
Probe suggestions:
- Baseline call with a minimal input.
- Same call with a slightly different instruction shape.
- Repeat the same call to check output stability where that matters.
### Structured output behavior
Use when you need to observe:
- Schema rejection versus best-effort completion.
- Handling of missing required fields.
- Differences between model-compliant output and transport-level errors.
Probe suggestions:
- Valid schema and valid prompt.
- Prompt likely to produce omitted fields.
- Clearly incompatible schema or unsupported option when relevant.
### Tool invocation behavior
Use when you need to learn:
- When tool calls are emitted.
- How arguments are shaped at runtime.
- What happens when the tool fails or returns malformed output.
Probe suggestions:
- Baseline tool-call success.
- Tool failure with a realistic exception.
- Tool result that is syntactically valid but semantically incomplete.
### Hosted shell and code interpreter failure shields
When probing hosted tools through the Responses API, eliminate common setup ambiguity first:
- Force the tool path you want to test with the matching `tool_choice`. A text-only completion without forced tool choice is not a reliable negative result.
- Treat `container_auto` and `container_reference` differently. Use `container_auto` when the probe needs fresh container provisioning or skill attachment, and use `container_reference` only to reuse existing container state.
- Do not assume every environment field is accepted on every container mode. If the probe is about skills, validate that the chosen container mode actually supports skill attachment before treating an API error as a runtime defect.
- Check model-specific option support before chasing unrelated failures. Unsupported reasoning or model settings can invalidate the probe before the tool path is exercised.
- For hosted package installation, treat network-dependent setup as best-effort and separate install failures from the underlying tool behavior you are trying to observe.
- For prompt cache investigations, keep model, instructions, tool configuration, and cache key effectively identical across repeated runs before interpreting `cached_tokens`.
### Streaming behavior
Use when the uncertainty involves:
- Event ordering.
- Partial text delivery.
- Termination after interruption.
- Tool-call events in streams.
Probe suggestions:
- Normal streamed completion.
- Early local cancellation.
- Network interruption if it can be reproduced safely.
## What to Capture
For OpenAI probes, try to record:
- Request options that materially affect behavior.
- Response item types and their order.
- Whether fields are absent, null, empty, or transformed.
- Server status and error payload details for failures.
- Retry and backoff hints when present.
- Stable identifiers that help compare repeated runs, such as request IDs, response IDs, tool call IDs, or container IDs when available.
- Which environment-variable names were approved for the probe when live credentials were required.
Do not spend time rediscovering static documentation unless the runtime result seems to contradict what you expected. The value of this skill is in the observed behavior.
@@ -0,0 +1,118 @@
# Reporting Format
Lead with findings, not process. The user asked for investigation results, so the answer should start with the most important observed behaviors. Put the real news first.
## Recommended Order
1. Findings.
2. Validation approach.
3. Case matrix or condensed case summary.
4. Artifact status and brief run summary.
5. Optional implementation note.
## Findings Section
Make each finding answer one user-relevant question. Good findings usually include:
- What was observed.
- Why it matters.
- The condition under which it happens.
- What was held constant when the finding comes from a comparison probe.
- `scope`: The boundary of the finding, such as commit, model, Python version, live vs local, or repeat mode.
- `confidence`: `high`, `medium`, or `low`.
Avoid burying the main result under setup details.
Put `unexpected` or `negative` findings first. If there were no unexpected or negative findings in the executed cases, say that explicitly before the rest of the findings section.
If the probe was comparative, say whether the result supports:
- Pattern parity only.
- A broader quality claim.
Do not imply a broader quality equivalence than the executed cases justify.
## Validation Approach Section
Summarize:
- The runtime surface you exercised.
- The shape of the probe code, in overview only.
- Which categories of cases you covered.
- Which execution modes you used, including repeat counts or warm-up handling when relevant.
- Whether live credentials or external services were used.
- Any important state controls such as fresh state, cache reuse, cache busting, unique IDs, or cleanup verification.
- For comparison probes, what was held constant, what was varied, and whether output-shape or usage differences could still influence the conclusion.
- Whether the usual docs path or an official-docs fallback was used for contract-sensitive checks.
Keep this concise. The user needs enough detail to trust the result, not a line-by-line replay of the script.
## Case Summary
Include either the full matrix or a condensed summary. At minimum, show:
- Which scenarios were executed.
- Whether the run was a quick pilot, an expanded matrix, or both.
- Which ones produced `unexpected` or `negative` results.
- Which ones passed or failed when a real comparison basis existed.
- Which cases were blocked.
- Where the supporting evidence lived, or that it was deleted.
If the matrix is large, show the highest-value cases in the main response and keep the rest as a compact appendix or note.
## Artifact Status And Brief Run Summary
State one of these explicitly:
- Temporary artifacts were kept until the final response was drafted, then deleted after validation.
- Temporary artifacts were kept at `<path>` because the user asked to keep them.
- Temporary artifacts were kept at `<path>` because they are needed for follow-up analysis.
Even if artifacts were deleted, retain a short run summary such as:
- Probe command or runner shape.
- Runtime context summary such as commit, Python executable, Python version, or model.
- Artifact path and final status.
For benchmark or repeat-heavy probes, keeping artifacts for follow-up is often the right default even when the immediate report is done.
## Optional Implementation Note
Include this only when one clear defect was isolated and a short implementation hypothesis or minimal repro direction would help. Keep it brief. Do not turn the report into a broader next-step plan unless the user asked for that.
## Compact Template
Use this outline when you need a fast structure:
Findings:
- <finding 1>
held constant: <prompt/tool/state settings kept the same, if comparative>
scope: <commit/model/python/live-local/repeat-mode>
confidence: <high|medium|low>
- <finding 2>
held constant: <prompt/tool/state settings kept the same, if comparative>
scope: <commit/model/python/live-local/repeat-mode>
confidence: <high|medium|low>
Validation approach:
- Surface: <what was exercised>
- Probe code: <brief overview>
- Coverage: <success, edge, error, repeat-sensitive, and quality categories>
- Execution modes: <single-shot|repeat-N|warm-up + repeat-N>
- Comparison parity: <what was held constant and what varied, if comparative>
- Docs source: <MCP or official-docs fallback, if relevant>
Case summary:
| case_id | scenario | result_flag | status | note |
| --- | --- | --- | --- | --- |
| S1 | ... | expected | pass | ... |
| E1 | ... | negative | fail | ... |
Artifact status and brief run summary:
- Temporary artifacts were kept until the final response was drafted, then deleted.
- Summary: <command/runtime-context/artifact-status summary>
Optional implementation note:
- <brief hypothesis or minimal repro direction>
Adjust the format to the task, but preserve the ordering.
@@ -0,0 +1,137 @@
# Validation Matrix
Use the matrix to decide what to probe before writing scripts. The goal is not exhaustive combinatorics; the goal is high-value coverage that is visible, explainable, and likely to reveal runtime surprises. The matrix should make the real news easy to scan.
## Minimum Columns
Use these columns unless the task clearly needs more:
- `case_id`: Stable identifier such as `S1`, `E3`, or `R2`.
- `scenario`: Short description of the behavior under test.
- `mode`: `single-shot`, `repeat-N`, or `warm-up + repeat-N`.
- `question`: The concrete runtime uncertainty this case is answering.
- `setup`: Inputs, environment, or preconditions required for the case.
- `observation_summary`: A compact summary of what actually happened.
- `result_flag`: `unexpected`, `negative`, `expected`, or `blocked`.
- `evidence`: Path, log reference, or `deleted`.
Add these columns when they materially improve the investigation:
- `comparison_basis`: The baseline, docs, or prior behavior you are comparing against.
- `variable_under_test`: The single factor that is intentionally changing in a comparison case.
- `held_constant`: Prompt shape, tool setup, model settings, or state rules that were intentionally kept the same.
- `output_constraint`: Any schema, length, or response-shape constraint used to keep the comparison fair.
- `status`: Use `pass`, `fail`, `unexpected-pass`, `unexpected-fail`, or `blocked` only when there is a credible comparison basis or control.
- `confidence`: `high`, `medium`, or `low`.
- `state_setup`: Fresh or reused state, cache strategy, unique IDs, and cleanup checks.
- `repeats`: Number of measured runs.
- `warm_up`: Whether a warm-up run was used and why.
- `variance`: Any useful spread or instability note across repeated runs.
- `usage_note`: Token, usage, or output-length note when it materially affects interpretation.
- `control`: Known-good comparison point for regression or behavior-change questions.
- `risk_profile`: `read-only`, `mutating`, or `costly` for live probes.
- `env_vars`: Exact environment-variable names the case plans to read.
- `approval`: `not-needed`, `pending`, or `approved` for cases that need user permission before execution.
Use `result_flag` as the fast scan field. It is what makes unexpected or negative findings jump out before the reader studies the full report.
Use `status` only when you have a real comparison basis. If the case is exploratory and there is no trustworthy baseline, prefer a strong `observation_summary` plus `result_flag` and `confidence` instead of pretending the result is a clean pass or fail.
## Choosing Execution Mode
Pick an execution mode before you run the case:
- Use `single-shot` for deterministic, one-run checks.
- Use `repeat-N` automatically when the question involves cache behavior, retries, streaming, interruptions, rate limiting, concurrency, or other run-to-run-sensitive behavior.
- Use `warm-up + repeat-N` when the first run is likely to include cold-start effects such as container provisioning, import caches, or prompt-cache population.
Use these defaults unless the task clearly needs something else:
- `repeat-3` for a quick screen of a repeat-sensitive question.
- `warm-up + repeat-10` for decision-grade latency comparisons or release-facing recommendations.
- For costly live probes, start at `repeat-3` and expand only if the answer is still unclear.
If it is genuinely unclear whether extra runs are worth the time or cost, ask the user before expanding the probe.
## Phase The Matrix
When the question is comparative or benchmark-like, do not jump straight to the largest matrix.
Start with a pilot:
- One control.
- One or two highest-signal success cases.
- The smallest repeat count that can disqualify a weak candidate quickly.
Expand only when:
- The candidate survives the pilot.
- The results are close enough that more samples matter.
- A major runtime surface is still uncovered.
- The user explicitly wants decision-grade evidence.
## Coverage Categories
Try to cover at least one case from each relevant category:
- `success`: Normal behavior that should work.
- `control`: Known-good comparison such as `origin/main`, the latest release, or the same request without the suspected option.
- `boundary`: Size, count, or parameter limits near a plausible edge.
- `invalid`: Bad inputs or unsupported combinations.
- `misconfig`: Missing key, wrong endpoint, bad permissions, or incompatible local setup.
- `transient`: Timeout, temporary server issue, network breakage, or rate limiting.
- `recovery`: Retry behavior, partial completion, duplicate submission, or cleanup.
- `concurrency`: Overlapping operations when shared state, ordering, or isolation may matter.
- `quality`: A harder or more open-ended sample when the user is asking about model intelligence, not just workflow parity.
If time is limited, prioritize categories in this order:
1. Known-good control when the question implies regression or drift.
2. Highest-risk success case.
3. Most plausible user-facing failure.
4. Most likely edge case with ambiguous behavior.
5. Cleanup or retry semantics.
6. Lower-probability extremes.
## Matrix Template
Use this compact template:
| case_id | scenario | mode | question | setup | state_setup | variable_under_test | held_constant | comparison_basis | observation_summary | result_flag | status | evidence |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| K1 | Known-good control | single-shot | Does the baseline still show the expected behavior? | Same probe against baseline target | Fresh state | none | current probe shape | `origin/main` or latest release | pending | pending | pending | pending |
| S1 | Baseline success | single-shot | What does the normal success path look like at runtime? | Valid config and representative input | Fresh state | none | representative input and setup | current docs or local expectation | pending | pending | pending | pending |
| R1 | Cache or retry behavior | warm-up + repeat-N | Does behavior change after the first run or across retries? | Same request repeated under controlled settings | Cache key or retry setup recorded | reuse versus fresh state | prompt shape and tool setup | same request without reuse, or docs if available | pending | pending | pending | pending |
| C1 | Model comparison pilot | warm-up + repeat-N | Does candidate B preserve the covered behavior while improving latency? | Same scenario across two models | Fresh state and stable IDs | model name | prompt shape, tool choice, and model settings parity | control model in the same probe | pending | pending | pending | pending |
| E1 | Invalid input | single-shot | How does the runtime reject a realistic bad input? | Missing required field | Fresh state | invalid field value | same request with valid field | same request with valid field | pending | pending | pending | pending |
| X1 | Concurrent overlap | repeat-N | Do overlapping runs interfere with each other? | Two or more overlapping operations | Unique IDs plus cleanup verification | overlap timing | same logical input | same request serialized, if available | pending | pending | pending | pending |
## Recording Results
Keep `question` unchanged after execution. Put the actual behavior in `observation_summary`, then mark the scan-friendly `result_flag`.
Use these `result_flag` values consistently:
- `unexpected`: The result diverged from the best current understanding in a surprising way.
- `negative`: The result exposed a user-relevant failure, risk, or sharp edge.
- `expected`: The result matched the current understanding and did not reveal new risk.
- `blocked`: The case did not produce a trustworthy observation.
Only fill `status` when there is a credible comparison basis. Otherwise use `observation_summary`, `result_flag`, and `confidence` to communicate what was learned without over-claiming certainty.
For comparison cases, use `observation_summary` and the final report to say whether the evidence supports pattern parity only or a broader quality claim.
If a case reveals a new branch of behavior, add a follow-up case instead of overloading the original one.
## Evidence Discipline
Treat a case as incomplete when:
- The observed output omits the key result you were testing.
- The script mixed multiple questions and the result is ambiguous.
- Hidden state, cache behavior, or previous runs may have influenced the result and were not controlled or documented.
- The question is whether behavior changed, but the case has no credible control or baseline to compare against.
- The case plans to read environment variables, but the exact variable names were not approved by the user before execution.
- The case was repeat-sensitive, but it ran only once without a clear rationale.
When this happens, narrow the probe and rerun. A smaller script with a cleaner result is better than a more complicated script that is hard to trust.
@@ -0,0 +1,227 @@
"""Disposable Python probe scaffold.
Copy this file to a temporary location and adapt it for one narrow question.
Recommended usage from the repository root:
uv run python /tmp/probe.py
If you want structured artifacts for repeat-heavy or benchmark probes:
PROBE_OUTPUT_DIR=/tmp/probe-run uv run python /tmp/probe.py
"""
from __future__ import annotations
import json
import os
import platform
import shutil
import statistics
import subprocess
import sys
import time
import uuid
from collections import Counter, defaultdict
from importlib import metadata
from pathlib import Path
SCENARIO = "replace-me"
RUN_LABEL = "replace-me"
MODE = "single-shot"
APPROVED_ENV_VARS: list[str] = []
OUTPUT_DIR_ENV = "PROBE_OUTPUT_DIR"
RESULTS: list[dict[str, object]] = []
def _git_value(*args: str) -> str:
result = subprocess.run(
["git", *args],
check=False,
capture_output=True,
text=True,
)
if result.returncode != 0:
return "unknown"
return result.stdout.strip() or "unknown"
def _package_version(name: str) -> str | None:
try:
return metadata.version(name)
except metadata.PackageNotFoundError:
return None
def _output_dir() -> Path | None:
value = os.getenv(OUTPUT_DIR_ENV)
if not value:
return None
return Path(value)
def _write_json(path: Path, payload: object) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
def emit(kind: str, **payload: object) -> None:
print(
json.dumps(
{
"ts": round(time.time(), 3),
"kind": kind,
**payload,
},
sort_keys=True,
)
)
def runtime_context() -> dict[str, object]:
approved = {name: ("set" if os.getenv(name) else "unset") for name in APPROVED_ENV_VARS}
package_versions = {
name: version
for name in ("openai", "agents")
if (version := _package_version(name)) is not None
}
return {
"scenario": SCENARIO,
"run_label": RUN_LABEL,
"mode": MODE,
"cwd": os.getcwd(),
"script_path": str(Path(__file__).resolve()),
"python_executable": sys.executable,
"python_version": sys.version.split()[0],
"platform": platform.platform(),
"git_commit": _git_value("rev-parse", "HEAD"),
"git_branch": _git_value("rev-parse", "--abbrev-ref", "HEAD"),
"uv_path": shutil.which("uv"),
"package_versions": package_versions,
"approved_env_vars": approved,
"output_dir": str(_output_dir()) if _output_dir() else None,
}
def start_case(case_id: str, *, mode: str = MODE, note: str | None = None) -> None:
emit("case_start", case_id=case_id, mode=mode, note=note)
def record_case_result(
case_id: str,
observation_summary: str,
result_flag: str,
*,
mode: str = MODE,
is_warmup: bool = False,
total_latency_s: float | None = None,
first_token_latency_s: float | None = None,
metrics: dict[str, object] | None = None,
error: str | None = None,
) -> None:
payload: dict[str, object] = {
"case_id": case_id,
"mode": mode,
"is_warmup": is_warmup,
"observation_summary": observation_summary,
"result_flag": result_flag,
"metrics": metrics or {},
"error": error,
}
if total_latency_s is not None:
payload["total_latency_s"] = total_latency_s
if first_token_latency_s is not None:
payload["first_token_latency_s"] = first_token_latency_s
RESULTS.append(payload)
emit("case_result", **payload)
def summarize_results() -> dict[str, object]:
by_case: defaultdict[str, list[dict[str, object]]] = defaultdict(list)
for result in RESULTS:
by_case[str(result["case_id"])].append(result)
summary_cases: dict[str, object] = {}
for case_id, items in by_case.items():
measured = [item for item in items if not bool(item.get("is_warmup"))]
latencies = [
float(item["total_latency_s"])
for item in measured
if item.get("total_latency_s") is not None
]
first_token_latencies = [
float(item["first_token_latency_s"])
for item in measured
if item.get("first_token_latency_s") is not None
]
result_flags = Counter(str(item["result_flag"]) for item in measured or items)
observations = [str(item["observation_summary"]) for item in (measured or items)[:3]]
summary_cases[case_id] = {
"mode": str(items[-1]["mode"]),
"runs": len(measured),
"warmups": len(items) - len(measured),
"result_flags": dict(result_flags),
"median_total_latency_s": (statistics.median(latencies) if latencies else None),
"mean_total_latency_s": statistics.mean(latencies) if latencies else None,
"median_first_token_latency_s": (
statistics.median(first_token_latencies) if first_token_latencies else None
),
"observations": observations,
}
return {
"scenario": SCENARIO,
"run_label": RUN_LABEL,
"mode": MODE,
"result_count": len(RESULTS),
"cases": summary_cases,
"result_flags": dict(Counter(str(item["result_flag"]) for item in RESULTS)),
}
def finalize(exit_code: int) -> None:
metadata_payload = {
"exit_code": exit_code,
"runtime_context": runtime_context(),
}
summary_payload = summarize_results()
emit("summary", metadata=metadata_payload, summary=summary_payload)
output_dir = _output_dir()
if not output_dir:
return
metadata_path = output_dir / "metadata.json"
results_path = output_dir / "results.json"
summary_path = output_dir / "summary.json"
_write_json(metadata_path, metadata_payload)
_write_json(results_path, RESULTS)
_write_json(summary_path, summary_payload)
emit(
"artifact_paths",
metadata_path=str(metadata_path),
results_path=str(results_path),
summary_path=str(summary_path),
)
def main() -> int:
case_id = os.getenv("PROBE_CASE_ID", f"case-{uuid.uuid4().hex[:8]}")
emit("banner", context=runtime_context())
start_case(case_id)
# Replace this block with the narrow runtime question you want to test.
observation = "replace-me"
result_flag = "expected"
record_case_result(
case_id=case_id,
observation_summary=observation,
result_flag=result_flag,
)
finalize(exit_code=0)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,42 @@
---
name: test-coverage-improver
description: 'Improve test coverage in the OpenAI Agents Python repository: run `make coverage`, inspect coverage artifacts, identify low-coverage files, propose high-impact tests, and confirm with the user before writing tests.'
---
# Test Coverage Improver
## Overview
Use this skill whenever coverage needs assessment or improvement (coverage regressions, failing thresholds, or user requests for stronger tests). It runs the coverage suite, analyzes results, highlights the biggest gaps, and prepares test additions while confirming with the user before changing code.
## Quick Start
1. From the repo root run `make coverage` to regenerate `.coverage` data and `coverage.xml`.
2. Collect artifacts: `.coverage` and `coverage.xml`, plus the console output from `coverage report -m` for drill-downs.
3. Summarize coverage: total percentages, lowest files, and uncovered lines/paths.
4. Draft test ideas per file: scenario, behavior under test, expected outcome, and likely coverage gain.
5. Ask the user for approval to implement the proposed tests; pause until they agree.
6. After approval, write the tests in `tests/`, rerun `make coverage`, and then run `$code-change-verification` before marking work complete.
## Workflow Details
- **Run coverage**: Execute `make coverage` at repo root. Avoid watch flags and keep prior coverage artifacts only if comparing trends.
- **Parse summaries efficiently**:
- Prefer the console output from `coverage report -m` for file-level totals; fallback to `coverage.xml` for tooling or spreadsheets.
- Use `uv run coverage html` to generate `htmlcov/index.html` if you need an interactive drill-down.
- **Prioritize targets**:
- Public APIs or shared utilities in `src/agents/` before examples or docs.
- Files with low statement coverage or newly added code at 0%.
- Recent bug fixes or risky code paths (error handling, retries, timeouts, concurrency).
- **Design impactful tests**:
- Hit uncovered paths: error cases, boundary inputs, optional flags, and cancellation/timeouts.
- Cover combinational logic rather than trivial happy paths.
- Place tests under `tests/` and avoid flaky async timing.
- **Coordinate with the user**: Present a numbered, concise list of proposed test additions and expected coverage gains. Ask explicitly before editing code or fixtures.
- **After implementation**: Rerun coverage, report the updated summary, and note any remaining low-coverage areas.
## Notes
- Keep any added comments or code in English.
- Do not create `scripts/`, `references/`, or `assets/` unless needed later.
- If coverage artifacts are missing or stale, rerun `pnpm test:coverage` instead of guessing.
@@ -0,0 +1,4 @@
interface:
display_name: "Test Coverage Improver"
short_description: "Analyze coverage gaps and propose high-impact tests"
default_prompt: "Use $test-coverage-improver to analyze coverage gaps, propose high-impact tests, and update coverage after approval."
+4
View File
@@ -0,0 +1,4 @@
#:schema https://developers.openai.com/codex/config-schema.json
[features]
codex_hooks = true
+15
View File
@@ -0,0 +1,15 @@
{
"hooks": {
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "uv run python \"$(git rev-parse --show-toplevel)/.codex/hooks/stop_repo_tidy.py\"",
"timeout": 20
}
]
}
]
}
}
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env python3
from __future__ import annotations
import hashlib
import json
import subprocess
import sys
import tempfile
from dataclasses import asdict, dataclass
from pathlib import Path
MAX_RUFF_FIX_FILES = 20
PYTHON_SUFFIXES = {".py", ".pyi"}
@dataclass
class HookState:
last_tidy_fingerprint: str | None = None
def write_stop_block(reason: str, system_message: str) -> None:
sys.stdout.write(
json.dumps(
{
"decision": "block",
"reason": reason,
"systemMessage": system_message,
}
)
)
def run_command(cwd: str, *args: str) -> subprocess.CompletedProcess[str]:
try:
return subprocess.run(
args,
cwd=cwd,
capture_output=True,
check=False,
text=True,
)
except FileNotFoundError as exc:
return subprocess.CompletedProcess(args, returncode=127, stdout="", stderr=str(exc))
def run_git(cwd: str, *args: str) -> subprocess.CompletedProcess[str]:
return run_command(cwd, "git", *args)
def git_root(cwd: str) -> str:
result = run_git(cwd, "rev-parse", "--show-toplevel")
if result.returncode != 0:
raise RuntimeError(result.stderr.strip() or "git root lookup failed")
return result.stdout.strip()
def parse_status_paths(repo_root: str) -> list[str]:
unstaged = run_git(repo_root, "diff", "--name-only", "--diff-filter=ACMR")
untracked = run_git(repo_root, "ls-files", "--others", "--exclude-standard")
if unstaged.returncode != 0 or untracked.returncode != 0:
return []
paths = {
line.strip()
for result in (unstaged, untracked)
for line in result.stdout.splitlines()
if line.strip()
}
return sorted(paths)
def untracked_paths(repo_root: str, paths: list[str]) -> set[str]:
if not paths:
return set()
result = run_git(repo_root, "ls-files", "--others", "--exclude-standard", "--", *paths)
if result.returncode != 0:
return set()
return {line.strip() for line in result.stdout.splitlines() if line.strip()}
def fingerprint_for_paths(repo_root: str, paths: list[str]) -> str | None:
if not paths:
return None
repo_root_path = Path(repo_root)
untracked = untracked_paths(repo_root, paths)
tracked_paths = [file_path for file_path in paths if file_path not in untracked]
diff_parts: list[str] = []
if tracked_paths:
diff = run_git(repo_root, "diff", "--no-ext-diff", "--binary", "--", *tracked_paths)
if diff.returncode == 0:
diff_parts.append(diff.stdout)
for file_path in sorted(untracked):
try:
digest = hashlib.sha256((repo_root_path / file_path).read_bytes()).hexdigest()
except OSError:
continue
diff_parts.append(f"untracked:{file_path}:{digest}")
if not diff_parts:
return None
return hashlib.sha256("\n".join(diff_parts).encode("utf-8")).hexdigest()
def state_dir() -> Path:
return Path(tempfile.gettempdir()) / "openai-agents-python-codex-hooks"
def state_path(session_id: str, repo_root: str) -> Path:
root_hash = hashlib.sha256(repo_root.encode("utf-8")).hexdigest()[:12]
safe_session_id = "".join(
ch if ch.isascii() and (ch.isalnum() or ch in "._-") else "_" for ch in session_id
)
return state_dir() / f"{safe_session_id}-{root_hash}.json"
def load_state(session_id: str, repo_root: str) -> HookState:
file_path = state_path(session_id, repo_root)
if not file_path.exists():
return HookState()
try:
payload = json.loads(file_path.read_text())
except (OSError, json.JSONDecodeError):
return HookState()
return HookState(last_tidy_fingerprint=payload.get("last_tidy_fingerprint"))
def save_state(session_id: str, repo_root: str, state: HookState) -> None:
file_path = state_path(session_id, repo_root)
file_path.parent.mkdir(parents=True, exist_ok=True)
file_path.write_text(json.dumps(asdict(state), indent=2))
def lint_fix_paths(repo_root: str) -> list[str]:
return [
file_path
for file_path in parse_status_paths(repo_root)
if Path(file_path).suffix in PYTHON_SUFFIXES
]
def main() -> None:
try:
payload = json.loads(sys.stdin.read() or "null")
except json.JSONDecodeError:
return
if not isinstance(payload, dict):
return
session_id = payload.get("session_id")
cwd = payload.get("cwd")
if not isinstance(session_id, str) or not isinstance(cwd, str):
return
if payload.get("stop_hook_active"):
return
repo_root = git_root(cwd)
current_paths = lint_fix_paths(repo_root)
if not current_paths or len(current_paths) > MAX_RUFF_FIX_FILES:
return
state = load_state(session_id, repo_root)
current_fingerprint = fingerprint_for_paths(repo_root, current_paths)
if current_fingerprint is None or state.last_tidy_fingerprint == current_fingerprint:
return
format_result = run_command(repo_root, "uv", "run", "ruff", "format", "--", *current_paths)
check_result: subprocess.CompletedProcess[str] | None = None
if format_result.returncode == 0:
check_result = run_command(
repo_root,
"uv",
"run",
"ruff",
"check",
"--fix",
"--",
*current_paths,
)
if format_result.returncode != 0:
write_stop_block(
"`uv run ruff format -- ...` failed for the touched Python files. "
"Review the formatting step before wrapping up.",
"Repo hook: targeted Ruff format failed.",
)
return
if check_result and check_result.returncode != 0:
write_stop_block(
"`uv run ruff check --fix -- ...` failed for the touched Python files. "
"Review the lint output before wrapping up.",
"Repo hook: targeted Ruff lint fix failed.",
)
return
updated_paths = lint_fix_paths(repo_root)
updated_fingerprint = fingerprint_for_paths(repo_root, updated_paths)
state.last_tidy_fingerprint = updated_fingerprint
save_state(session_id, repo_root, state)
if updated_fingerprint != current_fingerprint:
write_stop_block(
"I ran targeted tidy steps on the touched Python files "
"(`ruff format` and `ruff check --fix`). Review the updated diff, "
"then continue or wrap up.",
"Repo hook: ran targeted Ruff tidy on touched files.",
)
if __name__ == "__main__":
main()
+28
View File
@@ -0,0 +1,28 @@
---
name: Bug report
about: Report a bug
title: ''
labels: bug
assignees: ''
---
### Please read this first
- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have faced similar issues.
### Describe the bug
A clear and concise description of what the bug is.
### Debug information
- Agents SDK version: (e.g. `v0.0.3`)
- Python version (e.g. Python 3.14)
### Repro steps
Ideally provide a minimal python script that can be run to reproduce the bug.
### Expected behavior
A clear and concise description of what you expected to happen.
+16
View File
@@ -0,0 +1,16 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''
---
### Please read this first
- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have had similar requests
### Describe the feature
What is the feature you're requesting? How would it work? Please provide examples and details if possible.
+26
View File
@@ -0,0 +1,26 @@
---
name: Custom model providers
about: Questions or bugs about using non-OpenAI models
title: ''
labels: bug
assignees: ''
---
### Please read this first
- **Have you read the custom model provider docs, including the 'Common issues' section?** [Model provider docs](https://openai.github.io/openai-agents-python/models/#using-other-llm-providers)
- **Have you searched for related issues?** Others may have faced similar issues.
### Describe the question
A clear and concise description of what the question or bug is.
### Debug information
- Agents SDK version: (e.g. `v0.0.3`)
- Python version (e.g. Python 3.14)
### Repro steps
Ideally provide a minimal python script that can be run to reproduce the issue.
### Expected behavior
A clear and concise description of what you expected to happen.
+16
View File
@@ -0,0 +1,16 @@
---
name: Question
about: Questions about the SDK
title: ''
labels: question
assignees: ''
---
### Please read this first
- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have had similar requests
### Question
Describe your question. Provide details if available.
@@ -0,0 +1,19 @@
### Summary
<!-- Please give a short summary of the change and the problem this solves. -->
### Test plan
<!-- Please explain how this was tested -->
<!-- If verification could not complete because of local environment setup, include the failing command, the missing dependency, and why it is unrelated to this PR. Leave the pass checkbox below unchecked until all verification steps pass. -->
### Issue number
<!-- For example: "Closes #1234" -->
### Checks
- [ ] I've added new tests, if relevant
- [ ] I've run `.agents/skills/code-change-verification/scripts/run.sh`
- [ ] I've confirmed all verification steps pass
- [ ] If using Codex, I've run `/review` before submitting this PR
+9
View File
@@ -0,0 +1,9 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
open-pull-requests-limit: 5
labels:
- "dependencies"
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env bash
set -euo pipefail
mode="${1:-code}"
base_sha="${2:-${BASE_SHA:-}}"
head_sha="${3:-${HEAD_SHA:-}}"
if [ -z "${GITHUB_OUTPUT:-}" ]; then
echo "GITHUB_OUTPUT is not set." >&2
exit 1
fi
if [ -z "$head_sha" ]; then
head_sha="$(git rev-parse HEAD 2>/dev/null || true)"
fi
if [ -z "$base_sha" ]; then
if ! git rev-parse --verify origin/main >/dev/null 2>&1; then
git fetch --no-tags --depth=1 origin main || true
fi
if git rev-parse --verify origin/main >/dev/null 2>&1 && [ -n "$head_sha" ]; then
base_sha="$(git merge-base origin/main "$head_sha" 2>/dev/null || true)"
fi
fi
if [ -z "$base_sha" ] || [ -z "$head_sha" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "$base_sha" = "0000000000000000000000000000000000000000" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if ! git cat-file -e "$base_sha" 2>/dev/null; then
git fetch --no-tags --depth=1 origin "$base_sha" || true
fi
if ! git cat-file -e "$base_sha" 2>/dev/null; then
echo "run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
changed_files=$(git diff --name-only "$base_sha" "$head_sha" || true)
case "$mode" in
code)
pattern='^(src/|tests/|examples/|pyproject.toml$|uv.lock$|Makefile$)'
;;
docs)
pattern='^(docs/|mkdocs.yml$)'
;;
*)
pattern="$mode"
;;
esac
if echo "$changed_files" | grep -Eq "$pattern"; then
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "run=false" >> "$GITHUB_OUTPUT"
fi
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
repeat_count="${1:-5}"
asyncio_progress_args=(
tests/test_asyncio_progress.py
)
run_step_execution_args=(
tests/test_run_step_execution.py
-k
"cancel or post_invoke"
)
for run in $(seq 1 "$repeat_count"); do
echo "Async teardown stability run ${run}/${repeat_count}"
uv run pytest -q "${asyncio_progress_args[@]}"
uv run pytest -q "${run_step_execution_args[@]}"
done
+184
View File
@@ -0,0 +1,184 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import os
import re
import subprocess
import sys
from urllib import error, request
def warn(message: str) -> None:
print(message, file=sys.stderr)
def parse_version(value: str | None) -> tuple[int, int, int] | None:
if not value:
return None
match = re.match(r"^v?(\d+)\.(\d+)(?:\.(\d+))?", value)
if not match:
return None
major = int(match.group(1))
minor = int(match.group(2))
patch = int(match.group(3) or 0)
return major, minor, patch
def latest_tag_version(exclude_version: tuple[int, int, int] | None) -> tuple[int, int, int] | None:
try:
output = subprocess.check_output(["git", "tag", "--list", "v*"], text=True)
except Exception as exc:
warn(f"Milestone assignment skipped (failed to list tags: {exc}).")
return None
versions: list[tuple[int, int, int]] = []
for tag in output.splitlines():
parsed = parse_version(tag)
if not parsed:
continue
if exclude_version and parsed == exclude_version:
continue
versions.append(parsed)
if not versions:
return None
return max(versions)
def classify_bump(
target: tuple[int, int, int] | None,
previous: tuple[int, int, int] | None,
) -> str | None:
if not target or not previous:
return None
if target < previous:
warn("Milestone assignment skipped (release version is behind latest tag).")
return None
if target[0] != previous[0]:
return "major"
if target[1] != previous[1]:
return "minor"
return "patch"
def parse_milestone_title(title: str | None) -> tuple[int, int] | None:
if not title:
return None
match = re.match(r"^(\d+)\.(\d+)\.x$", title)
if not match:
return None
return int(match.group(1)), int(match.group(2))
def fetch_open_milestones(owner: str, repo: str, token: str) -> list[dict]:
url = f"https://api.github.com/repos/{owner}/{repo}/milestones?state=open&per_page=100"
headers = {
"Accept": "application/vnd.github+json",
"Authorization": f"Bearer {token}",
}
req = request.Request(url, headers=headers)
try:
with request.urlopen(req) as response:
return json.load(response)
except error.HTTPError as exc:
warn(f"Milestone assignment skipped (failed to list milestones: {exc.code}).")
except Exception as exc:
warn(f"Milestone assignment skipped (failed to list milestones: {exc}).")
return []
def select_milestone(milestones: list[dict], required_bump: str) -> str | None:
parsed: list[dict] = []
for milestone in milestones:
parsed_title = parse_milestone_title(milestone.get("title"))
if not parsed_title:
continue
parsed.append(
{
"milestone": milestone,
"major": parsed_title[0],
"minor": parsed_title[1],
}
)
parsed.sort(key=lambda entry: (entry["major"], entry["minor"]))
if not parsed:
warn("Milestone assignment skipped (no open milestones matching X.Y.x).")
return None
majors = sorted({entry["major"] for entry in parsed})
current_major = majors[0]
next_major = majors[1] if len(majors) > 1 else None
current_major_entries = [entry for entry in parsed if entry["major"] == current_major]
patch_target = current_major_entries[0]
minor_target = current_major_entries[1] if len(current_major_entries) > 1 else patch_target
major_target = None
if next_major is not None:
next_major_entries = [entry for entry in parsed if entry["major"] == next_major]
if next_major_entries:
major_target = next_major_entries[0]
target_entry = None
if required_bump == "major":
target_entry = major_target
elif required_bump == "minor":
target_entry = minor_target
else:
target_entry = patch_target
if not target_entry:
warn("Milestone assignment skipped (not enough open milestones for selection).")
return None
return target_entry["milestone"].get("title")
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--version", help="Release version (e.g., 0.6.6).")
parser.add_argument(
"--required-bump",
choices=("major", "minor", "patch"),
help="Override bump type (major/minor/patch).",
)
parser.add_argument("--repo", help="GitHub repository (owner/repo).")
parser.add_argument("--token", help="GitHub token.")
args = parser.parse_args()
required_bump = args.required_bump
if not required_bump:
target_version = parse_version(args.version)
if not target_version:
warn("Milestone assignment skipped (missing or invalid release version).")
return 0
previous_version = latest_tag_version(target_version)
required_bump = classify_bump(target_version, previous_version)
if not required_bump:
warn("Milestone assignment skipped (unable to determine required bump).")
return 0
token = args.token or os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
if not token:
warn("Milestone assignment skipped (missing GitHub token).")
return 0
repo = args.repo or os.environ.get("GITHUB_REPOSITORY")
if not repo or "/" not in repo:
warn("Milestone assignment skipped (missing repository info).")
return 0
owner, name = repo.split("/", 1)
milestones = fetch_open_milestones(owner, name, token)
if not milestones:
return 0
milestone_title = select_milestone(milestones, required_bump)
if milestone_title:
print(milestone_title)
return 0
if __name__ == "__main__":
sys.exit(main())
+48
View File
@@ -0,0 +1,48 @@
name: Deploy docs
on:
push:
branches:
- main
paths:
- "docs/**"
- "mkdocs.yml"
permissions:
contents: write # This allows pushing to gh-pages
jobs:
deploy_docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Determine docs-only push
id: docs-only
run: |
if [ "${{ github.event_name }}" != "push" ]; then
echo "skip=false" >> "$GITHUB_OUTPUT"
exit 0
fi
set -euo pipefail
before="${{ github.event.before }}"
sha="${{ github.sha }}"
changed_files=$(git diff --name-only "$before" "$sha" || true)
non_docs=$(echo "$changed_files" | grep -vE '^(docs/|mkdocs.yml$)' || true)
if [ -n "$non_docs" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Setup uv
if: steps.docs-only.outputs.skip != 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
- name: Install dependencies
if: steps.docs-only.outputs.skip != 'true'
run: make sync
- name: Deploy docs
if: steps.docs-only.outputs.skip != 'true'
run: make deploy-docs
+28
View File
@@ -0,0 +1,28 @@
name: Close inactive issues
on:
schedule:
- cron: "30 1 * * *"
jobs:
close-issues:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899
with:
days-before-issue-stale: 7
days-before-issue-close: 3
stale-issue-label: "stale"
exempt-issue-labels: "skip-stale"
stale-issue-message: "This issue is stale because it has been open for 7 days with no activity."
close-issue-message: "This issue was closed because it has been inactive for 3 days since being marked as stale."
any-of-issue-labels: 'question,needs-more-info'
days-before-pr-stale: 10
days-before-pr-close: 7
stale-pr-label: "stale"
exempt-pr-labels: "skip-stale"
stale-pr-message: "This PR is stale because it has been open for 10 days with no activity."
close-pr-message: "This PR was closed because it has been inactive for 7 days since being marked as stale."
repo-token: ${{ secrets.GITHUB_TOKEN }}
+35
View File
@@ -0,0 +1,35 @@
name: Publish to PyPI
on:
release:
types:
- published
permissions:
contents: read
jobs:
publish:
environment:
name: pypi
url: https://pypi.org/p/openai-agents
permissions:
id-token: write # Important for trusted publishing to PyPI
runs-on: ubuntu-latest
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Setup uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
- name: Install dependencies
run: make sync
- name: Build package
run: uv build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b
+143
View File
@@ -0,0 +1,143 @@
name: Create release PR
on:
workflow_dispatch:
inputs:
version:
description: "Version to release (e.g., 0.6.6)"
required: true
permissions:
contents: write
pull-requests: write
jobs:
release-pr:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
with:
fetch-depth: 0
ref: main
- name: Setup uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
- name: Fetch tags
run: git fetch origin --tags --prune
- name: Ensure release branch does not exist
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
branch="release/v${RELEASE_VERSION}"
if git ls-remote --exit-code --heads origin "$branch" >/dev/null 2>&1; then
echo "Branch $branch already exists on origin." >&2
exit 1
fi
- name: Update version
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
python - <<'PY'
import os
import pathlib
import re
import sys
version = os.environ["RELEASE_VERSION"]
if version.startswith("v"):
print("Version must not start with 'v' (use x.y.z...).", file=sys.stderr)
sys.exit(1)
if ".." in version:
print("Version contains consecutive dots (use x.y.z...).", file=sys.stderr)
sys.exit(1)
if not re.match(r"^\d+\.\d+(\.\d+)*([a-zA-Z0-9\.-]+)?$", version):
print(
"Version must be semver-like (e.g., 0.6.6, 0.6.6-rc1, 0.6.6.dev1).",
file=sys.stderr,
)
sys.exit(1)
path = pathlib.Path("pyproject.toml")
text = path.read_text()
updated, count = re.subn(
r'(?m)^version\s*=\s*"[^\"]+"',
f'version = "{version}"',
text,
)
if count != 1:
print("Expected to update exactly one version line.", file=sys.stderr)
sys.exit(1)
if updated == text:
print("Version already set; no changes made.", file=sys.stderr)
sys.exit(1)
path.write_text(updated)
PY
- name: Sync dependencies
run: make sync
- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Create release branch and commit
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
branch="release/v${RELEASE_VERSION}"
git checkout -b "$branch"
git add pyproject.toml uv.lock
if git diff --cached --quiet; then
echo "No changes to commit." >&2
exit 1
fi
git commit -m "Bump version to ${RELEASE_VERSION}"
git push --set-upstream origin "$branch"
- name: Build PR body
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
printf 'Release PR for %s.\n\nThe release readiness report will be prepared manually.\n' "$RELEASE_VERSION" > pr-body.md
- name: Create or update PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
head_branch="release/v${RELEASE_VERSION}"
milestone_name="$(python .github/scripts/select-release-milestone.py --version "$RELEASE_VERSION")"
pr_number="$(gh pr list --head "$head_branch" --base "main" --json number --jq '.[0].number // empty')"
if [ -z "$pr_number" ]; then
create_args=(
--title "Release ${RELEASE_VERSION}"
--body-file pr-body.md
--base "main"
--head "$head_branch"
--label "project"
)
if [ -n "$milestone_name" ]; then
create_args+=(--milestone "$milestone_name")
fi
if ! gh pr create "${create_args[@]}"; then
echo "PR create with label/milestone failed; retrying without them." >&2
gh pr create \
--title "Release ${RELEASE_VERSION}" \
--body-file pr-body.md \
--base "main" \
--head "$head_branch"
fi
else
edit_args=(
--title "Release ${RELEASE_VERSION}"
--body-file pr-body.md
--add-label "project"
)
if [ -n "$milestone_name" ]; then
edit_args+=(--milestone "$milestone_name")
fi
if ! gh pr edit "$pr_number" "${edit_args[@]}"; then
echo "PR edit with label/milestone failed; retrying without them." >&2
gh pr edit "$pr_number" --title "Release ${RELEASE_VERSION}" --body-file pr-body.md
fi
fi
+84
View File
@@ -0,0 +1,84 @@
name: Tag release on merge
on:
pull_request:
types:
- closed
branches:
- main
permissions:
contents: write
jobs:
tag-release:
if: >-
github.event.pull_request.merged == true &&
github.event.pull_request.head.repo.full_name == github.repository &&
startsWith(github.event.pull_request.head.ref, 'release/v')
runs-on: ubuntu-latest
steps:
- name: Validate merge commit
env:
MERGE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
run: |
if [ -z "$MERGE_SHA" ]; then
echo "merge_commit_sha is empty; refusing to tag to avoid tagging the wrong commit." >&2
exit 1
fi
- name: Checkout merge commit
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.merge_commit_sha }}
- name: Setup Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: "3.11"
- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Fetch tags
run: git fetch origin --tags --prune
- name: Resolve version
id: version
env:
HEAD_REF: ${{ github.event.pull_request.head.ref }}
run: |
python - <<'PY'
import os
import pathlib
import sys
import tomllib
path = pathlib.Path("pyproject.toml")
data = tomllib.loads(path.read_text())
version = data.get("project", {}).get("version")
if not version:
print("Missing project.version in pyproject.toml.", file=sys.stderr)
sys.exit(1)
head_ref = os.environ.get("HEAD_REF", "")
if head_ref.startswith("release/v"):
expected = head_ref[len("release/v") :]
if expected != version:
print(
f"Version mismatch: branch {expected} vs pyproject {version}.",
file=sys.stderr,
)
sys.exit(1)
output_path = pathlib.Path(os.environ["GITHUB_OUTPUT"])
output_path.write_text(f"version={version}\n")
PY
- name: Create tag
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
if git tag -l "v${VERSION}" | grep -q "v${VERSION}"; then
echo "Tag v${VERSION} already exists; skipping."
exit 0
fi
git tag -a "v${VERSION}" -m "Release v${VERSION}"
git push origin "v${VERSION}"
+162
View File
@@ -0,0 +1,162 @@
name: Tests
on:
push:
branches:
- main
pull_request:
# All PRs, including stacked PRs
permissions:
contents: read
env:
UV_FROZEN: "1"
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Verify formatting
if: steps.changes.outputs.run == 'true'
run: make format-check
- name: Run lint
if: steps.changes.outputs.run == 'true'
run: make lint
- name: Skip lint
if: steps.changes.outputs.run != 'true'
run: echo "Skipping lint for non-code changes."
typecheck:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Run typecheck
if: steps.changes.outputs.run == 'true'
run: make typecheck
- name: Skip typecheck
if: steps.changes.outputs.run != 'true'
run: echo "Skipping typecheck for non-code changes."
tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
python-version: ${{ matrix.python-version }}
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Run tests with coverage
if: steps.changes.outputs.run == 'true' && matrix.python-version == '3.12'
run: make coverage
- name: Run tests
if: steps.changes.outputs.run == 'true' && matrix.python-version != '3.12'
run: make tests
- name: Run async teardown stability tests
if: steps.changes.outputs.run == 'true' && (matrix.python-version == '3.10' || matrix.python-version == '3.14')
run: make tests-asyncio-stability
- name: Skip tests
if: steps.changes.outputs.run != 'true'
run: echo "Skipping tests for non-code changes."
tests-windows:
runs-on: windows-latest
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Detect code changes
id: changes
shell: bash
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
python-version: "3.13"
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: uv sync --all-extras --all-packages --group dev
- name: Run tests
if: steps.changes.outputs.run == 'true'
run: uv run pytest
- name: Skip tests
if: steps.changes.outputs.run != 'true'
run: echo "Skipping tests for non-code changes."
build-docs:
runs-on: ubuntu-latest
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
- name: Detect docs changes
id: changes
run: ./.github/scripts/detect-changes.sh docs "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # setup-uv v8.1.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Build docs
if: steps.changes.outputs.run == 'true'
run: make build-docs
- name: Skip docs build
if: steps.changes.outputs.run != 'true'
run: echo "Skipping docs build for non-docs changes."
+161
View File
@@ -0,0 +1,161 @@
# macOS Files
.DS_Store
# Byte-compiled / optimized / DLL files
__pycache__/
**/__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
.tmp/
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pdm
.pdm.toml
.pdm-python
.pdm-build/
# PEP 582
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.python-version
.env*
.venv
.venv*
env/
venv/
ENV/
env.bak/
venv.bak/
.venv39
.venv_res
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
.idea/
# Ruff stuff:
.ruff_cache/
# Example runtime state
examples/sandbox/extensions/daytona/usaspending_text2sql/.audit_log.jsonl
examples/sandbox/extensions/daytona/usaspending_text2sql/.session_state.json
# PyPI configuration file
.pypirc
.aider*
# Redis database files
dump.rdb
tmp/
# execplans
plans/
.vercel
+11
View File
@@ -0,0 +1,11 @@
{
"tabWidth": 4,
"overrides": [
{
"files": "*.yml",
"options": {
"tabWidth": 2
}
}
]
}
+14
View File
@@ -0,0 +1,14 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python Debugger: Python File",
"type": "debugpy",
"request": "launch",
"program": "${file}"
}
]
}
+7
View File
@@ -0,0 +1,7 @@
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
+244
View File
@@ -0,0 +1,244 @@
# Contributor Guide
This guide helps new contributors get started with the OpenAI Agents Python repository. It covers repo structure, how to test your work, available utilities, and guidelines for commits and PRs.
**Location:** `AGENTS.md` at the repository root.
## Table of Contents
1. [Policies & Mandatory Rules](#policies--mandatory-rules)
2. [Project Structure Guide](#project-structure-guide)
3. [Operation Guide](#operation-guide)
## Policies & Mandatory Rules
### Mandatory Skill Usage
#### `$code-change-verification`
Run `$code-change-verification` before marking work complete when changes affect runtime code, tests, or build/test behavior.
Run it when you change:
- `src/agents/` (library code) or shared utilities.
- `tests/` or add or modify snapshot tests.
- `examples/`.
- Build or test configuration such as `pyproject.toml`, `Makefile`, `mkdocs.yml`, `docs/scripts/`, or CI workflows.
You can skip `$code-change-verification` for docs-only or repo-meta changes (for example, `docs/`, `.agents/`, `README.md`, `AGENTS.md`, `.github/`), unless a user explicitly asks to run the full verification stack.
#### `$openai-knowledge`
When working on OpenAI API or OpenAI platform integrations in this repo (Responses API, tools, streaming, Realtime API, auth, models, rate limits, MCP, Agents SDK or ChatGPT Apps SDK), use `$openai-knowledge` to pull authoritative docs via the OpenAI Developer Docs MCP server (and guide setup if it is not configured).
#### `$implementation-strategy`
Before changing runtime code, exported APIs, external configuration, persisted schemas, wire protocols, or other user-facing behavior, use `$implementation-strategy` to decide the compatibility boundary and implementation shape. Judge breaking changes against the latest release tag, not unreleased branch-local churn. Interfaces introduced or changed after the latest release tag may be rewritten without compatibility shims unless they define a released or explicitly supported durable external state boundary, or the user explicitly asks for a migration path. Unreleased persisted formats on `main` may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported.
#### `$pr-draft-summary`
Before every final response for a task that changed runtime code, tests, examples, build/test configuration, or docs with behavior impact, invoke `$pr-draft-summary` to generate the required PR summary block, branch suggestion, title, and draft description. Determine whether to invoke it from the changed files, not from a subjective assessment of change size.
Skip `$pr-draft-summary` only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, or when the user explicitly says not to include the PR draft block.
Producing the PR draft block is part of the local final handoff. It is required for eligible local-only or uncommitted changes and does not authorize creating a branch, committing, pushing, or opening a pull request.
### Git Worktree and Branch Safety
Work in the user's current checkout and on the current branch by default. If the Codex task is already running in a selected Git worktree, use that worktree without requesting additional permission. Do not create or switch to another Git worktree, and do not create or switch branches, unless the user explicitly asks for or approves that exact action in the current conversation. A request to implement, investigate, review, test, or verify changes does not by itself authorize changing the active worktree or branch.
If isolation or a different checkout is needed, explain why and ask the user before changing Git state. This requirement also applies when another rule or workflow recommends a linked worktree: stop and request approval instead of choosing or creating one automatically.
### ExecPlans
Call out compatibility risk early in your plan only when the change affects behavior shipped in the latest release tag or a released or explicitly supported durable external state boundary, and confirm the approach before implementing changes that could impact users.
Use an ExecPlan when work is multi-step, spans several files, involves new features or refactors, or is likely to take more than about an hour. Start with the template and rules in `PLANS.md`, keep milestones and living sections (Progress, Surprises & Discoveries, Decision Log, Outcomes & Retrospective) up to date as you execute, and rewrite the plan if scope shifts. Call out compatibility risk only when the plan changes behavior shipped in the latest release tag or a released or explicitly supported durable external state boundary. Do not treat branch-local interface churn or unreleased post-tag changes on `main` as breaking by default; prefer direct replacement over compatibility layers in those cases, and renumber or squash unreleased persisted schemas before release when the intermediate snapshots are intentionally unsupported. If you intentionally skip an ExecPlan for a complex task, note why in your response so reviewers understand the choice.
### Public API Compatibility
Treat the parameter and dataclass field order of exported runtime APIs as a compatibility contract.
- For public constructors (for example `RunConfig`, `FunctionTool`, `AgentHookContext`), preserve existing positional argument meaning. Do not insert new constructor parameters or dataclass fields in the middle of existing public order.
- When adding a new optional public field/parameter, append it to the end whenever possible and keep old fields in the same order.
- If reordering is unavoidable, add an explicit compatibility layer and regression tests that exercise the old positional call pattern.
- Prefer keyword arguments at call sites to reduce accidental breakage, but do not rely on this to justify breaking positional compatibility for public APIs.
- Treat intended import paths and `__all__` membership as compatibility contracts. When adding or moving a public symbol, update the owning module, intended top-level or subpackage re-exports, and an import regression test. Keep top-level imports free of optional-dependency failures and runtime side effects; use lazy exports when needed.
### Platform, Docs, and Security Review
- Documentation is published to the live site, so coordinate SDK behavior changes and docs carefully. If docs describe behavior that is not released yet, either delay the docs change until the SDK release is available or split it into a follow-up PR.
- Treat runnable docs snippets as API compatibility checks. Before adding OpenAI API, provider, Responses, Realtime, WebSocket, or SDK constructor examples, verify the shown arguments and call shape against the actual implementation.
- Do not let untrusted sandbox manifests opt themselves out of host filesystem or base-directory boundaries. Escape hatches for local source materialization must be controlled by trusted application code at the call site, not by serialized manifest data.
- When documenting sandbox or security grants, verify the actual implementation path enforces the grant or boundary. Do not claim a grant applies to `LocalDir`, `LocalFile`, archive extraction, or other materialization paths unless those paths actually consult it.
- When redacting OpenAI tool, MCP, model, or provider payloads, consider traceback display, exception chaining, `__context__`, logs, and telemetry. Suppressing display with `raise ... from None` is not enough if the original exception object still carries sensitive input data.
- For OpenAI platform or SDK-specific docs changes, prefer `$openai-knowledge` for authoritative platform behavior and inspect the local code path for SDK behavior. Do not rely on generic API assumptions when documenting Responses, Chat Completions, Realtime, tools, MCP, or provider adapters.
- For Realtime tracing changes, read [Realtime tracing architecture](.agents/references/realtime-tracing.md) before proposing SDK spans. Realtime API server traces and Agents SDK client traces are separate; `group_id` can correlate them but does not create a shared trace hierarchy.
## Project Structure Guide
### Overview
The OpenAI Agents Python repository provides the Python Agents SDK, examples, and documentation built with MkDocs. Use `uv run python ...` for Python commands to ensure a consistent environment.
### Repo Structure & Important Files
- `src/agents/`: Core library implementation.
- `tests/`: Test suite; see `tests/README.md` for snapshot guidance.
- `examples/`: Sample projects showing SDK usage.
- `docs/`: MkDocs documentation source; do not edit translated docs under `docs/ja`, `docs/ko`, or `docs/zh` (they are generated).
- `docs/scripts/`: Documentation utilities, including translation and reference generation.
- `mkdocs.yml`: Documentation site configuration.
- `Makefile`: Common developer commands.
- `pyproject.toml`, `uv.lock`: Python dependencies and tool configuration.
- `.github/PULL_REQUEST_TEMPLATE/pull_request_template.md`: Pull request template to use when opening PRs.
- `.agents/references/`: Durable SDK maintainer architecture references. Start with [the reference map](.agents/references/README.md) and open only the files relevant to the affected runtime boundary.
- `site/`: Built documentation output.
### Agents Core Runtime Guidelines
- For `Agent` fields, cloning, dynamic instructions, enabled tools or handoffs, output schemas, run context wrappers, usage aggregation, or public-versus-internal agent identity, read [Agent definition and run context](.agents/references/agent-definition-and-run-context.md).
- `src/agents/run.py` is the runtime entrypoint (`Runner`, `AgentRunner`). Keep it focused on orchestration and public flow control. Put new runtime logic under `src/agents/run_internal/` and import it into `run.py`.
- When `run.py` grows, refactor helpers into `run_internal/` modules (for example `run_loop.py`, `turn_resolution.py`, `tool_execution.py`, `session_persistence.py`) and leave only wiring and composition in `run.py`.
- For turn accounting, guardrail ordering, handoffs, interruptions, cancellation, hooks, or streaming behavior, read [Runner lifecycle](.agents/references/runner-lifecycle.md). Keep streaming and non-streaming paths behaviorally aligned.
- For new model output, tool call, approval, or run item variants, read [Run item lifecycle](.agents/references/run-item-lifecycle.md) and update every applicable processing, event, replay, persistence, tracing, and serialization surface.
- For function-tool parameter schemas, `Annotated` or `Field` metadata, strict JSON schema conversion, or structured output schemas, read [Function and output schema](.agents/references/function-and-output-schema.md).
- For function-tool naming, namespacing, lookup, approvals, tracing, or call-ID changes, read [Tool identity and routing](.agents/references/tool-identity.md) and use the canonical helpers in `src/agents/_tool_identity.py` instead of adding local normalization rules.
- For function-tool planning, approval ordering, tool guardrails, concurrency, cancellation, timeouts, hooks, or failure conversion, read [Tool execution lifecycle](.agents/references/tool-execution-lifecycle.md).
- For local MCP connection ownership, `MCPServerManager`, request serialization, tool caching or filtering, transport retries, cancellation, or cleanup, read [Local MCP server lifecycle](.agents/references/local-mcp-server-lifecycle.md).
- For trace or span context, processors, export, flush, shutdown, sensitive data, or resumed trace state, read [Tracing lifecycle](.agents/references/tracing-lifecycle.md).
- For `RealtimeSession` lifecycle, background-task, handoff, listener, connection, or cleanup changes, read [Realtime session lifecycle](.agents/references/realtime-session-lifecycle.md) and verify both normal and failure-path resource ownership.
- For `VoicePipeline`, streamed audio input, STT session ownership, TTS task ordering, voice lifecycle events, PCM framing, or voice tracing changes, read [Voice pipeline lifecycle](.agents/references/voice-pipeline-lifecycle.md).
- For server-managed conversation (`conversation_id`, `previous_response_id`, `auto_previous_response_id`), read [Conversation state ownership](.agents/references/conversation-state-ownership.md) before changing continuation, filtering, retry, compaction, handoffs, or resume behavior.
- For client-managed session input, per-turn saves, retry rewind, backend atomicity, or compaction replacement, read [Session persistence](.agents/references/session-persistence.md).
- For model resolution, `ModelSettings`, provider adapters, Responses versus Chat Completions capabilities, request conversion, terminal events, transport reuse, or model retries, read [Model and provider boundaries](.agents/references/model-provider-boundaries.md).
- If the serialized `RunState` shape changes, read [RunState schema and resume boundary](.agents/references/runstate-schema.md) and follow its release-boundary, schema-version, backward-read, and regression-test rules.
- For sandbox session ownership, agent preparation, manifests, host-path materialization, snapshots, resume state, or cleanup, read [Sandbox runtime boundary](.agents/references/sandbox-runtime-boundary.md).
## Operation Guide
### Prerequisites
- Python 3.10+.
- `uv` installed for dependency management (`uv sync`) and `uv run` for Python commands.
- `make` available to run repository tasks.
### Development Workflow
1. Stay in the user's current checkout and on the current branch unless the user explicitly asks for or approves a Git state change.
2. If the user explicitly requests a feature/fix branch, create one with a descriptive name:
```bash
git checkout -b feat/<short-description>
```
3. If dependencies changed or you are setting up the repo, run `make sync`.
4. Implement changes and add or update tests alongside code updates.
5. Highlight compatibility or API risks in your plan before implementing changes that alter the latest released behavior or a released or explicitly supported durable external state boundary.
6. Build docs when you touch documentation:
```bash
make build-docs
```
7. When `$code-change-verification` applies, run it to execute the full verification stack before marking work complete.
8. Commit with concise, imperative messages; keep commits small and focused, then open a pull request.
9. Before reporting eligible code changes as complete, invoke `$pr-draft-summary` as the final handoff step unless the task falls under the documented skip cases. Do not omit it based on perceived change size or because the work remains local or uncommitted.
### Testing & Automated Checks
Before submitting changes, ensure relevant checks pass and extend tests when you touch code.
When `$code-change-verification` applies, run it to execute the required verification stack from the repository root. Rerun the full stack after applying fixes.
#### Unit tests and type checking
- Run the full test suite:
```bash
make tests
```
- Run a focused test:
```bash
uv run pytest -s -k <pattern>
```
- Type checking:
```bash
make typecheck
```
#### Snapshot tests
Some tests rely on inline snapshots; see `tests/README.md` for details. Re-run `make tests` after updating snapshots.
- Fix snapshots:
```bash
make snapshots-fix
```
- Create new snapshots:
```bash
make snapshots-create
```
#### Coverage
- Generate coverage (fails if coverage drops below threshold):
```bash
make coverage
```
#### Formatting, linting, and type checking
- Formatting and linting use `ruff`; run `make format` (applies fixes) and `make lint` (checks only).
- Type hints must pass `make typecheck`.
- Write comments as full sentences ending with a period.
- Imports are managed by Ruff and should stay sorted.
#### Mandatory local run order
When `$code-change-verification` applies, run the full sequence in order (or use the skill scripts):
```bash
make format
make lint
make typecheck
make tests
```
### Utilities & Tips
- Install or refresh development dependencies:
```bash
make sync
```
- Run tests against the oldest supported version (Python 3.10) in an isolated environment:
```bash
UV_PROJECT_ENVIRONMENT=.venv_310 uv sync --python 3.10 --all-extras --all-packages --group dev
UV_PROJECT_ENVIRONMENT=.venv_310 uv run --python 3.10 -m pytest
```
- Documentation workflows:
```bash
make build-docs # build docs after editing docs
make serve-docs # preview docs locally
make build-full-docs # run translations and build
```
- Snapshot helpers:
```bash
make snapshots-fix
make snapshots-create
```
- Use `examples/` to see common SDK usage patterns.
- Review `Makefile` for common commands and use `uv run` for Python invocations.
- Explore `docs/` and `docs/scripts/` to understand the documentation pipeline.
- Consult `tests/README.md` for test and snapshot workflows.
- Check `mkdocs.yml` to understand how docs are organized.
### Pull Request & Commit Guidelines
- Use the template at `.github/PULL_REQUEST_TEMPLATE/pull_request_template.md`; include a summary, test plan, and issue number if applicable.
- Add tests for new behavior when feasible and update documentation for user-facing changes.
- Run `make format`, `make lint`, `make typecheck`, and `make tests` before marking work ready.
- Commit messages should be concise and written in the imperative mood. Small, focused commits are preferred.
### Review Process & What Reviewers Look For
- ✅ Checks pass (`make format`, `make lint`, `make typecheck`, `make tests`).
- ✅ Tests cover new behavior and edge cases.
- ✅ Code is readable, maintainable, and consistent with existing style.
- ✅ Public APIs and user-facing behavior changes are documented.
- ✅ Examples are updated if behavior changes.
- ✅ History is clean with a clear PR description.
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 OpenAI
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+88
View File
@@ -0,0 +1,88 @@
.PHONY: sync
sync:
uv sync --all-extras --all-packages --group dev
.PHONY: format
format:
uv run ruff format
uv run ruff check --fix
.PHONY: format-check
format-check:
uv run ruff format --check
.PHONY: lint
lint:
uv run ruff check
.PHONY: mypy
mypy:
uv run mypy . --exclude site
.PHONY: pyright
pyright:
uv run pyright --project pyrightconfig.json
.PHONY: typecheck
typecheck:
@set -eu; \
mypy_pid=''; \
pyright_pid=''; \
trap 'test -n "$$mypy_pid" && kill $$mypy_pid 2>/dev/null || true; test -n "$$pyright_pid" && kill $$pyright_pid 2>/dev/null || true' EXIT INT TERM; \
echo "Running make mypy and make pyright in parallel..."; \
$(MAKE) mypy & mypy_pid=$$!; \
$(MAKE) pyright & pyright_pid=$$!; \
wait $$mypy_pid; \
wait $$pyright_pid; \
trap - EXIT
.PHONY: tests
tests: tests-parallel tests-serial
.PHONY: tests-asyncio-stability
tests-asyncio-stability:
bash .github/scripts/run-asyncio-teardown-stability.sh
.PHONY: tests-parallel
tests-parallel:
uv run pytest -n auto --dist loadfile -m "not serial"
.PHONY: tests-serial
tests-serial:
uv run pytest -m serial
.PHONY: coverage
coverage:
uv run coverage run -m pytest
uv run coverage xml -o coverage.xml
uv run coverage report -m --fail-under=85
.PHONY: snapshots-fix
snapshots-fix:
uv run pytest --inline-snapshot=fix
.PHONY: snapshots-create
snapshots-create:
uv run pytest --inline-snapshot=create
.PHONY: build-docs
build-docs:
uv run docs/scripts/generate_ref_files.py
uv run mkdocs build
.PHONY: build-full-docs
build-full-docs:
uv run docs/scripts/translate_docs.py
uv run mkdocs build
.PHONY: serve-docs
serve-docs:
uv run mkdocs serve
.PHONY: deploy-docs
deploy-docs:
uv run mkdocs gh-deploy --force --verbose
.PHONY: check
check: format-check lint typecheck tests
+100
View File
@@ -0,0 +1,100 @@
# Codex Execution Plans (ExecPlans)
This file defines how to write and maintain an ExecPlan: a self-contained, living specification that a novice can follow to deliver observable, working behavior in this repository.
## When to Use an ExecPlan
- Required for multi-step or multi-file work, new features, refactors, or tasks expected to take more than about an hour.
- Optional for trivial fixes (typos, small docs), but if you skip it for a substantial task, state the reason in your response.
## How to Use This File
- Authoring: read this file end to end before drafting; start from the skeleton; embed all context (paths, commands, definitions) so no external docs are needed.
- Implementing: move directly to the next milestone without asking for next steps; keep the living sections current at every stopping point.
- Discussing: record decisions and rationale inside the plan so work can be resumed later using only the ExecPlan.
## Non-Negotiable Requirements
- Self-contained and beginner-friendly: define every term; include needed repo knowledge; avoid assuming prior plans or external links.
- Living document: revise Progress, Surprises & Discoveries, Decision Log, and Outcomes & Retrospective as work proceeds while keeping the plan self-contained.
- Outcome-focused: describe what the user can do after the change and how to see it working; the plan must lead to demonstrably working behavior, not just code edits.
- Explicit acceptance: state behaviors, commands, and observable outputs that prove success.
## Formatting Rules
- Default envelope is a single fenced code block labeled `md`; do not nest other triple backticks inside—indent commands, transcripts, and diffs instead.
- If the file contains only the ExecPlan, omit the enclosing code fence.
- Use blank lines after headings; prefer prose over lists. Checklists are permitted only in the Progress section (and are mandatory there).
## Guidelines
- Define jargon immediately and tie it to concrete files or commands in this repo.
- Anchor on outcomes: acceptance should be phrased as observable behavior; for internal changes, show tests or scenarios that demonstrate the effect.
- Specify repository context explicitly: full paths, functions, modules, working directory for commands, and environment assumptions.
- Be idempotent and safe: describe retries or rollbacks for risky steps; prefer additive, testable changes.
- Validation is required: state exact test commands and expected outputs; include concise evidence (logs, transcripts, diffs) as indented examples.
## Milestones
- Tell a story (goal → work → result → proof) for each milestone; keep them narrative rather than bureaucratic.
- Each milestone must be independently verifiable and incrementally advance the overall goal.
- Milestones are distinct from Progress: milestones explain the plan; Progress tracks real-time execution.
## Living Sections (must be present and maintained)
- Progress: checkbox list with timestamps; every pause should update what is done and what remains.
- Surprises & Discoveries: unexpected behaviors, performance notes, or bugs with brief evidence.
- Decision Log: each decision with rationale and date/author.
- Outcomes & Retrospective: what was achieved, remaining gaps, and lessons learned.
## Prototyping and Parallel Paths
- Prototypes are encouraged to de-risk changes; keep them additive, clearly labeled, and validated.
- Parallel implementations are acceptable when reducing risk; describe how to validate each path and how to retire one safely.
## ExecPlan Skeleton
```md
# <Short, action-oriented description>
This ExecPlan is a living document. The sections Progress, Surprises & Discoveries, Decision Log, and Outcomes & Retrospective must stay up to date as work proceeds.
If PLANS.md is present in the repo, maintain this document in accordance with it and link back to it by path.
## Purpose / Big Picture
Explain the user-visible behavior gained after this change and how to observe it.
## Progress
- [x] (2025-10-01 13:00Z) Example completed step.
- [ ] Example incomplete step.
- [ ] Example partially completed step (completed: X; remaining: Y).
## Surprises & Discoveries
- Observation: …
Evidence: …
## Decision Log
- Decision: …
Rationale: …
Date/Author: …
## Outcomes & Retrospective
Summarize outcomes, gaps, and lessons learned; compare to the original purpose.
## Context and Orientation
Describe the current state relevant to this task as if the reader knows nothing. Name key files and modules by full path; define any non-obvious terms.
## Plan of Work
Prose description of the sequence of edits and additions. For each edit, name the file and location and what to change.
## Concrete Steps
Exact commands to run (with working directory). Include short expected outputs for comparison.
## Validation and Acceptance
Behavioral acceptance criteria plus test commands and expected results.
## Idempotence and Recovery
How to retry or roll back safely; ensure steps can be rerun without harm.
## Artifacts and Notes
Concise transcripts, diffs, or snippets as indented examples.
## Interfaces and Dependencies
Prescribe libraries, modules, and function signatures that must exist at the end. Use stable names and paths.
```
## Revising a Plan
- When the scope shifts, rewrite affected sections so the document remains coherent and self-contained.
- After significant edits, add a short note at the end explaining what changed and why.
+149
View File
@@ -0,0 +1,149 @@
# OpenAI Agents SDK [![PyPI](https://img.shields.io/pypi/v/openai-agents?label=pypi%20package)](https://pypi.org/project/openai-agents/)
The OpenAI Agents SDK is a lightweight yet powerful framework for building multi-agent workflows. It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.
<img src="https://cdn.openai.com/API/docs/images/orchestration.png" alt="Image of the Agents Tracing UI" style="max-height: 803px;">
> [!NOTE]
> Looking for the JavaScript/TypeScript version? Check out [Agents SDK JS/TS](https://github.com/openai/openai-agents-js).
### Core concepts:
1. [**Agents**](https://openai.github.io/openai-agents-python/agents): LLMs configured with instructions, tools, guardrails, and handoffs
1. [**Sandbox Agents**](https://openai.github.io/openai-agents-python/sandbox_agents): Agents preconfigured to work with a container to perform work over long time horizons.
1. **[Agents as tools](https://openai.github.io/openai-agents-python/tools/#agents-as-tools) / [Handoffs](https://openai.github.io/openai-agents-python/handoffs/)**: Delegating to other agents for specific tasks
1. [**Tools**](https://openai.github.io/openai-agents-python/tools/): Various Tools let agents take actions (functions, MCP, hosted tools)
1. [**Guardrails**](https://openai.github.io/openai-agents-python/guardrails/): Configurable safety checks for input and output validation
1. [**Human in the loop**](https://openai.github.io/openai-agents-python/human_in_the_loop/): Built-in mechanisms for involving humans across agent runs
1. [**Sessions**](https://openai.github.io/openai-agents-python/sessions/): Automatic conversation history management across agent runs
1. [**Tracing**](https://openai.github.io/openai-agents-python/tracing/): Built-in tracking of agent runs, allowing you to view, debug and optimize your workflows
1. [**Realtime Agents**](https://openai.github.io/openai-agents-python/realtime/quickstart/): Build powerful voice agents with `gpt-realtime-2.1` and full agent features
Explore the [examples](https://github.com/openai/openai-agents-python/tree/main/examples) directory to see the SDK in action, and read our [documentation](https://openai.github.io/openai-agents-python/) for more details.
## Get started
To get started, set up your Python environment (Python 3.10 or newer required), and then install OpenAI Agents SDK package.
### venv
```bash
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install openai-agents
```
For voice support, install with the optional `voice` group: `pip install 'openai-agents[voice]'`. For Redis session support, install with the optional `redis` group: `pip install 'openai-agents[redis]'`.
### uv
If you're familiar with [uv](https://docs.astral.sh/uv/), installing the package would be even easier:
```bash
uv init
uv add openai-agents
```
For voice support, install with the optional `voice` group: `uv add 'openai-agents[voice]'`. For Redis session support, install with the optional `redis` group: `uv add 'openai-agents[redis]'`.
## Run your first agents
The SDK supports three primary ways to run agents. Set the `OPENAI_API_KEY` environment variable before running any of these examples.
### Run a sandbox agent
Use a [`SandboxAgent`](https://openai.github.io/openai-agents-python/sandbox_agents) when the agent needs to inspect files, run commands, apply patches, or preserve workspace state across longer tasks.
```python
from agents import Runner
from agents.run import RunConfig
from agents.sandbox import Manifest, SandboxAgent, SandboxRunConfig
from agents.sandbox.entries import GitRepo
from agents.sandbox.sandboxes import UnixLocalSandboxClient
agent = SandboxAgent(
name="Workspace Assistant",
instructions="Inspect the sandbox workspace before answering.",
default_manifest=Manifest(entries={"repo": GitRepo(repo="openai/openai-agents-python", ref="main")}),
)
result = Runner.run_sync(
agent,
"Inspect the repo README and summarize what this project does.",
run_config=RunConfig(sandbox=SandboxRunConfig(client=UnixLocalSandboxClient())),
)
print(result.final_output)
```
### Run a text agent
Use a text `Agent` for workflows that do not need a persistent realtime connection or a sandbox workspace.
```python
from agents import Agent, Runner
agent = Agent(name="Assistant", instructions="You are a helpful assistant")
result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
print(result.final_output)
# Code within the code,
# Functions calling themselves,
# Infinite loop's dance.
```
(_For Jupyter notebook users, see [hello_world_jupyter.ipynb](https://github.com/openai/openai-agents-python/blob/main/examples/basic/hello_world_jupyter.ipynb)_)
### Run a realtime agent
Use a [`RealtimeAgent`](https://openai.github.io/openai-agents-python/realtime/quickstart/) for low-latency, server-side voice and multimodal experiences over WebSocket.
```python
import asyncio
from agents.realtime import RealtimeAgent, RealtimeRunner
async def main() -> None:
agent = RealtimeAgent(name="Assistant", instructions="You are a helpful voice assistant. Keep responses short.")
runner = RealtimeRunner(starting_agent=agent)
session = await runner.run()
async with session:
await session.send_message("Say hello in one short sentence.")
async for event in session:
if event.type == "audio":
# Forward or play event.audio.data.
pass
elif event.type == "history_added":
print(event.item)
elif event.type == "agent_end":
break
if __name__ == "__main__":
asyncio.run(main())
```
Explore the [examples](https://github.com/openai/openai-agents-python/tree/main/examples) directory to see the SDK in action, and read our [documentation](https://openai.github.io/openai-agents-python/) for more details.
## Acknowledgements
We'd like to acknowledge the excellent work of the open-source community, especially:
- [Pydantic](https://docs.pydantic.dev/latest/)
- [Requests](https://github.com/psf/requests)
- [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)
- [Griffe](https://github.com/mkdocstrings/griffe)
This library has these optional dependencies:
- [websockets](https://github.com/python-websockets/websockets)
- [SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy)
- [any-llm](https://github.com/mozilla-ai/any-llm) and [LiteLLM](https://github.com/BerriAI/litellm)
We also rely on the following tools to manage the project:
- [uv](https://github.com/astral-sh/uv) and [ruff](https://github.com/astral-sh/ruff)
- [mypy](https://github.com/python/mypy) and [Pyright](https://github.com/microsoft/pyright)
- [pytest](https://github.com/pytest-dev/pytest) and [Coverage.py](https://github.com/coveragepy/coveragepy)
- [MkDocs](https://github.com/squidfunk/mkdocs-material)
We're committed to continuing to build the Agents SDK as an open source framework so others in the community can expand on our approach.
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目:`openai/openai-agents-python`
- 原始仓库:https://github.com/openai/openai-agents-python
- 导入方式:上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源,不代表 WeHub 是原项目作者
+5
View File
@@ -0,0 +1,5 @@
# Security Policy
For a more in-depth look at our security policy, please check out our [Coordinated Vulnerability Disclosure Policy](https://openai.com/security/disclosure/#:~:text=Disclosure%20Policy,-Security%20is%20essential&text=OpenAI%27s%20coordinated%20vulnerability%20disclosure%20policy,expect%20from%20us%20in%20return.).
Our PGP key can located [at this address.](https://cdn.openai.com/security.txt)
+424
View File
@@ -0,0 +1,424 @@
# Agents
Agents are the core building block in your apps. An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, and structured outputs.
Use this page when you want to define or customize a single plain `Agent`. If you are deciding how multiple agents should collaborate, read [Agent orchestration](multi_agent.md). If the agent should run inside an isolated workspace with manifest-defined files and sandbox-native capabilities, read [Sandbox agent concepts](sandbox/guide.md).
The SDK uses the Responses API by default for OpenAI models, but the distinction here is orchestration: `Agent` plus `Runner` lets the SDK manage turns, tools, guardrails, handoffs, and sessions for you. If you want to own that loop yourself, use the Responses API directly instead.
## Choose the next guide
Use this page as the hub for agent definition. Jump to the adjacent guide that matches the next decision you need to make.
| If you want to... | Read next |
| --- | --- |
| Choose a model or provider setup | [Models](models/index.md) |
| Add capabilities to the agent | [Tools](tools.md) |
| Run an agent against a real repo, document bundle, or isolated workspace | [Sandbox agents quickstart](sandbox_agents.md) |
| Decide between manager-style orchestration and handoffs | [Agent orchestration](multi_agent.md) |
| Configure handoff behavior | [Handoffs](handoffs.md) |
| Run turns, stream events, or manage conversation state | [Running agents](running_agents.md) |
| Inspect final output, run items, or resumable state | [Results](results.md) |
| Share local dependencies and runtime state | [Context management](context.md) |
## Basic configuration
The most common properties of an agent are:
| Property | Required | Description |
| --- | --- | --- |
| `name` | yes | Human-readable agent name. |
| `instructions` | no | System prompt or dynamic instructions callback. Strongly recommended. See [Dynamic instructions](#dynamic-instructions). |
| `prompt` | no | OpenAI Responses API prompt configuration. Accepts a static prompt object or a function. See [Prompt templates](#prompt-templates). |
| `handoff_description` | no | Short description exposed when this agent is offered as a handoff target. |
| `handoffs` | no | Delegate the conversation to specialist agents. See [handoffs](handoffs.md). |
| `model` | no | Which LLM to use. See [Models](models/index.md). |
| `model_settings` | no | Model tuning parameters such as `temperature`, `top_p`, and `tool_choice`. |
| `tools` | no | Tools the agent can call. See [Tools](tools.md). |
| `mcp_servers` | no | MCP-backed tools for the agent. See the [MCP guide](mcp.md). |
| `mcp_config` | no | Fine-tune how MCP tools are prepared, such as strict schema conversion and MCP failure formatting. See the [MCP guide](mcp.md#agent-level-mcp-configuration). |
| `input_guardrails` | no | Guardrails that run on the first user input for this agent chain. See [Guardrails](guardrails.md). |
| `output_guardrails` | no | Guardrails that run on the final output for this agent. See [Guardrails](guardrails.md). |
| `output_type` | no | Structured output type instead of plain text. See [Output types](#output-types). |
| `hooks` | no | Agent-scoped lifecycle callbacks. See [Lifecycle events (hooks)](#lifecycle-events-hooks). |
| `tool_use_behavior` | no | Control whether tool results loop back to the model or end the run. See [Tool use behavior](#tool-use-behavior). |
| `reset_tool_choice` | no | Reset `tool_choice` after a tool call (default: `True`) to avoid tool-use loops. See [Forcing tool use](#forcing-tool-use). |
```python
from agents import Agent, ModelSettings, function_tool
@function_tool
def get_weather(city: str) -> str:
"""returns weather info for the specified city."""
return f"The weather in {city} is sunny"
agent = Agent(
name="Haiku agent",
instructions="Always respond in haiku form",
model="gpt-5-nano",
tools=[get_weather],
)
```
Everything in this section applies to `Agent`. `SandboxAgent` builds on the same ideas, then adds `default_manifest`, `base_instructions`, `capabilities`, and `run_as` for workspace-scoped runs. See [Sandbox agent concepts](sandbox/guide.md).
## Prompt templates
You can reference a prompt template created in the OpenAI platform by setting `prompt`. This works with OpenAI models using the Responses API.
To use it, please:
1. Go to https://platform.openai.com/playground/prompts
2. Create a new prompt variable, `poem_style`.
3. Create a system prompt with the content:
```
Write a poem in {{poem_style}}
```
4. Run the example with the `--prompt-id` flag.
```python
from agents import Agent
agent = Agent(
name="Prompted assistant",
prompt={
"id": "pmpt_123",
"version": "1",
"variables": {"poem_style": "haiku"},
},
)
```
You can also generate the prompt dynamically at run time:
```python
from dataclasses import dataclass
from agents import Agent, GenerateDynamicPromptData, Runner
@dataclass
class PromptContext:
prompt_id: str
poem_style: str
async def build_prompt(data: GenerateDynamicPromptData):
ctx: PromptContext = data.context.context
return {
"id": ctx.prompt_id,
"version": "1",
"variables": {"poem_style": ctx.poem_style},
}
agent = Agent(name="Prompted assistant", prompt=build_prompt)
result = await Runner.run(
agent,
"Say hello",
context=PromptContext(prompt_id="pmpt_123", poem_style="limerick"),
)
```
## Context
Agents are generic on their `context` type. Context is a dependency-injection tool: it's an object you create and pass to `Runner.run()`, that is passed to every agent, tool, handoff etc, and it serves as a grab bag of dependencies and state for the agent run. You can provide any Python object as the context.
Read the [context guide](context.md) for the full `RunContextWrapper` surface, shared usage tracking, nested `tool_input`, and serialization caveats.
```python
@dataclass
class UserContext:
name: str
uid: str
is_pro_user: bool
async def fetch_purchases() -> list[Purchase]:
return ...
agent = Agent[UserContext](
...,
)
```
## Output types
By default, agents produce plain text (i.e. `str`) outputs. If you want the agent to produce a particular type of output, you can use the `output_type` parameter. A common choice is to use [Pydantic](https://docs.pydantic.dev/) objects, but we support any type that can be wrapped in a Pydantic [TypeAdapter](https://docs.pydantic.dev/latest/api/type_adapter/) - dataclasses, lists, TypedDict, etc.
```python
from pydantic import BaseModel
from agents import Agent
class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
agent = Agent(
name="Calendar extractor",
instructions="Extract calendar events from text",
output_type=CalendarEvent,
)
```
!!! note
When you pass an `output_type`, that tells the model to use [structured outputs](https://platform.openai.com/docs/guides/structured-outputs) instead of regular plain text responses.
## Multi-agent system design patterns
There are many ways to design multiagent systems, but we commonly see two broadly applicable patterns:
1. Manager (agents as tools): A central manager/orchestrator invokes specialized subagents as tools and retains control of the conversation.
2. Handoffs: Peer agents hand off control to a specialized agent that takes over the conversation. This is decentralized.
See [our practical guide to building agents](https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf) for more details.
### Manager (agents as tools)
The `customer_facing_agent` handles all user interaction and invokes specialized subagents exposed as tools. Read more in the [tools](tools.md#agents-as-tools) documentation.
```python
from agents import Agent
booking_agent = Agent(...)
refund_agent = Agent(...)
customer_facing_agent = Agent(
name="Customer-facing agent",
instructions=(
"Handle all direct user communication. "
"Call the relevant tools when specialized expertise is needed."
),
tools=[
booking_agent.as_tool(
tool_name="booking_expert",
tool_description="Handles booking questions and requests.",
),
refund_agent.as_tool(
tool_name="refund_expert",
tool_description="Handles refund questions and requests.",
)
],
)
```
### Handoffs
Handoffs are subagents the agent can delegate to. When a handoff occurs, the delegated agent receives the conversation history and takes over the conversation. This pattern enables modular, specialized agents that excel at a single task. Read more in the [handoffs](handoffs.md) documentation.
```python
from agents import Agent
booking_agent = Agent(...)
refund_agent = Agent(...)
triage_agent = Agent(
name="Triage agent",
instructions=(
"Help the user with their questions. "
"If they ask about booking, hand off to the booking agent. "
"If they ask about refunds, hand off to the refund agent."
),
handoffs=[booking_agent, refund_agent],
)
```
## Dynamic instructions
In most cases, you can provide instructions when you create the agent. However, you can also provide dynamic instructions via a function. The function will receive the agent and context, and must return the prompt. Both regular and `async` functions are accepted.
```python
def dynamic_instructions(
context: RunContextWrapper[UserContext], agent: Agent[UserContext]
) -> str:
return f"The user's name is {context.context.name}. Help them with their questions."
agent = Agent[UserContext](
name="Triage agent",
instructions=dynamic_instructions,
)
```
## Lifecycle events (hooks)
Sometimes, you want to observe the lifecycle of an agent. For example, you may want to log events, pre-fetch data, or record usage when certain events occur.
There are two hook scopes:
- [`RunHooks`][agents.lifecycle.RunHooks] observe the entire `Runner.run(...)` invocation, including handoffs to other agents.
- [`AgentHooks`][agents.lifecycle.AgentHooks] are attached to a specific agent instance via `agent.hooks`.
The callback context also changes depending on the event:
- Agent start/end hooks receive [`AgentHookContext`][agents.run_context.AgentHookContext], which wraps your original context and carries the shared run usage state.
- LLM, tool, and handoff hooks receive [`RunContextWrapper`][agents.run_context.RunContextWrapper].
Typical hook timing:
- `on_agent_start` / `on_agent_end`: when a specific agent begins or finishes producing a final output.
- `on_llm_start` / `on_llm_end`: immediately around each model call.
- `on_tool_start` / `on_tool_end`: around each local tool invocation. For function tools, the hook `context` is typically a `ToolContext`, so you can inspect tool-call metadata such as `tool_call_id`.
- `on_handoff`: when control moves from one agent to another.
Use `RunHooks` when you want a single observer for the whole workflow, and `AgentHooks` when one agent needs custom side effects.
```python
from agents import Agent, RunHooks, Runner
class LoggingHooks(RunHooks):
async def on_agent_start(self, context, agent):
print(f"Starting {agent.name}")
async def on_llm_end(self, context, agent, response):
print(f"{agent.name} produced {len(response.output)} output items")
async def on_agent_end(self, context, agent, output):
print(f"{agent.name} finished with usage: {context.usage}")
agent = Agent(name="Assistant", instructions="Be concise.")
result = await Runner.run(agent, "Explain quines", hooks=LoggingHooks())
print(result.final_output)
```
For the full callback surface, see the [Lifecycle API reference](ref/lifecycle.md).
## Guardrails
Guardrails allow you to run checks/validations on user input in parallel to the agent running, and on the agent's output once it is produced. For example, you could screen the user's input and agent's output for relevance. Read more in the [guardrails](guardrails.md) documentation.
## Cloning/copying agents
By using the `clone()` method on an agent, you can duplicate an Agent, and optionally change any properties you like.
```python
pirate_agent = Agent(
name="Pirate",
instructions="Write like a pirate",
model="gpt-5.6-sol",
)
robot_agent = pirate_agent.clone(
name="Robot",
instructions="Write like a robot",
)
```
## Forcing tool use
Supplying a list of tools doesn't always mean the LLM will use a tool. You can force tool use by setting [`ModelSettings.tool_choice`][agents.model_settings.ModelSettings.tool_choice]. Valid values are:
1. `auto`, which allows the LLM to decide whether or not to use a tool.
2. `required`, which requires the LLM to use a tool (but it can intelligently decide which tool).
3. `none`, which requires the LLM to _not_ use a tool.
4. Setting a specific string e.g. `my_tool`, which requires the LLM to use that specific tool.
When you are using OpenAI Responses tool search, named tool choices are more limited: you cannot target bare namespace names or deferred-only tools with `tool_choice`, and `tool_choice="tool_search"` does not target [`ToolSearchTool`][agents.tool.ToolSearchTool]. In those cases, prefer `auto` or `required`. See [Hosted tool search](tools.md#hosted-tool-search) for the Responses-specific constraints.
```python
from agents import Agent, Runner, function_tool, ModelSettings
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
agent = Agent(
name="Weather Agent",
instructions="Retrieve weather details.",
tools=[get_weather],
model_settings=ModelSettings(tool_choice="get_weather")
)
```
## Tool use behavior
The `tool_use_behavior` parameter in the `Agent` configuration controls how tool outputs are handled:
- `"run_llm_again"`: The default. Tools are run, and the LLM processes the results to produce a final response.
- `"stop_on_first_tool"`: The output of the first tool call is used as the final response, without further LLM processing.
```python
from agents import Agent, Runner, function_tool, ModelSettings
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
agent = Agent(
name="Weather Agent",
instructions="Retrieve weather details.",
tools=[get_weather],
tool_use_behavior="stop_on_first_tool"
)
```
- `StopAtTools(stop_at_tool_names=[...])`: Stops if any specified tool is called, using its output as the final response.
```python
from agents import Agent, Runner, function_tool
from agents.agent import StopAtTools
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
@function_tool
def sum_numbers(a: int, b: int) -> int:
"""Adds two numbers."""
return a + b
agent = Agent(
name="Stop At Stock Agent",
instructions="Get weather or sum numbers.",
tools=[get_weather, sum_numbers],
tool_use_behavior=StopAtTools(stop_at_tool_names=["get_weather"])
)
```
- `ToolsToFinalOutputFunction`: A custom function that processes tool results and decides whether to stop or continue with the LLM.
```python
from agents import Agent, Runner, function_tool, FunctionToolResult, RunContextWrapper
from agents.agent import ToolsToFinalOutputResult
from typing import List, Any
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
def custom_tool_handler(
context: RunContextWrapper[Any],
tool_results: List[FunctionToolResult]
) -> ToolsToFinalOutputResult:
"""Processes tool results to decide final output."""
for result in tool_results:
if result.output and "sunny" in result.output:
return ToolsToFinalOutputResult(
is_final_output=True,
final_output=f"Final weather: {result.output}"
)
return ToolsToFinalOutputResult(
is_final_output=False,
final_output=None
)
agent = Agent(
name="Weather Agent",
instructions="Retrieve weather details.",
tools=[get_weather],
tool_use_behavior=custom_tool_handler
)
```
!!! note
To prevent infinite loops, the framework automatically resets `tool_choice` to "auto" after a tool call. This behavior is configurable via [`agent.reset_tool_choice`][agents.agent.Agent.reset_tool_choice]. The infinite loop is because tool results are sent to the LLM, which then generates another tool call because of `tool_choice`, ad infinitum.
+16
View File
@@ -0,0 +1,16 @@
<svg width="512" height="512" viewBox="0 0 512 512" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_1497_2713)">
<rect width="512" height="512" rx="256" fill="#0000FF"/>
<g clip-path="url(#clip1_1497_2713)">
<path d="M215.923 209.432V177.018C215.923 174.288 216.947 172.24 219.334 170.876L284.506 133.344C293.378 128.227 303.955 125.839 314.872 125.839C355.816 125.839 381.75 157.572 381.75 191.35C381.75 193.737 381.75 196.467 381.407 199.197L313.848 159.617C309.755 157.229 305.658 157.229 301.564 159.617L215.923 209.432ZM368.099 335.679V258.224C368.099 253.446 366.051 250.034 361.958 247.646L276.316 197.831L304.294 181.793C306.682 180.43 308.73 180.43 311.118 181.793L376.289 219.325C395.057 230.245 407.68 253.446 407.68 275.964C407.68 301.894 392.327 325.78 368.099 335.676V335.679ZM195.792 267.438L167.813 251.061C165.425 249.698 164.401 247.649 164.401 244.919V169.855C164.401 133.347 192.38 105.708 230.254 105.708C244.586 105.708 257.891 110.486 269.153 119.016L201.937 157.914C197.843 160.302 195.795 163.714 195.795 168.492V267.441L195.792 267.438ZM256.015 302.24L215.923 279.722V231.954L256.015 209.436L296.104 231.954V279.722L256.015 302.24ZM281.776 405.968C267.444 405.968 254.14 401.19 242.877 392.66L310.094 353.762C314.187 351.374 316.235 347.962 316.235 343.184V244.235L344.557 260.611C346.944 261.975 347.968 264.023 347.968 266.753V341.817C347.968 378.325 319.647 405.965 281.776 405.965V405.968ZM200.909 329.88L135.738 292.348C116.97 281.427 104.347 258.227 104.347 235.709C104.347 209.436 120.042 185.893 144.267 175.997V253.791C144.267 258.57 146.315 261.981 150.409 264.369L235.711 313.842L207.733 329.88C205.345 331.243 203.297 331.243 200.909 329.88ZM197.158 385.837C158.602 385.837 130.281 356.834 130.281 321.008C130.281 318.278 130.623 315.548 130.963 312.818L198.179 351.717C202.273 354.104 206.369 354.104 210.463 351.717L296.104 302.243V334.658C296.104 337.388 295.08 339.436 292.693 340.8L227.521 378.332C218.649 383.449 208.072 385.837 197.155 385.837H197.158ZM281.776 426.438C323.062 426.438 357.522 397.096 365.373 358.197C403.586 348.302 428.153 312.475 428.153 275.967C428.153 252.082 417.918 228.882 399.493 212.162C401.199 204.997 402.223 197.831 402.223 190.668C402.223 141.877 362.643 105.365 316.92 105.365C307.709 105.365 298.838 106.729 289.966 109.801C274.61 94.7878 253.455 85.2344 230.254 85.2344C188.968 85.2344 154.509 114.576 146.658 153.475C108.444 163.371 83.877 199.197 83.877 235.705C83.877 259.59 94.1121 282.791 112.537 299.51C110.831 306.676 109.807 313.842 109.807 321.005C109.807 369.796 149.388 406.307 195.11 406.307C204.321 406.307 213.193 404.944 222.064 401.871C237.417 416.885 258.572 426.438 281.776 426.438Z" fill="white"/>
</g>
</g>
<defs>
<clipPath id="clip0_1497_2713">
<rect width="512" height="512" rx="256" fill="white"/>
</clipPath>
<clipPath id="clip1_1497_2713">
<rect width="344.276" height="341.204" fill="white" transform="translate(83.877 85.2344)"/>
</clipPath>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 398 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 423 KiB

+15
View File
@@ -0,0 +1,15 @@
<svg width="721" height="721" viewBox="0 0 721 721" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_1637_2935)">
<g clip-path="url(#clip1_1637_2935)">
<path d="M304.246 295.411V249.828C304.246 245.989 305.687 243.109 309.044 241.191L400.692 188.412C413.167 181.215 428.042 177.858 443.394 177.858C500.971 177.858 537.44 222.482 537.44 269.982C537.44 273.34 537.44 277.179 536.959 281.018L441.954 225.358C436.197 222 430.437 222 424.68 225.358L304.246 295.411ZM518.245 472.945V364.024C518.245 357.304 515.364 352.507 509.608 349.149L389.174 279.096L428.519 256.543C431.877 254.626 434.757 254.626 438.115 256.543L529.762 309.323C556.154 324.679 573.905 357.304 573.905 388.971C573.905 425.436 552.315 459.024 518.245 472.941V472.945ZM275.937 376.982L236.592 353.952C233.235 352.034 231.794 349.154 231.794 345.315V239.756C231.794 188.416 271.139 149.548 324.4 149.548C344.555 149.548 363.264 156.268 379.102 168.262L284.578 222.964C278.822 226.321 275.942 231.119 275.942 237.838V376.986L275.937 376.982ZM360.626 425.922L304.246 394.255V327.083L360.626 295.416L417.002 327.083V394.255L360.626 425.922ZM396.852 571.789C376.698 571.789 357.989 565.07 342.151 553.075L436.674 498.374C442.431 495.017 445.311 490.219 445.311 483.499V344.352L485.138 367.382C488.495 369.299 489.936 372.179 489.936 376.018V481.577C489.936 532.917 450.109 571.785 396.852 571.785V571.789ZM283.134 464.79L191.486 412.01C165.094 396.654 147.343 364.029 147.343 332.362C147.343 295.416 169.415 262.309 203.48 248.393V357.791C203.48 364.51 206.361 369.308 212.117 372.665L332.074 442.237L292.729 464.79C289.372 466.707 286.491 466.707 283.134 464.79ZM277.859 543.48C223.639 543.48 183.813 502.695 183.813 452.314C183.813 448.475 184.294 444.636 184.771 440.797L279.295 495.498C285.051 498.856 290.812 498.856 296.568 495.498L417.002 425.927V471.509C417.002 475.349 415.562 478.229 412.204 480.146L320.557 532.926C308.081 540.122 293.206 543.48 277.854 543.48H277.859ZM396.852 600.576C454.911 600.576 503.37 559.313 514.41 504.612C568.149 490.696 602.696 440.315 602.696 388.976C602.696 355.387 588.303 322.762 562.392 299.25C564.791 289.173 566.231 279.096 566.231 269.024C566.231 200.411 510.571 149.067 446.274 149.067C433.322 149.067 420.846 150.984 408.37 155.305C386.775 134.192 357.026 120.758 324.4 120.758C266.342 120.758 217.883 162.02 206.843 216.721C153.104 230.637 118.557 281.018 118.557 332.357C118.557 365.946 132.95 398.571 158.861 422.083C156.462 432.16 155.022 442.237 155.022 452.309C155.022 520.922 210.682 572.266 274.978 572.266C287.931 572.266 300.407 570.349 312.883 566.028C334.473 587.141 364.222 600.576 396.852 600.576Z" fill="white"/>
</g>
</g>
<defs>
<clipPath id="clip0_1637_2935">
<rect width="720" height="720" fill="white" transform="translate(0.606934 0.899902)"/>
</clipPath>
<clipPath id="clip1_1637_2935">
<rect width="484.139" height="479.818" fill="white" transform="translate(118.557 120.758)"/>
</clipPath>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 2.9 KiB

+201
View File
@@ -0,0 +1,201 @@
# Configuration
This page covers SDK-wide defaults that you usually set once during application startup, such as the default OpenAI key or client, the default OpenAI API shape, tracing export defaults, and logging behavior.
These defaults still apply to sandbox-based workflows, but sandbox workspaces, sandbox clients, and session reuse are configured separately.
If you need to configure a specific agent or run instead, start with:
- [Agents](agents.md) for instructions, tools, output types, handoffs, and guardrails on a plain `Agent`.
- [Running agents](running_agents.md) for `RunConfig`, sessions, and conversation-state options.
- [Sandbox agents](sandbox/guide.md) for `SandboxRunConfig`, manifests, capabilities, and sandbox-client-specific workspace setup.
- [Models](models/index.md) for model selection and provider configuration.
- [Tracing](tracing.md) for per-run tracing metadata and custom trace processors.
## API keys and clients
By default, the SDK uses the `OPENAI_API_KEY` environment variable for LLM requests and tracing. The key is resolved when the SDK first creates an OpenAI client (lazy initialization), so set the environment variable before your first model call. If you are unable to set that environment variable before your app starts, you can use the [set_default_openai_key()][agents.set_default_openai_key] function to set the key.
```python
from agents import set_default_openai_key
set_default_openai_key("sk-...")
```
Alternatively, you can also configure an OpenAI client to be used. By default, the SDK creates an `AsyncOpenAI` instance, using the API key from the environment variable or the default key set above. You can change this by using the [set_default_openai_client()][agents.set_default_openai_client] function.
```python
from openai import AsyncOpenAI
from agents import set_default_openai_client
custom_client = AsyncOpenAI(base_url="...", api_key="...")
set_default_openai_client(custom_client)
```
If you prefer environment-based endpoint configuration, the default OpenAI provider also reads `OPENAI_BASE_URL`. When you enable Responses websocket transport, it also reads `OPENAI_WEBSOCKET_BASE_URL` for the websocket `/responses` endpoint.
```bash
export OPENAI_BASE_URL="https://your-openai-compatible-endpoint.example/v1"
export OPENAI_WEBSOCKET_BASE_URL="wss://your-openai-compatible-endpoint.example/v1"
```
Finally, you can also customize the OpenAI API that is used. By default, we use the OpenAI Responses API. You can override this to use the Chat Completions API by using the [set_default_openai_api()][agents.set_default_openai_api] function.
```python
from agents import set_default_openai_api
set_default_openai_api("chat_completions")
```
## OpenAI provider defaults
OpenAI-backed providers also read SDK-wide defaults when they resolve model names. Use [`set_default_openai_responses_transport()`][agents.set_default_openai_responses_transport] to make OpenAI Responses models use websocket transport by default:
```python
from agents import set_default_openai_responses_transport
set_default_openai_responses_transport("websocket")
```
This affects OpenAI Responses models resolved by the default OpenAI provider. For provider-level setup, connection reuse, keepalive options, and custom websocket endpoints, see [Responses WebSocket transport](models/index.md#responses-websocket-transport).
If your OpenAI setup expects provider-level agent registration metadata, configure a default harness ID once at startup:
```python
from agents import set_default_openai_harness
set_default_openai_harness("your-harness-id")
```
You can also pass the full registration object:
```python
from agents import OpenAIAgentRegistrationConfig, set_default_openai_agent_registration
set_default_openai_agent_registration(
OpenAIAgentRegistrationConfig(harness_id="your-harness-id")
)
```
If no SDK default is set, OpenAI-backed providers fall back to the `OPENAI_AGENT_HARNESS_ID` environment variable. When a harness ID is configured, the SDK adds it to trace metadata as `agent_harness_id` unless that key is already present in `RunConfig.trace_metadata`.
## Tracing
Tracing is enabled by default. By default it uses the same OpenAI API key as your model requests from the section above (that is, the environment variable or the default key you set). You can specifically set the API key used for tracing by using the [`set_tracing_export_api_key`][agents.set_tracing_export_api_key] function.
```python
from agents import set_tracing_export_api_key
set_tracing_export_api_key("sk-...")
```
If your model traffic uses one key or client but tracing should use a different OpenAI key, pass `use_for_tracing=False` when setting the default key or client, then configure tracing separately. The same pattern works with [`set_default_openai_key()`][agents.set_default_openai_key] if you are not using a custom client.
```python
from openai import AsyncOpenAI
from agents import (
set_default_openai_client,
set_tracing_export_api_key,
)
custom_client = AsyncOpenAI(base_url="https://your-openai-compatible-endpoint.example/v1", api_key="provider-key")
set_default_openai_client(custom_client, use_for_tracing=False)
set_tracing_export_api_key("sk-tracing")
```
If you need to attribute traces to a specific organization or project when using the default exporter, set these environment variables before your app starts:
```bash
export OPENAI_ORG_ID="org_..."
export OPENAI_PROJECT_ID="proj_..."
```
You can also set a tracing API key per run without changing the global exporter.
```python
from agents import Runner, RunConfig
await Runner.run(
agent,
input="Hello",
run_config=RunConfig(tracing={"api_key": "sk-tracing-123"}),
)
```
You can also disable tracing entirely by using the [`set_tracing_disabled()`][agents.set_tracing_disabled] function.
```python
from agents import set_tracing_disabled
set_tracing_disabled(True)
```
If you want to keep tracing enabled but exclude potentially sensitive inputs/outputs from trace payloads, set [`RunConfig.trace_include_sensitive_data`][agents.run.RunConfig.trace_include_sensitive_data] to `False`:
```python
from agents import Runner, RunConfig
await Runner.run(
agent,
input="Hello",
run_config=RunConfig(trace_include_sensitive_data=False),
)
```
You can also change the default without code by setting this environment variable before your app starts:
```bash
export OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA=0
```
For full tracing controls, see the [tracing guide](tracing.md).
## Debug logging
The SDK defines two Python loggers (`openai.agents` and `openai.agents.tracing`) and does not attach handlers by default. Logs follow your application's Python logging configuration.
To enable verbose logging, use the [`enable_verbose_stdout_logging()`][agents.enable_verbose_stdout_logging] function.
```python
from agents import enable_verbose_stdout_logging
enable_verbose_stdout_logging()
```
Alternatively, you can customize the logs by adding handlers, filters, formatters, etc. You can read more in the [Python logging guide](https://docs.python.org/3/howto/logging.html).
```python
import logging
logger = logging.getLogger("openai.agents") # or openai.agents.tracing for the Tracing logger
# To make all logs show up
logger.setLevel(logging.DEBUG)
# To make info and above show up
logger.setLevel(logging.INFO)
# To make warning and above show up
logger.setLevel(logging.WARNING)
# etc
# You can customize this as needed, but this will output to `stderr` by default
logger.addHandler(logging.StreamHandler())
```
### Sensitive data in logs
Certain logs may contain sensitive data (for example, user data).
By default, the SDK does **not** log LLM inputs/outputs or tool inputs/outputs. These protections are controlled by:
```bash
OPENAI_AGENTS_DONT_LOG_MODEL_DATA=1
OPENAI_AGENTS_DONT_LOG_TOOL_DATA=1
```
If you need to include this data temporarily for debugging, set either variable to `0` (or `false`) before your app starts:
```bash
export OPENAI_AGENTS_DONT_LOG_MODEL_DATA=0
export OPENAI_AGENTS_DONT_LOG_TOOL_DATA=0
```
+144
View File
@@ -0,0 +1,144 @@
# Context management
Context is an overloaded term. There are two main classes of context you might care about:
1. Context available locally to your code: this is data and dependencies you might need when tool functions run, during callbacks like `on_handoff`, in lifecycle hooks, etc.
2. Context available to LLMs: this is data the LLM sees when generating a response.
## Local context
This is represented via the [`RunContextWrapper`][agents.run_context.RunContextWrapper] class and the [`context`][agents.run_context.RunContextWrapper.context] property within it. The way this works is:
1. You create any Python object you want. A common pattern is to use a dataclass or a Pydantic object.
2. You pass that object to the various run methods (e.g. `Runner.run(..., context=whatever)`).
3. All your tool calls, lifecycle hooks etc will be passed a wrapper object, `RunContextWrapper[T]`, where `T` represents your context object type which you can access via `wrapper.context`.
For some runtime-specific callbacks, the SDK may pass a more specialized subclass of `RunContextWrapper[T]`. For example, function-tool lifecycle hooks typically receive `ToolContext`, which also exposes tool-call metadata like `tool_call_id`, `tool_name`, and `tool_arguments`.
The **most important** thing to be aware of: every agent, tool function, lifecycle etc for a given agent run must use the same _type_ of context.
You can use the context for things like:
- Contextual data for your run (e.g. things like a username/uid or other information about the user)
- Dependencies (e.g. logger objects, data fetchers, etc)
- Helper functions
!!! danger "Note"
The context object is **not** sent to the LLM. It is purely a local object that you can read from, write to and call methods on it.
Within a single run, derived wrappers share the same underlying app context, approval state, and usage tracking. Nested [`Agent.as_tool()`][agents.agent.Agent.as_tool] runs may attach a different `tool_input`, but they do not get an isolated copy of your app state by default.
### What `RunContextWrapper` exposes
[`RunContextWrapper`][agents.run_context.RunContextWrapper] is a wrapper around your app-defined context object. In practice you will most often use:
- [`wrapper.context`][agents.run_context.RunContextWrapper.context] for your own mutable app state and dependencies.
- [`wrapper.usage`][agents.run_context.RunContextWrapper.usage] for aggregated request and token usage across the current run.
- [`wrapper.tool_input`][agents.run_context.RunContextWrapper.tool_input] for structured input when the current run is executing inside [`Agent.as_tool()`][agents.agent.Agent.as_tool].
- [`wrapper.approve_tool(...)`][agents.run_context.RunContextWrapper.approve_tool] / [`wrapper.reject_tool(...)`][agents.run_context.RunContextWrapper.reject_tool] when you need to update approval state programmatically.
Only `wrapper.context` is your app-defined object. The other fields are runtime metadata managed by the SDK.
If you later serialize a [`RunState`][agents.run_state.RunState] for human-in-the-loop or durable job workflows, that runtime metadata is saved with the state. Avoid putting secrets in [`RunContextWrapper.context`][agents.run_context.RunContextWrapper.context] if you intend to persist or transmit serialized state.
Conversation state is a separate concern. Use `result.to_input_list()`, `session`, `conversation_id`, or `previous_response_id` depending on how you want to carry turns forward. See [results](results.md), [running agents](running_agents.md), and [sessions](sessions/index.md) for that decision.
```python
import asyncio
from dataclasses import dataclass
from agents import Agent, RunContextWrapper, Runner, function_tool
@dataclass
class UserInfo: # (1)!
name: str
uid: int
@function_tool
async def fetch_user_age(wrapper: RunContextWrapper[UserInfo]) -> str: # (2)!
"""Fetch the age of the user. Call this function to get user's age information."""
return f"The user {wrapper.context.name} is 47 years old"
async def main():
user_info = UserInfo(name="John", uid=123)
agent = Agent[UserInfo]( # (3)!
name="Assistant",
tools=[fetch_user_age],
)
result = await Runner.run( # (4)!
starting_agent=agent,
input="What is the age of the user?",
context=user_info,
)
print(result.final_output) # (5)!
# The user John is 47 years old.
if __name__ == "__main__":
asyncio.run(main())
```
1. This is the context object. We've used a dataclass here, but you can use any type.
2. This is a tool. You can see it takes a `RunContextWrapper[UserInfo]`. The tool implementation reads from the context.
3. We mark the agent with the generic `UserInfo`, so that the typechecker can catch errors (for example, if we tried to pass a tool that took a different context type).
4. The context is passed to the `run` function.
5. The agent correctly calls the tool and gets the age.
---
### Advanced: `ToolContext`
In some cases, you might want to access extra metadata about the tool being executed — such as its name, call ID, or raw argument string.
For this, you can use the [`ToolContext`][agents.tool_context.ToolContext] class, which extends `RunContextWrapper`.
```python
from typing import Annotated
from pydantic import BaseModel, Field
from agents import Agent, Runner, function_tool
from agents.tool_context import ToolContext
class WeatherContext(BaseModel):
user_id: str
class Weather(BaseModel):
city: str = Field(description="The city name")
temperature_range: str = Field(description="The temperature range in Celsius")
conditions: str = Field(description="The weather conditions")
@function_tool
def get_weather(ctx: ToolContext[WeatherContext], city: Annotated[str, "The city to get the weather for"]) -> Weather:
print(f"[debug] Tool context: (name: {ctx.tool_name}, call_id: {ctx.tool_call_id}, args: {ctx.tool_arguments})")
return Weather(city=city, temperature_range="14-20C", conditions="Sunny with wind.")
agent = Agent(
name="Weather Agent",
instructions="You are a helpful agent that can tell the weather of a given city.",
tools=[get_weather],
)
```
`ToolContext` provides the same `.context` property as `RunContextWrapper`,
plus additional fields specific to the current tool call:
- `tool_name` the name of the tool being invoked
- `tool_call_id` a unique identifier for this tool call
- `tool_arguments` the raw argument string passed to the tool
- `tool_namespace` the Responses namespace for the tool call, when the tool was loaded through `tool_namespace()` or another namespaced surface
- `qualified_tool_name` the tool name qualified with the namespace when one is available
Use `ToolContext` when you need tool-level metadata during execution.
For general context sharing between agents and tools, `RunContextWrapper` remains sufficient. Because `ToolContext` extends `RunContextWrapper`, it can also expose `.tool_input` when a nested `Agent.as_tool()` run supplied structured input.
---
## Agent/LLM context
When an LLM is called, the **only** data it can see is from the conversation history. This means that if you want to make some new data available to the LLM, you must do it in a way that makes it available in that history. There are a few ways to do this:
1. You can add it to the Agent `instructions`. This is also known as a "system prompt" or "developer message". System prompts can be static strings, or they can be dynamic functions that receive the context and output a string. This is a common tactic for information that is always useful (for example, the user's name or the current date).
2. Add it to the `input` when calling the `Runner.run` functions. This is similar to the `instructions` tactic, but allows you to have messages that are lower in the [chain of command](https://cdn.openai.com/spec/model-spec-2024-05-08.html#follow-the-chain-of-command).
3. Expose it via function tools. This is useful for _on-demand_ context - the LLM decides when it needs some data, and can call the tool to fetch that data.
4. Use retrieval or web search. These are special tools that are able to fetch relevant data from files or databases (retrieval), or from the web (web search). This is useful for "grounding" the response in relevant contextual data.
+132
View File
@@ -0,0 +1,132 @@
# Examples
Check out a variety of sample implementations of the SDK in the examples section of the [repo](https://github.com/openai/openai-agents-python/tree/main/examples). The examples are organized into several categories that demonstrate different patterns and capabilities.
## Categories
- **[agent_patterns](https://github.com/openai/openai-agents-python/tree/main/examples/agent_patterns):** Examples in this category illustrate common agent design patterns, such as
- Deterministic workflows
- Agents as tools
- Agents as tools with streaming events (`examples/agent_patterns/agents_as_tools_streaming.py`)
- Agents as tools with structured input parameters (`examples/agent_patterns/agents_as_tools_structured.py`)
- Parallel agent execution
- Conditional tool usage
- Forcing tool use with different behaviors (`examples/agent_patterns/forcing_tool_use.py`)
- Input/output guardrails
- LLM as a judge
- Routing
- Streaming guardrails
- Human-in-the-loop with tool approval and state serialization (`examples/agent_patterns/human_in_the_loop.py`)
- Human-in-the-loop with streaming (`examples/agent_patterns/human_in_the_loop_stream.py`)
- Custom rejection messages for approval flows (`examples/agent_patterns/human_in_the_loop_custom_rejection.py`)
- **[basic](https://github.com/openai/openai-agents-python/tree/main/examples/basic):** These examples showcase foundational capabilities of the SDK, such as
- Hello world examples (Default model, GPT-5, open-weight model)
- Agent lifecycle management
- Run hooks and agent hooks lifecycle example (`examples/basic/lifecycle_example.py`)
- Dynamic system prompts
- Basic tool usage (`examples/basic/tools.py`)
- Tool input/output guardrails (`examples/basic/tool_guardrails.py`)
- Image tool output (`examples/basic/image_tool_output.py`)
- Streaming outputs (text, items, function call args)
- Responses websocket transport with a shared session helper across turns (`examples/basic/stream_ws.py`)
- Prompt templates
- File handling (local and remote, images and PDFs)
- Usage tracking
- Runner-managed retry settings (`examples/basic/retry.py`)
- Runner-managed retries through a third-party adapter (`examples/basic/retry_litellm.py`)
- Non-strict output types
- Previous response ID usage
- **[customer_service](https://github.com/openai/openai-agents-python/tree/main/examples/customer_service):** Example customer service system for an airline.
- **[financial_research_agent](https://github.com/openai/openai-agents-python/tree/main/examples/financial_research_agent):** A financial research agent that demonstrates structured research workflows with agents and tools for financial data analysis.
- **[handoffs](https://github.com/openai/openai-agents-python/tree/main/examples/handoffs):** Practical examples of agent handoffs with message filtering, including:
- Message filter example (`examples/handoffs/message_filter.py`)
- Message filter with streaming (`examples/handoffs/message_filter_streaming.py`)
- **[hosted_mcp](https://github.com/openai/openai-agents-python/tree/main/examples/hosted_mcp):** Examples demonstrating how to use hosted MCP (Model Context Protocol) with the OpenAI Responses API, including:
- Simple hosted MCP without approval (`examples/hosted_mcp/simple.py`)
- MCP connectors such as Google Calendar (`examples/hosted_mcp/connectors.py`)
- Human-in-the-loop with interruption-based approvals (`examples/hosted_mcp/human_in_the_loop.py`)
- On-approval callback for MCP tool calls (`examples/hosted_mcp/on_approval.py`)
- **[mcp](https://github.com/openai/openai-agents-python/tree/main/examples/mcp):** Learn how to build agents with MCP (Model Context Protocol), including:
- Filesystem examples
- Git examples
- MCP prompt server examples
- SSE (Server-Sent Events) examples
- SSE remote server connection (`examples/mcp/sse_remote_example`)
- Streamable HTTP examples
- Streamable HTTP remote connection (`examples/mcp/streamable_http_remote_example`)
- Custom HTTP client factory for Streamable HTTP (`examples/mcp/streamablehttp_custom_client_example`)
- Prefetching all MCP tools with `MCPUtil.get_all_function_tools` (`examples/mcp/get_all_mcp_tools_example`)
- MCPServerManager with FastAPI (`examples/mcp/manager_example`)
- MCP tool filtering (`examples/mcp/tool_filter_example`)
- **[memory](https://github.com/openai/openai-agents-python/tree/main/examples/memory):** Examples of different memory implementations for agents, including:
- SQLite session storage
- Advanced SQLite session storage
- Redis session storage
- SQLAlchemy session storage
- Dapr state store session storage
- Encrypted session storage
- OpenAI Conversations session storage
- Responses compaction session storage
- Stateless Responses compaction with `ModelSettings(store=False)` (`examples/memory/compaction_session_stateless_example.py`)
- File-backed session storage (`examples/memory/file_session.py`)
- File-backed session with human-in-the-loop (`examples/memory/file_hitl_example.py`)
- SQLite in-memory session with human-in-the-loop (`examples/memory/memory_session_hitl_example.py`)
- OpenAI Conversations session with human-in-the-loop (`examples/memory/openai_session_hitl_example.py`)
- HITL approval/rejection scenario across sessions (`examples/memory/hitl_session_scenario.py`)
- **[model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers):** Explore how to use non-OpenAI models with the SDK, including custom providers and third-party adapters.
- **[realtime](https://github.com/openai/openai-agents-python/tree/main/examples/realtime):** Examples showing how to build real-time experiences using the SDK, including:
- Web application patterns with structured text and image messages
- Command-line audio loops and playback handling
- Twilio Media Streams integration over WebSocket
- Twilio SIP integration using Realtime Calls API attach flows
- **[reasoning_content](https://github.com/openai/openai-agents-python/tree/main/examples/reasoning_content):** Examples demonstrating how to work with reasoning content, including:
- Reasoning content with the Runner API, streaming and non-streaming (`examples/reasoning_content/runner_example.py`)
- Reasoning content with OSS models via OpenRouter (`examples/reasoning_content/gpt_oss_stream.py`)
- Basic reasoning content example (`examples/reasoning_content/main.py`)
- **[research_bot](https://github.com/openai/openai-agents-python/tree/main/examples/research_bot):** Simple deep research clone that demonstrates complex multi-agent research workflows.
- **[sandbox](https://github.com/openai/openai-agents-python/tree/main/examples/sandbox):** Examples for running agents in isolated workspaces, including:
- Basic sandbox agent setup (`examples/sandbox/basic.py`)
- Unix-local and Docker sandbox lifecycle examples
- Sandbox-backed handoffs (`examples/sandbox/handoffs.py`)
- Sandbox memory and snapshot resume (`examples/sandbox/memory.py`)
- Sandbox agents exposed as tools (`examples/sandbox/sandbox_agents_as_tools.py`)
- **[tools](https://github.com/openai/openai-agents-python/tree/main/examples/tools):** Learn how to implement OAI hosted tools and experimental Codex tooling such as:
- Web search and web search with filters
- File search
- Code interpreter
- Apply patch tool with file editing and approval (`examples/tools/apply_patch.py`)
- Shell tool execution with approval callbacks (`examples/tools/shell.py`)
- Shell tool with human-in-the-loop interruption-based approvals (`examples/tools/shell_human_in_the_loop.py`)
- Hosted container shell with inline skills (`examples/tools/container_shell_inline_skill.py`)
- Hosted container shell with skill references (`examples/tools/container_shell_skill_reference.py`)
- Local shell with local skills (`examples/tools/local_shell_skill.py`)
- Tool search with namespaces and deferred tools (`examples/tools/tool_search.py`)
- Computer use
- Image generation
- Experimental Codex tool workflows (`examples/tools/codex.py`)
- Experimental Codex same-thread workflows (`examples/tools/codex_same_thread.py`)
- **[voice](https://github.com/openai/openai-agents-python/tree/main/examples/voice):** See examples of voice agents, using our TTS and STT models, including streamed voice examples.
+230
View File
@@ -0,0 +1,230 @@
# Guardrails
Guardrails enable you to do checks and validations of user input and agent output. For example, imagine you have an agent that uses a very smart (and hence slow/expensive) model to help with customer requests. You wouldn't want malicious users to ask the model to help them with their math homework. So, you can run a guardrail with a fast/cheap model. If the guardrail detects malicious usage, it can immediately raise an error and prevent the expensive model from running, saving you time and money (**when using blocking guardrails; for parallel guardrails, the expensive model may have already started running before the guardrail completes. See "Execution modes" below for details**).
There are two kinds of guardrails:
1. Input guardrails run on the initial user input
2. Output guardrails run on the final agent output
## Workflow boundaries
Guardrails are attached to agents and tools, but they do not all run at the same points in a workflow:
- **Input guardrails** run only for the first agent in the chain.
- **Output guardrails** run only for the agent that produces the final output.
- **Tool guardrails** run on every custom function-tool invocation, with input guardrails before execution and output guardrails after execution.
If you need checks around each custom function-tool call in a workflow that includes managers, handoffs, or delegated specialists, use tool guardrails instead of relying only on agent-level input/output guardrails.
## Input guardrails
Input guardrails run in 3 steps:
1. First, the guardrail receives the same input passed to the agent.
2. Next, the guardrail function runs to produce a [`GuardrailFunctionOutput`][agents.guardrail.GuardrailFunctionOutput], which is then wrapped in an [`InputGuardrailResult`][agents.guardrail.InputGuardrailResult]
3. Finally, we check if [`.tripwire_triggered`][agents.guardrail.GuardrailFunctionOutput.tripwire_triggered] is true. If true, an [`InputGuardrailTripwireTriggered`][agents.exceptions.InputGuardrailTripwireTriggered] exception is raised, so you can appropriately respond to the user or handle the exception.
!!! Note
Input guardrails are intended to run on user input, so an agent's guardrails only run if the agent is the *first* agent. You might wonder, why is the `guardrails` property on the agent instead of passed to `Runner.run`? It's because guardrails tend to be related to the actual Agent - you'd run different guardrails for different agents, so colocating the code is useful for readability.
### Execution modes
Input guardrails support two execution modes:
- **Parallel execution** (default, `run_in_parallel=True`): The guardrail runs concurrently with the agent's execution. This provides the best latency since both start at the same time. However, if the guardrail fails, the agent may have already consumed tokens and executed tools before being cancelled.
- **Blocking execution** (`run_in_parallel=False`): The guardrail runs and completes *before* the agent starts. If the guardrail tripwire is triggered, the agent never executes, preventing token consumption and tool execution. This is ideal for cost optimization and when you want to avoid potential side effects from tool calls.
## Output guardrails
Output guardrails run in 3 steps:
1. First, the guardrail receives the output produced by the agent.
2. Next, the guardrail function runs to produce a [`GuardrailFunctionOutput`][agents.guardrail.GuardrailFunctionOutput], which is then wrapped in an [`OutputGuardrailResult`][agents.guardrail.OutputGuardrailResult]
3. Finally, we check if [`.tripwire_triggered`][agents.guardrail.GuardrailFunctionOutput.tripwire_triggered] is true. If true, an [`OutputGuardrailTripwireTriggered`][agents.exceptions.OutputGuardrailTripwireTriggered] exception is raised, so you can appropriately respond to the user or handle the exception.
!!! Note
Output guardrails are intended to run on the final agent output, so an agent's guardrails only run if the agent is the *last* agent. Similar to the input guardrails, we do this because guardrails tend to be related to the actual Agent - you'd run different guardrails for different agents, so colocating the code is useful for readability.
Output guardrails always run after the agent completes, so they don't support the `run_in_parallel` parameter.
## Tool guardrails
Tool guardrails wrap **function tools** and let you validate or block tool calls before and after execution. They are configured on the tool itself and run every time that tool is invoked.
- Input tool guardrails run before the tool executes and can skip the call, replace the output with a message, or raise a tripwire.
- Output tool guardrails run after the tool executes and can replace the output or raise a tripwire.
- If a function tool requires approval, input tool guardrails normally run after approval and immediately before execution. Set [`RunConfig.tool_execution`][agents.run.RunConfig.tool_execution] to [`ToolExecutionConfig(pre_approval_tool_input_guardrails=True)`][agents.run.ToolExecutionConfig] when you want those input checks to run before the pending approval interruption is emitted. Calls that pass this pre-approval check are still checked again after approval before the tool executes.
- Tool guardrails apply only to function tools created with [`function_tool`][agents.tool.function_tool]. Handoffs run through the SDK's handoff pipeline rather than the normal function-tool pipeline, so tool guardrails do not apply to the handoff call itself. Hosted tools (`WebSearchTool`, `FileSearchTool`, `HostedMCPTool`, `CodeInterpreterTool`, `ImageGenerationTool`) and built-in execution tools (`ComputerTool`, `ShellTool`, `ApplyPatchTool`, `LocalShellTool`) also do not use this guardrail pipeline, and [`Agent.as_tool()`][agents.agent.Agent.as_tool] does not currently expose tool-guardrail options directly.
See the code snippet below for details.
## Tripwires
If the input or output fails the guardrail, the Guardrail can signal this with a tripwire. As soon as we see a guardrail that has triggered the tripwires, we immediately raise a `{Input,Output}GuardrailTripwireTriggered` exception and halt the Agent execution.
## Implementing a guardrail
You need to provide a function that receives input, and returns a [`GuardrailFunctionOutput`][agents.guardrail.GuardrailFunctionOutput]. In this example, we'll do this by running an Agent under the hood.
```python
from pydantic import BaseModel
from agents import (
Agent,
GuardrailFunctionOutput,
InputGuardrailTripwireTriggered,
RunContextWrapper,
Runner,
TResponseInputItem,
input_guardrail,
)
class MathHomeworkOutput(BaseModel):
is_math_homework: bool
reasoning: str
guardrail_agent = Agent( # (1)!
name="Guardrail check",
instructions="Check if the user is asking you to do their math homework.",
output_type=MathHomeworkOutput,
)
@input_guardrail
async def math_guardrail( # (2)!
ctx: RunContextWrapper[None], agent: Agent, input: str | list[TResponseInputItem]
) -> GuardrailFunctionOutput:
result = await Runner.run(guardrail_agent, input, context=ctx.context)
return GuardrailFunctionOutput(
output_info=result.final_output, # (3)!
tripwire_triggered=result.final_output.is_math_homework,
)
agent = Agent( # (4)!
name="Customer support agent",
instructions="You are a customer support agent. You help customers with their questions.",
input_guardrails=[math_guardrail],
)
async def main():
# This should trip the guardrail
try:
await Runner.run(agent, "Hello, can you help me solve for x: 2x + 3 = 11?")
print("Guardrail didn't trip - this is unexpected")
except InputGuardrailTripwireTriggered:
print("Math homework guardrail tripped")
```
1. We'll use this agent in our guardrail function.
2. This is the guardrail function that receives the agent's input/context, and returns the result.
3. We can include extra information in the guardrail result.
4. This is the actual agent that defines the workflow.
Output guardrails are similar.
```python
from pydantic import BaseModel
from agents import (
Agent,
GuardrailFunctionOutput,
OutputGuardrailTripwireTriggered,
RunContextWrapper,
Runner,
output_guardrail,
)
class MessageOutput(BaseModel): # (1)!
response: str
class MathOutput(BaseModel): # (2)!
reasoning: str
is_math: bool
guardrail_agent = Agent(
name="Guardrail check",
instructions="Check if the output includes any math.",
output_type=MathOutput,
)
@output_guardrail
async def math_guardrail( # (3)!
ctx: RunContextWrapper, agent: Agent, output: MessageOutput
) -> GuardrailFunctionOutput:
result = await Runner.run(guardrail_agent, output.response, context=ctx.context)
return GuardrailFunctionOutput(
output_info=result.final_output,
tripwire_triggered=result.final_output.is_math,
)
agent = Agent( # (4)!
name="Customer support agent",
instructions="You are a customer support agent. You help customers with their questions.",
output_guardrails=[math_guardrail],
output_type=MessageOutput,
)
async def main():
# This should trip the guardrail
try:
await Runner.run(agent, "Hello, can you help me solve for x: 2x + 3 = 11?")
print("Guardrail didn't trip - this is unexpected")
except OutputGuardrailTripwireTriggered:
print("Math output guardrail tripped")
```
1. This is the actual agent's output type.
2. This is the guardrail's output type.
3. This is the guardrail function that receives the agent's output, and returns the result.
4. This is the actual agent that defines the workflow.
Lastly, here are examples of tool guardrails.
```python
import json
from agents import (
Agent,
Runner,
ToolGuardrailFunctionOutput,
function_tool,
tool_input_guardrail,
tool_output_guardrail,
)
@tool_input_guardrail
def block_secrets(data):
args = json.loads(data.context.tool_arguments or "{}")
if "sk-" in json.dumps(args):
return ToolGuardrailFunctionOutput.reject_content(
"Remove secrets before calling this tool."
)
return ToolGuardrailFunctionOutput.allow()
@tool_output_guardrail
def redact_output(data):
text = str(data.output or "")
if "sk-" in text:
return ToolGuardrailFunctionOutput.reject_content("Output contained sensitive data.")
return ToolGuardrailFunctionOutput.allow()
@function_tool(
tool_input_guardrails=[block_secrets],
tool_output_guardrails=[redact_output],
)
def classify_text(text: str) -> str:
"""Classify text for internal routing."""
return f"length:{len(text)}"
agent = Agent(name="Classifier", tools=[classify_text])
result = Runner.run_sync(agent, "hello world")
print(result.final_output)
```
+152
View File
@@ -0,0 +1,152 @@
# Handoffs
Handoffs allow an agent to delegate tasks to another agent. This is particularly useful in scenarios where different agents specialize in distinct areas. For example, a customer support app might have agents that each specifically handle tasks like order status, refunds, FAQs, etc.
Handoffs are represented as tools to the LLM. So if there's a handoff to an agent named `Refund Agent`, the tool would be called `transfer_to_refund_agent`.
## Creating a handoff
All agents have a [`handoffs`][agents.agent.Agent.handoffs] param, which can either take an `Agent` directly, or a `Handoff` object that customizes the Handoff.
If you pass plain `Agent` instances, their [`handoff_description`][agents.agent.Agent.handoff_description] (when set) is appended to the default tool description. Use it to hint when the model should pick that handoff without writing a full `handoff()` object.
You can create a handoff using the [`handoff()`][agents.handoffs.handoff] function provided by the Agents SDK. This function allows you to specify the agent to hand off to, along with optional overrides and input filters.
### Basic usage
Here's how you can create a simple handoff:
```python
from agents import Agent, handoff
billing_agent = Agent(name="Billing agent")
refund_agent = Agent(name="Refund agent")
# (1)!
triage_agent = Agent(name="Triage agent", handoffs=[billing_agent, handoff(refund_agent)])
```
1. You can use the agent directly (as in `billing_agent`), or you can use the `handoff()` function.
### Customizing handoffs via the `handoff()` function
The [`handoff()`][agents.handoffs.handoff] function lets you customize things.
- `agent`: This is the agent to which things will be handed off.
- `tool_name_override`: By default, the `Handoff.default_tool_name()` function is used, which resolves to `transfer_to_<agent_name>`. You can override this.
- `tool_description_override`: Override the default tool description from `Handoff.default_tool_description()`
- `on_handoff`: A callback function executed when the handoff is invoked. This is useful for things like kicking off some data fetching as soon as you know a handoff is being invoked. This function receives the agent context, and can optionally also receive LLM generated input. The input data is controlled by the `input_type` param.
- `input_type`: The schema for the handoff tool-call arguments. When set, the parsed payload is passed to `on_handoff`.
- `input_filter`: This lets you filter the input received by the next agent. See below for more.
- `is_enabled`: Whether the handoff is enabled. This can be a boolean or a function that returns a boolean, allowing you to dynamically enable or disable the handoff at runtime.
- `nest_handoff_history`: Optional per-call override for the RunConfig-level `nest_handoff_history` setting. If `None`, the value defined in the active run configuration is used instead.
The [`handoff()`][agents.handoffs.handoff] helper always transfers control to the specific `agent` you passed in. If you have multiple possible destinations, register one handoff per destination and let the model choose among them. Use a custom [`Handoff`][agents.handoffs.Handoff] only when your own handoff code must decide which agent to return at invocation time.
```python
from agents import Agent, handoff, RunContextWrapper
def on_handoff(ctx: RunContextWrapper[None]):
print("Handoff called")
agent = Agent(name="My agent")
handoff_obj = handoff(
agent=agent,
on_handoff=on_handoff,
tool_name_override="custom_handoff_tool",
tool_description_override="Custom description",
)
```
## Handoff inputs
In certain situations, you want the LLM to provide some data when it calls a handoff. For example, imagine a handoff to an "Escalation agent". You might want the model to provide a reason so you can log it.
```python
from pydantic import BaseModel
from agents import Agent, handoff, RunContextWrapper
class EscalationData(BaseModel):
reason: str
async def on_handoff(ctx: RunContextWrapper[None], input_data: EscalationData):
print(f"Escalation agent called with reason: {input_data.reason}")
agent = Agent(name="Escalation agent")
handoff_obj = handoff(
agent=agent,
on_handoff=on_handoff,
input_type=EscalationData,
)
```
`input_type` describes the arguments for the handoff tool call itself. The SDK exposes that schema to the model as the handoff tool's `parameters`, validates the returned JSON locally, and passes the parsed value to `on_handoff`.
It does not replace the next agent's main input, and it does not choose a different destination. The [`handoff()`][agents.handoffs.handoff] helper still transfers to the specific agent you wrapped, and the receiving agent still sees the conversation history unless you change it with an [`input_filter`][agents.handoffs.Handoff.input_filter] or nested handoff history settings.
`input_type` is also separate from [`RunContextWrapper.context`][agents.run_context.RunContextWrapper.context]. Use `input_type` for metadata the model decides at handoff time, not for application state or dependencies you already have locally.
### When to use `input_type`
Use `input_type` when the handoff needs a small piece of model-generated metadata such as `reason`, `language`, `priority`, or `summary`. For example, a triage agent can hand off to a refund agent with `{ "reason": "duplicate_charge", "priority": "high" }`, and `on_handoff` can log or persist that metadata before the refund agent takes over.
Choose a different mechanism when the goal is different:
- Put existing application state and dependencies in [`RunContextWrapper.context`][agents.run_context.RunContextWrapper.context]. See the [context guide](context.md).
- Use [`input_filter`][agents.handoffs.Handoff.input_filter], [`RunConfig.nest_handoff_history`][agents.run.RunConfig.nest_handoff_history], or [`RunConfig.handoff_history_mapper`][agents.run.RunConfig.handoff_history_mapper] if you want to change what history the receiving agent sees.
- Register one handoff per destination if there are multiple possible specialists. `input_type` can add metadata to the chosen handoff, but it does not dispatch between destinations.
- If you want structured input for a nested specialist without transferring the conversation, prefer [`Agent.as_tool(parameters=...)`][agents.agent.Agent.as_tool]. See [tools](tools.md#structured-input-for-tool-agents).
## Input filters
When a handoff occurs, it's as though the new agent takes over the conversation, and gets to see the entire previous conversation history. If you want to change this, you can set an [`input_filter`][agents.handoffs.Handoff.input_filter]. An input filter is a function that receives the existing input via a [`HandoffInputData`][agents.handoffs.HandoffInputData], and must return a new `HandoffInputData`.
[`HandoffInputData`][agents.handoffs.HandoffInputData] includes:
- `input_history`: the input history before `Runner.run(...)` started.
- `pre_handoff_items`: items generated before the agent turn where the handoff was invoked.
- `new_items`: items generated during the current turn, including the handoff call and handoff output items.
- `input_items`: optional items to forward to the next agent instead of `new_items`, allowing you to filter model input while keeping `new_items` intact for session history.
- `run_context`: the active [`RunContextWrapper`][agents.run_context.RunContextWrapper] at the time the handoff was invoked.
Nested handoffs are available as an opt-in beta and are disabled by default while we stabilize them. When you enable [`RunConfig.nest_handoff_history`][agents.run.RunConfig.nest_handoff_history], the runner collapses the prior transcript into a single assistant summary message and wraps it in a `<CONVERSATION HISTORY>` block that keeps appending new turns when multiple handoffs happen during the same run. You can provide your own mapping function via [`RunConfig.handoff_history_mapper`][agents.run.RunConfig.handoff_history_mapper] to replace the generated message without writing a full `input_filter`. The opt-in only applies when neither the handoff nor the run supplies an explicit `input_filter`, so existing code that already customizes the payload (including the examples in this repository) keeps its current behavior without changes. You can override the nesting behaviour for a single handoff by passing `nest_handoff_history=True` or `False` to [`handoff(...)`][agents.handoffs.handoff], which sets [`Handoff.nest_handoff_history`][agents.handoffs.Handoff.nest_handoff_history]. If you just need to change the wrapper text for the generated summary, call [`set_conversation_history_wrappers`][agents.handoffs.set_conversation_history_wrappers] (and optionally [`reset_conversation_history_wrappers`][agents.handoffs.reset_conversation_history_wrappers]) before running your agents.
If both the handoff and the active [`RunConfig.handoff_input_filter`][agents.run.RunConfig.handoff_input_filter] define a filter, the per-handoff [`input_filter`][agents.handoffs.Handoff.input_filter] takes precedence for that specific handoff.
!!! note
Handoffs stay within a single run. Input guardrails still apply only to the first agent in the chain, and output guardrails only to the agent that produces the final output. Use tool guardrails when you need checks around each custom function-tool call inside the workflow.
There are some common patterns (for example removing all tool calls from the history), which are implemented for you in [`agents.extensions.handoff_filters`][]
```python
from agents import Agent, handoff
from agents.extensions import handoff_filters
agent = Agent(name="FAQ agent")
handoff_obj = handoff(
agent=agent,
input_filter=handoff_filters.remove_all_tools, # (1)!
)
```
1. This will automatically remove all tools from the history when `FAQ agent` is called.
## Recommended prompts
To make sure that LLMs understand handoffs properly, we recommend including information about handoffs in your agents. We have a suggested prefix in [`agents.extensions.handoff_prompt.RECOMMENDED_PROMPT_PREFIX`][], or you can call [`agents.extensions.handoff_prompt.prompt_with_handoff_instructions`][] to automatically add recommended data to your prompts.
```python
from agents import Agent
from agents.extensions.handoff_prompt import RECOMMENDED_PROMPT_PREFIX
billing_agent = Agent(
name="Billing agent",
instructions=f"""{RECOMMENDED_PROMPT_PREFIX}
<Fill in the rest of your prompt here>.""",
)
```
+197
View File
@@ -0,0 +1,197 @@
# Human-in-the-loop
Use the human-in-the-loop (HITL) flow to pause agent execution until a person approves or rejects sensitive tool calls. Tools declare when they need approval, run results surface pending approvals as interruptions, and `RunState` lets you serialize and resume runs after decisions are made.
That approval surface is run-wide, not limited to the current top-level agent. The same pattern applies when the tool belongs to the current agent, to an agent reached through a handoff, or to a nested [`Agent.as_tool()`][agents.agent.Agent.as_tool] execution. In the nested `Agent.as_tool()` case, the interruption still surfaces on the outer run, so you approve or reject it on the outer `RunState` and resume the original top-level run.
With `Agent.as_tool()`, approvals can happen at two different layers: the agent tool itself can require approval via `Agent.as_tool(..., needs_approval=...)`, and tools inside the nested agent can later raise their own approvals after the nested run starts. Both are handled through the same outer-run interruption flow.
This page focuses on the manual approval flow via `interruptions`. If your app can decide in code, some tool types also support programmatic approval callbacks so the run can continue without pausing.
## Marking tools that need approval
Set `needs_approval` to `True` to always require approval or provide an async function that decides per call. The callable receives the run context, parsed tool parameters, and the tool call ID.
```python
from agents import Agent, Runner, function_tool
@function_tool(needs_approval=True)
async def cancel_order(order_id: int) -> str:
return f"Cancelled order {order_id}"
async def requires_review(_ctx, params, _call_id) -> bool:
return "refund" in params.get("subject", "").lower()
@function_tool(needs_approval=requires_review)
async def send_email(subject: str, body: str) -> str:
return f"Sent '{subject}'"
agent = Agent(
name="Support agent",
instructions="Handle tickets and ask for approval when needed.",
tools=[cancel_order, send_email],
)
```
`needs_approval` is available on [`function_tool`][agents.tool.function_tool], [`Agent.as_tool`][agents.agent.Agent.as_tool], [`ShellTool`][agents.tool.ShellTool], and [`ApplyPatchTool`][agents.tool.ApplyPatchTool]. Local MCP servers also support approvals through `require_approval` on [`MCPServerStdio`][agents.mcp.server.MCPServerStdio], [`MCPServerSse`][agents.mcp.server.MCPServerSse], and [`MCPServerStreamableHttp`][agents.mcp.server.MCPServerStreamableHttp]. Hosted MCP servers support approvals via [`HostedMCPTool`][agents.tool.HostedMCPTool] with `tool_config={"require_approval": "always"}` and an optional `on_approval_request` callback. Shell and apply_patch tools accept an `on_approval` callback if you want to auto-approve or auto-reject without surfacing an interruption.
## How the approval flow works
1. When the model emits a tool call, the runner evaluates its approval rule (`needs_approval`, `require_approval`, or the hosted MCP equivalent).
2. If an approval decision for that tool call is already stored in the [`RunContextWrapper`][agents.run_context.RunContextWrapper], the runner proceeds without prompting. Per-call approvals are scoped to the specific call ID; pass `always_approve=True` or `always_reject=True` to persist the same decision for future calls to that tool during the rest of the run.
3. Otherwise, execution pauses and `RunResult.interruptions` (or `RunResultStreaming.interruptions`) contains [`ToolApprovalItem`][agents.items.ToolApprovalItem] entries with details such as `agent.name`, `tool_name`, and `arguments`. This includes approvals raised after a handoff or inside nested `Agent.as_tool()` executions.
4. Convert the result to a `RunState` with `result.to_state()`, call `state.approve(...)` or `state.reject(...)`, and then resume with `Runner.run(agent, state)` or `Runner.run_streamed(agent, state)`, where `agent` is the original top-level agent for the run.
5. The resumed run continues where it left off and will re-enter this flow if new approvals are needed.
Sticky decisions created with `always_approve=True` or `always_reject=True` are stored in the run state, so they survive `state.to_string()` / `RunState.from_string(...)` and `state.to_json()` / `RunState.from_json(...)` when you resume the same paused run later.
You do not need to resolve every pending approval in the same pass. `interruptions` can contain a mix of regular function tools, hosted MCP approvals, and nested `Agent.as_tool()` approvals. If you rerun after approving or rejecting only some items, those resolved calls can continue while unresolved ones remain in `interruptions` and pause the run again.
## Custom rejection messages
By default, a rejected tool call returns the SDK's standard rejection text back into the run. You can customize that message in two layers:
- Run-wide fallback: set [`RunConfig.tool_error_formatter`][agents.run.RunConfig.tool_error_formatter] to control the default model-visible message for approval rejections across the whole run.
- Per-call override: pass `rejection_message=...` to `state.reject(...)` when you want one specific rejected tool call to surface a different message.
If both are provided, the per-call `rejection_message` takes precedence over the run-wide formatter.
```python
from agents import RunConfig, ToolErrorFormatterArgs
def format_rejection(args: ToolErrorFormatterArgs[None]) -> str | None:
if args.kind != "approval_rejected":
return None
return "Publish action was canceled because approval was rejected."
run_config = RunConfig(tool_error_formatter=format_rejection)
# Later, while resolving a specific interruption:
state.reject(
interruption,
rejection_message="Publish action was canceled because the reviewer denied approval.",
)
```
See [`examples/agent_patterns/human_in_the_loop_custom_rejection.py`](https://github.com/openai/openai-agents-python/tree/main/examples/agent_patterns/human_in_the_loop_custom_rejection.py) for a complete example that shows both layers together.
## Automatic approval decisions
Manual `interruptions` are the most general pattern, but they are not the only one:
- Local [`ShellTool`][agents.tool.ShellTool] and [`ApplyPatchTool`][agents.tool.ApplyPatchTool] can use `on_approval` to approve or reject immediately in code.
- [`HostedMCPTool`][agents.tool.HostedMCPTool] can use `tool_config={"require_approval": "always"}` together with `on_approval_request` for the same kind of programmatic decision.
- Plain [`function_tool`][agents.tool.function_tool] tools and [`Agent.as_tool()`][agents.agent.Agent.as_tool] use the manual interruption flow on this page.
When these callbacks return a decision, the run continues without pausing for a human response. For Realtime and voice session APIs, see the approval flow in the [Realtime guide](realtime/guide.md).
## Streaming and sessions
The same interruption flow works in streaming runs. After a streamed run pauses, keep consuming [`RunResultStreaming.stream_events()`][agents.result.RunResultStreaming.stream_events] until the iterator finishes, inspect [`RunResultStreaming.interruptions`][agents.result.RunResultStreaming.interruptions], resolve them, and resume with [`Runner.run_streamed(...)`][agents.run.Runner.run_streamed] if you want the resumed output to keep streaming. See [Streaming](streaming.md) for the streamed version of this pattern.
If you are also using a session, keep passing the same session instance when you resume from `RunState`, or pass another session object that points at the same backing store. The resumed turn is then appended to the same stored conversation history. See [Sessions](sessions/index.md) for the session lifecycle details.
## Example: pause, approve, resume
The snippet below mirrors the JavaScript HITL guide: it pauses when a tool needs approval, persists state to disk, reloads it, and resumes after collecting a decision.
```python
import asyncio
import json
from pathlib import Path
from agents import Agent, Runner, RunState, function_tool
async def needs_oakland_approval(_ctx, params, _call_id) -> bool:
return "Oakland" in params.get("city", "")
@function_tool(needs_approval=needs_oakland_approval)
async def get_temperature(city: str) -> str:
return f"The temperature in {city} is 20° Celsius"
agent = Agent(
name="Weather assistant",
instructions="Answer weather questions with the provided tools.",
tools=[get_temperature],
)
STATE_PATH = Path(".cache/hitl_state.json")
def prompt_approval(tool_name: str, arguments: str | None) -> bool:
answer = input(f"Approve {tool_name} with {arguments}? [y/N]: ").strip().lower()
return answer in {"y", "yes"}
async def main() -> None:
result = await Runner.run(agent, "What is the temperature in Oakland?")
while result.interruptions:
# Persist the paused state.
state = result.to_state()
STATE_PATH.parent.mkdir(parents=True, exist_ok=True)
STATE_PATH.write_text(state.to_string())
# Load the state later (could be a different process).
stored = json.loads(STATE_PATH.read_text())
state = await RunState.from_json(agent, stored)
for interruption in result.interruptions:
approved = await asyncio.get_running_loop().run_in_executor(
None, prompt_approval, interruption.name or "unknown_tool", interruption.arguments
)
if approved:
state.approve(interruption, always_approve=False)
else:
state.reject(interruption)
result = await Runner.run(agent, state)
print(result.final_output)
if __name__ == "__main__":
asyncio.run(main())
```
In this example, `prompt_approval` is synchronous because it uses `input()` and is executed with `run_in_executor(...)`. If your approval source is already asynchronous (for example, an HTTP request or async database query), you can use an `async def` function and `await` it directly instead.
To stream output while waiting for approvals, call `Runner.run_streamed`, consume `result.stream_events()` until it completes, and then follow the same `result.to_state()` and resume steps shown above.
## Repository patterns and examples
- **Streaming approvals**: `examples/agent_patterns/human_in_the_loop_stream.py` shows how to drain `stream_events()` and then approve pending tool calls before resuming with `Runner.run_streamed(agent, state)`.
- **Custom rejection text**: `examples/agent_patterns/human_in_the_loop_custom_rejection.py` shows how to combine run-level `tool_error_formatter` with per-call `rejection_message` overrides when approvals are rejected.
- **Agent as tool approvals**: `Agent.as_tool(..., needs_approval=...)` applies the same interruption flow when delegated agent tasks need review. Nested interruptions still surface on the outer run, so resume the original top-level agent rather than the nested one.
- **Local shell and apply_patch tools**: `ShellTool` and `ApplyPatchTool` also support `needs_approval`. Use `state.approve(interruption, always_approve=True)` or `state.reject(..., always_reject=True)` to cache the decision for future calls. For automatic decisions, provide `on_approval` (see `examples/tools/shell.py`); for manual decisions, handle interruptions (see `examples/tools/shell_human_in_the_loop.py`). Hosted shell environments do not support `needs_approval` or `on_approval`; see the [tools guide](tools.md).
- **Local MCP servers**: Use `require_approval` on `MCPServerStdio` / `MCPServerSse` / `MCPServerStreamableHttp` to gate MCP tool calls (see `examples/mcp/get_all_mcp_tools_example/main.py` and `examples/mcp/tool_filter_example/main.py`).
- **Hosted MCP servers**: Set `require_approval` to `"always"` on `HostedMCPTool` to force HITL, optionally providing `on_approval_request` to auto-approve or reject (see `examples/hosted_mcp/human_in_the_loop.py` and `examples/hosted_mcp/on_approval.py`). Use `"never"` for trusted servers (`examples/hosted_mcp/simple.py`).
- **Sessions and memory**: Pass a session to `Runner.run` so approvals and conversation history survive multiple turns. SQLite and OpenAI Conversations session variants are in `examples/memory/memory_session_hitl_example.py` and `examples/memory/openai_session_hitl_example.py`.
- **Realtime agents**: The realtime demo exposes WebSocket messages that approve or reject tool calls via `approve_tool_call` / `reject_tool_call` on the `RealtimeSession` (see `examples/realtime/app/server.py` for the server-side handlers and [Realtime guide](realtime/guide.md#tool-approvals) for the API surface).
## Long-running approvals
`RunState` is designed to be durable. Use `state.to_json()` or `state.to_string()` to store pending work in a database or queue and recreate it later with `RunState.from_json(...)` or `RunState.from_string(...)`.
Useful serialization options:
- `context_serializer`: Customize how non-mapping context objects are serialized.
- `context_deserializer`: Rebuild non-mapping context objects when loading state with `RunState.from_json(...)` or `RunState.from_string(...)`.
- `strict_context=True`: Fail serialization or deserialization unless the context is already a mapping or you provide the appropriate serializer/deserializer.
- `context_override`: Replace the serialized context when loading state. This is useful when you do not want to restore the original context object, but it does not remove that context from an already serialized payload.
- `include_tracing_api_key=True`: Include the tracing API key in the serialized trace payload when you need resumed work to keep exporting traces with the same credentials.
Serialized run state includes your app context plus SDK-managed runtime metadata such as approvals, usage, serialized `tool_input`, nested agent-as-tool resumptions, trace metadata, and server-managed conversation settings. If you plan to store or transmit serialized state, treat `RunContextWrapper.context` as persisted data and avoid placing secrets there unless you intentionally want them to travel with the state.
## Versioning pending tasks
If approvals may sit for a while, store a version marker for your agent definitions or SDK alongside the serialized state. You can then route deserialization to the matching code path to avoid incompatibilities when models, prompts, or tool definitions change.
+97
View File
@@ -0,0 +1,97 @@
# OpenAI Agents SDK
The [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) enables you to build agentic AI apps in a lightweight, easy-to-use package with very few abstractions. It's a production-ready upgrade of our previous experimentation for agents, [Swarm](https://github.com/openai/swarm/tree/main). The Agents SDK has a very small set of primitives:
- **Agents**, which are LLMs equipped with instructions and tools
- **Agents as tools / Handoffs**, which allow agents to delegate to other agents for specific tasks
- **Guardrails**, which enable validation of agent inputs and outputs
In combination with Python, these primitives are powerful enough to express complex relationships between tools and agents, and allow you to build real-world applications without a steep learning curve. In addition, the SDK comes with built-in **tracing** that lets you visualize and debug your agentic flows, as well as evaluate them and even fine-tune models for your application.
## Why use the Agents SDK
The SDK has two driving design principles:
1. Enough features to be worth using, but few enough primitives to make it quick to learn.
2. Works great out of the box, but you can customize exactly what happens.
Here are the main features of the SDK:
- **Agent loop**: A built-in agent loop that handles tool invocation, sends results back to the LLM, and continues until the task is complete.
- **Python-first**: Use built-in language features to orchestrate and chain agents, rather than needing to learn new abstractions.
- **Agents as tools / Handoffs**: A powerful mechanism for coordinating and delegating work across multiple agents.
- **Sandbox agents**: Run specialists inside real isolated workspaces with manifest-defined files, sandbox client choice, and resumable sandbox sessions.
- **Guardrails**: Run input validation and safety checks in parallel with agent execution, and fail fast when checks do not pass.
- **Function tools**: Turn any Python function into a tool with automatic schema generation and Pydantic-powered validation.
- **MCP server tool calling**: Built-in MCP server tool integration that works the same way as function tools.
- **Sessions**: A persistent memory layer for maintaining working context within an agent loop.
- **Human in the loop**: Built-in mechanisms for involving humans across agent runs.
- **Tracing**: Built-in tracing for visualizing, debugging, and monitoring workflows, with support for the OpenAI suite of evaluation, fine-tuning, and distillation tools.
- **Realtime Agents**: Build powerful voice agents with `gpt-realtime-2.1`, automatic interruption detection, context management, guardrails, and more.
## Agents SDK or Responses API?
The SDK uses the Responses API by default for OpenAI models, but it adds a higher-level runtime around model calls.
Use the Responses API directly when:
- you want to own the loop, tool dispatch, and state handling yourself
- your workflow is short-lived and mainly about returning the model's response
Use the Agents SDK when:
- you want the runtime to manage turns, tool execution, guardrails, handoffs, or sessions
- your agent should produce artifacts or operate across multiple coordinated steps
- you need a real workspace or resumable execution through [Sandbox agents](sandbox_agents.md)
You do not need to choose one globally. Many applications use the SDK for managed workflows and call the Responses API directly for lower-level paths.
## Installation
```bash
pip install openai-agents
```
## Hello world example
```python
from agents import Agent, Runner
agent = Agent(name="Assistant", instructions="You are a helpful assistant")
result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
print(result.final_output)
# Code within the code,
# Functions calling themselves,
# Infinite loop's dance.
```
(_If running this, ensure you set the `OPENAI_API_KEY` environment variable_)
```bash
export OPENAI_API_KEY=sk-...
```
## Start here
- Build your first text-based agent with the [Quickstart](quickstart.md).
- Then decide how you want to carry state across turns in [Running agents](running_agents.md#choose-a-memory-strategy).
- If the task depends on real files, repos, or isolated per-agent workspace state, read the [Sandbox agents quickstart](sandbox_agents.md).
- If you are deciding between handoffs and manager-style orchestration, read [Agent orchestration](multi_agent.md).
## Choose your path
Use this table when you know the job you want to do, but not which page explains it.
| Goal | Start here |
| --- | --- |
| Build the first text agent and see one complete run | [Quickstart](quickstart.md) |
| Add function tools, hosted tools, or agents as tools | [Tools](tools.md) |
| Run a coding, review, or document agent inside a real isolated workspace | [Sandbox agents quickstart](sandbox_agents.md) and [Sandbox clients](sandbox/clients.md) |
| Decide between handoffs and manager-style orchestration | [Agent orchestration](multi_agent.md) |
| Keep memory across turns | [Running agents](running_agents.md#choose-a-memory-strategy) and [Sessions](sessions/index.md) |
| Use OpenAI models, websocket transport, or non-OpenAI providers | [Models](models/index.md) |
| Review outputs, run items, interruptions, and resume state | [Results](results.md) |
| Build a low-latency voice agent with `gpt-realtime-2.1` | [Realtime agents quickstart](realtime/quickstart.md) and [Realtime transport](realtime/transport.md) |
| Build a speech-to-text / agent / text-to-speech pipeline | [Voice pipeline quickstart](voice/quickstart.md) |
+428
View File
@@ -0,0 +1,428 @@
---
search:
exclude: true
---
# エージェント
エージェントは、アプリの中核となる構成要素です。エージェントとは、instructions、ツール、およびハンドオフ、ガードレール、structured outputs などのオプションの実行時動作を設定した大規模言語モデル(LLM)です。
単一の標準的な `Agent` を定義またはカスタマイズする場合は、このページを参照してください。複数のエージェントをどのように連携させるかを検討している場合は、[エージェントオーケストレーション](multi_agent.md)を参照してください。マニフェストで定義されたファイルとサンドボックスネイティブの機能を備えた分離ワークスペース内でエージェントを実行する場合は、[サンドボックスエージェントの概念](sandbox/guide.md)を参照してください。
SDK は、OpenAI モデルに対してデフォルトで Responses API を使用しますが、ここでの違いはオーケストレーションにあります。`Agent``Runner` を組み合わせることで、SDK がターン、ツール、ガードレール、ハンドオフ、セッションを管理します。このループを自分で管理したい場合は、代わりに Responses API を直接使用してください。
## 次のガイドの選択
このページをエージェント定義のハブとして使用してください。次に行う必要がある判断に合った関連ガイドに進んでください。
| 目的 | 次に読むガイド |
| --- | --- |
| モデルまたはプロバイダーの設定を選択する | [モデル](models/index.md) |
| エージェントに機能を追加する | [ツール](tools.md) |
| 実際のリポジトリ、ドキュメント一式、または分離ワークスペースを対象にエージェントを実行する | [サンドボックスエージェントのクイックスタート](sandbox_agents.md) |
| マネージャー形式のオーケストレーションとハンドオフのどちらを使用するか決定する | [エージェントオーケストレーション](multi_agent.md) |
| ハンドオフの動作を設定する | [ハンドオフ](handoffs.md) |
| ターンの実行、イベントのストリーミング、または会話状態の管理を行う | [エージェントの実行](running_agents.md) |
| 最終出力、実行項目、または再開可能な状態を確認する | [実行結果](results.md) |
| ローカル依存関係と実行時状態を共有する | [コンテキスト管理](context.md) |
## 基本設定
エージェントで最も一般的なプロパティは次のとおりです。
| プロパティ | 必須 | 説明 |
| --- | --- | --- |
| `name` | はい | 人が理解しやすいエージェント名です。 |
| `instructions` | いいえ | システムプロンプトまたは動的 instructions コールバックです。使用を強く推奨します。[動的 instructions](#dynamic-instructions)を参照してください。 |
| `prompt` | いいえ | OpenAI Responses API のプロンプト設定です。静的なプロンプトオブジェクトまたは関数を受け取ります。[プロンプトテンプレート](#prompt-templates)を参照してください。 |
| `handoff_description` | いいえ | このエージェントがハンドオフ先として提示される際に公開される短い説明です。 |
| `handoffs` | いいえ | 会話を専門エージェントに委任します。[ハンドオフ](handoffs.md)を参照してください。 |
| `model` | いいえ | 使用する LLM です。[モデル](models/index.md)を参照してください。 |
| `model_settings` | いいえ | `temperature``top_p``tool_choice` などのモデル調整パラメーターです。 |
| `tools` | いいえ | エージェントが呼び出せるツールです。[ツール](tools.md)を参照してください。 |
| `mcp_servers` | いいえ | エージェント向けの MCP ベースのツールです。[MCP ガイド](mcp.md)を参照してください。 |
| `mcp_config` | いいえ | 厳密なスキーマ変換や MCP エラー形式など、MCP ツールの準備方法を詳細に調整します。[MCP ガイド](mcp.md#agent-level-mcp-configuration)を参照してください。 |
| `input_guardrails` | いいえ | このエージェントチェーンへの最初のユーザー入力に対して実行されるガードレールです。[ガードレール](guardrails.md)を参照してください。 |
| `output_guardrails` | いいえ | このエージェントの最終出力に対して実行されるガードレールです。[ガードレール](guardrails.md)を参照してください。 |
| `output_type` | いいえ | プレーンテキストの代わりに使用する構造化出力型です。[出力型](#output-types)を参照してください。 |
| `hooks` | いいえ | エージェントスコープのライフサイクルコールバックです。[ライフサイクルイベント(フック)](#lifecycle-events-hooks)を参照してください。 |
| `tool_use_behavior` | いいえ | ツールの実行結果をモデルに戻すか、実行を終了するかを制御します。[ツール使用時の動作](#tool-use-behavior)を参照してください。 |
| `reset_tool_choice` | いいえ | ツール使用ループを回避するため、ツール呼び出し後に `tool_choice` をリセットします(デフォルト:`True`)。[ツール使用の強制](#forcing-tool-use)を参照してください。 |
```python
from agents import Agent, ModelSettings, function_tool
@function_tool
def get_weather(city: str) -> str:
"""returns weather info for the specified city."""
return f"The weather in {city} is sunny"
agent = Agent(
name="Haiku agent",
instructions="Always respond in haiku form",
model="gpt-5-nano",
tools=[get_weather],
)
```
このセクションの内容はすべて `Agent` に適用されます。`SandboxAgent` は同じ概念を基盤とし、さらにワークスペーススコープの実行向けに `default_manifest``base_instructions``capabilities``run_as` を追加します。[サンドボックスエージェントの概念](sandbox/guide.md)を参照してください。
## プロンプトテンプレート
`prompt` を設定すると、OpenAI プラットフォームで作成したプロンプトテンプレートを参照できます。これは、Responses API を使用する OpenAI モデルで機能します。
使用するには、次の手順に従ってください。
1. https://platform.openai.com/playground/prompts に移動します。
2. 新しいプロンプト変数 `poem_style` を作成します。
3. 次の内容でシステムプロンプトを作成します。
```
Write a poem in {{poem_style}}
```
4. `--prompt-id` フラグを指定してコード例を実行します。
```python
from agents import Agent
agent = Agent(
name="Prompted assistant",
prompt={
"id": "pmpt_123",
"version": "1",
"variables": {"poem_style": "haiku"},
},
)
```
実行時にプロンプトを動的に生成することもできます。
```python
from dataclasses import dataclass
from agents import Agent, GenerateDynamicPromptData, Runner
@dataclass
class PromptContext:
prompt_id: str
poem_style: str
async def build_prompt(data: GenerateDynamicPromptData):
ctx: PromptContext = data.context.context
return {
"id": ctx.prompt_id,
"version": "1",
"variables": {"poem_style": ctx.poem_style},
}
agent = Agent(name="Prompted assistant", prompt=build_prompt)
result = await Runner.run(
agent,
"Say hello",
context=PromptContext(prompt_id="pmpt_123", poem_style="limerick"),
)
```
## コンテキスト
エージェントは `context` 型に対してジェネリックです。コンテキストは依存性注入のためのツールです。これは、作成して `Runner.run()` に渡すオブジェクトであり、すべてのエージェント、ツール、ハンドオフなどに渡され、エージェント実行に必要な依存関係と状態をまとめて保持します。任意の Python オブジェクトをコンテキストとして指定できます。
`RunContextWrapper` の全機能、共有使用量の追跡、ネストされた `tool_input`、およびシリアライズ時の注意事項については、[コンテキストガイド](context.md)を参照してください。
```python
@dataclass
class UserContext:
name: str
uid: str
is_pro_user: bool
async def fetch_purchases() -> list[Purchase]:
return ...
agent = Agent[UserContext](
...,
)
```
## 出力型
デフォルトでは、エージェントはプレーンテキスト(つまり `str`)の出力を生成します。エージェントに特定の型の出力を生成させる場合は、`output_type` パラメーターを使用できます。一般的には [Pydantic](https://docs.pydantic.dev/) オブジェクトを使用しますが、Pydantic の [TypeAdapter](https://docs.pydantic.dev/latest/api/type_adapter/) でラップできる任意の型(データクラス、リスト、TypedDict など)をサポートしています。
```python
from pydantic import BaseModel
from agents import Agent
class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
agent = Agent(
name="Calendar extractor",
instructions="Extract calendar events from text",
output_type=CalendarEvent,
)
```
!!! note
`output_type` を渡すと、通常のプレーンテキスト応答の代わりに [structured outputs](https://platform.openai.com/docs/guides/structured-outputs) を使用するようモデルに指示します。
## マルチエージェントシステムの設計パターン
マルチエージェントシステムにはさまざまな設計方法がありますが、一般的に広く適用できる次の 2 つのパターンが使用されます。
1. マネージャー(agents as tools):中央のマネージャー/オーケストレーターが、専門化されたサブエージェントをツールとして呼び出し、会話の制御を維持します。
2. ハンドオフ:対等なエージェントが、会話を引き継ぐ専門エージェントに制御をハンドオフします。これは分散型のパターンです。
詳細については、[エージェント構築の実践ガイド](https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf)を参照してください。
### マネージャー(agents as tools
`customer_facing_agent` はすべてのユーザー対応を処理し、ツールとして公開された専門化されたサブエージェントを呼び出します。詳細については、[ツール](tools.md#agents-as-tools)のドキュメントを参照してください。
```python
from agents import Agent
booking_agent = Agent(...)
refund_agent = Agent(...)
customer_facing_agent = Agent(
name="Customer-facing agent",
instructions=(
"Handle all direct user communication. "
"Call the relevant tools when specialized expertise is needed."
),
tools=[
booking_agent.as_tool(
tool_name="booking_expert",
tool_description="Handles booking questions and requests.",
),
refund_agent.as_tool(
tool_name="refund_expert",
tool_description="Handles refund questions and requests.",
)
],
)
```
### ハンドオフ
ハンドオフは、エージェントが処理を委任できるサブエージェントです。ハンドオフが発生すると、委任先のエージェントが会話履歴を受け取り、会話を引き継ぎます。このパターンにより、単一のタスクに特化したモジュール式の専門エージェントを実現できます。詳細については、[ハンドオフ](handoffs.md)のドキュメントを参照してください。
```python
from agents import Agent
booking_agent = Agent(...)
refund_agent = Agent(...)
triage_agent = Agent(
name="Triage agent",
instructions=(
"Help the user with their questions. "
"If they ask about booking, hand off to the booking agent. "
"If they ask about refunds, hand off to the refund agent."
),
handoffs=[booking_agent, refund_agent],
)
```
## 動的 instructions
ほとんどの場合、エージェントの作成時に instructions を指定できます。ただし、関数を介して動的な instructions を指定することもできます。この関数はエージェントとコンテキストを受け取り、プロンプトを返す必要があります。通常の関数と `async` 関数の両方を使用できます。
```python
def dynamic_instructions(
context: RunContextWrapper[UserContext], agent: Agent[UserContext]
) -> str:
return f"The user's name is {context.context.name}. Help them with their questions."
agent = Agent[UserContext](
name="Triage agent",
instructions=dynamic_instructions,
)
```
## ライフサイクルイベント(フック)
エージェントのライフサイクルを監視したい場合があります。たとえば、特定のイベントが発生した際に、イベントのログ記録、データの事前取得、または使用量の記録を行う場合があります。
フックには次の 2 つのスコープがあります。
- [`RunHooks`][agents.lifecycle.RunHooks] は、他のエージェントへのハンドオフを含む `Runner.run(...)` 呼び出し全体を監視します。
- [`AgentHooks`][agents.lifecycle.AgentHooks] は、`agent.hooks` を介して特定のエージェントインスタンスにアタッチされます。
コールバックのコンテキストも、イベントによって異なります。
- エージェントの開始/終了フックは [`AgentHookContext`][agents.run_context.AgentHookContext] を受け取ります。これは元のコンテキストをラップし、共有される実行使用量の状態を保持します。
- LLM、ツール、ハンドオフのフックは [`RunContextWrapper`][agents.run_context.RunContextWrapper] を受け取ります。
一般的なフックのタイミングは次のとおりです。
- `on_agent_start` / `on_agent_end`:特定のエージェントが最終出力の生成を開始または完了したとき。
- `on_llm_start` / `on_llm_end`:各モデル呼び出しの直前と直後。
- `on_tool_start` / `on_tool_end`:各ローカルツール呼び出しの前後。関数ツールの場合、フックの `context` は通常 `ToolContext` であるため、`tool_call_id` などのツール呼び出しメタデータを確認できます。
- `on_handoff`:制御があるエージェントから別のエージェントに移るとき。
ワークフロー全体を単一のオブザーバーで監視する場合は `RunHooks` を使用し、1 つのエージェントにカスタムの副作用が必要な場合は `AgentHooks` を使用してください。
```python
from agents import Agent, RunHooks, Runner
class LoggingHooks(RunHooks):
async def on_agent_start(self, context, agent):
print(f"Starting {agent.name}")
async def on_llm_end(self, context, agent, response):
print(f"{agent.name} produced {len(response.output)} output items")
async def on_agent_end(self, context, agent, output):
print(f"{agent.name} finished with usage: {context.usage}")
agent = Agent(name="Assistant", instructions="Be concise.")
result = await Runner.run(agent, "Explain quines", hooks=LoggingHooks())
print(result.final_output)
```
コールバックの全機能については、[ライフサイクル API リファレンス](ref/lifecycle.md)を参照してください。
## ガードレール
ガードレールを使用すると、エージェントの実行と並行してユーザー入力に対するチェック/検証を行い、エージェントの出力が生成された後にその出力をチェックできます。たとえば、ユーザー入力とエージェント出力が関連性のある内容かどうかを検査できます。詳細については、[ガードレール](guardrails.md)のドキュメントを参照してください。
## エージェントのクローン作成/コピー
エージェントの `clone()` メソッドを使用すると、Agent を複製し、必要に応じて任意のプロパティを変更できます。
```python
pirate_agent = Agent(
name="Pirate",
instructions="Write like a pirate",
model="gpt-5.6-sol",
)
robot_agent = pirate_agent.clone(
name="Robot",
instructions="Write like a robot",
)
```
## ツール使用の強制
ツールのリストを指定しても、LLM が必ずツールを使用するとは限りません。[`ModelSettings.tool_choice`][agents.model_settings.ModelSettings.tool_choice] を設定すると、ツールの使用を強制できます。有効な値は次のとおりです。
1. `auto`:ツールを使用するかどうかを LLM が判断できます。
2. `required`:LLM にツールの使用を要求します。ただし、どのツールを使用するかは LLM が適切に判断できます。
3. `none`:LLM にツールを _使用しない_ よう要求します。
4. `my_tool` などの特定の文字列:LLM にその特定のツールを使用するよう要求します。
OpenAI Responses のツール検索を使用する場合、名前付きツールの選択にはさらに制限があります。`tool_choice` では、単独の名前空間名や遅延専用ツールを指定できず、`tool_choice="tool_search"` で [`ToolSearchTool`][agents.tool.ToolSearchTool] を指定することもできません。このような場合は、`auto` または `required` を使用してください。Responses 固有の制約については、[ホスト型ツール検索](tools.md#hosted-tool-search)を参照してください。
```python
from agents import Agent, Runner, function_tool, ModelSettings
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
agent = Agent(
name="Weather Agent",
instructions="Retrieve weather details.",
tools=[get_weather],
model_settings=ModelSettings(tool_choice="get_weather")
)
```
## ツール使用時の動作
`Agent` 設定の `tool_use_behavior` パラメーターは、ツール出力の処理方法を制御します。
- `"run_llm_again"`:デフォルトです。ツールが実行され、その実行結果を LLM が処理して最終応答を生成します。
- `"stop_on_first_tool"`:最初のツール呼び出しの出力を、それ以降の LLM 処理を行わずに最終応答として使用します。
```python
from agents import Agent, Runner, function_tool, ModelSettings
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
agent = Agent(
name="Weather Agent",
instructions="Retrieve weather details.",
tools=[get_weather],
tool_use_behavior="stop_on_first_tool"
)
```
- `StopAtTools(stop_at_tool_names=[...])`:指定したツールのいずれかが呼び出された場合に停止し、その出力を最終応答として使用します。
```python
from agents import Agent, Runner, function_tool
from agents.agent import StopAtTools
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
@function_tool
def sum_numbers(a: int, b: int) -> int:
"""Adds two numbers."""
return a + b
agent = Agent(
name="Stop At Stock Agent",
instructions="Get weather or sum numbers.",
tools=[get_weather, sum_numbers],
tool_use_behavior=StopAtTools(stop_at_tool_names=["get_weather"])
)
```
- `ToolsToFinalOutputFunction`:ツールの実行結果を処理し、停止するか LLM での処理を続けるかを判断するカスタム関数です。
```python
from agents import Agent, Runner, function_tool, FunctionToolResult, RunContextWrapper
from agents.agent import ToolsToFinalOutputResult
from typing import List, Any
@function_tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
def custom_tool_handler(
context: RunContextWrapper[Any],
tool_results: List[FunctionToolResult]
) -> ToolsToFinalOutputResult:
"""Processes tool results to decide final output."""
for result in tool_results:
if result.output and "sunny" in result.output:
return ToolsToFinalOutputResult(
is_final_output=True,
final_output=f"Final weather: {result.output}"
)
return ToolsToFinalOutputResult(
is_final_output=False,
final_output=None
)
agent = Agent(
name="Weather Agent",
instructions="Retrieve weather details.",
tools=[get_weather],
tool_use_behavior=custom_tool_handler
)
```
!!! note
無限ループを防ぐため、フレームワークはツール呼び出し後に `tool_choice` を自動的に `"auto"` にリセットします。この動作は [`agent.reset_tool_choice`][agents.agent.Agent.reset_tool_choice] で設定できます。無限ループが発生するのは、ツールの実行結果が LLM に送信され、その後 `tool_choice` によって LLM が再びツール呼び出しを生成し、この処理が際限なく繰り返されるためです。
+205
View File
@@ -0,0 +1,205 @@
---
search:
exclude: true
---
# 設定
このページでは、デフォルトの OpenAI キーまたはクライアント、デフォルトの OpenAI API の形式、トレーシングエクスポートのデフォルト、ログ動作など、通常はアプリケーションの起動時に一度だけ設定する SDK 全体のデフォルトについて説明します。
これらのデフォルトはサンドボックスベースのワークフローにも引き続き適用されますが、サンドボックスワークスペース、サンドボックスクライアント、セッションの再利用は別途設定します。
代わりに特定のエージェントまたは実行を設定する必要がある場合は、まず次を参照してください:
- [エージェント](agents.md): 通常の `Agent` の instructions、tools、出力タイプ、ハンドオフ、ガードレールについて。
- [エージェントの実行](running_agents.md): `RunConfig`、セッション、会話状態オプションについて。
- [サンドボックスエージェント](sandbox/guide.md): `SandboxRunConfig`、マニフェスト、ケイパビリティ、サンドボックスクライアント固有のワークスペース設定について。
- [モデル](models/index.md): モデル選択とプロバイダー設定について。
- [トレーシング](tracing.md): 実行ごとのトレーシングメタデータとカスタムトレースプロセッサーについて。
## API キーとクライアント
デフォルトでは、SDK は LLM リクエストとトレーシングに `OPENAI_API_KEY` 環境変数を使用します。このキーは、SDK が初めて OpenAI クライアントを作成するときに解決されます(遅延初期化)。そのため、最初のモデル呼び出しの前に環境変数を設定してください。アプリの起動前にその環境変数を設定できない場合は、[set_default_openai_key()][agents.set_default_openai_key] 関数を使用してキーを設定できます。
```python
from agents import set_default_openai_key
set_default_openai_key("sk-...")
```
また、使用する OpenAI クライアントを設定することもできます。デフォルトでは、SDK は環境変数の API キー、または上で設定したデフォルトキーを使用して、`AsyncOpenAI` インスタンスを作成します。これは [set_default_openai_client()][agents.set_default_openai_client] 関数を使用して変更できます。
```python
from openai import AsyncOpenAI
from agents import set_default_openai_client
custom_client = AsyncOpenAI(base_url="...", api_key="...")
set_default_openai_client(custom_client)
```
環境変数ベースのエンドポイント設定を使用したい場合、デフォルトの OpenAI プロバイダーは `OPENAI_BASE_URL` も読み取ります。Responses WebSocket トランスポートを有効にすると、WebSocket の `/responses` エンドポイント用に `OPENAI_WEBSOCKET_BASE_URL` も読み取ります。
```bash
export OPENAI_BASE_URL="https://your-openai-compatible-endpoint.example/v1"
export OPENAI_WEBSOCKET_BASE_URL="wss://your-openai-compatible-endpoint.example/v1"
```
最後に、使用される OpenAI API もカスタマイズできます。デフォルトでは OpenAI Responses API を使用します。[set_default_openai_api()][agents.set_default_openai_api] 関数を使用すると、これを上書きして Chat Completions API を使用できます。
```python
from agents import set_default_openai_api
set_default_openai_api("chat_completions")
```
## OpenAI プロバイダーのデフォルト
OpenAI を基盤とするプロバイダーは、モデル名を解決するときにも SDK 全体のデフォルトを読み取ります。OpenAI Responses モデルがデフォルトで WebSocket トランスポートを使用するようにするには、[`set_default_openai_responses_transport()`][agents.set_default_openai_responses_transport] を使用します:
```python
from agents import set_default_openai_responses_transport
set_default_openai_responses_transport("websocket")
```
これは、デフォルトの OpenAI プロバイダーによって解決される OpenAI Responses モデルに影響します。プロバイダーレベルのセットアップ、接続の再利用、キープアライブオプション、カスタム WebSocket エンドポイントについては、[Responses WebSocket トランスポート](models/index.md#responses-websocket-transport) を参照してください。
OpenAI セットアップでプロバイダーレベルのエージェント登録メタデータが必要な場合は、起動時にデフォルトのハーネス ID を一度設定してください:
```python
from agents import set_default_openai_harness
set_default_openai_harness("your-harness-id")
```
完全な登録オブジェクトを渡すこともできます:
```python
from agents import OpenAIAgentRegistrationConfig, set_default_openai_agent_registration
set_default_openai_agent_registration(
OpenAIAgentRegistrationConfig(harness_id="your-harness-id")
)
```
SDK のデフォルトが設定されていない場合、OpenAI を基盤とするプロバイダーは `OPENAI_AGENT_HARNESS_ID` 環境変数にフォールバックします。ハーネス ID が設定されている場合、`agent_harness_id``RunConfig.trace_metadata` にすでに存在する場合を除き、SDK はそれを `agent_harness_id` としてトレースメタデータに追加します。
## トレーシング
トレーシングはデフォルトで有効です。デフォルトでは、上のセクションで説明したモデルリクエストと同じ OpenAI API キー(つまり、環境変数または設定したデフォルトキー)を使用します。トレーシングに使用する API キーは、[`set_tracing_export_api_key`][agents.set_tracing_export_api_key] 関数を使用して個別に設定できます。
```python
from agents import set_tracing_export_api_key
set_tracing_export_api_key("sk-...")
```
モデルのトラフィックではあるキーまたはクライアントを使用し、トレーシングでは別の OpenAI キーを使用したい場合は、デフォルトのキーまたはクライアントを設定するときに `use_for_tracing=False` を渡し、そのうえでトレーシングを個別に設定してください。カスタムクライアントを使用していない場合は、同じパターンを [`set_default_openai_key()`][agents.set_default_openai_key] にも適用できます。
```python
from openai import AsyncOpenAI
from agents import (
set_default_openai_client,
set_tracing_export_api_key,
)
custom_client = AsyncOpenAI(base_url="https://your-openai-compatible-endpoint.example/v1", api_key="provider-key")
set_default_openai_client(custom_client, use_for_tracing=False)
set_tracing_export_api_key("sk-tracing")
```
デフォルトエクスポーターを使用するときに、トレースを特定の組織またはプロジェクトに関連付ける必要がある場合は、アプリの起動前に次の環境変数を設定してください:
```bash
export OPENAI_ORG_ID="org_..."
export OPENAI_PROJECT_ID="proj_..."
```
グローバルエクスポーターを変更せずに、実行ごとにトレーシング API キーを設定することもできます。
```python
from agents import Runner, RunConfig
await Runner.run(
agent,
input="Hello",
run_config=RunConfig(tracing={"api_key": "sk-tracing-123"}),
)
```
[`set_tracing_disabled()`][agents.set_tracing_disabled] 関数を使用して、トレーシングを完全に無効化することもできます。
```python
from agents import set_tracing_disabled
set_tracing_disabled(True)
```
トレーシングを有効のままにしつつ、機微情報を含む可能性のある入力/出力をトレースペイロードから除外したい場合は、[`RunConfig.trace_include_sensitive_data`][agents.run.RunConfig.trace_include_sensitive_data] を `False` に設定してください:
```python
from agents import Runner, RunConfig
await Runner.run(
agent,
input="Hello",
run_config=RunConfig(trace_include_sensitive_data=False),
)
```
コードを使わずにデフォルトを変更することもできます。アプリの起動前にこの環境変数を設定してください:
```bash
export OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA=0
```
トレーシング制御の詳細については、[トレーシングガイド](tracing.md) を参照してください。
## デバッグログ
SDK は 2 つの Python ロガー(`openai.agents``openai.agents.tracing`)を定義しており、デフォルトではハンドラーをアタッチしません。ログは、アプリケーションの Python ロギング設定に従います。
詳細ログを有効にするには、[`enable_verbose_stdout_logging()`][agents.enable_verbose_stdout_logging] 関数を使用します。
```python
from agents import enable_verbose_stdout_logging
enable_verbose_stdout_logging()
```
また、ハンドラー、フィルター、フォーマッターなどを追加してログをカスタマイズすることもできます。詳しくは [Python ロギングガイド](https://docs.python.org/3/howto/logging.html) を参照してください。
```python
import logging
logger = logging.getLogger("openai.agents") # or openai.agents.tracing for the Tracing logger
# To make all logs show up
logger.setLevel(logging.DEBUG)
# To make info and above show up
logger.setLevel(logging.INFO)
# To make warning and above show up
logger.setLevel(logging.WARNING)
# etc
# You can customize this as needed, but this will output to `stderr` by default
logger.addHandler(logging.StreamHandler())
```
### ログ内の機微データ
一部のログには、機微データ(たとえば、ユーザーデータ)が含まれる場合があります。
デフォルトでは、SDK は LLM の入力/出力やツールの入力/出力を **ログに記録しません** 。これらの保護は次の項目で制御されます:
```bash
OPENAI_AGENTS_DONT_LOG_MODEL_DATA=1
OPENAI_AGENTS_DONT_LOG_TOOL_DATA=1
```
デバッグのためにこのデータを一時的に含める必要がある場合は、アプリの起動前にいずれかの変数を `0`(または `false`)に設定してください:
```bash
export OPENAI_AGENTS_DONT_LOG_MODEL_DATA=0
export OPENAI_AGENTS_DONT_LOG_TOOL_DATA=0
```
+148
View File
@@ -0,0 +1,148 @@
---
search:
exclude: true
---
# コンテキスト管理
コンテキストは多義的な用語です。考慮すべきコンテキストには、主に 2 つの種類があります:
1. コードからローカルに利用できるコンテキスト: これは、ツール関数の実行時、`on_handoff` のようなコールバック内、ライフサイクルフック内などで必要になる可能性のあるデータや依存関係です。
2. LLM が利用できるコンテキスト: これは、LLM が応答を生成するときに参照するデータです。
## ローカルコンテキスト
これは [`RunContextWrapper`][agents.run_context.RunContextWrapper] クラス、およびその中の [`context`][agents.run_context.RunContextWrapper.context] プロパティで表現されます。仕組みは次のとおりです:
1. 任意の Python オブジェクトを作成します。一般的なパターンは dataclass や Pydantic オブジェクトを使用することです。
2. そのオブジェクトを各種実行メソッドに渡します (例: `Runner.run(..., context=whatever)`)。
3. すべてのツール呼び出し、ライフサイクルフックなどには、ラッパーオブジェクト `RunContextWrapper[T]` が渡されます。ここで `T` はコンテキストオブジェクトの型を表し、`wrapper.context` を通じてアクセスできます。
一部のランタイム固有のコールバックでは、SDK はより特殊化された `RunContextWrapper[T]` のサブクラスを渡す場合があります。たとえば、関数ツールのライフサイクルフックは通常 `ToolContext` を受け取り、これは `tool_call_id``tool_name``tool_arguments` などのツール呼び出しメタデータも公開します。
認識すべき **最も重要な** 点は、あるエージェント実行におけるすべてのエージェント、ツール関数、ライフサイクルなどが、同じ __ のコンテキストを使用しなければならないということです。
コンテキストは、たとえば次の用途に使用できます:
- 実行時のコンテキストデータ (例: ユーザー名 / uid や、ユーザーに関するその他の情報)
- 依存関係 (例: ロガーオブジェクト、データ取得器など)
- ヘルパー関数
!!! danger "注記"
コンテキストオブジェクトは LLM に **送信されません**。これは完全にローカルなオブジェクトであり、読み取り、書き込み、メソッドの呼び出しができます。
1 回の実行内では、派生したラッパーは同じ基盤となるアプリコンテキスト、承認状態、使用状況の追跡を共有します。ネストされた [`Agent.as_tool()`][agents.agent.Agent.as_tool] の実行では異なる `tool_input` を付加する場合がありますが、デフォルトではアプリ状態の分離コピーは取得しません。
### `RunContextWrapper` の公開内容
[`RunContextWrapper`][agents.run_context.RunContextWrapper] は、アプリで定義したコンテキストオブジェクトを包むラッパーです。実際には、ほとんどの場合、次のものを使用します:
- [`wrapper.context`][agents.run_context.RunContextWrapper.context]: 独自の可変なアプリ状態と依存関係に使用します。
- [`wrapper.usage`][agents.run_context.RunContextWrapper.usage]: 現在の実行全体で集計されたリクエストおよびトークン使用量に使用します。
- [`wrapper.tool_input`][agents.run_context.RunContextWrapper.tool_input]: 現在の実行が [`Agent.as_tool()`][agents.agent.Agent.as_tool] の内部で実行されている場合の構造化入力に使用します。
- [`wrapper.approve_tool(...)`][agents.run_context.RunContextWrapper.approve_tool] / [`wrapper.reject_tool(...)`][agents.run_context.RunContextWrapper.reject_tool]: 承認状態をプログラムで更新する必要がある場合に使用します。
アプリで定義したオブジェクトは `wrapper.context` のみです。その他のフィールドは SDK が管理するランタイムメタデータです。
後で human-in-the-loop や耐久ジョブワークフローのために [`RunState`][agents.run_state.RunState] をシリアライズする場合、そのランタイムメタデータは状態とともに保存されます。シリアライズされた状態を永続化または送信する予定がある場合、[`RunContextWrapper.context`][agents.run_context.RunContextWrapper.context] にシークレットを入れないでください。
会話状態は別の関心事です。ターンをどのように引き継ぐかに応じて、`result.to_input_list()``session``conversation_id`、または `previous_response_id` を使用してください。その判断については、[実行結果](results.md)、[エージェントの実行](running_agents.md)、[セッション](sessions/index.md) を参照してください。
```python
import asyncio
from dataclasses import dataclass
from agents import Agent, RunContextWrapper, Runner, function_tool
@dataclass
class UserInfo: # (1)!
name: str
uid: int
@function_tool
async def fetch_user_age(wrapper: RunContextWrapper[UserInfo]) -> str: # (2)!
"""Fetch the age of the user. Call this function to get user's age information."""
return f"The user {wrapper.context.name} is 47 years old"
async def main():
user_info = UserInfo(name="John", uid=123)
agent = Agent[UserInfo]( # (3)!
name="Assistant",
tools=[fetch_user_age],
)
result = await Runner.run( # (4)!
starting_agent=agent,
input="What is the age of the user?",
context=user_info,
)
print(result.final_output) # (5)!
# The user John is 47 years old.
if __name__ == "__main__":
asyncio.run(main())
```
1. これがコンテキストオブジェクトです。ここでは dataclass を使用していますが、任意の型を使用できます。
2. これはツールです。`RunContextWrapper[UserInfo]` を受け取っていることがわかります。ツール実装はコンテキストから読み取ります。
3. 型チェッカーがエラーを検出できるように、エージェントにジェネリック `UserInfo` を指定します (たとえば、異なるコンテキスト型を受け取るツールを渡そうとした場合)。
4. コンテキストは `run` 関数に渡されます。
5. エージェントは正しくツールを呼び出し、年齢を取得します。
---
### 高度な内容: `ToolContext`
場合によっては、実行中のツールに関する追加メタデータ (名前、呼び出し ID、生の引数文字列など) にアクセスしたいことがあります。
この場合、`RunContextWrapper` を拡張する [`ToolContext`][agents.tool_context.ToolContext] クラスを使用できます。
```python
from typing import Annotated
from pydantic import BaseModel, Field
from agents import Agent, Runner, function_tool
from agents.tool_context import ToolContext
class WeatherContext(BaseModel):
user_id: str
class Weather(BaseModel):
city: str = Field(description="The city name")
temperature_range: str = Field(description="The temperature range in Celsius")
conditions: str = Field(description="The weather conditions")
@function_tool
def get_weather(ctx: ToolContext[WeatherContext], city: Annotated[str, "The city to get the weather for"]) -> Weather:
print(f"[debug] Tool context: (name: {ctx.tool_name}, call_id: {ctx.tool_call_id}, args: {ctx.tool_arguments})")
return Weather(city=city, temperature_range="14-20C", conditions="Sunny with wind.")
agent = Agent(
name="Weather Agent",
instructions="You are a helpful agent that can tell the weather of a given city.",
tools=[get_weather],
)
```
`ToolContext``RunContextWrapper` と同じ `.context` プロパティを提供し、
現在のツール呼び出しに固有の追加フィールドも提供します:
- `tool_name` – 呼び出されているツールの名前
- `tool_call_id` – このツール呼び出しの一意の識別子
- `tool_arguments` – ツールに渡された生の引数文字列
- `tool_namespace` ツールが `tool_namespace()` または別の名前空間付きサーフェスを通じて読み込まれた場合の、ツール呼び出しに対する Responses 名前空間
- `qualified_tool_name` – 名前空間が利用できる場合に、その名前空間で修飾されたツール名
実行中にツールレベルのメタデータが必要な場合は、`ToolContext` を使用してください。
エージェントとツール間で一般的なコンテキスト共有を行うには、`RunContextWrapper` のままで十分です。`ToolContext``RunContextWrapper` を拡張しているため、ネストされた `Agent.as_tool()` 実行が構造化入力を提供した場合には `.tool_input` も公開できます。
---
## エージェント / LLM コンテキスト
LLM が呼び出されるとき、その LLM が参照できる **唯一の** データは会話履歴に含まれるものです。つまり、新しいデータを LLM に利用可能にしたい場合は、その履歴内で利用可能になるような方法で行う必要があります。これにはいくつかの方法があります:
1. エージェントの `instructions` に追加できます。これは「システムプロンプト」または「開発者メッセージ」とも呼ばれます。システムプロンプトは静的文字列にも、コンテキストを受け取って文字列を出力する動的関数にもできます。これは、常に役立つ情報 (たとえば、ユーザーの名前や現在の日付) に対する一般的な手法です。
2. `Runner.run` 関数を呼び出すときに `input` に追加します。これは `instructions` の手法に似ていますが、[指揮系統](https://cdn.openai.com/spec/model-spec-2024-05-08.html#follow-the-chain-of-command) においてより下位のメッセージにできます。
3. 関数ツールを介して公開します。これは _オンデマンド_ のコンテキストに便利です。LLM がデータを必要とするタイミングを判断し、そのデータを取得するためにツールを呼び出せます。
4. リトリーバルまたは Web 検索を使用します。これらは、ファイルやデータベースから関連データを取得する (リトリーバル)、または Web から取得する (Web 検索) ことができる特殊なツールです。これは、関連するコンテキストデータに基づいて応答を「グラウンディング」するのに便利です。
+136
View File
@@ -0,0 +1,136 @@
---
search:
exclude: true
---
# コード例
[リポジトリ](https://github.com/openai/openai-agents-python/tree/main/examples) のコード例セクションで、SDK のさまざまな実装サンプルをご覧ください。コード例は複数のカテゴリーに整理され、それぞれ異なるパターンと機能を示します。
## カテゴリー
- **[agent_patterns](https://github.com/openai/openai-agents-python/tree/main/examples/agent_patterns):** このカテゴリーのコード例では、次のような一般的なエージェント設計パターンを示します。
- 決定論的ワークフロー
- Agents as tools
- ストリーミングイベントを使用する Agents as tools (`examples/agent_patterns/agents_as_tools_streaming.py`)
- 構造化入力パラメーターを使用する Agents as tools (`examples/agent_patterns/agents_as_tools_structured.py`)
- エージェントの並列実行
- 条件付きのツール使用
- 異なる動作でのツール使用の強制 (`examples/agent_patterns/forcing_tool_use.py`)
- 入出力ガードレール
- 判定役としての LLM
- ルーティング
- ストリーミングガードレール
- ツール承認と状態のシリアル化を伴う人間参加型フロー (`examples/agent_patterns/human_in_the_loop.py`)
- ストリーミングを伴う人間参加型フロー (`examples/agent_patterns/human_in_the_loop_stream.py`)
- 承認フロー向けのカスタム拒否メッセージ (`examples/agent_patterns/human_in_the_loop_custom_rejection.py`)
- **[basic](https://github.com/openai/openai-agents-python/tree/main/examples/basic):** これらのコード例では、次のような SDK の基本機能を紹介します。
- Hello world のコード例(デフォルトモデル、GPT-5、オープンウェイトモデル)
- エージェントのライフサイクル管理
- 実行フックとエージェントフックのライフサイクルのコード例 (`examples/basic/lifecycle_example.py`)
- 動的システムプロンプト
- 基本的なツールの使用 (`examples/basic/tools.py`)
- ツールの入出力ガードレール (`examples/basic/tool_guardrails.py`)
- 画像形式のツール出力 (`examples/basic/image_tool_output.py`)
- ストリーミング出力(テキスト、アイテム、関数呼び出しの引数)
- ターン間で共有されるセッションヘルパーを使用する Responses WebSocket トランスポート (`examples/basic/stream_ws.py`)
- プロンプトテンプレート
- ファイル処理(ローカルおよびリモート、画像および PDF)
- 使用量の追跡
- Runner が管理する再試行設定 (`examples/basic/retry.py`)
- サードパーティー製アダプターを介して Runner が管理する再試行 (`examples/basic/retry_litellm.py`)
- 厳密でない出力型
- 以前のレスポンス ID の使用
- **[customer_service](https://github.com/openai/openai-agents-python/tree/main/examples/customer_service):** 航空会社向けカスタマーサービスシステムのコード例です。
- **[financial_research_agent](https://github.com/openai/openai-agents-python/tree/main/examples/financial_research_agent):** エージェントとツールを使用した、金融データ分析向けの構造化された調査ワークフローを示す金融調査エージェントです。
- **[handoffs](https://github.com/openai/openai-agents-python/tree/main/examples/handoffs):** メッセージフィルタリングを伴うエージェントのハンドオフの実践的なコード例です。以下が含まれます。
- メッセージフィルターのコード例 (`examples/handoffs/message_filter.py`)
- ストリーミングを伴うメッセージフィルター (`examples/handoffs/message_filter_streaming.py`)
- **[hosted_mcp](https://github.com/openai/openai-agents-python/tree/main/examples/hosted_mcp):** OpenAI Responses API でホスト型 MCP (Model Context Protocol) を使用する方法を示すコード例です。以下が含まれます。
- 承認不要のシンプルなホスト型 MCP (`examples/hosted_mcp/simple.py`)
- Google Calendar などの MCP コネクター (`examples/hosted_mcp/connectors.py`)
- 中断ベースの承認を使用する人間参加型フロー (`examples/hosted_mcp/human_in_the_loop.py`)
- MCP ツール呼び出しの承認時コールバック (`examples/hosted_mcp/on_approval.py`)
- **[mcp](https://github.com/openai/openai-agents-python/tree/main/examples/mcp):** MCP (Model Context Protocol) を使用してエージェントを構築する方法を学びます。以下が含まれます。
- ファイルシステムのコード例
- Git のコード例
- MCP プロンプトサーバーのコード例
- SSE (Server-Sent Events) のコード例
- SSE リモートサーバー接続 (`examples/mcp/sse_remote_example`)
- Streamable HTTP のコード例
- Streamable HTTP リモート接続 (`examples/mcp/streamable_http_remote_example`)
- Streamable HTTP 向けのカスタム HTTP クライアントファクトリー (`examples/mcp/streamablehttp_custom_client_example`)
- `MCPUtil.get_all_function_tools` を使用したすべての MCP ツールの事前取得 (`examples/mcp/get_all_mcp_tools_example`)
- FastAPI と組み合わせた MCPServerManager (`examples/mcp/manager_example`)
- MCP ツールのフィルタリング (`examples/mcp/tool_filter_example`)
- **[memory](https://github.com/openai/openai-agents-python/tree/main/examples/memory):** エージェント向けのさまざまなメモリ実装のコード例です。以下が含まれます。
- SQLite セッションストレージ
- 高度な SQLite セッションストレージ
- Redis セッションストレージ
- SQLAlchemy セッションストレージ
- Dapr ステートストアのセッションストレージ
- 暗号化されたセッションストレージ
- OpenAI Conversations セッションストレージ
- Responses 圧縮セッションストレージ
- `ModelSettings(store=False)` を使用したステートレスな Responses 圧縮 (`examples/memory/compaction_session_stateless_example.py`)
- ファイルベースのセッションストレージ (`examples/memory/file_session.py`)
- 人間参加型フローを伴うファイルベースのセッション (`examples/memory/file_hitl_example.py`)
- 人間参加型フローを伴う SQLite インメモリセッション (`examples/memory/memory_session_hitl_example.py`)
- 人間参加型フローを伴う OpenAI Conversations セッション (`examples/memory/openai_session_hitl_example.py`)
- セッションをまたぐ HITL の承認/拒否シナリオ (`examples/memory/hitl_session_scenario.py`)
- **[model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers):** カスタムプロバイダーやサードパーティー製アダプターなど、OpenAI 以外のモデルを SDK で使用する方法を紹介します。
- **[realtime](https://github.com/openai/openai-agents-python/tree/main/examples/realtime):** SDK を使用してリアルタイム体験を構築する方法を示すコード例です。以下が含まれます。
- 構造化されたテキストメッセージと画像メッセージを使用する Web アプリケーションパターン
- コマンドラインでの音声ループと再生処理
- WebSocket を介した Twilio Media Streams 統合
- Realtime Calls API のアタッチフローを使用した Twilio SIP 統合
- **[reasoning_content](https://github.com/openai/openai-agents-python/tree/main/examples/reasoning_content):** 推論コンテンツの扱い方を示すコード例です。以下が含まれます。
- Runner API を使用した推論コンテンツ、ストリーミングおよび非ストリーミング (`examples/reasoning_content/runner_example.py`)
- OpenRouter 経由の OSS モデルを使用した推論コンテンツ (`examples/reasoning_content/gpt_oss_stream.py`)
- 基本的な推論コンテンツのコード例 (`examples/reasoning_content/main.py`)
- **[research_bot](https://github.com/openai/openai-agents-python/tree/main/examples/research_bot):** 複雑なマルチエージェント調査ワークフローを示す、シンプルなディープリサーチのクローンです。
- **[sandbox](https://github.com/openai/openai-agents-python/tree/main/examples/sandbox):** 分離されたワークスペースでエージェントを実行するためのコード例です。以下が含まれます。
- 基本的なサンドボックスエージェントのセットアップ (`examples/sandbox/basic.py`)
- Unix ローカルおよび Docker サンドボックスのライフサイクルのコード例
- サンドボックスを利用したハンドオフ (`examples/sandbox/handoffs.py`)
- サンドボックスのメモリとスナップショットからの再開 (`examples/sandbox/memory.py`)
- ツールとして公開されるサンドボックスエージェント (`examples/sandbox/sandbox_agents_as_tools.py`)
- **[tools](https://github.com/openai/openai-agents-python/tree/main/examples/tools):** OpenAI がホストするツールや実験的な Codex ツール機能の実装方法を学びます。以下が含まれます。
- Web 検索、およびフィルター付き Web 検索
- ファイル検索
- Code interpreter
- ファイル編集と承認を伴うパッチ適用ツール (`examples/tools/apply_patch.py`)
- 承認コールバックを伴うシェルツールの実行 (`examples/tools/shell.py`)
- 中断ベースの人間参加型承認を伴うシェルツール (`examples/tools/shell_human_in_the_loop.py`)
- インラインスキルを備えたホスト型コンテナシェル (`examples/tools/container_shell_inline_skill.py`)
- スキル参照を備えたホスト型コンテナシェル (`examples/tools/container_shell_skill_reference.py`)
- ローカルスキルを備えたローカルシェル (`examples/tools/local_shell_skill.py`)
- 名前空間と遅延ツールを使用したツール検索 (`examples/tools/tool_search.py`)
- コンピュータ操作
- 画像生成
- 実験的な Codex ツールワークフロー (`examples/tools/codex.py`)
- 実験的な Codex の同一スレッドワークフロー (`examples/tools/codex_same_thread.py`)
- **[voice](https://github.com/openai/openai-agents-python/tree/main/examples/voice):** TTS および STT モデルを使用した音声エージェントのコード例をご覧ください。ストリーミング音声のコード例も含まれます。
+234
View File
@@ -0,0 +1,234 @@
---
search:
exclude: true
---
# ガードレール
ガードレールを使用すると、ユーザー入力とエージェント出力のチェックおよび検証を行えます。たとえば、顧客からのリクエストを支援するために非常に賢い(そのため低速で高コストな)モデルを使用するエージェントがあるとします。悪意のあるユーザーに、そのモデルへ数学の宿題を手伝わせたくはないはずです。そこで、高速 / 低コストなモデルでガードレールを実行できます。ガードレールが悪意のある使用を検出した場合、即座にエラーを送出して高価なモデルの実行を防ぎ、時間と費用を節約できます( **ブロッキングガードレールを使用する場合に限ります。並列ガードレールでは、ガードレールが完了する前に、高価なモデルがすでに実行を開始している可能性があります。詳細は下記の「実行モード」を参照してください** )。
ガードレールには 2 種類あります。
1. 入力ガードレールは、最初のユーザー入力に対して実行されます。
2. 出力ガードレールは、最終的なエージェント出力に対して実行されます。
## ワークフローの境界
ガードレールはエージェントとツールにアタッチされますが、すべてがワークフロー内の同じ時点で実行されるわけではありません。
- **入力ガードレール** は、チェーン内の最初のエージェントに対してのみ実行されます。
- **出力ガードレール** は、最終出力を生成するエージェントに対してのみ実行されます。
- **ツールガードレール** は、すべてのカスタム関数ツール呼び出しで実行され、実行前に入力ガードレール、実行後に出力ガードレールが実行されます。
マネージャー、ハンドオフ、または委任先の専門家を含むワークフローで、各カスタム関数ツール呼び出しの前後にチェックが必要な場合は、エージェントレベルの入力 / 出力ガードレールだけに頼るのではなく、ツールガードレールを使用してください。
## 入力ガードレール
入力ガードレールは 3 ステップで実行されます。
1. まず、ガードレールはエージェントに渡されたものと同じ入力を受け取ります。
2. 次に、ガードレール関数が実行され、 [`GuardrailFunctionOutput`][agents.guardrail.GuardrailFunctionOutput] が生成されます。これはその後 [`InputGuardrailResult`][agents.guardrail.InputGuardrailResult] にラップされます。
3. 最後に、 [`.tripwire_triggered`][agents.guardrail.GuardrailFunctionOutput.tripwire_triggered] が true かどうかを確認します。true の場合、 [`InputGuardrailTripwireTriggered`][agents.exceptions.InputGuardrailTripwireTriggered] 例外が送出されるため、ユーザーに適切に応答したり、例外を処理したりできます。
!!! Note
入力ガードレールはユーザー入力に対して実行されることを想定しているため、エージェントのガードレールは、そのエージェントが *最初の* エージェントである場合にのみ実行されます。なぜ `guardrails` プロパティが `Runner.run` に渡されるのではなく、エージェント上にあるのか疑問に思うかもしれません。これは、ガードレールが実際のエージェントに関連していることが多いためです。エージェントごとに異なるガードレールを実行するため、コードを同じ場所に配置しておくと可読性の面で役立ちます。
### 実行モード
入力ガードレールは 2 つの実行モードをサポートします。
- **並列実行** (デフォルト、 `run_in_parallel=True` ): ガードレールはエージェントの実行と並行して実行されます。両方が同時に開始されるため、レイテンシが最も良くなります。ただし、ガードレールが失敗した場合、キャンセルされる前にエージェントがすでにトークンを消費し、ツールを実行している可能性があります。
- **ブロッキング実行** `run_in_parallel=False` ): ガードレールはエージェントが開始する *前に* 実行され、完了します。ガードレールのトリップワイヤーが発火した場合、エージェントは一切実行されないため、トークン消費とツール実行を防げます。これは、コスト最適化や、ツール呼び出しによる潜在的な副作用を避けたい場合に最適です。
## 出力ガードレール
出力ガードレールは 3 ステップで実行されます。
1. まず、ガードレールはエージェントによって生成された出力を受け取ります。
2. 次に、ガードレール関数が実行され、 [`GuardrailFunctionOutput`][agents.guardrail.GuardrailFunctionOutput] が生成されます。これはその後 [`OutputGuardrailResult`][agents.guardrail.OutputGuardrailResult] にラップされます。
3. 最後に、 [`.tripwire_triggered`][agents.guardrail.GuardrailFunctionOutput.tripwire_triggered] が true かどうかを確認します。true の場合、 [`OutputGuardrailTripwireTriggered`][agents.exceptions.OutputGuardrailTripwireTriggered] 例外が送出されるため、ユーザーに適切に応答したり、例外を処理したりできます。
!!! Note
出力ガードレールは最終的なエージェント出力に対して実行されることを想定しているため、エージェントのガードレールは、そのエージェントが *最後の* エージェントである場合にのみ実行されます。入力ガードレールと同様に、これはガードレールが実際のエージェントに関連していることが多いためです。エージェントごとに異なるガードレールを実行するため、コードを同じ場所に配置しておくと可読性の面で役立ちます。
出力ガードレールは必ずエージェントの完了後に実行されるため、 `run_in_parallel` パラメーターはサポートしません。
## ツールガードレール
ツールガードレールは **関数ツール** をラップし、実行の前後でツール呼び出しを検証またはブロックできるようにします。これはツール自体に設定され、そのツールが呼び出されるたびに実行されます。
- 入力ツールガードレールはツール実行前に実行され、呼び出しをスキップしたり、出力をメッセージに置き換えたり、トリップワイヤーを送出したりできます。
- 出力ツールガードレールはツール実行後に実行され、出力を置き換えたり、トリップワイヤーを送出したりできます。
- 関数ツールに承認が必要な場合、入力ツールガードレールは通常、承認後かつ実行直前に実行されます。保留中の承認割り込みが発行される前にこれらの入力チェックを実行したい場合は、 [`RunConfig.tool_execution`][agents.run.RunConfig.tool_execution] を [`ToolExecutionConfig(pre_approval_tool_input_guardrails=True)`][agents.run.ToolExecutionConfig] に設定してください。この承認前チェックに合格した呼び出しも、ツールが実行される前に、承認後に再度チェックされます。
- ツールガードレールは、 [`function_tool`][agents.tool.function_tool] で作成された関数ツールにのみ適用されます。ハンドオフは通常の関数ツールパイプラインではなく、 SDK のハンドオフパイプラインを通じて実行されるため、ツールガードレールはハンドオフ呼び出し自体には適用されません。ホスト型ツール( `WebSearchTool``FileSearchTool``HostedMCPTool``CodeInterpreterTool``ImageGenerationTool` )と組み込み実行ツール( `ComputerTool``ShellTool``ApplyPatchTool``LocalShellTool` )もこのガードレールパイプラインを使用しません。また、 [`Agent.as_tool()`][agents.agent.Agent.as_tool] は現在、ツールガードレールのオプションを直接公開していません。
詳細は、下記のコードスニペットを参照してください。
## トリップワイヤー
入力または出力がガードレールに合格しなかった場合、ガードレールはトリップワイヤーでこれを知らせることができます。トリップワイヤーが発火したガードレールを検出した時点で、即座に `{Input,Output}GuardrailTripwireTriggered` 例外を送出し、エージェントの実行を停止します。
## ガードレールの実装
入力を受け取り、 [`GuardrailFunctionOutput`][agents.guardrail.GuardrailFunctionOutput] を返す関数を用意する必要があります。この例では、内部でエージェントを実行することでこれを行います。
```python
from pydantic import BaseModel
from agents import (
Agent,
GuardrailFunctionOutput,
InputGuardrailTripwireTriggered,
RunContextWrapper,
Runner,
TResponseInputItem,
input_guardrail,
)
class MathHomeworkOutput(BaseModel):
is_math_homework: bool
reasoning: str
guardrail_agent = Agent( # (1)!
name="Guardrail check",
instructions="Check if the user is asking you to do their math homework.",
output_type=MathHomeworkOutput,
)
@input_guardrail
async def math_guardrail( # (2)!
ctx: RunContextWrapper[None], agent: Agent, input: str | list[TResponseInputItem]
) -> GuardrailFunctionOutput:
result = await Runner.run(guardrail_agent, input, context=ctx.context)
return GuardrailFunctionOutput(
output_info=result.final_output, # (3)!
tripwire_triggered=result.final_output.is_math_homework,
)
agent = Agent( # (4)!
name="Customer support agent",
instructions="You are a customer support agent. You help customers with their questions.",
input_guardrails=[math_guardrail],
)
async def main():
# This should trip the guardrail
try:
await Runner.run(agent, "Hello, can you help me solve for x: 2x + 3 = 11?")
print("Guardrail didn't trip - this is unexpected")
except InputGuardrailTripwireTriggered:
print("Math homework guardrail tripped")
```
1. このエージェントをガードレール関数で使用します。
2. これは、エージェントの入力 / コンテキストを受け取り、実行結果を返すガードレール関数です。
3. ガードレールの実行結果に追加情報を含めることができます。
4. これは、ワークフローを定義する実際のエージェントです。
出力ガードレールも同様です。
```python
from pydantic import BaseModel
from agents import (
Agent,
GuardrailFunctionOutput,
OutputGuardrailTripwireTriggered,
RunContextWrapper,
Runner,
output_guardrail,
)
class MessageOutput(BaseModel): # (1)!
response: str
class MathOutput(BaseModel): # (2)!
reasoning: str
is_math: bool
guardrail_agent = Agent(
name="Guardrail check",
instructions="Check if the output includes any math.",
output_type=MathOutput,
)
@output_guardrail
async def math_guardrail( # (3)!
ctx: RunContextWrapper, agent: Agent, output: MessageOutput
) -> GuardrailFunctionOutput:
result = await Runner.run(guardrail_agent, output.response, context=ctx.context)
return GuardrailFunctionOutput(
output_info=result.final_output,
tripwire_triggered=result.final_output.is_math,
)
agent = Agent( # (4)!
name="Customer support agent",
instructions="You are a customer support agent. You help customers with their questions.",
output_guardrails=[math_guardrail],
output_type=MessageOutput,
)
async def main():
# This should trip the guardrail
try:
await Runner.run(agent, "Hello, can you help me solve for x: 2x + 3 = 11?")
print("Guardrail didn't trip - this is unexpected")
except OutputGuardrailTripwireTriggered:
print("Math output guardrail tripped")
```
1. これは、実際のエージェントの出力型です。
2. これは、ガードレールの出力型です。
3. これは、エージェントの出力を受け取り、実行結果を返すガードレール関数です。
4. これは、ワークフローを定義する実際のエージェントです。
最後に、ツールガードレールのコード例を示します。
```python
import json
from agents import (
Agent,
Runner,
ToolGuardrailFunctionOutput,
function_tool,
tool_input_guardrail,
tool_output_guardrail,
)
@tool_input_guardrail
def block_secrets(data):
args = json.loads(data.context.tool_arguments or "{}")
if "sk-" in json.dumps(args):
return ToolGuardrailFunctionOutput.reject_content(
"Remove secrets before calling this tool."
)
return ToolGuardrailFunctionOutput.allow()
@tool_output_guardrail
def redact_output(data):
text = str(data.output or "")
if "sk-" in text:
return ToolGuardrailFunctionOutput.reject_content("Output contained sensitive data.")
return ToolGuardrailFunctionOutput.allow()
@function_tool(
tool_input_guardrails=[block_secrets],
tool_output_guardrails=[redact_output],
)
def classify_text(text: str) -> str:
"""Classify text for internal routing."""
return f"length:{len(text)}"
agent = Agent(name="Classifier", tools=[classify_text])
result = Runner.run_sync(agent, "hello world")
print(result.final_output)
```
+156
View File
@@ -0,0 +1,156 @@
---
search:
exclude: true
---
# ハンドオフ
ハンドオフにより、エージェントはタスクを別のエージェントに委任できます。これは、異なるエージェントがそれぞれ別の領域を専門とするシナリオで特に役立ちます。たとえば、カスタマーサポートアプリには、注文ステータス、返金、 FAQ などのタスクをそれぞれ専門に扱うエージェントがあるかもしれません。
ハンドオフは LLM に対してツールとして表現されます。そのため、 `Refund Agent` という名前のエージェントへのハンドオフがある場合、そのツールは `transfer_to_refund_agent` と呼ばれます。
## ハンドオフの作成
すべてのエージェントには [`handoffs`][agents.agent.Agent.handoffs] パラメーターがあり、 `Agent` を直接受け取ることも、ハンドオフをカスタマイズする `Handoff` オブジェクトを受け取ることもできます。
通常の `Agent` インスタンスを渡す場合、その [`handoff_description`][agents.agent.Agent.handoff_description] (設定されている場合)がデフォルトのツール説明に追加されます。完全な `handoff()` オブジェクトを書かずに、そのハンドオフをモデルが選ぶべきタイミングを示唆するために使用してください。
Agents SDK が提供する [`handoff()`][agents.handoffs.handoff] 関数を使用してハンドオフを作成できます。この関数では、必要に応じた上書きや入力フィルターとともに、引き渡し先のエージェントを指定できます。
### 基本的な使用法
シンプルなハンドオフを作成する方法は次のとおりです。
```python
from agents import Agent, handoff
billing_agent = Agent(name="Billing agent")
refund_agent = Agent(name="Refund agent")
# (1)!
triage_agent = Agent(name="Triage agent", handoffs=[billing_agent, handoff(refund_agent)])
```
1. エージェントを直接使用することも( `billing_agent` のように)、 `handoff()` 関数を使用することもできます。
### `handoff()` 関数によるハンドオフのカスタマイズ
[`handoff()`][agents.handoffs.handoff] 関数を使用すると、さまざまな項目をカスタマイズできます。
- `agent`: 処理を引き渡す先のエージェントです。
- `tool_name_override`: デフォルトでは `Handoff.default_tool_name()` 関数が使用され、 `transfer_to_<agent_name>` に解決されます。これは上書きできます。
- `tool_description_override`: `Handoff.default_tool_description()` から得られるデフォルトのツール説明を上書きします。
- `on_handoff`: ハンドオフが呼び出されたときに実行されるコールバック関数です。ハンドオフが呼び出されることが分かった時点ですぐにデータ取得を開始する、といった用途に便利です。この関数はエージェントコンテキストを受け取り、任意で LLM が生成した入力も受け取れます。入力データは `input_type` パラメーターによって制御されます。
- `input_type`: ハンドオフツール呼び出し引数のスキーマです。設定されている場合、解析されたペイロードが `on_handoff` に渡されます。
- `input_filter`: これにより、次のエージェントが受け取る入力をフィルタリングできます。詳細は以下を参照してください。
- `is_enabled`: ハンドオフが有効かどうかです。これはブール値、またはブール値を返す関数にでき、実行時にハンドオフを動的に有効化または無効化できます。
- `nest_handoff_history`: RunConfig レベルの `nest_handoff_history` 設定に対する、呼び出しごとの任意の上書きです。 `None` の場合は、アクティブな実行設定で定義された値が代わりに使用されます。
[`handoff()`][agents.handoffs.handoff] ヘルパーは、渡された特定の `agent` に常に制御を移します。複数の宛先候補がある場合は、宛先ごとに 1 つのハンドオフを登録し、モデルにその中から選ばせてください。独自のハンドオフコードが呼び出し時にどのエージェントを返すかを決定する必要がある場合にのみ、カスタム [`Handoff`][agents.handoffs.Handoff] を使用してください。
```python
from agents import Agent, handoff, RunContextWrapper
def on_handoff(ctx: RunContextWrapper[None]):
print("Handoff called")
agent = Agent(name="My agent")
handoff_obj = handoff(
agent=agent,
on_handoff=on_handoff,
tool_name_override="custom_handoff_tool",
tool_description_override="Custom description",
)
```
## ハンドオフ入力
状況によっては、 LLM がハンドオフを呼び出すときに何らかのデータを提供してほしい場合があります。たとえば、「エスカレーションエージェント」へのハンドオフを想像してみてください。ログに記録できるように、モデルに理由を提供してほしい場合があります。
```python
from pydantic import BaseModel
from agents import Agent, handoff, RunContextWrapper
class EscalationData(BaseModel):
reason: str
async def on_handoff(ctx: RunContextWrapper[None], input_data: EscalationData):
print(f"Escalation agent called with reason: {input_data.reason}")
agent = Agent(name="Escalation agent")
handoff_obj = handoff(
agent=agent,
on_handoff=on_handoff,
input_type=EscalationData,
)
```
`input_type` は、ハンドオフツール呼び出し自体の引数を表します。 SDK はそのスキーマをハンドオフツールの `parameters` としてモデルに公開し、返された JSON をローカルで検証して、解析済みの値を `on_handoff` に渡します。
これは次のエージェントのメイン入力を置き換えるものではなく、別の宛先を選択するものでもありません。 [`handoff()`][agents.handoffs.handoff] ヘルパーは引き続き、ラップした特定のエージェントへ転送し、受け取り側のエージェントは [`input_filter`][agents.handoffs.Handoff.input_filter] またはネストされたハンドオフ履歴設定で変更しない限り、引き続き会話履歴を参照します。
`input_type` は [`RunContextWrapper.context`][agents.run_context.RunContextWrapper.context] とも別のものです。ローカルにすでにあるアプリケーション状態や依存関係ではなく、ハンドオフ時にモデルが決定するメタデータには `input_type` を使用してください。
### `input_type` の使用タイミング
ハンドオフに `reason``language``priority``summary` など、モデルが生成する小さなメタデータが必要な場合に `input_type` を使用してください。たとえば、トリアージエージェントは `{ "reason": "duplicate_charge", "priority": "high" }` とともに返金エージェントへハンドオフでき、返金エージェントが引き継ぐ前に `on_handoff` でそのメタデータをログに記録したり永続化したりできます。
目的が異なる場合は、別の仕組みを選んでください。
- 既存のアプリケーション状態と依存関係は [`RunContextWrapper.context`][agents.run_context.RunContextWrapper.context] に置いてください。[コンテキストガイド](context.md)を参照してください。
- 受け取り側のエージェントが参照する履歴を変更したい場合は、 [`input_filter`][agents.handoffs.Handoff.input_filter] 、 [`RunConfig.nest_handoff_history`][agents.run.RunConfig.nest_handoff_history] 、または [`RunConfig.handoff_history_mapper`][agents.run.RunConfig.handoff_history_mapper] を使用してください。
- 複数の専門エージェント候補がある場合は、宛先ごとに 1 つのハンドオフを登録してください。 `input_type` は選択されたハンドオフにメタデータを追加できますが、宛先間の振り分けは行いません。
- 会話を引き渡さずに、ネストされた専門エージェントに構造化入力を渡したい場合は、 [`Agent.as_tool(parameters=...)`][agents.agent.Agent.as_tool] を優先してください。[ツール](tools.md#structured-input-for-tool-agents)を参照してください。
## 入力フィルター
ハンドオフが発生すると、新しいエージェントが会話を引き継ぎ、以前の会話履歴全体を参照できるようになります。これを変更したい場合は、 [`input_filter`][agents.handoffs.Handoff.input_filter] を設定できます。入力フィルターは、 [`HandoffInputData`][agents.handoffs.HandoffInputData] を通じて既存の入力を受け取り、新しい `HandoffInputData` を返す必要がある関数です。
[`HandoffInputData`][agents.handoffs.HandoffInputData] には次が含まれます。
- `input_history`: `Runner.run(...)` が開始する前の入力履歴です。
- `pre_handoff_items`: ハンドオフが呼び出されたエージェントターンより前に生成されたアイテムです。
- `new_items`: ハンドオフ呼び出しとハンドオフ出力アイテムを含む、現在のターン中に生成されたアイテムです。
- `input_items`: セッション履歴用に `new_items` をそのまま保ちながらモデル入力をフィルタリングできるよう、 `new_items` の代わりに次のエージェントへ転送する任意のアイテムです。
- `run_context`: ハンドオフが呼び出された時点でアクティブな [`RunContextWrapper`][agents.run_context.RunContextWrapper] です。
ネストされたハンドオフはオプトインのベータとして利用でき、安定化が進むまではデフォルトで無効です。 [`RunConfig.nest_handoff_history`][agents.run.RunConfig.nest_handoff_history] を有効にすると、ランナーは以前の会話記録を 1 つの assistant 要約メッセージにまとめ、それを `<CONVERSATION HISTORY>` ブロックで包みます。このブロックには、同じ実行中に複数のハンドオフが発生した場合に新しいターンが追加され続けます。 [`RunConfig.handoff_history_mapper`][agents.run.RunConfig.handoff_history_mapper] を通じて独自のマッピング関数を提供し、完全な `input_filter` を書くことなく、生成されたメッセージを置き換えることができます。このオプトインは、ハンドオフと実行のどちらも明示的な `input_filter` を指定していない場合にのみ適用されます。そのため、ペイロードをすでにカスタマイズしている既存のコード(このリポジトリ内のコード例を含む)は、変更なしで現在の動作を維持します。単一のハンドオフに対してネスト動作を上書きするには、 [`handoff(...)`][agents.handoffs.handoff] に `nest_handoff_history=True` または `False` を渡します。これにより [`Handoff.nest_handoff_history`][agents.handoffs.Handoff.nest_handoff_history] が設定されます。生成された要約のラッパーテキストだけを変更したい場合は、エージェントを実行する前に [`set_conversation_history_wrappers`][agents.handoffs.set_conversation_history_wrappers] を呼び出してください(必要に応じて [`reset_conversation_history_wrappers`][agents.handoffs.reset_conversation_history_wrappers] も呼び出せます)。
ハンドオフとアクティブな [`RunConfig.handoff_input_filter`][agents.run.RunConfig.handoff_input_filter] の両方がフィルターを定義している場合、その特定のハンドオフではハンドオフごとの [`input_filter`][agents.handoffs.Handoff.input_filter] が優先されます。
!!! note
ハンドオフは単一の実行内にとどまります。入力ガードレールは引き続きチェーン内の最初のエージェントにのみ適用され、出力ガードレールは最終出力を生成するエージェントにのみ適用されます。ワークフロー内の各カスタム関数ツール呼び出しの周囲でチェックが必要な場合は、ツールガードレールを使用してください。
一般的なパターン(たとえば、履歴からすべてのツール呼び出しを削除するなど)がいくつかあり、 [`agents.extensions.handoff_filters`][] に実装されています。
```python
from agents import Agent, handoff
from agents.extensions import handoff_filters
agent = Agent(name="FAQ agent")
handoff_obj = handoff(
agent=agent,
input_filter=handoff_filters.remove_all_tools, # (1)!
)
```
1. これにより、 `FAQ agent` が呼び出されたときに、履歴からすべてのツールが自動的に削除されます。
## 推奨プロンプト
LLM がハンドオフを適切に理解できるように、エージェントにハンドオフに関する情報を含めることを推奨します。推奨されるプレフィックスを [`agents.extensions.handoff_prompt.RECOMMENDED_PROMPT_PREFIX`][] に用意しています。または、 [`agents.extensions.handoff_prompt.prompt_with_handoff_instructions`][] を呼び出して、推奨データをプロンプトに自動的に追加できます。
```python
from agents import Agent
from agents.extensions.handoff_prompt import RECOMMENDED_PROMPT_PREFIX
billing_agent = Agent(
name="Billing agent",
instructions=f"""{RECOMMENDED_PROMPT_PREFIX}
<Fill in the rest of your prompt here>.""",
)
```
+201
View File
@@ -0,0 +1,201 @@
---
search:
exclude: true
---
# ヒューマンインザループ
ヒューマンインザループ (HITL) フローを使用すると、人が慎重な扱いが必要なツール呼び出しを承認または拒否するまで、エージェントの実行を一時停止できます。ツールは承認が必要なタイミングを宣言し、実行結果は保留中の承認を中断として提示し、`RunState` によって判定後に実行をシリアライズして再開できます。
その承認の提示先は実行全体であり、現在のトップレベルのエージェントに限定されません。同じパターンは、ツールが現在のエージェントに属する場合、ハンドオフを通じて到達したエージェントに属する場合、またはネストされた [`Agent.as_tool()`][agents.agent.Agent.as_tool] 実行に属する場合にも適用されます。ネストされた `Agent.as_tool()` の場合でも、中断は外側の実行に提示されるため、外側の `RunState` で承認または拒否し、元のトップレベルの実行を再開します。
`Agent.as_tool()` では、承認が 2 つの異なるレイヤーで発生する可能性があります。エージェントツール自体が `Agent.as_tool(..., needs_approval=...)` によって承認を要求でき、ネストされたエージェント内のツールも、ネストされた実行が開始した後に独自の承認を要求できます。どちらも同じ外側の実行の中断フローを通じて処理されます。
このページでは、`interruptions` を介した手動承認フローに焦点を当てます。アプリがコード内で判定できる場合、一部のツールタイプはプログラムによる承認コールバックにも対応しているため、実行を一時停止せずに続行できます。
## 承認が必要なツールの指定
常に承認を要求するには `needs_approval``True` に設定するか、呼び出しごとに判定する async 関数を指定します。この呼び出し可能オブジェクトは、実行コンテキスト、解析済みのツールパラメーター、ツール呼び出し ID を受け取ります。
```python
from agents import Agent, Runner, function_tool
@function_tool(needs_approval=True)
async def cancel_order(order_id: int) -> str:
return f"Cancelled order {order_id}"
async def requires_review(_ctx, params, _call_id) -> bool:
return "refund" in params.get("subject", "").lower()
@function_tool(needs_approval=requires_review)
async def send_email(subject: str, body: str) -> str:
return f"Sent '{subject}'"
agent = Agent(
name="Support agent",
instructions="Handle tickets and ask for approval when needed.",
tools=[cancel_order, send_email],
)
```
`needs_approval` は、[`function_tool`][agents.tool.function_tool]、[`Agent.as_tool`][agents.agent.Agent.as_tool]、[`ShellTool`][agents.tool.ShellTool]、[`ApplyPatchTool`][agents.tool.ApplyPatchTool] で利用できます。ローカル MCP サーバーも、[`MCPServerStdio`][agents.mcp.server.MCPServerStdio]、[`MCPServerSse`][agents.mcp.server.MCPServerSse]、[`MCPServerStreamableHttp`][agents.mcp.server.MCPServerStreamableHttp] の `require_approval` を通じて承認に対応しています。ホスト型 MCP サーバーは、`tool_config={"require_approval": "always"}` と任意の `on_approval_request` コールバックを設定した [`HostedMCPTool`][agents.tool.HostedMCPTool] によって承認に対応します。Shell と apply_patch ツールは、中断を提示せずに自動承認または自動拒否したい場合に `on_approval` コールバックを受け付けます。
## 承認フローの仕組み
1. モデルがツール呼び出しを出力すると、ランナーはその承認ルール (`needs_approval``require_approval`、またはホスト型 MCP の同等機能) を評価します。
2. そのツール呼び出しの承認判定がすでに [`RunContextWrapper`][agents.run_context.RunContextWrapper] に保存されている場合、ランナーは確認を求めずに処理を続行します。呼び出しごとの承認は特定の呼び出し ID にスコープされます。そのツールに対する今後の呼び出しに、実行の残りの間同じ判定を保持するには、`always_approve=True` または `always_reject=True` を渡します。
3. それ以外の場合、実行は一時停止し、`RunResult.interruptions` (または `RunResultStreaming.interruptions`) に、`agent.name``tool_name``arguments` などの詳細を含む [`ToolApprovalItem`][agents.items.ToolApprovalItem] エントリが入ります。これには、ハンドオフ後やネストされた `Agent.as_tool()` 実行内で発生した承認も含まれます。
4. 実行結果を `result.to_state()``RunState` に変換し、`state.approve(...)` または `state.reject(...)` を呼び出してから、`Runner.run(agent, state)` または `Runner.run_streamed(agent, state)` で再開します。ここで `agent` は、その実行における元のトップレベルのエージェントです。
5. 再開された実行は中断した場所から続行し、新しい承認が必要になった場合はこのフローに再び入ります。
`always_approve=True` または `always_reject=True` で作成された固定判定は実行状態に保存されるため、後で同じ一時停止中の実行を再開するときに `state.to_string()` / `RunState.from_string(...)` および `state.to_json()` / `RunState.from_json(...)` を使っても保持されます。
すべての保留中承認を同じ 1 回の処理で解決する必要はありません。`interruptions` には、通常の関数ツール、ホスト型 MCP の承認、ネストされた `Agent.as_tool()` の承認が混在する場合があります。一部の項目だけを承認または拒否した後に再実行すると、解決済みの呼び出しは続行でき、未解決のものは `interruptions` に残って実行を再び一時停止します。
## カスタム拒否メッセージ
既定では、拒否されたツール呼び出しは SDK 標準の拒否テキストを実行内に返します。このメッセージは 2 つのレイヤーでカスタマイズできます。
- 実行全体のフォールバック: 実行全体で承認拒否に対するモデルに見える既定メッセージを制御するには、[`RunConfig.tool_error_formatter`][agents.run.RunConfig.tool_error_formatter] を設定します。
- 呼び出しごとのオーバーライド: 特定の拒否されたツール呼び出しだけに異なるメッセージを提示したい場合は、`state.reject(...)``rejection_message=...` を渡します。
両方が指定されている場合、呼び出しごとの `rejection_message` が実行全体のフォーマッターより優先されます。
```python
from agents import RunConfig, ToolErrorFormatterArgs
def format_rejection(args: ToolErrorFormatterArgs[None]) -> str | None:
if args.kind != "approval_rejected":
return None
return "Publish action was canceled because approval was rejected."
run_config = RunConfig(tool_error_formatter=format_rejection)
# Later, while resolving a specific interruption:
state.reject(
interruption,
rejection_message="Publish action was canceled because the reviewer denied approval.",
)
```
両方のレイヤーをまとめて示す完全なコード例については、[`examples/agent_patterns/human_in_the_loop_custom_rejection.py`](https://github.com/openai/openai-agents-python/tree/main/examples/agent_patterns/human_in_the_loop_custom_rejection.py) を参照してください。
## 自動承認判定
手動の `interruptions` は最も汎用的なパターンですが、唯一の方法ではありません。
- ローカルの [`ShellTool`][agents.tool.ShellTool] と [`ApplyPatchTool`][agents.tool.ApplyPatchTool] は、`on_approval` を使用してコード内で即座に承認または拒否できます。
- [`HostedMCPTool`][agents.tool.HostedMCPTool] は、`tool_config={"require_approval": "always"}``on_approval_request` を組み合わせて、同じ種類のプログラムによる判定を行えます。
- 通常の [`function_tool`][agents.tool.function_tool] ツールと [`Agent.as_tool()`][agents.agent.Agent.as_tool] は、このページの手動中断フローを使用します。
これらのコールバックが判定を返すと、人間の応答を待って一時停止することなく実行が続行されます。Realtime および音声セッション API については、[Realtime ガイド](realtime/guide.md) の承認フローを参照してください。
## ストリーミングとセッション
同じ中断フローはストリーミング実行でも機能します。ストリーミング実行が一時停止した後は、イテレーターが終了するまで [`RunResultStreaming.stream_events()`][agents.result.RunResultStreaming.stream_events] を消費し続け、[`RunResultStreaming.interruptions`][agents.result.RunResultStreaming.interruptions] を確認して解決し、再開後の出力もストリーミングし続けたい場合は [`Runner.run_streamed(...)`][agents.run.Runner.run_streamed] で再開します。このパターンのストリーミング版については、[ストリーミング](streaming.md) を参照してください。
セッションも使用している場合は、`RunState` から再開するときに同じセッションインスタンスを渡し続けるか、同じバッキングストアを指す別のセッションオブジェクトを渡します。これにより、再開されたターンは同じ保存済み会話履歴に追加されます。セッションのライフサイクル詳細については、[セッション](sessions/index.md) を参照してください。
## 例: 一時停止・承認・再開
以下のスニペットは JavaScript の HITL ガイドと同じ流れです。ツールに承認が必要な場合に一時停止し、状態をディスクに永続化して再読み込みし、判定を収集した後に再開します。
```python
import asyncio
import json
from pathlib import Path
from agents import Agent, Runner, RunState, function_tool
async def needs_oakland_approval(_ctx, params, _call_id) -> bool:
return "Oakland" in params.get("city", "")
@function_tool(needs_approval=needs_oakland_approval)
async def get_temperature(city: str) -> str:
return f"The temperature in {city} is 20° Celsius"
agent = Agent(
name="Weather assistant",
instructions="Answer weather questions with the provided tools.",
tools=[get_temperature],
)
STATE_PATH = Path(".cache/hitl_state.json")
def prompt_approval(tool_name: str, arguments: str | None) -> bool:
answer = input(f"Approve {tool_name} with {arguments}? [y/N]: ").strip().lower()
return answer in {"y", "yes"}
async def main() -> None:
result = await Runner.run(agent, "What is the temperature in Oakland?")
while result.interruptions:
# Persist the paused state.
state = result.to_state()
STATE_PATH.parent.mkdir(parents=True, exist_ok=True)
STATE_PATH.write_text(state.to_string())
# Load the state later (could be a different process).
stored = json.loads(STATE_PATH.read_text())
state = await RunState.from_json(agent, stored)
for interruption in result.interruptions:
approved = await asyncio.get_running_loop().run_in_executor(
None, prompt_approval, interruption.name or "unknown_tool", interruption.arguments
)
if approved:
state.approve(interruption, always_approve=False)
else:
state.reject(interruption)
result = await Runner.run(agent, state)
print(result.final_output)
if __name__ == "__main__":
asyncio.run(main())
```
この例では、`prompt_approval``input()` を使用し、`run_in_executor(...)` で実行されるため同期的です。承認の取得元がすでに非同期である場合 (たとえば、HTTP リクエストや非同期データベースクエリ)、代わりに `async def` 関数を使用して直接 `await` できます。
承認を待つ間に出力をストリーミングするには、`Runner.run_streamed` を呼び出し、完了するまで `result.stream_events()` を消費してから、上記と同じ `result.to_state()` と再開手順に従います。
## リポジトリのパターンとコード例
- **ストリーミング承認**: `examples/agent_patterns/human_in_the_loop_stream.py` は、`stream_events()` を最後まで読み出し、その後 `Runner.run_streamed(agent, state)` で再開する前に保留中のツール呼び出しを承認する方法を示します。
- **カスタム拒否テキスト**: `examples/agent_patterns/human_in_the_loop_custom_rejection.py` は、承認が拒否された場合に、実行レベルの `tool_error_formatter` と呼び出しごとの `rejection_message` オーバーライドを組み合わせる方法を示します。
- **ツールとしてのエージェントの承認**: `Agent.as_tool(..., needs_approval=...)` は、委譲されたエージェントタスクにレビューが必要な場合に同じ中断フローを適用します。ネストされた中断も外側の実行に提示されるため、ネストされたエージェントではなく元のトップレベルのエージェントを再開してください。
- **ローカル shell と apply_patch ツール**: `ShellTool``ApplyPatchTool``needs_approval` に対応しています。将来の呼び出しに備えて判定をキャッシュするには、`state.approve(interruption, always_approve=True)` または `state.reject(..., always_reject=True)` を使用します。自動判定には `on_approval` を指定します (`examples/tools/shell.py` を参照)。手動判定には中断を処理します (`examples/tools/shell_human_in_the_loop.py` を参照)。ホスト型 shell 環境は `needs_approval` または `on_approval` に対応していません。[ツールガイド](tools.md) を参照してください。
- **ローカル MCP サーバー**: MCP ツール呼び出しを制御するには、`MCPServerStdio` / `MCPServerSse` / `MCPServerStreamableHttp``require_approval` を使用します (`examples/mcp/get_all_mcp_tools_example/main.py``examples/mcp/tool_filter_example/main.py` を参照)。
- **ホスト型 MCP サーバー**: HITL を強制するには、`HostedMCPTool``require_approval``"always"` に設定し、必要に応じて自動承認または拒否のために `on_approval_request` を指定します (`examples/hosted_mcp/human_in_the_loop.py``examples/hosted_mcp/on_approval.py` を参照)。信頼済みサーバーには `"never"` を使用します (`examples/hosted_mcp/simple.py`)。
- **セッションとメモリ**: セッションを `Runner.run` に渡すと、承認と会話履歴が複数ターンにわたって保持されます。SQLite と OpenAI Conversations のセッション版は、`examples/memory/memory_session_hitl_example.py``examples/memory/openai_session_hitl_example.py` にあります。
- **Realtime エージェント**: Realtime デモでは、`RealtimeSession``approve_tool_call` / `reject_tool_call` を介してツール呼び出しを承認または拒否する WebSocket メッセージを公開しています (サーバー側ハンドラーについては `examples/realtime/app/server.py`、API サーフェスについては [Realtime ガイド](realtime/guide.md#tool-approvals) を参照)。
## 長時間にわたる承認
`RunState` は耐久性を持つように設計されています。`state.to_json()` または `state.to_string()` を使用して保留中の作業をデータベースまたはキューに保存し、後で `RunState.from_json(...)` または `RunState.from_string(...)` で再作成します。
便利なシリアライズオプション:
- `context_serializer`: 非マッピングのコンテキストオブジェクトのシリアライズ方法をカスタマイズします。
- `context_deserializer`: `RunState.from_json(...)` または `RunState.from_string(...)` で状態を読み込むときに、非マッピングのコンテキストオブジェクトを再構築します。
- `strict_context=True`: コンテキストがすでにマッピングであるか、適切なシリアライザー / デシリアライザーを指定している場合を除き、シリアライズまたはデシリアライズを失敗させます。
- `context_override`: 状態を読み込むときに、シリアライズされたコンテキストを置き換えます。これは、元のコンテキストオブジェクトを復元したくない場合に便利ですが、すでにシリアライズ済みのペイロードからそのコンテキストを削除するわけではありません。
- `include_tracing_api_key=True`: 再開された作業で同じ認証情報を使ってトレースのエクスポートを継続する必要がある場合、シリアライズされたトレースペイロードにトレーシング API キーを含めます。
シリアライズされた実行状態には、アプリのコンテキストに加えて、承認、使用量、シリアライズ済みの `tool_input`、ネストされた agent-as-tool の再開情報、トレースメタデータ、サーバー管理の会話設定など、SDK 管理のランタイムメタデータが含まれます。シリアライズされた状態を保存または送信する予定がある場合は、`RunContextWrapper.context` を永続化データとして扱い、状態と一緒に移動させる意図がある場合を除き、そこにシークレットを置かないでください。
## 保留中タスクのバージョニング
承認がしばらく保留される可能性がある場合は、エージェント定義または SDK のバージョンマーカーを、シリアライズされた状態と一緒に保存してください。これにより、モデル、プロンプト、ツール定義が変更された場合の非互換性を避けるために、対応するコードパスへデシリアライズ処理を振り分けられます。

Some files were not shown because too many files have changed in this diff Show More