chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,63 @@
|
||||
# Common agentic patterns
|
||||
|
||||
This folder contains examples of different common patterns for agents.
|
||||
|
||||
## Hosted multi-agent (experimental)
|
||||
|
||||
The Responses API can coordinate server-hosted GPT-5.6 subagents while the normal SDK Runner executes developer-defined function tools locally. See [`hosted_multi_agent_beta.py`](./hosted_multi_agent_beta.py) for non-streaming and streaming examples using the experimental model module.
|
||||
|
||||
## Deterministic flows
|
||||
|
||||
A common tactic is to break down a task into a series of smaller steps. Each task can be performed by an agent, and the output of one agent is used as input to the next. For example, if your task was to generate a story, you could break it down into the following steps:
|
||||
|
||||
1. Generate an outline
|
||||
2. Generate the story
|
||||
3. Generate the ending
|
||||
|
||||
Each of these steps can be performed by an agent. The output of one agent is used as input to the next.
|
||||
|
||||
See the [`deterministic.py`](./deterministic.py) file for an example of this.
|
||||
|
||||
## Handoffs and routing
|
||||
|
||||
In many situations, you have specialized sub-agents that handle specific tasks. You can use handoffs to route the task to the right agent.
|
||||
|
||||
For example, you might have a frontline agent that receives a request, and then hands off to a specialized agent based on the language of the request. See the [`routing.py`](./routing.py) file for an example of this.
|
||||
|
||||
## Agents as tools
|
||||
|
||||
The mental model for handoffs is that the new agent "takes over". It sees the previous conversation history, and owns the conversation from that point onwards. However, this is not the only way to use agents. You can also use agents as a tool - the tool agent goes off and runs on its own, and then returns the result to the original agent.
|
||||
|
||||
For example, you could model the translation task above as tool calls instead: rather than handing over to the language-specific agent, you could call the agent as a tool, and then use the result in the next step. This enables things like translating multiple languages at once.
|
||||
|
||||
See the [`agents_as_tools.py`](./agents_as_tools.py) file for an example of this. See the [`agents_as_tools_streaming.py`](./agents_as_tools_streaming.py) file for a streaming variant that taps into nested agent events via `on_stream`. See the [`agents_as_tools_structured.py`](./agents_as_tools_structured.py) file for a structured-input variant using `Agent.as_tool()` parameters.
|
||||
|
||||
## LLM-as-a-judge
|
||||
|
||||
LLMs can often improve the quality of their output if given feedback. A common pattern is to generate a response using a model, and then use a second model to provide feedback. You can even use a small model for the initial generation and a larger model for the feedback, to optimize cost.
|
||||
|
||||
For example, you could use an LLM to generate an outline for a story, and then use a second LLM to evaluate the outline and provide feedback. You can then use the feedback to improve the outline, and repeat until the LLM is satisfied with the outline.
|
||||
|
||||
See the [`llm_as_a_judge.py`](./llm_as_a_judge.py) file for an example of this.
|
||||
|
||||
## Parallelization
|
||||
|
||||
Running multiple agents in parallel is a common pattern. This can be useful for both latency (e.g. if you have multiple steps that don't depend on each other) and also for other reasons e.g. generating multiple responses and picking the best one.
|
||||
|
||||
See the [`parallelization.py`](./parallelization.py) file for an example of this. It runs a translation agent multiple times in parallel, and then picks the best translation.
|
||||
|
||||
## Guardrails
|
||||
|
||||
Related to parallelization, you often want to run input guardrails to make sure the inputs to your agents are valid. For example, if you have a customer support agent, you might want to make sure that the user isn't trying to ask for help with a math problem.
|
||||
|
||||
You can definitely do this without any special Agents SDK features by using parallelization, but we support a special guardrail primitive. Guardrails can have a "tripwire" - if the tripwire is triggered, the agent execution will immediately stop and a `GuardrailTripwireTriggered` exception will be raised.
|
||||
|
||||
This is really useful for latency: for example, you might have a very fast model that runs the guardrail and a slow model that runs the actual agent. You wouldn't want to wait for the slow model to finish, so guardrails let you quickly reject invalid inputs.
|
||||
|
||||
See the [`input_guardrails.py`](./input_guardrails.py) and [`output_guardrails.py`](./output_guardrails.py) files for examples.
|
||||
|
||||
## Human in the loop
|
||||
|
||||
You can pause runs for manual approval before executing sensitive tools. This is useful for operations like sending money, deleting data, or running destructive commands.
|
||||
|
||||
See [`human_in_the_loop.py`](./human_in_the_loop.py) for the base approval flow and [`human_in_the_loop_custom_rejection.py`](./human_in_the_loop_custom_rejection.py) for run-level tool error formatting when approvals are rejected.
|
||||
@@ -0,0 +1,83 @@
|
||||
import asyncio
|
||||
|
||||
from agents import Agent, ItemHelpers, MessageOutputItem, Runner, trace
|
||||
from examples.auto_mode import input_with_fallback
|
||||
|
||||
"""
|
||||
This example shows the agents-as-tools pattern. The frontline agent receives a user message and
|
||||
then picks which agents to call, as tools. In this case, it picks from a set of translation
|
||||
agents.
|
||||
"""
|
||||
|
||||
spanish_agent = Agent(
|
||||
name="spanish_agent",
|
||||
instructions="You translate the user's message to Spanish",
|
||||
handoff_description="An english to spanish translator",
|
||||
)
|
||||
|
||||
french_agent = Agent(
|
||||
name="french_agent",
|
||||
instructions="You translate the user's message to French",
|
||||
handoff_description="An english to french translator",
|
||||
)
|
||||
|
||||
italian_agent = Agent(
|
||||
name="italian_agent",
|
||||
instructions="You translate the user's message to Italian",
|
||||
handoff_description="An english to italian translator",
|
||||
)
|
||||
|
||||
orchestrator_agent = Agent(
|
||||
name="orchestrator_agent",
|
||||
instructions=(
|
||||
"You are a translation agent. You use the tools given to you to translate."
|
||||
"If asked for multiple translations, you call the relevant tools in order."
|
||||
"You never translate on your own, you always use the provided tools."
|
||||
),
|
||||
tools=[
|
||||
spanish_agent.as_tool(
|
||||
tool_name="translate_to_spanish",
|
||||
tool_description="Translate the user's message to Spanish",
|
||||
),
|
||||
french_agent.as_tool(
|
||||
tool_name="translate_to_french",
|
||||
tool_description="Translate the user's message to French",
|
||||
),
|
||||
italian_agent.as_tool(
|
||||
tool_name="translate_to_italian",
|
||||
tool_description="Translate the user's message to Italian",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
synthesizer_agent = Agent(
|
||||
name="synthesizer_agent",
|
||||
instructions="You inspect translations, correct them if needed, and produce a final concatenated response.",
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
msg = input_with_fallback(
|
||||
"Hi! What would you like translated, and to which languages? ",
|
||||
"Translate 'Hello, world!' to French and Spanish.",
|
||||
)
|
||||
|
||||
# Run the entire orchestration in a single trace
|
||||
with trace("Orchestrator evaluator"):
|
||||
orchestrator_result = await Runner.run(orchestrator_agent, msg)
|
||||
|
||||
for item in orchestrator_result.new_items:
|
||||
if isinstance(item, MessageOutputItem):
|
||||
text = ItemHelpers.text_message_output(item)
|
||||
if text:
|
||||
print(f" - Translation step: {text}")
|
||||
|
||||
synthesizer_result = await Runner.run(
|
||||
synthesizer_agent, orchestrator_result.to_input_list()
|
||||
)
|
||||
|
||||
print(f"\n\nFinal response:\n{synthesizer_result.final_output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,159 @@
|
||||
import asyncio
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from agents import Agent, AgentBase, ModelSettings, RunContextWrapper, Runner, trace
|
||||
from agents.tool import function_tool
|
||||
from examples.auto_mode import confirm_with_fallback, input_with_fallback, is_auto_mode
|
||||
|
||||
"""
|
||||
This example demonstrates the agents-as-tools pattern with conditional tool enabling.
|
||||
Agent tools are dynamically enabled/disabled based on user access levels using the
|
||||
is_enabled parameter.
|
||||
"""
|
||||
|
||||
|
||||
class AppContext(BaseModel):
|
||||
language_preference: str = "spanish_only" # "spanish_only", "french_spanish", "european"
|
||||
|
||||
|
||||
def french_spanish_enabled(ctx: RunContextWrapper[AppContext], agent: AgentBase) -> bool:
|
||||
"""Enable for French+Spanish and European preferences."""
|
||||
return ctx.context.language_preference in ["french_spanish", "european"]
|
||||
|
||||
|
||||
def european_enabled(ctx: RunContextWrapper[AppContext], agent: AgentBase) -> bool:
|
||||
"""Only enable for European preference."""
|
||||
return ctx.context.language_preference == "european"
|
||||
|
||||
|
||||
@function_tool(needs_approval=True)
|
||||
async def get_user_name() -> str:
|
||||
print("Getting the user's name...")
|
||||
return "Kaz"
|
||||
|
||||
|
||||
# Create specialized agents
|
||||
spanish_agent = Agent(
|
||||
name="spanish_agent",
|
||||
instructions=(
|
||||
"Respond in Spanish. Call get_user_name exactly once before replying, then greet that "
|
||||
"user by name and answer the user's question in a non-empty final response."
|
||||
),
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
tools=[get_user_name],
|
||||
)
|
||||
|
||||
french_agent = Agent(
|
||||
name="french_agent",
|
||||
instructions="You respond in French. Always reply to the user's question in French.",
|
||||
)
|
||||
|
||||
italian_agent = Agent(
|
||||
name="italian_agent",
|
||||
instructions="You respond in Italian. Always reply to the user's question in Italian.",
|
||||
)
|
||||
|
||||
# Create orchestrator with conditional tools
|
||||
orchestrator = Agent(
|
||||
name="orchestrator",
|
||||
instructions=(
|
||||
"You are a multilingual assistant. Call each available language tool requested by the "
|
||||
"user exactly once, including every requested tool when multiple languages are requested. "
|
||||
"Wait for all tool calls to finish, then combine their responses into a non-empty final "
|
||||
"response. Never translate the user's request yourself."
|
||||
),
|
||||
tools=[
|
||||
spanish_agent.as_tool(
|
||||
tool_name="respond_spanish",
|
||||
tool_description="Respond to the user's question in Spanish",
|
||||
is_enabled=True, # Always enabled
|
||||
needs_approval=True, # HITL
|
||||
),
|
||||
french_agent.as_tool(
|
||||
tool_name="respond_french",
|
||||
tool_description="Respond to the user's question in French",
|
||||
is_enabled=french_spanish_enabled,
|
||||
),
|
||||
italian_agent.as_tool(
|
||||
tool_name="respond_italian",
|
||||
tool_description="Respond to the user's question in Italian",
|
||||
is_enabled=european_enabled,
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
"""Interactive demo with LLM interaction."""
|
||||
print("Agents-as-Tools with Conditional Enabling\n")
|
||||
print(
|
||||
"This demonstrates how language response tools are dynamically enabled based on user preferences.\n"
|
||||
)
|
||||
|
||||
print("Choose language preference:")
|
||||
print("1. Spanish only (1 tool)")
|
||||
print("2. French and Spanish (2 tools)")
|
||||
print("3. European languages (3 tools)")
|
||||
|
||||
choice = input_with_fallback("\nSelect option (1-3): ", "2").strip()
|
||||
preference_map = {"1": "spanish_only", "2": "french_spanish", "3": "european"}
|
||||
language_preference = preference_map.get(choice, "spanish_only")
|
||||
|
||||
# Create context and show available tools
|
||||
context = RunContextWrapper(AppContext(language_preference=language_preference))
|
||||
available_tools = await orchestrator.get_all_tools(context)
|
||||
tool_names = [tool.name for tool in available_tools]
|
||||
|
||||
print(f"\nLanguage preference: {language_preference}")
|
||||
print(f"Available tools: {', '.join(tool_names)}")
|
||||
print(f"The LLM will only see and can use these {len(available_tools)} tools\n")
|
||||
|
||||
# Get user request
|
||||
user_request = input_with_fallback(
|
||||
"Ask a question and see responses in available languages:\n",
|
||||
"Answer in Spanish and French: How do you say good morning?",
|
||||
)
|
||||
|
||||
# Run with LLM interaction
|
||||
print("\nProcessing request...")
|
||||
with trace("Conditional tool access"):
|
||||
result = await Runner.run(
|
||||
starting_agent=orchestrator,
|
||||
input=user_request,
|
||||
context=context.context,
|
||||
)
|
||||
while result.interruptions:
|
||||
|
||||
async def confirm(question: str) -> bool:
|
||||
return confirm_with_fallback(f"{question} (y/n): ", default=True)
|
||||
|
||||
state = result.to_state()
|
||||
for interruption in result.interruptions:
|
||||
prompt = f"\nDo you approve this tool call: {interruption.name} with arguments {interruption.arguments}?"
|
||||
confirmed = await confirm(prompt)
|
||||
if confirmed:
|
||||
state.approve(interruption)
|
||||
print(f"✓ Approved: {interruption.name}")
|
||||
else:
|
||||
state.reject(interruption)
|
||||
print(f"✗ Rejected: {interruption.name}")
|
||||
result = await Runner.run(orchestrator, state)
|
||||
|
||||
if is_auto_mode():
|
||||
called_tools: list[str] = []
|
||||
for item in result.new_items:
|
||||
tool_name = getattr(item.raw_item, "name", None)
|
||||
if item.type == "tool_call_item" and isinstance(tool_name, str):
|
||||
if tool_name.startswith("respond_"):
|
||||
called_tools.append(tool_name)
|
||||
if sorted(called_tools) != ["respond_french", "respond_spanish"]:
|
||||
raise RuntimeError(f"Expected Spanish and French responses once, got {called_tools}")
|
||||
if not result.final_output:
|
||||
raise RuntimeError("Expected a non-empty multilingual response")
|
||||
|
||||
print(f"\nResponse:\n{result.final_output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,59 @@
|
||||
import asyncio
|
||||
|
||||
from agents import Agent, AgentToolStreamEvent, ModelSettings, Runner, function_tool, trace
|
||||
|
||||
|
||||
@function_tool(
|
||||
name_override="billing_status_checker",
|
||||
description_override="Answer questions about customer billing status.",
|
||||
)
|
||||
def billing_status_checker(customer_id: str | None = None, question: str = "") -> str:
|
||||
"""Return a canned billing answer or a fallback when the question is unrelated."""
|
||||
normalized = question.lower()
|
||||
if "bill" in normalized or "billing" in normalized:
|
||||
return f"This customer (ID: {customer_id})'s bill is $100"
|
||||
return "I can only answer questions about billing."
|
||||
|
||||
|
||||
def handle_stream(event: AgentToolStreamEvent) -> None:
|
||||
"""Print streaming events emitted by the nested billing agent."""
|
||||
stream = event["event"]
|
||||
tool_call = event.get("tool_call")
|
||||
tool_call_info = tool_call.call_id if tool_call is not None else "unknown"
|
||||
print(f"[stream] agent={event['agent'].name} call={tool_call_info} type={stream.type} {stream}")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
with trace("Agents as tools streaming example"):
|
||||
billing_agent = Agent(
|
||||
name="Billing Agent",
|
||||
instructions="You are a billing agent that answers billing questions.",
|
||||
model_settings=ModelSettings(tool_choice="required"),
|
||||
tools=[billing_status_checker],
|
||||
)
|
||||
|
||||
billing_agent_tool = billing_agent.as_tool(
|
||||
tool_name="billing_agent",
|
||||
tool_description="You are a billing agent that answers billing questions.",
|
||||
on_stream=handle_stream,
|
||||
)
|
||||
|
||||
main_agent = Agent(
|
||||
name="Customer Support Agent",
|
||||
instructions=(
|
||||
"You are a customer support agent. Always call the billing agent to answer billing "
|
||||
"questions and return the billing agent response to the user."
|
||||
),
|
||||
tools=[billing_agent_tool],
|
||||
)
|
||||
|
||||
result = await Runner.run(
|
||||
main_agent,
|
||||
"Hello, my customer ID is ABC123. How much is my bill for this month?",
|
||||
)
|
||||
|
||||
print(f"\nFinal response:\n{result.final_output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,64 @@
|
||||
import asyncio
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from agents import Agent, Runner
|
||||
|
||||
"""
|
||||
This example shows structured input for agent-as-tool calls.
|
||||
"""
|
||||
|
||||
|
||||
class TranslationInput(BaseModel):
|
||||
text: str = Field(description="Text to translate.")
|
||||
source: str = Field(description="Source language code or name.")
|
||||
target: str = Field(description="Target language code or name.")
|
||||
|
||||
|
||||
translator = Agent(
|
||||
name="translator",
|
||||
instructions=(
|
||||
"Translate the input text into the target language. "
|
||||
"If the target is not clear, ask the user for clarification."
|
||||
),
|
||||
)
|
||||
|
||||
orchestrator = Agent(
|
||||
name="orchestrator",
|
||||
instructions=(
|
||||
"You are a task dispatcher. Always call the tool with sufficient input. "
|
||||
"Do not handle the translation yourself."
|
||||
),
|
||||
tools=[
|
||||
translator.as_tool(
|
||||
tool_name="translate_text",
|
||||
tool_description=(
|
||||
"Translate text between languages. Provide text, source language, "
|
||||
"and target language."
|
||||
),
|
||||
parameters=TranslationInput,
|
||||
# By default, the input schema will be included in a simpler format.
|
||||
# Set include_input_schema to true to include the full JSON Schema:
|
||||
# include_input_schema=True,
|
||||
# Build a custom prompt from structured input data:
|
||||
# input_builder=lambda options: (
|
||||
# f'Translate the text "{options["params"]["text"]}" '
|
||||
# f'from {options["params"]["source"]} to {options["params"]["target"]}.'
|
||||
# ),
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
query = 'Translate "Hola" from Spanish to French.'
|
||||
|
||||
response1 = await Runner.run(translator, query)
|
||||
print(f"Translator agent direct run: {response1.final_output}")
|
||||
|
||||
response2 = await Runner.run(orchestrator, query)
|
||||
print(f"Translator agent as tool: {response2.final_output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,84 @@
|
||||
import asyncio
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from agents import Agent, Runner, trace
|
||||
from examples.auto_mode import input_with_fallback
|
||||
|
||||
"""
|
||||
This example demonstrates a deterministic flow, where each step is performed by an agent.
|
||||
1. The first agent generates a story outline
|
||||
2. We feed the outline into the second agent
|
||||
3. The second agent checks if the outline is good quality and if it is a scifi story
|
||||
4. If the outline is not good quality or not a scifi story, we stop here
|
||||
5. If the outline is good quality and a scifi story, we feed the outline into the third agent
|
||||
6. The third agent writes the story
|
||||
"""
|
||||
|
||||
story_outline_agent = Agent(
|
||||
name="story_outline_agent",
|
||||
instructions="Generate a very short story outline based on the user's input.",
|
||||
)
|
||||
|
||||
|
||||
class OutlineCheckerOutput(BaseModel):
|
||||
good_quality: bool
|
||||
is_scifi: bool
|
||||
|
||||
|
||||
outline_checker_agent = Agent(
|
||||
name="outline_checker_agent",
|
||||
instructions="Read the given story outline, and judge the quality. Also, determine if it is a scifi story.",
|
||||
output_type=OutlineCheckerOutput,
|
||||
)
|
||||
|
||||
story_agent = Agent(
|
||||
name="story_agent",
|
||||
instructions="Write a short story based on the given outline.",
|
||||
output_type=str,
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
input_prompt = input_with_fallback(
|
||||
"What kind of story do you want? ",
|
||||
"Write a short sci-fi story.",
|
||||
)
|
||||
|
||||
# Ensure the entire workflow is a single trace
|
||||
with trace("Deterministic story flow"):
|
||||
# 1. Generate an outline
|
||||
outline_result = await Runner.run(
|
||||
story_outline_agent,
|
||||
input_prompt,
|
||||
)
|
||||
print("Outline generated")
|
||||
|
||||
# 2. Check the outline
|
||||
outline_checker_result = await Runner.run(
|
||||
outline_checker_agent,
|
||||
outline_result.final_output,
|
||||
)
|
||||
|
||||
# 3. Add a gate to stop if the outline is not good quality or not a scifi story
|
||||
assert isinstance(outline_checker_result.final_output, OutlineCheckerOutput)
|
||||
if not outline_checker_result.final_output.good_quality:
|
||||
print("Outline is not good quality, so we stop here.")
|
||||
exit(0)
|
||||
|
||||
if not outline_checker_result.final_output.is_scifi:
|
||||
print("Outline is not a scifi story, so we stop here.")
|
||||
exit(0)
|
||||
|
||||
print("Outline is good quality and a scifi story, so we continue to write the story.")
|
||||
|
||||
# 4. Write the story
|
||||
story_result = await Runner.run(
|
||||
story_agent,
|
||||
outline_result.final_output,
|
||||
)
|
||||
print(f"Story: {story_result.final_output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,112 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from agents import (
|
||||
Agent,
|
||||
FunctionToolResult,
|
||||
ModelSettings,
|
||||
RunContextWrapper,
|
||||
Runner,
|
||||
ToolsToFinalOutputFunction,
|
||||
ToolsToFinalOutputResult,
|
||||
function_tool,
|
||||
)
|
||||
from examples.auto_mode import is_auto_mode
|
||||
|
||||
"""
|
||||
This example shows how to force the agent to use a tool. It uses `ModelSettings(tool_choice="required")`
|
||||
to force the agent to use any tool.
|
||||
|
||||
You can run it with 3 options:
|
||||
1. `default`: The default behavior, which is to send the tool output to the LLM. In this case,
|
||||
`tool_choice` is not set, because otherwise it would result in an infinite loop - the LLM would
|
||||
call the tool, the tool would run and send the results to the LLM, and that would repeat
|
||||
(because the model is forced to use a tool every time.)
|
||||
2. `first_tool_result`: The first tool result is used as the final output.
|
||||
3. `custom`: A custom tool use behavior function is used. The custom function receives all the tool
|
||||
results, and chooses to use the first tool result to generate the final output.
|
||||
|
||||
Usage:
|
||||
python examples/agent_patterns/forcing_tool_use.py -t default
|
||||
python examples/agent_patterns/forcing_tool_use.py -t first_tool
|
||||
python examples/agent_patterns/forcing_tool_use.py -t custom
|
||||
"""
|
||||
|
||||
|
||||
class Weather(BaseModel):
|
||||
city: str
|
||||
temperature_range: str
|
||||
conditions: str
|
||||
|
||||
|
||||
@function_tool
|
||||
def get_weather(city: str) -> Weather:
|
||||
print("[debug] get_weather called")
|
||||
return Weather(city=city, temperature_range="14-20C", conditions="Sunny with wind")
|
||||
|
||||
|
||||
async def custom_tool_use_behavior(
|
||||
context: RunContextWrapper[Any], results: list[FunctionToolResult]
|
||||
) -> ToolsToFinalOutputResult:
|
||||
weather: Weather = results[0].output
|
||||
return ToolsToFinalOutputResult(
|
||||
is_final_output=True, final_output=f"{weather.city} is {weather.conditions}."
|
||||
)
|
||||
|
||||
|
||||
async def main(tool_use_behavior: Literal["default", "first_tool", "custom"] = "default"):
|
||||
if tool_use_behavior == "default":
|
||||
behavior: Literal["run_llm_again", "stop_on_first_tool"] | ToolsToFinalOutputFunction = (
|
||||
"run_llm_again"
|
||||
)
|
||||
elif tool_use_behavior == "first_tool":
|
||||
behavior = "stop_on_first_tool"
|
||||
elif tool_use_behavior == "custom":
|
||||
behavior = custom_tool_use_behavior
|
||||
|
||||
agent = Agent(
|
||||
name="Weather agent",
|
||||
instructions="You are a helpful agent.",
|
||||
tools=[get_weather],
|
||||
tool_use_behavior=behavior,
|
||||
model_settings=ModelSettings(
|
||||
tool_choice="required" if tool_use_behavior != "default" else None
|
||||
),
|
||||
)
|
||||
|
||||
result = await Runner.run(agent, input="What's the weather in Tokyo?")
|
||||
print(result.final_output)
|
||||
|
||||
|
||||
async def auto_demo() -> None:
|
||||
for behavior in ("default", "first_tool", "custom"):
|
||||
print(f"=== {behavior} ===")
|
||||
await main(behavior)
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"-t",
|
||||
"--tool-use-behavior",
|
||||
type=str,
|
||||
default="default",
|
||||
choices=["default", "first_tool", "custom"],
|
||||
help=(
|
||||
"The behavior to use for tool use. "
|
||||
"default sends tool outputs back to the model, first_tool uses the first tool result as the final output, "
|
||||
"custom runs a custom tool use behavior function."
|
||||
),
|
||||
)
|
||||
args = parser.parse_args()
|
||||
if is_auto_mode():
|
||||
asyncio.run(auto_demo())
|
||||
else:
|
||||
asyncio.run(main(args.tool_use_behavior))
|
||||
@@ -0,0 +1,92 @@
|
||||
# Copy/paste command (does not modify uv.lock):
|
||||
# uv run -m examples.agent_patterns.hosted_multi_agent_beta --mode stream
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
from collections.abc import Mapping
|
||||
from typing import Any
|
||||
|
||||
from agents import Agent, Runner, function_tool
|
||||
from agents.extensions.experimental.hosted_multi_agent import (
|
||||
OpenAIHostedMultiAgentModel,
|
||||
get_hosted_agent_metadata,
|
||||
)
|
||||
from agents.tool_context import ToolContext
|
||||
|
||||
PROPOSALS = {
|
||||
"alpha": {"estimated_weeks": 6, "risk": "medium"},
|
||||
"beta": {"estimated_weeks": 8, "risk": "low"},
|
||||
}
|
||||
|
||||
|
||||
@function_tool
|
||||
def get_proposal(ctx: ToolContext[Any], proposal: str) -> dict[str, object]:
|
||||
"""Return deterministic details for one proposal."""
|
||||
metadata = get_hosted_agent_metadata(ctx)
|
||||
caller = metadata.agent_name if metadata else "unknown"
|
||||
print(f"local tool caller={caller} call_id={ctx.tool_call_id} proposal={proposal}")
|
||||
if proposal not in PROPOSALS:
|
||||
raise ValueError(f"Unknown proposal: {proposal}")
|
||||
return PROPOSALS[proposal]
|
||||
|
||||
|
||||
def build_agent() -> Agent[Any]:
|
||||
return Agent(
|
||||
name="Hosted proposal coordinator",
|
||||
instructions=(
|
||||
"Create two subagents. Ask one to inspect proposal alpha and the other to inspect "
|
||||
"proposal beta. Each subagent must call get_proposal for its assigned proposal. "
|
||||
"Wait for both results, then return one concise comparison from the root agent."
|
||||
),
|
||||
model=OpenAIHostedMultiAgentModel(
|
||||
model="gpt-5.6-sol",
|
||||
config={"max_concurrent_subagents": 3},
|
||||
),
|
||||
tools=[get_proposal],
|
||||
)
|
||||
|
||||
|
||||
def log_hosted_event(event: object) -> None:
|
||||
if getattr(event, "type", None) != "response.output_item.done":
|
||||
return
|
||||
item = getattr(event, "item", None)
|
||||
item_type = item.get("type") if isinstance(item, Mapping) else getattr(item, "type", None)
|
||||
metadata = get_hosted_agent_metadata(item)
|
||||
is_hosted_item = item_type in {
|
||||
"agent_message",
|
||||
"multi_agent_call",
|
||||
"multi_agent_call_output",
|
||||
}
|
||||
is_subagent_message = (
|
||||
item_type == "message" and metadata is not None and metadata.agent_name != "/root"
|
||||
)
|
||||
if is_hosted_item or is_subagent_message:
|
||||
caller = metadata.agent_name if metadata else "unknown"
|
||||
print(f"hosted item type={item_type} agent={caller}")
|
||||
|
||||
|
||||
async def run_nonstreamed() -> None:
|
||||
result = await Runner.run(build_agent(), "Compare proposal alpha and proposal beta.")
|
||||
print(f"\nFinal response:\n{result.final_output}")
|
||||
|
||||
|
||||
async def run_streamed() -> None:
|
||||
result = Runner.run_streamed(build_agent(), "Compare proposal alpha and proposal beta.")
|
||||
async for event in result.stream_events():
|
||||
if event.type == "raw_response_event":
|
||||
log_hosted_event(event.data)
|
||||
print(f"\nFinal response:\n{result.final_output}")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--mode", choices=("nonstream", "stream"), default="nonstream")
|
||||
args = parser.parse_args()
|
||||
if args.mode == "stream":
|
||||
await run_streamed()
|
||||
else:
|
||||
await run_nonstreamed()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Human-in-the-loop example with tool approval.
|
||||
|
||||
This example demonstrates how to:
|
||||
1. Define tools that require approval before execution
|
||||
2. Handle interruptions when tool approval is needed
|
||||
3. Serialize/deserialize run state to continue execution later
|
||||
4. Approve or reject tool calls based on user input
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from agents import Agent, Runner, RunState, function_tool
|
||||
from examples.auto_mode import confirm_with_fallback
|
||||
|
||||
|
||||
@function_tool
|
||||
async def get_weather(city: str) -> str:
|
||||
"""Get the weather for a given city.
|
||||
|
||||
Args:
|
||||
city: The city to get weather for.
|
||||
|
||||
Returns:
|
||||
Weather information for the city.
|
||||
"""
|
||||
return f"The weather in {city} is sunny"
|
||||
|
||||
|
||||
async def _needs_temperature_approval(_ctx, params, _call_id) -> bool:
|
||||
"""Check if temperature tool needs approval."""
|
||||
return "Oakland" in params.get("city", "")
|
||||
|
||||
|
||||
@function_tool(
|
||||
# Dynamic approval: only require approval for Oakland
|
||||
needs_approval=_needs_temperature_approval
|
||||
)
|
||||
async def get_temperature(city: str) -> str:
|
||||
"""Get the temperature for a given city.
|
||||
|
||||
Args:
|
||||
city: The city to get temperature for.
|
||||
|
||||
Returns:
|
||||
Temperature information for the city.
|
||||
"""
|
||||
return f"The temperature in {city} is 20° Celsius"
|
||||
|
||||
|
||||
# Main agent with tool that requires approval
|
||||
agent = Agent(
|
||||
name="Weather Assistant",
|
||||
instructions=(
|
||||
"You are a helpful weather assistant. "
|
||||
"Answer questions about weather and temperature using the available tools."
|
||||
),
|
||||
tools=[get_weather, get_temperature],
|
||||
)
|
||||
|
||||
RESULT_PATH = Path(".cache/agent_patterns/human_in_the_loop/result.json")
|
||||
|
||||
|
||||
async def confirm(question: str) -> bool:
|
||||
"""Prompt user for yes/no confirmation.
|
||||
|
||||
Args:
|
||||
question: The question to ask.
|
||||
|
||||
Returns:
|
||||
True if user confirms, False otherwise.
|
||||
"""
|
||||
return confirm_with_fallback(f"{question} (y/n): ", default=True)
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run the human-in-the-loop example."""
|
||||
result = await Runner.run(
|
||||
agent,
|
||||
"What is the weather and temperature in Oakland?",
|
||||
)
|
||||
|
||||
has_interruptions = len(result.interruptions) > 0
|
||||
|
||||
while has_interruptions:
|
||||
print("\n" + "=" * 80)
|
||||
print("Run interrupted - tool approval required")
|
||||
print("=" * 80)
|
||||
|
||||
# Storing state to file (demonstrating serialization)
|
||||
state = result.to_state()
|
||||
state_json = state.to_json()
|
||||
RESULT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with RESULT_PATH.open("w") as f:
|
||||
json.dump(state_json, f, indent=2)
|
||||
|
||||
print(f"State saved to {RESULT_PATH}")
|
||||
|
||||
# From here on you could run things on a different thread/process
|
||||
|
||||
# Reading state from file (demonstrating deserialization)
|
||||
print(f"Loading state from {RESULT_PATH}")
|
||||
with RESULT_PATH.open() as f:
|
||||
stored_state_json = json.load(f)
|
||||
|
||||
state = await RunState.from_json(agent, stored_state_json)
|
||||
|
||||
# Process each interruption
|
||||
for interruption in result.interruptions:
|
||||
print("\nTool call details:")
|
||||
print(f" Agent: {interruption.agent.name}")
|
||||
print(f" Tool: {interruption.name}")
|
||||
print(f" Arguments: {interruption.arguments}")
|
||||
|
||||
confirmed = await confirm("\nDo you approve this tool call?")
|
||||
|
||||
if confirmed:
|
||||
print(f"✓ Approved: {interruption.name}")
|
||||
state.approve(interruption)
|
||||
else:
|
||||
print(f"✗ Rejected: {interruption.name}")
|
||||
state.reject(interruption)
|
||||
|
||||
# Resume execution with the updated state
|
||||
print("\nResuming agent execution...")
|
||||
result = await Runner.run(agent, state)
|
||||
has_interruptions = len(result.interruptions) > 0
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("Final Output:")
|
||||
print("=" * 80)
|
||||
print(result.final_output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Human-in-the-loop example with a custom rejection message.
|
||||
|
||||
This example is intentionally minimal:
|
||||
1. A single sensitive tool requires human approval.
|
||||
2. The first turn always issues that tool call.
|
||||
3. ``tool_error_formatter`` defines the universal fallback message shape.
|
||||
4. A per-call ``rejection_message`` passed to ``state.reject(...)`` overrides that fallback.
|
||||
5. The example prints both the tool output and the assistant's final reply.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from agents import (
|
||||
Agent,
|
||||
ModelSettings,
|
||||
RunConfig,
|
||||
Runner,
|
||||
ToolErrorFormatterArgs,
|
||||
function_tool,
|
||||
)
|
||||
from examples.auto_mode import confirm_with_fallback
|
||||
|
||||
|
||||
async def tool_error_formatter(args: ToolErrorFormatterArgs[None]) -> str | None:
|
||||
"""Build the universal fallback output message for rejected tool calls."""
|
||||
if args.kind != "approval_rejected":
|
||||
return None
|
||||
# The default message is "Tool execution was not approved."
|
||||
return "Publish action was canceled because approval was rejected."
|
||||
|
||||
|
||||
@function_tool(needs_approval=True)
|
||||
async def publish_announcement(title: str, body: str) -> str:
|
||||
"""Simulate publishing an announcement to users."""
|
||||
return f"Published announcement '{title}' with body: {body}"
|
||||
|
||||
|
||||
def _find_formatter_output(result: object) -> str | None:
|
||||
items = getattr(result, "new_items", None)
|
||||
if not isinstance(items, list):
|
||||
return None
|
||||
|
||||
for item in items:
|
||||
if getattr(item, "type", None) != "tool_call_output_item":
|
||||
continue
|
||||
output = getattr(item, "output", None)
|
||||
if isinstance(output, str):
|
||||
return output
|
||||
return None
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
agent = Agent(
|
||||
name="Operations Assistant",
|
||||
instructions=(
|
||||
"When a user asks to publish an announcement, call the publish_announcement tool directly. "
|
||||
"Do not ask the user for approval in plain text; runtime approvals handle that. "
|
||||
"If the tool call is rejected, respond with the exact rejection message and nothing else."
|
||||
),
|
||||
model_settings=ModelSettings(tool_choice="publish_announcement"),
|
||||
tools=[publish_announcement],
|
||||
)
|
||||
run_config = RunConfig(tool_error_formatter=tool_error_formatter)
|
||||
# ``tool_error_formatter`` is the universal fallback for approval rejects.
|
||||
# A specific ``rejection_message`` passed to ``state.reject(...)`` below overrides it.
|
||||
|
||||
result = await Runner.run(
|
||||
agent,
|
||||
"Please publish an announcement titled 'Office maintenance' with body "
|
||||
"'The office will close at 6 PM today.'",
|
||||
run_config=run_config,
|
||||
)
|
||||
|
||||
while result.interruptions:
|
||||
print("\nApproval required:")
|
||||
state = result.to_state()
|
||||
for interruption in result.interruptions:
|
||||
print(f"- Tool: {interruption.name}")
|
||||
print(f" Arguments: {interruption.arguments}")
|
||||
approved = confirm_with_fallback(
|
||||
"Approve this tool call? [y/N]: ",
|
||||
default=False,
|
||||
)
|
||||
if approved:
|
||||
state.approve(interruption)
|
||||
else:
|
||||
# This per-call rejection message takes precedence over ``tool_error_formatter``.
|
||||
state.reject(
|
||||
interruption,
|
||||
rejection_message=(
|
||||
"Publish action was canceled because the reviewer denied approval."
|
||||
),
|
||||
)
|
||||
|
||||
result = await Runner.run(agent, state, run_config=run_config)
|
||||
|
||||
formatter_output = _find_formatter_output(result)
|
||||
if formatter_output:
|
||||
print("\nFormatter output:")
|
||||
print(formatter_output)
|
||||
|
||||
print("\nFinal output:")
|
||||
print(result.final_output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Human-in-the-loop example with streaming.
|
||||
|
||||
This example demonstrates the human-in-the-loop (HITL) pattern with streaming.
|
||||
The agent will pause execution when a tool requiring approval is called,
|
||||
allowing you to approve or reject the tool call before continuing.
|
||||
|
||||
The streaming version provides real-time feedback as the agent processes
|
||||
the request, then pauses for approval when needed.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from agents import Agent, Runner, function_tool
|
||||
from examples.auto_mode import confirm_with_fallback
|
||||
|
||||
|
||||
async def _needs_temperature_approval(_ctx, params, _call_id) -> bool:
|
||||
"""Check if temperature tool needs approval."""
|
||||
return "Oakland" in params.get("city", "")
|
||||
|
||||
|
||||
@function_tool(
|
||||
# Dynamic approval: only require approval for Oakland
|
||||
needs_approval=_needs_temperature_approval
|
||||
)
|
||||
async def get_temperature(city: str) -> str:
|
||||
"""Get the temperature for a given city.
|
||||
|
||||
Args:
|
||||
city: The city to get temperature for.
|
||||
|
||||
Returns:
|
||||
Temperature information for the city.
|
||||
"""
|
||||
return f"The temperature in {city} is 20° Celsius"
|
||||
|
||||
|
||||
@function_tool
|
||||
async def get_weather(city: str) -> str:
|
||||
"""Get the weather for a given city.
|
||||
|
||||
Args:
|
||||
city: The city to get weather for.
|
||||
|
||||
Returns:
|
||||
Weather information for the city.
|
||||
"""
|
||||
return f"The weather in {city} is sunny."
|
||||
|
||||
|
||||
async def confirm(question: str) -> bool:
|
||||
"""Prompt user for yes/no confirmation.
|
||||
|
||||
Args:
|
||||
question: The question to ask.
|
||||
|
||||
Returns:
|
||||
True if user confirms, False otherwise.
|
||||
"""
|
||||
return confirm_with_fallback(f"{question} (y/n): ", default=True)
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run the human-in-the-loop example."""
|
||||
main_agent = Agent(
|
||||
name="Weather Assistant",
|
||||
instructions=(
|
||||
"You are a helpful weather assistant. "
|
||||
"Answer questions about weather and temperature using the available tools."
|
||||
),
|
||||
tools=[get_temperature, get_weather],
|
||||
)
|
||||
|
||||
# Run the agent with streaming
|
||||
result = Runner.run_streamed(
|
||||
main_agent,
|
||||
"What is the weather and temperature in Oakland?",
|
||||
)
|
||||
async for _ in result.stream_events():
|
||||
pass # Process streaming events silently or could print them
|
||||
|
||||
# Handle interruptions
|
||||
while len(result.interruptions) > 0:
|
||||
print("\n" + "=" * 80)
|
||||
print("Human-in-the-loop: approval required for the following tool calls:")
|
||||
print("=" * 80)
|
||||
|
||||
state = result.to_state()
|
||||
|
||||
for interruption in result.interruptions:
|
||||
print("\nTool call details:")
|
||||
print(f" Agent: {interruption.agent.name}")
|
||||
print(f" Tool: {interruption.name}")
|
||||
print(f" Arguments: {interruption.arguments}")
|
||||
|
||||
confirmed = await confirm("\nDo you approve this tool call?")
|
||||
|
||||
if confirmed:
|
||||
print(f"✓ Approved: {interruption.name}")
|
||||
state.approve(interruption)
|
||||
else:
|
||||
print(f"✗ Rejected: {interruption.name}")
|
||||
state.reject(interruption)
|
||||
|
||||
# Resume execution with streaming
|
||||
print("\nResuming agent execution...")
|
||||
result = Runner.run_streamed(main_agent, state)
|
||||
async for _ in result.stream_events():
|
||||
pass # Process streaming events silently or could print them
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("Final Output:")
|
||||
print("=" * 80)
|
||||
print(result.final_output)
|
||||
print("\nDone!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,122 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from agents import (
|
||||
Agent,
|
||||
GuardrailFunctionOutput,
|
||||
InputGuardrailTripwireTriggered,
|
||||
RunContextWrapper,
|
||||
Runner,
|
||||
TResponseInputItem,
|
||||
input_guardrail,
|
||||
)
|
||||
from examples.auto_mode import input_with_fallback, is_auto_mode
|
||||
|
||||
"""
|
||||
This example shows how to use guardrails.
|
||||
|
||||
Guardrails are checks that run in parallel to the agent's execution.
|
||||
They can be used to do things like:
|
||||
- Check if input messages are off-topic
|
||||
- Check that input messages don't violate any policies
|
||||
- Take over control of the agent's execution if an unexpected input is detected
|
||||
|
||||
In this example, we'll setup an input guardrail that trips if the user is asking to do math homework.
|
||||
If the guardrail trips, we'll respond with a refusal message.
|
||||
"""
|
||||
|
||||
|
||||
### 1. An agent-based guardrail that is triggered if the user is asking to do math homework
|
||||
class MathHomeworkOutput(BaseModel):
|
||||
reasoning: str
|
||||
is_math_homework: bool
|
||||
|
||||
|
||||
guardrail_agent = Agent(
|
||||
name="Guardrail check",
|
||||
instructions="Check if the user is asking you to do their math homework.",
|
||||
output_type=MathHomeworkOutput,
|
||||
)
|
||||
|
||||
|
||||
@input_guardrail
|
||||
async def math_guardrail(
|
||||
context: RunContextWrapper[None], agent: Agent, input: str | list[TResponseInputItem]
|
||||
) -> GuardrailFunctionOutput:
|
||||
"""This is an input guardrail function, which happens to call an agent to check if the input
|
||||
is a math homework question.
|
||||
"""
|
||||
result = await Runner.run(guardrail_agent, input, context=context.context)
|
||||
final_output = result.final_output_as(MathHomeworkOutput)
|
||||
|
||||
return GuardrailFunctionOutput(
|
||||
output_info=final_output,
|
||||
tripwire_triggered=final_output.is_math_homework,
|
||||
)
|
||||
|
||||
|
||||
### 2. The run loop
|
||||
|
||||
|
||||
async def main():
|
||||
agent = Agent(
|
||||
name="Customer support agent",
|
||||
instructions="You are a customer support agent. You help customers with their questions.",
|
||||
input_guardrails=[math_guardrail],
|
||||
)
|
||||
|
||||
input_data: list[TResponseInputItem] = []
|
||||
auto_mode = is_auto_mode()
|
||||
scripted_inputs = [
|
||||
"What's the capital of California?",
|
||||
"Can you help me solve for x: 2x + 5 = 11",
|
||||
]
|
||||
|
||||
while True:
|
||||
if auto_mode:
|
||||
if not scripted_inputs:
|
||||
break
|
||||
user_input = scripted_inputs.pop(0)
|
||||
print(f"[auto-input] Enter a message: -> {user_input}")
|
||||
else:
|
||||
user_input = input_with_fallback(
|
||||
"Enter a message: ",
|
||||
"What's the capital of California?",
|
||||
)
|
||||
input_data.append(
|
||||
{
|
||||
"role": "user",
|
||||
"content": user_input,
|
||||
}
|
||||
)
|
||||
|
||||
try:
|
||||
result = await Runner.run(agent, input_data)
|
||||
print(result.final_output)
|
||||
# If the guardrail didn't trigger, we use the result as the input for the next run
|
||||
input_data = result.to_input_list()
|
||||
except InputGuardrailTripwireTriggered:
|
||||
# If the guardrail triggered, we instead add a refusal message to the input
|
||||
message = "Sorry, I can't help you with your math homework."
|
||||
print(message)
|
||||
input_data.append(
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": message,
|
||||
}
|
||||
)
|
||||
if auto_mode and not scripted_inputs:
|
||||
break
|
||||
|
||||
# Sample run:
|
||||
# Enter a message: What's the capital of California?
|
||||
# The capital of California is Sacramento.
|
||||
# Enter a message: Can you help me solve for x: 2x + 5 = 11
|
||||
# Sorry, I can't help you with your math homework.
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,89 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from agents import Agent, ItemHelpers, Runner, TResponseInputItem, trace
|
||||
from examples.auto_mode import input_with_fallback, is_auto_mode
|
||||
|
||||
"""
|
||||
This example shows the LLM as a judge pattern. The first agent generates an outline for a story.
|
||||
The second agent judges the outline and provides feedback. We loop until the judge is satisfied
|
||||
with the outline.
|
||||
"""
|
||||
|
||||
story_outline_generator = Agent(
|
||||
name="story_outline_generator",
|
||||
instructions=(
|
||||
"You generate a very short story outline based on the user's input. "
|
||||
"If there is any feedback provided, use it to improve the outline."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class EvaluationFeedback:
|
||||
feedback: str
|
||||
score: Literal["pass", "needs_improvement", "fail"]
|
||||
|
||||
|
||||
evaluator = Agent[None](
|
||||
name="evaluator",
|
||||
instructions=(
|
||||
"You evaluate a story outline and decide if it's good enough. "
|
||||
"If it's not good enough, you provide feedback on what needs to be improved. "
|
||||
"Never give it a pass on the first try. After 5 attempts, you can give it a pass if the story outline is good enough - do not go for perfection"
|
||||
),
|
||||
output_type=EvaluationFeedback,
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
msg = input_with_fallback(
|
||||
"What kind of story would you like to hear? ",
|
||||
"A detective story in space.",
|
||||
)
|
||||
input_items: list[TResponseInputItem] = [{"content": msg, "role": "user"}]
|
||||
|
||||
latest_outline: str | None = None
|
||||
auto_mode = is_auto_mode()
|
||||
max_rounds = 3 if auto_mode else None
|
||||
rounds = 0
|
||||
|
||||
# We'll run the entire workflow in a single trace
|
||||
with trace("LLM as a judge"):
|
||||
while True:
|
||||
story_outline_result = await Runner.run(
|
||||
story_outline_generator,
|
||||
input_items,
|
||||
)
|
||||
|
||||
input_items = story_outline_result.to_input_list()
|
||||
latest_outline = ItemHelpers.text_message_outputs(story_outline_result.new_items)
|
||||
print("Story outline generated")
|
||||
|
||||
evaluator_result = await Runner.run(evaluator, input_items)
|
||||
result: EvaluationFeedback = evaluator_result.final_output
|
||||
|
||||
print(f"Evaluator score: {result.score}")
|
||||
|
||||
if result.score == "pass":
|
||||
print("Story outline is good enough, exiting.")
|
||||
break
|
||||
|
||||
if auto_mode:
|
||||
rounds += 1
|
||||
if max_rounds is not None and rounds >= max_rounds:
|
||||
print("Auto mode: stopping after limited rounds.")
|
||||
break
|
||||
|
||||
print("Re-running with feedback")
|
||||
|
||||
input_items.append({"content": f"Feedback: {result.feedback}", "role": "user"})
|
||||
|
||||
print(f"Final story outline: {latest_outline}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,80 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from agents import (
|
||||
Agent,
|
||||
GuardrailFunctionOutput,
|
||||
OutputGuardrailTripwireTriggered,
|
||||
RunContextWrapper,
|
||||
Runner,
|
||||
output_guardrail,
|
||||
)
|
||||
|
||||
"""
|
||||
This example shows how to use output guardrails.
|
||||
|
||||
Output guardrails are checks that run on the final output of an agent.
|
||||
They can be used to do things like:
|
||||
- Check if the output contains sensitive data
|
||||
- Check if the output is a valid response to the user's message
|
||||
|
||||
In this example, we'll use a (contrived) example where we check if the agent's response contains
|
||||
a phone number.
|
||||
"""
|
||||
|
||||
|
||||
# The agent's output type
|
||||
class MessageOutput(BaseModel):
|
||||
reasoning: str = Field(description="Thoughts on how to respond to the user's message")
|
||||
response: str = Field(description="The response to the user's message")
|
||||
user_name: str | None = Field(description="The name of the user who sent the message, if known")
|
||||
|
||||
|
||||
@output_guardrail
|
||||
async def sensitive_data_check(
|
||||
context: RunContextWrapper, agent: Agent, output: MessageOutput
|
||||
) -> GuardrailFunctionOutput:
|
||||
phone_number_in_response = "650" in output.response
|
||||
phone_number_in_reasoning = "650" in output.reasoning
|
||||
|
||||
return GuardrailFunctionOutput(
|
||||
output_info={
|
||||
"phone_number_in_response": phone_number_in_response,
|
||||
"phone_number_in_reasoning": phone_number_in_reasoning,
|
||||
},
|
||||
tripwire_triggered=phone_number_in_response or phone_number_in_reasoning,
|
||||
)
|
||||
|
||||
|
||||
agent = Agent(
|
||||
name="Assistant",
|
||||
instructions="You are a helpful assistant.",
|
||||
output_type=MessageOutput,
|
||||
output_guardrails=[sensitive_data_check],
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
# This should be ok
|
||||
await Runner.run(agent, "What's the capital of California?")
|
||||
print("First message passed")
|
||||
|
||||
# This should trip the guardrail
|
||||
try:
|
||||
result = await Runner.run(
|
||||
agent, "My phone number is 650-123-4567. Where do you think I live?"
|
||||
)
|
||||
print(
|
||||
f"Guardrail didn't trip - this is unexpected. Output: {json.dumps(result.final_output.model_dump(), indent=2)}"
|
||||
)
|
||||
|
||||
except OutputGuardrailTripwireTriggered as e:
|
||||
print(f"Guardrail tripped. Info: {e.guardrail_result.output.output_info}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,65 @@
|
||||
import asyncio
|
||||
|
||||
from agents import Agent, ItemHelpers, Runner, trace
|
||||
from examples.auto_mode import input_with_fallback
|
||||
|
||||
"""
|
||||
This example shows the parallelization pattern. We run the agent three times in parallel, and pick
|
||||
the best result.
|
||||
"""
|
||||
|
||||
spanish_agent = Agent(
|
||||
name="spanish_agent",
|
||||
instructions="You translate the user's message to Spanish",
|
||||
)
|
||||
|
||||
translation_picker = Agent(
|
||||
name="translation_picker",
|
||||
instructions="You pick the best Spanish translation from the given options.",
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
msg = input_with_fallback(
|
||||
"Hi! Enter a message, and we'll translate it to Spanish.\n\n",
|
||||
"Good morning!",
|
||||
)
|
||||
|
||||
# Ensure the entire workflow is a single trace
|
||||
with trace("Parallel translation"):
|
||||
res_1, res_2, res_3 = await asyncio.gather(
|
||||
Runner.run(
|
||||
spanish_agent,
|
||||
msg,
|
||||
),
|
||||
Runner.run(
|
||||
spanish_agent,
|
||||
msg,
|
||||
),
|
||||
Runner.run(
|
||||
spanish_agent,
|
||||
msg,
|
||||
),
|
||||
)
|
||||
|
||||
outputs = [
|
||||
ItemHelpers.text_message_outputs(res_1.new_items),
|
||||
ItemHelpers.text_message_outputs(res_2.new_items),
|
||||
ItemHelpers.text_message_outputs(res_3.new_items),
|
||||
]
|
||||
|
||||
translations = "\n\n".join(outputs)
|
||||
print(f"\n\nTranslations:\n\n{translations}")
|
||||
|
||||
best_translation = await Runner.run(
|
||||
translation_picker,
|
||||
f"Input: {msg}\n\nTranslations:\n{translations}",
|
||||
)
|
||||
|
||||
print("\n\n-----")
|
||||
|
||||
print(f"Best translation: {best_translation.final_output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,77 @@
|
||||
import asyncio
|
||||
import uuid
|
||||
|
||||
from openai.types.responses import ResponseContentPartDoneEvent, ResponseTextDeltaEvent
|
||||
|
||||
from agents import Agent, RawResponsesStreamEvent, Runner, TResponseInputItem, trace
|
||||
from examples.auto_mode import input_with_fallback, is_auto_mode
|
||||
|
||||
"""
|
||||
This example shows the handoffs/routing pattern. The triage agent receives the first message, and
|
||||
then hands off to the appropriate agent based on the language of the request. Responses are
|
||||
streamed to the user.
|
||||
"""
|
||||
|
||||
french_agent = Agent(
|
||||
name="french_agent",
|
||||
instructions="You only speak French",
|
||||
)
|
||||
|
||||
spanish_agent = Agent(
|
||||
name="spanish_agent",
|
||||
instructions="You only speak Spanish",
|
||||
)
|
||||
|
||||
english_agent = Agent(
|
||||
name="english_agent",
|
||||
instructions="You only speak English",
|
||||
)
|
||||
|
||||
triage_agent = Agent(
|
||||
name="triage_agent",
|
||||
instructions="Handoff to the appropriate agent based on the language of the request.",
|
||||
handoffs=[french_agent, spanish_agent, english_agent],
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
# We'll create an ID for this conversation, so we can link each trace
|
||||
conversation_id = str(uuid.uuid4().hex[:16])
|
||||
|
||||
msg = input_with_fallback(
|
||||
"Hi! We speak French, Spanish and English. How can I help? ",
|
||||
"Hello, how do I say good evening in French?",
|
||||
)
|
||||
agent = triage_agent
|
||||
inputs: list[TResponseInputItem] = [{"content": msg, "role": "user"}]
|
||||
auto_mode = is_auto_mode()
|
||||
|
||||
while True:
|
||||
# Each conversation turn is a single trace. Normally, each input from the user would be an
|
||||
# API request to your app, and you can wrap the request in a trace()
|
||||
with trace("Routing example", group_id=conversation_id):
|
||||
result = Runner.run_streamed(
|
||||
agent,
|
||||
input=inputs,
|
||||
)
|
||||
async for event in result.stream_events():
|
||||
if not isinstance(event, RawResponsesStreamEvent):
|
||||
continue
|
||||
data = event.data
|
||||
if isinstance(data, ResponseTextDeltaEvent):
|
||||
print(data.delta, end="", flush=True)
|
||||
elif isinstance(data, ResponseContentPartDoneEvent):
|
||||
print("\n")
|
||||
|
||||
inputs = result.to_input_list()
|
||||
print("\n")
|
||||
|
||||
if auto_mode:
|
||||
break
|
||||
user_msg = input_with_fallback("Enter a message: ", "Thanks!")
|
||||
inputs.append({"content": user_msg, "role": "user"})
|
||||
agent = result.current_agent
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,93 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
from openai.types.responses import ResponseTextDeltaEvent
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from agents import Agent, Runner
|
||||
|
||||
"""
|
||||
This example shows how to use guardrails as the model is streaming. Output guardrails run after the
|
||||
final output has been generated; this example runs guardails every N tokens, allowing for early
|
||||
termination if bad output is detected.
|
||||
|
||||
The expected output is that you'll see a bunch of tokens stream in, then the guardrail will trigger
|
||||
and stop the streaming.
|
||||
"""
|
||||
|
||||
|
||||
agent = Agent(
|
||||
name="Assistant",
|
||||
instructions=(
|
||||
"You are a helpful assistant. You ALWAYS write long responses, making sure to be verbose "
|
||||
"and detailed."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class GuardrailOutput(BaseModel):
|
||||
reasoning: str = Field(
|
||||
description="Reasoning about whether the response could be understood by a ten year old."
|
||||
)
|
||||
is_readable_by_ten_year_old: bool = Field(
|
||||
description="Whether the response is understandable by a ten year old."
|
||||
)
|
||||
|
||||
|
||||
guardrail_agent = Agent(
|
||||
name="Checker",
|
||||
instructions=(
|
||||
"You will be given a question and a response. Your goal is to judge whether the response "
|
||||
"is simple enough to be understood by a ten year old."
|
||||
),
|
||||
output_type=GuardrailOutput,
|
||||
model="gpt-4o-mini",
|
||||
)
|
||||
|
||||
|
||||
async def check_guardrail(text: str) -> GuardrailOutput:
|
||||
result = await Runner.run(guardrail_agent, text)
|
||||
return result.final_output_as(GuardrailOutput)
|
||||
|
||||
|
||||
async def main():
|
||||
question = "What is a black hole, and how does it behave?"
|
||||
result = Runner.run_streamed(agent, question)
|
||||
current_text = ""
|
||||
|
||||
# We will check the guardrail every N characters
|
||||
next_guardrail_check_len = 300
|
||||
guardrail_task = None
|
||||
|
||||
async for event in result.stream_events():
|
||||
if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
|
||||
print(event.data.delta, end="", flush=True)
|
||||
current_text += event.data.delta
|
||||
|
||||
# Check if it's time to run the guardrail check
|
||||
# Note that we don't run the guardrail check if there's already a task running. An
|
||||
# alternate implementation is to have N guardrails running, or cancel the previous
|
||||
# one.
|
||||
if len(current_text) >= next_guardrail_check_len and not guardrail_task:
|
||||
print("Running guardrail check")
|
||||
guardrail_task = asyncio.create_task(check_guardrail(current_text))
|
||||
next_guardrail_check_len += 300
|
||||
|
||||
# Every iteration of the loop, check if the guardrail has been triggered
|
||||
if guardrail_task and guardrail_task.done():
|
||||
guardrail_result = guardrail_task.result()
|
||||
if not guardrail_result.is_readable_by_ten_year_old:
|
||||
print("\n\n================\n\n")
|
||||
print(f"Guardrail triggered. Reasoning:\n{guardrail_result.reasoning}")
|
||||
break
|
||||
|
||||
# Do one final check on the final output
|
||||
guardrail_result = await check_guardrail(current_text)
|
||||
if not guardrail_result.is_readable_by_ten_year_old:
|
||||
print("\n\n================\n\n")
|
||||
print(f"Guardrail triggered. Reasoning:\n{guardrail_result.reasoning}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
Reference in New Issue
Block a user