chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,470 @@
|
||||
---
|
||||
title: "Agent"
|
||||
id: agent
|
||||
slug: "/agent"
|
||||
description: "The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`."
|
||||
---
|
||||
|
||||
# Agent
|
||||
|
||||
The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or user input |
|
||||
| **Mandatory init variables** | `chat_generator`: An instance of a Chat Generator that supports tools |
|
||||
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)s |
|
||||
| **Output variables** | `messages`: Chat history with tool and model responses |
|
||||
| **API reference** | [Agents](/reference/agents-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/agents/agent.py |
|
||||
| **Package name** | `haystack-ai` |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `Agent` component is a loop-based system that uses a chat-based large language model (LLM) and external tools to solve complex user queries.
|
||||
It works iteratively—calling tools, updating state, and generating prompts—until one of the configurable `exit_conditions` is met.
|
||||
|
||||
It can:
|
||||
|
||||
- Dynamically select tools based on user input,
|
||||
- Maintain and validate runtime state using a schema,
|
||||
- Stream token-level outputs from the LLM.
|
||||
|
||||
The `Agent` returns a dictionary containing:
|
||||
|
||||
- `messages`: the full conversation history,
|
||||
- `last_message`: the final `ChatMessage` from the agent,
|
||||
- Additional dynamic keys based on `state_schema`.
|
||||
|
||||
## Parameters
|
||||
|
||||
`chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional.
|
||||
|
||||
- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx).
|
||||
- `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`.
|
||||
- `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided.
|
||||
- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`.
|
||||
- `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details.
|
||||
- `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output.
|
||||
- `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`.
|
||||
- `raise_on_tool_invocation_failure`: If `True`, raises an exception when a tool call fails. If `False` (default), the error is passed back to the LLM as a message so it can recover.
|
||||
- `confirmation_strategies`: A dict mapping tool names (or tuples of tool names) to a `ConfirmationStrategy`, enabling human review of tool calls before execution. See [Human in the Loop](./human-in-the-loop.mdx).
|
||||
- `tool_invoker_kwargs`: Additional keyword arguments forwarded to the internal `ToolInvoker`.
|
||||
|
||||
### Runtime overrides
|
||||
|
||||
`run()` also accepts parameters that override the init-time configuration for a single call:
|
||||
|
||||
- `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run.
|
||||
- `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`).
|
||||
|
||||
:::info
|
||||
For the full parameter reference, see the [Agents API Documentation](/reference/agents-api).
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools import tool
|
||||
from haystack.components.agents import Agent
|
||||
from typing import Annotated
|
||||
|
||||
|
||||
@tool(outputs_to_state={"calc_result": {"source": "result"}})
|
||||
def calculator(
|
||||
expression: Annotated[str, "Math expression to evaluate, e.g. '7 * (4 + 2)'"],
|
||||
) -> dict:
|
||||
"""Evaluate basic math expressions."""
|
||||
try:
|
||||
result = eval(expression, {"__builtins__": {}})
|
||||
return {"result": result}
|
||||
except Exception as e:
|
||||
return {"error": str(e)}
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[calculator],
|
||||
system_prompt="You are a helpful assistant. Always use the calculator tool to evaluate math expressions.",
|
||||
state_schema={"calc_result": {"type": int}},
|
||||
)
|
||||
|
||||
response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
|
||||
|
||||
print(response["last_message"].text)
|
||||
print("Calc Result:", response.get("calc_result"))
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The example pipeline below creates a database assistant using `OpenAIChatGenerator`, `LinkContentFetcher`, and custom database tool.
|
||||
It reads the given URL and processes the page content, then builds a prompt for the AI.
|
||||
The assistant uses this information to write people's names and titles from the given page to the database.
|
||||
|
||||
```python
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.converters.html import HTMLToDocument
|
||||
from haystack.components.fetchers.link_content import LinkContentFetcher
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.tools import tool
|
||||
from typing import Annotated, Optional
|
||||
|
||||
document_store = InMemoryDocumentStore() # create a document store or an SQL database
|
||||
|
||||
|
||||
@tool
|
||||
def add_database_tool(
|
||||
name: Annotated[str, "First name of the person"],
|
||||
surname: Annotated[str, "Last name of the person"],
|
||||
job_title: Annotated[Optional[str], "Job title or role of the person"] = None,
|
||||
other: Annotated[Optional[str], "Any other relevant information"] = None,
|
||||
) -> str:
|
||||
"""Add a person to the database with information about them."""
|
||||
document_store.write_documents(
|
||||
[
|
||||
Document(
|
||||
content=name + " " + surname + " " + (job_title or ""),
|
||||
meta={"other": other},
|
||||
),
|
||||
],
|
||||
)
|
||||
# Returning a confirmation lets the agent know the tool call succeeded
|
||||
return f"Successfully added {name} {surname} to the database."
|
||||
|
||||
|
||||
database_assistant = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[add_database_tool],
|
||||
system_prompt="""
|
||||
You are a database assistant.
|
||||
Your task is to extract the names of people mentioned in the given context and add them to a knowledge base,
|
||||
along with additional relevant information about them that can be extracted from the context.
|
||||
Do not use your own knowledge, stay grounded to the given context.
|
||||
Do not ask the user for confirmation.
|
||||
Instead, automatically update the knowledge base and return a brief summary of the people added,
|
||||
including the information stored for each.
|
||||
""",
|
||||
)
|
||||
|
||||
extraction_agent = Pipeline()
|
||||
extraction_agent.add_component("fetcher", LinkContentFetcher())
|
||||
extraction_agent.add_component("converter", HTMLToDocument())
|
||||
extraction_agent.add_component(
|
||||
"builder",
|
||||
ChatPromptBuilder(
|
||||
template=[
|
||||
ChatMessage.from_user("""
|
||||
{% for doc in docs %}
|
||||
{{ doc.content|default|truncate(25000) }}
|
||||
{% endfor %}
|
||||
"""),
|
||||
],
|
||||
required_variables=["docs"],
|
||||
),
|
||||
)
|
||||
|
||||
extraction_agent.add_component("database_agent", database_assistant)
|
||||
extraction_agent.connect("fetcher.streams", "converter.sources")
|
||||
extraction_agent.connect("converter.documents", "builder.docs")
|
||||
extraction_agent.connect("builder", "database_agent")
|
||||
|
||||
agent_output = extraction_agent.run(
|
||||
{
|
||||
"fetcher": {
|
||||
"urls": ["https://github.com/deepset-ai/haystack/releases/tag/v2.27.0"],
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
print(agent_output["database_agent"]["last_message"].text)
|
||||
|
||||
# Inspect what was written to the document store
|
||||
written_docs = document_store.filter_documents()
|
||||
print(f"\n{len(written_docs)} people added to the database:")
|
||||
for doc in written_docs:
|
||||
print(f" - {doc.content}")
|
||||
```
|
||||
|
||||
### In YAML
|
||||
The example pipeline below fetches a webpage, converts its HTML to text, and builds a chat prompt combining the page content with a user query.
|
||||
The `Agent` then answers the question based on the provided content and can use its web search tool to find additional information if needed.
|
||||
|
||||
<details>
|
||||
<summary>View YAML</summary>
|
||||
|
||||
```yaml
|
||||
components:
|
||||
agent:
|
||||
init_parameters:
|
||||
chat_generator:
|
||||
init_parameters:
|
||||
api_base_url: null
|
||||
api_key:
|
||||
env_vars:
|
||||
- OPENAI_API_KEY
|
||||
strict: true
|
||||
type: env_var
|
||||
generation_kwargs: {}
|
||||
http_client_kwargs: null
|
||||
max_retries: null
|
||||
model: gpt-5.4-nano
|
||||
organization: null
|
||||
streaming_callback: null
|
||||
timeout: null
|
||||
tools: null
|
||||
tools_strict: false
|
||||
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
|
||||
confirmation_strategies: null
|
||||
exit_conditions:
|
||||
- text
|
||||
max_agent_steps: 5
|
||||
raise_on_tool_invocation_failure: false
|
||||
required_variables: null
|
||||
state_schema: {}
|
||||
streaming_callback: null
|
||||
system_prompt: You are a helpful assistant. Use the web search tool to find
|
||||
information when needed.
|
||||
tool_invoker_kwargs: null
|
||||
tools:
|
||||
- data:
|
||||
component:
|
||||
init_parameters:
|
||||
allowed_domains: null
|
||||
api_key:
|
||||
env_vars:
|
||||
- SERPERDEV_API_KEY
|
||||
strict: true
|
||||
type: env_var
|
||||
exclude_subdomains: false
|
||||
search_params: {}
|
||||
top_k: 3
|
||||
type: haystack.components.websearch.serper_dev.SerperDevWebSearch
|
||||
description: Search the web for current information on any topic
|
||||
inputs_from_state: null
|
||||
name: web_search
|
||||
outputs_to_state: null
|
||||
outputs_to_string: null
|
||||
parameters: null
|
||||
type: haystack.tools.component_tool.ComponentTool
|
||||
user_prompt: null
|
||||
type: haystack.components.agents.agent.Agent
|
||||
converter:
|
||||
init_parameters:
|
||||
extraction_kwargs: {}
|
||||
store_full_path: false
|
||||
type: haystack.components.converters.html.HTMLToDocument
|
||||
fetcher:
|
||||
init_parameters:
|
||||
client_kwargs:
|
||||
follow_redirects: true
|
||||
timeout: 3
|
||||
http2: false
|
||||
raise_on_failure: true
|
||||
request_headers: {}
|
||||
retry_attempts: 2
|
||||
timeout: 3
|
||||
user_agents:
|
||||
- haystack/LinkContentFetcher/2.27.0rc0
|
||||
type: haystack.components.fetchers.link_content.LinkContentFetcher
|
||||
prompt_builder:
|
||||
init_parameters:
|
||||
required_variables:
|
||||
- docs
|
||||
- query
|
||||
template:
|
||||
- content:
|
||||
- text: 'Based on the following content:
|
||||
|
||||
{% for doc in docs %}
|
||||
|
||||
{{ doc.content }}
|
||||
|
||||
{% endfor %}
|
||||
|
||||
Answer this question: {{ query }}'
|
||||
meta: {}
|
||||
name: null
|
||||
role: user
|
||||
variables: null
|
||||
type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
|
||||
connection_type_validation: true
|
||||
connections:
|
||||
- receiver: converter.sources
|
||||
sender: fetcher.streams
|
||||
- receiver: prompt_builder.docs
|
||||
sender: converter.documents
|
||||
- receiver: agent.messages
|
||||
sender: prompt_builder.prompt
|
||||
max_runs_per_component: 100
|
||||
metadata: {}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Streaming
|
||||
|
||||
You can stream output as it's generated. Pass a callback to `streaming_callback`.
|
||||
Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
|
||||
|
||||
```python
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[...],
|
||||
system_prompt="...",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
)
|
||||
```
|
||||
|
||||
See our [Streaming Support](../generators/guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
|
||||
|
||||
Give preference to `print_streaming_chunk` by default.
|
||||
Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
|
||||
|
||||
## Multimodal Inputs
|
||||
|
||||
Agents support multimodal inputs when paired with a vision-capable model such as `gpt-5` (OpenAI) or `gemini-2.5-flash` (Google).
|
||||
Pass images alongside text by including `ImageContent` objects in the `content_parts` of a `ChatMessage`:
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import ChatMessage, ImageContent
|
||||
|
||||
image = ImageContent.from_url("https://example.com/chart.png")
|
||||
result = agent.run(
|
||||
messages=[
|
||||
ChatMessage.from_user(content_parts=["What does this chart show?", image]),
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
Tools can also return `ImageContent` directly, letting the agent fetch and reason about images dynamically during its loop.
|
||||
Two things are required: set `outputs_to_string={"raw_result": True}` so the `ToolInvoker` skips string conversion, and return a `list[ImageContent]` (the tool result type is `str | Sequence[TextContent | ImageContent]`).
|
||||
|
||||
The standard Chat Completions API doesn't support images in tool results — use `OpenAIResponsesChatGenerator` (OpenAI's Responses API) instead:
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage, ImageContent
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool(outputs_to_string={"raw_result": True})
|
||||
def fetch_image(
|
||||
url: Annotated[str, "URL of the image to fetch and analyze"],
|
||||
) -> list[ImageContent]:
|
||||
"""Fetch an image from a URL so the agent can analyze its contents."""
|
||||
return [ImageContent.from_url(url)]
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5"),
|
||||
tools=[fetch_image],
|
||||
system_prompt="You are a helpful assistant that can fetch and analyze images from URLs.",
|
||||
)
|
||||
|
||||
result = agent.run(
|
||||
messages=[
|
||||
ChatMessage.from_user(
|
||||
"Fetch the image at https://picsum.photos/seed/haystack/640/480 and describe what you see.",
|
||||
),
|
||||
],
|
||||
)
|
||||
print(result["last_message"].text)
|
||||
```
|
||||
|
||||
`ImageContent` can be created from a URL, a local file path, or a PDF page using the `PDFToImageContent` converter.
|
||||
|
||||
### In a pipeline
|
||||
|
||||
When an `Agent` sits inside a pipeline, use `ChatPromptBuilder` with its string template format and the `| templatize_part` filter to pass images as structured content parts:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ImageContent
|
||||
|
||||
template = """
|
||||
{% message role="user" %}
|
||||
{{ question }}
|
||||
{{ image | templatize_part }}
|
||||
{% endmessage %}
|
||||
"""
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5"),
|
||||
system_prompt="You are a helpful assistant that can analyze images.",
|
||||
)
|
||||
prompt_builder = ChatPromptBuilder(
|
||||
template=template,
|
||||
required_variables=["question", "image"],
|
||||
)
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("prompt_builder", prompt_builder)
|
||||
pipeline.add_component("agent", agent)
|
||||
pipeline.connect("prompt_builder.prompt", "agent.messages")
|
||||
|
||||
# Download or provide your own chart image as "chart.png"
|
||||
image = ImageContent.from_file_path("chart.png")
|
||||
result = pipeline.run(
|
||||
{
|
||||
"prompt_builder": {"question": "What does this chart show?", "image": image},
|
||||
},
|
||||
)
|
||||
print(result["agent"]["last_message"].text)
|
||||
```
|
||||
|
||||
:::tip
|
||||
See these cookbooks for complete multimodal agent examples:
|
||||
- [Multimodal Agents](https://haystack.deepset.ai/cookbook/multimodal_intro#multimodal-agent) — image inputs and tool use with agents
|
||||
- [Gemma Chat RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag) — vision model in a RAG pipeline
|
||||
:::
|
||||
|
||||
## Multi-Agent Systems
|
||||
|
||||
You can wrap an `Agent` as a tool to build multi-agent systems where specialist agents handle focused subtasks and a coordinator agent plans and delegates.
|
||||
|
||||
See [Multi-Agent Systems](../../concepts/agents/multi-agent-systems.mdx) for a full guide, including the recommended `@tool` decorator approach for full interface control and `ComponentTool` for declarative configuration.
|
||||
|
||||
## MCP Integration
|
||||
|
||||
Agents work with MCP in two directions:
|
||||
|
||||
- **Consuming MCP tools**: Pass `MCPTool` or `MCPToolset` instances in the `tools` list to call tools on any MCP-compatible server (filesystem, browser, databases, and more). See [MCPTool](../../tools/mcptool.mdx) and [MCPToolset](../../tools/mcptoolset.mdx).
|
||||
- **Exposing as an MCP server**: Use [Hayhooks](../../development/hayhooks.mdx) to deploy your agent and expose it as an MCP server, making it callable from any MCP-compatible client such as Claude Desktop or Cursor.
|
||||
|
||||
## Additional References
|
||||
|
||||
📖 Related docs:
|
||||
|
||||
- [State](./state.mdx) — managing shared data between tools
|
||||
- [Human in the Loop](./human-in-the-loop.mdx) — intercepting tool calls for human review
|
||||
|
||||
📚 Tutorials:
|
||||
|
||||
- [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)
|
||||
- [Creating a Multi-Agent System](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)
|
||||
- [Human-in-the-Loop with Haystack Agents](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent/)
|
||||
|
||||
🧑🍳 Cookbook:
|
||||
|
||||
- [Build a GitHub Issue Resolver Agent](https://haystack.deepset.ai/cookbook/github_issue_resolver_agent)
|
||||
- [Multimodal Agents](https://haystack.deepset.ai/cookbook/multimodal_intro#multimodal-agent)
|
||||
- [Gemma Chat RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag)
|
||||
+305
@@ -0,0 +1,305 @@
|
||||
---
|
||||
title: "Human in the Loop"
|
||||
id: human-in-the-loop
|
||||
slug: "/human-in-the-loop"
|
||||
description: "Human-in-the-loop allows you to intercept agent tool calls before execution, letting a human confirm, reject, or modify the tool parameters."
|
||||
---
|
||||
|
||||
# Human in the Loop
|
||||
|
||||
Human-in-the-loop (HITL) lets you intercept an agent's tool calls before they are executed.
|
||||
A human can **confirm**, **reject**, or **modify** the parameters of each tool call in real time.
|
||||
This is useful for high-stakes operations - such as sending emails, modifying databases, or making API calls - where you want a human to review the action first.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Configured on** | The [`Agent`](./agent.mdx) component via `confirmation_strategies` |
|
||||
| **Key classes** | `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` |
|
||||
| **Import path** | `haystack.human_in_the_loop` |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/human_in_the_loop/ |
|
||||
| **Package name** | `haystack-ai` |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The HITL system is composed of three layers:
|
||||
|
||||
- **Strategy** - decides what to do when a tool is about to be called. The built-in `BlockingConfirmationStrategy` pauses execution and asks a human.
|
||||
- **Policy** - decides *when* to ask. Built-in policies: `AlwaysAskPolicy`, `NeverAskPolicy`, `AskOncePolicy`.
|
||||
- **UI** - the interface used to ask the human. Built-in UIs: `RichConsoleUI` (requires `rich`) and `SimpleConsoleUI` (stdlib only).
|
||||
|
||||
When the agent is about to invoke a tool, the strategy checks the policy.
|
||||
If the policy says to ask, the UI prompts the human with the tool name, description, and parameters. The human can:
|
||||
|
||||
- **Confirm** (`y`) - execute as-is
|
||||
- **Reject** (`n`) - skip execution and feed rejection feedback back to the LLM
|
||||
- **Modify** (`m`) - edit the parameters before execution
|
||||
|
||||
The agent then continues with the human's decision.
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic setup
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.human_in_the_loop import (
|
||||
AlwaysAskPolicy,
|
||||
BlockingConfirmationStrategy,
|
||||
SimpleConsoleUI,
|
||||
)
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool
|
||||
def send_email(
|
||||
to: Annotated[str, "The recipient email address"],
|
||||
subject: Annotated[str, "The email subject line"],
|
||||
body: Annotated[str, "The email body"],
|
||||
) -> str:
|
||||
"""Send an email to a recipient."""
|
||||
return f"Email sent to {to}."
|
||||
|
||||
|
||||
strategy = BlockingConfirmationStrategy(
|
||||
confirmation_policy=AlwaysAskPolicy(),
|
||||
confirmation_ui=SimpleConsoleUI(),
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
|
||||
tools=[send_email],
|
||||
confirmation_strategies={"send_email": strategy},
|
||||
)
|
||||
|
||||
result = agent.run(
|
||||
messages=[ChatMessage.from_user("Send a welcome email to alice@example.com")],
|
||||
)
|
||||
```
|
||||
|
||||
When the agent calls `send_email`, the terminal will pause and show:
|
||||
|
||||
```
|
||||
--- Tool Execution Request ---
|
||||
Tool: send_email
|
||||
Description: Send an email to a recipient.
|
||||
Arguments:
|
||||
to: alice@example.com
|
||||
subject: Welcome!
|
||||
body: Hi Alice, welcome aboard!
|
||||
------------------------------
|
||||
Confirm execution? (y=confirm / n=reject / m=modify):
|
||||
```
|
||||
|
||||
### Using RichConsoleUI
|
||||
|
||||
`RichConsoleUI` provides a styled terminal prompt using the [`rich`](https://github.com/Textualize/rich) library:
|
||||
|
||||
```shell
|
||||
pip install rich
|
||||
```
|
||||
|
||||
```python
|
||||
from haystack.human_in_the_loop import RichConsoleUI
|
||||
|
||||
strategy = BlockingConfirmationStrategy(
|
||||
confirmation_policy=AlwaysAskPolicy(),
|
||||
confirmation_ui=RichConsoleUI(),
|
||||
)
|
||||
```
|
||||
|
||||
### Applying strategies to multiple tools
|
||||
|
||||
You can configure different strategies per tool, or share one strategy across a group of tools using a tuple key:
|
||||
|
||||
```python
|
||||
@tool
|
||||
def delete_record(record_id: Annotated[str, "The ID of the record to delete"]) -> str:
|
||||
"""Delete a record from the database."""
|
||||
return f"Record {record_id} deleted."
|
||||
|
||||
|
||||
@tool
|
||||
def update_record(
|
||||
record_id: Annotated[str, "The ID of the record to update"],
|
||||
data: Annotated[str, "The new data as a JSON string"],
|
||||
) -> str:
|
||||
"""Update a record in the database."""
|
||||
return f"Record {record_id} updated."
|
||||
|
||||
|
||||
@tool
|
||||
def search(query: Annotated[str, "The search query"]) -> str:
|
||||
"""Search the knowledge base."""
|
||||
return f"Results for: {query}"
|
||||
|
||||
|
||||
ask_strategy = BlockingConfirmationStrategy(
|
||||
confirmation_policy=AlwaysAskPolicy(),
|
||||
confirmation_ui=SimpleConsoleUI(),
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
|
||||
tools=[send_email, delete_record, update_record, search],
|
||||
confirmation_strategies={
|
||||
# Share one strategy across multiple sensitive tools using a tuple key
|
||||
("send_email", "delete_record", "update_record"): ask_strategy,
|
||||
# search has no strategy - always executes without asking
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
### Customizing feedback messages
|
||||
|
||||
When a tool call is rejected or modified, `BlockingConfirmationStrategy` sends a message back to the LLM explaining what happened. Three optional template parameters control these messages — each has a sensible default, so you only need to set them if you want different wording:
|
||||
|
||||
- `reject_template`: Sent to the LLM when the user rejects a tool call. Must include a `{tool_name}` placeholder. Default: `"Tool execution for '{tool_name}' was rejected by the user."`
|
||||
- `modify_template`: Sent when the user modifies the parameters. Must include `{tool_name}` and `{final_tool_params}` placeholders. Default: `"The parameters for tool '{tool_name}' were updated by the user to:\n{final_tool_params}"`
|
||||
- `user_feedback_template`: Appends the user's optional free-text feedback to either message. Must include a `{feedback}` placeholder. Default: `"With user feedback: {feedback}"`
|
||||
|
||||
```python
|
||||
strategy = BlockingConfirmationStrategy(
|
||||
confirmation_policy=AlwaysAskPolicy(),
|
||||
confirmation_ui=SimpleConsoleUI(),
|
||||
reject_template="Skipping '{tool_name}' — rejected by operator.",
|
||||
modify_template="Updated parameters for '{tool_name}': {final_tool_params}",
|
||||
user_feedback_template="Reason: {feedback}",
|
||||
)
|
||||
```
|
||||
|
||||
## Policies
|
||||
|
||||
Policies control *when* the human is asked.
|
||||
|
||||
| Policy | Behavior |
|
||||
| --- | --- |
|
||||
| `AlwaysAskPolicy` | Ask every time the tool is called |
|
||||
| `NeverAskPolicy` | Never ask - always proceed (useful for toggling HITL off without removing the strategy) |
|
||||
| `AskOncePolicy` | Ask once per unique `(tool_name, parameters)` combination. Remembers confirmed calls and skips asking on repeats. |
|
||||
|
||||
### Custom policy
|
||||
|
||||
You can implement your own policy by subclassing `ConfirmationPolicy` from `haystack.human_in_the_loop.types`:
|
||||
|
||||
```python
|
||||
from haystack.human_in_the_loop.types import ConfirmationPolicy, ConfirmationUIResult
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AskForSensitiveParamsPolicy(ConfirmationPolicy):
|
||||
"""Only ask when the 'to' parameter looks like an external email domain."""
|
||||
|
||||
def should_ask(
|
||||
self,
|
||||
tool_name: str,
|
||||
tool_description: str,
|
||||
tool_params: dict[str, Any],
|
||||
) -> bool:
|
||||
to = tool_params.get("to", "")
|
||||
return not to.endswith("@mycompany.com")
|
||||
```
|
||||
|
||||
For stateful policies, also implement `update_after_confirmation`.
|
||||
It is called after the user responds and receives the full `ConfirmationUIResult`, letting you update internal state based on the outcome.
|
||||
The following policy asks once per tool name and skips re-asking for any tool the user has already confirmed:
|
||||
|
||||
```python
|
||||
from haystack.human_in_the_loop.types import ConfirmationPolicy
|
||||
from haystack.human_in_the_loop import ConfirmationUIResult
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AskOncePerToolPolicy(ConfirmationPolicy):
|
||||
"""Ask once per tool name, regardless of parameters. Skip on repeat confirmed calls."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._confirmed_tools: set[str] = set()
|
||||
|
||||
def should_ask(
|
||||
self,
|
||||
tool_name: str,
|
||||
tool_description: str,
|
||||
tool_params: dict[str, Any],
|
||||
) -> bool:
|
||||
return tool_name not in self._confirmed_tools
|
||||
|
||||
def update_after_confirmation(
|
||||
self,
|
||||
tool_name: str,
|
||||
tool_description: str,
|
||||
tool_params: dict[str, Any],
|
||||
confirmation_result: ConfirmationUIResult,
|
||||
) -> None:
|
||||
if confirmation_result.action == "confirm":
|
||||
self._confirmed_tools.add(tool_name)
|
||||
```
|
||||
|
||||
## Dataclasses
|
||||
|
||||
### `ConfirmationUIResult`
|
||||
|
||||
Returned by the UI after the human responds.
|
||||
|
||||
| Field | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| `action` | `str` | `"confirm"`, `"reject"`, or `"modify"` |
|
||||
| `feedback` | `str \| None` | Optional free-text feedback from the human |
|
||||
| `new_tool_params` | `dict \| None` | Replacement parameters when action is `"modify"` |
|
||||
|
||||
### `ToolExecutionDecision`
|
||||
|
||||
Returned by the strategy to the agent.
|
||||
|
||||
| Field | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| `tool_name` | `str` | Name of the tool |
|
||||
| `execute` | `bool` | Whether to execute the tool |
|
||||
| `tool_call_id` | `str \| None` | ID of the tool call |
|
||||
| `feedback` | `str \| None` | Feedback message passed back to the LLM on rejection or modification |
|
||||
| `final_tool_params` | `dict \| None` | Final parameters to use for execution |
|
||||
|
||||
## Example: HITL with Hayhooks and Open WebUI
|
||||
|
||||
The [hitl-hayhooks-redis-openwebui](https://github.com/deepset-ai/hitl-hayhooks-redis-openwebui) repository shows a full production-style HITL setup using a Haystack Agent served via [Hayhooks](https://github.com/deepset-ai/hayhooks) with approval dialogs rendered in [Open WebUI](https://github.com/open-webui/open-webui).
|
||||
|
||||
The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that uses `confirmation_strategy_context` to pass per-request resources - a Redis client and an async event queue - into the strategy at runtime:
|
||||
|
||||
- When a tool call is about to execute, the strategy emits a `tool_call_start` SSE event and blocks on `Redis BLPOP` waiting for an approval decision.
|
||||
- The Open WebUI Pipe function receives the SSE event, shows the user a confirmation dialog, then writes `approved` or `rejected` to Redis via `LPUSH`.
|
||||
- Once Redis unblocks, the strategy returns a `ToolExecutionDecision` and the agent continues.
|
||||
|
||||
This is a good reference if you need non-blocking HITL in a web or server environment where `SimpleConsoleUI` and `RichConsoleUI` are not suitable.
|
||||
|
||||
## Custom UI
|
||||
|
||||
Implement `ConfirmationUI` from `haystack.human_in_the_loop.types` to build your own interface - for example, a web-based approval queue:
|
||||
|
||||
```python
|
||||
from haystack.human_in_the_loop.types import ConfirmationUI
|
||||
from haystack.human_in_the_loop import ConfirmationUIResult
|
||||
from typing import Any
|
||||
|
||||
|
||||
class WebhookApprovalUI(ConfirmationUI):
|
||||
"""Sends a webhook and waits for an async approval response."""
|
||||
|
||||
def get_user_confirmation(
|
||||
self,
|
||||
tool_name: str,
|
||||
tool_description: str,
|
||||
tool_params: dict[str, Any],
|
||||
) -> ConfirmationUIResult:
|
||||
# Send approval request to your system and wait for response
|
||||
response = send_approval_request_and_wait(tool_name, tool_params)
|
||||
return ConfirmationUIResult(
|
||||
action=response["action"],
|
||||
feedback=response.get("feedback"),
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,415 @@
|
||||
---
|
||||
title: "State"
|
||||
id: state
|
||||
slug: "/state"
|
||||
description: "`State` is a container for storing shared information during Agent and Tool execution. It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer."
|
||||
---
|
||||
|
||||
# State
|
||||
|
||||
`State` is a container for storing shared information during Agent and Tool execution.
|
||||
It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer.
|
||||
|
||||
## Overview
|
||||
|
||||
When building agents that use multiple tools, you often need tools to share information or accumulate results across iterations.
|
||||
State provides centralized storage that all tools can read from and write to.
|
||||
For example, a search tool called multiple times can append its results to a shared `documents` list, which is then returned alongside the agent's final answer for source inspection.
|
||||
|
||||
State uses a schema-based approach where you define:
|
||||
|
||||
- What data can be stored,
|
||||
- The type of each piece of data,
|
||||
- How values are merged when updated.
|
||||
|
||||
The Agent creates and manages the `State` object internally. You shouldn't need to instantiate it directly.
|
||||
You interact with it through tool definitions (`inputs_from_state`, `outputs_to_state`, or a `state: State` parameter) and read results from the agent's output dict.
|
||||
|
||||
### Supported Types
|
||||
|
||||
State supports standard Python types:
|
||||
|
||||
- Basic types: `str`, `int`, `float`, `bool`, `dict`
|
||||
- List types: `list`, `list[str]`, `list[int]`, `list[Document]`
|
||||
- Union types: `str | int`, `str | None`
|
||||
- Custom classes and data classes.
|
||||
|
||||
### Automatic Message Handling
|
||||
|
||||
State automatically includes a `messages` field that stores the full conversation history during execution.
|
||||
You don't need to define this in your schema.
|
||||
It uses `list[ChatMessage]` type with the `merge_lists` handler, so new messages are appended on each iteration.
|
||||
|
||||
### State API
|
||||
|
||||
| Method | Description |
|
||||
| --- | --- |
|
||||
| `state.get(key, default=None)` | Read a value; returns `default` if the key doesn't exist |
|
||||
| `state.set(key, value)` | Write a value, merged using the schema's handler |
|
||||
| `state.has(key)` | Returns `True` if the key exists in state |
|
||||
| `state.data` | Returns a snapshot of all current state as a `dict` |
|
||||
|
||||
## Schema Definition
|
||||
|
||||
The schema defines what data can be stored and how values are updated. Each schema entry consists of:
|
||||
|
||||
- `type` (required): The Python type for this field (for example, `str`, `int`, `list`)
|
||||
- `handler` (optional): A callable that determines how new values are merged when `set()` is called
|
||||
|
||||
```python
|
||||
{
|
||||
"parameter_name": {
|
||||
"type": SomeType, # Required: expected Python type
|
||||
"handler": some_func, # Optional: merge function
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
If you don't specify a handler, State automatically assigns a default based on the type.
|
||||
|
||||
### Default Handlers
|
||||
|
||||
State provides two built-in merge behaviors (importable from `haystack.components.agents.state`):
|
||||
|
||||
- **`merge_lists`**: Appends to the existing list (default for list types)
|
||||
- **`replace_values`**: Overwrites the existing value (default for non-list types)
|
||||
|
||||
```python
|
||||
from haystack.components.agents import State
|
||||
|
||||
schema = {
|
||||
"documents": {"type": list}, # uses merge_lists by default
|
||||
"user_name": {"type": str}, # uses replace_values by default
|
||||
}
|
||||
|
||||
state = State(schema=schema)
|
||||
|
||||
state.set("documents", [1, 2])
|
||||
state.set("documents", [3, 4])
|
||||
print(state.get("documents")) # [1, 2, 3, 4]
|
||||
|
||||
state.set("user_name", "Alice")
|
||||
state.set("user_name", "Bob")
|
||||
print(state.get("user_name")) # "Bob"
|
||||
```
|
||||
|
||||
### Custom Handlers
|
||||
|
||||
Custom handlers are useful when the default `merge_lists` or `replace_values` behaviors don't fit your needs.
|
||||
A handler takes the current state value and the new value and returns the merged result.
|
||||
|
||||
The example below uses a deduplication handler, useful when multiple tool calls might return overlapping results and you want to avoid accumulating duplicates in state:
|
||||
|
||||
```python
|
||||
def deduplicate(current_value: list | None, new_value: list) -> list:
|
||||
"""Append new items, skipping any already in the list."""
|
||||
existing = set(current_value or [])
|
||||
return (current_value or []) + [item for item in new_value if item not in existing]
|
||||
|
||||
|
||||
schema = {"doc_ids": {"type": list, "handler": deduplicate}}
|
||||
|
||||
state = State(schema=schema)
|
||||
state.set("doc_ids", ["doc-1", "doc-2"])
|
||||
state.set("doc_ids", ["doc-2", "doc-3"])
|
||||
print(state.get("doc_ids")) # ["doc-1", "doc-2", "doc-3"]
|
||||
```
|
||||
|
||||
You can also override the handler for a single `set()` call:
|
||||
|
||||
```python
|
||||
from haystack.components.agents import State
|
||||
|
||||
|
||||
def concatenate_strings(current: str | None, new: str) -> str:
|
||||
return f"{current}-{new}" if current else new
|
||||
|
||||
|
||||
state = State(schema={"user_name": {"type": str}})
|
||||
state.set("user_name", "Alice")
|
||||
state.set("user_name", "Bob", handler_override=concatenate_strings)
|
||||
print(state.get("user_name")) # "Alice-Bob"
|
||||
```
|
||||
|
||||
## Using State
|
||||
|
||||
Define a `state_schema` when creating the Agent.
|
||||
State keys declared in `state_schema` are exposed as output keys on the agent's result dict alongside `messages` and `last_message`.
|
||||
|
||||
Tools interact with State through three mechanisms:
|
||||
|
||||
- **`outputs_to_state`**: Write tool results to state keys after the tool runs.
|
||||
- **`inputs_from_state`**: Inject state values into tool parameters before the tool runs.
|
||||
- **Direct `State` injection**: Add a `state: State` parameter to your tool function's signature. The Agent detects the `State` annotation and injects the live `State` object automatically, so you can read or write any key defined in the schema. The `State` object is never exposed to the LLM's parameter schema.
|
||||
|
||||
### Reading from State: `inputs_from_state`
|
||||
|
||||
`inputs_from_state` maps state keys to function parameter names using the format `{"state_key": "param_name"}`.
|
||||
The value is injected from state before the tool runs, so the LLM never needs to provide it.
|
||||
|
||||
Parameters mapped via `inputs_from_state` are automatically excluded from the LLM's parameter schema.
|
||||
The model never sees or provides them:
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool(inputs_from_state={"user_name": "user_context"})
|
||||
def search_documents(
|
||||
query: Annotated[str, "The search query"],
|
||||
user_context: str, # injected from state; excluded from LLM schema
|
||||
) -> dict:
|
||||
"""Search documents using query and user context."""
|
||||
return {"results": [f"Found results for '{query}' (user: {user_context})"]}
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[search_documents],
|
||||
system_prompt="Use the search_documents tool to find information.",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
state_schema={"user_name": {"type": str}},
|
||||
)
|
||||
|
||||
result = agent.run(
|
||||
messages=[ChatMessage.from_user("Search for Python tutorials")],
|
||||
user_name="Alice", # state key "user_name" is pre-populated by passing user_name= to agent.run()
|
||||
)
|
||||
|
||||
print(result["last_message"].text)
|
||||
```
|
||||
|
||||
### Writing to State: `outputs_to_state`
|
||||
|
||||
The `outputs_to_state` parameter maps tool output keys to state keys. Each entry supports two optional fields:
|
||||
|
||||
```python
|
||||
{
|
||||
"state_key": {
|
||||
"source": "tool_output_key", # which key to read from the tool's return dict; omit to store the entire dict
|
||||
"handler": some_func, # override the schema's merge handler for this mapping only
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool(
|
||||
outputs_to_state={
|
||||
"documents": {"source": "documents"},
|
||||
"result_count": {"source": "count"},
|
||||
"last_query": {"source": "query"},
|
||||
},
|
||||
)
|
||||
def retrieve_documents(
|
||||
query: Annotated[str, "The search query"],
|
||||
) -> dict:
|
||||
"""Retrieve relevant documents."""
|
||||
return {
|
||||
"documents": [
|
||||
{"title": "Doc 1", "content": "Content about Python"},
|
||||
{"title": "Doc 2", "content": "More about Python"},
|
||||
],
|
||||
"count": 2,
|
||||
"query": query,
|
||||
}
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[retrieve_documents],
|
||||
system_prompt="Use the retrieve_documents tool to find information.",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
state_schema={
|
||||
"documents": {"type": list},
|
||||
"result_count": {"type": int},
|
||||
"last_query": {"type": str},
|
||||
},
|
||||
)
|
||||
|
||||
result = agent.run(messages=[ChatMessage.from_user("Find information about Python")])
|
||||
|
||||
print(f"Documents: {result['documents']}")
|
||||
print(f"Result count: {result['result_count']}")
|
||||
print(f"Last query: {result['last_query']}")
|
||||
```
|
||||
|
||||
If you omit `source`, the entire tool result dict is stored under the state key:
|
||||
|
||||
```python
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool(outputs_to_state={"user_info": {}})
|
||||
def get_user_info() -> dict:
|
||||
"""Get user information."""
|
||||
return {"name": "Alice", "email": "alice@example.com", "role": "admin"}
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[get_user_info],
|
||||
system_prompt="Use the get_user_info tool to look up user details.",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
state_schema={"user_info": {"type": dict}},
|
||||
)
|
||||
|
||||
result = agent.run(messages=[ChatMessage.from_user("What are the user's details?")])
|
||||
|
||||
print(result["last_message"].text)
|
||||
print(f"User info: {result['user_info']}")
|
||||
```
|
||||
|
||||
### Combining Inputs and Outputs
|
||||
|
||||
Tools can both read from and write to State, enabling tool chaining across iterations.
|
||||
This example builds on `retrieve_documents` from the previous section:
|
||||
|
||||
```python
|
||||
@tool(
|
||||
inputs_from_state={"documents": "documents"},
|
||||
outputs_to_state={
|
||||
"final_docs": {"source": "processed_docs"},
|
||||
"final_count": {"source": "processed_count"},
|
||||
},
|
||||
)
|
||||
def process_documents(
|
||||
max_results: Annotated[int, "Maximum number of documents to return"],
|
||||
documents: list = None, # injected from state; LLM does not provide this
|
||||
) -> dict:
|
||||
"""Process retrieved documents and return a filtered subset."""
|
||||
processed = (documents or [])[:max_results]
|
||||
return {"processed_docs": processed, "processed_count": len(processed)}
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[retrieve_documents, process_documents], # chained through state
|
||||
system_prompt="Use the available tools to retrieve and process documents.",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
state_schema={
|
||||
"documents": {"type": list},
|
||||
"result_count": {"type": int},
|
||||
"last_query": {"type": str},
|
||||
"final_docs": {"type": list},
|
||||
"final_count": {"type": int},
|
||||
},
|
||||
)
|
||||
|
||||
result = agent.run(
|
||||
messages=[ChatMessage.from_user("Find and process 3 documents about Python")],
|
||||
)
|
||||
print(f"Processed {result['final_count']} documents")
|
||||
```
|
||||
|
||||
### Injecting State Directly into Tools
|
||||
|
||||
As an alternative to `inputs_from_state` and `outputs_to_state`, a tool can declare a `state: State` parameter to receive the live `State` object at invocation time.
|
||||
This lets the tool read from and write to any number of state keys without declaring mappings upfront.
|
||||
|
||||
The ToolInvoker detects the `State` annotation and injects the object automatically.
|
||||
It is excluded from the LLM-facing schema. The model is never asked to supply it.
|
||||
Both `State` and `State | None` annotations are supported.
|
||||
|
||||
For function-based tools, add the `state` parameter and use the `@tool` decorator:
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from haystack.components.agents import Agent, State
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.dataclasses import ChatMessage, Document
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool
|
||||
def retrieve_and_store(
|
||||
query: Annotated[str, "The search query"],
|
||||
state: State,
|
||||
) -> str:
|
||||
"""Retrieve documents and store them directly in state."""
|
||||
documents = [Document(content=f"Result for '{query}'")]
|
||||
state.set("documents", documents)
|
||||
user_name = state.get("user_name", "unknown")
|
||||
return f"Retrieved {len(documents)} document(s) for {user_name}"
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[retrieve_and_store],
|
||||
system_prompt="Use the retrieve_and_store tool to find documents.",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
state_schema={"documents": {"type": list[Document]}, "user_name": {"type": str}},
|
||||
)
|
||||
|
||||
result = agent.run(
|
||||
messages=[ChatMessage.from_user("Find documents about Python")],
|
||||
user_name="Alice",
|
||||
)
|
||||
|
||||
print(result["last_message"].text)
|
||||
print(result["documents"])
|
||||
```
|
||||
|
||||
For component-based tools, declare a `State` input socket on the `run` method and wrap it with `ComponentTool`:
|
||||
|
||||
```python
|
||||
from haystack import component
|
||||
from haystack.components.agents import Agent, State
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.dataclasses import ChatMessage, Document
|
||||
from haystack.tools import ComponentTool
|
||||
|
||||
|
||||
@component
|
||||
class DocumentRetriever:
|
||||
"""Retrieve documents and store them in state."""
|
||||
|
||||
@component.output_types(reply=str)
|
||||
def run(self, query: str, state: State) -> dict[str, str]:
|
||||
"""
|
||||
Retrieve documents based on query and store them in state.
|
||||
|
||||
:param query: The search query
|
||||
"""
|
||||
documents = [Document(content=f"Result for '{query}'")]
|
||||
state.set("documents", documents)
|
||||
return {"reply": f"Retrieved {len(documents)} document(s)"}
|
||||
|
||||
|
||||
retriever_tool = ComponentTool(
|
||||
component=DocumentRetriever(),
|
||||
name="retrieve",
|
||||
description="Retrieve documents based on a search query",
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[retriever_tool],
|
||||
system_prompt="Use the retrieve tool to find documents.",
|
||||
streaming_callback=print_streaming_chunk,
|
||||
state_schema={"documents": {"type": list[Document]}},
|
||||
)
|
||||
|
||||
result = agent.run(messages=[ChatMessage.from_user("Find documents about Python")])
|
||||
|
||||
print(result["last_message"].text)
|
||||
print(result["documents"])
|
||||
```
|
||||
Reference in New Issue
Block a user