chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:22:28 +08:00
commit c56bef871b
9296 changed files with 1854228 additions and 0 deletions
@@ -0,0 +1,470 @@
---
title: "Agent"
id: agent
slug: "/agent"
description: "The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`."
---
# Agent
The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`.
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or user input |
| **Mandatory init variables** | `chat_generator`: An instance of a Chat Generator that supports tools |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)s |
| **Output variables** | `messages`: Chat history with tool and model responses |
| **API reference** | [Agents](/reference/agents-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/agents/agent.py |
| **Package name** | `haystack-ai` |
</div>
## Overview
The `Agent` component is a loop-based system that uses a chat-based large language model (LLM) and external tools to solve complex user queries.
It works iteratively—calling tools, updating state, and generating prompts—until one of the configurable `exit_conditions` is met.
It can:
- Dynamically select tools based on user input,
- Maintain and validate runtime state using a schema,
- Stream token-level outputs from the LLM.
The `Agent` returns a dictionary containing:
- `messages`: the full conversation history,
- `last_message`: the final `ChatMessage` from the agent,
- Additional dynamic keys based on `state_schema`.
## Parameters
`chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional.
- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx).
- `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`.
- `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided.
- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`.
- `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details.
- `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output.
- `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`.
- `raise_on_tool_invocation_failure`: If `True`, raises an exception when a tool call fails. If `False` (default), the error is passed back to the LLM as a message so it can recover.
- `confirmation_strategies`: A dict mapping tool names (or tuples of tool names) to a `ConfirmationStrategy`, enabling human review of tool calls before execution. See [Human in the Loop](./human-in-the-loop.mdx).
- `tool_invoker_kwargs`: Additional keyword arguments forwarded to the internal `ToolInvoker`.
### Runtime overrides
`run()` also accepts parameters that override the init-time configuration for a single call:
- `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run.
- `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`).
:::info
For the full parameter reference, see the [Agents API Documentation](/reference/agents-api).
:::
## Usage
### On its own
```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
from haystack.components.agents import Agent
from typing import Annotated
@tool(outputs_to_state={"calc_result": {"source": "result"}})
def calculator(
expression: Annotated[str, "Math expression to evaluate, e.g. '7 * (4 + 2)'"],
) -> dict:
"""Evaluate basic math expressions."""
try:
result = eval(expression, {"__builtins__": {}})
return {"result": result}
except Exception as e:
return {"error": str(e)}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[calculator],
system_prompt="You are a helpful assistant. Always use the calculator tool to evaluate math expressions.",
state_schema={"calc_result": {"type": int}},
)
response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
print(response["last_message"].text)
print("Calc Result:", response.get("calc_result"))
```
### In a pipeline
The example pipeline below creates a database assistant using `OpenAIChatGenerator`, `LinkContentFetcher`, and custom database tool.
It reads the given URL and processes the page content, then builds a prompt for the AI.
The assistant uses this information to write people's names and titles from the given page to the database.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.converters.html import HTMLToDocument
from haystack.components.fetchers.link_content import LinkContentFetcher
from haystack import Document, Pipeline
from haystack.dataclasses import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.tools import tool
from typing import Annotated, Optional
document_store = InMemoryDocumentStore() # create a document store or an SQL database
@tool
def add_database_tool(
name: Annotated[str, "First name of the person"],
surname: Annotated[str, "Last name of the person"],
job_title: Annotated[Optional[str], "Job title or role of the person"] = None,
other: Annotated[Optional[str], "Any other relevant information"] = None,
) -> str:
"""Add a person to the database with information about them."""
document_store.write_documents(
[
Document(
content=name + " " + surname + " " + (job_title or ""),
meta={"other": other},
),
],
)
# Returning a confirmation lets the agent know the tool call succeeded
return f"Successfully added {name} {surname} to the database."
database_assistant = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[add_database_tool],
system_prompt="""
You are a database assistant.
Your task is to extract the names of people mentioned in the given context and add them to a knowledge base,
along with additional relevant information about them that can be extracted from the context.
Do not use your own knowledge, stay grounded to the given context.
Do not ask the user for confirmation.
Instead, automatically update the knowledge base and return a brief summary of the people added,
including the information stored for each.
""",
)
extraction_agent = Pipeline()
extraction_agent.add_component("fetcher", LinkContentFetcher())
extraction_agent.add_component("converter", HTMLToDocument())
extraction_agent.add_component(
"builder",
ChatPromptBuilder(
template=[
ChatMessage.from_user("""
{% for doc in docs %}
{{ doc.content|default|truncate(25000) }}
{% endfor %}
"""),
],
required_variables=["docs"],
),
)
extraction_agent.add_component("database_agent", database_assistant)
extraction_agent.connect("fetcher.streams", "converter.sources")
extraction_agent.connect("converter.documents", "builder.docs")
extraction_agent.connect("builder", "database_agent")
agent_output = extraction_agent.run(
{
"fetcher": {
"urls": ["https://github.com/deepset-ai/haystack/releases/tag/v2.27.0"],
},
},
)
print(agent_output["database_agent"]["last_message"].text)
# Inspect what was written to the document store
written_docs = document_store.filter_documents()
print(f"\n{len(written_docs)} people added to the database:")
for doc in written_docs:
print(f" - {doc.content}")
```
### In YAML
The example pipeline below fetches a webpage, converts its HTML to text, and builds a chat prompt combining the page content with a user query.
The `Agent` then answers the question based on the provided content and can use its web search tool to find additional information if needed.
<details>
<summary>View YAML</summary>
```yaml
components:
agent:
init_parameters:
chat_generator:
init_parameters:
api_base_url: null
api_key:
env_vars:
- OPENAI_API_KEY
strict: true
type: env_var
generation_kwargs: {}
http_client_kwargs: null
max_retries: null
model: gpt-5.4-nano
organization: null
streaming_callback: null
timeout: null
tools: null
tools_strict: false
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
confirmation_strategies: null
exit_conditions:
- text
max_agent_steps: 5
raise_on_tool_invocation_failure: false
required_variables: null
state_schema: {}
streaming_callback: null
system_prompt: You are a helpful assistant. Use the web search tool to find
information when needed.
tool_invoker_kwargs: null
tools:
- data:
component:
init_parameters:
allowed_domains: null
api_key:
env_vars:
- SERPERDEV_API_KEY
strict: true
type: env_var
exclude_subdomains: false
search_params: {}
top_k: 3
type: haystack.components.websearch.serper_dev.SerperDevWebSearch
description: Search the web for current information on any topic
inputs_from_state: null
name: web_search
outputs_to_state: null
outputs_to_string: null
parameters: null
type: haystack.tools.component_tool.ComponentTool
user_prompt: null
type: haystack.components.agents.agent.Agent
converter:
init_parameters:
extraction_kwargs: {}
store_full_path: false
type: haystack.components.converters.html.HTMLToDocument
fetcher:
init_parameters:
client_kwargs:
follow_redirects: true
timeout: 3
http2: false
raise_on_failure: true
request_headers: {}
retry_attempts: 2
timeout: 3
user_agents:
- haystack/LinkContentFetcher/2.27.0rc0
type: haystack.components.fetchers.link_content.LinkContentFetcher
prompt_builder:
init_parameters:
required_variables:
- docs
- query
template:
- content:
- text: 'Based on the following content:
{% for doc in docs %}
{{ doc.content }}
{% endfor %}
Answer this question: {{ query }}'
meta: {}
name: null
role: user
variables: null
type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
connection_type_validation: true
connections:
- receiver: converter.sources
sender: fetcher.streams
- receiver: prompt_builder.docs
sender: converter.documents
- receiver: agent.messages
sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}
```
</details>
## Streaming
You can stream output as it's generated. Pass a callback to `streaming_callback`.
Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
```python
from haystack.components.generators.utils import print_streaming_chunk
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[...],
system_prompt="...",
streaming_callback=print_streaming_chunk,
)
```
See our [Streaming Support](../generators/guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
Give preference to `print_streaming_chunk` by default.
Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
## Multimodal Inputs
Agents support multimodal inputs when paired with a vision-capable model such as `gpt-5` (OpenAI) or `gemini-2.5-flash` (Google).
Pass images alongside text by including `ImageContent` objects in the `content_parts` of a `ChatMessage`:
```python
from haystack.dataclasses import ChatMessage, ImageContent
image = ImageContent.from_url("https://example.com/chart.png")
result = agent.run(
messages=[
ChatMessage.from_user(content_parts=["What does this chart show?", image]),
],
)
```
Tools can also return `ImageContent` directly, letting the agent fetch and reason about images dynamically during its loop.
Two things are required: set `outputs_to_string={"raw_result": True}` so the `ToolInvoker` skips string conversion, and return a `list[ImageContent]` (the tool result type is `str | Sequence[TextContent | ImageContent]`).
The standard Chat Completions API doesn't support images in tool results — use `OpenAIResponsesChatGenerator` (OpenAI's Responses API) instead:
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage, ImageContent
from haystack.tools import tool
@tool(outputs_to_string={"raw_result": True})
def fetch_image(
url: Annotated[str, "URL of the image to fetch and analyze"],
) -> list[ImageContent]:
"""Fetch an image from a URL so the agent can analyze its contents."""
return [ImageContent.from_url(url)]
agent = Agent(
chat_generator=OpenAIResponsesChatGenerator(model="gpt-5"),
tools=[fetch_image],
system_prompt="You are a helpful assistant that can fetch and analyze images from URLs.",
)
result = agent.run(
messages=[
ChatMessage.from_user(
"Fetch the image at https://picsum.photos/seed/haystack/640/480 and describe what you see.",
),
],
)
print(result["last_message"].text)
```
`ImageContent` can be created from a URL, a local file path, or a PDF page using the `PDFToImageContent` converter.
### In a pipeline
When an `Agent` sits inside a pipeline, use `ChatPromptBuilder` with its string template format and the `| templatize_part` filter to pass images as structured content parts:
```python
from haystack import Pipeline
from haystack.components.agents import Agent
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ImageContent
template = """
{% message role="user" %}
{{ question }}
{{ image | templatize_part }}
{% endmessage %}
"""
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5"),
system_prompt="You are a helpful assistant that can analyze images.",
)
prompt_builder = ChatPromptBuilder(
template=template,
required_variables=["question", "image"],
)
pipeline = Pipeline()
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("agent", agent)
pipeline.connect("prompt_builder.prompt", "agent.messages")
# Download or provide your own chart image as "chart.png"
image = ImageContent.from_file_path("chart.png")
result = pipeline.run(
{
"prompt_builder": {"question": "What does this chart show?", "image": image},
},
)
print(result["agent"]["last_message"].text)
```
:::tip
See these cookbooks for complete multimodal agent examples:
- [Multimodal Agents](https://haystack.deepset.ai/cookbook/multimodal_intro#multimodal-agent) — image inputs and tool use with agents
- [Gemma Chat RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag) — vision model in a RAG pipeline
:::
## Multi-Agent Systems
You can wrap an `Agent` as a tool to build multi-agent systems where specialist agents handle focused subtasks and a coordinator agent plans and delegates.
See [Multi-Agent Systems](../../concepts/agents/multi-agent-systems.mdx) for a full guide, including the recommended `@tool` decorator approach for full interface control and `ComponentTool` for declarative configuration.
## MCP Integration
Agents work with MCP in two directions:
- **Consuming MCP tools**: Pass `MCPTool` or `MCPToolset` instances in the `tools` list to call tools on any MCP-compatible server (filesystem, browser, databases, and more). See [MCPTool](../../tools/mcptool.mdx) and [MCPToolset](../../tools/mcptoolset.mdx).
- **Exposing as an MCP server**: Use [Hayhooks](../../development/hayhooks.mdx) to deploy your agent and expose it as an MCP server, making it callable from any MCP-compatible client such as Claude Desktop or Cursor.
## Additional References
📖 Related docs:
- [State](./state.mdx) — managing shared data between tools
- [Human in the Loop](./human-in-the-loop.mdx) — intercepting tool calls for human review
📚 Tutorials:
- [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)
- [Creating a Multi-Agent System](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)
- [Human-in-the-Loop with Haystack Agents](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent/)
🧑‍🍳 Cookbook:
- [Build a GitHub Issue Resolver Agent](https://haystack.deepset.ai/cookbook/github_issue_resolver_agent)
- [Multimodal Agents](https://haystack.deepset.ai/cookbook/multimodal_intro#multimodal-agent)
- [Gemma Chat RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag)
@@ -0,0 +1,305 @@
---
title: "Human in the Loop"
id: human-in-the-loop
slug: "/human-in-the-loop"
description: "Human-in-the-loop allows you to intercept agent tool calls before execution, letting a human confirm, reject, or modify the tool parameters."
---
# Human in the Loop
Human-in-the-loop (HITL) lets you intercept an agent's tool calls before they are executed.
A human can **confirm**, **reject**, or **modify** the parameters of each tool call in real time.
This is useful for high-stakes operations - such as sending emails, modifying databases, or making API calls - where you want a human to review the action first.
<div className="key-value-table">
| | |
| --- | --- |
| **Configured on** | The [`Agent`](./agent.mdx) component via `confirmation_strategies` |
| **Key classes** | `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` |
| **Import path** | `haystack.human_in_the_loop` |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/human_in_the_loop/ |
| **Package name** | `haystack-ai` |
</div>
## Overview
The HITL system is composed of three layers:
- **Strategy** - decides what to do when a tool is about to be called. The built-in `BlockingConfirmationStrategy` pauses execution and asks a human.
- **Policy** - decides *when* to ask. Built-in policies: `AlwaysAskPolicy`, `NeverAskPolicy`, `AskOncePolicy`.
- **UI** - the interface used to ask the human. Built-in UIs: `RichConsoleUI` (requires `rich`) and `SimpleConsoleUI` (stdlib only).
When the agent is about to invoke a tool, the strategy checks the policy.
If the policy says to ask, the UI prompts the human with the tool name, description, and parameters. The human can:
- **Confirm** (`y`) - execute as-is
- **Reject** (`n`) - skip execution and feed rejection feedback back to the LLM
- **Modify** (`m`) - edit the parameters before execution
The agent then continues with the human's decision.
## Usage
### Basic setup
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.human_in_the_loop import (
AlwaysAskPolicy,
BlockingConfirmationStrategy,
SimpleConsoleUI,
)
from haystack.tools import tool
@tool
def send_email(
to: Annotated[str, "The recipient email address"],
subject: Annotated[str, "The email subject line"],
body: Annotated[str, "The email body"],
) -> str:
"""Send an email to a recipient."""
return f"Email sent to {to}."
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=SimpleConsoleUI(),
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
tools=[send_email],
confirmation_strategies={"send_email": strategy},
)
result = agent.run(
messages=[ChatMessage.from_user("Send a welcome email to alice@example.com")],
)
```
When the agent calls `send_email`, the terminal will pause and show:
```
--- Tool Execution Request ---
Tool: send_email
Description: Send an email to a recipient.
Arguments:
to: alice@example.com
subject: Welcome!
body: Hi Alice, welcome aboard!
------------------------------
Confirm execution? (y=confirm / n=reject / m=modify):
```
### Using RichConsoleUI
`RichConsoleUI` provides a styled terminal prompt using the [`rich`](https://github.com/Textualize/rich) library:
```shell
pip install rich
```
```python
from haystack.human_in_the_loop import RichConsoleUI
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=RichConsoleUI(),
)
```
### Applying strategies to multiple tools
You can configure different strategies per tool, or share one strategy across a group of tools using a tuple key:
```python
@tool
def delete_record(record_id: Annotated[str, "The ID of the record to delete"]) -> str:
"""Delete a record from the database."""
return f"Record {record_id} deleted."
@tool
def update_record(
record_id: Annotated[str, "The ID of the record to update"],
data: Annotated[str, "The new data as a JSON string"],
) -> str:
"""Update a record in the database."""
return f"Record {record_id} updated."
@tool
def search(query: Annotated[str, "The search query"]) -> str:
"""Search the knowledge base."""
return f"Results for: {query}"
ask_strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=SimpleConsoleUI(),
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
tools=[send_email, delete_record, update_record, search],
confirmation_strategies={
# Share one strategy across multiple sensitive tools using a tuple key
("send_email", "delete_record", "update_record"): ask_strategy,
# search has no strategy - always executes without asking
},
)
```
### Customizing feedback messages
When a tool call is rejected or modified, `BlockingConfirmationStrategy` sends a message back to the LLM explaining what happened. Three optional template parameters control these messages — each has a sensible default, so you only need to set them if you want different wording:
- `reject_template`: Sent to the LLM when the user rejects a tool call. Must include a `{tool_name}` placeholder. Default: `"Tool execution for '{tool_name}' was rejected by the user."`
- `modify_template`: Sent when the user modifies the parameters. Must include `{tool_name}` and `{final_tool_params}` placeholders. Default: `"The parameters for tool '{tool_name}' were updated by the user to:\n{final_tool_params}"`
- `user_feedback_template`: Appends the user's optional free-text feedback to either message. Must include a `{feedback}` placeholder. Default: `"With user feedback: {feedback}"`
```python
strategy = BlockingConfirmationStrategy(
confirmation_policy=AlwaysAskPolicy(),
confirmation_ui=SimpleConsoleUI(),
reject_template="Skipping '{tool_name}' — rejected by operator.",
modify_template="Updated parameters for '{tool_name}': {final_tool_params}",
user_feedback_template="Reason: {feedback}",
)
```
## Policies
Policies control *when* the human is asked.
| Policy | Behavior |
| --- | --- |
| `AlwaysAskPolicy` | Ask every time the tool is called |
| `NeverAskPolicy` | Never ask - always proceed (useful for toggling HITL off without removing the strategy) |
| `AskOncePolicy` | Ask once per unique `(tool_name, parameters)` combination. Remembers confirmed calls and skips asking on repeats. |
### Custom policy
You can implement your own policy by subclassing `ConfirmationPolicy` from `haystack.human_in_the_loop.types`:
```python
from haystack.human_in_the_loop.types import ConfirmationPolicy, ConfirmationUIResult
from typing import Any
class AskForSensitiveParamsPolicy(ConfirmationPolicy):
"""Only ask when the 'to' parameter looks like an external email domain."""
def should_ask(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
) -> bool:
to = tool_params.get("to", "")
return not to.endswith("@mycompany.com")
```
For stateful policies, also implement `update_after_confirmation`.
It is called after the user responds and receives the full `ConfirmationUIResult`, letting you update internal state based on the outcome.
The following policy asks once per tool name and skips re-asking for any tool the user has already confirmed:
```python
from haystack.human_in_the_loop.types import ConfirmationPolicy
from haystack.human_in_the_loop import ConfirmationUIResult
from typing import Any
class AskOncePerToolPolicy(ConfirmationPolicy):
"""Ask once per tool name, regardless of parameters. Skip on repeat confirmed calls."""
def __init__(self) -> None:
self._confirmed_tools: set[str] = set()
def should_ask(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
) -> bool:
return tool_name not in self._confirmed_tools
def update_after_confirmation(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
confirmation_result: ConfirmationUIResult,
) -> None:
if confirmation_result.action == "confirm":
self._confirmed_tools.add(tool_name)
```
## Dataclasses
### `ConfirmationUIResult`
Returned by the UI after the human responds.
| Field | Type | Description |
| --- | --- | --- |
| `action` | `str` | `"confirm"`, `"reject"`, or `"modify"` |
| `feedback` | `str \| None` | Optional free-text feedback from the human |
| `new_tool_params` | `dict \| None` | Replacement parameters when action is `"modify"` |
### `ToolExecutionDecision`
Returned by the strategy to the agent.
| Field | Type | Description |
| --- | --- | --- |
| `tool_name` | `str` | Name of the tool |
| `execute` | `bool` | Whether to execute the tool |
| `tool_call_id` | `str \| None` | ID of the tool call |
| `feedback` | `str \| None` | Feedback message passed back to the LLM on rejection or modification |
| `final_tool_params` | `dict \| None` | Final parameters to use for execution |
## Example: HITL with Hayhooks and Open WebUI
The [hitl-hayhooks-redis-openwebui](https://github.com/deepset-ai/hitl-hayhooks-redis-openwebui) repository shows a full production-style HITL setup using a Haystack Agent served via [Hayhooks](https://github.com/deepset-ai/hayhooks) with approval dialogs rendered in [Open WebUI](https://github.com/open-webui/open-webui).
The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that uses `confirmation_strategy_context` to pass per-request resources - a Redis client and an async event queue - into the strategy at runtime:
- When a tool call is about to execute, the strategy emits a `tool_call_start` SSE event and blocks on `Redis BLPOP` waiting for an approval decision.
- The Open WebUI Pipe function receives the SSE event, shows the user a confirmation dialog, then writes `approved` or `rejected` to Redis via `LPUSH`.
- Once Redis unblocks, the strategy returns a `ToolExecutionDecision` and the agent continues.
This is a good reference if you need non-blocking HITL in a web or server environment where `SimpleConsoleUI` and `RichConsoleUI` are not suitable.
## Custom UI
Implement `ConfirmationUI` from `haystack.human_in_the_loop.types` to build your own interface - for example, a web-based approval queue:
```python
from haystack.human_in_the_loop.types import ConfirmationUI
from haystack.human_in_the_loop import ConfirmationUIResult
from typing import Any
class WebhookApprovalUI(ConfirmationUI):
"""Sends a webhook and waits for an async approval response."""
def get_user_confirmation(
self,
tool_name: str,
tool_description: str,
tool_params: dict[str, Any],
) -> ConfirmationUIResult:
# Send approval request to your system and wait for response
response = send_approval_request_and_wait(tool_name, tool_params)
return ConfirmationUIResult(
action=response["action"],
feedback=response.get("feedback"),
)
```
@@ -0,0 +1,415 @@
---
title: "State"
id: state
slug: "/state"
description: "`State` is a container for storing shared information during Agent and Tool execution. It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer."
---
# State
`State` is a container for storing shared information during Agent and Tool execution.
It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer.
## Overview
When building agents that use multiple tools, you often need tools to share information or accumulate results across iterations.
State provides centralized storage that all tools can read from and write to.
For example, a search tool called multiple times can append its results to a shared `documents` list, which is then returned alongside the agent's final answer for source inspection.
State uses a schema-based approach where you define:
- What data can be stored,
- The type of each piece of data,
- How values are merged when updated.
The Agent creates and manages the `State` object internally. You shouldn't need to instantiate it directly.
You interact with it through tool definitions (`inputs_from_state`, `outputs_to_state`, or a `state: State` parameter) and read results from the agent's output dict.
### Supported Types
State supports standard Python types:
- Basic types: `str`, `int`, `float`, `bool`, `dict`
- List types: `list`, `list[str]`, `list[int]`, `list[Document]`
- Union types: `str | int`, `str | None`
- Custom classes and data classes.
### Automatic Message Handling
State automatically includes a `messages` field that stores the full conversation history during execution.
You don't need to define this in your schema.
It uses `list[ChatMessage]` type with the `merge_lists` handler, so new messages are appended on each iteration.
### State API
| Method | Description |
| --- | --- |
| `state.get(key, default=None)` | Read a value; returns `default` if the key doesn't exist |
| `state.set(key, value)` | Write a value, merged using the schema's handler |
| `state.has(key)` | Returns `True` if the key exists in state |
| `state.data` | Returns a snapshot of all current state as a `dict` |
## Schema Definition
The schema defines what data can be stored and how values are updated. Each schema entry consists of:
- `type` (required): The Python type for this field (for example, `str`, `int`, `list`)
- `handler` (optional): A callable that determines how new values are merged when `set()` is called
```python
{
"parameter_name": {
"type": SomeType, # Required: expected Python type
"handler": some_func, # Optional: merge function
},
}
```
If you don't specify a handler, State automatically assigns a default based on the type.
### Default Handlers
State provides two built-in merge behaviors (importable from `haystack.components.agents.state`):
- **`merge_lists`**: Appends to the existing list (default for list types)
- **`replace_values`**: Overwrites the existing value (default for non-list types)
```python
from haystack.components.agents import State
schema = {
"documents": {"type": list}, # uses merge_lists by default
"user_name": {"type": str}, # uses replace_values by default
}
state = State(schema=schema)
state.set("documents", [1, 2])
state.set("documents", [3, 4])
print(state.get("documents")) # [1, 2, 3, 4]
state.set("user_name", "Alice")
state.set("user_name", "Bob")
print(state.get("user_name")) # "Bob"
```
### Custom Handlers
Custom handlers are useful when the default `merge_lists` or `replace_values` behaviors don't fit your needs.
A handler takes the current state value and the new value and returns the merged result.
The example below uses a deduplication handler, useful when multiple tool calls might return overlapping results and you want to avoid accumulating duplicates in state:
```python
def deduplicate(current_value: list | None, new_value: list) -> list:
"""Append new items, skipping any already in the list."""
existing = set(current_value or [])
return (current_value or []) + [item for item in new_value if item not in existing]
schema = {"doc_ids": {"type": list, "handler": deduplicate}}
state = State(schema=schema)
state.set("doc_ids", ["doc-1", "doc-2"])
state.set("doc_ids", ["doc-2", "doc-3"])
print(state.get("doc_ids")) # ["doc-1", "doc-2", "doc-3"]
```
You can also override the handler for a single `set()` call:
```python
from haystack.components.agents import State
def concatenate_strings(current: str | None, new: str) -> str:
return f"{current}-{new}" if current else new
state = State(schema={"user_name": {"type": str}})
state.set("user_name", "Alice")
state.set("user_name", "Bob", handler_override=concatenate_strings)
print(state.get("user_name")) # "Alice-Bob"
```
## Using State
Define a `state_schema` when creating the Agent.
State keys declared in `state_schema` are exposed as output keys on the agent's result dict alongside `messages` and `last_message`.
Tools interact with State through three mechanisms:
- **`outputs_to_state`**: Write tool results to state keys after the tool runs.
- **`inputs_from_state`**: Inject state values into tool parameters before the tool runs.
- **Direct `State` injection**: Add a `state: State` parameter to your tool function's signature. The Agent detects the `State` annotation and injects the live `State` object automatically, so you can read or write any key defined in the schema. The `State` object is never exposed to the LLM's parameter schema.
### Reading from State: `inputs_from_state`
`inputs_from_state` maps state keys to function parameter names using the format `{"state_key": "param_name"}`.
The value is injected from state before the tool runs, so the LLM never needs to provide it.
Parameters mapped via `inputs_from_state` are automatically excluded from the LLM's parameter schema.
The model never sees or provides them:
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
@tool(inputs_from_state={"user_name": "user_context"})
def search_documents(
query: Annotated[str, "The search query"],
user_context: str, # injected from state; excluded from LLM schema
) -> dict:
"""Search documents using query and user context."""
return {"results": [f"Found results for '{query}' (user: {user_context})"]}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[search_documents],
system_prompt="Use the search_documents tool to find information.",
streaming_callback=print_streaming_chunk,
state_schema={"user_name": {"type": str}},
)
result = agent.run(
messages=[ChatMessage.from_user("Search for Python tutorials")],
user_name="Alice", # state key "user_name" is pre-populated by passing user_name= to agent.run()
)
print(result["last_message"].text)
```
### Writing to State: `outputs_to_state`
The `outputs_to_state` parameter maps tool output keys to state keys. Each entry supports two optional fields:
```python
{
"state_key": {
"source": "tool_output_key", # which key to read from the tool's return dict; omit to store the entire dict
"handler": some_func, # override the schema's merge handler for this mapping only
},
}
```
```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
@tool(
outputs_to_state={
"documents": {"source": "documents"},
"result_count": {"source": "count"},
"last_query": {"source": "query"},
},
)
def retrieve_documents(
query: Annotated[str, "The search query"],
) -> dict:
"""Retrieve relevant documents."""
return {
"documents": [
{"title": "Doc 1", "content": "Content about Python"},
{"title": "Doc 2", "content": "More about Python"},
],
"count": 2,
"query": query,
}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retrieve_documents],
system_prompt="Use the retrieve_documents tool to find information.",
streaming_callback=print_streaming_chunk,
state_schema={
"documents": {"type": list},
"result_count": {"type": int},
"last_query": {"type": str},
},
)
result = agent.run(messages=[ChatMessage.from_user("Find information about Python")])
print(f"Documents: {result['documents']}")
print(f"Result count: {result['result_count']}")
print(f"Last query: {result['last_query']}")
```
If you omit `source`, the entire tool result dict is stored under the state key:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import tool
@tool(outputs_to_state={"user_info": {}})
def get_user_info() -> dict:
"""Get user information."""
return {"name": "Alice", "email": "alice@example.com", "role": "admin"}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[get_user_info],
system_prompt="Use the get_user_info tool to look up user details.",
streaming_callback=print_streaming_chunk,
state_schema={"user_info": {"type": dict}},
)
result = agent.run(messages=[ChatMessage.from_user("What are the user's details?")])
print(result["last_message"].text)
print(f"User info: {result['user_info']}")
```
### Combining Inputs and Outputs
Tools can both read from and write to State, enabling tool chaining across iterations.
This example builds on `retrieve_documents` from the previous section:
```python
@tool(
inputs_from_state={"documents": "documents"},
outputs_to_state={
"final_docs": {"source": "processed_docs"},
"final_count": {"source": "processed_count"},
},
)
def process_documents(
max_results: Annotated[int, "Maximum number of documents to return"],
documents: list = None, # injected from state; LLM does not provide this
) -> dict:
"""Process retrieved documents and return a filtered subset."""
processed = (documents or [])[:max_results]
return {"processed_docs": processed, "processed_count": len(processed)}
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retrieve_documents, process_documents], # chained through state
system_prompt="Use the available tools to retrieve and process documents.",
streaming_callback=print_streaming_chunk,
state_schema={
"documents": {"type": list},
"result_count": {"type": int},
"last_query": {"type": str},
"final_docs": {"type": list},
"final_count": {"type": int},
},
)
result = agent.run(
messages=[ChatMessage.from_user("Find and process 3 documents about Python")],
)
print(f"Processed {result['final_count']} documents")
```
### Injecting State Directly into Tools
As an alternative to `inputs_from_state` and `outputs_to_state`, a tool can declare a `state: State` parameter to receive the live `State` object at invocation time.
This lets the tool read from and write to any number of state keys without declaring mappings upfront.
The ToolInvoker detects the `State` annotation and injects the object automatically.
It is excluded from the LLM-facing schema. The model is never asked to supply it.
Both `State` and `State | None` annotations are supported.
For function-based tools, add the `state` parameter and use the `@tool` decorator:
```python
from typing import Annotated
from haystack.components.agents import Agent, State
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage, Document
from haystack.tools import tool
@tool
def retrieve_and_store(
query: Annotated[str, "The search query"],
state: State,
) -> str:
"""Retrieve documents and store them directly in state."""
documents = [Document(content=f"Result for '{query}'")]
state.set("documents", documents)
user_name = state.get("user_name", "unknown")
return f"Retrieved {len(documents)} document(s) for {user_name}"
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retrieve_and_store],
system_prompt="Use the retrieve_and_store tool to find documents.",
streaming_callback=print_streaming_chunk,
state_schema={"documents": {"type": list[Document]}, "user_name": {"type": str}},
)
result = agent.run(
messages=[ChatMessage.from_user("Find documents about Python")],
user_name="Alice",
)
print(result["last_message"].text)
print(result["documents"])
```
For component-based tools, declare a `State` input socket on the `run` method and wrap it with `ComponentTool`:
```python
from haystack import component
from haystack.components.agents import Agent, State
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage, Document
from haystack.tools import ComponentTool
@component
class DocumentRetriever:
"""Retrieve documents and store them in state."""
@component.output_types(reply=str)
def run(self, query: str, state: State) -> dict[str, str]:
"""
Retrieve documents based on query and store them in state.
:param query: The search query
"""
documents = [Document(content=f"Result for '{query}'")]
state.set("documents", documents)
return {"reply": f"Retrieved {len(documents)} document(s)"}
retriever_tool = ComponentTool(
component=DocumentRetriever(),
name="retrieve",
description="Retrieve documents based on a search query",
)
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
tools=[retriever_tool],
system_prompt="Use the retrieve tool to find documents.",
streaming_callback=print_streaming_chunk,
state_schema={"documents": {"type": list[Document]}},
)
result = agent.run(messages=[ChatMessage.from_user("Find documents about Python")])
print(result["last_message"].text)
print(result["documents"])
```