chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,307 @@
|
||||
---
|
||||
title: "OpenAIResponsesChatGenerator"
|
||||
id: openairesponseschatgenerator
|
||||
slug: "/openairesponseschatgenerator"
|
||||
description: "`OpenAIResponsesChatGenerator` enables chat completion using OpenAI's Responses API with support for reasoning models."
|
||||
---
|
||||
|
||||
# OpenAIResponsesChatGenerator
|
||||
|
||||
`OpenAIResponsesChatGenerator` enables chat completion using OpenAI's Responses API with support for reasoning models.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
|
||||
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects representing the chat or a plain string |
|
||||
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects containing the generated responses |
|
||||
| **API reference** | [Generators](/reference/generators-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/openai_responses.py |
|
||||
| **Package name** | `haystack-ai` |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`OpenAIResponsesChatGenerator` uses OpenAI's Responses API to generate chat completions. It supports gpt-4 and o-series models (reasoning models like o1, o3-mini). The default model is `gpt-5-mini`.
|
||||
|
||||
The Responses API is designed for reasoning-capable models and supports features like reasoning summaries, multi-turn conversations with previous response IDs, and structured outputs.
|
||||
|
||||
The component requires a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`), and optional metadata. If a string is passed, it is converted into a list containing a single `ChatMessage` with the `user` role. See the [usage](#usage) section for examples.
|
||||
|
||||
You can pass any parameters valid for the OpenAI Responses API directly to `OpenAIResponsesChatGenerator` using the `generation_kwargs` parameter, both at initialization and to the `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI Responses API documentation](https://platform.openai.com/docs/api-reference/responses).
|
||||
|
||||
`OpenAIResponsesChatGenerator` can support custom deployments of your OpenAI models through the `api_base_url` init parameter.
|
||||
|
||||
### Authentication
|
||||
|
||||
`OpenAIResponsesChatGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key` using a [`Secret`](../../concepts/secret-management.mdx):
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.utils import Secret
|
||||
|
||||
generator = OpenAIResponsesChatGenerator(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
### Reasoning Support
|
||||
|
||||
One of the key features of the Responses API is support for reasoning models. You can configure reasoning behavior using the `reasoning` parameter in `generation_kwargs`:
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
client = OpenAIResponsesChatGenerator(
|
||||
generation_kwargs={"reasoning": {"effort": "medium", "summary": "auto"}},
|
||||
)
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_user(
|
||||
"What's the most efficient sorting algorithm for nearly sorted data?",
|
||||
),
|
||||
]
|
||||
response = client.run(messages)
|
||||
print(response)
|
||||
```
|
||||
|
||||
The `reasoning` parameter accepts:
|
||||
- `effort`: Level of reasoning effort - `"low"`, `"medium"`, or `"high"`
|
||||
- `summary`: How to generate reasoning summaries - `"auto"` or `"generate_summary": True/False`
|
||||
|
||||
:::note
|
||||
OpenAI does not return the actual reasoning tokens, but you can view the summary if enabled. For more details, see the [OpenAI Reasoning documentation](https://platform.openai.com/docs/guides/reasoning).
|
||||
:::
|
||||
|
||||
### Multi-turn Conversations
|
||||
|
||||
The Responses API supports multi-turn conversations using `previous_response_id`. You can pass the response ID from a previous turn to maintain conversation context:
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
client = OpenAIResponsesChatGenerator()
|
||||
|
||||
# First turn
|
||||
messages = [ChatMessage.from_user("What's quantum computing?")]
|
||||
response = client.run(messages)
|
||||
response_id = response["replies"][0].meta.get("id")
|
||||
|
||||
# Second turn - reference previous response
|
||||
messages = [ChatMessage.from_user("Can you explain that in simpler terms?")]
|
||||
response = client.run(messages, generation_kwargs={"previous_response_id": response_id})
|
||||
```
|
||||
|
||||
### Structured Output
|
||||
|
||||
`OpenAIResponsesChatGenerator` supports structured output generation through the `text_format` and `text` parameters in `generation_kwargs`:
|
||||
|
||||
- **`text_format`**: Pass a Pydantic model to define the structure
|
||||
- **`text`**: Pass a JSON schema directly
|
||||
|
||||
**Using a Pydantic model**:
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
|
||||
class BookInfo(BaseModel):
|
||||
title: str
|
||||
author: str
|
||||
year: int
|
||||
genre: str
|
||||
|
||||
|
||||
client = OpenAIResponsesChatGenerator(
|
||||
model="gpt-4o",
|
||||
generation_kwargs={"text_format": BookInfo},
|
||||
)
|
||||
|
||||
response = client.run(
|
||||
messages=[
|
||||
ChatMessage.from_user(
|
||||
"Extract book information: '1984 by George Orwell, published in 1949, is a dystopian novel.'",
|
||||
),
|
||||
],
|
||||
)
|
||||
print(response["replies"][0].text)
|
||||
```
|
||||
|
||||
**Using a JSON schema**:
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
json_schema = {
|
||||
"format": {
|
||||
"type": "json_schema",
|
||||
"name": "BookInfo",
|
||||
"strict": True,
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"title": {"type": "string"},
|
||||
"author": {"type": "string"},
|
||||
"year": {"type": "integer"},
|
||||
"genre": {"type": "string"},
|
||||
},
|
||||
"required": ["title", "author", "year", "genre"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
client = OpenAIResponsesChatGenerator(
|
||||
model="gpt-4o",
|
||||
generation_kwargs={"text": json_schema},
|
||||
)
|
||||
|
||||
response = client.run(
|
||||
messages=[
|
||||
ChatMessage.from_user(
|
||||
"Extract book information: '1984 by George Orwell, published in 1949, is a dystopian novel.'",
|
||||
),
|
||||
],
|
||||
)
|
||||
print(response["replies"][0].text)
|
||||
```
|
||||
|
||||
:::info[Model Compatibility and Limitations]
|
||||
- Both Pydantic models and JSON schemas are supported for latest models starting from GPT-4o.
|
||||
- If both `text_format` and `text` are provided, `text_format` takes precedence and the JSON schema passed to `text` is ignored.
|
||||
- Streaming is not supported when using structured outputs.
|
||||
- Older models only support basic JSON mode through `{"type": "json_object"}`. For details, see [OpenAI JSON mode documentation](https://platform.openai.com/docs/guides/structured-outputs#json-mode).
|
||||
- For complete information, check the [OpenAI Structured Outputs documentation](https://platform.openai.com/docs/guides/structured-outputs).
|
||||
:::
|
||||
|
||||
### Tool Support
|
||||
|
||||
`OpenAIResponsesChatGenerator` supports function calling through the `tools` parameter. It accepts flexible tool configurations:
|
||||
|
||||
- **Haystack Tool objects and Toolsets**: Pass Haystack `Tool` objects or `Toolset` objects, including mixed lists of both
|
||||
- **OpenAI/MCP tool definitions**: Pass pre-defined OpenAI or MCP tool definitions as dictionaries
|
||||
|
||||
Note that you cannot mix Haystack tools and OpenAI/MCP tools in the same call - choose one format or the other.
|
||||
|
||||
```python
|
||||
from haystack.tools import Tool
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
|
||||
def get_weather(city: str) -> str:
|
||||
"""Get weather information for a city."""
|
||||
return f"Weather in {city}: Sunny, 22°C"
|
||||
|
||||
|
||||
weather_tool = Tool(
|
||||
name="get_weather",
|
||||
description="Get current weather for a city",
|
||||
function=get_weather,
|
||||
parameters={"type": "object", "properties": {"city": {"type": "string"}}},
|
||||
)
|
||||
|
||||
generator = OpenAIResponsesChatGenerator(tools=[weather_tool])
|
||||
messages = [ChatMessage.from_user("What's the weather in Paris?")]
|
||||
response = generator.run(messages)
|
||||
```
|
||||
|
||||
You can control strict schema adherence with the `tools_strict` parameter. When set to `True` (default is `False`), the model will follow the tool schema exactly. Note that the Responses API has its own strictness enforcement mechanisms independent of this parameter.
|
||||
|
||||
For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
|
||||
|
||||
### Streaming
|
||||
|
||||
You can stream output as it's generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
|
||||
|
||||
```python
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
|
||||
# Configure any `Generator` or `ChatGenerator` with a streaming callback
|
||||
component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
|
||||
|
||||
# If this is a `ChatGenerator`, pass a list of messages:
|
||||
# from haystack.dataclasses import ChatMessage
|
||||
# component.run([ChatMessage.from_user("Your question here")])
|
||||
|
||||
# If this is a (non-chat) `Generator`, pass a prompt:
|
||||
# component.run({"prompt": "Your prompt here"})
|
||||
```
|
||||
|
||||
:::info
|
||||
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
|
||||
:::
|
||||
|
||||
See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
|
||||
|
||||
Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is an example of using `OpenAIResponsesChatGenerator` independently with reasoning and streaming:
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
|
||||
client = OpenAIResponsesChatGenerator(
|
||||
streaming_callback=print_streaming_chunk,
|
||||
generation_kwargs={"reasoning": {"effort": "high", "summary": "auto"}},
|
||||
)
|
||||
response = client.run(
|
||||
[
|
||||
ChatMessage.from_user(
|
||||
"Solve this logic puzzle: If all roses are flowers and some flowers fade quickly, can we conclude that some roses fade quickly?",
|
||||
),
|
||||
],
|
||||
)
|
||||
print(response["replies"][0].reasoning) # Access reasoning summary if available
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
This example shows a pipeline that uses `ChatPromptBuilder` to create dynamic prompts and `OpenAIResponsesChatGenerator` with reasoning enabled to generate explanations of complex topics:
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack import Pipeline
|
||||
|
||||
prompt_builder = ChatPromptBuilder()
|
||||
llm = OpenAIResponsesChatGenerator(
|
||||
generation_kwargs={"reasoning": {"effort": "low", "summary": "auto"}},
|
||||
)
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", prompt_builder)
|
||||
pipe.add_component("llm", llm)
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
topic = "quantum computing"
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"You are a helpful assistant that explains complex topics clearly.",
|
||||
),
|
||||
ChatMessage.from_user("Explain {{topic}} in simple terms"),
|
||||
]
|
||||
result = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"topic": topic},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
Reference in New Issue
Block a user