---
title: "OpenAIGenerator"
id: openaigenerator
slug: "/openaigenerator"
description: "`OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs)."
---

# OpenAIGenerator

`OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs).

<div className="key-value-table">

|  |  |
| --- | --- |
| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM  <br /> <br />`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
| **API reference** | [Generators](/reference/generators-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai.py |
| **Package name** | `haystack-ai` |

</div>

## Overview

`OpenAIGenerator` supports OpenAI models starting from gpt-3.5-turbo and later (gpt-4, gpt-4-turbo, and so on).

`OpenAIGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:

```
generator = OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"), model="gpt-4o-mini")
```

Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).

`OpenAIGenerator` supports custom deployments of your OpenAI models through the `api_base_url` init parameter.

### Streaming

`OpenAIGenerator` supports streaming the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter. Note that streaming the tokens is only compatible with generating a single response, so `n` must be set to 1 for streaming to work.

:::info
This component is designed for text generation, not for chat. If you want to use OpenAI LLMs for chat, use [`OpenAIChatGenerator`](openaichatgenerator.mdx) instead.
:::

## Usage

### On its own

Basic usage:

```python
from haystack.components.generators import OpenAIGenerator
from haystack.utils import Secret

client = OpenAIGenerator(model="gpt-4o-mini", api_key=Secret.from_token("<your-api-key>"))
response = client.run("What's Natural Language Processing? Be brief.")
print(response)

>>> {'replies': ['Natural Language Processing, often abbreviated as NLP, is a field
    of artificial intelligence that focuses on the interaction between computers
    and humans through natural language. The primary aim of NLP is to enable
    computers to understand, interpret, and generate human language in a valuable way.'],
    'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
    'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 53,
    'total_tokens': 69}}]}
```

With streaming:

```python
from haystack.components.generators import OpenAIGenerator
from haystack.utils import Secret

client = OpenAIGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
response = client.run("What's Natural Language Processing? Be brief.")
print(response)

>>> Natural Language Processing (NLP) is a branch of artificial
	intelligence that focuses on the interaction between computers and human
  language. It involves enabling computers to understand, interpret,and respond
  to natural human language in a way that is both meaningful and useful.
>>> {'replies': ['Natural Language Processing (NLP) is a branch of artificial
	intelligence that focuses on the interaction between computers and human
  language. It involves enabling computers to understand, interpret,and respond
  to natural human language in a way that is both meaningful and useful.'],
  'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
  'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 49,
  'total_tokens': 65}}]}
```

### In a Pipeline

Here's an example of RAG Pipeline:

```python
from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack import Document
from haystack.utils import Secret

docstore = InMemoryDocumentStore()
docstore.write_documents(
    [
        Document(content="Rome is the capital of Italy"),
        Document(content="Paris is the capital of France"),
    ],
)

query = "What is the capital of France?"

template = """
Given the following information, answer the question.

Context:
{% for document in documents %}
    {{ document.content }}
{% endfor %}

Question: {{ query }}?
"""
pipe = Pipeline()

pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component(
    "llm",
    OpenAIGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
)
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")

res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})

print(res)
```

### In YAML

This is the YAML representation of the RAG pipeline shown above. It retrieves documents based on a query, constructs a prompt using a template, and generates an answer using a chat model.

```yaml
components:
  llm:
    init_parameters:
      api_base_url: null
      api_key:
        env_vars:
        - OPENAI_API_KEY
        strict: true
        type: env_var
      generation_kwargs: {}
      http_client_kwargs: null
      max_retries: null
      model: gpt-5-mini
      organization: null
      streaming_callback: null
      system_prompt: null
      timeout: null
    type: haystack.components.generators.openai.OpenAIGenerator
  prompt_builder:
    init_parameters:
      required_variables: null
      template: "\nGiven the following information, answer the question.\n\nContext:\n\
        {% for document in documents %}\n    {{ document.content }}\n{% endfor %}\n\n\
        Question: {{ query }}?\n"
      variables: null
    type: haystack.components.builders.prompt_builder.PromptBuilder
  retriever:
    init_parameters:
      document_store:
        init_parameters:
          bm25_algorithm: BM25L
          bm25_parameters: {}
          bm25_tokenization_regex: (?u)\b\w+\b
          embedding_similarity_function: dot_product
          index: 64e4f9ab-87fb-47fd-b390-dabcfda61447
          return_embedding: true
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
      filter_policy: replace
      filters: null
      scale_score: false
      top_k: 10
    type: haystack.components.retrievers.in_memory.bm25_retriever.InMemoryBM25Retriever
connection_type_validation: true
connections:
- receiver: prompt_builder.documents
  sender: retriever.documents
- receiver: llm.prompt
  sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}
```