chore: import upstream snapshot with attribution

docs-website/versioned_docs/version-2.18/pipeline-components/generators/amazonbedrockchatgenerator.mdx

@@ -0,0 +1,108 @@
+---
+title: "AmazonBedrockChatGenerator"
+id: amazonbedrockchatgenerator
+slug: "/amazonbedrockchatgenerator"
+description: "This component enables chat completion using models through Amazon Bedrock service."
+---
+
+# AmazonBedrockChatGenerator
+
+This component enables chat completion using models through Amazon Bedrock service.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "model": The model to use  <br /> <br />"aws_access_key_id": AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.  <br /> <br />"aws_secret_access_key": AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.  <br /> <br />"aws_region_name": AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. |
+| **Mandatory run variables** | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  instances |
+| **Output variables** | "replies": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock |
+
+[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes high-performing foundation models from leading AI startups and Amazon available through a unified API. You can choose from various foundation models to find the one best suited for your use case.
+
+`AmazonBedrockChatGenerator` enables chat completion using chat models from Anthropic, Cohere, Meta Llama 2, and Mistral with a single component.
+
+The models that we currently support are Anthropic's _Claude_, Meta's _Llama 2_, and _Mistral_, but as more chat models are added, their support will be provided through `AmazonBedrockChatGenerator`.
+
+## Overview
+
+This component uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. For more information on setting up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
+
+:::note
+Using AWS CLI
+
+Consider using AWS CLI as a more straightforward tool to manage your AWS services. With AWS CLI, you can quickly configure your [boto3 credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). This way, you won't need to provide detailed authentication parameters when initializing Amazon Bedrock Generator in Haystack.
+:::
+
+To use this component for text generation, initialize an AmazonBedrockGenerator with the model name, the AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`) should be set as environment variables, be configured as described above or passed as [Secret](../../concepts/secret-management.mdx) arguments. Note, make sure the region you set supports Amazon Bedrock.
+
+To start using Amazon Bedrock with Haystack, install the `amazon-bedrock-haystack` package:
+
+```shell
+pip install amazon-bedrock-haystack
+```
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack_integrations.components.generators.amazon_bedrock import (
+    AmazonBedrockChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+
+generator = AmazonBedrockChatGenerator(model="meta.llama2-70b-chat-v1")
+messages = [
+    ChatMessage.from_system(
+        "You are a helpful assistant that answers question in Spanish only",
+    ),
+    ChatMessage.from_user("What's Natural Language Processing? Be brief."),
+]
+
+response = generator.run(messages)
+print(response)
+```
+
+### In a pipeline
+
+In a RAG pipeline:
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.amazon_bedrock import (
+    AmazonBedrockChatGenerator,
+)
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", AmazonBedrockChatGenerator(model="meta.llama2-70b-chat-v1"))
+pipe.connect("prompt_builder", "llm")
+
+country = "Germany"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving out valuable information to language learners.",
+)
+messages = [
+    system_message,
+    ChatMessage.from_user("What's the official language of {{ country }}?"),
+]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"country": country},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/amazonbedrockgenerator.mdx

@@ -0,0 +1,117 @@
+---
+title: "AmazonBedrockGenerator"
+id: amazonbedrockgenerator
+slug: "/amazonbedrockgenerator"
+description: "This component enables text generation using models through Amazon Bedrock service."
+---
+
+# AmazonBedrockGenerator
+
+This component enables text generation using models through Amazon Bedrock service.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "model": The model to use  <br /> <br />"aws_access_key_id": AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.  <br /> <br />"aws_secret_access_key": AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var.  <br /> <br />"aws_region_name": AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. |
+| **Mandatory run variables** | “prompt”: The instructions for the Generator |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the model |
+| **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock |
+
+[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes high-performing foundation models from leading AI startups and Amazon available through a unified API. You can choose from various foundation models to find the one best suited for your use case.
+
+`AmazonBedrockGenerator` enables text generation using models from AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single component.
+
+The models that we currently support are Anthropic's Claude, AI21 Labs' Jurassic-2, Stability AI's Stable Diffusion, Cohere's Command and Embed, Meta's Llama 2, and the Amazon Titan language and embeddings models.
+
+## Overview
+
+This component uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. For more information on setting up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
+
+:::note
+Using AWS CLI
+
+Consider using AWS CLI as a more straightforward tool to manage your AWS services. With AWS CLI, you can quickly configure your [boto3 credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). This way, you won't need to provide detailed authentication parameters when initializing Amazon Bedrock Generator in Haystack.
+:::
+
+To use this component for text generation, initialize an AmazonBedrockGenerator with the model name, the AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`) should be set as environment variables, be configured as described above or passed as [Secret](../../concepts/secret-management.mdx) arguments. Note, make sure the region you set supports Amazon Bedrock.
+
+To start using Amazon Bedrock with Haystack, install the `amazon-bedrock-haystack` package:
+
+```shell
+pip install amazon-bedrock-haystack
+```
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack_integrations.components.generators.amazon_bedrock import (
+    AmazonBedrockGenerator,
+)
+
+aws_access_key_id = "..."
+aws_secret_access_key = "..."
+aws_region_name = "eu-central-1"
+
+generator = AmazonBedrockGenerator(model="anthropic.claude-v2")
+result = generator.run("Who is the best American actor?")
+for reply in result["replies"]:
+    print(reply)
+
+## >>> 'There is no definitive "best" American actor, as acting skill and talent a# re subjective. However, some of the most acclaimed and influential American act# ors include Tom Hanks, Daniel Day-Lewis, Denzel Washington, Meryl Streep, Rober# t De Niro, Al Pacino, Marlon Brando, Jack Nicholson, Leonardo DiCaprio and John# ny Depp. Choosing a single "best" actor comes down to personal preference.'
+```
+
+### In a pipeline
+
+In a RAG pipeline:
+
+```python
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders import PromptBuilder
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack import Pipeline
+
+from haystack_integrations.components.generators.amazon_bedrock import (
+    AmazonBedrockGenerator,
+)
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: What's the official language of {{ country }}?
+"""
+
+aws_access_key_id = "..."
+aws_secret_access_key = "..."
+aws_region_name = "eu-central-1"
+generator = AmazonBedrockGenerator(model="anthropic.claude-v2")
+docstore = InMemoryDocumentStore()
+
+pipe = Pipeline()
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("generator", generator)
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "generator")
+
+pipe.run({"retriever": {"query": "France"}, "prompt_builder": {"country": "France"}})
+
+## {'generator': {'replies': ['Based on the context provided, the official language of France is French.']}}
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/anthropicchatgenerator.mdx

@@ -0,0 +1,160 @@
+---
+title: "AnthropicChatGenerator"
+id: anthropicchatgenerator
+slug: "/anthropicchatgenerator"
+description: "This component enables chat completions using Anthropic large language models (LLMs)."
+---
+
+# AnthropicChatGenerator
+
+This component enables chat completions using Anthropic large language models (LLMs).
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": An Anthropic API key. Can be set with `ANTHROPIC_API_KEY` env var. |
+| **Mandatory run variables** | “messages” A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
+| **Output variables** | "replies": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Anthropic](/reference/integrations-anthropic) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
+
+## Overview
+
+This integration supports Anthropic `chat` models such as `claude-3-5-sonnet-20240620`,`claude-3-opus-20240229`, `claude-3-haiku-20240307`, and similar. Check out the most recent full list in [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models).
+
+### Parameters
+
+`AnthropicChatGenerator` needs an Anthropic API key to work. You can provide this key in:
+
+- The `ANTHROPIC_API_KEY` environment variable (recommended)
+- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
+
+Set your preferred Anthropic model with the `model` parameter when initializing the component.
+
+`AnthropicChatGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and when running the component. For more details on the parameters supported by the Anthropic API, see the [Anthropic documentation](https://docs.anthropic.com).
+
+Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata.
+
+Only text input modality is supported at this time.
+
+### Streaming
+
+You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+## Configure any `Generator` or `ChatGenerator` with a streaming callback
+component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
+
+## If this is a `ChatGenerator`, pass a list of messages:
+## from haystack.dataclasses import ChatMessage
+## component.run([ChatMessage.from_user("Your question here")])
+
+## If this is a (non-chat) `Generator`, pass a prompt:
+## component.run({"prompt": "Your prompt here"})
+```
+
+:::note
+Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
+
+:::
+
+See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
+
+Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
+
+### Prompt caching
+
+Prompt caching is a feature for Anthropic LLMs that stores large text inputs for reuse. It allows you to send a large text block once and then refer to it in later requests without resending the entire text.
+This feature is particularly useful for coding assistants that need full codebase context and for processing large documents. It can help reduce costs and improve response times.
+
+Here's an example of an instance of `AnthropicChatGenerator` being initialized with prompt caching and tagging a message to be cached:
+
+```python python
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generation_kwargs = {"extra_headers": {"anthropic-beta": "prompt-caching-2024-07-31"}}
+
+claude_llm = AnthropicChatGenerator(
+    api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), generation_kwargs=generation_kwargs
+)
+
+system_message = ChatMessage.from_system("Replace with some long text documents, code or instructions")
+system_message.meta["cache_control"] = {"type": "ephemeral"}
+
+messages = [system_message, ChatMessage.from_user("A query about the long text for example")]
+result = claude_llm.run(messages)
+
+## and now invoke again with
+
+messages = [system_message, ChatMessage.from_user("Another query about the long text etc")]
+result = claude_llm.run(messages)
+
+## and so on, either invoking component directly or in the pipeline
+```
+
+For more details, refer to Anthropic's [documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) and integration [examples](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic/example).
+
+## Usage
+
+Install the`anthropic-haystack` package to use the `AnthropicChatGenerator`:
+
+```shell
+pip install anthropic-haystack
+```
+
+### On its own
+
+```python
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = AnthropicChatGenerator()
+message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
+print(generator.run([message]))
+```
+
+### In a pipeline
+
+You can also use `AnthropicChatGenerator`with the Anthropic chat models in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.utils import Secret
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component(
+    "llm",
+    AnthropicChatGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")),
+)
+pipe.connect("prompt_builder", "llm")
+
+country = "Germany"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving out valuable information to language learners.",
+)
+messages = [
+    system_message,
+    ChatMessage.from_user("What's the official language of {{ country }}?"),
+]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"country": country},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/anthropicgenerator.mdx

@@ -0,0 +1,84 @@
+---
+title: "AnthropicGenerator"
+id: anthropicgenerator
+slug: "/anthropicgenerator"
+description: "This component enables text completions using Anthropic large language models (LLMs)."
+---
+
+# AnthropicGenerator
+
+This component enables text completions using Anthropic large language models (LLMs).
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [PromptBuilder](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": An Anthropic API key. Can be set with `ANTHROPIC_API_KEY` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Anthropic](/reference/integrations-anthropic) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
+
+## Overview
+
+This integration supports Anthropic models such as `claude-3-5-sonnet-20240620`,`claude-3-opus-20240229`, `claude-3-haiku-20240307`, and similar. Although these LLMs are called chat models, the main prompt interface works with the string prompts. Check out the most recent full list in the [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models).
+
+### Parameters
+
+`AnthropicGenerator` needs an Anthropic API key to work. You can provide this key in:
+
+- The `ANTHROPIC_API_KEY` environment variable (recommended)
+- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
+
+Set your preferred Anthropic model in the `model` parameter when initializing the component.
+
+`AnthropicGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the Anthropic API, see [Anthropic documentation](https://docs.anthropic.com).
+
+Finally, the component run method requires a single string prompt to generate text.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+Install the `anthropic-haystack` package to use the `AnthropicGenerator`:
+
+```shell
+pip install anthropic-haystack
+```
+
+### On its own
+
+```python
+from haystack_integrations.components.generators.anthropic import AnthropicGenerator
+
+generator = AnthropicGenerator()
+print(generator.run("What's Natural Language Processing? Be brief."))
+```
+
+### In a pipeline
+
+You can also use `AnthropicGenerator` with the Anthropic models in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import PromptBuilder
+from haystack_integrations.components.generators.anthropic import AnthropicGenerator
+from haystack.utils import Secret
+
+template = """
+You are an assistant giving out valuable information to language learners.
+Answer this question, be brief.
+
+Question: {{ query }}?
+"""
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", PromptBuilder(template))
+pipe.add_component("llm", AnthropicGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")))
+pipe.connect("prompt_builder", "llm")
+
+query = "What language is spoke in Germany?"
+res = pipe.run(data={"prompt_builder": {"query": {query}}})
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/anthropicvertexchatgenerator.mdx

@@ -0,0 +1,184 @@
+---
+title: "AnthropicVertexChatGenerator"
+id: anthropicvertexchatgenerator
+slug: "/anthropicvertexchatgenerator"
+description: "This component enables chat completions using AnthropicVertex API."
+---
+
+# AnthropicVertexChatGenerator
+
+This component enables chat completions using AnthropicVertex API.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
+| **Mandatory init variables** | "region": The region where the Anthropic model is deployed  <br /> <br />”project_id”: GCP project ID where the Anthropic model is deployed |
+| **Mandatory run variables** | "messages": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)   objects |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and others |
+| **API reference** | [Anthropic](/reference/integrations-anthropic) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
+
+## Overview
+
+`AnthropicVertexChatGenerator` enables text generation using state-of-the-art Claude 3 LLMs using the Anthropic Vertex AI API.
+It supports   `Claude 3.5 Sonnet`, `Claude 3 Opus`, `Claude 3 Sonnet`, and `Claude 3 Haiku` models, that are accessible through the Vertex AI API endpoint. For more details about the models, refer to [Anthropic Vertex AI documentation](https://docs.anthropic.com/en/api/claude-on-vertex-ai).
+
+### Parameters
+
+To use the `AnthropicVertexChatGenerator`, ensure you have a GCP project with Vertex AI enabled. You need to specify your GCP `project_id` and `region`.
+
+You can provide these keys in the following ways:
+
+- The `REGION` and `PROJECT_ID` environment variables (recommended)
+- The `region` and `project_id` init parameters
+
+Before making requests, you may need to authenticate with GCP using `gcloud auth login`.
+
+Set your preferred supported Anthropic model with the `model` parameter when initializing the component. Additionally, ensure that the desired Anthropic model is activated in the Vertex AI Model Garden.
+
+`AnthropicVertexChatGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and when running the component. For more details on the parameters supported by the Anthropic API, see the [Anthropic documentation](https://docs.anthropic.com/).
+
+Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata.
+
+Only text input modality is supported at this time.
+
+### Streaming
+
+You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+## Configure any `Generator` or `ChatGenerator` with a streaming callback
+component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
+
+## If this is a `ChatGenerator`, pass a list of messages:
+## from haystack.dataclasses import ChatMessage
+## component.run([ChatMessage.from_user("Your question here")])
+
+## If this is a (non-chat) `Generator`, pass a prompt:
+## component.run({"prompt": "Your prompt here"})
+```
+
+:::note
+Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
+
+:::
+
+See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
+
+Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
+
+### Prompt Caching
+
+Prompt caching is a feature for Anthropic LLMs that stores large text inputs for reuse. It allows you to send a large text block once and then refer to it in later requests without resending the entire text.
+
+This feature is particularly useful for coding assistants that need full codebase context and for processing large documents. It can help reduce costs and improve response times.
+
+Here's an example of an instance of `AnthropicVertexChatGenerator` being initialized with prompt caching and tagging a message to be cached:
+
+```python
+from haystack_integrations.components.generators.anthropic import (
+    AnthropicVertexChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+
+generation_kwargs = {"extra_headers": {"anthropic-beta": "prompt-caching-2024-07-31"}}
+
+claude_llm = AnthropicVertexChatGenerator(
+    region="your_region",
+    project_id="test_id",
+    generation_kwargs=generation_kwargs,
+)
+
+system_message = ChatMessage.from_system(
+    "Replace with some long text documents, code or instructions",
+)
+system_message.meta["cache_control"] = {"type": "ephemeral"}
+
+messages = [
+    system_message,
+    ChatMessage.from_user("A query about the long text for example"),
+]
+result = claude_llm.run(messages)
+
+## and now invoke again with
+
+messages = [
+    system_message,
+    ChatMessage.from_user("Another query about the long text etc"),
+]
+result = claude_llm.run(messages)
+
+## and so on, either invoking component directly or in the pipeline
+```
+
+For more details, refer to Anthropic's [documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) and integration [examples](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic/example).
+
+## Usage
+
+Install the`anthropic-haystack` package to use the `AnthropicVertexChatGenerator`:
+
+```shell
+pip install anthropic-haystack
+```
+
+### On its own
+
+```python
+from haystack_integrations.components.generators.anthropic import (
+    AnthropicVertexChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+
+messages = [ChatMessage.from_user("What's Natural Language Processing?")]
+client = AnthropicVertexChatGenerator(
+    model="claude-3-sonnet@20240229",
+    project_id="your-project-id",
+    region="us-central1",
+)
+
+response = client.run(messages)
+print(response)
+```
+
+### In a pipeline
+
+You can also use `AnthropicVertexChatGenerator`with the Anthropic chat models in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.anthropic import (
+    AnthropicVertexChatGenerator,
+)
+from haystack.utils import Secret
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component(
+    "llm",
+    AnthropicVertexChatGenerator(project_id="test_id", region="us-central1"),
+)
+pipe.connect("prompt_builder", "llm")
+
+country = "Germany"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving out valuable information to language learners.",
+)
+messages = [
+    system_message,
+    ChatMessage.from_user("What's the official language of {{ country }}?"),
+]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"country": country},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/azureopenaichatgenerator.mdx

@@ -0,0 +1,185 @@
+---
+title: "AzureOpenAIChatGenerator"
+id: azureopenaichatgenerator
+slug: "/azureopenaichatgenerator"
+description: "This component enables chat completion using OpenAI’s large language models (LLMs) through Azure services."
+---
+
+# AzureOpenAIChatGenerator
+
+This component enables chat completion using OpenAI’s large language models (LLMs) through Azure services.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var.  <br /> <br />"azure_ad_token": Microsoft Entra ID token. Can be set with `AZURE_OPENAI_AD_TOKEN` env var. |
+| **Mandatory run variables** | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables** | “replies”: A list of alternative replies of the LLM to the input chat |
+| **API reference** | [Generators](/reference/generators-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/azure.py |
+
+## Overview
+
+`AzureOpenAIChatGenerator` supports OpenAI models deployed through Azure services. To see the list of supported models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model used with the component is `gpt-4o-mini`.
+
+To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
+
+The component uses `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` and `azure_ad_token` at initialization:
+
+```python
+client = AzureOpenAIChatGenerator(
+    azure_endpoint="<Your Azure endpoint e.g. `https://your-company.azure.openai.com/>",
+    api_key=Secret.from_token("<your-api-key>"),
+    azure_deployment="<a model name>",
+)
+```
+
+:::note
+We recommend using environment variables instead of initialization parameters.
+
+:::
+
+Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. See the [usage](https://www.notion.so/AzureOpenAIChatGenerator-c20636ac8b914ab798439a5f7a273ff0?pvs=21) section for an example.
+
+You can pass any chat completion parameters that are valid for the `openai.ChatCompletion.create` method directly to `AzureOpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
+
+You can also specify a model for this component through the `azure_deployment` init parameter.
+
+### Structured Output
+
+`AzureOpenAIChatGenerator` supports structured output generation, allowing you to receive responses in a predictable format. You can use Pydantic models or JSON schemas to define the structure of the output through the `response_format` parameter in `generation_kwargs`.
+
+This is useful when you need to extract structured data from text or generate responses that match a specific format.
+
+```python
+from pydantic import BaseModel
+from haystack.components.generators.chat import AzureOpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+
+class NobelPrizeInfo(BaseModel):
+    recipient_name: str
+    award_year: int
+    category: str
+    achievement_description: str
+    nationality: str
+
+
+client = AzureOpenAIChatGenerator(
+    azure_endpoint="<Your Azure endpoint>",
+    azure_deployment="gpt-4o",
+    generation_kwargs={"response_format": NobelPrizeInfo},
+)
+
+response = client.run(
+    messages=[
+        ChatMessage.from_user(
+            "In 2021, American scientist David Julius received the Nobel Prize in"
+            " Physiology or Medicine for his groundbreaking discoveries on how the human body"
+            " senses temperature and touch.",
+        ),
+    ],
+)
+print(response["replies"][0].text)
+```
+
+:::note
+Model Compatibility and Limitations
+
+- Pydantic models and JSON schemas are supported for latest models starting from GPT-4o.
+- Older models only support basic JSON mode through `{"type": "json_object"}`. For details, see [OpenAI JSON mode documentation](https://platform.openai.com/docs/guides/structured-outputs#json-mode).
+- Streaming limitation: When using streaming with structured outputs, you must provide a JSON schema instead of a Pydantic model for `response_format`.
+- For complete information, check the [Azure OpenAI Structured Outputs documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs).
+:::
+
+### Streaming
+
+You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+## Configure any `Generator` or `ChatGenerator` with a streaming callback
+component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
+
+## If this is a `ChatGenerator`, pass a list of messages:
+## from haystack.dataclasses import ChatMessage
+## component.run([ChatMessage.from_user("Your question here")])
+
+## If this is a (non-chat) `Generator`, pass a prompt:
+## component.run({"prompt": "Your prompt here"})
+```
+
+:::note
+Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
+
+:::
+
+See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
+
+Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
+
+## Usage
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack.components.generators.chat import AzureOpenAIChatGenerator
+
+client = AzureOpenAIChatGenerator()
+response = client.run(
+    [ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
+)
+print(response)
+```
+
+With streaming:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack.components.generators.chat import AzureOpenAIChatGenerator
+
+client = AzureOpenAIChatGenerator(
+    streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
+)
+response = client.run(
+    [ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
+)
+print(response)
+```
+
+### In a pipeline
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import AzureOpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+llm = AzureOpenAIChatGenerator()
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+location = "Berlin"
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/azureopenaigenerator.mdx

@@ -0,0 +1,131 @@
+---
+title: "AzureOpenAIGenerator"
+id: azureopenaigenerator
+slug: "/azureopenaigenerator"
+description: "This component enables text generation using OpenAI's large language models (LLMs) through Azure services."
+---
+
+# AzureOpenAIGenerator
+
+This component enables text generation using OpenAI's large language models (LLMs) through Azure services.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var.  <br /> <br />"azure_ad_token": Microsoft Entra ID token. Can be set with `AZURE_OPENAI_AD_TOKEN` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Generators](/reference/generators-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/azure.py |
+
+## Overview
+
+`AzureOpenAIGenerator` supports OpenAI models deployed through Azure services. To see the list of supported models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model used with the component is `gpt-4o-mini`.
+
+To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
+
+The component uses `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` and `azure_ad_token` at initialization:
+
+```python
+client = AzureOpenAIGenerator(
+    azure_endpoint="<Your Azure endpoint e.g. `https://your-company.azure.openai.com/>",
+    api_key=Secret.from_token("<your-api-key>"),
+    azure_deployment="<a model name>",
+)
+```
+
+:::note
+We recommend using environment variables instead of initialization parameters.
+
+:::
+
+Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
+
+You can also specify a model for this component through the `azure_deployment` init parameter.
+
+### Streaming
+
+`AzureOpenAIGenerator` supports streaming the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter. Note that streaming the tokens is only compatible with generating a single response, so `n` must be set to 1 for streaming to work.
+
+:::note
+This component is designed for text generation, not for chat. If you want to use LLMs for chat, use [`AzureOpenAIChatGenerator`](azureopenaichatgenerator.mdx) instead.
+
+:::
+
+## Usage
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack.components.generators import AzureOpenAIGenerator
+
+client = AzureOpenAIGenerator()
+response = client.run("What's Natural Language Processing? Be brief.")
+print(response)
+```
+
+With streaming:
+
+```python
+from haystack.components.generators import AzureOpenAIGenerator
+
+client = AzureOpenAIGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
+response = client.run("What's Natural Language Processing? Be brief.")
+print(response)
+
+	intelligence that focuses on the interaction between computers and human
+  language. It involves enabling computers to understand, interpret,and respond
+  to natural human language in a way that is both meaningful and useful.
+	intelligence that focuses on the interaction between computers and human
+  language. It involves enabling computers to understand, interpret,and respond
+  to natural human language in a way that is both meaningful and useful.'],
+  'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
+  'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 49,
+  'total_tokens': 65}}]}
+
+```
+
+### In a Pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.components.generators import AzureOpenAIGenerator
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack import Document
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="Rome is the capital of Italy"),
+        Document(content="Paris is the capital of France"),
+    ],
+)
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("llm", AzureOpenAIGenerator())
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/coherechatgenerator.mdx

@@ -0,0 +1,91 @@
+---
+title: "CohereChatGenerator"
+id: coherechatgenerator
+slug: "/coherechatgenerator"
+description: "CohereChatGenerator enables chat completions using Cohere's large language models (LLMs)."
+---
+
+# CohereChatGenerator
+
+CohereChatGenerator enables chat completions using Cohere's large language models (LLMs).
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
+| **Mandatory run variables** | “messages” A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
+| **Output variables** | "replies": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Cohere](/reference/integrations-cohere) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
+
+This integration supports Cohere `chat` models such as `command`,`command-r` and `comman-r-plus`. Check out the most recent full list in [Cohere documentation](https://docs.cohere.com/reference/chat).
+
+## Overview
+
+`CohereChatGenerator` needs a Cohere API key to work. You can set this key in:
+
+- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
+- The `COHERE_API_KEY` environment variable (recommended)
+
+Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `Co.chat` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the Cohere API, refer to the [Cohere documentation](https://docs.cohere.com/reference/chat).
+
+Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+You need to install `cohere-haystack` package to use the  `CohereChatGenerator`:
+
+```shell
+pip install cohere-haystack
+```
+
+#### On its own
+
+```python
+from haystack_integrations.components.generators.cohere import CohereChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = CohereChatGenerator()
+message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
+print(generator.run([message]))
+```
+
+#### In a Pipeline
+
+You can also use `CohereChatGenerator` to use cohere chat models in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.cohere import CohereChatGenerator
+from haystack.utils import Secret
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", CohereChatGenerator())
+pipe.connect("prompt_builder", "llm")
+
+country = "Germany"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving out valuable information to language learners.",
+)
+messages = [
+    system_message,
+    ChatMessage.from_user("What's the official language of {{ country }}?"),
+]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"country": country},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/coheregenerator.mdx

@@ -0,0 +1,113 @@
+---
+title: "CohereGenerator"
+id: coheregenerator
+slug: "/coheregenerator"
+description: "`CohereGenerator` enables text generation using Cohere's large language models (LLMs)."
+---
+
+# CohereGenerator
+
+`CohereGenerator` enables text generation using Cohere's large language models (LLMs).
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Cohere](/reference/integrations-cohere) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
+
+ This integration supports Cohere models such as `command`, `command-r` and `comman-r-plus`. Check out the most recent full list in [Cohere documentation](https://docs.cohere.com/reference/chat).
+
+## Overview
+
+`CohereGenerator` needs a Cohere API key to work. You can write this key in:
+
+- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
+- The `COHERE_API_KEY` environment variable (recommended)
+
+Then, the component needs a prompt to operate, but you can pass any text generation parameters directly to this component using the `generation_kwargs` parameter at initialization. For more details on the parameters supported by the Cohere API, refer to the [Cohere documentation](https://docs.cohere.com/reference/chat).
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+You need to install `cohere-haystack` package to use the  `CohereGenerator`:
+
+```shell
+pip install cohere-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack_integrations.components.generators.cohere import CohereGenerator
+
+client = CohereGenerator()
+response = client.run("Briefly explain what NLP is in one sentence.")
+print(response)
+
+ 'meta': [{'finish_reason': 'COMPLETE'}]}
+```
+
+With streaming:
+
+```python
+from haystack_integrations.components.generators.cohere import CohereGenerator
+
+client = CohereGenerator(
+    streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
+)
+response = client.run("Briefly explain what NLP is in one sentence.")
+print(response)
+```
+
+### In a pipeline
+
+In a RAG pipeline:
+
+```python
+from haystack import Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.generators.cohere import CohereGenerator
+from haystack import Document
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="Rome is the capital of Italy"),
+        Document(content="Paris is the capital of France"),
+    ],
+)
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("llm", CohereGenerator())
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/dalleimagegenerator.mdx

@@ -0,0 +1,93 @@
+---
+title: "DALLEImageGenerator"
+id: dalleimagegenerator
+slug: "/dalleimagegenerator"
+description: "Generate images using OpenAI's DALL-E model."
+---
+
+# DALLEImageGenerator
+
+Generate images using OpenAI's DALL-E model.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx), flexible |
+| **Mandatory init variables** | "api_key": An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the model |
+| **Output variables** | “images”: A list of generated images  <br /> <br />”revised_prompt”: A string containing the prompt that was used to generate the image, if there was any revision to the prompt made by OpenAI |
+| **API reference** | [Generators](/reference/generators-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai_dalle.py |
+
+## Overview
+
+The `DALLEImageGenerator` component generates images using OpenAI's DALL-E model.
+
+By default, the component uses `dall-e-3` model, standard picture quality, and 1024x1024 resolution. You can change these parameters using `model` (during component initialization), `quality`, and `size` (during component initialization or run) parameters.
+
+`DALLEImageGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
+
+```
+image_generator = DALLEImageGenerator(api_key=Secret.from_token("<your-api-key>"))
+```
+
+Check our [API reference](/reference/generators-api#dalleimagegenerator) for the detailed component parameters description, or the [OpenAI documentation](https://platform.openai.com/docs/api-reference/images/create) for the details on OpenAI API parameters.
+
+## Usage
+
+### On its own
+
+```python
+from haystack.components.generators import DALLEImageGenerator
+
+image_generator = DALLEImageGenerator()
+response = image_generator.run("Show me a picture of a black cat.")
+
+print(response)
+```
+
+### In a pipeline
+
+In the following pipeline, we first set up a `PromptBuilder` that will structure the image description with a detailed template describing various artistic elements. The pipeline then passes this structured prompt into a `DALLEImageGenerator` to generate the image based on this detailed description.
+
+```python
+from haystack import Pipeline
+from haystack.components.generators import DALLEImageGenerator
+from haystack.components.builders import PromptBuilder
+
+prompt_builder = PromptBuilder(
+    template="""Create a {style} image with the following details:
+
+                Main subject: {prompt}
+                Artistic style: {art_style}
+                Lighting: {lighting}
+                Color palette: {colors}
+                Composition: {composition}
+                Additional details: {details}""",
+)
+
+image_generator = DALLEImageGenerator()
+
+pipeline = Pipeline()
+pipeline.add_component("prompt_builder", prompt_builder)
+pipeline.add_component("image_generator", image_generator)
+
+pipeline.connect("prompt_builder.prompt", "image_generator.prompt")
+
+results = pipeline.run(
+    {
+        "prompt": "a mystical treehouse library",
+        "style": "photorealistic",
+        "art_style": "fantasy concept art with intricate details",
+        "lighting": "dusk with warm lantern light glowing from within",
+        "colors": "rich earth tones, deep greens, and golden accents",
+        "composition": "wide angle view showing the entire structure nestled in an ancient oak tree",
+        "details": "spiral staircases wrapping around branches, stained glass windows, floating books, and magical fireflies providing ambient illumination",
+    },
+)
+
+generated_images = results["image_generator"]["images"]
+revised_prompt = results["image_generator"]["revised_prompt"]
+
+print(f"Generated image URL: {generated_images[0]}")
+print(f"Revised prompt: {revised_prompt}")
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/external-integrations-generators.mdx

@@ -0,0 +1,17 @@
+---
+title: "External Integrations"
+id: external-integrations-generators
+slug: "/external-integrations-generators"
+description: "External integrations that enable RAG pipeline creation."
+---
+
+# External Integrations
+
+External integrations that enable RAG pipeline creation.
+
+| Name                                                                            | Description                                                                                    |
+| :------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------- |
+| [DeepL](https://haystack.deepset.ai/integrations/deepl)                         | Translate your text and documents using DeepL services.                                        |
+| [fastRAG](https://haystack.deepset.ai/integrations/fastrag/)                    | Enables the creation of efficient and optimized retrieval augmented generative pipelines.      |
+| [LM Format Enforcer](https://haystack.deepset.ai/integrations/lmformatenforcer) | Enforce JSON Schema / Regex output of your local models with `LMFormatEnforcerLocalGenerator`. |
+| [Titan](https://haystack.deepset.ai/integrations/titanml-takeoff)               | Run local open-source LLMs from Meta, Mistral and Alphabet directly in your computer.          |

docs-website/versioned_docs/version-2.18/pipeline-components/generators/googleaigeminichatgenerator.mdx

@@ -0,0 +1,168 @@
+---
+title: "GoogleAIGeminiChatGenerator"
+id: googleaigeminichatgenerator
+slug: "/googleaigeminichatgenerator"
+description: "This component enables chat completion using Google Gemini models."
+---
+
+# GoogleAIGeminiChatGenerator
+
+This component enables chat completion using Google Gemini models.
+
+:::warning
+Deprecation Notice
+
+This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
+
+We recommend switching to the new [GoogleGenAIChatGenerator](googlegenaichatgenerator.mdx) integration instead.
+:::
+
+|                                        |                                                                                                      |
+| :------------------------------------- | :--------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                 |
+| **Mandatory init variables**           | "api_key": A Google AI Studio API key. Can be set with `GOOGLE_API_KEY` env var.                     |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables**                   | “replies”: A list of alternative replies of the model to the input chat                              |
+| **API reference**                      | [Google AI](/reference/integrations-google-ai)                                                              |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_ai          |
+
+`GoogleAIGeminiChatGenerator` supports `gemini-2.5-pro-exp-03-25`, `gemini-2.0-flash`, `gemini-1.5-pro`, and `gemini-1.5-flash` models.
+
+For available models, see https://ai.google.dev/gemini-api/docs/models/gemini.
+
+### Parameters Overview
+
+`GoogleAIGeminiChatGenerator` uses a Google Studio API key for authentication. You can write this key in an `api_key` parameter or as a `GOOGLE_API_KEY` environment variable (recommended).
+
+To get an API key, visit the [Google AI Studio](https://aistudio.google.com/) website.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+To begin working with `GoogleAIGeminiChatGenerator`, install the `google-ai-haystack` package:
+
+```shell
+pip install google-ai-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+import os
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.google_ai import (
+    GoogleAIGeminiChatGenerator,
+)
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+gemini_chat = GoogleAIGeminiChatGenerator()
+
+messages = [ChatMessage.from_user("Tell me the name of a movie")]
+res = gemini_chat.run(messages)
+
+print(res["replies"][0].text)
+
+messages += [res["replies"], ChatMessage.from_user("Who's the main actor?")]
+res = gemini_chat.run(messages)
+
+print(res["replies"][0].text)
+```
+
+When chatting with Gemini, you can also easily use function calls. First, define the function locally and convert into a [Tool](../../tools/tool.mdx):
+
+```python
+from typing import Annotated
+from haystack.tools import create_tool_from_function
+
+
+## example function to get the current weather
+def get_current_weather(
+    location: Annotated[
+        str,
+        "The city for which to get the weather, e.g. 'San Francisco'",
+    ] = "Munich",
+    unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius",
+) -> str:
+    return f"The weather in {location} is sunny. The temperature is 20 {unit}."
+
+
+tool = create_tool_from_function(get_current_weather)
+```
+
+Create a new instance of `GoogleAIGeminiChatGenerator` to set the tools and a [ToolInvoker](../tools/toolinvoker.mdx) to invoke the tools.
+
+```python
+import os
+from haystack_integrations.components.generators.google_ai import (
+    GoogleAIGeminiChatGenerator,
+)
+from haystack.components.tools import ToolInvoker
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+
+gemini_chat = GoogleAIGeminiChatGenerator(model="gemini-2.0-flash", tools=[tool])
+
+tool_invoker = ToolInvoker(tools=[tool])
+```
+
+And then ask a question:
+
+```python
+from haystack.dataclasses import ChatMessage
+
+messages = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")]
+res = gemini_chat.run(messages=messages)
+
+print(res["replies"][0].tool_calls)
+
+tool_messages = tool_invoker.run(messages=replies)["tool_messages"]
+messages = user_message + replies + tool_messages
+
+messages += res["replies"][0] + [
+    ChatMessage.from_function(content=weather, name="get_current_weather"),
+]
+
+final_replies = gemini_chat.run(messages=messages)["replies"]
+print(final_replies[0].text)
+```
+
+### In a pipeline
+
+```python
+import os
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack_integrations.components.generators.google_ai import (
+    GoogleAIGeminiChatGenerator,
+)
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+gemini_chat = GoogleAIGeminiChatGenerator()
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("gemini", gemini_chat)
+pipe.connect("prompt_builder.prompt", "gemini.messages")
+
+location = "Rome"
+messages = [ChatMessage.from_user("Tell me briefly about {{location}} history")]
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/googleaigeminigenerator.mdx

@@ -0,0 +1,134 @@
+---
+title: "GoogleAIGeminiGenerator"
+id: googleaigeminigenerator
+slug: "/googleaigeminigenerator"
+description: "This component enables text generation using the Google Gemini models."
+---
+
+# GoogleAIGeminiGenerator
+
+This component enables text generation using the Google Gemini models.
+
+:::warning
+Deprecation Notice
+
+This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
+
+We recommend switching to the new [GoogleGenAIChatGenerator](googlegenaichatgenerator.mdx) integration instead.
+:::
+
+|                                        |                                                                                              |
+| :------------------------------------- | :------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)                                               |
+| **Mandatory init variables**           | "api_key": A Google AI Studio API key. Can be set with `GOOGLE_API_KEY` env var.             |
+| **Mandatory run variables**            | “parts”: A variadic list containing a mix of images, audio, video, and text to prompt Gemini |
+| **Output variables**                   | “replies”: A list of strings or dictionaries with all the replies generated by the model     |
+| **API reference**                      | [Google AI](/reference/integrations-google-ai)                                                      |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_ai  |
+
+`GoogleAIGeminiGenerator` supports `gemini-2.5-pro-exp-03-25`, `gemini-2.0-flash`, `gemini-1.5-pro`, and `gemini-1.5-flash` models.
+
+For available models, see https://ai.google.dev/gemini-api/docs/models/gemini.
+
+### Parameters Overview
+
+`GoogleAIGeminiGenerator` uses a Google AI Studio API key for authentication. You can write this key in an `api_key` parameter or as a `GOOGLE_API_KEY` environment variable (recommended).
+
+To get an API key, visit the [Google AI Studio](https://ai.google.dev/gemini-api/docs/api-key) website.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+Start by installing the `google-ai-haystack` package to use the  `GoogleAIGeminiGenerator`:
+
+```shell
+pip install google-ai-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+import os
+from haystack_integrations.components.generators.google_ai import (
+    GoogleAIGeminiGenerator,
+)
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+
+gemini = GoogleAIGeminiGenerator(model="gemini-1.5-pro")
+res = gemini.run(parts=["What is the most interesting thing you know?"])
+for answer in res["replies"]:
+    print(answer)
+```
+
+This is a more advanced usage that also uses text and images as input:
+
+```python
+import requests
+import os
+from haystack.dataclasses.byte_stream import ByteStream
+from haystack_integrations.components.generators.google_ai import (
+    GoogleAIGeminiGenerator,
+)
+
+URLS = [
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg",
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot2.jpg",
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot3.jpg",
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot4.jpg",
+]
+images = [
+    ByteStream(data=requests.get(url).content, mime_type="image/jpeg") for url in URLS
+]
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+
+gemini = GoogleAIGeminiGenerator(model="gemini-1.5-pro")
+result = gemini.run(parts=["What can you tell me about this robots?", *images])
+for answer in result["replies"]:
+    print(answer)
+```
+
+### In a pipeline
+
+In a RAG pipeline:
+
+```python
+import os
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders import PromptBuilder
+from haystack import Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.generators.google_ai import (
+    GoogleAIGeminiGenerator,
+)
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+
+docstore = InMemoryDocumentStore()
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: What's the official language of {{ country }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("gemini", GoogleAIGeminiGenerator(model="gemini-pro"))
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "gemini")
+
+pipe.run({"prompt_builder": {"country": "France"}})
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/googlegenaichatgenerator.mdx

@@ -0,0 +1,221 @@
+---
+title: "GoogleGenAIChatGenerator"
+id: googlegenaichatgenerator
+slug: "/googlegenaichatgenerator"
+description: "This component enables chat completion using Google Gemini models through Google Gen AI SDK."
+---
+
+# GoogleGenAIChatGenerator
+
+This component enables chat completion using Google Gemini models through Google Gen AI SDK.
+
+|                                        |                                                                                                |
+| -------------------------------------- | ---------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                             |
+| **Mandatory init variables**           | "api_key": A Google API key. Can be set with `GOOGLE_API_KEY` env var.                         |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat        |
+| **Output variables**                   | “replies”: A list of alternative replies of the model to the input chat                        |
+| **API reference**                      | [Google GenAI](/reference/integrations-google-genai)                                                  |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai |
+
+## Overview
+
+`GoogleGenAIChatGenerator` supports `gemini-2.0-flash` (default), `gemini-2.5-pro-exp-03-25`, `gemini-1.5-pro`, and `gemini-1.5-flash` models.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+### Authentication
+
+Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
+
+To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
+To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
+
+The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
+
+```python
+embedder = GoogleGenAITextEmbedder(api_key=Secret.from_token("<your-api-key>"))
+```
+
+The following examples show how to use the component with the Gemini Developer API and the Vertex AI API.
+
+#### Gemini Developer API (API Key Authentication)
+
+```python
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+
+## set the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY)
+chat_generator = GoogleGenAIChatGenerator()
+```
+
+#### Vertex AI (Application Default Credentials)
+
+```python
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+
+## Using Application Default Credentials (requires gcloud auth setup)
+chat_generator = GoogleGenAIChatGenerator(
+    api="vertex",
+    vertex_ai_project="my-project",
+    vertex_ai_location="us-central1",
+)
+```
+
+#### Vertex AI (API Key Authentication)
+
+```python
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+
+## set the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY)
+chat_generator = GoogleGenAIChatGenerator(api="vertex")
+```
+
+## Usage
+
+To start using this integration, install the package with:
+
+```shell
+pip install google-genai-haystack
+```
+
+### On its own
+
+```python
+from haystack.dataclasses.chat_message import ChatMessage
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+
+## Initialize the chat generator
+chat_generator = GoogleGenAIChatGenerator()
+
+## Generate a response
+messages = [ChatMessage.from_user("Tell me about movie Shawshank Redemption")]
+response = chat_generator.run(messages=messages)
+print(response["replies"][0].text)
+```
+
+You can also easily use function calls. First, define the function locally and convert into a [Tool](https://www.notion.so/docs/tool):
+
+```python
+from typing import Annotated
+from haystack.tools import create_tool_from_function
+
+
+## example function to get the current weather
+def get_current_weather(
+    location: Annotated[
+        str,
+        "The city for which to get the weather, e.g. 'San Francisco'",
+    ] = "Munich",
+    unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius",
+) -> str:
+    return f"The weather in {location} is sunny. The temperature is 20 {unit}."
+
+
+tool = create_tool_from_function(get_current_weather)
+```
+
+Create a new instance of `GoogleGenAIChatGenerator` to set the tools and a [ToolInvoker](https://www.notion.so/docs/toolinvoker) to invoke the tools.
+
+```python
+import os
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+from haystack.components.tools import ToolInvoker
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+
+genai_chat = GoogleGenAIChatGenerator(tools=[tool])
+
+tool_invoker = ToolInvoker(tools=[tool])
+```
+
+And then ask a question:
+
+```python
+from haystack.dataclasses import ChatMessage
+
+messages = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")]
+res = genai_chat.run(messages=messages)
+
+print(res["replies"][0].tool_calls)
+
+tool_messages = tool_invoker.run(messages=replies)["tool_messages"]
+messages = user_message + replies + tool_messages
+
+messages += res["replies"][0] + [
+    ChatMessage.from_function(content=weather, name="get_current_weather"),
+]
+
+final_replies = genai_chat.run(messages=messages)["replies"]
+print(final_replies[0].text)
+```
+
+#### With Streaming
+
+```python
+from haystack.dataclasses.chat_message import ChatMessage
+from haystack.dataclasses import StreamingChunk
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+
+
+def streaming_callback(chunk: StreamingChunk):
+    print(chunk.content, end="", flush=True)
+
+
+## Initialize with streaming callback
+chat_generator = GoogleGenAIChatGenerator(streaming_callback=streaming_callback)
+
+## Generate a streaming response
+messages = [ChatMessage.from_user("Write a short story")]
+response = chat_generator.run(messages=messages)
+## Text will stream in real-time through the callback
+```
+
+### In a pipeline
+
+```python
+import os
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+
+os.environ["GOOGLE_API_KEY"] = "<MY_API_KEY>"
+genai_chat = GoogleGenAIChatGenerator()
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("genai", genai_chat)
+pipe.connect("prompt_builder.prompt", "genai.messages")
+
+location = "Rome"
+messages = [ChatMessage.from_user("Tell me briefly about {{location}} history")]
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/guides-to-generators/choosing-the-right-generator.mdx

@@ -0,0 +1,199 @@
+---
+title: "Choosing the Right Generator"
+id: choosing-the-right-generator
+slug: "/choosing-the-right-generator"
+description: "This page provides information on choosing the right Generator for interacting with Generative Language Models in Haystack. It explains the distinction between Generators and ChatGenerators, discusses using proprietary and open models from various providers, and explores options for using open models on-premise."
+---
+
+# Choosing the Right Generator
+
+This page provides information on choosing the right Generator for interacting with Generative Language Models in Haystack. It explains the distinction between Generators and ChatGenerators, discusses using proprietary and open models from various providers, and explores options for using open models on-premise.
+
+In Haystack, Generators are the main interface for interacting with Generative Language Models.
+This guide aims to simplify the process of choosing the right Generator based on your preferences and computing resources. This guide does not focus on selecting a specific model itself but rather a model type and a Haystack Generator: as you will see, in several cases, you have different options to use the same model.
+
+## Generators vs ChatGenerators
+
+The first distinction we are talking about is between Generators and ChatGenerators, for example, OpenAIGenerator and OpenAIChatGenerator, HuggingFaceAPIGenerator and HuggingFaceAPIChatGenerator, and so on.
+
+- **Generators** are components that expect a prompt (a string) and return the generated text in “replies”.
+- **ChatGenerators** support the [ChatMessage data class](../../../concepts/data-classes/chatmessage.mdx) out of the box. They expect a list of Chat Messages and return a Chat Message in “replies”.
+
+The choice between Generators and ChatGenerators depends on your use case and the underlying model. If you anticipate a multi-turn interaction with the Language Model in a chat scenario, opting for a ChatGenerator is generally better.
+
+:::tip
+To learn more about this comparison, check out our [Generators vs Chat Generators](generators-vs-chat-generators.mdx) guide.
+:::
+
+## Streaming Support
+
+Streaming refers to outputting LLM responses word by word rather than waiting for the entire response to be generated before outputting everything at once.
+
+You can check which Generators have streaming support on the [Generators overview page](../../generators.mdx).
+
+When you enable streaming, the generator calls your `streaming_callback` for every `StreamingChunk`. Each chunk represents exactly one of the following:
+
+- **Tool calls**: The model is building a tool/function call. Read `chunk.tool_calls`.
+- **Tool result**: A tool finished and returned output. Read `chunk.tool_call_result`.
+- **Text tokens**: Normal assistant text. Read `chunk.content`.
+- **Reasoning tokens**: Extended thinking output (for models that support it). Read `chunk.reasoning`.
+
+Only one of these fields appears per chunk. Use `chunk.start` and `chunk.finish_reason` to detect boundaries. Use `chunk.index` and `chunk.component_info` for tracing.
+
+For providers that support multiple candidates, set `n=1` to stream.
+
+:::info[Parameter Details]
+
+Check out the parameter details in our [API Reference for StreamingChunk](/reference/data-classes-api#streamingchunk).
+:::
+
+The simplest way is to use the built-in `print_streaming_chunk` function. It handles all chunk types and prints formatted output to stdout:
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+generator = SomeGenerator(streaming_callback=print_streaming_chunk)
+# For ChatGenerators, pass a list[ChatMessage]. For text generators, pass a prompt string.
+```
+
+### Custom Callback
+
+If you need custom rendering, write your own callback. Handle the four chunk types in order:
+
+```python
+from haystack.dataclasses import StreamingChunk
+
+
+def my_streaming_callback(chunk: StreamingChunk) -> None:
+    if chunk.start and chunk.index and chunk.index > 0:
+        print("\n\n", flush=True, end="")
+
+    # Tool Call streaming
+    if chunk.tool_calls:
+        for tool_call in chunk.tool_calls:
+            if chunk.start:
+                if chunk.index and tool_call.index > chunk.index:
+                    print("\n\n", flush=True, end="")
+                print(
+                    f">>> Tool Call: {tool_call.tool_name}\n>>> Arguments: ",
+                    flush=True,
+                    end="",
+                )
+            if tool_call.arguments:
+                print(tool_call.arguments, flush=True, end="")
+
+    # Tool Result streaming
+    if chunk.tool_call_result:
+        print(f">>> Tool Result\n{chunk.tool_call_result.result}", flush=True, end="")
+
+    # Text streaming
+    if chunk.content:
+        if chunk.start:
+            print(">>> Assistant\n", flush=True, end="")
+        print(chunk.content, flush=True, end="")
+
+    # Reasoning streaming
+    if chunk.reasoning:
+        if chunk.start:
+            print(">>> Reasoning\n", flush=True, end="")
+        print(chunk.reasoning.reasoning_text, flush=True, end="")
+
+    if chunk.finish_reason is not None:
+        print("\n\n", flush=True, end="")
+```
+
+### Agents and Tools
+
+Agents and `ToolInvoker` forward your `streaming_callback`. They also emit a final tool-result chunk with a `finish_reason` so UIs can close the “tool phase” cleanly before assistant text resumes. The default `print_streaming_chunk` formats this for you.
+
+## Proprietary Models
+
+Using proprietary models is a quick way to start with Generative Language Models. The typical approach involves calling these hosted models using an API Key. You are paying based on the number of tokens, both sent and generated.
+You don’t need significant resources on your local machine, as the computation is executed on the provider’s infrastructure. When using these models, your data exits your machine and is transmitted to the model provider.
+
+Haystack supports the models offered by a variety of providers: OpenAI, Azure, Google VertexAI and Makersuite, Cohere, and Mistral, with more being added constantly.
+
+We also support [Amazon Bedrock](../amazonbedrockgenerator.mdx): it provides access to proprietary models from Amazon Titan family, AI21 Labs, Anthropic, Cohere, and several open source models, such as Llama from Meta.
+
+## Open Models
+
+When discussing open (weights) models, we're referring to models with public weights that anyone can deploy on their infrastructure. The datasets used for training are shared less frequently. One could choose to use an open model for several reasons, including more transparency and control of the model.
+
+:::info[Commercial Use]
+
+Not all open models are suitable for commercial use. We advise thoroughly reviewing the license, typically available on Hugging Face, before considering their adoption.
+:::
+
+Even if the model is open, you might still want to rely on model providers to use it, mostly because you want someone else to host the model and take care of the infrastructural aspects. In these scenarios, your data transitions from your machine to the provider facilitating the model.
+
+The costs associated with these solutions can vary. Depending on the solution you choose, you pay for the tokens consumed, both sent and generated or for the hosting of the mode, often billed per hour.
+
+In Haystack, several Generators support these solutions through privately hosted or shared hosted models.
+
+### Shared Hosted Models
+
+With this type, you leverage an instance of the model shared with other users, with payment typically based on consumed tokens, both sent and generated.
+
+Here are the components that support shared hosted models in Haystack:
+
+- Hugging Face API Generators, when querying the [free Hugging Face Inference API](https://huggingface.co/inference-api). The free Inference API provides access to some popular models for quick experimentation, although it comes with rate limitations and is not intended for production use.
+- Various cloud providers offer interfaces compatible with OpenAI Generators. These include Anyscale, Deep Infra, Fireworks, Lemonfox.ai, OctoAI, Together AI, and many others.
+  Here is an example using OctoAI and [`OpenAIChatGenerator`](../openaichatgenerator.mdx):
+
+```python
+
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+generator = OpenAIChatGenerator(
+    api_key=Secret.from_env_var("ENVVAR_WITH_API_KEY"),
+    api_base_url="https://text.octoai.run/v1",
+    model="mixtral-8x7b-instruct-fp16",
+)
+
+generator.run(messages=[ChatMessage.from_user("What is the best French cheese?")])
+```
+
+### Privately Hosted Models
+
+In this case, a private instance of the model is deployed by the provider, and you typically pay per hour.
+
+Here are the components that support privately hosted models in Haystack:
+
+- Amazon [SagemakerGenerator](../sagemakergenerator.mdx)
+- HuggingFace API Generators, when used to query [HuggingFace Inference endpoints](https://huggingface.co/inference-endpoints).
+
+### Shared Hosted Model vs Privately Hosted Model
+
+**Why choose a shared hosted model:**
+
+- Cost Savings: Access cost-effective solutions especially suitable for users with varying usage patterns or limited budgets.
+- Ease of Use: Setup and maintenance are simplified as the provider manages the infrastructure and updates, making it user-friendly.
+
+**Why choose a privately hosted model:**
+
+- Dedicated Resources: Ensure consistent performance with dedicated resources for your instance and avoid any impact from other users.
+- Scalability: Scale resources based on requirements while ensuring optimal performance during peak times and cost savings during off-peak hours.
+- Predictable Costs: Billing per hour leads to more predictable costs, especially when there is a clear understanding of usage patterns.
+
+## Open Models On-Premise
+
+On-premise models mean that you host open models on your machine/infrastructure.
+
+This choice is ideal for local experimentation.
+
+It is suitable in production scenarios where data privacy concerns drive the decision not to transmit data to external providers and you have ample computational resources.
+
+### Local Experimentation
+
+- GPU: [`HuggingFaceLocalGenerator`](../huggingfacelocalgenerator.mdx) is based on the Hugging Face Transformers library. This is good for experimentation when you have some GPU resources (for example, in Colab). If GPU resources are limited, alternative quantization options like bitsandbytes, GPTQ, and AWQ are supported. For more performant solutions in production use cases, refer to the options below.
+- CPU (+ GPU if available): [`LlamaCppGenerator`](../llamacppgenerator.mdx) uses the Llama.cpp library –  a project written in C/C++ for efficient inference of LLMs. In particular, it employs the quantized GGUF format, suitable for running these models on standard machines (even without GPUs). If GPU resources are available, some model layers can be offloaded to GPU for enhanced speed.
+- CPU (+ GPU if available): [`OllamaGenerator`](../ollamagenerator.mdx) is based on the Ollama project, acting like Docker for LLMs. It provides a simple way to package and deploy these models. Internally based on the Llama.cpp library, it offers a more streamlined process for running on various platforms.
+
+### Serving LLMs in Production
+
+The following solutions are suitable if you want to run Language Models in production and have GPU resources available. They use innovative techniques for fast inference and efficient handling of numerous concurrent requests.
+
+- vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. Haystack supports vLLM through the OpenAI Generators.
+- Hugging Face API Generators, when used to query a TGI instance deployed on-premise. Hugging Face Text Generation Inference is a toolkit for efficiently deploying and serving LLMs.

docs-website/versioned_docs/version-2.18/pipeline-components/generators/guides-to-generators/function-calling.mdx

@@ -0,0 +1,126 @@
+---
+title: "Function Calling"
+id: function-calling
+slug: "/function-calling"
+description: "Learn about function calling and how to use it as a tool in Haystack."
+---
+
+# Function Calling
+
+Learn about function calling and how to use it as a tool in Haystack.
+
+Function calling is a powerful feature that significantly enhances the capabilities of Large Language Models (LLMs). It enables better functionality, immediate data access, and interaction, and sets up for integration with external APIs and services. Function calling turns LLMs into adaptable tools for various use case scenarios.
+
+## Use Cases
+
+Function calling is useful for a variety of purposes, but two main points are particularly notable:
+
+1. **Enhanced LLM Functionality**: Function calling enhances the capabilities of LLMs beyond just text generation. It allows to convert human-generated prompts into precise function invocation descriptors. These descriptors can then be used by connected LLM frameworks to perform computations, manipulate data, and interact with external APIs. This expansion of functionality makes LLMs adaptable tools for a wide array of tasks and industries.
+2. **Real-Time Data Access and Interaction**: Function calling lets LLMs create function calls that access and interact with real-time data. This is necessary for apps that need current data, like news, weather, or financial market updates. By giving access to the latest information, this feature greatly improves the usefulness and trustworthiness of LLMs in changing and time-critical situations.
+
+:::warning
+Important to Note
+
+The model doesn't actually call the function. Function calling returns JSON with the name of a function and the arguments to invoke it.
+:::
+
+## Example
+
+In the most simple form, Haystack users can invoke function calling by interacting directly with ChatGenerators. In this example, the human prompt “What's the weather like in Berlin?” is converted into a method parameter invocation descriptor that can, in turn, be passed off to some hypothetical weather service:
+
+```python
+import json
+
+from typing import Dict, Any, List
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "get_current_weather",
+            "description": "Get the current weather",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "location": {
+                        "type": "string",
+                        "description": "The city and state, e.g. San Francisco, CA",
+                    },
+                    "format": {
+                        "type": "string",
+                        "enum": ["celsius", "fahrenheit"],
+                        "description": "The temperature unit to use. Infer this from the users location.",
+                    },
+                },
+                "required": ["location", "format"],
+            },
+        },
+    },
+]
+messages = [ChatMessage.from_user("What's the weather like in Berlin?")]
+generator = OpenAIChatGenerator()
+response = generator.run(messages=messages, generation_kwargs={"tools": tools})
+response_msg = response["replies"][0]
+
+messages.append(response_msg)
+print(response_msg)
+```
+
+Let’s pretend that the hypothetical weather service responded with some JSON response of the current weather data in Berlin:
+
+```python
+weather_response = [
+    {
+        "id": "response_uhGNifLfopt5JrCUxXw1L3zo",
+        "status": "success",
+        "function": {
+            "name": "get_current_weather",
+            "arguments": {"location": "Berlin", "format": "celsius"},
+        },
+        "data": {
+            "location": "Berlin",
+            "temperature": 18,
+            "weather_condition": "Partly Cloudy",
+            "humidity": "60%",
+            "wind_speed": "15 km/h",
+            "observation_time": "2024-03-05T14:00:00Z",
+        },
+    },
+]
+```
+
+We would normally pack the response back into [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx) and add it to a list of messages:
+
+```python
+fcm = ChatMessage.from_function(
+    content=json.dumps(weather_response),
+    name="get_current_weather",
+)
+messages.append(fcm)
+```
+
+Sending these messages back to LLM enables the model to understand the context of the ongoing LLM interaction through `ChatMessage` list and respond back with a human-readable weather report for Berlin:
+
+```python
+response = generator.run(messages=messages)
+response_msg = response["replies"][0]
+
+print(response_msg.content)
+```
+
+## Additional References
+
+Haystack 2.0 introduces a better way to call functions using pipelines.
+
+For example, you can easily connect an LLM with a ChatGenerator to an external service using an OpenAPI specification. This lets you resolve service parameters with function calls and then use those parameters to invoke the external service. The service's response is added back into the LLM's context window. This method supports real-time, retriever-augmented generation that works with any OpenAPI-compliant service. It's a big improvement in how LLMs can use external structured data and functionalities.
+
+For more information and examples, see the documentation on [`OpenAPIServiceToFunctions`](../../converters/openapiservicetofunctions.mdx) and [`OpenAPIServiceConnector`](../../connectors/openapiserviceconnector.mdx).
+
+:notebook: **Tutorial: **[Building a Chat Application with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling)
+
+🧑‍🍳 **Cookbooks:**
+
+- [Function Calling with OpenAIChatGenerator](https://haystack.deepset.ai/cookbook/function_calling_with_openaichatgenerator)
+- [Information Extraction with Gorilla](https://haystack.deepset.ai/cookbook/information-extraction-gorilla)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/guides-to-generators/generators-vs-chat-generators.mdx

@@ -0,0 +1,130 @@
+---
+title: "Generators vs Chat Generators"
+id: generators-vs-chat-generators
+slug: "/generators-vs-chat-generators"
+description: "This page explains the difference between Generators and Chat Generators in Haystack. It emphasizes choosing the right Generator based on the use case and model."
+---
+
+# Generators vs Chat Generators
+
+This page explains the difference between Generators and Chat Generators in Haystack. It emphasizes choosing the right Generator based on the use case and model.
+
+## Input/Output
+
+|             | **Generators**    | **Chat Generators**                                      |
+| ----------- | ----------------- | -------------------------------------------------------- |
+| **Inputs**  | String (a prompt) | A list of [ChatMessages](../../../concepts/data-classes/chatmessage.mdx) |
+| **Outputs** | Text              | ChatMessage (in "replies")                               |
+
+## Pick the Right Class
+
+### Overview
+
+The choice between Generators (or text Generators) and Chat Generators depends on your use case and the underlying model.
+
+As highlighted by the different input and output characteristics above, Generators and Chat Generators are distinct, often interacting with different models through calls to different APIs. Therefore, they are not automatically interchangeable.
+
+:::tip
+Multi-turn Interactions
+
+If you anticipate a two-way interaction with the Language Model in a chat scenario, opting for a Chat Generator is generally better. This choice ensures a more structured and straightforward interaction with the Language Model.
+:::
+
+Chat Generators use Chat Messages. They can accommodate roles like "system", "user", "assistant", and even "function", enabling a more structured and nuanced interaction with Language Models. Chat Generators can handle many interactions, including complex queries, mixed conversations using tools, resolving function names and parameters from free text, and more. The format of Chat Messages is also helpful in reducing off-topic responses. Chat Generators are better at keeping the conversation on track by providing a consistent context.
+
+### Function Calling
+
+Some Chat Generators allow to leverage the function-calling capabilities of the models by passing tool/function definitions.
+
+If you'd like to learn more, read the introduction to [Function Calling](function-calling.mdx) in our docs.
+
+Or, you can find more information in relevant providers’ documentation:
+
+- [Function calling](https://platform.openai.com/docs/guides/function-calling) for [`OpenAIChatGenerator`](../openaichatgenerator.mdx)
+- Gemini [function calling](https://codelabs.developers.google.com/codelabs/gemini-function-calling#0) for [`VertexAIGeminiChatGenerator`](../vertexaigeminichatgenerator.mdx)
+
+### Compatibility Exceptions
+
+- The [`HuggingFaceLocalGenerator`](../huggingfacelocalgenerator.mdx) is compatible with Chat models, although the [`HuggingFaceLocalChatGenerator`](../huggingfacelocalchatgenerator.mdx) is more suitable.
+
+In such cases, opting for a Chat Generator simplifies the process, as Haystack handles the conversion of Chat Messages to a prompt that’s fit for the selected model.
+
+### No Corresponding Chat Generator
+
+If a Generator does not have a corresponding Chat Generator, this does not imply that the Generator cannot be utilized in a chat scenario.
+
+For example, [`LlamaCppGenerator`](../llamacppgenerator.mdx) can be used with both chat and non-chat models.
+However, without the `ChatMessage` data class, you need to pay close attention to the model's prompt template and adhere to it.
+
+#### Chat (Prompt) Template
+
+The chat template may be available on the Model card on Hugging Face for open Language Models in a human-readable form.
+See an example for [argilla/notus-7b-v1](https://huggingface.co/argilla/notus-7b-v1#prompt-template) model on the Hugging Face.
+
+Usually, it is also available as a Jinja template in the tokenizer_config.json.
+Here’s an example for [argilla/notus-7b-v1](https://huggingface.co/argilla/notus-7b-v1/blob/main/tokenizer_config.json#L34):
+
+```json
+{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}
+```
+
+## Different Types of Language Models
+
+:::note
+Topic Exploration
+
+This field is young, constantly evolving, and distinctions are not always possible and precise.
+:::
+
+The training of Generative Language Models involves several phases, yielding distinct models.
+
+### From Pretraining to Base Language Models
+
+In the pretraining phase, models are trained on vast amounts of raw text in an unsupervised manner. During this stage, the model acquires the ability to generate statistically plausible text completions.
+
+For instance, given the prompt “What is music...” the pretrained model can generate diverse plausible completions:
+
+- Adding more context: “...to your ears?”
+- Adding follow-up questions: “? What is sound? What is harmony?”
+- Providing an answer: “Music is a form of artistic expression…”
+
+The model that emerges from this pretraining is commonly referred to as the **base Language Model**.
+Examples include [meta-llama/Llama-2-70b](https://huggingface.co/meta-llama/Llama-2-70b-hf) and [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1).
+
+Using base Language Models is infrequent in practical applications, as they cannot follow instructions or engage in conversation.
+
+If you want to experiment with them, use the Haystack text Generators.
+
+### Supervised Fine Tuning (SFT) and Alignment with Human Preferences
+
+To make the language model helpful in real applications, two additional training steps are usually performed.
+
+- Supervised Fine Tuning: The language Model is further trained on a dataset containing instruction-response pairs or multi-turn interactions. Depending on the dataset, the model can acquire the capability to follow instructions or engage in chat.
+  _If model training stops at this point, it may perform well on some benchmarks, but it does not behave in a way that aligns with human user preferences._
+- Alignment with Human Preferences: This crucial step ensures that the Language Model aligns with human intent. Various techniques, such as RLHF and DPO, can be employed.
+  _To learn more about these techniques and this evolving landscape, you can read [this blog post](https://ai-scholar.tech/en/articles/rlhf%2FDirect-Preference-Optimization)._
+
+After these phases, a Language Model suitable for practical applications is obtained.
+Examples include [meta-llama/Llama-2-70b-chat-hf](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf) and [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2).
+
+#### Instruct vs Chat Language Models
+
+Instruct models are trained to follow instructions, while Chat models are trained for multi-turn conversations.
+
+This information is sometimes evident in the model name (meta-llama/Llama-2-70b-**chat**-hf, mistralai/Mistral-7B-**Instruct**-v0.2) or within the accompanying model card.
+
+- For Chat Models, employing Chat Generators is the most natural choice.
+- Should you opt to utilize Instruct models for single-turn interactions, turning to text Generators is recommended.
+
+It's worth noting that many recent Instruct models are equipped with a [chat template](#chat-prompt-template). An example of this is mistralai/Mistral-7B-Instruct-v0.2 [chat template](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/blob/main/tokenizer_config.json#L42).
+
+Utilizing a Chat Generator is the optimal choice if the model features a Chat template and you intend to use it in chat scenarios. In these cases, you can expect out-of-the-box support for Chat Messages, and you don’t need to manually apply the aforementioned template.
+
+:::warning
+Caution
+
+The distinction between Instruct and Chat models is not a strict dichotomy.
+
+- Following pre-training, Supervised Fine Tuning (SFT) and Alignment with Human Preferences can be executed multiple times using diverse datasets. In some cases, the differentiation between Instruct and Chat models may not be particularly meaningful.
+- Some open Language Models on Hugging Face lack explicit indications of their nature.
+:::

docs-website/versioned_docs/version-2.18/pipeline-components/generators/huggingfaceapichatgenerator.mdx

@@ -0,0 +1,222 @@
+---
+title: "HuggingFaceAPIChatGenerator"
+id: huggingfaceapichatgenerator
+slug: "/huggingfaceapichatgenerator"
+description: "This generator enables chat completion using various Hugging Face APIs."
+---
+
+# HuggingFaceAPIChatGenerator
+
+This generator enables chat completion using various Hugging Face APIs.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "api_type": The type of Hugging Face API to use  <br /> <br />"api_params": A dictionary with one of the following keys:  <br /> <br />- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`."token": The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var. |
+| **Mandatory run variables** | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables** | “replies”: A list of replies of the LLM to the input chat |
+| **API reference** | [Generators](/reference/generators-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/hugging_face_api.py |
+
+## Overview
+
+`HuggingFaceAPIChatGenerator` can be used to generate chat completions using different Hugging Face APIs:
+
+- [Serverless Inference API (Inference Providers)](https://huggingface.co/docs/inference-providers) - free tier available
+- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
+- [Self-hosted Text Generation Inference](https://github.com/huggingface/text-generation-inference)
+
+This component's main input is a list of `ChatMessage` objects. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. For more information, check out our [`ChatMessage` docs](../../concepts/data-classes/chatmessage.mdx).
+
+:::note
+This component is designed for chat completion, so it expects a list of messages, not a single string. If you want to use Hugging Face APIs for simple text generation (such as translation or summarization tasks) or don't want to use the `ChatMessage` object, use [`HuggingFaceAPIGenerator`](huggingfaceapigenerator.mdx) instead.
+
+:::
+
+The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
+The token is needed:
+
+- If you use the Serverless Inference API, or
+- If you use the Inference Endpoints.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+### On its own
+
+#### Using Serverless Inference API (Inference Providers) - Free Tier Available
+
+This API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It's rate-limited and not meant for production.
+
+To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
+The Generator expects the `model` in `api_params`. It's also recommended to specify a `provider` for better performance and reliability.
+
+```python
+from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+from haystack.utils.hf import HFGenerationAPIType
+
+messages = [
+    ChatMessage.from_system("\\nYou are a helpful, respectful and honest assistant"),
+    ChatMessage.from_user("What's Natural Language Processing?"),
+]
+
+## the api_type can be expressed using the HFGenerationAPIType enum or as a string
+api_type = HFGenerationAPIType.SERVERLESS_INFERENCE_API
+api_type = "serverless_inference_api"  # this is equivalent to the above
+
+generator = HuggingFaceAPIChatGenerator(
+    api_type=api_type,
+    api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"},
+    token=Secret.from_env_var("HF_API_TOKEN"),
+)
+
+result = generator.run(messages)
+print(result)
+```
+
+#### Using Paid Inference Endpoints
+
+In this case, a private instance of the model is deployed by Hugging Face, and you typically pay per hour.
+
+To understand how to spin up an Inference Endpoint, visit [Hugging Face documentation](https://huggingface.co/inference-endpoints/dedicated).
+
+Additionally, in this case, you need to provide your Hugging Face token.
+The Generator expects the `url` of your endpoint in `api_params`.
+
+```python
+from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+
+messages = [
+    ChatMessage.from_system("\\nYou are a helpful, respectful and honest assistant"),
+    ChatMessage.from_user("What's Natural Language Processing?"),
+]
+
+generator = HuggingFaceAPIChatGenerator(
+    api_type="inference_endpoints",
+    api_params={"url": "<your-inference-endpoint-url>"},
+    token=Secret.from_env_var("HF_API_TOKEN"),
+)
+
+result = generator.run(messages)
+print(result)
+```
+
+#### Using Serverless Inference API (Inference Providers) with Text+Image Input
+
+You can also use this component with multimodal models that support both text and image input:
+
+```python
+from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
+from haystack.dataclasses import ChatMessage, ImageContent
+from haystack.utils import Secret
+from haystack.utils.hf import HFGenerationAPIType
+
+## Create an image from file path, URL, or base64
+image = ImageContent.from_file_path("path/to/your/image.jpg")
+
+## Create a multimodal message with both text and image
+messages = [
+    ChatMessage.from_user(content_parts=["Describe this image in detail", image]),
+]
+
+generator = HuggingFaceAPIChatGenerator(
+    api_type=HFGenerationAPIType.SERVERLESS_INFERENCE_API,
+    api_params={
+        "model": "Qwen/Qwen2.5-VL-7B-Instruct",  # Vision Language Model
+        "provider": "hyperbolic",
+    },
+    token=Secret.from_token("<your-api-key>"),
+)
+
+result = generator.run(messages)
+print(result)
+```
+
+#### Using Self-Hosted Text Generation Inference (TGI)
+
+[Hugging Face Text Generation Inference](https://github.com/huggingface/text-generation-inference) is a toolkit for efficiently deploying and serving LLMs.
+
+While it powers the most recent versions of Serverless Inference API and Inference Endpoints, it can be used easily on-premise through Docker.
+
+For example, you can run a TGI container as follows:
+
+```shell
+model=HuggingFaceH4/zephyr-7b-beta
+volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
+
+docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.4 --model-id $model
+```
+
+For more information, refer to the [official TGI repository](https://github.com/huggingface/text-generation-inference).
+
+The Generator expects the `url` of your TGI instance in `api_params`.
+
+```python
+from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+messages = [
+    ChatMessage.from_system("\\nYou are a helpful, respectful and honest assistant"),
+    ChatMessage.from_user("What's Natural Language Processing?"),
+]
+
+generator = HuggingFaceAPIChatGenerator(
+    api_type="text_generation_inference",
+    api_params={"url": "http://localhost:8080"},
+)
+
+result = generator.run(messages)
+print(result)
+```
+
+### In a pipeline
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack.utils import Secret
+from haystack.utils.hf import HFGenerationAPIType
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+llm = HuggingFaceAPIChatGenerator(
+    api_type=HFGenerationAPIType.SERVERLESS_INFERENCE_API,
+    api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"},
+    token=Secret.from_env_var("HF_API_TOKEN"),
+)
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+location = "Berlin"
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+result = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+
+print(result)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Build with Google Gemma: chat and RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/huggingfaceapigenerator.mdx

@@ -0,0 +1,188 @@
+---
+title: "HuggingFaceAPIGenerator"
+id: huggingfaceapigenerator
+slug: "/huggingfaceapigenerator"
+description: "This generator enables text generation using various Hugging Face APIs."
+---
+
+# HuggingFaceAPIGenerator
+
+This generator enables text generation using various Hugging Face APIs.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_type": The type of Hugging Face API to use  <br /> <br />"api_params": A dictionary with one of the following keys:  <br /> <br />- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`."token": The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and others |
+| **API reference** | [Generators](/reference/generators-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/hugging_face_api.py |
+
+## Overview
+
+`HuggingFaceAPIGenerator` can be used to generate text using different Hugging Face APIs:
+
+- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
+- [Self-hosted Text Generation Inference](https://github.com/huggingface/text-generation-inference)
+
+:::warning
+Important Note
+
+As of July 2025, the Hugging Face Inference API no longer offers generative models through the `text_generation` endpoint. Generative models are now only available through providers supporting the `chat_completion` endpoint. As a result, this component might no longer work with the Hugging Face Inference API.
+
+Use the [`HuggingFaceAPIChatGenerator`](huggingfaceapichatgenerator.mdx) component instead, which supports the `chat_completion` endpoint and works with the free Serverless Inference API.
+:::
+
+:::note
+This component is designed for text generation, not for chat. If you want to use these LLMs for chat, use [`HuggingFaceAPIChatGenerator`](huggingfaceapichatgenerator.mdx) instead.
+
+:::
+
+The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
+The token is needed when you use the Inference Endpoints.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+### On its own
+
+#### Using Paid Inference Endpoints
+
+In this case, a private instance of the model is deployed by Hugging Face, and you typically pay per hour.
+
+To understand how to spin up an Inference Endpoint, visit [Hugging Face documentation](https://huggingface.co/inference-endpoints/dedicated).
+
+Additionally, in this case, you need to provide your Hugging Face token.
+The Generator expects the `url` of your endpoint in `api_params`.
+
+```python
+from haystack.components.generators import HuggingFaceAPIGenerator
+from haystack.utils import Secret
+
+generator = HuggingFaceAPIGenerator(
+    api_type="inference_endpoints",
+    api_params={"url": "<your-inference-endpoint-url>"},
+    token=Secret.from_token("<your-api-key>"),
+)
+
+result = generator.run(prompt="What's Natural Language Processing?")
+print(result)
+```
+
+#### Using Self-Hosted Text Generation Inference (TGI)
+
+[Hugging Face Text Generation Inference](https://github.com/huggingface/text-generation-inference) is a toolkit for efficiently deploying and serving LLMs.
+
+While it powers the most recent versions of Serverless Inference API and Inference Endpoints, it can be used easily on-premise through Docker.
+
+For example, you can run a TGI container as follows:
+
+```shell
+model=mistralai/Mistral-7B-v0.1
+volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
+
+docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.4 --model-id $model
+```
+
+For more information, refer to the [official TGI repository](https://github.com/huggingface/text-generation-inference).
+
+The Generator expects the `url` of your TGI instance in `api_params`.
+
+```python
+from haystack.components.generators import HuggingFaceAPIGenerator
+
+generator = HuggingFaceAPIGenerator(
+    api_type="text_generation_inference",
+    api_params={"url": "http://localhost:8080"},
+)
+
+result = generator.run(prompt="What's Natural Language Processing?")
+print(result)
+```
+
+#### Using the Free Serverless Inference API (Not Recommended)
+
+:::warning
+This example might not work as the Hugging Face Inference API no longer offers models that support the `text_generation` endpoint. Use the [`HuggingFaceAPIChatGenerator`](huggingfaceapichatgenerator.mdx) for generative models through the `chat_completion` endpoint.
+
+:::
+
+Formerly known as (free) Hugging Face Inference API, this API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It's rate-limited and not meant for production.
+
+To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
+The Generator expects the `model` in `api_params`.
+
+```python
+from haystack.components.generators import HuggingFaceAPIGenerator
+from haystack.utils import Secret
+
+generator = HuggingFaceAPIGenerator(
+    api_type="serverless_inference_api",
+    api_params={"model": "HuggingFaceH4/zephyr-7b-beta"},
+    token=Secret.from_token("<your-api-key>"),
+)
+
+result = generator.run(prompt="What's Natural Language Processing?")
+print(result)
+```
+
+### In a pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.components.generators import HuggingFaceAPIGenerator
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack import Document
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="Rome is the capital of Italy"),
+        Document(content="Paris is the capital of France"),
+    ],
+)
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+
+generator = HuggingFaceAPIGenerator(
+    api_type="inference_endpoints",
+    api_params={"url": "<your-inference-endpoint-url>"},
+    token=Secret.from_token("<your-api-key>"),
+)
+
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("llm", generator)
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(res)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbooks:
+
+- [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)
+- [Information Extraction with Raven](https://haystack.deepset.ai/cookbook/information_extraction_raven)
+- [Web QA with Mixtral-8x7B-Instruct-v0.1](https://haystack.deepset.ai/cookbook/mixtral-8x7b-for-web-qa)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/huggingfacelocalchatgenerator.mdx

@@ -0,0 +1,90 @@
+---
+title: "HuggingFaceLocalChatGenerator"
+id: huggingfacelocalchatgenerator
+slug: "/huggingfacelocalchatgenerator"
+description: "Provides an interface for chat completion using a Hugging Face model that runs locally."
+---
+
+# HuggingFaceLocalChatGenerator
+
+Provides an interface for chat completion using a Hugging Face model that runs locally.
+
+|                                        |                                                                                                              |
+| :------------------------------------- | :----------------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                         |
+| **Mandatory init variables**           | "token": The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var.                   |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat         |
+| **Output variables**                   | “replies”: A list of strings with all the replies generated by the LLM                                       |
+| **API reference**                      | [Generators](/reference/generators-api)                                                                             |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/hugging_face_local.py |
+
+## Overview
+
+Keep in mind that if LLMs run locally, you may need a powerful machine to run them. This depends strongly on the model you select and its parameter count.
+
+:::note
+This component is designed for chat completion, not for text generation. If you want to use Hugging Face LLMs for text generation, use [`HuggingFaceLocalGenerator`](huggingfacelocalgenerator.mdx) instead.
+
+:::
+
+For remote file authorization, this component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token`:
+
+```python
+local_generator = HuggingFaceLocalChatGenerator(
+    token=Secret.from_token("<your-api-key>"),
+)
+```
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+### On its own
+
+```python
+from haystack.components.generators.chat import HuggingFaceLocalChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = HuggingFaceLocalChatGenerator(model="HuggingFaceH4/zephyr-7b-beta")
+generator.warm_up()
+messages = [ChatMessage.from_user("What's Natural Language Processing? Be brief.")]
+print(generator.run(messages))
+```
+
+### In a Pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.builders.prompt_builder import ChatPromptBuilder
+from haystack.components.generators.chat import HuggingFaceLocalChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+
+prompt_builder = ChatPromptBuilder()
+llm = HuggingFaceLocalChatGenerator(
+    model="HuggingFaceH4/zephyr-7b-beta",
+    token=Secret.from_env_var("HF_API_TOKEN"),
+)
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+location = "Berlin"
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/huggingfacelocalgenerator.mdx

@@ -0,0 +1,121 @@
+---
+title: "HuggingFaceLocalGenerator"
+id: huggingfacelocalgenerator
+slug: "/huggingfacelocalgenerator"
+description: "`HuggingFaceLocalGenerator` provides an interface to generate text using a Hugging Face model that runs locally."
+---
+
+# HuggingFaceLocalGenerator
+
+`HuggingFaceLocalGenerator` provides an interface to generate text using a Hugging Face model that runs locally.
+
+|                                        |                                                                                                         |
+| :------------------------------------- | :------------------------------------------------------------------------------------------------------ |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)                                                          |
+| **Mandatory init variables**           | "token": The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var.              |
+| **Mandatory run variables**            | “prompt”: A string containing the prompt for the LLM                                                    |
+| **Output variables**                   | “replies”: A list of strings with all the replies generated by the LLM                                  |
+| **API reference**                      | [Generators](/reference/generators-api)                                                                        |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/hugging_face_local.py |
+
+## Overview
+
+Keep in mind that if LLMs run locally, you may need a powerful machine to run them. This depends strongly on the model you select and its parameter count.
+
+:::note
+Looking for chat completion?
+
+This component is designed for text generation, not for chat. If you want to use Hugging Face LLMs for chat, consider using [`HuggingFaceLocalChatGenerator`](huggingfacelocalchatgenerator.mdx) instead.
+:::
+
+For remote files authorization, this component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token`:
+
+```python
+local_generator = HuggingFaceLocalGenerator(token=Secret.from_token("<your-api-key>"))
+```
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+### On its own
+
+```python
+from haystack.components.generators import HuggingFaceLocalGenerator
+
+generator = HuggingFaceLocalGenerator(
+    model="google/flan-t5-large",
+    task="text2text-generation",
+    generation_kwargs={
+        "max_new_tokens": 100,
+        "temperature": 0.9,
+    },
+)
+
+generator.warm_up()
+print(generator.run("Who is the best American actor?"))
+## {'replies': ['john wayne']}
+```
+
+### In a Pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.components.generators import HuggingFaceLocalGenerator
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack import Document
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="Rome is the capital of Italy"),
+        Document(content="Paris is the capital of France"),
+    ],
+)
+
+generator = HuggingFaceLocalGenerator(
+    model="google/flan-t5-large",
+    task="text2text-generation",
+    generation_kwargs={
+        "max_new_tokens": 100,
+        "temperature": 0.9,
+    },
+)
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("llm", generator)
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(res)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbooks:
+
+- [Use Zephyr 7B Beta with Hugging Face for RAG](https://haystack.deepset.ai/cookbook/zephyr-7b-beta-for-rag)
+- [Information Extraction with Gorilla](https://haystack.deepset.ai/cookbook/information-extraction-gorilla)
+- [RAG on the Oscars using Llama 3.1 models](https://haystack.deepset.ai/cookbook/llama3_rag)
+- [Agentic RAG with Llama 3.2 3B](https://haystack.deepset.ai/cookbook/llama32_agentic_rag)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/llamacppchatgenerator.mdx

@@ -0,0 +1,268 @@
+---
+title: "LlamaCppChatGenerator"
+id: llamacppchatgenerator
+slug: "/llamacppchatgenerator"
+description: "`LlamaCppGenerator` enables chat completion using an LLM running on Llama.cpp."
+---
+
+# LlamaCppChatGenerator
+
+`LlamaCppGenerator` enables chat completion using an LLM running on Llama.cpp.
+
+|                                        |                                                                                                                           |
+| :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------ |
+| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx)                                                                    |
+| **Mandatory init variables**           | "model": The path of the model to use                                                                                     |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  instances representing the input messages          |
+| **Output variables**                   | “replies”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  instances with all the replies generated by the LLM |
+| **API reference**                      | [Llama.cpp](/reference/integrations-llama-cpp)                                                                                   |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp                               |
+
+## Overview
+
+[Llama.cpp](https://github.com/ggml-org/llama.cpp) is a library written in C/C++ for efficient inference of Large Language Models. It leverages the efficient quantized GGUF format, dramatically reducing memory requirements and accelerating inference. This means it is possible to run LLMs efficiently on standard machines (even without GPUs).
+
+`Llama.cpp` uses the quantized binary file of the LLM in GGUF format, which can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf). `LlamaCppChatGenerator` supports models running on `Llama.cpp`  by taking the path to the locally saved GGUF file as `model` parameter at initialization.
+
+## Installation
+
+Install the `llama-cpp-haystack` package to use this integration:
+
+```shell
+pip install llama-cpp-haystack
+```
+
+### Using a different compute backend
+
+The default installation behavior is to build `llama.cpp` for CPU on Linux and Windows and use Metal on MacOS. To use other compute backends:
+
+1. Follow instructions on the [llama.cpp installation page](https://github.com/abetlen/llama-cpp-python#installation) to install [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) for your preferred compute backend.
+2. Install [llama-cpp-haystack](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp) using the command above.
+
+For example, to use `llama-cpp-haystack` with the **cuBLAS backend**, you have to run the following commands:
+
+```shell
+export GGML_CUDA=1
+CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python
+pip install llama-cpp-haystack
+```
+
+## Usage
+
+1. Download the GGUF version of the desired LLM. The GGUF versions of popular models can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf).
+2. Initialize `LlamaCppChatGenerator` with the path to the GGUF file and specify the required model and text generation parameters:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
+
+generator = LlamaCppChatGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+    model_kwargs={"n_gpu_layers": -1},
+    generation_kwargs={"max_tokens": 128, "temperature": 0.1},
+)
+generator.warm_up()
+messages = [ChatMessage.from_user("Who is the best American actor?")]
+result = generator.run(messages)
+```
+
+### Passing additional model parameters
+
+The `model`, `n_ctx`, `n_batch` arguments have been exposed for convenience and can be directly passed to the Generator during initialization as keyword arguments. Note that `model` translates to `llama.cpp`'s `model_path` parameter.
+
+The `model_kwargs` parameter can pass additional arguments when initializing the model. In case of duplication, these parameters override the `model`, `n_ctx`, and `n_batch` initialization parameters.
+
+See [Llama.cpp's LLM documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__init__) for more information on the available model arguments.
+
+**Note**: Llama.cpp automatically extracts the `chat_template` from the model metadata for applying formatting to ChatMessages. You can override the `chat_template` used by passing in a custom `chat_handler` or `chat_format` as a model parameter.
+
+For example, to offload the model to GPU during initialization:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = LlamaCppChatGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+    model_kwargs={"n_gpu_layers": -1},
+)
+generator.warm_up()
+messages = [ChatMessage.from_user("Who is the best American actor?")]
+result = generator.run(messages, generation_kwargs={"max_tokens": 128})
+generated_reply = result["replies"][0].content
+print(generated_reply)
+```
+
+### Passing text generation parameters
+
+The `generation_kwargs` parameter can pass additional generation arguments like `max_tokens`, `temperature`, `top_k`, `top_p`, and others to the model during inference.
+
+See [Llama.cpp's Chat Completion API documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.create_chat_completion) for more information on the available generation arguments.
+
+**Note**: JSON mode, Function Calling, and Tools are all supported as `generation_kwargs`. Please see the [llama-cpp-python GitHub README](https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#json-and-json-schema-mode) for more information on how to use them.
+
+For example, to set the `max_tokens` and `temperature`:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = LlamaCppChatGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+    generation_kwargs={"max_tokens": 128, "temperature": 0.1},
+)
+generator.warm_up()
+messages = [ChatMessage.from_user("Who is the best American actor?")]
+result = generator.run(messages)
+```
+
+The `generation_kwargs` can also be passed to the `run` method of the generator directly:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = LlamaCppChatGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+)
+generator.warm_up()
+messages = [ChatMessage.from_user("Who is the best American actor?")]
+result = generator.run(
+    messages,
+    generation_kwargs={"max_tokens": 128, "temperature": 0.1},
+)
+```
+
+### In a pipeline
+
+We use the `LlamaCppChatGenerator` in a Retrieval Augmented Generation pipeline on the [Simple Wikipedia](https://huggingface.co/datasets/pszemraj/simple_wikipedia) Dataset from Hugging Face and generate answers using the [OpenChat-3.5](https://huggingface.co/openchat/openchat-3.5-1210) LLM.
+
+Load the dataset:
+
+```python
+## Install HuggingFace Datasets using "pip install datasets"
+from datasets import load_dataset
+from haystack import Document, Pipeline
+from haystack.components.builders.answer_builder import AnswerBuilder
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.embedders import (
+    SentenceTransformersDocumentEmbedder,
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
+from haystack.components.writers import DocumentWriter
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.dataclasses import ChatMessage
+
+## Import LlamaCppChatGenerator
+from haystack_integrations.components.generators.llama_cpp import LlamaCppChatGenerator
+
+## Load first 100 rows of the Simple Wikipedia Dataset from HuggingFace
+dataset = load_dataset("pszemraj/simple_wikipedia", split="validation[:100]")
+
+docs = [
+    Document(
+        content=doc["text"],
+        meta={
+            "title": doc["title"],
+            "url": doc["url"],
+        },
+    )
+    for doc in dataset
+]
+```
+
+Index the documents to the `InMemoryDocumentStore` using the `SentenceTransformersDocumentEmbedder` and `DocumentWriter`:
+
+```python
+doc_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
+## Install sentence transformers using "pip install sentence-transformers"
+doc_embedder = SentenceTransformersDocumentEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+
+## Indexing Pipeline
+indexing_pipeline = Pipeline()
+indexing_pipeline.add_component(instance=doc_embedder, name="DocEmbedder")
+indexing_pipeline.add_component(
+    instance=DocumentWriter(document_store=doc_store),
+    name="DocWriter",
+)
+indexing_pipeline.connect("DocEmbedder", "DocWriter")
+
+indexing_pipeline.run({"DocEmbedder": {"documents": docs}})
+```
+
+Create the RAG pipeline and add the `LlamaCppChatGenerator` to it:
+
+```python
+system_message = ChatMessage.from_system(
+    """
+    Answer the question using the provided context.
+    Context:
+    {% for doc in documents %}
+        {{ doc.content }}
+    {% endfor %}
+    """,
+)
+user_message = ChatMessage.from_user("Question: {{question}}")
+assistent_message = ChatMessage.from_assistant("Answer: ")
+
+chat_template = [system_message, user_message, assistent_message]
+
+rag_pipeline = Pipeline()
+
+text_embedder = SentenceTransformersTextEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+
+## Load the LLM using LlamaCppChatGenerator
+model_path = "openchat-3.5-1210.Q3_K_S.gguf"
+generator = LlamaCppChatGenerator(model=model_path, n_ctx=4096, n_batch=128)
+
+rag_pipeline.add_component(
+    instance=text_embedder,
+    name="text_embedder",
+)
+rag_pipeline.add_component(
+    instance=InMemoryEmbeddingRetriever(document_store=doc_store, top_k=3),
+    name="retriever",
+)
+rag_pipeline.add_component(
+    instance=ChatPromptBuilder(template=chat_template),
+    name="prompt_builder",
+)
+rag_pipeline.add_component(instance=generator, name="llm")
+rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
+
+rag_pipeline.connect("text_embedder", "retriever")
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+rag_pipeline.connect("llm", "answer_builder")
+rag_pipeline.connect("retriever", "answer_builder.documents")
+```
+
+Run the pipeline:
+
+```python
+question = "Which year did the Joker movie release?"
+result = rag_pipeline.run(
+    {
+        "text_embedder": {"text": question},
+        "prompt_builder": {"question": question},
+        "llm": {"generation_kwargs": {"max_tokens": 128, "temperature": 0.1}},
+        "answer_builder": {"query": question},
+    },
+)
+
+generated_answer = result["answer_builder"]["answers"][0]
+print(generated_answer.data)
+## The Joker movie was released on October 4, 2019.
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/llamacppgenerator.mdx

@@ -0,0 +1,256 @@
+---
+title: "LlamaCppGenerator"
+id: llamacppgenerator
+slug: "/llamacppgenerator"
+description: "`LlamaCppGenerator` provides an interface to generate text using an LLM running on Llama.cpp."
+---
+
+# LlamaCppGenerator
+
+`LlamaCppGenerator` provides an interface to generate text using an LLM running on Llama.cpp.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "model": The path of the model to use |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count and others |
+| **API reference** | [Llama.cpp](/reference/integrations-llama-cpp) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp |
+
+## Overview
+
+[Llama.cpp](https://github.com/ggml-org/llama.cpp) is a library written in C/C++ for efficient inference of Large Language Models. It leverages the efficient quantized GGUF format, dramatically reducing memory requirements and accelerating inference. This means it is possible to run LLMs efficiently on standard machines (even without GPUs).
+
+`Llama.cpp` uses the quantized binary file of the LLM in GGUF format that can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf).  `LlamaCppGenerator` supports models running on `Llama.cpp`  by taking the path to the locally saved GGUF file as `model` parameter at initialization.
+
+## Installation
+
+Install the `llama-cpp-haystack` package:
+
+```bash
+pip install llama-cpp-haystack
+```
+
+### Using a different compute backend
+
+The default installation behavior is to build `llama.cpp` for CPU on Linux and Windows and use Metal on MacOS. To use other compute backends:
+
+1. Follow instructions on the [llama.cpp installation page](https://github.com/abetlen/llama-cpp-python#installation) to install [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) for your preferred compute backend.
+2. Install [llama-cpp-haystack](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/llama_cpp) using the command above.
+
+For example, to use `llama-cpp-haystack` with the **cuBLAS backend**, you have to run the following commands:
+
+```bash
+export GGML_CUDA=1
+CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python
+pip install llama-cpp-haystack
+```
+
+## Usage
+
+1. You need to download the GGUF version of the desired LLM. The GGUF versions of popular models can be downloaded from [Hugging Face](https://huggingface.co/models?library=gguf).
+2. Initialize a `LlamaCppGenerator` with the path to the GGUF file and also specify the required model and text generation parameters:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
+
+generator = LlamaCppGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+    model_kwargs={"n_gpu_layers": -1},
+    generation_kwargs={"max_tokens": 128, "temperature": 0.1},
+)
+generator.warm_up()
+prompt = f"Who is the best American actor?"
+result = generator.run(prompt)
+```
+
+### Passing additional model parameters
+
+The `model`, `n_ctx`, `n_batch` arguments have been exposed for convenience and can be directly passed to the Generator during initialization as keyword arguments. Note that `model` translates to `llama.cpp`'s `model_path` parameter.
+
+The `model_kwargs` parameter can pass additional arguments when initializing the model. In case of duplication, these parameters override the `model`, `n_ctx`, and `n_batch` initialization parameters.
+
+See [Llama.cpp's LLM documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__init__) for more information on the available model arguments.
+
+For example, to offload the model to GPU during initialization:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
+
+generator = LlamaCppGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+    model_kwargs={"n_gpu_layers": -1},
+)
+generator.warm_up()
+prompt = f"Who is the best American actor?"
+result = generator.run(prompt, generation_kwargs={"max_tokens": 128})
+generated_text = result["replies"][0]
+print(generated_text)
+```
+
+### Passing text generation parameters
+
+The `generation_kwargs` parameter can pass additional generation arguments like `max_tokens`, `temperature`, `top_k`, `top_p`, and others to the model during inference.
+
+See [Llama.cpp's Completion API documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.create_completion) for more information on the available generation arguments.
+
+For example, to set the `max_tokens` and `temperature`:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
+
+generator = LlamaCppGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+    generation_kwargs={"max_tokens": 128, "temperature": 0.1},
+)
+generator.warm_up()
+prompt = f"Who is the best American actor?"
+result = generator.run(prompt)
+```
+
+The `generation_kwargs` can also be passed to the `run` method of the generator directly:
+
+```python
+from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
+
+generator = LlamaCppGenerator(
+    model="/content/openchat-3.5-1210.Q3_K_S.gguf",
+    n_ctx=512,
+    n_batch=128,
+)
+generator.warm_up()
+prompt = f"Who is the best American actor?"
+result = generator.run(
+    prompt,
+    generation_kwargs={"max_tokens": 128, "temperature": 0.1},
+)
+```
+
+### Using in a Pipeline
+
+We use the `LlamaCppGenerator` in a Retrieval Augmented Generation pipeline on the [Simple Wikipedia](https://huggingface.co/datasets/pszemraj/simple_wikipedia) Dataset from HuggingFace and generate answers using the [OpenChat-3.5](https://huggingface.co/openchat/openchat-3.5-1210) LLM.
+
+Load the dataset:
+
+```python
+## Install HuggingFace Datasets using "pip install datasets"
+from datasets import load_dataset
+from haystack import Document, Pipeline
+from haystack.components.builders.answer_builder import AnswerBuilder
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.components.embedders import (
+    SentenceTransformersDocumentEmbedder,
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
+from haystack.components.writers import DocumentWriter
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+## Import LlamaCppGenerator
+from haystack_integrations.components.generators.llama_cpp import LlamaCppGenerator
+
+## Load first 100 rows of the Simple Wikipedia Dataset from HuggingFace
+dataset = load_dataset("pszemraj/simple_wikipedia", split="validation[:100]")
+
+docs = [
+    Document(
+        content=doc["text"],
+        meta={
+            "title": doc["title"],
+            "url": doc["url"],
+        },
+    )
+    for doc in dataset
+]
+```
+
+Index the documents to the `InMemoryDocumentStore` using the `SentenceTransformersDocumentEmbedder` and `DocumentWriter`:
+
+```python
+doc_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
+doc_embedder = SentenceTransformersDocumentEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+
+## Indexing Pipeline
+indexing_pipeline = Pipeline()
+indexing_pipeline.add_component(instance=doc_embedder, name="DocEmbedder")
+indexing_pipeline.add_component(
+    instance=DocumentWriter(document_store=doc_store),
+    name="DocWriter",
+)
+indexing_pipeline.connect(connect_from="DocEmbedder", connect_to="DocWriter")
+
+indexing_pipeline.run({"DocEmbedder": {"documents": docs}})
+```
+
+Create the Retrieval Augmented Generation (RAG) pipeline and add the `LlamaCppGenerator` to it:
+
+```python
+## Prompt Template for the https://huggingface.co/openchat/openchat-3.5-1210 LLM
+prompt_template = """GPT4 Correct User: Answer the question using the provided context.
+Question: {{question}}
+Context:
+{% for doc in documents %}
+    {{ doc.content }}
+{% endfor %}
+<|end_of_turn|>
+GPT4 Correct Assistant:
+"""
+
+rag_pipeline = Pipeline()
+
+text_embedder = SentenceTransformersTextEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+
+## Load the LLM using LlamaCppGenerator
+model_path = "openchat-3.5-1210.Q3_K_S.gguf"
+generator = LlamaCppGenerator(model=model_path, n_ctx=4096, n_batch=128)
+
+rag_pipeline.add_component(
+    instance=text_embedder,
+    name="text_embedder",
+)
+rag_pipeline.add_component(
+    instance=InMemoryEmbeddingRetriever(document_store=doc_store, top_k=3),
+    name="retriever",
+)
+rag_pipeline.add_component(
+    instance=PromptBuilder(template=prompt_template),
+    name="prompt_builder",
+)
+rag_pipeline.add_component(instance=generator, name="llm")
+rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
+
+rag_pipeline.connect("text_embedder", "retriever")
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+rag_pipeline.connect("llm.replies", "answer_builder.replies")
+rag_pipeline.connect("retriever", "answer_builder.documents")
+```
+
+Run the pipeline:
+
+```python
+question = "Which year did the Joker movie release?"
+result = rag_pipeline.run(
+    {
+        "text_embedder": {"text": question},
+        "prompt_builder": {"question": question},
+        "llm": {"generation_kwargs": {"max_tokens": 128, "temperature": 0.1}},
+        "answer_builder": {"query": question},
+    },
+)
+
+generated_answer = result["answer_builder"]["answers"][0]
+print(generated_answer.data)
+## The Joker movie was released on October 4, 2019.
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/llamastackchatgenerator.mdx

@@ -0,0 +1,118 @@
+---
+title: "LlamaStackChatGenerator"
+id: llamastackchatgenerator
+slug: "/llamastackchatgenerator"
+description: "This component enables chat completions using any model made available by inference providers on a Llama Stack server."
+---
+
+# LlamaStackChatGenerator
+
+This component enables chat completions using any model made available by inference providers on a Llama Stack server.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) |
+| **Mandatory init variables** | "model": The name of the model to use for chat completion.   This depends on the inference provider used for the Llama Stack Server. |
+| **Mandatory run variables** | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables** | “replies”: A list of alternative replies of the model to the input chat |
+| **API reference** | [Llama Stack](/reference/integrations-llama-stack) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/llama_stack |
+
+## Overview
+
+[Llama Stack](https://llama-stack.readthedocs.io/en/latest/index.html) provides building blocks and unified APIs to streamline the development of AI applications across various environments.
+
+The `LlamaStackChatGenerator` enables you to access any LLMs exposed by inference providers hosted on a Llama Stack server. It abstracts away the underlying provider details, allowing you to reuse the same client-side code regardless of the inference backend. For a list of supported providers and configuration options, refer to the [Llama Stack documentation](https://llama-stack.readthedocs.io/en/latest/providers/inference/index.html).
+
+This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
+
+It is also fully compatible with Haystack **Tools / Toolsets**, enabling function-calling capabilities with supported models.
+
+## Initialization
+
+To use this integration, you must have:
+
+- A running instance of a Llama Stack server (local or remote)
+- A valid model name supported by your selected inference provider
+
+Then initialize the `LlamaStackChatGenerator` by specifying the `model` name or ID. The value depends on the inference provider running on your server.
+
+**Examples:**
+
+- For Ollama: `model="ollama/llama3.2:3b"`
+- For vLLM: `model="meta-llama/Llama-3.2-3B"`
+
+**Note:** Switching the inference provider only requires updating the model name.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+To start using this integration, install the package with:
+
+```shell
+pip install llama-stack-haystack
+```
+
+### On its own
+
+```python
+import os
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.llama_stack import (
+    LlamaStackChatGenerator,
+)
+
+client = LlamaStackChatGenerator(model="ollama/llama3.2:3b")
+response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
+print(response["replies"])
+```
+
+#### With Streaming
+
+```python
+import os
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.llama_stack import (
+    LlamaStackChatGenerator,
+)
+from haystack.components.generators.utils import print_streaming_chunk
+
+client = LlamaStackChatGenerator(
+    model="ollama/llama3.2:3b",
+    streaming_callback=print_streaming_chunk,
+)
+response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
+print(response["replies"])
+```
+
+### In a pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.llama_stack import (
+    LlamaStackChatGenerator,
+)
+
+prompt_builder = ChatPromptBuilder()
+llm = LlamaStackChatGenerator(model="ollama/llama3.2:3b")
+
+pipe = Pipeline()
+pipe.add_component("builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("builder.prompt", "llm.messages")
+
+messages = [
+    ChatMessage.from_system("Give brief answers."),
+    ChatMessage.from_user("Tell me about {{city}}"),
+]
+
+response = pipe.run(
+    data={"builder": {"template": messages, "template_variables": {"city": "Berlin"}}},
+)
+print(response)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/metallamachatgenerator.mdx

@@ -0,0 +1,150 @@
+---
+title: "MetaLlamaChatGenerator"
+id: metallamachatgenerator
+slug: "/metallamachatgenerator"
+description: "This component enables chat completion with any model hosted available with Meta Llama API."
+---
+
+# MetaLlamaChatGenerator
+
+This component enables chat completion with any model hosted available with Meta Llama API.
+
+|                                        |                                                                                                             |
+| -------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                          |
+| **Mandatory init variables**           | “api_key”: A Meta Llama API key. Can be set with `LLAMA_API_KEY` env variable or passed to `init()` method. |
+| **Mandatory run variables**            | “messages:” A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                |
+| **Output variables**                   | “replies”: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                 |
+| **API reference**                      | [Meta Llama API](/reference/integrations-meta-llama)                                                               |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/meta_llama                |
+
+## Overview
+
+The `MetaLlamaChatGenerator` enables you to use multiple Meta Llama models by making chat completion calls to the Meta [Llama API](https://llama.developer.meta.com/?utm_source=partner-haystack&utm_medium=website). The default model is `Llama-4-Scout-17B-16E-Instruct-FP8`.
+
+Currently available models are:
+
+| Model ID                                 | Input context length | Output context length | Input Modalities | Output Modalities |
+| ---------------------------------------- | -------------------- | --------------------- | ---------------- | ----------------- |
+| `Llama-4-Scout-17B-16E-Instruct-FP8`     | 128k                 | 4028                  | Text, Image      | Text              |
+| `Llama-4-Maverick-17B-128E-Instruct-FP8` | 128k                 | 4028                  | Text, Image      | Text              |
+| `Llama-3.3-70B-Instruct`                 | 128k                 | 4028                  | Text             | Text              |
+| `Llama-3.3-8B-Instruct`                  | 128k                 | 4028                  | Text             | Text              |
+
+This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
+
+It is also fully compatible with Haystack [Tools](../../tools/tool.mdx) and [Toolsets](../../tools/toolset.mdx) that allow function-calling capabilities with supported models.
+
+### Initialization
+
+To use this integration, you must have a Meta Llama API key. You can provide it with the `LLAMA_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
+
+Then, install the `meta-llama-haystack` integration:
+
+```shell
+pip install meta-llama-haystack
+```
+
+### Streaming
+
+`MetaLlamaChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
+
+## Usage
+
+### On its own
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.meta_llama import (
+    MetaLlamaChatGenerator,
+)
+
+llm = MetaLlamaChatGenerator()
+response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
+print(response["replies"][0].text)
+```
+
+With streaming and model routing:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.meta_llama import (
+    MetaLlamaChatGenerator,
+)
+
+llm = MetaLlamaChatGenerator(
+    model="Llama-3.3-8B-Instruct",
+    streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
+)
+
+response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
+
+## check the model used for the response
+print("\n\n Model used: ", response["replies"][0].meta["model"])
+```
+
+### In a pipeline
+
+```python
+## To run this example, you will need to set a `LLAMA_API_KEY` environment variable.
+
+from haystack import Document, Pipeline
+from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.dataclasses import ChatMessage
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.utils import Secret
+
+from haystack_integrations.components.generators.meta_llama import (
+    MetaLlamaChatGenerator,
+)
+
+## Write documents to InMemoryDocumentStore
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+## Build a RAG pipeline
+prompt_template = [
+    ChatMessage.from_user(
+        "Given these documents, answer the question.\n"
+        "Documents:\n{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
+        "Question: {{question}}\n"
+        "Answer:",
+    ),
+]
+
+## Define required variables explicitly
+prompt_builder = ChatPromptBuilder(
+    template=prompt_template,
+    required_variables={"question", "documents"},
+)
+
+retriever = InMemoryBM25Retriever(document_store=document_store)
+llm = MetaLlamaChatGenerator(
+    api_key=Secret.from_env_var("LLAMA_API_KEY"),
+    streaming_callback=print_streaming_chunk,
+)
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm.messages")
+
+## Ask a question
+question = "Who lives in Paris?"
+rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/mistralchatgenerator.mdx

@@ -0,0 +1,124 @@
+---
+title: "MistralChatGenerator"
+id: mistralchatgenerator
+slug: "/mistralchatgenerator"
+description: "This component enables chat completion using Mistral’s text generation models."
+---
+
+# MistralChatGenerator
+
+This component enables chat completion using Mistral’s text generation models.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": The Mistral API key. Can be set with `MISTRAL_API_KEY` env var. |
+| **Mandatory run variables** | “messages” A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
+| **Output variables** | "replies": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Mistral](/reference/integrations-mistral) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral |
+
+## Overview
+
+This integration supports Mistral’s models provided through the generative endpoint. For a full list of available models, check out the [Mistral documentation](https://docs.mistral.ai/platform/endpoints/#generative-endpoints).
+
+`MistralChatGenerator` needs a Mistral API key to work. You can write this key in:
+
+- The `api_key` init parameter using [Secret API](../../concepts/secret-management.mdx)
+- The `MISTRAL_API_KEY` environment variable (recommended)
+
+Currently, available models are:
+
+- `mistral-tiny` (default)
+- `mistral-small`
+- `mistral-medium`(soon to be deprecated)
+- `mistral-large-latest`
+- `codestral-latest`
+
+This component needs a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata.
+
+Refer to the [Mistral API documentation](https://docs.mistral.ai/api/#operation/createChatCompletion) for more details on the parameters supported by the Mistral API, which you can provide with `generation_kwargs` when running the component.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+Install the `mistral-haystack` package to use the  `MistralChatGenerator`:
+
+```shell
+pip install mistral-haystack
+```
+
+#### On its own
+
+```python
+from haystack_integrations.components.generators.mistral import MistralChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+
+generator = MistralChatGenerator(
+    api_key=Secret.from_env_var("MISTRAL_API_KEY"),
+    streaming_callback=print_streaming_chunk,
+)
+message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
+print(generator.run([message]))
+```
+
+#### In a Pipeline
+
+Below is an example RAG Pipeline where we answer questions based on the URL contents. We add the contents of the URL into our `messages` in the `ChatPromptBuilder` and generate an answer with the `MistralChatGenerator`.
+
+```python
+from haystack import Document
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.components.fetchers import LinkContentFetcher
+from haystack.components.converters import HTMLToDocument
+from haystack.dataclasses import ChatMessage
+
+from haystack_integrations.components.generators.mistral import MistralChatGenerator
+
+fetcher = LinkContentFetcher()
+converter = HTMLToDocument()
+prompt_builder = ChatPromptBuilder(variables=["documents"])
+llm = MistralChatGenerator(
+    streaming_callback=print_streaming_chunk,
+    model="mistral-small",
+)
+
+message_template = """Answer the following question based on the contents of the article: {{query}}\n
+               Article: {{documents[0].content}} \n
+           """
+messages = [ChatMessage.from_user(message_template)]
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component(name="fetcher", instance=fetcher)
+rag_pipeline.add_component(name="converter", instance=converter)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+
+rag_pipeline.connect("fetcher.streams", "converter.sources")
+rag_pipeline.connect("converter.documents", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
+
+question = "What are the capabilities of Mixtral?"
+
+result = rag_pipeline.run(
+    {
+        "fetcher": {"urls": ["https://mistral.ai/news/mixtral-of-experts"]},
+        "prompt_builder": {
+            "template_variables": {"query": question},
+            "template": messages,
+        },
+        "llm": {"generation_kwargs": {"max_tokens": 165}},
+    },
+)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Web QA with Mixtral-8x7B-Instruct-v0.1](https://haystack.deepset.ai/cookbook/mixtral-8x7b-for-web-qa)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/nvidiachatgenerator.mdx

@@ -0,0 +1,98 @@
+---
+title: "NvidiaChatGenerator"
+id: nvidiachatgenerator
+slug: "/nvidiachatgenerator"
+description: "This Generator enables chat completion using Nvidia-hosted models."
+---
+
+# NvidiaChatGenerator
+
+This Generator enables chat completion using Nvidia-hosted models.
+
+|                                        |                                                                                          |
+| -------------------------------------- | ---------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                     |
+| **Mandatory init variables**           | "api_key": API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var.         |
+| **Mandatory run variables**            | "messages": A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                           |
+| **Output variables**                   | "replies": A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                            |
+| **API reference**                      | [NVIDIA API](https://build.nvidia.com/models)                                            |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia |
+
+## Overview
+
+`NvidiaChatGenerator` enables chat completions using NVIDIA's generative models via the NVIDIA API. It is compatible with the [ChatMessage](../../concepts/data-classes/chatmessage.mdx) format for both input and output, ensuring seamless integration in chat-based pipelines.
+
+You can use LLMs self-hosted with NVIDIA NIM or models hosted on the [NVIDIA API catalog](https://build.nvidia.com/explore/discover). The default model for this component is `meta/llama-3.1-8b-instruct`.
+
+To use this integration, you must have a NVIDIA API key. You can provide it with the `NVIDIA_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
+
+This generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
+
+## Usage
+
+To start using `NvidiaChatGenerator`, first, install the `nvidia-haystack` package:
+
+```shell
+pip install nvidia-haystack
+```
+
+You can use the `NvidiaChatGenerator` with all the LLMs available in the [NVIDIA API catalog](https://docs.api.nvidia.com/nim/reference) or a model deployed with NVIDIA NIM. Follow the [NVIDIA NIM for LLMs Playbook](https://developer.nvidia.com/docs/nemo-microservices/inference/playbooks/nmi_playbook.html) to learn how to deploy your desired model on your infrastructure.
+
+### On its own
+
+ To use LLMs from the NVIDIA API catalog, you need to specify the correct `api_url` if needed (the default one is `https://integrate.api.nvidia.com/v1`), and your API key. You can get your API key directly from the [catalog website](https://build.nvidia.com/explore/discover).
+
+```python
+from haystack_integrations.components.generators.nvidia import NvidiaChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = NvidiaChatGenerator(
+    model="meta/llama-3.1-8b-instruct",  # or any supported NVIDIA model
+    api_key=Secret.from_env_var("NVIDIA_API_KEY"),
+)
+
+messages = [ChatMessage.from_user("What's Natural Language Processing? Be brief.")]
+result = generator.run(messages)
+print(result["replies"])
+print(result["meta"])
+```
+
+### In a Pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.nvidia import NvidiaChatGenerator
+from haystack.utils import Secret
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component(
+    "llm",
+    NvidiaChatGenerator(
+        model="meta/llama-3.1-8b-instruct",
+        api_key=Secret.from_env_var("NVIDIA_API_KEY"),
+    ),
+)
+pipe.connect("prompt_builder", "llm")
+
+country = "Germany"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving out valuable information to language learners.",
+)
+messages = [
+    system_message,
+    ChatMessage.from_user("What's the official language of {{ country }}?"),
+]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"country": country},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/nvidiagenerator.mdx

@@ -0,0 +1,141 @@
+---
+title: "NvidiaGenerator"
+id: nvidiagenerator
+slug: "/nvidiagenerator"
+description: "This Generator enables text generation using Nvidia-hosted models."
+---
+
+# NvidiaGenerator
+
+This Generator enables text generation using Nvidia-hosted models.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count and others |
+| **API reference** | [Nvidia](/reference/integrations-nvidia) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia |
+
+## Overview
+
+The `NvidiaGenerator` provides an interface for generating text using LLMs self-hosted with NVIDIA NIM or models hosted on the [NVIDIA API catalog](https://build.nvidia.com/explore/discover).
+
+## Usage
+
+To start using `NvidiaGenerator`, first, install the `nvidia-haystack` package:
+
+```shell
+pip install nvidia-haystack
+```
+
+You can use the `NvidiaGenerator` with all the LLMs available in the [NVIDIA API catalog](https://docs.api.nvidia.com/nim/reference) or a model deployed with NVIDIA NIM. Follow the [NVIDIA NIM for LLMs Playbook](https://developer.nvidia.com/docs/nemo-microservices/inference/playbooks/nmi_playbook.html) to learn how to deploy your desired model on your infrastructure.
+
+### On its own
+
+To use LLMs from the NVIDIA API catalog, you need to specify the correct `api_url` and your API key. You can get your API key directly from the [catalog website](https://build.nvidia.com/explore/discover).
+
+The `NvidiaGenerator` needs an Nvidia API key to work. It uses the `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`, as in the following example.
+
+```python
+from haystack.utils.auth import Secret
+from haystack_integrations.components.generators.nvidia import NvidiaGenerator
+
+generator = NvidiaGenerator(
+    model="meta/llama3-70b-instruct",
+    api_url="https://integrate.api.nvidia.com/v1",
+    api_key=Secret.from_token("<your-api-key>"),
+    model_arguments={
+        "temperature": 0.2,
+        "top_p": 0.7,
+        "max_tokens": 1024,
+    },
+)
+generator.warm_up()
+
+result = generator.run(prompt="What is the answer?")
+print(result["replies"])
+print(result["meta"])
+```
+
+To use a locally deployed model, you need to set the `api_url` to your localhost and unset your `api_key`.
+
+```python
+from haystack_integrations.components.generators.nvidia import NvidiaGenerator
+
+generator = NvidiaGenerator(
+    model="llama-2-7b",
+    api_url="http://0.0.0.0:9999/v1",
+    api_key=None,
+    model_arguments={
+        "temperature": 0.2,
+    },
+)
+generator.warm_up()
+
+result = generator.run(prompt="What is the answer?")
+print(result["replies"])
+print(result["meta"])
+```
+
+### In a Pipeline
+
+Here's an example of a RAG pipeline:
+
+```python
+from haystack import Pipeline, Document
+from haystack.utils.auth import Secret
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.generators.nvidia import NvidiaGenerator
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="Rome is the capital of Italy"),
+        Document(content="Paris is the capital of France"),
+    ],
+)
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component(
+    "llm",
+    NvidiaGenerator(
+        model="meta/llama3-70b-instruct",
+        api_url="https://integrate.api.nvidia.com/v1",
+        api_key=Secret.from_token("<your-api-key>"),
+        model_arguments={
+            "temperature": 0.2,
+            "top_p": 0.7,
+            "max_tokens": 1024,
+        },
+    ),
+)
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(res)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Haystack RAG Pipeline with Self-Deployed AI models using NVIDIA NIMs](https://haystack.deepset.ai/cookbook/rag-with-nims)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/ollamachatgenerator.mdx

@@ -0,0 +1,182 @@
+---
+title: "OllamaChatGenerator"
+id: ollamachatgenerator
+slug: "/ollamachatgenerator"
+description: "This component enables chat completion using an LLM running on Ollama."
+---
+
+# OllamaChatGenerator
+
+This component enables chat completion using an LLM running on Ollama.
+
+|                                        |                                                                                                      |
+| :------------------------------------- | :--------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                 |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables**                   | “replies”: A list of LLM’s alternative replies                                                       |
+| **API reference**                      | [Ollama](/reference/integrations-ollama)                                                                    |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama             |
+
+## Overview
+
+[Ollama](https://github.com/jmorganca/ollama) is a project focused on running LLMs locally. Internally, it uses the quantized GGUF format by default. This means it is possible to run LLMs on standard machines (even without GPUs) without having to handle complex installation procedures.
+
+`OllamaChatGenerator` supports models running on Ollama, such as `llama2` and `mixtral`. Find the full list of supported models [here](https://ollama.ai/library).
+
+`OllamaChatGenerator`  needs a `model`  name and a `url` to work. By default, it uses `"orca-mini"` model and `"http://localhost:11434"` url.
+
+The way to operate with `OllamaChatGenerator` is by using  `ChatMessage` objects. [ChatMessage](../../concepts/data-classes/chatmessage.mdx)  is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. See the [usage](#usage) section for an example.
+
+### Streaming
+
+You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+## Configure any `Generator` or `ChatGenerator` with a streaming callback
+component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
+
+## If this is a `ChatGenerator`, pass a list of messages:
+## from haystack.dataclasses import ChatMessage
+## component.run([ChatMessage.from_user("Your question here")])
+
+## If this is a (non-chat) `Generator`, pass a prompt:
+## component.run({"prompt": "Your prompt here"})
+```
+
+:::note
+Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
+
+:::
+
+See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
+
+Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
+
+## Usage
+
+1. You need a running instance of Ollama. The installation instructions are [in the Ollama GitHub repository](https://github.com/jmorganca/ollama).
+   A fast way to run Ollama is using Docker:
+
+```bash
+docker run -d -p 11434:11434 --name ollama ollama/ollama:latest
+```
+
+2. You need to download or pull the desired LLM. The model library is available on the [Ollama website](https://ollama.ai/library).
+   If you are using Docker, you can, for example, pull the Zephyr model:
+
+```bash
+docker exec ollama ollama pull zephyr
+```
+
+If you already installed Ollama in your system, you can execute:
+
+```bash
+ollama pull zephyr
+```
+
+:::tip
+Choose a specific version of a model
+
+You can also specify a tag to choose a specific (quantized) version of your model. The available tags are shown in the model card of the Ollama models library. This is an [example](https://ollama.ai/library/zephyr/tags) for Zephyr.
+In this case, simply run
+
+```shell
+## ollama pull model:tag
+ollama pull zephyr:7b-alpha-q3_K_S
+```
+:::
+
+3. You also need to install the `ollama-haystack` package:
+
+```bash
+pip install ollama-haystack
+```
+
+### On its own
+
+```python
+from haystack_integrations.components.generators.ollama import OllamaChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = OllamaChatGenerator(model="zephyr",
+                            url = "http://localhost:11434",
+                            generation_kwargs={
+                              "num_predict": 100,
+                              "temperature": 0.9,
+                              })
+
+messages = [ChatMessage.from_system("\nYou are a helpful, respectful and honest assistant"),
+ChatMessage.from_user("What's Natural Language Processing?")]
+
+print(generator.run(messages=messages))
+    "replies": [
+        ChatMessage(
+            _role=<ChatRole.ASSISTANT: 'assistant'>,
+            _content=[
+                TextContent(
+                    text=(
+                        "Natural Language Processing (NLP) is a subfield of "
+                        "Artificial Intelligence that deals with understanding, "
+                        "interpreting, and generating human language in a meaningful "
+                        "way. It enables tasks such as language translation, sentiment "
+                        "analysis, and text summarization."
+                    )
+                )
+            ],
+            _name=None,
+            _meta={
+                "model": "zephyr",...
+            }
+        )
+    ]
+}
+```
+
+### In a Pipeline
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack_integrations.components.generators.ollama import OllamaChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+generator = OllamaChatGenerator(model="zephyr",
+                            url = "http://localhost:11434",
+                            generation_kwargs={
+                              "temperature": 0.9,
+                              })
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", generator)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+location = "Berlin"
+messages = [ChatMessage.from_system("Always respond in Spanish even if some input data is in other languages."),
+            ChatMessage.from_user("Tell me about {{location}}")]
+print(pipe.run(data={"prompt_builder": {"template_variables":{"location": location}, "template": messages}}))
+
+    "llm": {
+        "replies": [
+            ChatMessage(
+                _role=<ChatRole.ASSISTANT: 'assistant'>,
+                _content=[
+                    TextContent(
+                        text=(
+                            "Berlín es la capital y la mayor ciudad de Alemania. "
+                            "Está ubicada en el estado federado de Berlín, y tiene más..."
+                        )
+                    )
+                ],
+                _name=None,
+                _meta={
+                    "model": "zephyr",...
+                }
+            )
+        ]
+    }
+}
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/ollamagenerator.mdx

@@ -0,0 +1,151 @@
+---
+title: "OllamaGenerator"
+id: ollamagenerator
+slug: "/ollamagenerator"
+description: "A component that provides an interface to generate text using an LLM running on Ollama."
+---
+
+# OllamaGenerator
+
+A component that provides an interface to generate text using an LLM running on Ollama.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count and others |
+| **API reference** | [Ollama](/reference/integrations-ollama) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama |
+
+## Overview
+
+`OllamaGenerator` provides an interface to generate text using an LLM running on Ollama.
+
+`OllamaGenerator`  needs a `model`  name and a `url` to work. By default, it uses `"orca-mini"` model and `"http://localhost:11434"` url.
+
+[Ollama](https://github.com/jmorganca/ollama) is a project focused on running LLMs locally. Internally, it uses the quantized GGUF format by default. This means it is possible to run LLMs on standard machines (even without GPUs) without having to go through complex installation procedures.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+1. You need a running instance of Ollama. You can find the installation instructions [here](https://github.com/jmorganca/ollama).
+   A fast way to run Ollama is using Docker:
+
+```shell
+docker run -d -p 11434:11434 --name ollama ollama/ollama:latest
+```
+
+2. You need to download or pull the desired LLM. The model library is available on the [Ollama website](https://ollama.ai/library).
+   If you are using Docker, you can, for example, pull the Zephyr model:
+
+```shell
+docker exec ollama ollama pull zephyr
+```
+
+If you have already installed Ollama in your system, you can execute:
+
+```shell
+ollama pull zephyr
+```
+
+:::tip
+Choose a specific version of a model
+
+You can also specify a tag to choose a specific (quantized) version of your model. The available tags are shown in the model card of the Ollama models library. This is an [example](https://ollama.ai/library/zephyr/tags) for Zephyr.
+In this case, simply run
+
+```shell
+## ollama pull model:tag
+ollama pull zephyr:7b-alpha-q3_K_S
+```
+:::
+
+3. You also need to install the `ollama-haystack` package:
+
+```shell
+pip install ollama-haystack
+```
+
+### On its own
+
+Here's how the `OllamaGenerator` would work just on its own:
+
+```python
+from haystack_integrations.components.generators.ollama import OllamaGenerator
+
+generator = OllamaGenerator(
+    model="zephyr",
+    url="http://localhost:11434",
+    generation_kwargs={
+        "num_predict": 100,
+        "temperature": 0.9,
+    },
+)
+
+print(generator.run("Who is the best American actor?"))
+
+## {'replies': ['I do not have the ability to form opinions or preferences.
+## However, some of the most acclaimed american actors in recent years include
+## denzel washington, tom hanks, leonardo dicaprio, matthew mcconaughey...'],
+#'meta': [{'model': 'zephyr', ...}]}
+```
+
+### In a Pipeline
+
+```python
+from haystack_integrations.components.generators.ollama import OllamaGenerator
+
+from haystack import Pipeline, Document
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="I really like summer"),
+        Document(content="My favorite sport is soccer"),
+        Document(content="I don't like reading sci-fi books"),
+        Document(content="I don't like crowded places"),
+    ],
+)
+
+generator = OllamaGenerator(
+    model="zephyr",
+    url="http://localhost:11434",
+    generation_kwargs={
+        "num_predict": 100,
+        "temperature": 0.9,
+    },
+)
+
+pipe = Pipeline()
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("llm", generator)
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+result = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(result)
+
+## {'llm': {'replies': ['Based on the provided context, it seems that you enjoy
+## soccer and summer. Unfortunately, there is no direct information given about
+## what else you enjoy...'],
+## 'meta': [{'model': 'zephyr', ...]}}
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/openaichatgenerator.mdx

@@ -0,0 +1,200 @@
+---
+title: "OpenAIChatGenerator"
+id: openaichatgenerator
+slug: "/openaichatgenerator"
+description: "`OpenAIChatGenerator` enables chat completion using OpenAI’s large language models (LLMs)."
+---
+
+# OpenAIChatGenerator
+
+`OpenAIChatGenerator` enables chat completion using OpenAI’s large language models (LLMs).
+
+|                                        |                                                                                                      |
+| :------------------------------------- | :--------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                 |
+| **Mandatory init variables**           | "api_key": An OpenAI API key. Can be set with `OPENAI_API_KEY` env var.                              |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables**                   | “replies”: A list of alternative replies of the LLM to the input chat                                |
+| **API reference**                      | [Generators](/reference/generators-api)                                                                     |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/openai.py     |
+
+## Overview
+
+`OpenAIChatGenerator` supports OpenAI models starting from gpt-3.5-turbo and later (gpt-4, gpt-4-turbo, and so on).
+
+`OpenAIChatGenerator` needs an OpenAI key to work. It uses an ` OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
+
+```python
+generator = OpenAIChatGenerator(model="gpt-4o-mini")
+```
+
+Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. See the [usage](#usage) section for an example.
+
+You can pass any chat completion parameters valid for the `openai.ChatCompletion.create` method directly to `OpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).
+
+`OpenAIChatGenerator` can support custom deployments of your OpenAI models through the `api_base_url` init parameter.
+
+### Structured Output
+
+`OpenAIChatGenerator` supports structured output generation, allowing you to receive responses in a predictable format. You can use Pydantic models or JSON schemas to define the structure of the output through the `response_format` parameter in `generation_kwargs`.
+
+This is useful when you need to extract structured data from text or generate responses that match a specific format.
+
+```python
+from pydantic import BaseModel
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+
+class NobelPrizeInfo(BaseModel):
+    recipient_name: str
+    award_year: int
+    category: str
+    achievement_description: str
+    nationality: str
+
+
+client = OpenAIChatGenerator(
+    model="gpt-4o-2024-08-06",
+    generation_kwargs={"response_format": NobelPrizeInfo},
+)
+
+response = client.run(
+    messages=[
+        ChatMessage.from_user(
+            "In 2021, American scientist David Julius received the Nobel Prize in"
+            " Physiology or Medicine for his groundbreaking discoveries on how the human body"
+            " senses temperature and touch.",
+        ),
+    ],
+)
+print(response["replies"][0].text)
+```
+
+:::note
+Model Compatibility and Limitations
+
+- Pydantic models and JSON schemas are supported for latest models starting from `gpt-4o-2024-08-06`.
+- Older models only support basic JSON mode through `{"type": "json_object"}`. For details, see [OpenAI JSON mode documentation](https://platform.openai.com/docs/guides/structured-outputs#json-mode).
+- Streaming limitation: When using streaming with structured outputs, you must provide a JSON schema instead of a Pydantic model for `response_format`.
+- For complete information, check the [OpenAI Structured Outputs documentation](https://platform.openai.com/docs/guides/structured-outputs).
+:::
+
+### Streaming
+
+You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+## Configure any `Generator` or `ChatGenerator` with a streaming callback
+component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
+
+## If this is a `ChatGenerator`, pass a list of messages:
+## from haystack.dataclasses import ChatMessage
+## component.run([ChatMessage.from_user("Your question here")])
+
+## If this is a (non-chat) `Generator`, pass a prompt:
+## component.run({"prompt": "Your prompt here"})
+```
+
+:::note
+Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
+
+:::
+
+See our [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
+
+Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
+
+## Usage
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack.components.generators.chat import OpenAIChatGenerator
+
+client = OpenAIChatGenerator()
+response = client.run(
+    [ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
+)
+print(response)
+```
+
+With streaming:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack.components.generators.chat import OpenAIChatGenerator
+
+client = OpenAIChatGenerator(
+    streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
+)
+response = client.run(
+    [ChatMessage.from_user("What's Natural Language Processing? Be brief.")],
+)
+print(response)
+```
+
+With multimodal inputs:
+
+```python
+from haystack.dataclasses import ChatMessage, ImageContent
+from haystack.components.generators.chat import OpenAIChatGenerator
+
+llm = OpenAIChatGenerator(model="gpt-4o-mini")
+
+image = ImageContent.from_file_path("apple.jpg", detail="low")
+user_message = ChatMessage.from_user(
+    content_parts=["What does the image show? Max 5 words.", image],
+)
+
+response = llm.run([user_message])["replies"][0].text
+print(response)
+```
+
+### In a Pipeline
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack.utils import Secret
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+llm = OpenAIChatGenerator(
+    api_key=Secret.from_env_var("OPENAI_API_KEY"),
+    model="gpt-4o-mini",
+)
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+location = "Berlin"
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+```
+
+## Additional References
+
+:notebook: Tutorial: [Building a Chat Application with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling)
+
+🧑‍🍳 Cookbook: [Function Calling with OpenAIChatGenerator](https://haystack.deepset.ai/cookbook/function_calling_with_openaichatgenerator)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/openaigenerator.mdx

@@ -0,0 +1,133 @@
+---
+title: "OpenAIGenerator"
+id: openaigenerator
+slug: "/openaigenerator"
+description: "`OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs)."
+---
+
+# OpenAIGenerator
+
+`OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs).
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Generators](/reference/generators-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai.py |
+
+## Overview
+
+`OpenAIGenerator` supports OpenAI models starting from gpt-3.5-turbo and later (gpt-4, gpt-4-turbo, and so on).
+
+`OpenAIGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
+
+```
+generator = OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"), model="gpt-4o-mini")
+```
+
+Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).
+
+`OpenAIGenerator` supports custom deployments of your OpenAI models through the `api_base_url` init parameter.
+
+### Streaming
+
+`OpenAIGenerator` supports streaming the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter. Note that streaming the tokens is only compatible with generating a single response, so `n` must be set to 1 for streaming to work.
+
+:::note
+This component is designed for text generation, not for chat. If you want to use OpenAI LLMs for chat, use [`OpenAIChatGenerator`](openaichatgenerator.mdx) instead.
+
+:::
+
+## Usage
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack.components.generators import OpenAIGenerator
+from haystack.utils import Secret
+
+client = OpenAIGenerator(model="gpt-4", api_key=Secret.from_token("<your-api-key>"))
+response = client.run("What's Natural Language Processing? Be brief.")
+print(response)
+
+    of artificial intelligence that focuses on the interaction between computers
+    and humans through natural language. The primary aim of NLP is to enable
+    computers to understand, interpret, and generate human language in a valuable way.'],
+    'meta': [{'model': 'gpt-4-0613', 'index': 0, 'finish_reason':
+    'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 53,
+    'total_tokens': 69}}]}
+```
+
+With streaming:
+
+```python
+from haystack.components.generators import OpenAIGenerator
+from haystack.utils import Secret
+
+client = OpenAIGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
+response = client.run("What's Natural Language Processing? Be brief.")
+print(response)
+
+	intelligence that focuses on the interaction between computers and human
+  language. It involves enabling computers to understand, interpret,and respond
+  to natural human language in a way that is both meaningful and useful.
+	intelligence that focuses on the interaction between computers and human
+  language. It involves enabling computers to understand, interpret,and respond
+  to natural human language in a way that is both meaningful and useful.'],
+  'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
+  'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 49,
+  'total_tokens': 65}}]}
+```
+
+### In a Pipeline
+
+Here's an example of RAG Pipeline:
+
+```python
+from haystack import Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders.prompt_builder import PromptBuilder
+from haystack.components.generators import OpenAIGenerator
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack import Document
+from haystack.utils import Secret
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents([Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")])
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("llm", OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"))
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+res=pipe.run({
+    "prompt_builder": {
+        "query": query
+    },
+    "retriever": {
+        "query": query
+    }
+})
+
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/openrouterchatgenerator.mdx

@@ -0,0 +1,109 @@
+---
+title: "OpenRouterChatGenerator"
+id: openrouterchatgenerator
+slug: "/openrouterchatgenerator"
+description: "This component enables chat completion with any model hosted on [OpenRouter](https://openrouter.ai/)."
+---
+
+# OpenRouterChatGenerator
+
+This component enables chat completion with any model hosted on [OpenRouter](https://openrouter.ai/).
+
+|                                        |                                                                                                                   |
+| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                                |
+| **Mandatory init variables**           | “api_key”: An OpenRouter API key. Can be set with `OPENROUTER_API_KEY` env variable or passed to `init()` method. |
+| **Mandatory run variables**            | “messages:” A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                      |
+| **Output variables**                   | “replies”: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                       |
+| **API reference**                      | [OpenRouter](/reference/integrations-openrouter)                                                                         |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/openrouter                      |
+
+## Overview
+
+The `OpenRouterChatGenerator` enables you to use models from multiple providers (such as `openai/gpt-4o`, `anthropic/claude-3.5-sonnet`, and others) by making chat completion calls to the [OpenRouter API](https://openrouter.ai/docs/quickstart).
+
+This generator also supports OpenRouter-specific features such as:
+
+- Provider routing and model fallback that are configurable with the `generation_kwargs` parameter during initialization or runtime.
+- Custom HTTP headers that can be supplied using the `extra_headers` parameter.
+
+This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
+
+It is also fully compatible with Haystack [Tools](../../tools/tool.mdx) and [Toolsets](../../tools/toolset.mdx) that allow function-calling capabilities with supported models.
+
+### Initialization
+
+To use this integration, you must have an active OpenRouter subscription with sufficient credits and an API key. You can provide it with the `OPENROUTER_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
+
+Then, install the `openrouter-haystack` integration:
+
+```shell
+pip install openrouter-haystack
+```
+
+### Streaming
+
+`OpenRouterChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
+
+## Usage
+
+### On its own
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.openrouter import (
+    OpenRouterChatGenerator,
+)
+
+client = OpenRouterChatGenerator()
+response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
+print(response["replies"][0].text)
+```
+
+With streaming and model routing:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.openrouter import (
+    OpenRouterChatGenerator,
+)
+
+client = OpenRouterChatGenerator(
+    model="openrouter/auto",
+    streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
+)
+
+response = client.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
+
+## check the model used for the response
+print("\n\n Model used: ", response["replies"][0].meta["model"])
+```
+
+### In a pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.openrouter import (
+    OpenRouterChatGenerator,
+)
+
+prompt_builder = ChatPromptBuilder()
+llm = OpenRouterChatGenerator(model="openai/gpt-4o-mini")
+
+pipe = Pipeline()
+pipe.add_component("builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("builder.prompt", "llm.messages")
+
+messages = [
+    ChatMessage.from_system("Give brief answers."),
+    ChatMessage.from_user("Tell me about {{city}}"),
+]
+
+response = pipe.run(
+    data={"builder": {"template": messages, "template_variables": {"city": "Berlin"}}},
+)
+print(response)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/sagemakergenerator.mdx

@@ -0,0 +1,103 @@
+---
+title: "SagemakerGenerator"
+id: sagemakergenerator
+slug: "/sagemakergenerator"
+description: "This component enables text generation using LLMs deployed on Amazon Sagemaker."
+---
+
+# SagemakerGenerator
+
+This component enables text generation using LLMs deployed on Amazon Sagemaker.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "model": The model to use  <br /> <br />"aws_access_key_id": AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var.  <br /> <br />"aws_secret_access_key": AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var. |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
+| **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Amazon Sagemaker](/reference/integrations-amazon-sagemaker) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_sagemaker |
+
+`SagemakerGenerator` allows you to make use of models deployed on [AWS SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html).
+
+## Parameters Overview
+
+`SagemakerGenerator` needs AWS credentials to work.  Set the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.
+
+You also need to specify your Sagemaker endpoint at initialization time for the component to work. Pass the endpoint name to the `model` parameter like this:
+
+```python
+generator = SagemakerGenerator(model="jumpstart-dft-hf-llm-falcon-7b-instruct-bf16")
+```
+
+Additionally, you can pass any text generation parameters valid for your specific model directly to `SagemakerGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method.
+
+If your model also needs custom attributes, pass those as a dictionary at initialization time by setting the `aws_custom_attributes` parameter.
+
+One notable family of models that needs these custom parameters is Llama2, which needs to be initialized with `{"accept_eula": True}` :
+
+```python
+generator = SagemakerGenerator(
+    model="jumpstart-dft-meta-textgenerationneuron-llama-2-7b",
+    aws_custom_attributes={"accept_eula": True},
+)
+```
+
+## Usage
+
+You need to install `amazon-sagemaker-haystack` package to use the  `SagemakerGenerator`:
+
+```shell
+pip install amazon-sagemaker-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack_integrations.components.generators.amazon_sagemaker import SagemakerGenerator
+
+client = SagemakerGenerator(model="jumpstart-dft-hf-llm-falcon-7b-instruct-bf16")
+client.warm_up()
+response = client.run("Briefly explain what NLP is in one sentence.")
+print(response)
+
+ 'metadata': [{}]}
+```
+
+### In a pipeline
+
+In a RAG pipeline:
+
+```python
+from haystack_integrations.components.generators.amazon_sagemaker import (
+    SagemakerGenerator,
+)
+from haystack import Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders import PromptBuilder
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: What's the official language of {{ country }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component(
+    "llm",
+    SagemakerGenerator(model="jumpstart-dft-hf-llm-falcon-7b-instruct-bf16"),
+)
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "llm")
+
+pipe.run({"prompt_builder": {"country": "France"}})
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/stackitchatgenerator.mdx

@@ -0,0 +1,96 @@
+---
+title: "STACKITChatGenerator"
+id: stackitchatgenerator
+slug: "/stackitchatgenerator"
+description: "This component enables chat completions using the STACKIT API."
+---
+
+# STACKITChatGenerator
+
+This component enables chat completions using the STACKIT API.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
+| **Mandatory init variables** | "model": The model used through the STACKIT API |
+| **Mandatory run variables** | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
+| **Output variables** | "replies": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply (such as token count, finish reason, and so on) |
+| **API reference** | [STACKIT](/reference/integrations-stackit) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |
+
+## Overview
+
+`STACKITChatGenerator` enables text generation models served by STACKIT through their API.
+
+### Parameters
+
+To use the `STACKITChatGenerator`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as another environment variable or a token by setting
+`api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).
+
+Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/stackit/en/models-licenses-319914532.html).
+
+Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.
+
+You can pass any text generation parameters valid for the STACKIT Chat Completion API directly to this component with the `generation_kwargs` parameter in the init or run methods.
+
+The component needs a list of `ChatMessage` objects to run. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. Find out more about it [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
+
+### Streaming
+
+This ChatGenerator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly into the output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+Install the `stackit-haystack` package to use the `STACKITChatGenerator`:
+
+```shell
+pip install stackit-haystack
+```
+
+### On its own
+
+```python
+from haystack_integrations.components.generators.stackit import STACKITChatGenerator
+from haystack.dataclasses import ChatMessage
+
+generator = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")
+
+result = generator.run([ChatMessage.from_user("Tell me a joke.")])
+print(result)
+```
+
+### In a pipeline
+
+You can also use `STACKITChatGenerator` in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+from haystack_integrations.components.generators.stackit import STACKITChatGenerator
+
+prompt_builder = ChatPromptBuilder()
+llm = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")
+
+messages = [ChatMessage.from_user("Question: {{question}} \\n")]
+
+pipeline = Pipeline()
+pipeline.add_component("prompt_builder", prompt_builder)
+pipeline.add_component("llm", llm)
+
+pipeline.connect("prompt_builder.prompt", "llm.messages")
+
+result = pipeline.run(
+    {
+        "prompt_builder": {
+            "template_variables": {"question": "Tell me a joke."},
+            "template": messages,
+        },
+    },
+)
+
+print(result)
+```
+
+For an example of streaming in a pipeline, refer to the examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and on its dedicated [integration page](https://haystack.deepset.ai/integrations/stackit).

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaicodegenerator.mdx

@@ -0,0 +1,70 @@
+---
+title: "VertexAICodeGenerator"
+id: vertexaicodegenerator
+slug: "/vertexaicodegenerator"
+description: "This component enables code generation using Google Vertex AI generative model."
+---
+
+# VertexAICodeGenerator
+
+This component enables code generation using Google Vertex AI generative model.
+
+|  |  |
+| --- | --- |
+| **Mandatory run variables** | “prefix”: A string of code before the current point  <br /> <br />”suffix”: An optional string of code after the current point |
+| **Output variables** | “replies”: Code generated by the model |
+| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
+
+`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`.
+
+### Parameters Overview
+
+`VertexAICodeGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+## Usage
+
+You need to install `google-vertex-haystack` package first to use the  `VertexAIImageCaptioner`:
+
+```shell
+pip install google-vertex-haystack
+```
+
+Basic usage:
+
+````python
+from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator
+
+generator = VertexAICodeGenerator()
+
+result = generator.run(prefix="def to_json(data):")
+
+for answer in result["replies"]:
+    print(answer)
+
+````
+
+You can also set other parameters like the number of output tokens, temperature, stop sequences, and the number of candidates.
+
+Let’s try a different model:
+
+```python
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAICodeGenerator,
+)
+
+generator = VertexAICodeGenerator(
+    model="code-gecko",
+    temperature=0.8,
+    candidate_count=3,
+)
+
+result = generator.run(prefix="def convert_temperature(degrees):")
+
+for answer in result["replies"]:
+    print(answer)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaigeminichatgenerator.mdx

@@ -0,0 +1,170 @@
+---
+title: "VertexAIGeminiChatGenerator"
+id: vertexaigeminichatgenerator
+slug: "/vertexaigeminichatgenerator"
+description: "`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models."
+---
+
+# VertexAIGeminiChatGenerator
+
+`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models.
+
+:::warning
+Deprecation Notice
+
+This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
+
+We recommend switching to the new [GoogleGenAIChatGenerator](googlegenaichatgenerator.mdx) integration instead.
+:::
+
+|                                        |                                                                                                      |
+| :------------------------------------- | :--------------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                 |
+| **Mandatory run variables**            | “messages”: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects representing the chat |
+| **Output variables**                   | “replies”: A list of alternative replies of the model to the input chat                              |
+| **API reference**                      | [Google Vertex](/reference/integrations-google-vertex)                                                      |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex      |
+
+`VertexAIGeminiGenerator` supports `gemini-1.5-pro` and `gemini-1.5-flash`/  `gemini-2.0-flash` models. Note that [Google recommends upgrading](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions) from `gemini-1.5-pro` to `gemini-2.0-flash`.
+
+For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models.
+
+:::note
+To explore the full capabilities of Gemini check out this [article](https://haystack.deepset.ai/blog/gemini-models-with-google-vertex-for-haystack) and the related [🧑‍🍳 Cookbook](https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/vertexai-gemini-examples.ipynb).
+
+:::
+
+### Parameters Overview
+
+`VertexAIGeminiChatGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+You need to install the `google-vertex-haystack` package to use the  `VertexAIGeminiChatGenerator`:
+
+```shell
+pip install google-vertex-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIGeminiChatGenerator,
+)
+
+gemini_chat = VertexAIGeminiChatGenerator()
+
+messages = [ChatMessage.from_user("Tell me the name of a movie")]
+res = gemini_chat.run(messages)
+
+print(res["replies"][0].text)
+
+messages += [res["replies"][0], ChatMessage.from_user("Who's the main actor?")]
+res = gemini_chat.run(messages)
+
+print(res["replies"][0].text)
+```
+
+When chatting with Gemini Pro, you can also easily use function calls. First, define the function locally and convert into a [Tool](../../tools/tool.mdx):
+
+```python
+from typing import Annotated
+from haystack.tools import create_tool_from_function
+
+
+## example function to get the current weather
+def get_current_weather(
+    location: Annotated[
+        str,
+        "The city for which to get the weather, e.g. 'San Francisco'",
+    ] = "Munich",
+    unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius",
+) -> str:
+    return f"The weather in {location} is sunny. The temperature is 20 {unit}."
+
+
+tool = create_tool_from_function(get_current_weather)
+```
+
+Create a new instance of `VertexAIGeminiChatGenerator` to set the tools and a [ToolInvoker](../tools/toolinvoker.mdx) to invoke the tools.:
+
+```python
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIGeminiChatGenerator,
+)
+from haystack.components.tools import ToolInvoker
+
+gemini_chat = VertexAIGeminiChatGenerator(model="gemini-2.0-flash-exp", tools=[tool])
+
+tool_invoker = ToolInvoker(tools=[tool])
+```
+
+And then ask our question:
+
+```python
+from haystack.dataclasses import ChatMessage
+
+messages = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")]
+res = gemini_chat.run(messages=messages)
+
+print(res["replies"][0].tool_calls)
+
+tool_messages = tool_invoker.run(messages=replies)["tool_messages"]
+messages = user_message + replies + tool_messages
+
+messages += res["replies"][0] + [
+    ChatMessage.from_function(content=weather, name="get_current_weather"),
+]
+
+final_replies = gemini_chat.run(messages=messages)["replies"]
+print(final_replies[0].text)
+```
+
+### In a pipeline
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIGeminiChatGenerator,
+)
+
+## no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+gemini_chat = VertexAIGeminiChatGenerator()
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("gemini", gemini)
+pipe.connect("prompt_builder.prompt", "gemini.messages")
+
+location = "Rome"
+messages = [ChatMessage.from_user("Tell me briefly about {{location}} history")]
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location},
+            "template": messages,
+        },
+    },
+)
+
+print(res)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Function Calling and Multimodal QA with Gemini](https://haystack.deepset.ai/cookbook/vertexai-gemini-examples)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaigeminigenerator.mdx

@@ -0,0 +1,145 @@
+---
+title: "VertexAIGeminiGenerator"
+id: vertexaigeminigenerator
+slug: "/vertexaigeminigenerator"
+description: "`VertexAIGeminiGenerator` enables text generation using Google Gemini models."
+---
+
+# VertexAIGeminiGenerator
+
+`VertexAIGeminiGenerator` enables text generation using Google Gemini models.
+
+:::warning
+Deprecation Notice
+
+This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
+
+We recommend switching to the new [GoogleGenAIChatGenerator](googlegenaichatgenerator.mdx) integration instead.
+:::
+
+|                                        |                                                                                                 |
+| :------------------------------------- | :---------------------------------------------------------------------------------------------- |
+| **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)                                                  |
+| **Mandatory run variables**            | “parts”: A variadic list containing a mix of images, audio, video, and text to prompt Gemini    |
+| **Output variables**                   | “replies”: A list of strings or dictionaries with all the replies generated by the model        |
+| **API reference**                      | [Google Vertex](/reference/integrations-google-vertex)                                                 |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
+
+`VertexAIGeminiGenerator` supports `gemini-1.5-pro` and `gemini-1.5-flash`/  `gemini-2.0-flash` models. Note that [Google recommends upgrading](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions) from `gemini-1.5-pro` to `gemini-2.0-flash`.
+
+For details on available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models.
+
+:::note
+To explore the full capabilities of Gemini check out this [article](https://haystack.deepset.ai/blog/gemini-models-with-google-vertex-for-haystack) and the related [Colab notebook](https://colab.research.google.com/drive/10SdXvH2ATSzqzA3OOmTM8KzD5ZdH_Q6Z?usp=sharing).
+
+:::
+
+### Parameters Overview
+
+`VertexAIGeminiGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+You should install `google-vertex-haystack` package to use the  `VertexAIGeminiGenerator`:
+
+```shell
+pip install google-vertex-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIGeminiGenerator,
+)
+
+gemini = VertexAIGeminiGenerator()
+result = gemini.run(parts=["What is the most interesting thing you know?"])
+for answer in result["replies"]:
+    print(answer)
+```
+
+Advanced usage, multi-modal prompting:
+
+```python
+import requests
+from haystack.dataclasses.byte_stream import ByteStream
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIGeminiGenerator,
+)
+
+URLS = [
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg",
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot2.jpg",
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot3.jpg",
+    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot4.jpg",
+]
+images = [
+    ByteStream(data=requests.get(url).content, mime_type="image/jpeg") for url in URLS
+]
+
+gemini = VertexAIGeminiGenerator()
+result = gemini.run(parts=["What can you tell me about this robots?", *images])
+for answer in result["replies"]:
+    print(answer)
+```
+
+### In a pipeline
+
+In a RAG pipeline:
+
+```python
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.builders import PromptBuilder
+from haystack import Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIGeminiGenerator,
+)
+
+docstore = InMemoryDocumentStore()
+docstore.write_documents(
+    [
+        Document(content="Rome is the capital of Italy"),
+        Document(content="Paris is the capital of France"),
+    ],
+)
+
+query = "What is the capital of France?"
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{ query }}?
+"""
+pipe = Pipeline()
+
+pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
+pipe.add_component("prompt_builder", PromptBuilder(template=template))
+pipe.add_component("gemini", VertexAIGeminiGenerator())
+pipe.connect("retriever", "prompt_builder.documents")
+pipe.connect("prompt_builder", "gemini")
+
+res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+print(res)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Function Calling and Multimodal QA with Gemini](https://haystack.deepset.ai/cookbook/vertexai-gemini-examples)

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaiimagecaptioner.mdx

@@ -0,0 +1,84 @@
+---
+title: "VertexAIImageCaptioner"
+id: vertexaiimagecaptioner
+slug: "/vertexaiimagecaptioner"
+description: "`VertexAIImageCaptioner` enables text generation using Google Vertex AI `imagetext` generative model."
+---
+
+# VertexAIImageCaptioner
+
+`VertexAIImageCaptioner` enables text generation using Google Vertex AI `imagetext` generative model.
+
+|                             |                                                                                                 |
+| :-------------------------- | :---------------------------------------------------------------------------------------------- |
+| **Mandatory run variables** | “image”: A [`ByteStream`](../../concepts/data-classes.mdx#bytestream)  object storing an image               |
+| **Output variables**        | “captions”: A list of strings generated by the model                                            |
+| **API reference**           | [Google Vertex](/reference/integrations-google-vertex)                                                 |
+| **GitHub link**             | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
+
+### Parameters Overview
+
+`VertexAIImageCaptioner` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+## Usage
+
+You need to install `google-vertex-haystack` package to use the  `VertexAIImageCaptioner`:
+
+```shell
+pip install google-vertex-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+import requests
+
+from haystack.dataclasses.byte_stream import ByteStream
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIImageCaptioner,
+)
+
+captioner = VertexAIImageCaptioner()
+
+image = ByteStream(
+    data=requests.get(
+        "https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg",
+    ).content,
+)
+result = captioner.run(image=image)
+
+for caption in result["captions"]:
+    print(caption)
+```
+
+You can also set the caption language and the number of results:
+
+```python
+import requests
+
+from haystack.dataclasses.byte_stream import ByteStream
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIImageCaptioner,
+)
+
+captioner = VertexAIImageCaptioner(
+    number_of_results=3,  # Can't be greater than 3
+    language="it",
+)
+
+image = ByteStream(
+    data=requests.get(
+        "https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg",
+    ).content,
+)
+result = captioner.run(image=image)
+
+for caption in result["captions"]:
+    print(caption)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaiimagegenerator.mdx

@@ -0,0 +1,76 @@
+---
+title: "VertexAIImageGenerator"
+id: vertexaiimagegenerator
+slug: "/vertexaiimagegenerator"
+description: "This component enables image generation using Google Vertex AI generative model."
+---
+
+# VertexAIImageGenerator
+
+This component enables image generation using Google Vertex AI generative model.
+
+|                             |                                                                                                             |
+| :-------------------------- | :---------------------------------------------------------------------------------------------------------- |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the model                                                      |
+| **Output variables**        | “images”: A list of [`ByteStream`](../../concepts/data-classes.mdx#bytestream)  containing images generated by the model |
+| **API reference**           | [Google Vertex](/reference/integrations-google-vertex)                                                             |
+| **GitHub link**             | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex             |
+
+`VertexAIImageGenerator` supports the `imagegeneration` model.
+
+### Parameters Overview
+
+`VertexAIImageGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+## Usage
+
+You need to install `google-vertex-haystack` package to use the  `VertexAIImageGenerator`:
+
+```python
+pip install google-vertex-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+from pathlib import Path
+
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIImageGenerator,
+)
+
+generator = VertexAIImageGenerator()
+result = generator.run(prompt="Generate an image of a cute cat")
+result["images"][0].to_file(Path("my_image.png"))
+```
+
+You can also set other parameters like the number of images generated and the guidance scale to change the strength of the prompt.
+
+Let’s also use a negative prompt to omit something from the image:
+
+```python
+from pathlib import Path
+
+from haystack_integrations.components.generators.google_vertex import (
+    VertexAIImageGenerator,
+)
+
+generator = VertexAIImageGenerator(
+    number_of_images=3,
+    guidance_scale=12,
+)
+
+result = generator.run(
+    prompt="Generate an image of a cute cat",
+    negative_prompt="window, chair",
+)
+
+for i, image in enumerate(result["images"]):
+    images.to_file(Path(f"image_{i}.png"))
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaiimageqa.mdx

@@ -0,0 +1,69 @@
+---
+title: "VertexAIImageQA"
+id: vertexaiimageqa
+slug: "/vertexaiimageqa"
+description: "This component enables text generation (image captioning) using Google Vertex AI generative models."
+---
+
+# VertexAIImageQA
+
+This component enables text generation (image captioning) using Google Vertex AI generative models.
+
+|  |  |
+| --- | --- |
+| **Mandatory run variables** | “image”: A [`ByteStream`](../../concepts/data-classes.mdx#bytestream)  containing an image data  <br /> <br />”question”: A string of a question about the image |
+| **Output variables** | “replies”: A list of strings containing answers generated by the model |
+| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
+
+`VertexAIImageQA` supports the `imagetext` model.
+
+### Parameters Overview
+
+`VertexAIImageQA` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+## Usage
+
+You need to install `google-vertex-haystack` package to use the  `VertexAIImageQA`:
+
+```python
+pip install google-vertex-haystack
+```
+
+### On its own
+
+Basic usage:
+
+```python
+from haystack.dataclasses.byte_stream import ByteStream
+from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
+
+qa = VertexAIImageQA()
+
+image = ByteStream.from_file_path("dog.jpg")
+
+res = qa.run(image=image, question="What color is this dog")
+
+print(res["replies"][0])
+```
+
+You can also set the number of answers generated:
+
+```python
+from haystack.dataclasses.byte_stream import ByteStream
+from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
+
+qa = VertexAIImageQA(
+    number_of_results=3,
+)
+image = ByteStream.from_file_path("dog.jpg")
+
+res = qa.run(image=image, question="Tell me something about this dog")
+
+for answer in res["replies"]:
+    print(answer)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/vertexaitextgenerator.mdx

@@ -0,0 +1,67 @@
+---
+title: "VertexAITextGenerator"
+id: vertexaitextgenerator
+slug: "/vertexaitextgenerator"
+description: "This component enables text generation using Google Vertex AI generative models."
+---
+
+# VertexAITextGenerator
+
+This component enables text generation using Google Vertex AI generative models.
+
+|  |  |
+| --- | --- |
+| **Mandatory run variables** | “prompt”: A string containing the prompt for the model |
+| **Output variables** | “replies”: A list of strings containing answers generated by the model  <br /> <br />”safety_attributes”: A dictionary containing scores for safety attributes  <br /> <br />”citations”: A list of dictionaries containing grounding citations |
+| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
+
+`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models.
+
+### Parameters Overview
+
+`VertexAITextGenerator` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
+
+You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
+
+## Usage
+
+You need to install `google-vertex-haystack` package to use the  `VertexAITextGenerator`:
+
+```python
+pip install google-vertex-haystack
+```
+
+### On its own
+
+Basic usage:
+
+````python
+from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator
+
+generator = VertexAITextGenerator()
+res = generator.run("Tell me a good interview question for a software engineer.")
+
+print(res["replies"][0])
+
+````
+
+You can also set other parameters like the number of answers generated, temperature to control the randomness, and stop sequences to stop generation. For a full list of possible parameters, see the documentation of [`TextGenerationModel.predict()`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.language_models.TextGenerationModel#vertexai_language_models_TextGenerationModel_predict).
+
+```python
+from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator
+
+generator = VertexAITextGenerator(
+    candidate_count=3,
+    temperature=0.2,
+    stop_sequences=["example", "Example"],
+)
+res = generator.run("Tell me a good interview question for a software engineer.")
+
+for answer in res["replies"]:
+    print(answer)
+		print("-----")
+
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/watsonxchatgenerator.mdx

@@ -0,0 +1,108 @@
+---
+title: "WatsonxChatGenerator"
+id: watsonxchatgenerator
+slug: "/watsonxchatgenerator"
+description: "Use this component with IBM watsonx models like `granite-3-2b-instruct` for chat generation."
+---
+
+# WatsonxChatGenerator
+
+Use this component with IBM watsonx models like `granite-3-2b-instruct` for chat generation.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": The IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var.  <br /> <br />"project_id": The IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. |
+| **Mandatory run variables** | "messages" A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
+| **Output variables** | "replies": A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
+| **API reference** | [Watsonx](/reference/integrations-watsonx) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx |
+
+This integration supports IBM watsonx.ai foundation models such as `ibm/granite-13b-chat-v2`, `ibm/llama-2-70b-chat`, `ibm/llama-3-70b-instruct`, and similar. These models provide high-quality chat completion capabilities through IBM's cloud platform. Check out the most recent full list in the [IBM watsonx.ai documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-ibm.html?context=wx).
+
+## Overview
+
+`WatsonxChatGenerator` needs IBM Cloud credentials to work. You can set these in:
+
+- The `api_key` and `project_id` init parameters using [Secret API](../../concepts/secret-management.mdx)
+- The `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` environment variables (recommended)
+
+Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the IBM watsonx.ai API directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the IBM watsonx.ai API, refer to the [IBM watsonx.ai documentation](https://cloud.ibm.com/apidocs/watsonx-ai).
+
+Finally, the component needs a list of `ChatMessage` objects to operate. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+You need to install `watsonx-haystack` package to use the `WatsonxChatGenerator`:
+
+```shell
+pip install watsonx-haystack
+```
+
+#### On its own
+
+```python
+from haystack_integrations.components.generators.watsonx.chat.chat_generator import (
+    WatsonxChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+
+generator = WatsonxChatGenerator(
+    api_key=Secret.from_env_var("WATSONX_API_KEY"),
+    project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
+    model="ibm/granite-13b-instruct-v2",
+)
+
+message = ChatMessage.from_user("What's Natural Language Processing? Be brief.")
+print(generator.run([message]))
+```
+
+#### In a Pipeline
+
+You can also use `WatsonxChatGenerator` to use IBM watsonx.ai chat models in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.watsonx.chat.chat_generator import (
+    WatsonxChatGenerator,
+)
+from haystack.utils import Secret
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component(
+    "llm",
+    WatsonxChatGenerator(
+        api_key=Secret.from_env_var("WATSONX_API_KEY"),
+        project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
+        model="ibm/granite-13b-instruct-v2",
+    ),
+)
+pipe.connect("prompt_builder", "llm")
+
+country = "Germany"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving out valuable information to language learners.",
+)
+messages = [
+    system_message,
+    ChatMessage.from_user("What's the official language of {{ country }}?"),
+]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"country": country},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```

docs-website/versioned_docs/version-2.18/pipeline-components/generators/watsonxgenerator.mdx

@@ -0,0 +1,103 @@
+---
+title: "WatsonxGenerator"
+id: watsonxgenerator
+slug: "/watsonxgenerator"
+description: "Use this component with IBM watsonx models like `granite-3-2b-instruct` for simple text generation tasks."
+---
+
+# WatsonxGenerator
+
+Use this component with IBM watsonx models like `granite-3-2b-instruct` for simple text generation tasks.
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [PromptBuilder](../builders/promptbuilder.mdx)  |
+| **Mandatory init variables** | "api_key": An IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var.  <br /> <br />"project_id": An IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. |
+| **Mandatory run variables** | "prompt": A string containing the prompt for the LLM |
+| **Output variables** | "replies": A list of strings with all the replies generated by the LLM  <br /> <br />"meta": A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
+| **API reference** | [Watsonx](/reference/integrations-watsonx) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx |
+
+## Overview
+
+This integration supports IBM watsonx.ai foundation models such as `ibm/granite-13b-chat-v2`, `ibm/llama-2-70b-chat`, `ibm/llama-3-70b-instruct`, and similar. These models provide high-quality text generation capabilities through IBM's cloud platform. Check out the most recent full list in the [IBM watsonx.ai documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-ibm.html?context=wx).
+
+### Parameters
+
+`WatsonxGenerator` needs IBM Cloud credentials to work. You can provide these in:
+
+- The `WATSONX_API_KEY` environment variable (recommended)
+- The `WATSONX_PROJECT_ID` environment variable (recommended)
+- The `api_key` and `project_id` init parameters using Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
+
+Set your preferred IBM watsonx.ai model in the `model` parameter when initializing the component. The default model is `ibm/granite-3-2b-instruct`.
+
+`WatsonxGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the IBM watsonx.ai API directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the IBM watsonx.ai API, see [IBM watsonx.ai documentation](https://cloud.ibm.com/apidocs/watsonx-ai).
+
+The component also supports system prompts that can be set at initialization or passed during runtime to provide context or instructions for the generation.
+
+Finally, the component run method requires a single string prompt to generate text.
+
+### Streaming
+
+This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
+
+## Usage
+
+Install the `watsonx-haystack` package to use the `WatsonxGenerator`:
+
+```shell
+pip install watsonx-haystack
+```
+
+### On its own
+
+```python
+from haystack_integrations.components.generators.watsonx.generator import (
+    WatsonxGenerator,
+)
+from haystack.utils import Secret
+
+generator = WatsonxGenerator(
+    api_key=Secret.from_env_var("WATSONX_API_KEY"),
+    project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
+)
+
+print(generator.run("What's Natural Language Processing? Be brief."))
+```
+
+### In a pipeline
+
+You can also use `WatsonxGenerator` with the IBM watsonx.ai models in your pipeline.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import PromptBuilder
+from haystack_integrations.components.generators.watsonx.generator import (
+    WatsonxGenerator,
+)
+from haystack.utils import Secret
+
+template = """
+You are an assistant giving out valuable information to language learners.
+Answer this question, be brief.
+
+Question: {{ query }}?
+"""
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", PromptBuilder(template))
+pipe.add_component(
+    "llm",
+    WatsonxGenerator(
+        api_key=Secret.from_env_var("WATSONX_API_KEY"),
+        project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
+    ),
+)
+pipe.connect("prompt_builder", "llm")
+
+query = "What language is spoken in Germany?"
+res = pipe.run(data={"prompt_builder": {"query": query}})
+
+print(res)
+```