chore: import upstream snapshot with attribution

docs-website/reference_versioned_docs/version-2.18/integrations-api/togetherai.md

@@ -0,0 +1,282 @@
+---
+title: "Together AI"
+id: integrations-togetherai
+description: "Together AI integration for Haystack"
+slug: "/integrations-togetherai"
+---
+
+
+## haystack_integrations.components.generators.togetherai.chat.chat_generator
+
+### TogetherAIChatGenerator
+
+Bases: <code>OpenAIChatGenerator</code>
+
+Enables text generation using Together AI generative models.
+
+For supported models, see [Together AI docs](https://docs.together.ai/docs).
+
+Users can pass any text generation parameters valid for the Together AI chat completion API
+directly to this component using the `generation_kwargs` parameter in `__init__` or the `generation_kwargs`
+parameter in `run` method.
+
+Key Features and Compatibility:
+
+- **Primary Compatibility**: Designed to work seamlessly with the Together AI chat completion endpoint.
+- **Streaming Support**: Supports streaming responses from the Together AI chat completion endpoint.
+- **Customizability**: Supports all parameters supported by the Together AI chat completion endpoint.
+
+This component uses the ChatMessage format for structuring both input and output,
+ensuring coherent and contextually relevant responses in chat-based text generation scenarios.
+Details on the ChatMessage format can be found in the
+[Haystack docs](https://docs.haystack.deepset.ai/docs/chatmessage)
+
+For more details on the parameters supported by the Together AI API, refer to the
+[Together AI API Docs](https://docs.together.ai/reference/chat-completions-1).
+
+Usage example:
+
+```python
+from haystack_integrations.components.generators.togetherai import TogetherAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+messages = [ChatMessage.from_user("What's Natural Language Processing?")]
+
+client = TogetherAIChatGenerator()
+response = client.run(messages)
+print(response)
+
+>>{'replies': [ChatMessage(_content='Natural Language Processing (NLP) is a branch of artificial intelligence
+>>that focuses on enabling computers to understand, interpret, and generate human language in a way that is
+>>meaningful and useful.', _role=<ChatRole.ASSISTANT: 'assistant'>, _name=None,
+>>_meta={'model': 'meta-llama/Llama-3.3-70B-Instruct-Turbo', 'index': 0, 'finish_reason': 'stop',
+>>'usage': {'prompt_tokens': 15, 'completion_tokens': 36, 'total_tokens': 51}})]}
+```
+
+#### __init__
+
+```python
+__init__(
+    *,
+    api_key: Secret = Secret.from_env_var("TOGETHER_API_KEY"),
+    model: str = "meta-llama/Llama-3.3-70B-Instruct-Turbo",
+    streaming_callback: StreamingCallbackT | None = None,
+    api_base_url: str | None = "https://api.together.xyz/v1",
+    generation_kwargs: dict[str, Any] | None = None,
+    tools: ToolsType | None = None,
+    timeout: float | None = None,
+    max_retries: int | None = None,
+    http_client_kwargs: dict[str, Any] | None = None
+) -> None
+```
+
+Creates an instance of TogetherAIChatGenerator.
+
+**Parameters:**
+
+- **api_key** (<code>Secret</code>) – The Together API key.
+- **model** (<code>str</code>) – The name of the Together AI chat completion model to use.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+  The callback function accepts StreamingChunk as an argument.
+- **api_base_url** (<code>str | None</code>) – The Together AI API Base url.
+  For more details, see Together AI [docs](https://docs.together.ai/docs/openai-api-compatibility).
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Other parameters to use for the model. These parameters are all sent directly to
+  the Together AI endpoint. See [Together AI API docs](https://docs.together.ai/reference/chat-completions-1)
+  for more details.
+  Some of the supported parameters:
+- `max_tokens`: The maximum number of tokens the output text can have.
+- `temperature`: What sampling temperature to use. Higher values mean the model will take more risks.
+  Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer.
+- `top_p`: An alternative to sampling with temperature, called nucleus sampling, where the model
+  considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens
+  comprising the top 10% probability mass are considered.
+- `stream`: Whether to stream back partial progress. If set, tokens will be sent as data-only server-sent
+  events as they become available, with the stream terminated by a data: [DONE] message.
+- `safe_prompt`: Whether to inject a safety prompt before all conversations.
+- `random_seed`: The seed to use for random sampling.
+- `response_format`: A JSON schema or a Pydantic model that enforces the structure of the model's response.
+  If provided, the output will always be validated against this
+  format (unless the model returns a tool call).
+  For details, see the [OpenAI Structured Outputs documentation](https://platform.openai.com/docs/guides/structured-outputs).
+  Notes:
+  - For structured outputs with streaming,
+    the `response_format` must be a JSON schema and not a Pydantic model.
+- **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
+  Each tool should have a unique name.
+- **timeout** (<code>float | None</code>) – The timeout for the Together AI API call.
+- **max_retries** (<code>int | None</code>) – Maximum number of retries to contact Together AI after an internal error.
+  If not set, it defaults to either the `OPENAI_MAX_RETRIES` environment variable, or set to 5.
+- **http_client_kwargs** (<code>dict\[str, Any\] | None</code>) – A dictionary of keyword arguments to configure a custom `httpx.Client`or `httpx.AsyncClient`.
+  For more information, see the [HTTPX documentation](https://www.python-httpx.org/api/#client).
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serialize this component to a dictionary.
+
+**Returns:**
+
+- <code>dict\[str, Any\]</code> – The serialized component as a dictionary.
+
+## haystack_integrations.components.generators.togetherai.generator
+
+### TogetherAIGenerator
+
+Bases: <code>TogetherAIChatGenerator</code>
+
+Provides an interface to generate text using an LLM running on Together AI.
+
+Usage example:
+
+```python
+from haystack_integrations.components.generators.togetherai import TogetherAIGenerator
+
+generator = TogetherAIGenerator(model="deepseek-ai/DeepSeek-R1",
+                            generation_kwargs={
+                            "temperature": 0.9,
+                            })
+
+print(generator.run("Who is the best Italian actor?"))
+```
+
+#### __init__
+
+```python
+__init__(
+    api_key: Secret = Secret.from_env_var("TOGETHER_API_KEY"),
+    model: str = "meta-llama/Llama-3.3-70B-Instruct-Turbo",
+    api_base_url: str | None = "https://api.together.xyz/v1",
+    streaming_callback: StreamingCallbackT | None = None,
+    system_prompt: str | None = None,
+    generation_kwargs: dict[str, Any] | None = None,
+    timeout: float | None = None,
+    max_retries: int | None = None,
+) -> None
+```
+
+Initialize the TogetherAIGenerator.
+
+**Parameters:**
+
+- **api_key** (<code>Secret</code>) – The Together API key.
+- **model** (<code>str</code>) – The name of the model to use.
+- **api_base_url** (<code>str | None</code>) – The base URL of the Together AI API.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+  The callback function accepts StreamingChunk as an argument.
+- **system_prompt** (<code>str | None</code>) – The system prompt to use for text generation. If not provided, the system prompt is
+  omitted, and the default system prompt of the model is used.
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Other parameters to use for the model. These parameters are all sent directly to
+  the Together AI endpoint. See Together AI
+  [documentation](https://docs.together.ai/reference/chat-completions-1) for more details.
+  Some of the supported parameters:
+- `max_tokens`: The maximum number of tokens the output text can have.
+- `temperature`: What sampling temperature to use. Higher values mean the model will take more risks.
+  Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer.
+- `top_p`: An alternative to sampling with temperature, called nucleus sampling, where the model
+  considers the results of the tokens with top_p probability mass. So, 0.1 means only the tokens
+  comprising the top 10% probability mass are considered.
+- `n`: How many completions to generate for each prompt. For example, if the LLM gets 3 prompts and n is 2,
+  it will generate two completions for each of the three prompts, ending up with 6 completions in total.
+- `stop`: One or more sequences after which the LLM should stop generating tokens.
+- `presence_penalty`: What penalty to apply if a token is already present at all. Bigger values mean
+  the model will be less likely to repeat the same token in the text.
+- `frequency_penalty`: What penalty to apply if a token has already been generated in the text.
+  Bigger values mean the model will be less likely to repeat the same token in the text.
+- `logit_bias`: Add a logit bias to specific tokens. The keys of the dictionary are tokens, and the
+  values are the bias to add to that token.
+- **timeout** (<code>float | None</code>) – Timeout for together.ai Client calls, if not set it is inferred from the `OPENAI_TIMEOUT` environment
+  variable or set to 30.
+- **max_retries** (<code>int | None</code>) – Maximum retries to establish contact with Together AI if it returns an internal error, if not set it is
+  inferred from the `OPENAI_MAX_RETRIES` environment variable or set to 5.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serialize this component to a dictionary.
+
+**Returns:**
+
+- <code>dict\[str, Any\]</code> – The serialized component as a dictionary.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> TogetherAIGenerator
+```
+
+Deserialize this component from a dictionary.
+
+**Parameters:**
+
+- **data** (<code>dict\[str, Any\]</code>) – The dictionary representation of this component.
+
+**Returns:**
+
+- <code>TogetherAIGenerator</code> – The deserialized component instance.
+
+#### run
+
+```python
+run(
+    *,
+    prompt: str,
+    system_prompt: str | None = None,
+    streaming_callback: StreamingCallbackT | None = None,
+    generation_kwargs: dict[str, Any] | None = None
+) -> dict[str, Any]
+```
+
+Generate text completions synchronously.
+
+**Parameters:**
+
+- **prompt** (<code>str</code>) – The input prompt string for text generation.
+- **system_prompt** (<code>str | None</code>) – An optional system prompt to provide context or instructions for the generation.
+  If not provided, the system prompt set in the `__init__` method will be used.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+  If provided, this will override the `streaming_callback` set in the `__init__` method.
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional keyword arguments for text generation. These parameters will potentially override the parameters
+  passed in the `__init__` method. Supported parameters include temperature, max_new_tokens, top_p, etc.
+
+**Returns:**
+
+- <code>dict\[str, Any\]</code> – A dictionary with the following keys:
+- `replies`: A list of generated text completions as strings.
+- `meta`: A list of metadata dictionaries containing information about each generation,
+  including model name, finish reason, and token usage statistics.
+
+#### run_async
+
+```python
+run_async(
+    *,
+    prompt: str,
+    system_prompt: str | None = None,
+    streaming_callback: StreamingCallbackT | None = None,
+    generation_kwargs: dict[str, Any] | None = None
+) -> dict[str, Any]
+```
+
+Generate text completions asynchronously.
+
+**Parameters:**
+
+- **prompt** (<code>str</code>) – The input prompt string for text generation.
+- **system_prompt** (<code>str | None</code>) – An optional system prompt to provide context or instructions for the generation.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+  If provided, this will override the `streaming_callback` set in the `__init__` method.
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional keyword arguments for text generation. These parameters will potentially override the parameters
+  passed in the `__init__` method. Supported parameters include temperature, max_new_tokens, top_p, etc.
+
+**Returns:**
+
+- <code>dict\[str, Any\]</code> – A dictionary with the following keys:
+- `replies`: A list of generated text completions as strings.
+- `meta`: A list of metadata dictionaries containing information about each generation,
+  including model name, finish reason, and token usage statistics.