chore: import upstream snapshot with attribution

docs-website/reference_versioned_docs/version-2.28/integrations-api/optimum.md

@@ -0,0 +1,629 @@
+---
+title: "Optimum"
+id: integrations-optimum
+description: "Optimum integration for Haystack"
+slug: "/integrations-optimum"
+---
+
+<a id="haystack_integrations.components.embedders.optimum.optimization"></a>
+
+## Module haystack\_integrations.components.embedders.optimum.optimization
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationMode"></a>
+
+### OptimumEmbedderOptimizationMode
+
+[ONXX Optimization modes](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/optimization)
+support by the Optimum Embedders.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationMode.O1"></a>
+
+#### O1
+
+Basic general optimizations.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationMode.O2"></a>
+
+#### O2
+
+Basic and extended general optimizations, transformers-specific fusions.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationMode.O3"></a>
+
+#### O3
+
+Same as O2 with Gelu approximation.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationMode.O4"></a>
+
+#### O4
+
+Same as O3 with mixed precision.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationMode.from_str"></a>
+
+#### OptimumEmbedderOptimizationMode.from\_str
+
+```python
+@classmethod
+def from_str(cls, string: str) -> "OptimumEmbedderOptimizationMode"
+```
+
+Create an optimization mode from a string.
+
+**Arguments**:
+
+- `string`: String to convert.
+
+**Returns**:
+
+Optimization mode.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationConfig"></a>
+
+### OptimumEmbedderOptimizationConfig
+
+Configuration for Optimum Embedder Optimization.
+
+**Arguments**:
+
+- `mode`: Optimization mode.
+- `for_gpu`: Whether to optimize for GPUs.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationConfig.to_optimum_config"></a>
+
+#### OptimumEmbedderOptimizationConfig.to\_optimum\_config
+
+```python
+def to_optimum_config() -> OptimizationConfig
+```
+
+Convert the configuration to a Optimum configuration.
+
+**Returns**:
+
+Optimum configuration.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationConfig.to_dict"></a>
+
+#### OptimumEmbedderOptimizationConfig.to\_dict
+
+```python
+def to_dict() -> dict[str, Any]
+```
+
+Convert the configuration to a dictionary.
+
+**Returns**:
+
+Dictionary with serialized data.
+
+<a id="haystack_integrations.components.embedders.optimum.optimization.OptimumEmbedderOptimizationConfig.from_dict"></a>
+
+#### OptimumEmbedderOptimizationConfig.from\_dict
+
+```python
+@classmethod
+def from_dict(cls, data: dict[str,
+                              Any]) -> "OptimumEmbedderOptimizationConfig"
+```
+
+Create an optimization configuration from a dictionary.
+
+**Arguments**:
+
+- `data`: Dictionary to deserialize from.
+
+**Returns**:
+
+Optimization configuration.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder"></a>
+
+## Module haystack\_integrations.components.embedders.optimum.optimum\_document\_embedder
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder.OptimumDocumentEmbedder"></a>
+
+### OptimumDocumentEmbedder
+
+A component for computing `Document` embeddings using models loaded with the
+[HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library,
+leveraging the ONNX runtime for high-speed inference.
+
+The embedding of each Document is stored in the `embedding` field of the Document.
+
+Usage example:
+```python
+from haystack.dataclasses import Document
+from haystack_integrations.components.embedders.optimum import OptimumDocumentEmbedder
+
+doc = Document(content="I love pizza!")
+
+document_embedder = OptimumDocumentEmbedder(model="sentence-transformers/all-mpnet-base-v2")
+document_embedder.warm_up()
+
+result = document_embedder.run([doc])
+print(result["documents"][0].embedding)
+
+# [0.017020374536514282, -0.023255806416273117, ...]
+```
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder.OptimumDocumentEmbedder.__init__"></a>
+
+#### OptimumDocumentEmbedder.\_\_init\_\_
+
+```python
+def __init__(model: str = "sentence-transformers/all-mpnet-base-v2",
+             token: Secret | None = Secret.from_env_var("HF_API_TOKEN",
+                                                        strict=False),
+             prefix: str = "",
+             suffix: str = "",
+             normalize_embeddings: bool = True,
+             onnx_execution_provider: str = "CPUExecutionProvider",
+             pooling_mode: str | OptimumEmbedderPooling | None = None,
+             model_kwargs: dict[str, Any] | None = None,
+             working_dir: str | None = None,
+             optimizer_settings: OptimumEmbedderOptimizationConfig
+             | None = None,
+             quantizer_settings: OptimumEmbedderQuantizationConfig
+             | None = None,
+             batch_size: int = 32,
+             progress_bar: bool = True,
+             meta_fields_to_embed: list[str] | None = None,
+             embedding_separator: str = "\n") -> None
+```
+
+Create a OptimumDocumentEmbedder component.
+
+**Arguments**:
+
+- `model`: A string representing the model id on HF Hub.
+- `token`: The HuggingFace token to use as HTTP bearer authorization.
+- `prefix`: A string to add to the beginning of each text.
+- `suffix`: A string to add to the end of each text.
+- `normalize_embeddings`: Whether to normalize the embeddings to unit length.
+- `onnx_execution_provider`: The [execution provider](https://onnxruntime.ai/docs/execution-providers/)
+to use for ONNX models.
+
+Note: Using the TensorRT execution provider
+TensorRT requires to build its inference engine ahead of inference,
+which takes some time due to the model optimization and nodes fusion.
+To avoid rebuilding the engine every time the model is loaded, ONNX
+Runtime provides a pair of options to save the engine: `trt_engine_cache_enable`
+and `trt_engine_cache_path`. We recommend setting these two provider
+options using the `model_kwargs` parameter, when using the TensorRT execution provider.
+The usage is as follows:
+```python
+embedder = OptimumDocumentEmbedder(
+    model="sentence-transformers/all-mpnet-base-v2",
+    onnx_execution_provider="TensorrtExecutionProvider",
+    model_kwargs={
+        "provider_options": {
+            "trt_engine_cache_enable": True,
+            "trt_engine_cache_path": "tmp/trt_cache",
+        }
+    },
+)
+```
+- `pooling_mode`: The pooling mode to use. When `None`, pooling mode will be inferred from the model config.
+- `model_kwargs`: Dictionary containing additional keyword arguments to pass to the model.
+In case of duplication, these kwargs override `model`, `onnx_execution_provider`
+and `token` initialization parameters.
+- `working_dir`: The directory to use for storing intermediate files
+generated during model optimization/quantization. Required
+for optimization and quantization.
+- `optimizer_settings`: Configuration for Optimum Embedder Optimization.
+If `None`, no additional optimization is be applied.
+- `quantizer_settings`: Configuration for Optimum Embedder Quantization.
+If `None`, no quantization is be applied.
+- `batch_size`: Number of Documents to encode at once.
+- `progress_bar`: Whether to show a progress bar or not.
+- `meta_fields_to_embed`: List of meta fields that should be embedded along with the Document text.
+- `embedding_separator`: Separator used to concatenate the meta fields to the Document text.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder.OptimumDocumentEmbedder.warm_up"></a>
+
+#### OptimumDocumentEmbedder.warm\_up
+
+```python
+def warm_up() -> None
+```
+
+Initializes the component.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder.OptimumDocumentEmbedder.to_dict"></a>
+
+#### OptimumDocumentEmbedder.to\_dict
+
+```python
+def to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns**:
+
+Dictionary with serialized data.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder.OptimumDocumentEmbedder.from_dict"></a>
+
+#### OptimumDocumentEmbedder.from\_dict
+
+```python
+@classmethod
+def from_dict(cls, data: dict[str, Any]) -> "OptimumDocumentEmbedder"
+```
+
+Deserializes the component from a dictionary.
+
+**Arguments**:
+
+- `data`: The dictionary to deserialize from.
+
+**Returns**:
+
+The deserialized component.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_document_embedder.OptimumDocumentEmbedder.run"></a>
+
+#### OptimumDocumentEmbedder.run
+
+```python
+@component.output_types(documents=list[Document])
+def run(documents: list[Document]) -> dict[str, list[Document]]
+```
+
+Embed a list of Documents.
+
+The embedding of each Document is stored in the `embedding` field of the Document.
+
+**Arguments**:
+
+- `documents`: A list of Documents to embed.
+
+**Raises**:
+
+- `TypeError`: If the input is not a list of Documents.
+
+**Returns**:
+
+The updated Documents with their embeddings.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder"></a>
+
+## Module haystack\_integrations.components.embedders.optimum.optimum\_text\_embedder
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder.OptimumTextEmbedder"></a>
+
+### OptimumTextEmbedder
+
+A component to embed text using models loaded with the
+[HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library,
+leveraging the ONNX runtime for high-speed inference.
+
+Usage example:
+```python
+from haystack_integrations.components.embedders.optimum import OptimumTextEmbedder
+
+text_to_embed = "I love pizza!"
+
+text_embedder = OptimumTextEmbedder(model="sentence-transformers/all-mpnet-base-v2")
+text_embedder.warm_up()
+
+print(text_embedder.run(text_to_embed))
+
+# {'embedding': [-0.07804739475250244, 0.1498992145061493,, ...]}
+```
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder.OptimumTextEmbedder.__init__"></a>
+
+#### OptimumTextEmbedder.\_\_init\_\_
+
+```python
+def __init__(
+        model: str = "sentence-transformers/all-mpnet-base-v2",
+        token: Secret | None = Secret.from_env_var("HF_API_TOKEN",
+                                                   strict=False),
+        prefix: str = "",
+        suffix: str = "",
+        normalize_embeddings: bool = True,
+        onnx_execution_provider: str = "CPUExecutionProvider",
+        pooling_mode: str | OptimumEmbedderPooling | None = None,
+        model_kwargs: dict[str, Any] | None = None,
+        working_dir: str | None = None,
+        optimizer_settings: OptimumEmbedderOptimizationConfig | None = None,
+        quantizer_settings: OptimumEmbedderQuantizationConfig | None = None)
+```
+
+Create a OptimumTextEmbedder component.
+
+**Arguments**:
+
+- `model`: A string representing the model id on HF Hub.
+- `token`: The HuggingFace token to use as HTTP bearer authorization.
+- `prefix`: A string to add to the beginning of each text.
+- `suffix`: A string to add to the end of each text.
+- `normalize_embeddings`: Whether to normalize the embeddings to unit length.
+- `onnx_execution_provider`: The [execution provider](https://onnxruntime.ai/docs/execution-providers/)
+to use for ONNX models.
+
+Note: Using the TensorRT execution provider
+TensorRT requires to build its inference engine ahead of inference,
+which takes some time due to the model optimization and nodes fusion.
+To avoid rebuilding the engine every time the model is loaded, ONNX
+Runtime provides a pair of options to save the engine: `trt_engine_cache_enable`
+and `trt_engine_cache_path`. We recommend setting these two provider
+options using the `model_kwargs` parameter, when using the TensorRT execution provider.
+The usage is as follows:
+```python
+embedder = OptimumDocumentEmbedder(
+    model="sentence-transformers/all-mpnet-base-v2",
+    onnx_execution_provider="TensorrtExecutionProvider",
+    model_kwargs={
+        "provider_options": {
+            "trt_engine_cache_enable": True,
+            "trt_engine_cache_path": "tmp/trt_cache",
+        }
+    },
+)
+```
+- `pooling_mode`: The pooling mode to use. When `None`, pooling mode will be inferred from the model config.
+- `model_kwargs`: Dictionary containing additional keyword arguments to pass to the model.
+In case of duplication, these kwargs override `model`, `onnx_execution_provider`
+and `token` initialization parameters.
+- `working_dir`: The directory to use for storing intermediate files
+generated during model optimization/quantization. Required
+for optimization and quantization.
+- `optimizer_settings`: Configuration for Optimum Embedder Optimization.
+If `None`, no additional optimization is be applied.
+- `quantizer_settings`: Configuration for Optimum Embedder Quantization.
+If `None`, no quantization is be applied.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder.OptimumTextEmbedder.warm_up"></a>
+
+#### OptimumTextEmbedder.warm\_up
+
+```python
+def warm_up()
+```
+
+Initializes the component.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder.OptimumTextEmbedder.to_dict"></a>
+
+#### OptimumTextEmbedder.to\_dict
+
+```python
+def to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns**:
+
+Dictionary with serialized data.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder.OptimumTextEmbedder.from_dict"></a>
+
+#### OptimumTextEmbedder.from\_dict
+
+```python
+@classmethod
+def from_dict(cls, data: dict[str, Any]) -> "OptimumTextEmbedder"
+```
+
+Deserializes the component from a dictionary.
+
+**Arguments**:
+
+- `data`: The dictionary to deserialize from.
+
+**Returns**:
+
+The deserialized component.
+
+<a id="haystack_integrations.components.embedders.optimum.optimum_text_embedder.OptimumTextEmbedder.run"></a>
+
+#### OptimumTextEmbedder.run
+
+```python
+@component.output_types(embedding=list[float])
+def run(text: str) -> dict[str, list[float]]
+```
+
+Embed a string.
+
+**Arguments**:
+
+- `text`: The text to embed.
+
+**Raises**:
+
+- `TypeError`: If the input is not a string.
+
+**Returns**:
+
+The embeddings of the text.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling"></a>
+
+## Module haystack\_integrations.components.embedders.optimum.pooling
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling"></a>
+
+### OptimumEmbedderPooling
+
+Pooling modes support by the Optimum Embedders.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.CLS"></a>
+
+#### CLS
+
+Perform CLS Pooling on the output of the embedding model
+using the first token (CLS token).
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.MEAN"></a>
+
+#### MEAN
+
+Perform Mean Pooling on the output of the embedding model.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.MAX"></a>
+
+#### MAX
+
+Perform Max Pooling on the output of the embedding model
+using the maximum value in each dimension over all the tokens.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.MEAN_SQRT_LEN"></a>
+
+#### MEAN\_SQRT\_LEN
+
+Perform mean-pooling on the output of the embedding model but
+divide by the square root of the sequence length.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.WEIGHTED_MEAN"></a>
+
+#### WEIGHTED\_MEAN
+
+Perform weighted (position) mean pooling on the output of the
+embedding model.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.LAST_TOKEN"></a>
+
+#### LAST\_TOKEN
+
+Perform Last Token Pooling on the output of the embedding model.
+
+<a id="haystack_integrations.components.embedders.optimum.pooling.OptimumEmbedderPooling.from_str"></a>
+
+#### OptimumEmbedderPooling.from\_str
+
+```python
+@classmethod
+def from_str(cls, string: str) -> "OptimumEmbedderPooling"
+```
+
+Create a pooling mode from a string.
+
+**Arguments**:
+
+- `string`: String to convert.
+
+**Returns**:
+
+Pooling mode.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization"></a>
+
+## Module haystack\_integrations.components.embedders.optimum.quantization
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationMode"></a>
+
+### OptimumEmbedderQuantizationMode
+
+[Dynamic Quantization modes](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/quantization)
+support by the Optimum Embedders.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationMode.ARM64"></a>
+
+#### ARM64
+
+Quantization for the ARM64 architecture.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationMode.AVX2"></a>
+
+#### AVX2
+
+Quantization with AVX-2 instructions.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationMode.AVX512"></a>
+
+#### AVX512
+
+Quantization with AVX-512 instructions.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationMode.AVX512_VNNI"></a>
+
+#### AVX512\_VNNI
+
+Quantization with AVX-512 and VNNI instructions.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationMode.from_str"></a>
+
+#### OptimumEmbedderQuantizationMode.from\_str
+
+```python
+@classmethod
+def from_str(cls, string: str) -> "OptimumEmbedderQuantizationMode"
+```
+
+Create an quantization mode from a string.
+
+**Arguments**:
+
+- `string`: String to convert.
+
+**Returns**:
+
+Quantization mode.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationConfig"></a>
+
+### OptimumEmbedderQuantizationConfig
+
+Configuration for Optimum Embedder Quantization.
+
+**Arguments**:
+
+- `mode`: Quantization mode.
+- `per_channel`: Whether to apply per-channel quantization.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationConfig.to_optimum_config"></a>
+
+#### OptimumEmbedderQuantizationConfig.to\_optimum\_config
+
+```python
+def to_optimum_config() -> QuantizationConfig
+```
+
+Convert the configuration to a Optimum configuration.
+
+**Returns**:
+
+Optimum configuration.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationConfig.to_dict"></a>
+
+#### OptimumEmbedderQuantizationConfig.to\_dict
+
+```python
+def to_dict() -> dict[str, Any]
+```
+
+Convert the configuration to a dictionary.
+
+**Returns**:
+
+Dictionary with serialized data.
+
+<a id="haystack_integrations.components.embedders.optimum.quantization.OptimumEmbedderQuantizationConfig.from_dict"></a>
+
+#### OptimumEmbedderQuantizationConfig.from\_dict
+
+```python
+@classmethod
+def from_dict(cls, data: dict[str,
+                              Any]) -> "OptimumEmbedderQuantizationConfig"
+```
+
+Create a configuration from a dictionary.
+
+**Arguments**:
+
+- `data`: Dictionary to deserialize from.
+
+**Returns**:
+
+Quantization configuration.
+