chore: import upstream snapshot with attribution

docs-website/reference_versioned_docs/version-2.21/integrations-api/vllm.md

@@ -0,0 +1,699 @@
+---
+title: "vLLM"
+id: integrations-vllm
+description: "vLLM integration for Haystack"
+slug: "/integrations-vllm"
+---
+
+
+## haystack_integrations.components.embedders.vllm.document_embedder
+
+### VLLMDocumentEmbedder
+
+A component for computing Document embeddings using models served with [vLLM](https://docs.vllm.ai/).
+
+The embedding of each Document is stored in the `embedding` field of the Document.
+It expects a vLLM server to be running and accessible at the `api_base_url` parameter and uses the
+OpenAI-compatible Embeddings API exposed by vLLM.
+
+### Starting the vLLM server
+
+Before using this component, start a vLLM server with an embedding model:
+
+```bash
+vllm serve google/embeddinggemma-300m
+```
+
+For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
+
+### Usage example
+
+```python
+from haystack import Document
+from haystack_integrations.components.embedders.vllm import VLLMDocumentEmbedder
+
+doc = Document(content="I love pizza!")
+
+document_embedder = VLLMDocumentEmbedder(model="google/embeddinggemma-300m")
+
+result = document_embedder.run([doc])
+print(result["documents"][0].embedding)
+```
+
+### Usage example with vLLM-specific parameters
+
+Pass vLLM-specific parameters via the `extra_parameters` dictionary. They are forwarded as `extra_body`
+to the OpenAI-compatible endpoint.
+
+```python
+document_embedder = VLLMDocumentEmbedder(
+    model="google/embeddinggemma-300m",
+    extra_parameters={"truncate_prompt_tokens": 256, "truncation_side": "right"},
+)
+```
+
+#### __init__
+
+```python
+__init__(
+    *,
+    model: str,
+    api_key: Secret | None = Secret.from_env_var("VLLM_API_KEY", strict=False),
+    api_base_url: str = "http://localhost:8000/v1",
+    prefix: str = "",
+    suffix: str = "",
+    dimensions: int | None = None,
+    batch_size: int = 32,
+    progress_bar: bool = True,
+    meta_fields_to_embed: list[str] | None = None,
+    embedding_separator: str = "\n",
+    timeout: float | None = None,
+    max_retries: int | None = None,
+    http_client_kwargs: dict[str, Any] | None = None,
+    raise_on_failure: bool = False,
+    extra_parameters: dict[str, Any] | None = None
+) -> None
+```
+
+Creates an instance of VLLMDocumentEmbedder.
+
+**Parameters:**
+
+- **model** (<code>str</code>) – The name of the model served by vLLM. Check
+  [vLLM documentation](https://docs.vllm.ai/en/stable/models/pooling_models) for more information.
+- **api_key** (<code>Secret | None</code>) – The vLLM API key. Defaults to the `VLLM_API_KEY` environment variable.
+  Only required if the vLLM server was started with `--api-key`.
+- **api_base_url** (<code>str</code>) – The base URL of the vLLM server.
+- **prefix** (<code>str</code>) – A string to add at the beginning of each text.
+- **suffix** (<code>str</code>) – A string to add at the end of each text.
+- **dimensions** (<code>int | None</code>) – The number of dimensions of the resulting embedding. Only models trained with
+  Matryoshka Representation Learning support this parameter. See
+  [vLLM documentation](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#matryoshka-embeddings)
+  for more information.
+- **batch_size** (<code>int</code>) – Number of documents to encode at once.
+- **progress_bar** (<code>bool</code>) – Whether to show a progress bar.
+- **meta_fields_to_embed** (<code>list\[str\] | None</code>) – List of meta fields to embed along with the document text.
+- **embedding_separator** (<code>str</code>) – Separator used to concatenate the meta fields to the document text.
+- **timeout** (<code>float | None</code>) – Timeout in seconds for vLLM client calls. If not set, the OpenAI client default applies.
+- **max_retries** (<code>int | None</code>) – Maximum number of retries for failed requests. If not set, the OpenAI client
+  default applies.
+- **http_client_kwargs** (<code>dict\[str, Any\] | None</code>) – A dictionary of keyword arguments to configure a custom `httpx.Client` or
+  `httpx.AsyncClient`. For more information, see the
+  [HTTPX documentation](https://www.python-httpx.org/api/#client).
+- **raise_on_failure** (<code>bool</code>) – Whether to raise an exception if the embedding request fails. If `False`,
+  the component logs the error and continues processing the remaining documents.
+- **extra_parameters** (<code>dict\[str, Any\] | None</code>) – Additional parameters forwarded as `extra_body` to the vLLM embeddings
+  endpoint. Use this to pass parameters not part of the standard OpenAI Embeddings API, such as
+  `truncate_prompt_tokens`, `truncation_side`, etc. See the
+  [vLLM Embeddings API docs](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#openai-compatible-embeddings-api).
+
+#### warm_up
+
+```python
+warm_up() -> None
+```
+
+Create the OpenAI clients.
+
+#### run
+
+```python
+run(documents: list[Document]) -> dict[str, list[Document] | dict[str, Any]]
+```
+
+Embed a list of Documents.
+
+**Parameters:**
+
+- **documents** (<code>list\[Document\]</code>) – Documents to embed.
+
+**Returns:**
+
+- <code>dict\[str, list\[Document\] | dict\[str, Any\]\]</code> – A dictionary with:
+- `documents`: The input documents with their `embedding` field populated.
+- `meta`: Information about the usage of the model.
+
+#### run_async
+
+```python
+run_async(
+    documents: list[Document],
+) -> dict[str, list[Document] | dict[str, Any]]
+```
+
+Asynchronously embed a list of Documents.
+
+**Parameters:**
+
+- **documents** (<code>list\[Document\]</code>) – Documents to embed.
+
+**Returns:**
+
+- <code>dict\[str, list\[Document\] | dict\[str, Any\]\]</code> – A dictionary with:
+- `documents`: The input documents with their `embedding` field populated.
+- `meta`: Information about the usage of the model.
+
+## haystack_integrations.components.embedders.vllm.text_embedder
+
+### VLLMTextEmbedder
+
+A component for embedding strings using models served with [vLLM](https://docs.vllm.ai/).
+
+It expects a vLLM server to be running and accessible at the `api_base_url` parameter and uses the
+OpenAI-compatible Embeddings API exposed by vLLM.
+
+### Starting the vLLM server
+
+Before using this component, start a vLLM server with an embedding model:
+
+```bash
+vllm serve google/embeddinggemma-300m
+```
+
+For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
+
+### Usage example
+
+```python
+from haystack_integrations.components.embedders.vllm import VLLMTextEmbedder
+
+text_embedder = VLLMTextEmbedder(model="google/embeddinggemma-300m")
+print(text_embedder.run("I love pizza!"))
+```
+
+### Usage example with vLLM-specific parameters
+
+Pass vLLM-specific parameters via the `extra_parameters` dictionary. They are forwarded as `extra_body`
+to the OpenAI-compatible endpoint.
+
+```python
+text_embedder = VLLMTextEmbedder(
+    model="google/embeddinggemma-300m",
+    extra_parameters={"truncate_prompt_tokens": 256, "truncation_side": "right"},
+)
+```
+
+#### __init__
+
+```python
+__init__(
+    *,
+    model: str,
+    api_key: Secret | None = Secret.from_env_var("VLLM_API_KEY", strict=False),
+    api_base_url: str = "http://localhost:8000/v1",
+    prefix: str = "",
+    suffix: str = "",
+    dimensions: int | None = None,
+    timeout: float | None = None,
+    max_retries: int | None = None,
+    http_client_kwargs: dict[str, Any] | None = None,
+    extra_parameters: dict[str, Any] | None = None
+) -> None
+```
+
+Creates an instance of VLLMTextEmbedder.
+
+**Parameters:**
+
+- **model** (<code>str</code>) – The name of the model served by vLLM (e.g., "intfloat/e5-mistral-7b-instruct").
+- **api_key** (<code>Secret | None</code>) – The vLLM API key. Defaults to the `VLLM_API_KEY` environment variable.
+  Only required if the vLLM server was started with `--api-key`.
+- **api_base_url** (<code>str</code>) – The base URL of the vLLM server.
+- **prefix** (<code>str</code>) – A string to add at the beginning of each text to embed.
+- **suffix** (<code>str</code>) – A string to add at the end of each text to embed.
+- **dimensions** (<code>int | None</code>) – The number of dimensions of the resulting embedding. Only models trained with
+  Matryoshka Representation Learning support this parameter. See
+  [vLLM documentation](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#matryoshka-embeddings)
+  for more information.
+- **timeout** (<code>float | None</code>) – Timeout in seconds for vLLM client calls. If not set, the OpenAI client default applies.
+- **max_retries** (<code>int | None</code>) – Maximum number of retries for failed requests. If not set, the OpenAI client
+  default applies.
+- **http_client_kwargs** (<code>dict\[str, Any\] | None</code>) – A dictionary of keyword arguments to configure a custom `httpx.Client` or
+  `httpx.AsyncClient`. For more information, see the
+  [HTTPX documentation](https://www.python-httpx.org/api/#client).
+- **extra_parameters** (<code>dict\[str, Any\] | None</code>) – Additional parameters forwarded as `extra_body` to the vLLM embeddings
+  endpoint. Use this to pass parameters not part of the standard OpenAI Embeddings API, such as
+  `truncate_prompt_tokens`, `truncation_side`, `additional_data`, `use_activation`, etc. See the
+  [vLLM Embeddings API docs](https://docs.vllm.ai/en/stable/models/pooling_models/embed/#openai-compatible-embeddings-api).
+
+#### warm_up
+
+```python
+warm_up() -> None
+```
+
+Create the OpenAI clients.
+
+#### run
+
+```python
+run(text: str) -> dict[str, list[float] | dict[str, Any]]
+```
+
+Embed a single string.
+
+**Parameters:**
+
+- **text** (<code>str</code>) – Text to embed.
+
+**Returns:**
+
+- <code>dict\[str, list\[float\] | dict\[str, Any\]\]</code> – A dictionary with:
+- `embedding`: The embedding of the input text.
+- `meta`: Information about the usage of the model.
+
+#### run_async
+
+```python
+run_async(text: str) -> dict[str, list[float] | dict[str, Any]]
+```
+
+Asynchronously embed a single string.
+
+**Parameters:**
+
+- **text** (<code>str</code>) – Text to embed.
+
+**Returns:**
+
+- <code>dict\[str, list\[float\] | dict\[str, Any\]\]</code> – A dictionary with:
+- `embedding`: The embedding of the input text.
+- `meta`: Information about the usage of the model.
+
+## haystack_integrations.components.generators.vllm.chat.chat_generator
+
+### VLLMChatGenerator
+
+A component for generating chat completions using models served with [vLLM](https://docs.vllm.ai/).
+
+It expects a vLLM server to be running and accessible at the `api_base_url` parameter.
+
+### Starting the vLLM server
+
+Before using this component, start a vLLM server:
+
+```bash
+vllm serve Qwen/Qwen3-4B-Instruct-2507
+```
+
+For reasoning models, start the server with the appropriate reasoning parser:
+
+```bash
+vllm serve Qwen/Qwen3-0.6B --reasoning-parser qwen3
+```
+
+For tool calling, the server must be started with `--enable-auto-tool-choice` and `--tool-call-parser`:
+
+```bash
+vllm serve Qwen/Qwen3-0.6B --enable-auto-tool-choice --tool-call-parser hermes
+```
+
+The available tool call parsers depend on the model. See the
+[vLLM tool calling docs](https://docs.vllm.ai/en/stable/features/tool_calling/) for the full list.
+
+For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
+
+### Usage example
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.vllm import VLLMChatGenerator
+
+generator = VLLMChatGenerator(
+    model="Qwen/Qwen3-0.6B",
+    generation_kwargs={"max_tokens": 512, "temperature": 0.7},
+)
+
+messages = [ChatMessage.from_user("What's Natural Language Processing?")]
+response = generator.run(messages=messages)
+print(response["replies"][0].text)
+```
+
+### Usage example with vLLM-specific parameters
+
+Pass the vLLM-specific parameters inside the `generation_kwargs`["extra_body"] dictionary.
+
+```python
+from haystack_integrations.components.generators.vllm import VLLMChatGenerator
+
+generator = VLLMChatGenerator(
+    model="Qwen/Qwen3-0.6B",
+    generation_kwargs={
+        "max_tokens": 512,
+        "extra_body": {
+            "top_k": 50,
+            "min_tokens": 10,
+            "repetition_penalty": 1.1,
+        },
+    },
+)
+```
+
+### Usage example with tool calling
+
+To use tool calling, start the vLLM server with `--enable-auto-tool-choice` and `--tool-call-parser`.
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack.tools import tool
+from haystack_integrations.components.generators.vllm import VLLMChatGenerator
+
+@tool
+def weather(city: str) -> str:
+    """Get the weather in a given city."""
+    return f"The weather in {city} is sunny"
+
+generator = VLLMChatGenerator(model="Qwen/Qwen3-0.6B", tools=[weather])
+
+messages = [ChatMessage.from_user("What is the weather in Paris?")]
+response = generator.run(messages=messages)
+print(response["replies"][0].tool_calls)
+```
+
+### Usage example with reasoning models
+
+To use reasoning models, start the vLLM server with `--reasoning-parser`.
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.vllm import VLLMChatGenerator
+
+generator = VLLMChatGenerator(model="Qwen/Qwen3-0.6B")
+
+messages = [ChatMessage.from_user("Solve step by step: what is 15 * 37?")]
+response = generator.run(messages=messages)
+reply = response["replies"][0]
+if reply.reasoning:
+    print("Reasoning:", reply.reasoning.reasoning_text)
+print("Answer:", reply.text)
+```
+
+#### __init__
+
+```python
+__init__(
+    *,
+    model: str,
+    api_key: Secret | None = Secret.from_env_var("VLLM_API_KEY", strict=False),
+    streaming_callback: StreamingCallbackT | None = None,
+    api_base_url: str = "http://localhost:8000/v1",
+    generation_kwargs: dict[str, Any] | None = None,
+    timeout: float | None = None,
+    max_retries: int | None = None,
+    tools: ToolsType | None = None,
+    http_client_kwargs: dict[str, Any] | None = None
+) -> None
+```
+
+Creates an instance of VLLMChatGenerator.
+
+**Parameters:**
+
+- **model** (<code>str</code>) – The name of the model served by vLLM (e.g., "Qwen/Qwen3-0.6B").
+- **api_key** (<code>Secret | None</code>) – The vLLM API key. Defaults to the `VLLM_API_KEY` environment variable.
+  Only required if the vLLM server was started with `--api-key`.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+  The callback function accepts
+  [StreamingChunk](https://docs.haystack.deepset.ai/docs/data-classes#streamingchunk)
+  as an argument.
+- **api_base_url** (<code>str</code>) – The base URL of the vLLM server.
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional parameters for text generation. These parameters are sent directly to
+  the vLLM OpenAI-compatible endpoint. See
+  [vLLM documentation](https://docs.vllm.ai/en/stable/serving/openai_compatible_server/)
+  for more details.
+  Some of the supported parameters:
+- `max_tokens`: Maximum number of tokens to generate.
+- `temperature`: Sampling temperature.
+- `top_p`: Nucleus sampling parameter.
+- `n`: Number of completions to generate for each prompt.
+- `stop`: One or more sequences after which the model should stop generating tokens.
+- `response_format`: A JSON schema or a Pydantic model that enforces the structure of the response.
+- `extra_body`: A dictionary of vLLM-specific parameters not part of the standard OpenAI API
+  (e.g., `top_k`, `min_tokens`, `repetition_penalty`).
+- **timeout** (<code>float | None</code>) – Timeout for vLLM client calls. If not set, it defaults to the default set by the OpenAI client.
+- **max_retries** (<code>int | None</code>) – Maximum number of retries to attempt for failed requests. If not set, it defaults to the default
+  set by the OpenAI client.
+- **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
+  Each tool should have a unique name. Not all models support tools.
+- **http_client_kwargs** (<code>dict\[str, Any\] | None</code>) – A dictionary of keyword arguments to configure a custom `httpx.Client` or `httpx.AsyncClient`.
+  For more information, see the [HTTPX documentation](https://www.python-httpx.org/api/#client).
+
+#### warm_up
+
+```python
+warm_up() -> None
+```
+
+Create the OpenAI clients and warm up tools.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serialize this component to a dictionary.
+
+**Returns:**
+
+- <code>dict\[str, Any\]</code> – The serialized component as a dictionary.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> VLLMChatGenerator
+```
+
+Deserialize this component from a dictionary.
+
+**Parameters:**
+
+- **data** (<code>dict\[str, Any\]</code>) – The dictionary representation of this component.
+
+**Returns:**
+
+- <code>VLLMChatGenerator</code> – The deserialized component instance.
+
+#### run
+
+```python
+run(
+    messages: list[ChatMessage] | str,
+    streaming_callback: StreamingCallbackT | None = None,
+    generation_kwargs: dict[str, Any] | None = None,
+    *,
+    tools: ToolsType | None = None
+) -> dict[str, list[ChatMessage]]
+```
+
+Run the VLLM chat generator on the given input data.
+
+**Parameters:**
+
+- **messages** (<code>list\[ChatMessage\] | str</code>) – A list of ChatMessage instances representing the input messages.
+  If a string is provided, it is converted to a list containing a ChatMessage with user role.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional keyword arguments for text generation. These parameters will
+  override the parameters passed during component initialization.
+  For details on vLLM API parameters, see
+  [vLLM documentation](https://docs.vllm.ai/en/stable/serving/openai_compatible_server/).
+- **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
+  If set, it will override the `tools` parameter provided during initialization.
+
+**Returns:**
+
+- <code>dict\[str, list\[ChatMessage\]\]</code> – A dictionary with the following key:
+- `replies`: A list containing the generated responses as ChatMessage instances.
+
+#### run_async
+
+```python
+run_async(
+    messages: list[ChatMessage] | str,
+    streaming_callback: StreamingCallbackT | None = None,
+    generation_kwargs: dict[str, Any] | None = None,
+    *,
+    tools: ToolsType | None = None
+) -> dict[str, list[ChatMessage]]
+```
+
+Run the VLLM chat generator on the given input data asynchronously.
+
+**Parameters:**
+
+- **messages** (<code>list\[ChatMessage\] | str</code>) – A list of ChatMessage instances representing the input messages.
+  If a string is provided, it is converted to a list containing a ChatMessage with user role.
+- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream.
+  Must be a coroutine.
+- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional keyword arguments for text generation. These parameters will
+  override the parameters passed during component initialization.
+  For details on vLLM API parameters, see
+  [vLLM documentation](https://docs.vllm.ai/en/stable/serving/openai_compatible_server/).
+- **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
+  If set, it will override the `tools` parameter provided during initialization.
+
+**Returns:**
+
+- <code>dict\[str, list\[ChatMessage\]\]</code> – A dictionary with the following key:
+- `replies`: A list containing the generated responses as ChatMessage instances.
+
+## haystack_integrations.components.rankers.vllm.ranker
+
+### VLLMRanker
+
+Ranks Documents based on their similarity to a query using models served with [vLLM](https://docs.vllm.ai/).
+
+It expects a vLLM server to be running and accessible at the `api_base_url` parameter and uses the
+`/rerank` endpoint exposed by vLLM.
+
+### Starting the vLLM server
+
+Before using this component, start a vLLM server with a reranker model:
+
+```bash
+vllm serve BAAI/bge-reranker-base
+```
+
+For details on server options, see the [vLLM CLI docs](https://docs.vllm.ai/en/stable/cli/serve/).
+
+### Usage example
+
+```python
+from haystack import Document
+from haystack_integrations.components.rankers.vllm import VLLMRanker
+
+ranker = VLLMRanker(model="BAAI/bge-reranker-base")
+docs = [
+    Document(content="The capital of Brazil is Brasilia."),
+    Document(content="The capital of France is Paris."),
+]
+result = ranker.run(query="What is the capital of France?", documents=docs)
+print(result["documents"][0].content)
+```
+
+### Usage example with vLLM-specific parameters
+
+Pass vLLM-specific parameters via the `extra_parameters` dictionary. They are merged into the
+request body sent to the `/rerank` endpoint.
+
+```python
+ranker = VLLMRanker(
+    model="BAAI/bge-reranker-base",
+    extra_parameters={"truncate_prompt_tokens": 256},
+)
+```
+
+#### __init__
+
+```python
+__init__(
+    *,
+    model: str,
+    api_key: Secret | None = Secret.from_env_var("VLLM_API_KEY", strict=False),
+    api_base_url: str = "http://localhost:8000/v1",
+    top_k: int | None = None,
+    score_threshold: float | None = None,
+    meta_fields_to_embed: list[str] | None = None,
+    meta_data_separator: str = "\n",
+    http_client_kwargs: dict[str, Any] | None = None,
+    extra_parameters: dict[str, Any] | None = None
+) -> None
+```
+
+Creates an instance of VLLMRanker.
+
+**Parameters:**
+
+- **model** (<code>str</code>) – The name of the reranker model served by vLLM. Check
+  [vLLM documentation](https://docs.vllm.ai/en/stable/models/pooling_models/scoring/#supported-models) for
+  information on supported models.
+- **api_key** (<code>Secret | None</code>) – The vLLM API key. Defaults to the `VLLM_API_KEY` environment variable.
+  Only required if the vLLM server was started with `--api-key`.
+- **api_base_url** (<code>str</code>) – The base URL of the vLLM server.
+- **top_k** (<code>int | None</code>) – The maximum number of Documents to return. If `None`, all documents are returned.
+- **score_threshold** (<code>float | None</code>) – If set, documents with a relevance score below this value are dropped.
+  Applied after `top_k`, so the output may contain fewer than `top_k` documents.
+- **meta_fields_to_embed** (<code>list\[str\] | None</code>) – List of meta fields that should be concatenated with the document
+  content before reranking.
+- **meta_data_separator** (<code>str</code>) – Separator used to concatenate the meta fields to the document content.
+- **http_client_kwargs** (<code>dict\[str, Any\] | None</code>) – A dictionary of keyword arguments to configure a custom `httpx.Client` or
+  `httpx.AsyncClient`. For more information, see the
+  [HTTPX documentation](https://www.python-httpx.org/api/#client).
+- **extra_parameters** (<code>dict\[str, Any\] | None</code>) – Additional parameters merged into the request body sent to the vLLM
+  `/rerank` endpoint. Use this to pass parameters not part of the standard rerank API, such as
+  `truncate_prompt_tokens`. See the
+  [vLLM docs](https://docs.vllm.ai/en/stable/models/pooling_models/scoring/#rerank-api) for more information.
+
+**Raises:**
+
+- <code>ValueError</code> – If `top_k` is not > 0.
+
+#### warm_up
+
+```python
+warm_up() -> None
+```
+
+Create the httpx clients.
+
+#### run
+
+```python
+run(
+    query: str,
+    documents: list[Document],
+    top_k: int | None = None,
+    score_threshold: float | None = None,
+) -> dict[str, list[Document] | dict[str, Any]]
+```
+
+Returns a list of Documents ranked by their similarity to the given query.
+
+**Parameters:**
+
+- **query** (<code>str</code>) – Query string.
+- **documents** (<code>list\[Document\]</code>) – List of Documents to rank.
+- **top_k** (<code>int | None</code>) – The maximum number of Documents to return. Overrides the value set at initialization.
+- **score_threshold** (<code>float | None</code>) – Minimum relevance score required for a document to be returned. Overrides
+  the value set at initialization.
+
+**Returns:**
+
+- <code>dict\[str, list\[Document\] | dict\[str, Any\]\]</code> – A dictionary with:
+- `documents`: Documents sorted from most to least relevant.
+- `meta`: Information about the model and usage.
+
+**Raises:**
+
+- <code>ValueError</code> – If `top_k` is not > 0.
+
+#### run_async
+
+```python
+run_async(
+    query: str,
+    documents: list[Document],
+    top_k: int | None = None,
+    score_threshold: float | None = None,
+) -> dict[str, list[Document] | dict[str, Any]]
+```
+
+Asynchronously returns a list of Documents ranked by their similarity to the given query.
+
+**Parameters:**
+
+- **query** (<code>str</code>) – Query string.
+- **documents** (<code>list\[Document\]</code>) – List of Documents to rank.
+- **top_k** (<code>int | None</code>) – The maximum number of Documents to return. Overrides the value set at initialization.
+- **score_threshold** (<code>float | None</code>) – Minimum relevance score required for a document to be returned. Overrides
+  the value set at initialization.
+
+**Returns:**
+
+- <code>dict\[str, list\[Document\] | dict\[str, Any\]\]</code> – A dictionary with:
+- `documents`: Documents sorted from most to least relevant.
+- `meta`: Information about the model and usage.
+
+**Raises:**
+
+- <code>ValueError</code> – If `top_k` is not > 0.