chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add run_async for `OpenAITextEmbedder`.
|
||||
fixes:
|
||||
- |
|
||||
`OpenAITextEmbedder` no longer replaces newlines with spaces in the text to embed. This was only required for the
|
||||
discontinued v1 embedding models.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
fixes:
|
||||
- |
|
||||
Forward declaration of `AnalyzeResult` type in `AzureOCRDocumentConverter`.
|
||||
|
||||
`AnalyzeResult` is already imported in a lazy import block.
|
||||
The forward declaration avoids issues when `azure-ai-formrecognizer>=3.2.0b2` is not installed.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
upgrade:
|
||||
- |
|
||||
``AsyncPipeline`` has been removed. Its asynchronous capabilities are now part of the single ``Pipeline``
|
||||
class, which provides both a synchronous ``run`` method and the asynchronous ``run_async``,
|
||||
``run_async_generator``, and ``stream`` methods (mirroring how components expose ``run`` and ``run_async``).
|
||||
|
||||
To migrate, replace ``AsyncPipeline`` with ``Pipeline``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Before
|
||||
from haystack import AsyncPipeline
|
||||
pipeline = AsyncPipeline()
|
||||
result = await pipeline.run_async(data)
|
||||
|
||||
# After
|
||||
from haystack import Pipeline
|
||||
pipeline = Pipeline()
|
||||
result = await pipeline.run_async(data)
|
||||
|
||||
Serialized pipelines, ``SuperComponent``s, and ``PipelineTool``s are loaded as ``Pipeline`` instances.
|
||||
|
||||
``Pipeline.run`` runs synchronously and blocks until completion; in an async context use
|
||||
``await pipeline.run_async(...)``. The former ``AsyncPipeline.run`` was a synchronous wrapper around the
|
||||
concurrent engine, so to preserve that behavior use ``asyncio.run(pipeline.run_async(...))``. Note that
|
||||
``Pipeline.run`` executes components sequentially and does not accept ``concurrency_limit``.
|
||||
|
||||
Both synchronous and asynchronous runs are traced under a single ``haystack.pipeline.run`` operation
|
||||
name, distinguished by a ``haystack.pipeline.execution_mode`` tag (``sync`` or ``async``). Previously,
|
||||
asynchronous runs used the ``haystack.async_pipeline.run`` operation name.
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
---
|
||||
fixes:
|
||||
- |
|
||||
Fixed a bug in ``NamedEntityExtractor`` where the spaCy/Thinc device state was not correctly
|
||||
restored after execution, potentially affecting the device configuration of other spaCy
|
||||
components in the same process.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
security:
|
||||
- |
|
||||
Made QUOTE_SPANS_RE regex ReDoS-safe. This prevents potential catastrophic backtracking on malicious inputs
|
||||
fixes:
|
||||
- |
|
||||
Fixed a potential ReDoS issue in QUOTE_SPANS_RE regex used inside the SentenceSplitter component.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Adapts how ChatPromptBuilder creates ChatMessages. Messages are deep copied to ensure all meta fields are copied correctly.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Adapt GPTGenerator to use strings for input and output
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add CohereGenerator compatible with Cohere generate endpoint
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add MarkdownToTextDocument, a file converter that converts Markdown files into a text Documents.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add HuggingFace TEI Embedders - `HuggingFaceTEITextEmbedder` and `HuggingFaceTEIDocumentEmbedder`.
|
||||
|
||||
An example using `HuggingFaceTEITextEmbedder` to embed a string:
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceTEITextEmbedder
|
||||
text_to_embed = "I love pizza!"
|
||||
text_embedder = HuggingFaceTEITextEmbedder(
|
||||
model="BAAI/bge-small-en-v1.5", url="<your-tei-endpoint-url>", token="<your-token>"
|
||||
)
|
||||
print(text_embedder.run(text_to_embed))
|
||||
# {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
```
|
||||
|
||||
An example using `HuggingFaceTEIDocumentEmbedder` to create Document embeddings:
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack.components.embedders import HuggingFaceTEIDocumentEmbedder
|
||||
doc = Document(content="I love pizza!")
|
||||
document_embedder = HuggingFaceTEIDocumentEmbedder(
|
||||
model="BAAI/bge-small-en-v1.5", url="<your-tei-endpoint-url>", token="<your-token>"
|
||||
)
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
# [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added an ``after_tool`` hook point to the ``Agent``. Hooks registered under ``after_tool`` run after tools execute,
|
||||
once their result messages are in ``state.data["messages"]``, and before the exit check and the next LLM call. Use it to
|
||||
rewrite the freshly produced tool-result messages, for example to offload, redact, truncate, or summarize results
|
||||
before the next LLM call sees them. It does not run on the plain-text exit step, where no tools run. Register it
|
||||
like any other hook: ``hooks={"after_tool": [my_hook]}``.
|
||||
@@ -0,0 +1,129 @@
|
||||
---
|
||||
upgrade:
|
||||
- |
|
||||
``continue_run`` is now a reserved key in ``Agent.state_schema`` (set by an ``on_exit`` hook to keep the Agent
|
||||
running). Passing it via the ``state_schema`` argument now raises ``ValueError``. Rename any conflicting state
|
||||
key (e.g. ``my_continue_run``) to migrate.
|
||||
features:
|
||||
- |
|
||||
Added hooks to the ``Agent``. Pass ``hooks`` as a dictionary mapping a hook point to a list of hooks the Agent runs
|
||||
at that point. Each hook receives the live ``State`` and influences the run by mutating it in place; hooks for a
|
||||
hook point run in list order, and the same hook can be registered under multiple hook points. Valid hook points are:
|
||||
- ``before_llm``: Runs before each chat-generator call.
|
||||
- ``before_tool``: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent
|
||||
re-reads the current last message from ``state["messages"]``. If that message contains tool calls, those calls are
|
||||
executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent
|
||||
loops back to the next LLM call unless ``max_agent_steps`` has been reached.
|
||||
- ``on_exit``: Runs when the Agent is about to stop on an exit condition. An ``on_exit`` hook can keep the Agent
|
||||
running by setting the ``continue_run`` control flag (``state.set("continue_run", True)``), usually alongside a
|
||||
message telling the model what to do next. ``on_exit`` hooks run when the Agent stops on an exit condition, but
|
||||
not when it stops because ``max_agent_steps`` is reached.
|
||||
|
||||
A hook is any object with a ``run(state)`` method (optionally ``run_async(state)``); use the ``@hook`` decorator
|
||||
to build one from a function. This enables patterns such as building run-time system context, retrieving memories,
|
||||
and requiring a tool to be called before finishing.
|
||||
|
||||
The example below registers one hook at each placement to show what each can do:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import Annotated
|
||||
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.agents.state import State, replace_values
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.hooks import hook
|
||||
from haystack.tools import tool
|
||||
|
||||
|
||||
@tool
|
||||
def search(query: Annotated[str, "The search query"]) -> str:
|
||||
"""Search the web."""
|
||||
# Placeholder: would call a real search API
|
||||
return "Fusion startups reported net-energy-gain milestones this year."
|
||||
|
||||
|
||||
@hook
|
||||
def build_context(state: State) -> None:
|
||||
# before_llm: build run-time system context once, before the first model call.
|
||||
if state.get("step_count") == 0:
|
||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
system = ChatMessage.from_system(f"You are a research assistant. The current time is {now}.")
|
||||
state.set("messages", [system, *state.data["messages"]], handler_override=replace_values)
|
||||
|
||||
|
||||
@hook
|
||||
def audit_tool_calls(state: State) -> None:
|
||||
# before_tool: see which tools the model is about to run.
|
||||
pending = state.data["messages"][-1].tool_calls
|
||||
print(f"about to run: {[tc.tool_name for tc in pending]}")
|
||||
|
||||
|
||||
@hook
|
||||
def require_search(state: State) -> None:
|
||||
# on_exit: keep going until the agent has actually searched.
|
||||
if state.get("tool_call_counts", {}).get("search", 0) == 0:
|
||||
state.set("messages", [ChatMessage.from_system("Search before answering.")])
|
||||
state.set("continue_run", True)
|
||||
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
|
||||
tools=[search],
|
||||
hooks={
|
||||
"before_llm": [build_context],
|
||||
"before_tool": [audit_tool_calls],
|
||||
"on_exit": [require_search],
|
||||
},
|
||||
)
|
||||
|
||||
result = agent.run(messages=[ChatMessage.from_user("What are the latest developments in fusion energy?")])
|
||||
print(result["last_message"].text)
|
||||
|
||||
Class-based hooks may also implement the optional lifecycle methods ``warm_up`` / ``warm_up_async`` and
|
||||
``close`` / ``close_async``. The Agent calls them from its own ``warm_up`` / ``warm_up_async`` and
|
||||
``close`` / ``close_async``, so a hook can defer opening clients or reading credentials until warm-up and release
|
||||
them on close. When a class-based hook should be serializable, store serializable constructor arguments on the hook
|
||||
and rebuild runtime clients from those values. For example, an ``on_exit`` hook can grade the Agent's answer with
|
||||
its own LLM and ask it to improve a weak answer before finishing, warming the judge's client during the Agent's
|
||||
warm-up:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from typing import Any
|
||||
from haystack.core.serialization import default_from_dict, default_to_dict
|
||||
|
||||
|
||||
class GradeFinalAnswer:
|
||||
"""Grade the Agent's answer with an LLM and ask it to improve a weak answer before finishing."""
|
||||
|
||||
def __init__(self, model: str = "gpt-5.4-nano"):
|
||||
self.model = model
|
||||
self._judge = OpenAIChatGenerator(model=self.model)
|
||||
|
||||
def warm_up(self) -> None:
|
||||
# Warm up the judge's own client during the Agent's warm-up.
|
||||
self._judge.warm_up()
|
||||
|
||||
def run(self, state: State) -> None:
|
||||
answer = state.data["messages"][-1].text or ""
|
||||
verdict = self._judge.run(
|
||||
messages=[ChatMessage.from_user(f"Reply with only PASS or FAIL. Is this answer complete?\n\n{answer}")]
|
||||
)["replies"][0].text or ""
|
||||
if "FAIL" in verdict.upper():
|
||||
state.set("messages", [ChatMessage.from_user("Your answer was incomplete. Please improve it.")])
|
||||
state.set("continue_run", True)
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return default_to_dict(self, model=self.model)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any]) -> "GradeFinalAnswer":
|
||||
return default_from_dict(cls, data)
|
||||
|
||||
|
||||
agent = Agent(chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"), hooks={"on_exit": [GradeFinalAnswer()]})
|
||||
result = agent.run(messages=[ChatMessage.from_user("Explain how vaccines work.")])
|
||||
print(result["last_message"].text)
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add the `AnswerBuilder` component for Haystack 2.0 that creates Answer objects from the string output of Generators.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added support for Apple Silicon GPU acceleration through "mps pytorch", enabling better performance on Apple M1 hardware.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added the apply_filter_policy function to standardize the application of filter policies across all document store-specific retrievers, allowing for consistent handling of initial and runtime filters based on the chosen policy (replace or merge).
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add `run_async` method to HuggingFaceAPIDocumentEmbedder. This method enriches Documents with embeddings.
|
||||
It supports the same parameters as the `run` method. It returns a coroutine that can be awaited.
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add native async tool invocation. ``Tool`` now accepts an optional ``async_function`` (a coroutine function) alongside the existing ``function`` field.
|
||||
When ``Agent.run_async`` invokes a tool, it now awaits the ``async_function`` of Tool if available.
|
||||
|
||||
``@tool`` / ``create_tool_from_function`` automatically routes ``async def`` callables to ``async_function``, so simply decorating an ``async def`` produces an async tool:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from typing import Annotated
|
||||
from haystack.tools import tool
|
||||
|
||||
@tool
|
||||
async def weather(city: Annotated[str, "The name of the city"]) -> str:
|
||||
"""Get the weather for a city."""
|
||||
...
|
||||
|
||||
``ComponentTool`` automatically wires an async invoker for components that define ``run_async`` (``__haystack_supports_async__`` is ``True``).
|
||||
``PipelineTool`` enables the async path when wrapping an ``AsyncPipeline`` and falls back to the threaded sync path when wrapping a regular ``Pipeline``.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added component_name and component_type attributes to PipelineRuntimeError.
|
||||
- Moved error message creation to within PipelineRuntimeError
|
||||
- Created a new subclass of PipelineRuntimeError called PipelineComponentsBlockedError for the specific case where the pipeline cannot run since no components are unblocked.
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Pipelines now natively support connecting multiple outputs directly to a single component input without requiring
|
||||
an explicit Joiner component. This only works when the connected outputs and inputs are of compatible list types,
|
||||
such as ``list[Document]``.
|
||||
|
||||
This simplifies pipeline definitions when multiple components produce compatible outputs.
|
||||
For example, multiple outputs from a ``FileTypeRouter`` can now be connected directly to a single converter or writer, without defining an intermediate ``ListJoiner`` or ``DocumentJoiner``.
|
||||
|
||||
.. code:: python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters import HTMLToDocument, TextFileToDocument
|
||||
from haystack.components.routers import FileTypeRouter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.dataclasses import ByteStream
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
sources = [
|
||||
ByteStream.from_string(text="Text file content", mime_type="text/plain", meta={"file_type": "txt"}),
|
||||
ByteStream.from_string(
|
||||
text="\n<html><body>Some content</body></html>\n", mime_type="text/html", meta={"file_type": "html"},
|
||||
),
|
||||
]
|
||||
|
||||
doc_store = InMemoryDocumentStore()
|
||||
pipe = Pipeline()
|
||||
|
||||
pipe.add_component("router", FileTypeRouter(mime_types=["text/plain", "text/html"]))
|
||||
pipe.add_component("txt_converter", TextFileToDocument())
|
||||
pipe.add_component("html_converter", HTMLToDocument())
|
||||
pipe.add_component("writer", DocumentWriter(doc_store))
|
||||
|
||||
pipe.connect("router.text/plain", "txt_converter.sources")
|
||||
pipe.connect("router.text/html", "html_converter.sources")
|
||||
# The DocumentWriter accepts documents from both converters without needing a DocumentJoiner
|
||||
pipe.connect("txt_converter.documents", "writer.documents")
|
||||
pipe.connect("html_converter.documents", "writer.documents")
|
||||
|
||||
result = pipe.run({"router": {"sources": sources}})
|
||||
# result["writer"]["documents_written"] == 2
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Improved AzureDocumentEmbedder to handle embedding generation failures gracefully.
|
||||
Errors are logged, and processing continues with the remaining batches.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Adds AzureOpenAIDocumentEmbedder and AzureOpenAITextEmbedder as new embedders. These embedders are very similar to
|
||||
their OpenAI counterparts, but they use the Azure API instead of the OpenAI API.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Adds support for Azure OpenAI models with AzureOpenAIGenerator and AzureOpenAIChatGenerator components.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Expose default_headers to pass custom headers to Azure API including APIM subscription key.
|
||||
- |
|
||||
Add optional azure_kwargs dictionary parameter to pass in parameters undefined in Haystack but supported by AzureOpenAI.
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added the `AzureOpenAIResponsesChatGenerator`, a new component that integrates Azure OpenAI's Responses API into Haystack.
|
||||
This unlocks several advanced capabilities from the Responses API:
|
||||
- Allowing retrieval of concise summaries of the model's reasoning process.
|
||||
- Allowing the use of native OpenAI or MCP tool formats, along with Haystack Tool objects and Toolset instances.
|
||||
|
||||
Example with reasoning and web search tool:
|
||||
```python
|
||||
from haystack.components.generators.chat import AzureOpenAIResponsesChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
chat_generator = AzureOpenAIResponsesChatGenerator(
|
||||
azure_endpoint="https://example-resource.azure.openai.com/",
|
||||
azure_deployment="gpt-5-mini",
|
||||
generation_kwargs={"reasoning": {"effort": "low", "summary": "auto"}},
|
||||
)
|
||||
|
||||
response = chat_generator.run(
|
||||
messages=[ChatMessage.from_user("What's Natural Language Processing?")]
|
||||
)
|
||||
print(response["replies"][0].text)
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add AzureOCRDocumentConverter to convert files of different types using Azure's Document Intelligence Service.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Treat bare types (e.g., List, Dict) as generic types with Any arguments during type compatibility checks.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Add batch_size to the __init__ method of FAISS Document Store. This works as the default value for all methods of
|
||||
FAISS Document Store that support batch_size.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add ByteStream type to send binary raw data across components
|
||||
in a pipeline.
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
highlights: >
|
||||
The `Multiplexer` component proved to be hard to explain and to understand. After reviewing its use cases, the documentation
|
||||
was rewritten and the component was renamed to `BranchJoiner` to better explain its functionalities.
|
||||
upgrade:
|
||||
- |
|
||||
`BranchJoiner` has the very same interface as `Multiplexer`. To upgrade your code, just rename any occurrence
|
||||
of `Multiplexer` to `BranchJoiner` and ajdust the imports accordingly.
|
||||
features:
|
||||
- |
|
||||
Add `BranchJoiner` to eventually replace `Multiplexer`
|
||||
deprecations:
|
||||
- |
|
||||
`Mulitplexer` is now deprecated.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Add `guess_mime_type` parameter to `Bytestream.from_file_path()`
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- Add support for ByteStream objects in MetadataRouter.
|
||||
It can now be used to route `list[Documents]` or `list[ByteStream]` based on metadata.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Adds `calculate_metrics()` function to EvaluationResult for computation of evaluation metrics.
|
||||
Adds `Metric` class to store list of available metrics.
|
||||
Adds `MetricsResult` class to store the metric values computed during the evaluation.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add compatibility for Callable types.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Introduce ChatMessage data class to facilitate structured handling and processing of message content
|
||||
within LLM chat interactions.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
fixes:
|
||||
- |
|
||||
In the `ChatMessage.to_openai_dict_format` utility method,
|
||||
include the `name` field in the returned dictionary, if present.
|
||||
Previously, the `name` field was erroneously skipped.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
`ChatPromptBuilder` now supports changing its template at runtime. This allows you to define a default template and then change it based on your needs at runtime.
|
||||
deprecations:
|
||||
- |
|
||||
`DynamicChatPromptBuilder` has been deprecated as `ChatPromptBuilder` fully covers its functionality. Use `ChatPromptBuilder` instead.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
- Add a `ComponentInfo` dataclass to the `haystack.dataclasses` module.
|
||||
This dataclass is used to store information about the component. We pass it to `StreamingChunk` so we can tell from which component a stream is coming from.
|
||||
|
||||
- Pass the `component_info` to the `StreamingChunk` in the `OpenAIChatGenerator`, `AzureOpenAIChatGenerator`, `HuggingFaceAPIChatGenerator` and `HuggingFaceLocalChatGenerator`.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
highlights: |
|
||||
Introduced ComponentTool, a powerful addition to the Haystack tooling architecture that enables any Haystack component to be used as a tool by LLMs.
|
||||
ComponentTool bridges the gap between Haystack's component ecosystem and LLM tool/function calling capabilities, allowing LLMs to
|
||||
directly interact with components like web search, document processing, or any custom user component. ComponentTool handles
|
||||
all the complexity of schema generation and type conversion, making it easy to expose component functionality to LLMs.
|
||||
|
||||
features:
|
||||
- |
|
||||
Introduced the ComponentTool, a new tool that wraps Haystack components allowing them to be utilized as tools for LLMs (various ChatGenerators).
|
||||
This ComponentTool supports automatic tool schema generation, input type conversion, and offering support for components with run methods that have input types:
|
||||
- Basic types (str, int, float, bool, dict)
|
||||
- Dataclasses (both simple and nested structures)
|
||||
- Lists of basic types (e.g., List[str])
|
||||
- Lists of dataclasses (e.g., List[Document])
|
||||
- Parameters with mixed types (e.g., List[Document], str etc.)
|
||||
|
||||
Example usage:
|
||||
```python
|
||||
from haystack.components.websearch import SerperDevWebSearch
|
||||
from haystack.tools import ComponentTool
|
||||
from haystack.utils import Secret
|
||||
|
||||
# Create a SerperDev search component
|
||||
search = SerperDevWebSearch(
|
||||
api_key=Secret.from_token("your-api-key"),
|
||||
top_k=3
|
||||
)
|
||||
|
||||
# Create a tool from the component
|
||||
tool = ComponentTool(
|
||||
component=search,
|
||||
name="web_search", # Optional: defaults to "serper_dev_web_search"
|
||||
description="Search the web for current information" # Optional: defaults to component docstring
|
||||
)
|
||||
|
||||
# You can now use the tool now in a pipeline, see docs for more examples
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add a CSV to Document converter component. Loads the file as bytes object. Adds the loaded string as a new document that can be used for further processing by the Document Splitter.
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Allow the ability to add the current date inside a template in `PromptBuilder` using the following syntax:
|
||||
|
||||
- `{% now 'UTC' %}`: Get the current date for the UTC timezone.
|
||||
|
||||
- `{% now 'America/Chicago' + 'hours=2' %}`: Add two hours to the current date in the Chicago timezone.
|
||||
|
||||
- `{% now 'Europe/Berlin' - 'weeks=2' %}`: Subtract two weeks from the current date in the Berlin timezone.
|
||||
|
||||
- `{% now 'Pacific/Fiji' + 'hours=2', '%H' %}`: Display only the number of hours after adding two hours to the Fiji timezone.
|
||||
|
||||
- `{% now 'Etc/GMT-4', '%I:%M %p' %}`: Change the date format to AM/PM for the GMT-4 timezone.
|
||||
|
||||
Note that if no date format is provided, the default will be `%Y-%m-%d %H:%M:%S`. Please refer to [list of tz database](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for a list of timezones.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add callable hook to PyPDFToDocument to enable easier customization of pdf to Document conversion.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added custom filters support to ConditionalRouter. Users can now pass in
|
||||
one or more custom Jinja2 filter callables and be able to access those
|
||||
filters when defining condition expressions in routes.
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
We've added a new **DALLEImageGenerator** component, bringing image generation with OpenAI's DALL-E to the Haystack
|
||||
|
||||
- **Easy to Use**: Just a few lines of code to get started:
|
||||
```python
|
||||
from haystack.components.generators import DALLEImageGenerator
|
||||
image_generator = DALLEImageGenerator()
|
||||
response = image_generator.run("Show me a picture of a black cat.")
|
||||
print(response)
|
||||
```
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
fixes:
|
||||
- |
|
||||
Add dataframe to legacy fields for the Document dataclass.
|
||||
This fixes a bug where Document.from_dict() in haystack-ai>=2.11.0 could not properly deserialize a Document
|
||||
dictionary obtained with document.to_dict(flatten=False) in haystack-ai<=2.10.0.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added `default_headers` parameter to `AzureOpenAIDocumentEmbedder` and `AzureOpenAITextEmbedder`.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Add a field called `default_value` to the `InputSocket` dataclass.
|
||||
Derive `is_mandatory` value from the presence of `default_value`.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
deprecations:
|
||||
- |
|
||||
The output of the ContextRelevanceEvaluator will change in Haystack 2.4.0. Contexts will be scored as a whole instead of individual statements and only the relevant sentences will be returned. A score of 1 is now returned if a relevant sentence is found, and 0 otherwise.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
add dimensions parameter to Azure OpenAI Embedders (AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder) to fully support new embedding models like text-embedding-3-small, text-embedding-3-large and upcoming ones
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
add dimensions parameter to OpenAI Embedders to fully support new embedding models like text-embedding-3-small, text-embedding-3-large and upcoming ones
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added a new mode in JoinDocuments,
|
||||
Distribution-based rank fusion as
|
||||
[the article](https://medium.com/plain-simple-software/distribution-based-score-fusion-dbsf-a-new-approach-to-vector-search-ranking-f87c37488b18)
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add `SentenceTransformersDiversityRanker`.
|
||||
The Diversity Ranker orders documents in such a way as to maximize the overall diversity of the given documents.
|
||||
The ranker leverages sentence-transformer models to calculate semantic embeddings for each document and the query.
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
upgrade:
|
||||
- |
|
||||
``DocumentNDCGEvaluator`` now matches documents by their ``content`` field
|
||||
by default instead of their auto-generated ``id``. Previously, ground
|
||||
truth and retrieved documents were matched only if they had identical
|
||||
``id`` values, which rarely happened in practice since IDs are generated
|
||||
independently for each Document instance. As a result, NDCG scores
|
||||
computed with this evaluator may change for existing pipelines. To keep
|
||||
the previous ``id``-based matching behavior, pass
|
||||
``document_comparison_field="id"`` when constructing the evaluator.
|
||||
enhancements:
|
||||
- |
|
||||
Added ``document_comparison_field`` parameter to ``DocumentNDCGEvaluator``,
|
||||
consistent with ``DocumentMAPEvaluator``, ``DocumentMRREvaluator``, and
|
||||
``DocumentRecallEvaluator``. Users can now match documents by ``"content"``,
|
||||
``"id"``, or any ``"meta.<key>"`` field when calculating NDCG scores.
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Document writer returns the number of documents written.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
highlights: >
|
||||
Adding the `DocxToDocument` component to convert Docx files to Documents.
|
||||
features:
|
||||
- |
|
||||
Adding the `DocxToDocument` component inside the `converters` category. It uses the `python-docx` library to convert Docx files to haystack Documents.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Adds `ChatMessage` templating in `PromptBuilder`
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Add `DynamicPromptBuilder` to dynamically generate prompts from either a list of ChatMessage instances or a string
|
||||
template, leveraging Jinja2 templating for flexible and efficient prompt construction.
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added the `enable_streaming_callback_passthrough` to the `ToolInovker` init, run and run_async methods. If set to True the ToolInvoker will try and pass the `streaming_callback` function to a tool's invoke method only if the tool's invoke method has `streaming_callback` in its signature.
|
||||
fixes:
|
||||
- |
|
||||
Fixed the `to_dict` and `from_dict` of `ToolInvoker` to properly serialize the `streaming_callback` init parameter.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Enhanced `SentenceTransformersDocumentEmbedder` and `SentenceTransformersTextEmbedder` to accept
|
||||
an additional parameter, which is passed directly to the underlying `SentenceTransformer.encode` method
|
||||
for greater flexibility in embedding customization.
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
---
|
||||
fixes:
|
||||
- |
|
||||
Add encoding format keyword argument to OpenAI client when creating embeddings.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Updated our serialization and deserialization of PipelineSnapshots to work with python Enum classes.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Improved error handling for component `run` failures by raising a runtime error that includes the component's name and type.
|
||||
@@ -0,0 +1,4 @@
|
||||
preview:
|
||||
- |
|
||||
Add eval function for evaluation of components and Pipelines.
|
||||
Adds EvaluationResult to store results of evaluation.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Adds support for the Exact Match metric to `EvaluationResult.calculate_metrics(...)`:
|
||||
```python
|
||||
from haystack.evaluation.metrics import Metric
|
||||
exact_match_metric = eval_result.calculate_metrics(Metric.EM, output_key="answers")
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add XLSXToDocument converter that loads an Excel file using Pandas + openpyxl and by default converts each sheet into a separate Document in a CSV format.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add an `extra` field to `ToolCall` and `ToolCallDelta` to store provider-specific information.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Adds support for the F1 metric to `EvaluationResult.calculate_metrics(...)`:
|
||||
```python
|
||||
from haystack.evaluation.metrics import Metric
|
||||
f1_metric = eval_result.calculate_metrics(Metric.F1, output_key="answers")
|
||||
```
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
If an LLM-based evaluator (e.g., `Faithfulness` or `ContextRelevance`) is initialised with `raise_on_failure=False`, and if a call to an LLM fails or an LLM outputs an invalid JSON, the score of the sample is set to `NaN` instead of raising an exception.
|
||||
The user is notified with a warning indicating the number of requests that failed.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
highlights: >
|
||||
Introduced `FallbackChatGenerator` that tries multiple chat providers one by one, improving reliability in production and making sure you get answers even when some provider fails.
|
||||
features:
|
||||
- |
|
||||
Added `FallbackChatGenerator` that automatically retries different chat generators and returns first successful response with detailed information about which providers were tried.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Adds FileExtensionClassifier to preview components.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add FilterRetriever.
|
||||
It retrieves documents that match the provided (either at init or runtime) filters.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added dedicated `finish_reason` field to `StreamingChunk` class to improve type safety and enable sophisticated streaming UI logic. The field uses a `FinishReason` type alias with standard values: "stop", "length", "tool_calls", "content_filter", plus Haystack-specific value "tool_call_results" (used by ToolInvoker to indicate tool execution completion).
|
||||
- |
|
||||
Updated `ToolInvoker` component to use the new `finish_reason` field when streaming tool results. The component now sets `finish_reason="tool_call_results"` in the final streaming chunk to indicate that tool execution has completed, while maintaining backward compatibility by also setting the value in `meta["finish_reason"]`.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added ``haystack.component.fully_qualified_type`` field to component tracing output.
|
||||
This new field provides the full module path and class name (e.g.,
|
||||
``haystack.components.generators.chat.openai.OpenAIChatGenerator``) alongside the
|
||||
existing ``haystack.component.type`` field that only contains the class name.
|
||||
This enables dynamic component loading and better tooling integration.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Adds `generation_kwargs` to the `Agent` component, allowing for more fine-grained control at run-time over the chat generation.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Add async variants of metadata methods to ``InMemoryDocumentStore``:
|
||||
``get_metadata_fields_info_async()``, ``get_metadata_field_min_max_async()``, and
|
||||
``get_metadata_field_unique_values_async()``. These rely on the store's thread-pool executor,
|
||||
consistent with the existing async method pattern.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added haystack-experimental to the project's dependencies to enable automatic use of cutting-edge features from Haystack. Users can now access components from haystack-experimental by simply importing them from haystack_experimental instead of haystack. For more information, visit https://github.com/deepset-ai/haystack-experimental.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
fixes:
|
||||
- |
|
||||
Resolves a bug where the HuggingFaceTGIGenerator and HuggingFaceTGIChatGenerator encountered issues if provided
|
||||
with valid models that were not available on the HuggingFace inference API rate-limited tier. The fix, detailed
|
||||
in [GitHub issue #6816](https://github.com/deepset-ai/haystack/issues/6816) and its GitHub PR, ensures these
|
||||
components now correctly handle model availability, eliminating previous limitations.
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added support for human confirmation on ``Agent`` tool calls.
|
||||
You can now configure per-tool confirmation strategies when building an ``Agent``, including always requiring confirmation,
|
||||
never requiring it, or requesting confirmation only on first use.
|
||||
The confirmation experience is customizable through different UI strategies, allowing fine-grained control over how and when users approve tool executions.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-4.1"),
|
||||
tools=[balance_tool, addition_tool, phone_tool],
|
||||
system_prompt=(
|
||||
"You are a helpful financial assistant. "
|
||||
"Use the provided tool to get bank balances when needed."
|
||||
),
|
||||
confirmation_strategies={
|
||||
balance_tool.name: BlockingConfirmationStrategy(
|
||||
confirmation_policy=AlwaysAskPolicy(),
|
||||
confirmation_ui=RichConsoleUI(console=cons),
|
||||
),
|
||||
addition_tool.name: BlockingConfirmationStrategy(
|
||||
confirmation_policy=NeverAskPolicy(),
|
||||
confirmation_ui=SimpleConsoleUI(),
|
||||
),
|
||||
phone_tool.name: BlockingConfirmationStrategy(
|
||||
confirmation_policy=AskOncePolicy(),
|
||||
confirmation_ui=SimpleConsoleUI(),
|
||||
),
|
||||
},
|
||||
)
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Adds HTMLToDocument component to convert HTML to a Document.
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Introducing the HuggingFaceLocalChatGenerator, a new chat-based generator designed for leveraging chat models from
|
||||
Hugging Face's (HF) model hub. Users can now perform inference with chat-based models in a local runtime, utilizing
|
||||
familiar HF generation parameters, stop words, and even employing custom chat templates for custom message formatting.
|
||||
This component also supports streaming responses and is optimized for compatibility with a variety of devices.
|
||||
|
||||
Here is an example of how to use the HuggingFaceLocalChatGenerator:
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import HuggingFaceLocalChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
generator = HuggingFaceLocalChatGenerator(model="HuggingFaceH4/zephyr-7b-beta")
|
||||
generator.warm_up()
|
||||
messages = [ChatMessage.from_user("What's Natural Language Processing? Be brief.")]
|
||||
print(generator.run(messages))
|
||||
```
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added new `HuggingFaceTEIRanker` component to enable reranking with Text Embeddings Inference (TEI) API.
|
||||
This component supports both self-hosted Text Embeddings Inference services and Hugging Face Inference Endpoints.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Adds `HuggingFaceTGIChatGenerator` for text and chat generation. This components support remote inferencing for
|
||||
Hugging Face LLMs via text-generation-inference (TGI) protocol.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Adds `HuggingFaceTGIGenerator` for text generation. This components support remote inferencing for
|
||||
Hugging Face LLMs via text-generation-inference (TGI) protocol.
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added ``link_format`` parameter to ``PPTXToDocument`` and ``XLSXToDocument`` converters,
|
||||
allowing extraction of hyperlink addresses from PPTX and XLSX files.
|
||||
|
||||
Supported formats:
|
||||
|
||||
- ``"markdown"``: ``[text](url)``
|
||||
- ``"plain"``: ``text (url)``
|
||||
- ``"none"`` (default): Only text is extracted, link addresses are ignored.
|
||||
|
||||
This follows the same pattern already available in ``DOCXToDocument``.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added multimodal support to `HuggingFaceAPIChatGenerator` to enable vision-language model (VLM) usage with images and text.
|
||||
Users can now send both text and images to VLM models through Hugging Face APIs. The implementation follows the HF VLM API format
|
||||
specification and maintains full backward compatibility with text-only messages.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add a indexing `build_indexing_pipeline` utility function
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Adds inference mode to model call of the ExtractiveReader. This prevents gradients from being calculated during inference time in pytorch.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
|
||||
enhancements:
|
||||
- |
|
||||
Added a new parameter to `EvaluationRunResult.comparative_individual_scores_report()` to specify columns to keep in the comparative DataFrame.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
The `DocumentCleaner` class has the optional attribute `keep_id` that if set to True it keeps the document ids unchanged after cleanup.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
preview:
|
||||
- |
|
||||
Introduced the LinkContentFetcher in Haystack 2.0. This component fetches content from specified
|
||||
URLs and converts them into ByteStream objects for further processing in Haystack pipelines.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added a new component `ListJoiner` which joins lists of values from different components to a single list.
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added a new ``LLM`` component (``haystack.components.generators.chat.LLM``) that provides a simplified
|
||||
interface for text generation powered by a large language model. The ``LLM`` component is a streamlined
|
||||
version of the ``Agent`` that focuses solely on single-turn text generation without tool usage.
|
||||
It supports system prompts, templated user prompts with required variables, streaming callbacks,
|
||||
and both synchronous (``run``) and asynchronous (``run_async``) execution.
|
||||
|
||||
Usage example:
|
||||
.. code:: python
|
||||
from haystack.components.generators.chat import LLM
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
llm = LLM(
|
||||
chat_generator=OpenAIChatGenerator(),
|
||||
system_prompt="You are a helpful translation assistant.",
|
||||
user_prompt=\"\"\"{% message role="user"%}
|
||||
Summarize the following document: {{ document }}
|
||||
{% endmessage %}\"\"\",
|
||||
required_variables=["document"],
|
||||
)
|
||||
|
||||
result = llm.run(document="The weather is lovely today and the sun is shining. ")
|
||||
print(result["last_message"].text)
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Added the LLMDocumentContentExtractor which extracts textual content from image-based documents using a vision-enabled LLM.
|
||||
This is good for creating a textual representation of an image which can be used when only text-based retrieval is available.
|
||||
@@ -0,0 +1,5 @@
|
||||
features:
|
||||
- |
|
||||
Added ``LLMRanker``, a new ranker component that uses a ``ChatGenerator`` and ``PromptBuilder`` to rerank
|
||||
documents based on JSON-formatted LLM output. ``LLMRanker`` supports configurable prompts, optional custom
|
||||
chat generators, runtime ``top_k`` overrides, and serialization.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
If logprobs are enabled in the generation kwargs, return logprobs in ChatMessage.meta for `OpenAIChatGenerator` and `OpenAIResponsesChatGenerator`.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Add warning logs to the PDFMinerToDocument and PyPDFToDocument to indicate when a processed PDF file has no content.
|
||||
This can happen if the PDF file is a scanned image.
|
||||
Also added an explicit check and warning message to the DocumentSplitter that warns the user that empty Documents are skipped.
|
||||
This behavior was already occurring, but now its clearer through logs that this is happening.
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
prelude: >
|
||||
We're excited to introduce a new ranker to Haystack - LostInTheMiddleRanker.
|
||||
It reorders documents based on the "Lost in the Middle" order, a strategy that
|
||||
places the most relevant paragraphs at the beginning or end of the context,
|
||||
while less relevant paragraphs are positioned in the middle. This ranker,
|
||||
based on the research paper "Lost in the Middle: How Language Models Use Long
|
||||
Contexts" by Liu et al., can be leveraged in Retrieval-Augmented Generation
|
||||
(RAG) pipelines.
|
||||
features:
|
||||
- |
|
||||
The LostInTheMiddleRanker can be used like other rankers in Haystack. After
|
||||
initializing LostInTheMiddleRanker with the desired parameters, it can be
|
||||
used to rank/reorder a list of documents based on the "Lost in the Middle"
|
||||
order - the most relevant documents are located at the top and bottom of
|
||||
the returned list, while the least relevant documents are found in the
|
||||
middle. We advise that you use this ranker in combination with other rankers,
|
||||
and to place it towards the end of the pipeline.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
|
||||
features:
|
||||
- |
|
||||
Add LostInTheMiddleRanker.
|
||||
It reorders documents based on the "Lost in the Middle" order, a strategy that
|
||||
places the most relevant paragraphs at the beginning or end of the context,
|
||||
while less relevant paragraphs are positioned in the middle.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Adds markdown mimetype support to the file type router i.e. `FileTypeRouter` class.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
enhancements:
|
||||
- |
|
||||
Added the Maximum Margin Relevance (MMR) strategy to the `SentenceTransformersDiversityRanker`. MMR scores are calculated for each document based on their relevance to the query and diversity from already selected documents.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Introduced the ``MarkdownHeaderSplitter`` component:
|
||||
- Splits documents into chunks at Markdown headers (``#``, ``##``, etc.), preserving header hierarchy as metadata.
|
||||
- Supports secondary splitting (by word, passage, period, or line) for further chunking after header-based splitting using Haystack's ``DocumentSplitter``.
|
||||
- Preserves and propagates metadata such as parent headers and page numbers.
|
||||
- Handles edge cases such as documents with no headers, empty content, and non-text documents.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user