chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:22:28 +08:00
commit c56bef871b
9296 changed files with 1854228 additions and 0 deletions
@@ -0,0 +1,184 @@
---
title: "LLMDocumentContentExtractor"
id: llmdocumentcontentextractor
slug: "/llmdocumentcontentextractor"
description: "Extracts textual content from image-based documents using a vision-enabled Large Language Model (LLM)."
---
# LLMDocumentContentExtractor
Extracts textual content from image-based documents using a vision-enabled Large Language Model (LLM).
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | After [Converters](../converters.mdx) in an indexing pipeline to extract text from image-based documents |
| **Mandatory init variables** | `chat_generator`: A ChatGenerator instance that supports vision-based input <br /> <br />`prompt`: Instructional text for the LLM on how to extract content (no Jinja variables allowed) |
| **Mandatory run variables** | `documents`: A list of documents with file paths in metadata |
| **Output variables** | `documents`: Successfully processed documents with extracted content <br /> <br />`failed_documents`: Documents that failed processing with error metadata |
| **API reference** | [Extractors](/reference/extractors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/image/llm_document_content_extractor.py |
</div>
## Overview
`LLMDocumentContentExtractor` extracts textual content from image-based documents using a vision-enabled Large Language Model (LLM). This component is particularly useful for processing scanned documents, images containing text, or PDF pages that need to be converted to searchable text.
The component works by:
1. Converting each input document into an image using the `DocumentToImageContent` component,
2. Using a predefined prompt to instruct the LLM on how to extract content,
3. Processing the image through a vision-capable ChatGenerator to extract structured textual content.
The prompt must not contain Jinja variables; it should only include instructions for the LLM. Image data and the prompt are passed together to the LLM as a Chat Message.
Documents for which the LLM fails to extract content are returned in a separate `failed_documents` list with a `content_extraction_error` entry in their metadata for debugging or reprocessing.
## Usage
### On its own
Below is an example that uses the `LLMDocumentContentExtractor` to extract text from image-based documents:
```python
from haystack import Document
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.extractors.image import LLMDocumentContentExtractor
## Initialize the chat generator with vision capabilities
chat_generator = OpenAIChatGenerator(
model="gpt-4o-mini",
generation_kwargs={"temperature": 0.0},
)
## Create the extractor
extractor = LLMDocumentContentExtractor(
chat_generator=chat_generator,
file_path_meta_field="file_path",
raise_on_failure=False,
)
## Create documents with image file paths
documents = [
Document(content="", meta={"file_path": "image.jpg"}),
Document(content="", meta={"file_path": "document.pdf", "page_number": 1}),
]
## Run the extractor
result = extractor.run(documents=documents)
## Check results
print(f"Successfully processed: {len(result['documents'])}")
print(f"Failed documents: {len(result['failed_documents'])}")
## Access extracted content
for doc in result["documents"]:
print(f"File: {doc.meta['file_path']}")
print(f"Extracted content: {doc.content[:100]}...")
```
### Using custom prompts
You can provide a custom prompt to instruct the LLM on how to extract content:
```python
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
custom_prompt = """
Extract all text content from this image-based document.
Instructions:
- Extract text exactly as it appears
- Preserve the reading order
- Format tables as markdown
- Describe any images or diagrams briefly
- Maintain document structure
Document:"""
chat_generator = OpenAIChatGenerator(model="gpt-4o-mini")
extractor = LLMDocumentContentExtractor(
chat_generator=chat_generator,
prompt=custom_prompt,
file_path_meta_field="file_path",
)
documents = [Document(content="", meta={"file_path": "scanned_document.pdf"})]
result = extractor.run(documents=documents)
```
### Handling failed documents
The component provides detailed error information for failed documents:
```python
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
chat_generator = OpenAIChatGenerator(model="gpt-4o-mini")
extractor = LLMDocumentContentExtractor(
chat_generator=chat_generator,
raise_on_failure=False, # Don't raise exceptions, return failed documents
)
documents = [Document(content="", meta={"file_path": "problematic_image.jpg"})]
result = extractor.run(documents=documents)
## Check for failed documents
for failed_doc in result["failed_documents"]:
print(f"Failed to process: {failed_doc.meta['file_path']}")
print(f"Error: {failed_doc.meta['content_extraction_error']}")
```
### In a pipeline
Below is an example of a pipeline that uses `LLMDocumentContentExtractor` to process image-based documents and store the extracted text:
```python
from haystack import Pipeline
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import Document
## Create document store
document_store = InMemoryDocumentStore()
## Create pipeline
p = Pipeline()
p.add_component(
instance=LLMDocumentContentExtractor(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
file_path_meta_field="file_path",
),
name="content_extractor",
)
p.add_component(instance=DocumentSplitter(), name="splitter")
p.add_component(instance=DocumentWriter(document_store=document_store), name="writer")
## Connect components
p.connect("content_extractor.documents", "splitter.documents")
p.connect("splitter.documents", "writer.documents")
## Create test documents
docs = [
Document(content="", meta={"file_path": "scanned_document.pdf"}),
Document(content="", meta={"file_path": "image_with_text.jpg"}),
]
## Run pipeline
result = p.run({"content_extractor": {"documents": docs}})
## Check results
print(f"Successfully processed: {len(result['content_extractor']['documents'])}")
print(f"Failed documents: {len(result['content_extractor']['failed_documents'])}")
## Access documents in the store
stored_docs = document_store.filter_documents()
print(f"Documents in store: {len(stored_docs)}")
```
@@ -0,0 +1,149 @@
---
title: "LLMMetadataExtractor"
id: llmmetadataextractor
slug: "/llmmetadataextractor"
description: "Extracts metadata from documents using a Large Language Model. The metadata is extracted by providing a prompt to a LLM that generates it."
---
# LLMMetadataExtractor
Extracts metadata from documents using a Large Language Model. The metadata is extracted by providing a prompt to a LLM that generates it.
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | After [PreProcessors](../preprocessors.mdx) in an indexing pipeline |
| **Mandatory init variables** | `prompt`: The prompt to instruct the LLM on how to extract metadata from the document <br /> <br />`chat_generator`: A Chat Generator instance which represents the LLM configured to return a JSON object |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Extractors](/reference/extractors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/llm_metadata_extractor.py |
</div>
## Overview
The `LLMMetadataExtractor` extraction relies on an LLM and a prompt to perform the metadata extraction. At initialization time, it expects an LLM, a Haystack Generator, and a prompt describing the metadata extraction process.
The prompt should have a variable called `document` that will point to a single document in the list of documents. So, to access the content of the document, you can use `{{ document.content }}` in the prompt.
At runtime, it expects a list of documents and will run the LLM on each document in the list, extracting metadata from the document. The metadata will be added to the document's metadata field.
If the LLM fails to extract metadata from a document, it will be added to the `failed_documents` list. The failed documents' metadata will contain the keys `metadata_extraction_error` and `metadata_extraction_response`.
These documents can be re-run with another extractor to extract metadata using the `metadata_extraction_response` and `metadata_extraction_error` in the prompt.
The current implementation supports the following Haystack Generators:
- [OpenAIChatGenerator](../generators/openaichatgenerator.mdx)
- [AzureOpenAIChatGenerator](../generators/azureopenaichatgenerator.mdx)
- [AmazonBedrockChatGenerator](../generators/amazonbedrockchatgenerator.mdx)
- [VertexAIGeminiChatGenerator](../generators/vertexaigeminichatgenerator.mdx)
## Usage
Here's an example of using the `LLMMetadataExtractor` to extract named entities and add them to the document's metadata.
First, the mandatory imports:
```python
from haystack import Document
from haystack.components.extractors.llm_metadata_extractor import LLMMetadataExtractor
from haystack.components.generators.chat import OpenAIChatGenerator
```
Then, define some documents:
```python
docs = [
Document(
content="deepset was founded in 2018 in Berlin, and is known for its Haystack framework",
),
Document(
content="Hugging Face is a company founded in New York, USA and is known for its Transformers library",
),
]
```
And now, a prompt that extracts named entities from the documents:
```python
NER_PROMPT = """
-Goal-
Given text and a list of entity types, identify all entities of those types from the text.
-Steps-
1. Identify all entities. For each identified entity, extract the following information:
- entity_name: Name of the entity, capitalized
- entity_type: One of the following types: [organization, product, service, industry]
Format each entity as a JSON like: {"entity": <entity_name>, "entity_type": <entity_type>}
2. Return output in a single list with all the entities identified in steps 1.
-Examples-
######################
Example 1:
entity_types: [organization, person, partnership, financial metric, product, service, industry, investment strategy, market trend]
text: Another area of strength is our co-brand issuance. Visa is the primary network partner for eight of the top
10 co-brand partnerships in the US today and we are pleased that Visa has finalized a multi-year extension of
our successful credit co-branded partnership with Alaska Airlines, a portfolio that benefits from a loyal customer
base and high cross-border usage.
We have also had significant co-brand momentum in CEMEA. First, we launched a new co-brand card in partnership
with Qatar Airways, British Airways and the National Bank of Kuwait. Second, we expanded our strong global
Marriott relationship to launch Qatar's first hospitality co-branded card with Qatar Islamic Bank. Across the
United Arab Emirates, we now have exclusive agreements with all the leading airlines marked by a recent
agreement with Emirates Skywards.
And we also signed an inaugural Airline co-brand agreement in Morocco with Royal Air Maroc. Now newer digital
issuers are equally
------------------------
output:
{"entities": [{"entity": "Visa", "entity_type": "company"}, {"entity": "Alaska Airlines", "entity_type": "company"}, {"entity": "Qatar Airways", "entity_type": "company"}, {"entity": "British Airways", "entity_type": "company"}, {"entity": "National Bank of Kuwait", "entity_type": "company"}, {"entity": "Marriott", "entity_type": "company"}, {"entity": "Qatar Islamic Bank", "entity_type": "company"}, {"entity": "Emirates Skywards", "entity_type": "company"}, {"entity": "Royal Air Maroc", "entity_type": "company"}]}
#############################
-Real Data-
######################
entity_types: [company, organization, person, country, product, service]
text: {{ document.content }}
######################
output:
"""
```
Now, define a simple indexing pipeline that uses the `LLMMetadataExtractor` to extract named entities from the documents:
```python
chat_generator = OpenAIChatGenerator(
generation_kwargs={
"max_tokens": 500,
"temperature": 0.0,
"seed": 0,
"response_format": {"type": "json_object"},
},
max_retries=1,
timeout=60.0,
)
extractor = LLMMetadataExtractor(
prompt=NER_PROMPT,
chat_generator=generator,
expected_keys=["entities"],
raise_on_failure=False,
)
extractor.warm_up()
extractor.run(documents=docs)
>> {'documents': [
Document(id=.., content: 'deepset was founded in 2018 in Berlin, and is known for its Haystack framework',
meta: {'entities': [{'entity': 'deepset', 'entity_type': 'company'}, {'entity': 'Berlin', 'entity_type': 'city'},
{'entity': 'Haystack', 'entity_type': 'product'}]}),
Document(id=.., content: 'Hugging Face is a company that was founded in New York, USA and is known for its Transformers library',
meta: {'entities': [
{'entity': 'Hugging Face', 'entity_type': 'company'}, {'entity': 'New York', 'entity_type': 'city'},
{'entity': 'USA', 'entity_type': 'country'}, {'entity': 'Transformers', 'entity_type': 'product'}
]})
]
'failed_documents': []
}
>>
```
@@ -0,0 +1,99 @@
---
title: "NamedEntityExtractor"
id: namedentityextractor
slug: "/namedentityextractor"
description: "This component extracts predefined entities out of a piece of text and writes them into documents meta field."
---
# NamedEntityExtractor
This component extracts predefined entities out of a piece of text and writes them into documents meta field.
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | After the [PreProcessor](../preprocessors.mdx) in an indexing pipeline or after a [Retriever](../retrievers.mdx) in a query pipeline |
| **Mandatory init variables** | `backend`: The backend to use for NER <br /> <br />`model`: Name or path of the model to use |
| **Mandatory run variables** | `documents`: A list of documents |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Extractors](/reference/extractors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/named_entity_extractor.py |
</div>
## Overview
`NamedEntityExtractor` looks for entities, which are spans in the text. The extractor automatically recognizes and groups them depending on their class, such as people's names, organizations, locations, and other types. The exact classes are determined by the model that you initialize the component with.
`NamedEntityExtractor` takes a list of documents as input and returns a list of the same documents with their `meta` data enriched with `NamedEntityAnnotations`. A `NamedEntityAnnotation` consists of the type of the entity, the start and end of the span, and a score calculated by the model, for example: `NamedEntityAnnotation(entity='PER', start=11, end=16, score=0.9)`.
When the `NamedEntityExtractor` is initialized, you need to set a `model` and a `backend`. The latter can be either `"hugging_face"` or `"spacy"`. Optionally, you can set `pipeline_kwargs`, which are then passed on to the Hugging Face pipeline or the spaCy pipeline. You can additionally set the `device` that is used to run the component.
## Usage
The current implementation supports two NER backends: Hugging Face and spaCy. These two backends work with any HF or spaCy model that supports token classification or NER.
Heres an example of how you could initialize different backends:
```python
## Initialize with HF backend
extractor = NamedEntityExtractor(backend="hugging_face", model="dslim/bert-base-NER")
## Initialize with spaCy backend
extractor = NamedEntityExtractor(backend="spacy", model="en_core_web_sm")
```
`NamedEntityExtractor` accepts a list of `Documents` as its input. The extractor annotates the raw text in the documents and stores the annotations in the document's `meta` dictionary under the `named_entities` key.
```python
from haystack.dataclasses import Document
from haystack.components.extractors import NamedEntityExtractor
extractor = NamedEntityExtractor(backend="hugging_face", model="dslim/bert-base-NER")
documents = [
Document(content="My name is Clara and I live in Berkeley, California."),
Document(content="I'm Merlin, the happy pig!"),
Document(content="New York State is home to the Empire State Building."),
]
extractor.warm_up()
extractor.run(documents)
print(documents)
```
Here is the example result:
```python
[Document(id=aec840d1b6c85609f4f16c3e222a5a25fd8c4c53bd981a40c1268ab9c72cee10, content: 'My name is Clara and I live in Berkeley, California.', meta: {'named_entities': [NamedEntityAnnotation(entity='PER', start=11, end=16, score=0.99641764), NamedEntityAnnotation(entity='LOC', start=31, end=39, score=0.996198), NamedEntityAnnotation(entity='LOC', start=41, end=51, score=0.9990196)]}),
Document(id=98f1dc5d0ccd9d9950cd191d1076db0f7af40c401dd7608f11c90cb3fc38c0c2, content: 'I'm Merlin, the happy pig!', meta: {'named_entities': [NamedEntityAnnotation(entity='PER', start=4, end=10, score=0.99054915)]}),
Document(id=44948ea0eec018b33aceaaedde4616eb9e93ce075e0090ec1613fc145f84b4a9, content: 'New York State is home to the Empire State Building.', meta: {'named_entities': [NamedEntityAnnotation(entity='LOC', start=0, end=14, score=0.9989541), NamedEntityAnnotation(entity='LOC', start=30, end=51, score=0.95746297)]})]
```
### Get stored annotations
This component includes the `get_stored_annotations` helper class method that allows you to retrieve the annotations stored in a `Document` transparently:
```python
from haystack.dataclasses import Document
from haystack.components.extractors import NamedEntityExtractor
extractor = NamedEntityExtractor(backend="hugging_face", model="dslim/bert-base-NER")
documents = [
Document(content="My name is Clara and I live in Berkeley, California."),
Document(content="I'm Merlin, the happy pig!"),
Document(content="New York State is home to the Empire State Building."),
]
extractor.warm_up()
extractor.run(documents)
annotations = [NamedEntityExtractor.get_stored_annotations(doc) for doc in documents]
print(annotations)
## If a Document doesn't contain any annotations, this returns None.
new_doc = Document(content="In one of many possible worlds...")
assert NamedEntityExtractor.get_stored_annotations(new_doc) is None
```
@@ -0,0 +1,146 @@
---
title: "RegexTextExtractor"
id: regextextextractor
slug: "/regextextextractor"
description: "Extracts text from chat messages or strings using a regular expression pattern."
---
# RegexTextExtractor
Extracts text from chat messages or strings using a regular expression pattern.
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | After a [Chat Generator](../generators.mdx) to parse structured output from LLM responses |
| **Mandatory init variables** | `regex_pattern`: The regular expression pattern used to extract text |
| **Mandatory run variables** | `text_or_messages`: A string or a list of `ChatMessage` objects to search through |
| **Output variables** | `captured_text`: The extracted text from the first capture group |
| **API reference** | [Extractors](/reference/extractors-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/extractors/regex_text_extractor.py |
</div>
## Overview
`RegexTextExtractor` parses text input or `ChatMessage` objects using a regular expression pattern and extracts text captured by capture groups. This is useful for extracting structured information from LLM outputs that follow specific formats, such as XML-like tags or other patterns.
The component works with both plain strings and lists of `ChatMessage` objects. When given a list of messages, it processes only the last message.
The regex pattern should include at least one capture group (text within parentheses) to specify what text to extract. If no capture group is provided, the entire match is returned instead.
### Handling no matches
By default, when the pattern doesn't match, the component returns an empty dictionary `{}`. You can change this behavior with the `return_empty_on_no_match` parameter:
```python
from haystack.components.extractors import RegexTextExtractor
# Default behavior - returns empty dict when no match
extractor_default = RegexTextExtractor(regex_pattern=r"<answer>(.*?)</answer>")
result = extractor_default.run(text_or_messages="No answer tags here")
print(result) # Output: {}
# Alternative behavior - returns empty string when no match
extractor_explicit = RegexTextExtractor(
regex_pattern=r"<answer>(.*?)</answer>",
return_empty_on_no_match=False,
)
result = extractor_explicit.run(text_or_messages="No answer tags here")
print(result) # Output: {'captured_text': ''}
```
:::note
The default behavior of returning `{}` when no match is found is deprecated and will change in a future release to return `{'captured_text': ''}` instead. Set `return_empty_on_no_match=False` explicitly if you want the new behavior now.
:::
## Usage
### On its own
This example extracts a URL from an XML-like tag structure:
```python
from haystack.components.extractors import RegexTextExtractor
# Create extractor with a pattern that captures the URL value
extractor = RegexTextExtractor(regex_pattern='<issue url="(.+?)">')
# Extract from a string
result = extractor.run(
text_or_messages='<issue url="github.com/example/issue/123">Issue description</issue>',
)
print(result)
# Output: {'captured_text': 'github.com/example/issue/123'}
```
### With ChatMessages
When working with LLM outputs in chat pipelines, you can extract structured data from `ChatMessage` objects:
```python
from haystack.components.extractors import RegexTextExtractor
from haystack.dataclasses import ChatMessage
extractor = RegexTextExtractor(
regex_pattern=r"```json\s*(.*?)\s*```",
return_empty_on_no_match=False,
)
# Simulating an LLM response with JSON in a code block
messages = [
ChatMessage.from_user("Extract the data"),
ChatMessage.from_assistant(
'Here is the data:\n```json\n{"name": "Alice", "age": 30}\n```',
),
]
result = extractor.run(text_or_messages=messages)
print(result)
# Output: {'captured_text': '{"name": "Alice", "age": 30}'}
```
### In a pipeline
This example demonstrates extracting a specific section from a structured LLM response. The pipeline asks an LLM to analyze a topic and format its response with XML-like tags for different sections. The `RegexTextExtractor` then pulls out only the summary, discarding the rest of the response.
The LLM generates a full response with both `<analysis>` and `<summary>` sections, but only the content inside `<summary>` tags is extracted and returned.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.extractors import RegexTextExtractor
from haystack.dataclasses import ChatMessage
pipe = Pipeline()
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", OpenAIChatGenerator())
pipe.add_component(
"extractor",
RegexTextExtractor(
regex_pattern=r"<summary>(.*?)</summary>",
return_empty_on_no_match=False,
),
)
pipe.connect("prompt_builder.prompt", "llm.messages")
pipe.connect("llm.replies", "extractor.text_or_messages")
# Instruct the LLM to use a specific structured format
messages = [
ChatMessage.from_system(
"Respond using this exact format:\n"
"<analysis>Your detailed analysis here</analysis>\n"
"<summary>A one-sentence summary</summary>",
),
ChatMessage.from_user("What are the main benefits and drawbacks of remote work?"),
]
# Run the pipeline (requires OPENAI_API_KEY environment variable)
result = pipe.run({"prompt_builder": {"template": messages}})
print(result["extractor"]["captured_text"])
# Output: 'Remote work offers flexibility and eliminates commuting but can lead to isolation and blurred work-life boundaries.'
```