Files
deepset-ai--haystack/docs-website/docs/concepts/data-classes/imagecontent.mdx
T
wehub-resource-sync c56bef871b
Sync docs with Docusaurus / sync (push) Waiting to run
Tests / Check if changed (push) Waiting to run
Tests / format (push) Blocked by required conditions
Tests / check-imports (push) Blocked by required conditions
Tests / Unit / macos-latest (push) Blocked by required conditions
Tests / Unit / ubuntu-latest (push) Blocked by required conditions
Tests / Unit / windows-latest (push) Blocked by required conditions
Tests / mypy (push) Blocked by required conditions
Tests / Integration / ubuntu-latest (push) Blocked by required conditions
Tests / Integration / macos-latest (push) Blocked by required conditions
Tests / Integration / windows-latest (push) Blocked by required conditions
Tests / notify-slack-on-failure (push) Blocked by required conditions
Tests / Mark tests as completed (push) Blocked by required conditions
Docker image release / Build base image (push) Waiting to run
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:22:28 +08:00

248 lines
5.9 KiB
Plaintext

---
title: "ImageContent"
id: imagecontent
slug: "/imagecontent"
description: "`ImageContent` represents image-based content in Haystack chat messages and multimodal pipelines."
---
# ImageContent
`ImageContent` is a Haystack data class used to represent image-based content in chat messages and multimodal AI pipelines.
It is commonly used with:
* multimodal LLMs
* vision-language models
* image-aware chat applications
* document/image processing workflows
`ImageContent` stores images as base64-encoded strings together with metadata such as MIME type and image detail level.
If you are looking for the full API reference, see the [API documentation](/reference/data-classes-api#imagecontent).
---
# Creating ImageContent
You can create an `ImageContent` object directly from a base64 string:
```python
from haystack.dataclasses import ImageContent
image = ImageContent(base64_image="your_base64_encoded_image", mime_type="image/png")
print(image)
```
---
# Loading Images from a File Path
The `from_file_path()` class method provides a convenient way to load local image files.
```python
from haystack.dataclasses import ImageContent
image = ImageContent.from_file_path("sample.png", detail="low")
print(image)
```
The optional `detail` parameter is currently supported by OpenAI vision models and accepts:
* `"auto"`
* `"high"`
* `"low"`
You can also resize images while loading:
```python
image = ImageContent.from_file_path("sample.png", size=(512, 512))
```
This helps reduce:
* memory usage
* processing time
* payload size
when working with multimodal LLM APIs.
---
# Loading Images from a URL
You can also create an `ImageContent` object directly from an image URL:
```python
from haystack.dataclasses import ImageContent
image = ImageContent.from_url(
"https://images.unsplash.com/photo-1546182990-dffeafbe841d",
detail="low",
)
print(image)
```
Internally, Haystack downloads the image and converts it into a base64 representation.
---
# Producing ImageContent with Converters
In a pipeline, you usually don't create `ImageContent` objects by hand. Instead, you use converter components that read files and produce `ImageContent` for you:
* [`ImageFileToImageContent`](../../pipeline-components/converters/imagefiletoimagecontent.mdx) converts local image files (such as PNG or JPEG) into `ImageContent` objects.
* [`PDFToImageContent`](../../pipeline-components/converters/pdftoimagecontent.mdx) renders the pages of PDF files into `ImageContent` objects.
```python
from haystack.components.converters.image import (
ImageFileToImageContent,
PDFToImageContent,
)
image_converter = ImageFileToImageContent()
image_contents = image_converter.run(sources=["image.jpg", "another_image.png"])[
"image_contents"
]
pdf_converter = PDFToImageContent()
pdf_image_contents = pdf_converter.run(sources=["file.pdf"])["image_contents"]
```
Both converters accept the optional `detail` and `size` parameters, which are forwarded to the `ImageContent` objects they create.
---
# Using ImageContent with ChatMessage
`ImageContent` is commonly used together with [`ChatMessage`](chatmessage.mdx) for multimodal conversations.
```python
from haystack.dataclasses import ChatMessage, ImageContent
image = ImageContent.from_url(
"https://images.unsplash.com/photo-1546182990-dffeafbe841d",
detail="low",
)
message = ChatMessage.from_user(content_parts=["What does this image show?", image])
print(message)
```
This allows multimodal LLMs to process both:
* textual prompts
* image inputs
within the same message.
For more dynamic prompts, you can build multimodal messages with [`ChatPromptBuilder`](../../pipeline-components/builders/chatpromptbuilder.mdx) using Jinja2 string templates. The `| templatize_part` filter inserts an `ImageContent` object as a structured content part instead of plain text:
```python
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage, ImageContent
template = """
{% message role="user" %}
Hello! I am {{user_name}}. What's the difference between the following images?
{% for image in images %}
{{ image | templatize_part }}
{% endfor %}
{% endmessage %}
"""
builder = ChatPromptBuilder(template=template)
images = [
ImageContent.from_file_path("apple.jpg"),
ImageContent.from_file_path("kiwi.jpg"),
]
result = builder.run(user_name="John", images=images)
print(result["prompt"])
```
---
# Metadata
The optional `meta` parameter allows you to attach custom metadata to the image.
```python
image = ImageContent.from_url(
"https://images.unsplash.com/photo-1546182990-dffeafbe841d",
meta={"source": "example-dataset"},
)
```
This can be useful for:
* tracing
* dataset tracking
* workflow metadata
* custom application logic
---
# Validation
By default, `ImageContent` validates:
* base64 encoding
* MIME type correctness
* image MIME compatibility
Validation can be disabled to improve performance:
```python
image = ImageContent(
base64_image="your_base64_encoded_image",
mime_type="image/png",
validation=False,
)
```
---
# Serialization
`ImageContent` supports dictionary serialization.
```python
image_dict = image.to_dict()
restored_image = ImageContent.from_dict(image_dict)
```
---
# Displaying Images
The `show()` method can display images directly in:
* Jupyter notebooks
* local desktop environments
```python
image.show()
```
This requires the `Pillow` package:
```bash
pip install pillow
```
---
# Related Components
`ImageContent` is frequently used with:
* [`ChatMessage`](chatmessage.mdx) — to build multimodal messages
* [`ChatPromptBuilder`](../../pipeline-components/builders/chatpromptbuilder.mdx) — to template multimodal prompts
* [`ImageFileToImageContent`](../../pipeline-components/converters/imagefiletoimagecontent.mdx) — to convert image files into `ImageContent`
* [`PDFToImageContent`](../../pipeline-components/converters/pdftoimagecontent.mdx) — to convert PDF pages into `ImageContent`