chore: import upstream snapshot with attribution
Sync docs with Docusaurus / sync (push) Waiting to run
Tests / Check if changed (push) Waiting to run
Tests / format (push) Blocked by required conditions
Tests / check-imports (push) Blocked by required conditions
Tests / Unit / macos-latest (push) Blocked by required conditions
Tests / Unit / ubuntu-latest (push) Blocked by required conditions
Tests / Unit / windows-latest (push) Blocked by required conditions
Tests / mypy (push) Blocked by required conditions
Tests / Integration / ubuntu-latest (push) Blocked by required conditions
Tests / Integration / macos-latest (push) Blocked by required conditions
Tests / Integration / windows-latest (push) Blocked by required conditions
Tests / notify-slack-on-failure (push) Blocked by required conditions
Tests / Mark tests as completed (push) Blocked by required conditions
Docker image release / Build base image (push) Waiting to run
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Waiting to run
Tests / Check if changed (push) Waiting to run
Tests / format (push) Blocked by required conditions
Tests / check-imports (push) Blocked by required conditions
Tests / Unit / macos-latest (push) Blocked by required conditions
Tests / Unit / ubuntu-latest (push) Blocked by required conditions
Tests / Unit / windows-latest (push) Blocked by required conditions
Tests / mypy (push) Blocked by required conditions
Tests / Integration / ubuntu-latest (push) Blocked by required conditions
Tests / Integration / macos-latest (push) Blocked by required conditions
Tests / Integration / windows-latest (push) Blocked by required conditions
Tests / notify-slack-on-failure (push) Blocked by required conditions
Tests / Mark tests as completed (push) Blocked by required conditions
Docker image release / Build base image (push) Waiting to run
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,135 @@
|
||||
---
|
||||
title: "Validators"
|
||||
id: validators-api
|
||||
description: "Validators validate LLM outputs"
|
||||
slug: "/validators-api"
|
||||
---
|
||||
|
||||
|
||||
## json_schema
|
||||
|
||||
### is_valid_json
|
||||
|
||||
```python
|
||||
is_valid_json(s: str) -> bool
|
||||
```
|
||||
|
||||
Check if the provided string is a valid JSON.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **s** (<code>str</code>) – The string to be checked.
|
||||
|
||||
**Returns:**
|
||||
|
||||
- <code>bool</code> – `True` if the string is a valid JSON; otherwise, `False`.
|
||||
|
||||
### JsonSchemaValidator
|
||||
|
||||
Validates JSON content of `ChatMessage` against a specified [JSON Schema](https://json-schema.org/).
|
||||
|
||||
If JSON content of a message conforms to the provided schema, the message is passed along the "validated" output.
|
||||
If the JSON content does not conform to the schema, the message is passed along the "validation_error" output.
|
||||
In the latter case, the error message is constructed using the provided `error_template` or a default template.
|
||||
These error ChatMessages can be used by LLMs in Haystack 2.x recovery loops.
|
||||
|
||||
Usage example:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.joiners import BranchJoiner
|
||||
from haystack.components.validators import JsonSchemaValidator
|
||||
from haystack import component
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
|
||||
@component
|
||||
class MessageProducer:
|
||||
|
||||
@component.output_types(messages=list[ChatMessage])
|
||||
def run(self, messages: list[ChatMessage]) -> dict:
|
||||
return {"messages": messages}
|
||||
|
||||
|
||||
p = Pipeline()
|
||||
p.add_component("llm", OpenAIChatGenerator(generation_kwargs={"response_format": {"type": "json_object"}}))
|
||||
p.add_component("schema_validator", JsonSchemaValidator())
|
||||
p.add_component("joiner_for_llm", BranchJoiner(list[ChatMessage]))
|
||||
p.add_component("message_producer", MessageProducer())
|
||||
|
||||
p.connect("message_producer.messages", "joiner_for_llm")
|
||||
p.connect("joiner_for_llm", "llm")
|
||||
p.connect("llm.replies", "schema_validator.messages")
|
||||
p.connect("schema_validator.validation_error", "joiner_for_llm")
|
||||
|
||||
result = p.run(data={
|
||||
"message_producer": {
|
||||
"messages":[ChatMessage.from_user("Generate JSON for person with name 'John' and age 30")]},
|
||||
"schema_validator": {
|
||||
"json_schema": {
|
||||
"type": "object",
|
||||
"properties": {"name": {"type": "string"},
|
||||
"age": {"type": "integer"}
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
print(result)
|
||||
# >> {'schema_validator': {'validated': [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>,
|
||||
# _content=[TextContent(text="\n{\n "name": "John",\n "age": 30\n}")],
|
||||
# _name=None, _meta={'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 17, 'prompt_tokens': 20,
|
||||
# 'total_tokens': 37}})]}}
|
||||
```
|
||||
|
||||
#### __init__
|
||||
|
||||
```python
|
||||
__init__(
|
||||
json_schema: dict[str, Any] | None = None, error_template: str | None = None
|
||||
) -> None
|
||||
```
|
||||
|
||||
Initialize the JsonSchemaValidator component.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **json_schema** (<code>dict\[str, Any\] | None</code>) – A dictionary representing the [JSON schema](https://json-schema.org/) against which
|
||||
the messages' content is validated.
|
||||
- **error_template** (<code>str | None</code>) – A custom template string for formatting the error message in case of validation failure.
|
||||
|
||||
#### run
|
||||
|
||||
```python
|
||||
run(
|
||||
messages: list[ChatMessage],
|
||||
json_schema: dict[str, Any] | None = None,
|
||||
error_template: str | None = None,
|
||||
) -> dict[str, list[ChatMessage]]
|
||||
```
|
||||
|
||||
Validates the last of the provided messages against the specified json schema.
|
||||
|
||||
If it does, the message is passed along the "validated" output. If it does not, the message is passed along
|
||||
the "validation_error" output.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **messages** (<code>list\[ChatMessage\]</code>) – A list of ChatMessage instances to be validated. The last message in this list is the one
|
||||
that is validated.
|
||||
- **json_schema** (<code>dict\[str, Any\] | None</code>) – A dictionary representing the [JSON schema](https://json-schema.org/)
|
||||
against which the messages' content is validated. If not provided, the schema from the component init
|
||||
is used.
|
||||
- **error_template** (<code>str | None</code>) – A custom template string for formatting the error message in case of validation. If not
|
||||
provided, the `error_template` from the component init is used.
|
||||
|
||||
**Returns:**
|
||||
|
||||
- <code>dict\[str, list\[ChatMessage\]\]</code> – A dictionary with the following keys:
|
||||
- "validated": A list of messages if the last message is valid.
|
||||
- "validation_error": A list of messages if the last message is invalid.
|
||||
|
||||
**Raises:**
|
||||
|
||||
- <code>ValueError</code> – If no JSON schema is provided or if the message content is not a dictionary or a list of
|
||||
dictionaries.
|
||||
Reference in New Issue
Block a user