Files
wehub-resource-sync c56bef871b
Sync docs with Docusaurus / sync (push) Waiting to run
Tests / Check if changed (push) Waiting to run
Tests / format (push) Blocked by required conditions
Tests / check-imports (push) Blocked by required conditions
Tests / Unit / macos-latest (push) Blocked by required conditions
Tests / Unit / ubuntu-latest (push) Blocked by required conditions
Tests / Unit / windows-latest (push) Blocked by required conditions
Tests / mypy (push) Blocked by required conditions
Tests / Integration / ubuntu-latest (push) Blocked by required conditions
Tests / Integration / macos-latest (push) Blocked by required conditions
Tests / Integration / windows-latest (push) Blocked by required conditions
Tests / notify-slack-on-failure (push) Blocked by required conditions
Tests / Mark tests as completed (push) Blocked by required conditions
Docker image release / Build base image (push) Waiting to run
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:22:28 +08:00

303 lines
11 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: "Presidio"
id: integrations-presidio
description: "Presidio integration for Haystack"
slug: "/integrations-presidio"
---
## haystack_integrations.components.extractors.presidio.presidio_entity_extractor
### PresidioEntityExtractor
Detects PII entities in Haystack Documents using Microsoft Presidio Analyzer.
See [Presidio Analyzer](https://microsoft.github.io/presidio/) for details.
Accepts a list of Documents and returns new Documents with detected PII entities stored
in each Document's metadata under the key `"entities"`. Each entry in the list contains
the entity type, start/end character offsets, and the confidence score.
Original Documents are not mutated. Documents without text content are passed through unchanged.
The analyzer engine is loaded on the first call to `run()`,
or by calling `warm_up()` explicitly beforehand.
### Usage example
```python
from haystack import Document
from haystack_integrations.components.extractors.presidio import PresidioEntityExtractor
extractor = PresidioEntityExtractor()
result = extractor.run(documents=[Document(content="Contact Alice at alice@example.com")])
print(result["documents"][0].meta["entities"])
# [{"entity_type": "PERSON", "start": 8, "end": 13, "score": 0.85},
# {"entity_type": "EMAIL_ADDRESS", "start": 17, "end": 34, "score": 1.0}]
```
#### SPACY_DEFAULT_MODELS
```python
SPACY_DEFAULT_MODELS: dict[str, str] = _SPACY_DEFAULT_MODELS
```
Mapping from ISO 639-1 language code to the largest available spaCy model for that language.
Used to automatically select an NLP model when `models` is not specified.
See [spaCy documentation](https://spacy.io/models) for the full list of available spaCy models.
#### __init__
```python
__init__(
*,
language: str = "en",
entities: list[str] | None = None,
score_threshold: float = 0.35,
models: list[dict[str, str]] | None = None
) -> None
```
Initializes the PresidioEntityExtractor.
**Parameters:**
- **language** (<code>str</code>) ISO 639-1 language code for PII detection. Defaults to `"en"`.
For languages in the built-in mapping (e.g. `"de"`, `"fr"`, `"es"`), the appropriate
spaCy model is loaded automatically at warm-up time — no need to set `models`.
For unsupported languages, use the `models` parameter to configure a custom model.
See [Presidio supported languages](https://microsoft.github.io/presidio/analyzer/languages/).
- **entities** (<code>list\[str\] | None</code>) List of PII entity types to detect (e.g. `["PERSON", "EMAIL_ADDRESS"]`).
If `None`, all supported entity types are detected.
See [Presidio supported entities](https://microsoft.github.io/presidio/supported_entities/).
- **score_threshold** (<code>float</code>) Minimum confidence score (0-1) for a detected entity to be included. Defaults to `0.35`.
See [Presidio analyzer documentation](https://microsoft.github.io/presidio/analyzer/).
- **models** (<code>list\[dict\[str, str\]\] | None</code>) Advanced override: list of spaCy model configurations.
Each entry must contain `"lang_code"` and `"model_name"` keys,
e.g. `[{"lang_code": "fr", "model_name": "fr_core_news_md"}]`.
Use this only when you need a specific model variant or a language not covered by the
built-in mapping. If `None`, the model is selected automatically from `SPACY_DEFAULT_MODELS`
based on `language`.
#### warm_up
```python
warm_up() -> None
```
Initializes the Presidio analyzer engine.
This method loads the underlying NLP models. In a Haystack Pipeline,
this is called automatically before the first run.
#### run
```python
run(documents: list[Document]) -> dict[str, list[Document]]
```
Detects PII entities in the provided Documents.
**Parameters:**
- **documents** (<code>list\[Document\]</code>) List of Documents to analyze for PII entities.
**Returns:**
- <code>dict\[str, list\[Document\]\]</code> A dictionary with key `documents` containing Documents with detected entities
stored in metadata under the key `"entities"`.
## haystack_integrations.components.preprocessors.presidio.presidio_document_cleaner
### PresidioDocumentCleaner
Anonymizes PII in Haystack Documents using [Microsoft Presidio](https://microsoft.github.io/presidio/).
Accepts a list of Documents, detects personally identifiable information (PII) in their
text content, and returns new Documents with PII replaced by entity type placeholders
(e.g. `<PERSON>`, `<EMAIL_ADDRESS>`). Original Documents are not mutated.
Documents without text content are passed through unchanged.
The analyzer and anonymizer engines are loaded on the first call to `run()`,
or by calling `warm_up()` explicitly beforehand.
### Usage example
```python
from haystack import Document
from haystack_integrations.components.preprocessors.presidio import PresidioDocumentCleaner
cleaner = PresidioDocumentCleaner()
result = cleaner.run(documents=[Document(content="My name is John and my email is john@example.com")])
print(result["documents"][0].content)
# My name is <PERSON> and my email is <EMAIL_ADDRESS>
```
#### SPACY_DEFAULT_MODELS
```python
SPACY_DEFAULT_MODELS: dict[str, str] = _SPACY_DEFAULT_MODELS
```
Mapping from ISO 639-1 language code to the largest available spaCy model for that language.
Used to automatically select an NLP model when `models` is not specified.
See [spaCy documentation](https://spacy.io/models) for the full list of available spaCy models.
#### __init__
```python
__init__(
*,
language: str = "en",
entities: list[str] | None = None,
score_threshold: float = 0.35,
models: list[dict[str, str]] | None = None
) -> None
```
Initializes the PresidioDocumentCleaner.
**Parameters:**
- **language** (<code>str</code>) ISO 639-1 language code for PII detection. Defaults to `"en"`.
For languages in the built-in mapping (e.g. `"de"`, `"fr"`, `"es"`), the appropriate
spaCy model is loaded automatically at warm-up time — no need to set `models`.
For unsupported languages, use the `models` parameter to configure a custom model.
See [Presidio supported languages](https://microsoft.github.io/presidio/analyzer/languages/).
- **entities** (<code>list\[str\] | None</code>) List of PII entity types to detect and anonymize (e.g. `["PERSON", "EMAIL_ADDRESS"]`).
If `None`, all supported entity types are used.
See [Presidio supported entities](https://microsoft.github.io/presidio/supported_entities/).
- **score_threshold** (<code>float</code>) Minimum confidence score (0-1) for a detected entity to be anonymized. Defaults to `0.35`.
See [Presidio analyzer documentation](https://microsoft.github.io/presidio/analyzer/).
- **models** (<code>list\[dict\[str, str\]\] | None</code>) Advanced override: list of spaCy model configurations.
Each entry must contain `"lang_code"` and `"model_name"` keys,
e.g. `[{"lang_code": "fr", "model_name": "fr_core_news_md"}]`.
Use this only when you need a specific model variant or a language not covered by the
built-in mapping. If `None`, the model is selected automatically from `SPACY_DEFAULT_MODELS`
based on `language`.
#### warm_up
```python
warm_up() -> None
```
Initializes the Presidio analyzer and anonymizer engines.
This method loads the underlying NLP models. In a Haystack Pipeline,
this is called automatically before the first run.
#### run
```python
run(documents: list[Document]) -> dict[str, list[Document]]
```
Anonymizes PII in the provided Documents.
**Parameters:**
- **documents** (<code>list\[Document\]</code>) List of Documents whose text content will be anonymized.
**Returns:**
- <code>dict\[str, list\[Document\]\]</code> A dictionary with key `documents` containing the cleaned Documents.
## haystack_integrations.components.preprocessors.presidio.presidio_text_cleaner
### PresidioTextCleaner
Anonymizes PII in plain strings using [Microsoft Presidio](https://microsoft.github.io/presidio/).
Accepts a list of strings, detects personally identifiable information (PII), and returns
a new list of strings with PII replaced by entity type placeholders (e.g. `<PERSON>`).
Useful for sanitizing user queries before they are sent to an LLM.
The analyzer and anonymizer engines are loaded on the first call to `run()`,
or by calling `warm_up()` explicitly beforehand.
### Usage example
```python
from haystack_integrations.components.preprocessors.presidio import PresidioTextCleaner
cleaner = PresidioTextCleaner()
result = cleaner.run(texts=["Hi, I am John Smith, call me at 212-555-1234"])
print(result["texts"][0])
# Hi, I am <PERSON>, call me at <PHONE_NUMBER>
```
#### SPACY_DEFAULT_MODELS
```python
SPACY_DEFAULT_MODELS: dict[str, str] = _SPACY_DEFAULT_MODELS
```
Mapping from ISO 639-1 language code to the largest available spaCy model for that language.
Used to automatically select an NLP model when `models` is not specified.
See [spaCy documentation](https://spacy.io/models) for the full list of available spaCy models.
#### __init__
```python
__init__(
*,
language: str = "en",
entities: list[str] | None = None,
score_threshold: float = 0.35,
models: list[dict[str, str]] | None = None
) -> None
```
Initializes the PresidioTextCleaner.
**Parameters:**
- **language** (<code>str</code>) ISO 639-1 language code for PII detection. Defaults to `"en"`.
For languages in the built-in mapping (e.g. `"de"`, `"fr"`, `"es"`), the appropriate
spaCy model is loaded automatically at warm-up time — no need to set `models`.
For unsupported languages, use the `models` parameter to configure a custom model.
See [Presidio supported languages](https://microsoft.github.io/presidio/analyzer/languages/).
- **entities** (<code>list\[str\] | None</code>) List of PII entity types to detect and anonymize (e.g. `["PERSON", "PHONE_NUMBER"]`).
If `None`, all supported entity types are used.
See [Presidio supported entities](https://microsoft.github.io/presidio/supported_entities/).
- **score_threshold** (<code>float</code>) Minimum confidence score (0-1) for a detected entity to be anonymized. Defaults to `0.35`.
See [Presidio analyzer documentation](https://microsoft.github.io/presidio/analyzer/).
- **models** (<code>list\[dict\[str, str\]\] | None</code>) Advanced override: list of spaCy model configurations.
Each entry must contain `"lang_code"` and `"model_name"` keys,
e.g. `[{"lang_code": "fr", "model_name": "fr_core_news_md"}]`.
Use this only when you need a specific model variant or a language not covered by the
built-in mapping. If `None`, the model is selected automatically from `SPACY_DEFAULT_MODELS`
based on `language`.
#### warm_up
```python
warm_up() -> None
```
Initializes the Presidio analyzer and anonymizer engines.
This method loads the underlying NLP models. In a Haystack Pipeline,
this is called automatically before the first run.
#### run
```python
run(texts: list[str]) -> dict[str, list[str]]
```
Anonymizes PII in the provided strings.
**Parameters:**
- **texts** (<code>list\[str\]</code>) List of strings to anonymize.
**Returns:**
- <code>dict\[str, list\[str\]\]</code> A dictionary with key `texts` containing the cleaned strings.