wehub-resource-sync
6b7e6b44f1
gh-pages / build (push) Waiting to run
Python Publish (pypi) / Upload release to PyPI (push) Waiting to run
Spellcheck / spellcheck (push) Waiting to run
Python Build and Type Check / python-ci (ubuntu-latest, 3.11) (push) Has been cancelled
Python Build and Type Check / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Build and Type Check / python-ci (windows-latest, 3.11) (push) Has been cancelled
Python Build and Type Check / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Integration Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Integration Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Notebook Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Notebook Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Smoke Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Smoke Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
Python Unit Tests / python-ci (ubuntu-latest, 3.13) (push) Has been cancelled
Python Unit Tests / python-ci (windows-latest, 3.13) (push) Has been cancelled
42 lines
1.6 KiB
Markdown
42 lines
1.6 KiB
Markdown
# GraphRAG Inputs
|
|
|
|
This package provides input document loading utilities for GraphRAG, supporting multiple file formats including CSV, JSON, JSON Lines, and plain text.
|
|
|
|
## Supported File Types
|
|
|
|
The following four standard file formats are supported out of the box:
|
|
|
|
- **CSV** - Tabular data with configurable column mappings
|
|
- **JSON** - JSON files with configurable property paths
|
|
- **JSON Lines** - Line-delimited JSON records
|
|
- **Text** - Plain text files
|
|
|
|
### Markitdown Support
|
|
|
|
Additionally, we support the `InputType.MarkItDown` format, which uses the [MarkItDown](https://github.com/microsoft/markitdown) library to import any supported file type. The MarkItDown converter can handle a wide variety of file formats including Office documents, PDFs, HTML, and more.
|
|
|
|
**Note:** Additional optional dependencies may need to be installed depending on the file type you're processing. The choice of converter is determined by MarkItDowns's processing logic, which primarily uses the file extension to select the appropriate converter. Please refer to the [MarkItDown repository](https://github.com/microsoft/markitdown) for installation instructions and detailed information about supported formats.
|
|
|
|
## Examples
|
|
|
|
Basic usage with the factory:
|
|
|
|
|
|
1. Import a pdf with MarkItDown:
|
|
|
|
```bash
|
|
pip install 'markitdown[pdf]' # required dependency for pdf processing
|
|
```
|
|
|
|
|
|
2. YAML config example for above:
|
|
```yaml
|
|
input:
|
|
type: markitdown
|
|
file_pattern: ".*\\.pdf$$"
|
|
input_storage:
|
|
type: file
|
|
base_dir: "input"
|
|
```
|
|
|
|
[Open the notebook to explore the input example code](example_notebooks/input_example.ipynb) |