chore: import upstream snapshot with attribution
Build/Publish Develop Docs / deploy (push) Failing after 1s
PaddleOCR Code Style Check / check-code-style (push) Failing after 1s
PaddleOCR PR Tests GPU / detect-changes (push) Failing after 1s
PaddleOCR PR Tests / detect-changes (push) Failing after 1s
PaddleOCR PR Tests GPU / test-pr-gpu (push) Has been cancelled
PaddleOCR PR Tests / test-pr (push) Has been cancelled
PaddleOCR PR Tests GPU / test-pr-gpu-impl (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.13) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.8) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.9) (push) Has been cancelled
Build/Publish Develop Docs / deploy (push) Failing after 1s
PaddleOCR Code Style Check / check-code-style (push) Failing after 1s
PaddleOCR PR Tests GPU / detect-changes (push) Failing after 1s
PaddleOCR PR Tests / detect-changes (push) Failing after 1s
PaddleOCR PR Tests GPU / test-pr-gpu (push) Has been cancelled
PaddleOCR PR Tests / test-pr (push) Has been cancelled
PaddleOCR PR Tests GPU / test-pr-gpu-impl (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.13) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.8) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.9) (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,64 @@
|
||||
# Introduction to PP-ChatOCRV4
|
||||
**PP-ChatOCRv4** is a unique document and image intelligent analysis solution from PaddlePaddle, combining LLM, MLLM, and OCR technologies to address complex document information extraction challenges such as layout analysis, rare characters, multi-page PDFs, tables, and seal recognition. Integrated with ERNIE Bot, it fuses massive data and knowledge, achieving high accuracy and wide applicability. This pipeline also provides flexible service deployment options, supporting deployment on various hardware. Furthermore, it offers custom development capabilities, allowing you to train and fine-tune models on your own datasets, with seamless integration of trained models.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
|
||||
<div align="center">
|
||||
<table>
|
||||
<thead>
|
||||
<tr >
|
||||
<th class>Solution</td>
|
||||
<th class>Avg Recall</td>
|
||||
</tr>
|
||||
<thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>GPT-4o</td>
|
||||
<td>63.47%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-ChatOCRv3</td>
|
||||
<td class>70.08%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>80.26%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-ChatOCRv4</b></td>
|
||||
<td><b>85.55%</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
# Demo
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
|
||||
</div>
|
||||
|
||||
|
||||
# FAQ
|
||||
|
||||
1. Does support other multimodal models?
|
||||
|
||||
Yes, only set on pipeline configuration.
|
||||
|
||||
2. How to reduce latency and improve throughput?
|
||||
|
||||
Use the High-performance inference plugin, and deploy multi instances.
|
||||
|
||||
3. How to further improve accuracy?
|
||||
|
||||
Firstly, it is necessary to check whether the extracted visual information is correct. If the visual information is incorrect, it is necessary to visualize the visual prediction results to determine which model performs poorly, and then fine-tune train the model with more data. If the visual information is correct but cannot extract the correct information, the prompt needs to be adjusted according to the analysing about the question and answer.
|
||||
@@ -0,0 +1,65 @@
|
||||
# 一、PP-ChatOCRV4简介
|
||||
**PP-ChatOCRv4**是飞桨特色的文档图像智能分析解决方案，结合了 LLM、MLLM 和 OCR 等技术，一站式解决版面分析、生僻字识别、多页 PDF 文件批量解析、复杂表格识别、印章识别等常见的复杂文档信息抽取难点问题，结合文心大模型将海量数据和知识相融合，信息抽取准确率高且应用广泛。本产线同时提供了灵活的服务化部署方式，支持在多种硬件上部署。不仅如此，本产线也提供了二次开发的能力，您可以基于本产线在您自己的数据集上训练调优，训练后的模型也可以无缝集成。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# 二、关键指标
|
||||
|
||||
<div align="center">
|
||||
<table>
|
||||
<thead>
|
||||
<tr >
|
||||
<th class>Solution</td>
|
||||
<th class>Avg Recall</td>
|
||||
</tr>
|
||||
<thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>GPT-4o</td>
|
||||
<td>63.47%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-ChatOCRv3</td>
|
||||
<td class>70.08%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>80.26%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-ChatOCRv4</b></td>
|
||||
<td><b>85.55%</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
# 三、PP-ChatOCRv4 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
|
||||
</div>
|
||||
|
||||
# 四、使用方法和常见问题
|
||||
|
||||
1. 多模态大模型除了DocBee，是否支持其他多模态模型？
|
||||
|
||||
支持，只需在配置文件中进行设置即可。
|
||||
|
||||
2. 如何降低时延、提升吞吐？
|
||||
|
||||
无论使用哪一种服务化部署方案，都可以通过启用高性能推理插件提升模型推理速度，从而降低处理时延。
|
||||
|
||||
此外，对于高稳定性服务化部署方案，通过调整服务配置，设置多个实例，也可以充分利用部署机器的资源，有效提升吞吐。
|
||||
|
||||
3. 如何进一步提升精度？
|
||||
|
||||
首先需要检查提取的视觉信息是否正确，如果视觉信息有误，则需要通过可视化视觉预测结果，判断哪个模型效果较差，从而针对性地训练微调较差的模型；如果视觉信息无误，但无法抽取正确信息，则需要根据问答的具体情况调整Prompt。
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,258 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 1. Introduction to PP-OCRv5 Multilingual Text Recognition
|
||||
|
||||
[PP-OCRv5](./PP-OCRv5.md) is the latest generation text recognition solution in the PP-OCR series, focusing on multi-scenario and multilingual text recognition tasks. In terms of supported text types, the default configuration of the recognition model can accurately identify five major types: Simplified Chinese, Pinyin, Traditional Chinese, English, and Japanese. Additionally, PP-OCRv5 offers multilingual text recognition capabilities covering 106 languages, including Korean, Spanish, French, Portuguese, German, Italian, Russian, Thai, Greek and more (for a full list of supported languages and abbreviations, see [Section 4](#4-supported-languages-and-abbreviations)). Compared to the previous PP-OCRv3 version, PP-OCRv5 achieves over a 30% improvement in accuracy for multilingual text recognition.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/french_0_res.jpg" alt="French recognition result" width="500"/>
|
||||
<br>
|
||||
<b>French Recognition Result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/german_0_res.png" alt="German recognition result" width="500"/>
|
||||
<br>
|
||||
<b>German Recognition Result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/korean_1_res.jpg" alt="Korean recognition result" width="500"/>
|
||||
<br>
|
||||
<b>Korean Recognition Result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/ru_0.jpeg" alt="Russian recognition result" width="500"/>
|
||||
<br>
|
||||
<b>Russian Recognition Result</b>
|
||||
</div>
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/th_0_res.jpg" alt="Thai recognition result" width="500"/>
|
||||
<br>
|
||||
<b>Thai recognition result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/el_0_res.jpg" alt="Greek recognition result" width="500"/>
|
||||
<br>
|
||||
<b>Greek recognition result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/ar_0_res.jpg" alt="Arabic OCR Result" width="500"/>
|
||||
<br>
|
||||
<b>Arabic recognition Result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/hi_0_res.jpg" alt="Hindi OCR Result" width="500"/>
|
||||
<br>
|
||||
<b>Hindi recognition Result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/ta_0_rec.jpg" alt="Tamil OCR Result" width="500"/>
|
||||
<br>
|
||||
<b>Tamil recognition Result</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/te_0_res.png" alt="Telugu OCR Result" width="500"/>
|
||||
<br>
|
||||
<b>Telugu recognition Result</b>
|
||||
</div>
|
||||
|
||||
|
||||
## 2. Quick Start
|
||||
|
||||
You can specify the language for text recognition by using the `--lang` parameter when running the general OCR pipeline in the command line:
|
||||
|
||||
```bash
|
||||
# Use the `--lang` parameter to specify the French recognition model
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png \
|
||||
--lang fr \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
For explanations of the other command-line parameters, please refer to the [Command Line Usage](../../pipeline_usage/OCR.en.md#21-command-line) section of the general OCR pipeline documentation. After running, the results will be displayed in the terminal:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_ocr_french01.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': True, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'page_index': None, 'model_settings': {'use_doc_orientation_classify': False, 'use_doc_unwarping': False}, 'angle': -1}, 'dt_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'text_det_params': {'limit_side_len': 64, 'limit_type': 'min', 'thresh': 0.3, 'max_side_limit': 4000, 'box_thresh': 0.6, 'unclip_ratio': 1.5}, 'text_type': 'general', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0.0, 'rec_texts': ['mifere; la profpérité & les fuccès ac-', 'compagnent l’homme induftrieux.', 'Quel eft celui qui a acquis des ri-', 'cheffes, qui eft devenu puiffant, qui', 's’eft couvert de gloire, dont l’éloge', 'retentit par-tout, qui fiege au confeil', "du Roi? C'eft celui qui bannit la pa-", "reffe de fa maifon, & qui a dit à l'oifi-", 'veté : tu es mon ennemie.'], 'rec_scores': array([0.98409832, ..., 0.98091048]), 'rec_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'rec_boxes': array([[118, ..., 81],
|
||||
...,
|
||||
[108, ..., 562]], dtype=int16)}}
|
||||
```
|
||||
|
||||
If you specify `save_path`, the visualization results will be saved to the specified path. An example of the visualized result is shown below:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/general_ocr_french01_res.png"/>
|
||||
|
||||
|
||||
You can also use Python code to specify the recognition model for a particular language when initializing the general OCR pipeline via the `lang` parameter:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
lang="fr", # Specify French recognition model with the lang parameter
|
||||
use_doc_orientation_classify=False, # Disable document orientation classification model
|
||||
use_doc_unwarping=False, # Disable text image unwarping model
|
||||
use_textline_orientation=False, # Disable text line orientation classification model
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
For more details on the `PaddleOCR` class parameters, please refer to the [Python Scripting Integration](../../pipeline_usage/OCR.en.md#22-python-script-integration) section of the general OCR pipeline documentation.
|
||||
|
||||
|
||||
## 3. Performance Comparison
|
||||
|
||||
| Model | Download Link | Accuracy on the corresponding dataset (%) | Improvement over the previous generation model (%) ｜
|
||||
|-|-|-|-|
|
||||
| korean_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/korean_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a> | 88.0| 65.0 |
|
||||
| latin_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/latin_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a> | 84.7 | 46.8 |
|
||||
| eslav_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/eslav_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/eslav_PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a> | 81.6 | 31.4 |
|
||||
| th_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/th_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/th_PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a> | 82.68 | - |
|
||||
| el_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/el_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/el_PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a> | 89.28 | - |
|
||||
| en_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/en_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a> | 85.25 | 11.0 |
|
||||
| cyrillic_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/cyrillic_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/cyrillic_PP-OCRv5_mobile_rec_pretrained.pdparams">Trained Model</a> | 80.27 | 21.2 |
|
||||
| arabic_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/arabic_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/arabic_PP-OCRv5_mobile_rec_pretrained.pdparams">Trained Model</a> | 81.27 | 22.83 |
|
||||
| devanagari_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/devanagari_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv5_mobile_rec_pretrained.pdparams">Trained Model</a> | 84.96 | 68.26 |
|
||||
| te_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/te_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/te_PP-OCRv5_mobile_rec_pretrained.pdparams">Trained Model</a> | 87.65 | 43.47 |
|
||||
| ta_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ta_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ta_PP-OCRv5_mobile_rec_pretrained.pdparams">Trained Model</a> | 94.2 | 39.23 |
|
||||
|
||||
**Notes:**
|
||||
- Korean Dataset: The latest PP-OCRv5 dataset containing 5,007 Korean text images.
|
||||
- Latin Script Language Dataset: The latest PP-OCRv5 dataset containing 3,111 images of Latin script languages.
|
||||
- East Slavic Language Dataset: The latest PP-OCRv5 dataset containing a total of 7,031 text images in Russian, Belarusian, and Ukrainian.
|
||||
- Thai dataset: The latest PP-OCRv5 constructed Thai dataset contains a total of 4,261 text images for recognition.
|
||||
- Greek dataset: The latest PP-OCRv5 constructed Greek dataset contains a total of 2,799 text images for recognition.
|
||||
- English dataset: The latest PP-OCRv5 constructed English dataset contains a total of 6,530 text images for recognition.
|
||||
- Cyrillic Dataset: The latest PP-OCRv5 Cyrillic recognition dataset contains a total of 7,600 text images.
|
||||
- Tamil Dataset: The latest PP-OCRv5 Tamil recognition dataset contains a total of 2,121 text images.
|
||||
- Telugu Dataset: The latest PP-OCRv5 Telugu recognition dataset contains a total of 2,478 text images.
|
||||
- Arabic Dataset: The latest PP-OCRv5 Arabic, Sanskrit, etc. recognition dataset contains a total of 2,676 text images.
|
||||
- Devanagari Dataset: The latest PP-OCRv5 Devanagari recognition dataset contains a total of 3,611 text images.
|
||||
|
||||
## 4. Supported Languages and Abbreviations
|
||||
|
||||
| Language | Description | Abbreviation | | Language | Description | Abbreviation |
|
||||
| --- | --- | --- | ---|--- | --- | --- |
|
||||
| Chinese | Chinese & English | ch | | Hungarian | Hungarian | hu |
|
||||
| English | English | en | | Serbian (latin) | Serbian (latin) | rs_latin |
|
||||
| French | French | fr | | Indonesian | Indonesian | id |
|
||||
| German | German | de | | Occitan | Occitan | oc |
|
||||
| Japanese | Japanese | japan | | Icelandic | Icelandic | is |
|
||||
| Korean | Korean | korean | | Lithuanian | Lithuanian | lt |
|
||||
| Traditional Chinese | Chinese Traditional | chinese_cht | | Maori | Maori | mi |
|
||||
| Afrikaans | Afrikaans | af | | Malay | Malay | ms |
|
||||
| Italian | Italian | it | | Dutch | Dutch | nl |
|
||||
| Spanish | Spanish | es | | Norwegian | Norwegian | no |
|
||||
| Bosnian | Bosnian | bs | | Polish | Polish | pl |
|
||||
| Portuguese | Portuguese | pt | | Slovak | Slovak | sk |
|
||||
| Czech | Czech | cs | | Slovenian | Slovenian | sl |
|
||||
| Welsh | Welsh | cy | | Albanian | Albanian | sq |
|
||||
| Danish | Danish | da | | Swedish | Swedish | sv |
|
||||
| Estonian | Estonian | et | | Swahili | Swahili | sw |
|
||||
| Irish | Irish | ga | | Tagalog | Tagalog | tl |
|
||||
| Croatian | Croatian | hr | | Turkish | Turkish | tr |
|
||||
| Uzbek | Uzbek | uz | | Latin | Latin | la |
|
||||
| Russian | Russian | ru | | Belarusian | Belarusian | be |
|
||||
| Ukrainian | Ukrainian | uk | | Thai | Thai | th |
|
||||
| Greek | Greek | el | | Azerbaijani | Azerbaijani | az |
|
||||
| Kurdish | Kurdish | ku | | Latvian | Latvian | lv |
|
||||
| Maltese | Maltese | mt | | Pali | Pali | pi |
|
||||
| Romanian | Romanian | ro | | Vietnamese | Vietnamese | vi |
|
||||
| Finnish | Finnish | fi | | Basque | Basque | eu |
|
||||
| Galician | Galician | gl | | Luxembourgish | Luxembourgish | lb |
|
||||
| Romansh | Romansh | rm | | Catalan | Catalan | ca |
|
||||
| Quechua | Quechua | qu | | Telugu | Telugu | te |
|
||||
| Serbian (Cyrillic) | Serbian (Cyrillic) | rs_cyrillic | | Bulgarian | Bulgarian | bg |
|
||||
| Mongolian | Mongolian | mn | | Abkhaz | Abkhaz | ab |
|
||||
| Adyghe | Adyghe | ady | | Kabardian | Kabardian | kbd |
|
||||
| Avar | Avar | av | | Dargwa | Dargwa | dar |
|
||||
| Ingush | Ingush | inh | | Chechen | Chechen | ce |
|
||||
| Lak | Lak | lki | | Lezgian | Lezgian | lez |
|
||||
| Tabasaran | Tabasaran | tab | | Kazakh | Kazakh | kk |
|
||||
| Kyrgyz | Kyrgyz | ky | | Tajik | Tajik | tg |
|
||||
| Macedonian | Macedonian | mk | | Tatar | Tatar | tt |
|
||||
| Chuvash | Chuvash | cv | | Bashkir | Bashkir | ba |
|
||||
| Mari | Mari | mhr | | Moldovan | Moldovan | mo |
|
||||
| Udmurt | Udmurt | udm | | Komi | Komi | kv |
|
||||
| Ossetian | Ossetian | os | | Buriat | Buriat | bua |
|
||||
| Kalmyk | Kalmyk | xal | | Tuvinian | Tuvinian | tyv |
|
||||
| Sakha | Sakha | sah | | Karakalpak | Karakalpak | kaa |
|
||||
| Arabic | Arabic | ar | | Persian | Persian | fa |
|
||||
| Uyghur | Uyghur | ug | | Urdu | Urdu | ur |
|
||||
| Pashto | Pashto | ps | | Kurdish | Kurdish | ku |
|
||||
| Sindhi | Sindhi | sd | | Balochi | Balochi | bal |
|
||||
| Hindi | Hindi | hi | | Marathi | Marathi | mr |
|
||||
| Nepali | Nepali | ne | | Bihari | Bihari | bh |
|
||||
| Maithili | Maithili | mai | | Old English | Old English | ang |
|
||||
| Bhojpuri | Bhojpuri | bho | | Magahi | Magahi | mah |
|
||||
| Sadri | Sadri | sck | | Newar | Newar | new |
|
||||
| Konkani | Konkani | gom | | Sanskrit | Sanskrit | sa |
|
||||
| Haryanvi | Haryanvi | bgc | | Tamil | Tamil | ta |
|
||||
|
||||
|
||||
## 5. Models and Their Supported Languages
|
||||
|
||||
| Model | Supported Languages |
|
||||
|-|-|
|
||||
| korean_PP-OCRv5_mobile_rec | Korean, English |
|
||||
| latin_PP-OCRv5_mobile_rec | French, German, Afrikaans, Italian, Spanish, Bosnian, Portuguese, Czech, Welsh, Danish, Estonian, Irish, Croatian, Uzbek, Hungarian, Serbian (Latin), Indonesian, Occitan, Icelandic, Lithuanian, Maori, Malay, Dutch, Norwegian, Polish, Slovak, Slovenian, Albanian, Swedish, Swahili, Tagalog, Turkish, Latin, Azerbaijani, Kurdish, Latvian, Maltese, Pali, Romanian, Vietnamese, Finnish, Basque, Galician, Luxembourgish, Romansh, Catalan, Quechua |
|
||||
| eslav_PP-OCRv5_mobile_rec | Russian, Belarusian, Ukrainian, English |
|
||||
| th_PP-OCRv5_mobile_rec | Thai, English |
|
||||
| el_PP-OCRv5_mobile_rec | Greek, English |
|
||||
| en_PP-OCRv5_mobile_rec | English |
|
||||
| cyrillic_PP-OCRv5_mobile_rec | Russian, Belarusian, Ukrainian, Serbian (Cyrillic), Bulgarian, Mongolian, Abkhazian, Adyghe, Kabardian, Avar, Dargin, Ingush, Chechen, Lak, Lezgin, Tabasaran, Kazakh, Kyrgyz, Tajik, Macedonian, Tatar, Chuvash, Bashkir, Malian, Moldovan, Udmurt, Komi, Ossetian, Buryat, Kalmyk, Tuvan, Sakha, Karakalpak, English |
|
||||
| arabic_PP-OCRv5_mobile_rec | Arabic, Persian, Uyghur, Urdu, Pashto, Kurdish, Sindhi, Balochi, English |
|
||||
| devanagari_PP-OCRv5_mobile_rec | Hindi, Marathi, Nepali, Bihari, Maithili, Angika, Bhojpuri, Magahi, Santali, Newari, Konkani, Sanskrit, Haryanvi, English |
|
||||
| ta_PP-OCRv5_mobile_rec | Tamil, English |
|
||||
| te_PP-OCRv5_mobile_rec | Telugu, English |
|
||||
|
||||
**note:** `en_PP-OCRv5_mobile_rec` is an optimized version based on the `PP-OCRv5` model, specifically fine-tuned for English scenarios. It demonstrates higher recognition accuracy and better adaptability when processing English text.
|
||||
@@ -0,0 +1,247 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 一、PP-OCRv5多语种文字识别介绍
|
||||
|
||||
|
||||
[PP-OCRv5](./PP-OCRv5.md) 是 PP-OCR 系列的最新一代文字识别解决方案，专注于多场景、多语种的文字识别任务。在文字类型支持方面，默认配置的识别模型可准确识别简体中文、中文拼音、繁体中文、英文和日文这五大主流文字类型。同时，PP-OCRv5还提供了覆盖106种语言的多语种文字识别能力，包括韩文、西班牙文、法文、葡萄牙文、德文、意大利文、俄罗斯文、泰文、希腊文等（具体支持语种及缩写详见[第四节](#_3)）。相较于前代 PP-OCRv3 版本，PP-OCRv5 在多语言文字识别准确率上实现了超过30%的提升。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/french_0_res.jpg" alt="法文识别结" width="500"/>
|
||||
<br>
|
||||
<b>法文识别结果</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/german_0_res.png" alt="德文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>德文识别结果</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/korean_1_res.jpg" alt="韩文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>韩文识别结果</b>
|
||||
</div>
|
||||
|
||||
<br>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/ru_0.jpeg" alt="俄文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>俄文识别结果</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/th_0_res.jpg" alt="泰文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>泰文识别结果</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/el_0_res.jpg" alt="希腊文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>希腊文识别结果</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/ar_0_res.jpg" alt="阿拉伯文文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>阿拉伯文识别结果</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/hi_0_res.jpg" alt="印地文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>印地文识别结果</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/ta_0_rec.jpg" alt="泰米尔文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>泰米尔文识别结果</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/te_0_res.png" alt="泰卢固文识别结果" width="500"/>
|
||||
<br>
|
||||
<b>泰卢固文识别结果</b>
|
||||
</div>
|
||||
|
||||
## 二、快速使用
|
||||
|
||||
您可以通过在命令行中使用 `--lang` 参数，来使用指定语种的文本识别模型进行通用 OCR 产线的推理：
|
||||
|
||||
```bash
|
||||
# 通过 `--lang` 参数指定使用法语的识别模型
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png \
|
||||
--lang fr \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
上述命令行的其他参数说明请参考通用 OCR 产线的[命令行使用方式](../../pipeline_usage/OCR.md#21), 运行后结果会被打印到终端上：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_ocr_french01.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': True, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'page_index': None, 'model_settings': {'use_doc_orientation_classify': False, 'use_doc_unwarping': False}, 'angle': -1}, 'dt_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'text_det_params': {'limit_side_len': 64, 'limit_type': 'min', 'thresh': 0.3, 'max_side_limit': 4000, 'box_thresh': 0.6, 'unclip_ratio': 1.5}, 'text_type': 'general', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0.0, 'rec_texts': ['mifere; la profpérité & les fuccès ac-', 'compagnent l’homme induftrieux.', 'Quel eft celui qui a acquis des ri-', 'cheffes, qui eft devenu puiffant, qui', 's’eft couvert de gloire, dont l’éloge', 'retentit par-tout, qui fiege au confeil', "du Roi? C'eft celui qui bannit la pa-", "reffe de fa maifon, & qui a dit à l'oifi-", 'veté : tu es mon ennemie.'], 'rec_scores': array([0.98409832, ..., 0.98091048]), 'rec_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'rec_boxes': array([[118, ..., 81],
|
||||
...,
|
||||
[108, ..., 562]], dtype=int16)}}
|
||||
```
|
||||
|
||||
若指定了`save_path`，则会保存可视化结果在`save_path`下。可视化结果如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/general_ocr_french01_res.png"/>
|
||||
|
||||
|
||||
您也可以使用 Python 代码，在通用 OCR 产线初始化时，通过 `lang` 参数来使用指定语种的识别模型：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
lang="fr" # 通过 lang 参数指定使用法语的识别模型
|
||||
use_doc_orientation_classify=False, # 通过 use_doc_orientation_classify 参数指定不使用文档方向分类模型
|
||||
use_doc_unwarping=False, # 通过 use_doc_unwarping 参数指定不使用文本图像矫正模型
|
||||
use_textline_orientation=False, # 通过 use_textline_orientation 参数指定不使用文本行方向分类模型
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
更过关于 `PaddleOCR` 类参数的说明参考通用 OCR 产线的[脚本方式集成](../../pipeline_usage/OCR.md#22-python)。
|
||||
|
||||
|
||||
## 三、指标对比
|
||||
|
||||
| 模型 | 模型下载链接 | 对应数据集精度（%） | 相比前代模型提升幅度 (%) |
|
||||
|-|-|-|-|
|
||||
| korean_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/korean_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 88.0| 65.0 |
|
||||
| latin_PP-OCRv5_mobile_rec | <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/latin_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 84.7 | 46.8 |
|
||||
| eslav_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/eslav_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/eslav_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 81.6 | 31.4 |
|
||||
| th_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/th_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/th_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 82.68 | - |
|
||||
| el_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/el_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/el_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 89.28 | - |
|
||||
| en_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/en_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 85.25 | 11.0 |
|
||||
| cyrillic_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/cyrillic_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/cyrillic_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 80.27 | 21.2 |
|
||||
| arabic_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/arabic_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/arabic_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 81.27 | 22.83 |
|
||||
| devanagari_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/devanagari_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 84.96 | 68.26 |
|
||||
| te_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/te_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/te_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 87.65 | 43.47 |
|
||||
| ta_PP-OCRv5_mobile_rec |<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ta_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ta_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a> | 94.2 | 39.23 |
|
||||
|
||||
**注：**
|
||||
- 韩语数据集：PP-OCRv5 最新构建的包含了 5007 张韩语文本图片的识别数据集。
|
||||
- 拉丁字母语言数据集：PP-OCRv5 最新构建的包含了 3111 张拉丁字母语言的文本图片识别数据集。
|
||||
- 东斯拉夫语言数据集：PP-OCRv5 最新构建的包含了俄语、 白俄罗斯语和乌克兰语共计 7031 张文本图片的识别数据集。
|
||||
- 泰文数据集：PP-OCRv5 最新构建的泰文共计 4261 张文本图片的识别数据集。
|
||||
- 希腊文数据集：PP-OCRv5 最新构建的希腊文共计 2799 张文本图片的识别数据集。
|
||||
- 英文数据集：PP-OCRv5 最新构建的英文共计 6530 张文本图片的识别数据集。
|
||||
- 西里尔语数据集：PP-OCRv5 最新构建的西里尔文共计 7600 张文本图片的识别数据集。
|
||||
- 泰米尔语数据集：PP-OCRv5 最新构建的泰米尔语共计 2121 张文本图片的识别数据集。
|
||||
- 泰卢固语数据集：PP-OCRv5 最新构建的泰卢固语共计 2478 张文本图片的识别数据集。
|
||||
- 阿拉伯语数据集：PP-OCRv5 最新构建的阿拉伯、梵语等共计 2676 张文本图片的识别数据集。
|
||||
- 天城文数据集：PP-OCRv5 最新构建的天城文共计 3611 张文本图片的识别数据集。
|
||||
|
||||
## 四、 支持语种及缩写
|
||||
|
||||
| 语种 | 描述 | 缩写 | | 语种 | 描述 | 缩写 |
|
||||
| --- | --- | --- | ---|--- | --- | --- |
|
||||
| 中文 | Chinese & English | ch | | 匈牙利文 | Hungarian | hu |
|
||||
| 英文 | English | en | | 塞尔维亚文（latin） | Serbian(latin) | rs_latin |
|
||||
| 法文 | French | fr | | 印度尼西亚文 | Indonesian | id |
|
||||
| 德文 | German | de | | 欧西坦文 | Occitan | oc |
|
||||
| 日文 | Japanese | japan | | 冰岛文 | Icelandic | is |
|
||||
| 韩文 | Korean | korean | | 立陶宛文 | Lithuanian | lt |
|
||||
| 中文繁体 | Chinese Traditional | chinese_cht | | 毛利文 | Maori | mi |
|
||||
| 南非荷兰文 | Afrikaans | af | | 马来文 | Malay | ms |
|
||||
| 意大利文 | Italian | it | | 荷兰文 | Dutch | nl |
|
||||
| 西班牙文 | Spanish | es | | 挪威文 | Norwegian | no |
|
||||
| 波斯尼亚文 | Bosnian | bs | | 波兰文 | Polish | pl |
|
||||
| 葡萄牙文 | Portuguese | pt | | 斯洛伐克文 | Slovak | sk |
|
||||
| 捷克文 | Czech | cs | | 斯洛文尼亚文 | Slovenian | sl |
|
||||
| 威尔士文 | Welsh | cy | | 阿尔巴尼亚文 | Albanian | sq |
|
||||
| 丹麦文 | Danish | da | | 瑞典文 | Swedish | sv |
|
||||
| 爱沙尼亚文 | Estonian | et | | 西瓦希里文 | Swahili | sw |
|
||||
| 爱尔兰文 | Irish | ga | | 塔加洛文 | Tagalog | tl |
|
||||
| 克罗地亚文 | Croatian | hr | | 土耳其文 | Turkish | tr |
|
||||
| 乌兹别克文 | Uzbek | uz | | 拉丁文 | Latin | la |
|
||||
| 俄罗斯文 | Russian | ru | | 白俄罗斯文 | Belarusian | be |
|
||||
| 乌克兰文 | Ukranian | uk | | 泰文 | Thai | th |
|
||||
| 希腊文 | Greek | el | | 阿塞拜疆文 | Azerbaijani | az |
|
||||
| 库尔德文 | Kurdish | ku | |拉脱维亚文 | Latvian|lv |
|
||||
| 马耳他文 | Maltese | mt | |巴利文 | Pali| pi |
|
||||
|罗马尼亚文 | Romanian | ro | |越南文 | Vietnamese| vi |
|
||||
| 芬兰文 | Finnish | fi | | 巴斯克文 | Basque | eu |
|
||||
| 加利西亚文| Galician | gl | | 卢森堡文 | Luxembourgish | lb |
|
||||
| 罗曼什文 | Romansh | rm | | 加泰罗尼亚文 | Catalan | ca |
|
||||
| 克丘亚文 | Quechua |qu | | 泰卢固文 |Telugu |te |
|
||||
| 塞尔维亚语（西里尔字母） | Serbian (Cyrillic) | rs_cyrillic | | 保加利亚文 | Bulgarian | bg |
|
||||
| 蒙古文 | Mongolian | mn | | 阿布哈兹文 | Abkhaz | ab |
|
||||
| 阿迪赫文 | Adyghe | ady | | 卡巴尔达文 | Kabardian | kbd |
|
||||
| 阿瓦尔文 | Avar | av | | 达尔格瓦文 | Dargwa | dar |
|
||||
| 印古什文 | Ingush | inh | | 车臣文 | Chechen | ce |
|
||||
| 拉克文 | Lak | lki | | 列兹金文 | Lezgian | lez |
|
||||
| 塔巴萨兰文 | Tabasaran | tab | | 哈萨克文 | Kazakh | kk |
|
||||
| 吉尔吉斯文 | Kyrgyz | ky | | 塔吉克文 | Tajik | tg |
|
||||
| 马其顿文 | Macedonian | mk | | 鞑靼文 | Tatar | tt |
|
||||
| 楚瓦什文 | Chuvash | cv | | 巴什基尔文 | Bashkir | ba |
|
||||
| 马里文 | Mari | mhr | | 莫尔多瓦文 | Moldovan | mo |
|
||||
| 乌德穆尔特文 | Udmurt | udm | | 科米文 | Komi | kv |
|
||||
| 奥塞梯文 | Ossetian | os | | 布里亚特文 | Buriat | bua |
|
||||
| 卡尔梅克文 | Kalmyk | xal | | 图瓦文 | Tuvinian | tyv |
|
||||
| 萨哈文 | Sakha | sah | | 卡拉卡尔帕克语 | Karakalpak | kaa |
|
||||
| 阿拉伯文 | Arabic | ar | | 波斯文 | Persian | fa |
|
||||
| 维吾尔文 | Uyghur | ug | | 乌尔都文 | Urdu | ur |
|
||||
| 普什图文 | Pashto | ps | | 库尔德文 | Kurdish | ku |
|
||||
| 信德文 | Sindhi | sd | | 俾路支文 | Balochi | bal |
|
||||
| 印地文 | Hindi | hi | | 马拉地文 | Marathi | mr |
|
||||
| 尼泊尔文 | Nepali | ne | | 比哈尔文 | Bihari | bh |
|
||||
| 迈蒂利文 | Maithili | mai | | 古英文 | Old English | ang |
|
||||
| 博杰普尔文 | Bhojpuri | bho | | 马加希文 | Magahi | mah |
|
||||
| 萨达里文 | Sadri | sck | | 尼瓦尔文 | Newar | new |
|
||||
| 孔卡尼文 | Konkani | gom | | 梵文 | Sanskrit | sa |
|
||||
| 哈里亚纳文 | Haryanvi | bgc | | 泰米尔语 | Tamil | ta |
|
||||
|
||||
## 五、模型及其支持的语种
|
||||
|
||||
| 模型 | 支持语种 |
|
||||
|-|-|
|
||||
| PP-OCRv5_server_rec | 简体中文、繁体中文、英文、日文 |
|
||||
| PP-OCRv5_mobile_rec | 简体中文、繁体中文、英文、日文 |
|
||||
| korean_PP-OCRv5_mobile_rec | 韩文、英文 |
|
||||
| latin_PP-OCRv5_mobile_rec |法文、德文、南非荷兰文、意大利文、西班牙文、波斯尼亚文、葡萄牙文、捷克文、威尔士文、丹麦文、爱沙尼亚文、爱尔兰文、克罗地亚文、乌兹别克文、匈牙利文、塞尔维亚文（latin）、印度尼西亚文、欧西坦文、冰岛文、立陶宛文、毛利文、马来文、荷兰文、挪威文、波兰文、斯洛伐克文、斯洛文尼亚文、阿尔巴尼亚文、瑞典文、西瓦希里文、塔加洛文、土耳其文、拉丁文、阿塞拜疆文、库尔德文、拉脱维亚文、马耳他文、巴利文、罗马尼亚文、越南文、芬兰文、巴斯克文、加利西亚文、卢森堡文、罗曼什文、加泰罗尼亚文、克丘亚文|
|
||||
| eslav_PP-OCRv5_mobile_rec | 俄罗斯文、白俄罗斯文、乌克兰文、英文 |
|
||||
| th_PP-OCRv5_mobile_rec | 泰文、英文 |
|
||||
| el_PP-OCRv5_mobile_rec | 希腊文、英文 |
|
||||
| en_PP-OCRv5_mobile_rec | 英文 |
|
||||
| cyrillic_PP-OCRv5_mobile_rec | 俄罗斯文、白俄罗斯文、乌克兰文、塞尔维亚文（cyrillic）、保加利亚文、蒙古文、阿布哈兹文、阿迪赫文、卡巴尔达文、阿瓦尔文、达尔格瓦文、印古什文、车臣文、拉克文、列兹金文、塔巴萨兰文、哈萨克文、吉尔吉斯文、塔吉克文、马其顿文、鞑靼文、楚瓦什文、巴什基尔文、马里文、莫尔多瓦文、乌德穆尔特文、科米文、奥塞梯文、布里亚特文、卡尔梅克文、图瓦文、萨哈文、卡拉卡尔帕克文、英文 |
|
||||
| arabic_PP-OCRv5_mobile_rec | 阿拉伯文、波斯文、维吾尔文、乌尔都文、普什图文、库尔德文、信德文、俾路支文、英文|
|
||||
| devanagari_PP-OCRv5_mobile_rec | 印地文，马拉地文，尼泊尔文，比哈尔文，迈蒂利文，古英文，博杰普尔文，马加希文，萨达里文，尼瓦尔文，孔卡尼文，梵文，哈里亚纳文、英文 |
|
||||
| ta_PP-OCRv5_mobile_rec | 泰米尔文、英文 |
|
||||
| te_PP-OCRv5_mobile_rec| 泰卢固文、英文 |
|
||||
|
||||
|
||||
**注：** `en_PP-OCRv5_mobile_rec` 是在 `PP-OCRv5` 模型基础上，针对英文场景进行了定向优化，在处理英文文本时表现出更高的识别精度和更强的场景适应能力。
|
||||
@@ -0,0 +1,226 @@
|
||||
# 1. PP-OCRv6 Introduction
|
||||
|
||||
**PP-OCRv6** is the latest generation of the PP-OCR universal text recognition solution. Built on the newly designed PPLCNetV4 unified backbone, it offers tiny, small, and medium tiers targeting edge/IoT, mobile/desktop, and server scenarios respectively. PP-OCRv6 achieves a major breakthrough in language coverage—the medium/small tiers support 50 languages with a single unified model, including Simplified Chinese, Traditional Chinese, English, Japanese, and 46 Latin-script languages (tiny supports 49, excluding Japanese). On our in-house multi-scenario benchmark, PP-OCRv6_medium achieves +5.1% recognition accuracy and +4.6% detection Hmean over PP-OCRv5_server, with 2.37× GPU inference speedup; with only 34.5M parameters, it surpasses VLMs such as Qwen3-VL-235B and GPT-5.5 in accuracy.
|
||||
|
||||
Main contributions:
|
||||
|
||||
1. **Unified and Scalable Model Family**: A three-tier OCR model family spanning 1.5M to 34.5M parameters. The medium tier achieves 86.2% detection Hmean and 83.2% recognition accuracy, serving as production-ready infrastructure for industrial deployment and large-scale data pipelines.
|
||||
2. **Tailored Lightweight Architectural Innovations**: (i) LCNetV4: a MetaFormer-style lightweight backbone with structural reparameterization; (ii) RepLKFPN: a detection neck with dilated reparameterizable depthwise convolutions for large receptive fields; (iii) EncoderWithLightSVTR: a recognition neck with local-global attention and additive skip connections.
|
||||
3. **Extensive Multi-Language and Scenario Generalization**: A single model scaled to support 50 languages and diverse challenging industrial scenes (e.g., digital displays, dot-matrix characters, tire prints), significantly improving OCR performance in scenarios traditionally underserved by general-purpose VLMs.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/v6acc_opt.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">Performance comparison between PP-OCRv6, PP-OCRv5, and Vision-Language Models. Left: text detection average Hmean (%); Right: text recognition weighted average accuracy (%).</p>
|
||||
|
||||
|
||||
# 2. Key Technical Improvements
|
||||
|
||||
## 2.1 Unified Backbone: PPLCNetV4
|
||||
|
||||
**LCNetV4Block**: Following the MetaFormer paradigm, each layer is decomposed into a Token Mixer and a Channel Mixer. Given input feature $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$:
|
||||
|
||||
$$\hat{\mathbf{x}} = \text{SE}(\text{DW}(\mathbf{x})) + \mathbf{x}$$
|
||||
|
||||
$$\mathbf{y} = W_2\,\sigma(W_1\,\hat{\mathbf{x}}) + \hat{\mathbf{x}}$$
|
||||
|
||||
where $\text{DW}(\cdot)$ is a 3×3 depthwise convolution (Token Mixer), SE is an optional channel attention module, $W_1 \in \mathbb{R}^{2C \times C}$ and $W_2 \in \mathbb{R}^{C \times 2C}$ form the Channel Mixer with expansion ratio 2, and $\sigma$ is GELU activation.
|
||||
|
||||
**Task-Adaptive Downsampling**: The same backbone serves both tasks via different stride strategies—detection mode uses standard stride-2 spatial downsampling producing multi-scale feature maps (stride 4/8/16/32); recognition mode uses asymmetric stride $(2,1)$ at Stage 3/4, reducing height only while preserving width, followed by height-axis average pooling to produce 1-D sequential features for CTC/NRTR decoding.
|
||||
|
||||
**Comparison with LCNetV3**:
|
||||
|
||||
| Design Aspect | LCNetV3 | LCNetV4 |
|
||||
|--------------|---------|---------|
|
||||
| Architecture | MobileNet-style (DW→SE→PW) | MetaFormer (TokenMixer + ChannelMixer) |
|
||||
| Channel Interaction | Single 1×1 PW Conv | Expand(2×)→Act→Compress + residual |
|
||||
| Spatial Mixing | Plain DW Conv | RepDWConv (3×3 + 1×1 + identity) |
|
||||
| BN Initialization | Standard | Zero-init on compress BN |
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/backbone.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<p align="center">PPLCNetV4 backbone architecture.</p>
|
||||
|
||||
## 2.2 Detection Module
|
||||
|
||||
- **RepLKFPN**: Lightweight large-kernel FPN using DilatedReparamBlock (7×7 depthwise conv + dilated branches), 31% fewer parameters than PP-OCRv5's RSEFPN (118K vs 172K) with receptive field expanded from 3×3 to 7×7.
|
||||
- **Auxiliary Deep Supervision**: Prediction heads at P2, P3, P4 levels for stronger gradient signals during training.
|
||||
- **DiceBCE Loss**: Combined DiceLoss + Focal Loss for better per-pixel supervision on small and dense text.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/ppocrv6_det_pip_ori.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<p align="center">PP-OCRv6 detection module architecture.</p>
|
||||
|
||||
## 2.3 Recognition Module
|
||||
|
||||
- **EncoderWithLightSVTR Neck**: Local context modeling (1×7 depthwise conv) + global self-attention (1-2 Transformer layers), with additive skip connections (instead of concatenation in PP-OCRv5) to reduce parameters.
|
||||
- **Multi-Head Decoder**: CTCHead for efficient parallel inference; NRTRHead for auxiliary training supervision (removed at inference).
|
||||
- **Tiny Model Design**: No neck (direct reshape + FC), trained with knowledge distillation from the medium model.
|
||||
- **Multilingual Unification**: Dictionary extended with ~200 diacritical characters, enabling single-model 50-language coverage.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/rec.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<p align="center">PP-OCRv6 recognition module architecture.</p>
|
||||
|
||||
# 3. Key Metrics
|
||||
|
||||
## 3.1 Text Detection
|
||||
|
||||
Text detection Hmean (%) on our in-house multi-scenario benchmark (16 categories):
|
||||
|
||||
| Model | AVG | HW-CN | HW-EN | Print-CN | Print-EN | TC | Anc. | JP | Blur | Emo. | Warp | Pin. | Art. | Tab. | Rot. | Indus. | Gen. |
|
||||
|-------|-----|-------|-------|----------|----------|-----|------|-----|------|------|------|------|------|------|------|--------|------|
|
||||
| **PP-OCRv6_medium** | **86.2** | **83.7** | 84.0 | **95.1** | **93.7** | **86.3** | **80.2** | **84.3** | **94.1** | 99.6 | **88.6** | **74.0** | **69.0** | 96.8 | **93.8** | **73.3** | **82.8** |
|
||||
| **PP-OCRv6_small** | **84.1** | 80.5 | **87.1** | 94.2 | 93.6 | 85.7 | 72.6 | 82.3 | 92.6 | 99.7 | 87.6 | 69.6 | 65.3 | 95.6 | 93.7 | 67.6 | 78.2 |
|
||||
| **PP-OCRv6_tiny** | **80.6** | 79.4 | 85.9 | 93.1 | 92.3 | 83.7 | 63.0 | 76.6 | 89.3 | **99.8** | 86.1 | 59.0 | 60.1 | 94.7 | 91.0 | 62.0 | 73.8 |
|
||||
| PP-OCRv5_server | 81.6 | 80.3 | 84.1 | 94.5 | 91.7 | 81.5 | 67.6 | 77.2 | 90.1 | 96.2 | 87.6 | 67.1 | 67.3 | **97.1** | 80.0 | 64.3 | 79.7 |
|
||||
| PP-OCRv5_mobile | 75.2 | 74.4 | 77.7 | 90.5 | 91.0 | 82.3 | 58.1 | 72.7 | 87.4 | 93.6 | 82.7 | 57.5 | 52.5 | 92.8 | 64.7 | 52.8 | 72.1 |
|
||||
| Gemini-3.1-Pro | 46.8 | 53.4 | 56.5 | 47.3 | 47.6 | 39.0 | 45.8 | 38.2 | 50.0 | 68.1 | 44.6 | 40.6 | 65.2 | 26.9 | 22.1 | 52.5 | 50.2 |
|
||||
| GPT-5.5 | 45.6 | 42.4 | 58.5 | 50.2 | 51.9 | 35.0 | 26.7 | 42.0 | 49.1 | 97.5 | 37.7 | 36.3 | 52.0 | 71.0 | 10.0 | 36.2 | 32.6 |
|
||||
| Qwen3-VL-235B | 38.3 | 56.5 | 66.0 | 41.7 | 37.0 | 19.3 | 13.1 | 27.0 | 38.5 | 81.2 | 28.5 | 33.0 | 68.3 | 19.6 | 2.1 | 48.4 | 32.3 |
|
||||
|
||||
## 3.2 Text Recognition
|
||||
|
||||
Text recognition accuracy (%) on our in-house multi-scenario benchmark (15 categories):
|
||||
|
||||
| Model | W-Avg | HW-CN | HW-EN | Print-CN | Print-EN | TC | Anc. | JP | Conf. | Spec. | Gen. | Pin. | Art. | Indus. | Screen | Card |
|
||||
|-------|-------|-------|-------|----------|----------|-----|------|-----|-------|-------|------|------|------|--------|--------|------|
|
||||
| **PP-OCRv6_medium** | **83.2** | **62.1** | 67.8 | **91.5** | **94.1** | **78.6** | **72.4** | **90.5** | **64.9** | **61.7** | **87.5** | **78.1** | **71.2** | **77.4** | **82.5** | **88.1** |
|
||||
| **PP-OCRv6_small** | **81.3** | 57.6 | 61.1 | 90.5 | 93.3 | 77.0 | 71.1 | 88.2 | 64.1 | 60.2 | 85.7 | 75.9 | 68.4 | 76.4 | 79.7 | 86.9 |
|
||||
| **PP-OCRv6_tiny** | 73.5 | 40.1 | 39.3 | 86.7 | 88.4 | 65.0 | 68.4 | 89.8 | 52.3 | 57.1 | 78.0 | 65.4 | 54.7 | 62.1 | 71.2 | 80.5 |
|
||||
| PP-OCRv5_server | 78.1 | 58.0 | 59.6 | 90.1 | 85.1 | 74.7 | 60.4 | 73.7 | 59.4 | 56.8 | 86.5 | 74.4 | 64.0 | 70.2 | 68.1 | 87.6 |
|
||||
| PP-OCRv5_mobile | 73.7 | 41.7 | 50.9 | 86.0 | 86.0 | 72.0 | 57.8 | 75.8 | 55.7 | 54.8 | 80.7 | 72.5 | 54.0 | 59.3 | 57.6 | 81.7 |
|
||||
| Qwen3-VL-235B | 74.9 | 49.7 | **73.2** | 82.3 | 86.2 | 76.4 | 33.6 | 66.2 | 56.1 | 49.0 | 82.5 | 76.5 | 69.6 | 74.7 | 73.8 | 78.7 |
|
||||
| Gemini-3.1-Pro | 71.4 | 46.4 | 73.0 | 80.0 | 90.5 | 69.5 | 18.0 | 67.2 | 54.4 | 50.3 | 74.6 | 75.9 | 63.1 | 69.1 | 73.2 | 75.9 |
|
||||
| GPT-5.5 | 64.2 | 19.2 | 56.9 | 75.7 | 82.2 | 57.5 | 63.7 | 58.6 | 49.1 | 48.3 | 67.7 | 50.4 | 53.0 | 62.4 | 67.7 | 71.1 |
|
||||
|
||||
## 3.3 End-to-End Inference Speed (s/image)
|
||||
|
||||
Tested on 200 images (general + document scenes), including image I/O, pre/post-processing, and model inference.
|
||||
|
||||
| Hardware | Backend | PP-OCRv6_medium | PP-OCRv6_small | PP-OCRv6_tiny | PP-OCRv5_server | PP-OCRv5_mobile | PP-OCRv4_mobile |
|
||||
|----------|---------|-----------------|----------------|---------------|-----------------|-----------------|-----------------|
|
||||
| NVIDIA A100 | PaddlePaddle | 0.29 | 0.25 | 0.13 | 0.32 | 0.25 | 0.14 |
|
||||
| NVIDIA A100 | TensorRT | -- | 0.32 | 0.16 | -- | 0.33 | 0.16 |
|
||||
| NVIDIA V100 | PaddlePaddle | 0.72 | 0.49 | 0.21 | 0.66 | 0.50 | 0.25 |
|
||||
| NVIDIA V100 | ONNX Runtime | 0.67 | 0.53 | 0.29 | 0.77 | 0.46 | 0.27 |
|
||||
| NVIDIA V100 | TensorRT | 0.77 | 0.60 | 0.23 | 0.73 | 0.59 | 0.27 |
|
||||
| Intel Xeon 8350C | PaddlePaddle | 2.05 | 0.79 | 0.32 | 2.04 | 0.80 | 0.62 |
|
||||
| Intel Xeon 8350C | OpenVINO | 1.40 | 0.59 | 0.20 | 7.30 | 0.78 | 0.60 |
|
||||
| Intel Xeon 8350C | ONNX Runtime | 3.31 | 0.61 | 0.22 | 6.36 | 0.61 | 0.49 |
|
||||
| Apple M4 | PaddlePaddle | 8.82 | 3.07 | 0.96 | >10 | 5.82 | 5.65 |
|
||||
| Apple M4 | ONNX Runtime | 5.55 | 1.29 | 0.35 | 7.20 | 1.10 | 1.02 |
|
||||
|
||||
- PP-OCRv6_medium matches or outperforms PP-OCRv5_server on all platforms: 1.1× faster on A100 (0.29s vs 0.32s), 1.15× on V100 ONNX Runtime (0.67s vs 0.77s), 5.2× on Intel Xeon OpenVINO (1.40s vs 7.30s).
|
||||
- PP-OCRv6_small matches PP-OCRv5_mobile in latency on most platforms with higher accuracy; 1.9× faster on Apple M4 PaddlePaddle (3.07s vs 5.82s).
|
||||
- PP-OCRv6_tiny is the fastest model across all platforms: 6.1× over PP-OCRv5_mobile on Apple M4 PaddlePaddle (0.96s vs 5.82s), 3.9× on Intel Xeon OpenVINO (0.20s vs 0.78s), reaching 0.13s on A100.
|
||||
|
||||
# 4. Visualization
|
||||
|
||||
## 4.1 Detection Comparison
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/ppocrv6_det_vis.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">Text detection comparison. Left to right: PP-OCRv6_medium, PP-OCRv5_server, Gemini-3.1-Pro, GPT-5.5.</p>
|
||||
|
||||
## 4.2 Hallucination Comparison
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/huanjue.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">PP-OCRv6_medium vs VLMs hallucination comparison. PP-OCRv6 faithfully reproduces visual text content, while VLMs introduce hallucinated corrections based on linguistic priors.</p>
|
||||
|
||||
## 4.3 End-to-End OCR Comparison
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/case1.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/case2.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/case3.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">End-to-end OCR comparison between PP-OCRv6_medium and PP-OCRv5_server across Chinese, English, Japanese, artistic fonts, industrial characters, rotated text, pinyin, and dot-matrix characters.</p>
|
||||
|
||||
# 5. Quick Start
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
# Default: PP-OCRv6_medium
|
||||
ocr = PaddleOCR(
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png")
|
||||
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
|
||||
```bash
|
||||
# CLI usage
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False
|
||||
```
|
||||
|
||||
**Using Transformers Engine:**
|
||||
|
||||
PP-OCRv6 supports inference via Hugging Face Transformers (requires `transformers>=5.8.0`):
|
||||
|
||||
```python
|
||||
from paddleocr import TextRecognition
|
||||
|
||||
model = TextRecognition(
|
||||
model_name="PP-OCRv6_medium_rec",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict(input="general_ocr_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
```
|
||||
|
||||
**Using High-Performance Inference (ONNX Runtime backend):**
|
||||
|
||||
Enable the high-performance inference plugin with `enable_hpi=True`:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
enable_hpi=True,
|
||||
)
|
||||
result = ocr.predict("general_ocr_002.png")
|
||||
```
|
||||
|
||||
> The HPI plugin requires additional installation. See [High-Performance Inference Guide](../../inference_deployment/local_inference/high_performance_inference.md).
|
||||
|
||||
# 6. Deployment and Custom Development
|
||||
|
||||
* **Multi-OS Support**: Compatible with Windows, Linux, and Mac.
|
||||
* **Multi-Hardware Support**: Supports NVIDIA GPU, Intel CPU, Kunlun, Ascend, and more.
|
||||
* **High-Performance Inference Plugin**: See [High-Performance Inference Guide](../../inference_deployment/local_inference/high_performance_inference.md).
|
||||
* **Serving Deployment**: See [Serving Deployment Guide](../../inference_deployment/serving/serving.md).
|
||||
* **Custom Development**: Supports custom dataset training, dictionary extension, and model fine-tuning. See [Text Detection Tutorial](../../module_usage/text_detection.md) and [Text Recognition Tutorial](../../module_usage/text_recognition.md).
|
||||
@@ -0,0 +1,258 @@
|
||||
# 一、PP-OCRv6简介
|
||||
|
||||
**PP-OCRv6** 是 PP-OCR 最新一代通用文字识别解决方案。PP-OCRv6 基于全新设计的 PPLCNetV4 统一骨干网络，提供 tiny、small、medium 三档模型，分别面向端侧/IoT、移动端/桌面端、服务端场景。PP-OCRv6 在语言覆盖方面实现重大突破，medium/small 档单一模型统一支持简体中文、繁体中文、英文、日文及 46 种拉丁语系语言共 50 种语言（tiny 档支持 49 种，不含日文）。在内部多场景综合评估集上，PP-OCRv6_medium 相比 PP-OCRv5_server 识别精度提升 5.1%、检测精度提升 4.6%，同时 GPU 推理速度提升 2.37×；以仅 34.5M 参数的规模，精度超越 Qwen3-VL-235B、GPT-5.5 等大型视觉语言模型。
|
||||
|
||||
PP-OCRv6 的主要贡献如下：
|
||||
|
||||
1. **统一可扩展的模型族**：提供覆盖 1.5M 至 34.5M 参数的三档完整 OCR 模型族。medium 档达到 86.2% 检测 Hmean 和 83.2% 识别准确率，可作为工业部署和大规模数据管线的高效生产级基础设施。
|
||||
2. **面向 OCR 的轻量级架构创新**：提出一系列专为 OCR 任务定制的轻量级架构组件——(i) LCNetV4：集成结构重参数化的 MetaFormer 风格轻量骨干；(ii) RepLKFPN：利用膨胀可重参数化深度卷积实现大感受野的检测颈部；(iii) EncoderWithLightSVTR：基于局部-全局注意力和加性跳跃连接的识别颈部。
|
||||
3. **广泛的多语言与多场景泛化**：单一模型扩展至支持 50 种语言和多种挑战性工业场景（如数码显示屏、点阵字符、轮胎印字等），显著提升了传统通用视觉语言模型难以覆盖的专业场景 OCR 性能。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/v6acc_opt.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：PP-OCRv6 与 PP-OCRv5 及视觉语言模型的性能对比。左：文本检测平均 Hmean（%）；右：文本识别加权平均准确率（%）。</p>
|
||||
|
||||
# 二、核心技术升级
|
||||
|
||||
## 1. 统一骨干网络 PPLCNetV4
|
||||
|
||||
PP-OCRv6 采用全新设计的 PPLCNetV4 作为检测和识别的统一骨干网络，核心创新包括：
|
||||
|
||||
**LCNetV4Block**：遵循 MetaFormer 范式，将每层解耦为 Token Mixer 和 Channel Mixer。设输入特征 $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$，Block 计算如下：
|
||||
|
||||
$$\hat{\mathbf{x}} = \text{SE}(\text{DW}(\mathbf{x})) + \mathbf{x}$$
|
||||
|
||||
$$\mathbf{y} = W_2\,\sigma(W_1\,\hat{\mathbf{x}}) + \hat{\mathbf{x}}$$
|
||||
|
||||
其中 $\text{DW}(\cdot)$ 是 3×3 深度卷积（Token Mixer），SE 是可选的通道注意力模块，$W_1 \in \mathbb{R}^{2C \times C}$、$W_2 \in \mathbb{R}^{C \times 2C}$ 构成扩展比为 2 的 Channel Mixer，$\sigma$ 为 GELU 激活。
|
||||
|
||||
**Task-Adaptive Downsampling**：同一骨干通过不同下采样策略服务两个任务——检测模式使用标准 stride-2 空间下采样产出多尺度特征图（stride 4/8/16/32）；识别模式在 Stage 3/4 使用非对称 stride $(2,1)$，仅缩减高度保留宽度，经 height-axis 平均池化后产出 1-D 序列特征用于 CTC/NRTR 解码。
|
||||
|
||||
**与 LCNetV3 对比**：
|
||||
|
||||
| 设计维度 | LCNetV3 | LCNetV4 |
|
||||
|---------|---------|---------|
|
||||
| 架构范式 | MobileNet-style (DW→SE→PW) | MetaFormer (TokenMixer + ChannelMixer) |
|
||||
| 通道交互 | 单个 1×1 PW Conv | Expand(2×)→Act→Compress + 残差 |
|
||||
| 空间混合 | 普通 DW Conv | RepDWConv（3×3 + 1×1 + identity 三分支） |
|
||||
| BN 初始化 | 标准 | Compress 层 BN 零初始化 |
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/backbone.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：PPLCNetV4 骨干网络结构</p>
|
||||
|
||||
## 2. 检测模块升级
|
||||
|
||||
- **RepLKFPN**：轻量级大核特征金字塔，使用 DilatedReparamBlock（7×7 深度卷积 + 膨胀分支），相比 PP-OCRv5 的 RSEFPN 参数减少 31%（118K vs 172K），同时感受野从 3×3 扩大到 7×7。
|
||||
- **辅助深度监督**：在 P2、P3、P4 层级添加预测头，训练时提供更强梯度信号。
|
||||
- **DiceBCE Loss**：组合 DiceLoss + Focal Loss，对小目标和密集文本提供更好的逐像素监督。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/ppocrv6_det_pip_ori.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：PP-OCRv6 检测模块结构</p>
|
||||
|
||||
## 3. 识别模块升级
|
||||
|
||||
- **EncoderWithLightSVTR 颈部**：局部上下文建模（1×7 深度卷积）+ 全局自注意力（1-2 层 Transformer），通过加性跳跃连接（而非 PP-OCRv5 的拼接）减少参数。
|
||||
- **多头解码器**：CTCHead 用于高效并行推理，NRTRHead 用于训练时辅助监督（推理时移除）。
|
||||
- **Tiny 模型特殊设计**：无颈部（直接 reshape + FC），使用 medium 模型蒸馏训练。
|
||||
- **多语言统一**：字典扩展约 200 个带变音符号字符，实现单模型 48 语言覆盖。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/rec.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：PP-OCRv6 识别模块结构</p>
|
||||
|
||||
# 三、关键指标
|
||||
|
||||
## 1. 文本检测指标
|
||||
|
||||
在内部多场景基准上对 16 类场景进行文本检测 Hmean(%) 评测：
|
||||
|
||||
| 模型 | AVG | 手写CN | 手写EN | 印刷CN | 印刷EN | 繁体 | 古籍 | 日文 | 模糊 | 表情 | 扭曲 | 拼音 | 艺术字 | 表格 | 旋转 | 工业 | 通用 |
|
||||
|------|-----|--------|--------|--------|--------|------|------|------|------|------|------|------|--------|------|------|------|------|
|
||||
| **PP-OCRv6_medium** | **86.2** | **83.7** | 84.0 | **95.1** | **93.7** | **86.3** | **80.2** | **84.3** | **94.1** | 99.6 | **88.6** | **74.0** | **69.0** | 96.8 | **93.8** | **73.3** | **82.8** |
|
||||
| **PP-OCRv6_small** | **84.1** | 80.5 | **87.1** | 94.2 | 93.6 | 85.7 | 72.6 | 82.3 | 92.6 | 99.7 | 87.6 | 69.6 | 65.3 | 95.6 | 93.7 | 67.6 | 78.2 |
|
||||
| **PP-OCRv6_tiny** | **80.6** | 79.4 | 85.9 | 93.1 | 92.3 | 83.7 | 63.0 | 76.6 | 89.3 | **99.8** | 86.1 | 59.0 | 60.1 | 94.7 | 91.0 | 62.0 | 73.8 |
|
||||
| PP-OCRv5_server | 81.6 | 80.3 | 84.1 | 94.5 | 91.7 | 81.5 | 67.6 | 77.2 | 90.1 | 96.2 | 87.6 | 67.1 | 67.3 | **97.1** | 80.0 | 64.3 | 79.7 |
|
||||
| PP-OCRv5_mobile | 75.2 | 74.4 | 77.7 | 90.5 | 91.0 | 82.3 | 58.1 | 72.7 | 87.4 | 93.6 | 82.7 | 57.5 | 52.5 | 92.8 | 64.7 | 52.8 | 72.1 |
|
||||
| Gemini-3.1-Pro | 46.8 | 53.4 | 56.5 | 47.3 | 47.6 | 39.0 | 45.8 | 38.2 | 50.0 | 68.1 | 44.6 | 40.6 | 65.2 | 26.9 | 22.1 | 52.5 | 50.2 |
|
||||
| GPT-5.5 | 45.6 | 42.4 | 58.5 | 50.2 | 51.9 | 35.0 | 26.7 | 42.0 | 49.1 | 97.5 | 37.7 | 36.3 | 52.0 | 71.0 | 10.0 | 36.2 | 32.6 |
|
||||
| Qwen3-VL-235B | 38.3 | 56.5 | 66.0 | 41.7 | 37.0 | 19.3 | 13.1 | 27.0 | 38.5 | 81.2 | 28.5 | 33.0 | 68.3 | 19.6 | 2.1 | 48.4 | 32.3 |
|
||||
|
||||
PP-OCRv6_medium 平均 Hmean 达 86.2%，相比 PP-OCRv5_server 提升 4.6 个百分点，在日文、古籍、旋转文本、工业字符等场景提升尤为显著。
|
||||
|
||||
## 2. 文本识别指标
|
||||
|
||||
在内部多场景基准上对 15 类场景进行文本识别准确率(%) 评测：
|
||||
|
||||
| 模型 | W-Avg | 手写CN | 手写EN | 印刷CN | 印刷EN | 繁体 | 古籍 | 日文 | 易混淆 | 特殊字符 | 通用 | 拼音 | 艺术字 | 工业 | 屏幕 | 卡片 |
|
||||
|------|-------|--------|--------|--------|--------|------|------|------|--------|----------|------|------|--------|------|------|------|
|
||||
| **PP-OCRv6_medium** | **83.2** | **62.1** | 67.8 | **91.5** | **94.1** | **78.6** | **72.4** | **90.5** | **64.9** | **61.7** | **87.5** | **78.1** | **71.2** | **77.4** | **82.5** | **88.1** |
|
||||
| **PP-OCRv6_small** | **81.3** | 57.6 | 61.1 | 90.5 | 93.3 | 77.0 | 71.1 | 88.2 | 64.1 | 60.2 | 85.7 | 75.9 | 68.4 | 76.4 | 79.7 | 86.9 |
|
||||
| **PP-OCRv6_tiny** | 73.5 | 40.1 | 39.3 | 86.7 | 88.4 | 65.0 | 68.4 | 89.8 | 52.3 | 57.1 | 78.0 | 65.4 | 54.7 | 62.1 | 71.2 | 80.5 |
|
||||
| PP-OCRv5_server | 78.1 | 58.0 | 59.6 | 90.1 | 85.1 | 74.7 | 60.4 | 73.7 | 59.4 | 56.8 | 86.5 | 74.4 | 64.0 | 70.2 | 68.1 | 87.6 |
|
||||
| PP-OCRv5_mobile | 73.7 | 41.7 | 50.9 | 86.0 | 86.0 | 72.0 | 57.8 | 75.8 | 55.7 | 54.8 | 80.7 | 72.5 | 54.0 | 59.3 | 57.6 | 81.7 |
|
||||
| Qwen3-VL-235B | 74.9 | 49.7 | **73.2** | 82.3 | 86.2 | 76.4 | 33.6 | 66.2 | 56.1 | 49.0 | 82.5 | 76.5 | 69.6 | 74.7 | 73.8 | 78.7 |
|
||||
| Gemini-3.1-Pro | 71.4 | 46.4 | 73.0 | 80.0 | 90.5 | 69.5 | 18.0 | 67.2 | 54.4 | 50.3 | 74.6 | 75.9 | 63.1 | 69.1 | 73.2 | 75.9 |
|
||||
| GPT-5.5 | 64.2 | 19.2 | 56.9 | 75.7 | 82.2 | 57.5 | 63.7 | 58.6 | 49.1 | 48.3 | 67.7 | 50.4 | 53.0 | 62.4 | 67.7 | 71.1 |
|
||||
|
||||
PP-OCRv6_medium 加权平均准确率 83.2%，相比 PP-OCRv5_server 提升 5.1%，在日文(+16.8%)、古籍(+12.0%)、屏幕显示(+14.4%) 等类别提升显著。即使是仅 1.1M 参数的 PP-OCRv6_tiny，也超越了 4/5 的 VLM 模型。
|
||||
|
||||
## 4. 端到端推理速度（s/image）
|
||||
|
||||
在 200 张图像（通用场景 + 文档场景）上测试端到端 OCR 产线速度，包含读图、前后处理、模型推理全流程。
|
||||
|
||||
| 硬件 | 推理后端 | PP-OCRv6_medium | PP-OCRv6_small | PP-OCRv6_tiny | PP-OCRv5_server | PP-OCRv5_mobile | PP-OCRv4_mobile |
|
||||
|------|---------|-----------------|----------------|---------------|-----------------|-----------------|-----------------|
|
||||
| NVIDIA A100 | PaddlePaddle | 0.29 | 0.25 | 0.13 | 0.32 | 0.25 | 0.14 |
|
||||
| NVIDIA A100 | TensorRT | -- | 0.32 | 0.16 | -- | 0.33 | 0.16 |
|
||||
| NVIDIA V100 | PaddlePaddle | 0.72 | 0.49 | 0.21 | 0.66 | 0.50 | 0.25 |
|
||||
| NVIDIA V100 | ONNX Runtime | 0.67 | 0.53 | 0.29 | 0.77 | 0.46 | 0.27 |
|
||||
| NVIDIA V100 | TensorRT | 0.77 | 0.60 | 0.23 | 0.73 | 0.59 | 0.27 |
|
||||
| Intel Xeon 8350C | PaddlePaddle | 2.05 | 0.79 | 0.32 | 2.04 | 0.80 | 0.62 |
|
||||
| Intel Xeon 8350C | OpenVINO | 1.40 | 0.59 | 0.20 | 7.30 | 0.78 | 0.60 |
|
||||
| Intel Xeon 8350C | ONNX Runtime | 3.31 | 0.61 | 0.22 | 6.36 | 0.61 | 0.49 |
|
||||
| Apple M4 | PaddlePaddle | 8.82 | 3.07 | 0.96 | >10 | 5.82 | 5.65 |
|
||||
| Apple M4 | ONNX Runtime | 5.55 | 1.29 | 0.35 | 7.20 | 1.10 | 1.02 |
|
||||
|
||||
- PP-OCRv6_medium 在所有平台上均匹配或优于 PP-OCRv5_server：A100 上快 1.1×（0.29s vs 0.32s），V100 ONNX Runtime 快 1.15×（0.67s vs 0.77s），Intel Xeon OpenVINO 快 5.2×（1.40s vs 7.30s）。
|
||||
- PP-OCRv6_small 在大多数平台上与 PP-OCRv5_mobile 速度持平但精度更高；Apple M4 PaddlePaddle 快 1.9×（3.07s vs 5.82s）。
|
||||
- PP-OCRv6_tiny 是所有平台上最快的模型，Apple M4 PaddlePaddle 快 6.1×（0.96s vs 5.82s），Intel Xeon OpenVINO 快 3.9×（0.20s vs 0.78s），A100 上仅需 0.13s。
|
||||
|
||||
# 四、语言支持
|
||||
|
||||
PP-OCRv6 medium/small 档支持以下 50 种语言：
|
||||
|
||||
**核心语言**：简体中文、繁体中文、英文、日文
|
||||
|
||||
**拉丁语系（46种）**：法文、德文、意大利文、西班牙文、葡萄牙文、荷兰文、波兰文、罗马尼亚文、捷克文、瑞典文、挪威文、丹麦文、芬兰文、匈牙利文、土耳其文、越南文、印尼文、马来文、阿塞拜疆文、南非荷兰文、波斯尼亚文、克罗地亚文、威尔士文、爱沙尼亚文、爱尔兰文、冰岛文、库尔德文、立陶宛文、拉脱维亚文、马耳他文、毛利文、奥克文、斯洛伐克文、斯洛文尼亚文、阿尔巴尼亚文、斯瓦希里文、他加禄文、乌兹别克文、拉丁文、塞尔维亚文(拉丁)、加泰罗尼亚文、巴斯克文、加利西亚文、卢森堡文、罗曼什文、克丘亚文
|
||||
|
||||
> PP-OCRv6_tiny 档支持 49 种语言（不含日文，以避免约 4000 个汉字/假名字符对 1.1M 参数输出层的影响）。
|
||||
|
||||
# 五、效果可视化
|
||||
|
||||
## 1. 检测效果对比
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/ppocrv6_det_vis.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：文本检测效果对比。从左到右：PP-OCRv6_medium、PP-OCRv5_server、Gemini-3.1-Pro、GPT-5.5。</p>
|
||||
|
||||
## 2. 幻觉对比
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/huanjue.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：PP-OCRv6_medium 与 VLM 的幻觉对比。PP-OCRv6 忠实还原图像中的文字内容，而 VLM 基于语言先验进行"纠正"，引入了图像中不存在的幻觉。</p>
|
||||
|
||||
## 3. 端到端 OCR 效果对比
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/case1.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/case2.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv6/case3.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
<p align="center">图：PP-OCRv6_medium 与 PP-OCRv5_server 端到端 OCR 效果对比，涵盖中文、英文、日文、艺术字、工业字符、旋转文本、拼音、点阵字符等场景。</p>
|
||||
|
||||
# 六、快速使用
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
# 默认使用 PP-OCRv6_medium
|
||||
ocr = PaddleOCR(
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png")
|
||||
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
|
||||
```
|
||||
|
||||
```bash
|
||||
# 命令行使用
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False
|
||||
```
|
||||
|
||||
**使用 Transformers 引擎推理：**
|
||||
|
||||
PP-OCRv6 支持通过 Hugging Face Transformers 引擎进行推理（需安装 `transformers>=5.8.0`）：
|
||||
|
||||
```python
|
||||
from paddleocr import TextRecognition
|
||||
|
||||
model = TextRecognition(
|
||||
model_name="PP-OCRv6_medium_rec",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict(input="general_ocr_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
```
|
||||
|
||||
```bash
|
||||
# 命令行方式
|
||||
paddleocr text_recognition -i general_ocr_rec_001.png \
|
||||
--text_recognition_model_name PP-OCRv6_medium_rec \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
**使用高性能推理（ONNX Runtime 后端）：**
|
||||
|
||||
通过 `enable_hpi=True` 启用高性能推理插件，底层会自动使用 ONNX Runtime 加速：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
enable_hpi=True,
|
||||
)
|
||||
result = ocr.predict("general_ocr_002.png")
|
||||
```
|
||||
|
||||
```bash
|
||||
# 命令行方式
|
||||
paddleocr ocr -i general_ocr_002.png \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--enable_hpi True
|
||||
```
|
||||
|
||||
> 高性能推理插件需额外安装，详见[高性能推理指南](../../inference_deployment/local_inference/high_performance_inference.md)。
|
||||
|
||||
# 七、部署与二次开发
|
||||
|
||||
- **多系统支持**：兼容 Windows、Linux、Mac 等主流操作系统。
|
||||
- **多硬件支持**：支持英伟达 GPU、Intel CPU、昆仑芯、昇腾等硬件推理和部署。
|
||||
- **高性能推理插件**：推荐结合高性能推理插件进一步提升推理速度，详见[高性能推理指南](../../inference_deployment/local_inference/high_performance_inference.md)。
|
||||
- **服务化部署**：支持高稳定性服务化部署方案，详见[服务化部署指南](../../inference_deployment/serving/serving.md)。
|
||||
- **二次开发能力**：支持自定义数据集训练、字典扩展、模型微调，详见[文本检测模块使用教程](../../module_usage/text_detection.md)及[文本识别模块使用教程](../../module_usage/text_recognition.md)。
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,124 @@
|
||||
## 1. PaddleOCR-VL-1.5 Introduction
|
||||
|
||||
**PaddleOCR-VL-1.5** is an advanced next-generation model of PaddleOCR-VL, achieving a new state-of-the-art accuracy of 94.5% on OmniDocBench v1.5. To rigorously evaluate robustness against real-world physical distortions—including scanning artifacts, skew, warping, screen photography, and illumination—we propose the Real5-OmniDocBench benchmark. Experimental results demonstrate that this enhanced model attains SOTA performance on the newly curated benchmark. Furthermore, we extend the model’s capabilities by incorporating seal recognition and text spotting tasks, while remaining a 0.9B ultra-compact VLM with high efficiency.
|
||||
|
||||
### Key Metrics:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/paddleocr-vl-1.5_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### **Core Features**
|
||||
|
||||
1. With a **parameter size of 0.9B**, PaddleOCR-VL-1.5 **achieves xxx% accuracy on OmniDocBench v1.5**, surpassing the previous SOTA model PaddleOCR-VL. Significant improvements are observed in **table, formula, and text understanding.**
|
||||
|
||||
2. **It introduces an innovative approach to document parsing by supporting irregular-shaped localization**, enabling accurate polygonal detection under skewed and curved document conditions. Evaluations across five real-world scenarios—scanning, curving, skewing, screen-photo capture, and light variation—demonstrate superior performance over mainstream open-source and proprietary models.
|
||||
|
||||
3. The model introduces **text spotting (text-line localization and recognition)**, along with **seal recognition**, with all corresponding metrics **setting new SOTA results** in their respective tasks.
|
||||
|
||||
4. PaddleOCR-VL-1.5 further strengthens its capability in **specialized scenarios and multilingual recognition.** Recognition performance is improved for **rare characters, ancient texts, multilingual tables, underlines, and checkboxes,** and language coverage is extended to include **China's Tibetan script and Bengali.**
|
||||
|
||||
5. The model supports **automatic cross-page table merging** and **cross-page paragraph heading recognition**, effectively mitigating content fragmentation issues in **long-document parsing.**
|
||||
|
||||
|
||||
## 2. Model Architecture
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/PaddleOCR-VL-1.5.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
## 3. Model Performance
|
||||
|
||||
### 1. OmniDocBench v1.5
|
||||
|
||||
#### PaddleOCR-VL achieves SOTA performance for overall, text, formula, tables and reading order on OmniDocBench v1.5.
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/omnidocbenchv1.5_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **Notes:**
|
||||
> - Performance metrics are cited from the [OmniDocBench official leaderboard](https://opendatalab.com/omnidocbench), except for Gemini-3 Pro, Qwen3-VL-235B-A22B-Instruct and our model, which were evaluated independently.
|
||||
|
||||
|
||||
### 2. Real5-OmniDocBench
|
||||
|
||||
#### Across all five diverse and challenging scenarios—scanning, warping, screen-photography, illumination, and skew—PaddleOCR-VL-1.5 consistently sets new SOTA records
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/real5-omnidocbench_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **Notes:**
|
||||
> - Real5-OmniDocBench is a brand-new benchmark oriented toward real-world scenarios, which we constructed based on the OmniDocBench v1.5 dataset. The dataset comprises five distinct scenarios: Scanning, Warping, Screen-photography, Illumination, and Skew. For further details, please refer to [Real5-OmniDocBench](https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench).
|
||||
|
||||
|
||||
|
||||
## 4、Inference and deployment Performance {#4-inference-and-deployment-performance}
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/inference_performance.png" width="600"/>
|
||||
</div>
|
||||
|
||||
> **Notes:**
|
||||
> - End-to-End Inference Performance Comparison on OmniDocBench v1.5. PDF documents were processed in batches of 512 on a single NVIDIA A100 GPU. The reported end-to-end runtime includes both PDF rendering and Markdown generation. All methods rely on their built-in PDF parsing modules and default DPI settings to reflect out-of-the-box performance.
|
||||
|
||||
|
||||
## 5. Visualization
|
||||
|
||||
### Real-word Document Parsing
|
||||
|
||||
|
||||
#### Illumination
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/light.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Skew
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/skew.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Screen Photography
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/screen.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Scanning
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/scaning.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
#### Warping
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/curving.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### Text Spotting
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/spotting.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### Seal Recognition
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/seal.jpg" width="800"/>
|
||||
</div>
|
||||
@@ -0,0 +1,124 @@
|
||||
## 1. PaddleOCR-VL-1.5 简介
|
||||
|
||||
**PaddleOCR-VL-1.5** 在1.0版本上进行了进一步能力的扩展和升级优化，在文档解析 OmniDocBench v1.5 上取得了 94.5% 的更高的新 SOTA（最佳）结果。为了严格评估其对现实世界物理畸变的鲁棒性——包括扫描伪影、倾斜、弯曲、屏摄和光照变化——我们提出了 Real5-OmniDocBench 基准测试。实验结果表明，该增强模型在这一新构建的基准测试中各个场景都达到了 SOTA 性能。此外，我们通过加入印章识别和文字检测识别任务扩展了模型能力，同时保持了 0.9B 的超紧凑 VLM 规模和高效率。
|
||||
|
||||
### **关键指标:**
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/paddleocr-vl-1.5_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### **核心特性:**
|
||||
|
||||
1. **文档解析的SOTA性能：** 凭借 0.9B 的参数量，PaddleOCR-VL-1.5 在 OmniDocBench v1.5 上达到了 94.5% 的准确率，超越了之前的 SOTA 模型 PaddleOCR-VL。在表格、公式和文本识别方面观察到了显著提升。
|
||||
|
||||
2. **现实5大场景文档解析的SOTA性能：** 引入了一种创新的文档解析方法，支持不规则形状定位，能够在文档倾斜和弯曲条件下实现精确的多边形检测。在扫描、弯曲、倾斜、屏摄和光照变化这五个现实场景的评估中，表现优于主流的开源和闭源模型。
|
||||
|
||||
3. **0.9B紧凑架构扩充能力：** 模型引入了文本行定位与识别 以及 印章识别，所有相关指标均在各自任务中创下了新的 SOTA 结果。
|
||||
|
||||
4. **强化多元素识别能力：** PaddleOCR-VL-1.5 进一步增强了在特定场景和多语言识别方面的能力。针对特殊符号、古籍、多语言表格、下划线和复选框的识别性能得到提升，语言覆盖范围扩展至包括中国藏文和孟加拉语。
|
||||
|
||||
5. **长文档跨页解析：** 模型支持跨页表格自动合并和跨页段落标题识别，有效缓解了长文档解析中的内容碎片化问题。
|
||||
|
||||
|
||||
## 二、技术架构
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/PaddleOCR-VL-1.5.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
## 三、 模型性能
|
||||
|
||||
### 1. OmniDocBench v1.5
|
||||
|
||||
#### PaddleOCR-VL 在 OmniDocBench v1.5 上的整体、文本、公式、表格和阅读顺序中均达到最先进的性能。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/omnidocbenchv1.5_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **注:**
|
||||
> - 性能指标引自 [OmniDocBench 官方排行榜](https://opendatalab.com/omnidocbench), Gemini-3 Pro、Qwen3-VL-235B-A22B-Instruct 和我们的模型除外。
|
||||
|
||||
|
||||
### 2. Real5-OmniDocBench
|
||||
|
||||
#### 在扫描、扭曲、屏摄、光照和倾斜这五个多样化且具挑战性的场景中，PaddleOCR-VL-1.5 均创下了新的 SOTA 记录。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/real5-omnidocbench_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **注:**
|
||||
> - Real5-OmniDocBench 是我们基于 OmniDocBench v1.5 数据集构建的、面向真实场景的全新基准测试。该数据集包含五个不同场景：扫描 (Scanning)、扭曲 (Warping)、屏摄 (Screen-photography)、光照 (Illumination) 和倾斜 (Skew)。更多详情请参阅 [Real5-OmniDocBench](https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench).
|
||||
|
||||
|
||||
|
||||
## 4、推理部署性能 {#4推理部署性能}
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/inference_performance.png" width="600"/>
|
||||
</div>
|
||||
|
||||
> **注:**
|
||||
> - OmniDocBench v1.5 上的端到端推理性能对比。PDF 文档在单张 NVIDIA A100 GPU 上以 512 的 batch size 进行处理。报告的端到端运行时间包含 PDF 渲染和 Markdown 生成。所有方法均依赖其内置的 PDF 解析模块和默认 DPI 设置，以反映开箱即用的性能。
|
||||
|
||||
|
||||
## 5. 可视化
|
||||
|
||||
### 现实场景文档
|
||||
|
||||
|
||||
#### 光照
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/light.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### 倾斜
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/skew.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### 屏摄
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/screen.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### 扫描
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/scaning.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
#### 弯曲/扭曲
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/curving.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### 文本定位与识别
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/spotting.jpg" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### 印章识别
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/seal.jpg" width="800"/>
|
||||
</div>
|
||||
@@ -0,0 +1,146 @@
|
||||
## 1. Introduction to PaddleOCR-VL-1.6
|
||||
|
||||
**PaddleOCR-VL-1.6** further optimizes PaddleOCR-VL-1.5 by systematically analyzing under-optimized areas in the current model, applying targeted data optimization, and adopting refined post-training strategies. It achieves a new state-of-the-art (SOTA) result of 96.33% on the OmniDocBench v1.6 document parsing benchmark. PaddleOCR-VL-1.6 also reaches SOTA performance across all scenarios on Real5-OmniDocBench, a benchmark designed to evaluate robustness against real-world physical distortions. In addition, PaddleOCR-VL-1.6 outperforms PaddleOCR-VL-1.5 on three subtasks: seal recognition, text detection and recognition, and chart recognition, while still maintaining an ultra-compact 0.9B-parameter VLM and high efficiency.
|
||||
|
||||
### **Key Metrics:**
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/paddleocr-vl-1.6_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### **Core Features:**
|
||||
|
||||
1. **SOTA performance in document parsing:** With only 0.9B parameters, PaddleOCR-VL-1.6 achieves 96.33% accuracy on OmniDocBench v1.6, surpassing the previous SOTA model, PaddleOCR-VL-1.5. Significant improvements are observed in table, formula, and text recognition.
|
||||
|
||||
2. **SOTA performance for document parsing across five real-world scenarios:** PaddleOCR-VL-1.6 offers stronger robustness and practicality in real-world use cases. In evaluations across five real-world distortion scenarios—scanning, warping, skew, screen photography, and illumination variation—it outperforms mainstream open-source and closed-source models.
|
||||
|
||||
3. **Enhanced multi-element recognition capabilities:** Beyond improved layout parsing, PaddleOCR-VL-1.6 substantially strengthens recognition of complex tables, ancient books, and rare Chinese characters, while further improving three existing capabilities: chart parsing, seal recognition, and text detection and recognition.
|
||||
|
||||
4. **Compact 0.9B architecture:** PaddleOCR-VL-1.6 follows the compact 0.9B architecture of the PaddleOCR-VL series, enabling zero-cost adaptation and drop-in replacement.
|
||||
|
||||
|
||||
## 2. Technical Architecture
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/overall.png" width="800"/>
|
||||
</div>
|
||||
|
||||
1. **Data engine:** Starting from PaddleOCR-VL-1.5, the data engine systematically identifies under-optimized areas in PaddleOCR-VL-1.5, designs strategies for obtaining high-quality labels, and performs targeted data optimization.
|
||||
2. **Progressive post-training strategy:** Data is carefully categorized from three perspectives: quality, difficulty, and improvement value. The training weights of PaddleOCR-VL-1.5 are loaded, and a three-stage post-training strategy—continued pre-training, supervised fine-tuning, and reinforcement learning—is applied according to different data quality levels to steadily improve model performance.
|
||||
|
||||
|
||||
## 3. Model Performance
|
||||
|
||||
### 1. OmniDocBench v1.6
|
||||
|
||||
#### PaddleOCR-VL-1.6 achieves state-of-the-art performance on OmniDocBench v1.6 in overall metrics, text, formulas, and tables. It also delivers leading results in reading order.
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/table2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **Note:**
|
||||
> - Performance metrics are cited from the [official OmniDocBench leaderboard](https://opendatalab.com/omnidocbench).
|
||||
|
||||
|
||||
### 2. Real5-OmniDocBench
|
||||
|
||||
#### PaddleOCR-VL-1.6 sets new SOTA records across five diverse and challenging scenarios: scanning, warping, screen photography, illumination, and skew.
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/table3.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **Note:**
|
||||
> - Real5-OmniDocBench is a new real-world benchmark built by the PaddleOCR team based on the OmniDocBench v1.5 dataset. It contains five scenarios: Scanning, Warping, Screen-photography, Illumination, and Skew. For more details, see [Real5-OmniDocBench](https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench).
|
||||
|
||||
|
||||
|
||||
## 4. Inference and Deployment Performance
|
||||
|
||||
PaddleOCR-VL-1.6 and PaddleOCR-VL-1.5 use exactly the same model architecture design, so they have identical inference speeds. For details about the inference speed of PaddleOCR-VL-1.5, refer to [PaddleOCR-VL-1.5 inference speed](./PaddleOCR-VL-1.5.md#4-inference-and-deployment-performance).
|
||||
|
||||
|
||||
## 5. Visualization
|
||||
|
||||
### Comparison with PaddleOCR-VL-1.5
|
||||
|
||||
|
||||
#### Ancient Book Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/ancient1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/ancient2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/ancient3.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Chart Parsing
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/chart1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/chart2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Formula Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/fomula1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/fomula2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/fomula3.png" width="800"/>
|
||||
</div>
|
||||
|
||||
#### Rare Chinese Character Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/rare-cha1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/rare-cha2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
#### Seal Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/seal1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/seal2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### Table Recognition
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/table1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/table2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/table3.png" width="800"/>
|
||||
</div>
|
||||
@@ -0,0 +1,146 @@
|
||||
## 1. PaddleOCR-VL-1.6 简介
|
||||
|
||||
**PaddleOCR-VL-1.6** 在 PaddleOCR-VL-1.5 的基础上进一步优化，通过系统分析当前模型中的欠优化区域，进行针对性数据优化，并采用精细化的后训练策略，在文档解析基准 OmniDocBench v1.6 上取得 96.33% 的最新 SOTA（最佳）结果。同时，PaddleOCR-VL-1.6 在面向真实世界物理畸变鲁棒性的 Real5-OmniDocBench 基准测试中，也在各个场景下均达到 SOTA 性能。此外，在印章识别、文字检测识别和图表识别三个子任务上，PaddleOCR-VL-1.6 均领先 PaddleOCR-VL-1.5，同时依然保持 0.9B 超紧凑 VLM 参数量和高效率。
|
||||
|
||||
### **关键指标:**
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/paddleocr-vl-1.6_metrics.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### **核心特性:**
|
||||
|
||||
1. **文档解析的SOTA性能：** 凭借 0.9B 的参数量，PaddleOCR-VL-1.6 在 OmniDocBench v1.6 上达到了 96.33% 的准确率，超越了之前的 SOTA 模型 PaddleOCR-VL-1.5。在表格、公式和文本识别方面观察到了显著提升。
|
||||
|
||||
2. **现实5大场景文档解析的SOTA性能：** 具备更强的鲁棒性和真实场景的实用性。在扫描、弯曲、倾斜、屏摄和光照变化这五个真实扰动场景的评估中，表现优于主流的开源和闭源模型。
|
||||
|
||||
3. **强化多元素识别能力：** 除了版面解析能力的提升外，PaddleOCR-VL-1.6 进一步大幅度增强了对复杂表格，古籍 和 生僻字的识别能力，同时在 图表解析，印章识别，文字检测识别这三个原有能力上进一步提升。
|
||||
|
||||
4. **0.9B紧凑架构：** 沿用 PaddleOCR-VL系列的 0.9B 紧凑构架，零成本适配，即换即用。
|
||||
|
||||
|
||||
## 二、技术架构
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/overall.png" width="800"/>
|
||||
</div>
|
||||
|
||||
1. **数据引擎:** 以 PaddleOCR-VL-1.5 为出发点，系统性定位 PaddleOCR-VL-1.5 的欠优化区域，并设计高质量标签获取策略，针对性的进行数据优化。
|
||||
2. **渐进式后训练策略** 从质量，难度，提升价值三个角度精细化划分数据，加载 PaddleOCR-VL-1.5 训练权重，结合不同的数据质量进行 继续预训练，监督微调，强化学习 三阶段的后训练策略，稳步提升模型性能。
|
||||
|
||||
|
||||
## 三、 模型性能
|
||||
|
||||
### 1. OmniDocBench v1.6
|
||||
|
||||
#### PaddleOCR-VL-1.6 在 OmniDocBench v1.6 上的整体指标、文本、公式、表格均达到最先进的性能，其中在在阅读顺序方面，PaddleOCR-VL-1.6也取得了较为领先的指标。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/table2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **注:**
|
||||
> - 性能指标引自 [OmniDocBench 官方排行榜](https://opendatalab.com/omnidocbench)。
|
||||
|
||||
|
||||
### 2. Real5-OmniDocBench
|
||||
|
||||
#### 在扫描、扭曲、屏摄、光照和倾斜这五个多样化且具挑战性的场景中，PaddleOCR-VL-1.6 均创下了新的 SOTA 记录。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/table3.png" width="800"/>
|
||||
</div>
|
||||
|
||||
> **注:**
|
||||
> - Real5-OmniDocBench 是 PaddleOCR团队 基于 OmniDocBench v1.5 数据集构建的、面向真实场景的全新基准测试。该数据集包含五个不同场景：扫描 (Scanning)、扭曲 (Warping)、屏摄 (Screen-photography)、光照 (Illumination) 和倾斜 (Skew)。更多详情请参阅 [Real5-OmniDocBench](https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench).
|
||||
|
||||
|
||||
|
||||
## 四、推理部署性能
|
||||
|
||||
PaddleOCR-VL-1.6 和 PaddleOCR-VL-1.5 采用完全相同的模型架构设计，因此有完全相同的推理速度。关于PaddleOCR-VL-1.5推理速度的说明可以参考 [PaddleOCR-VL-1.5推理速度](./PaddleOCR-VL-1.5.md#4推理部署性能) 。
|
||||
|
||||
|
||||
## 5. 可视化
|
||||
|
||||
### 和 PaddleOCR-VL-1.5 的对比
|
||||
|
||||
|
||||
#### 古籍识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/ancient1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/ancient2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/ancient3.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### 图表解析
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/chart1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/chart2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### 公式识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/fomula1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/fomula2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/fomula3.png" width="800"/>
|
||||
</div>
|
||||
|
||||
#### 生僻字识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/rare-cha1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/rare-cha2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
#### 印章识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/seal1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/seal2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### 表格识别
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/table1.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/table2.png" width="800"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_6/comparison/table3.png" width="800"/>
|
||||
</div>
|
||||
@@ -0,0 +1,196 @@
|
||||
## 1. PaddleOCR-VL Introduction
|
||||
|
||||
**PaddleOCR-VL** is a SOTA and resource-efficient model tailored for document parsing. Its core component is PaddleOCR-VL-0.9B, a compact yet powerful vision-language model (VLM) that integrates a NaViT-style dynamic resolution visual encoder with the ERNIE-4.5-0.3B language model to enable accurate element recognition. This innovative model efficiently supports 109 languages and excels in recognizing complex elements (e.g., text, tables, formulas, and charts), while maintaining minimal resource consumption. Through comprehensive evaluations on widely used public benchmarks and in-house benchmarks, PaddleOCR-VL achieves SOTA performance in both page-level document parsing and element-level recognition. It significantly outperforms existing solutions, exhibits strong competitiveness against top-tier VLMs, and delivers fast inference speeds. These strengths make it highly suitable for practical deployment in real-world scenarios.
|
||||
|
||||
### Key Metrics:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/allmetric.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### **Core Features**
|
||||
|
||||
1. **Compact yet Powerful VLM Architecture:** We present a novel vision-language model that is specifically designed for resource-efficient inference, achieving outstanding performance in element recognition. By integrating a NaViT-style dynamic high-resolution visual encoder with the lightweight ERNIE-4.5-0.3B language model, we significantly enhance the model’s recognition capabilities and decoding efficiency. This integration maintains high accuracy while reducing computational demands, making it well-suited for efficient and practical document processing applications.
|
||||
|
||||
|
||||
2. **SOTA Performance on Document Parsing:** PaddleOCR-VL achieves state-of-the-art performance in both page-level document parsing and element-level recognition. It significantly outperforms existing pipeline-based solutions and exhibiting strong competitiveness against leading vision-language models (VLMs) in document parsing. Moreover, it excels in recognizing complex document elements, such as text, tables, formulas, and charts, making it suitable for a wide range of challenging content types, including handwritten text and historical documents. This makes it highly versatile and suitable for a wide range of document types and scenarios.
|
||||
|
||||
|
||||
3. **Multilingual Support:** PaddleOCR-VL Supports 109 languages, covering major global languages, including but not limited to Chinese, English, Japanese, Latin, and Korean, as well as languages with different scripts and structures, such as Russian (Cyrillic script), Arabic, Hindi (Devanagari script), and Thai. This broad language coverage substantially enhances the applicability of our system to multilingual and globalized document processing scenarios.
|
||||
|
||||
|
||||
## 2. Model Architecture
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/methods/paddleocrvl.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
## 3. Model Performance
|
||||
|
||||
### Page-Level Document Parsing
|
||||
|
||||
|
||||
#### OmniDocBench v1.5
|
||||
|
||||
##### PaddleOCR-VL achieves SOTA performance for overall, text, formula, tables and reading order on OmniDocBench v1.5.
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/omni15.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
#### OmniDocBench v1.0
|
||||
|
||||
##### PaddleOCR-VL achieves SOTA performance for almost all metrics of overall, text, formula, tables and reading order on OmniDocBench v1.0.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/omni10.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
### Element-level Recognition
|
||||
|
||||
#### Text
|
||||
|
||||
**Comparison of OmniDocBench-OCR-block Performance**
|
||||
|
||||
PaddleOCR-VL’s robust and versatile capability in handling diverse document types, establishing it as the leading method in the OmniDocBench-OCR-block performance evaluation.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/omnibenchocr.png" width="800"/>
|
||||
</div>
|
||||
|
||||
**Comparison of In-house-OCR-block Performance**
|
||||
|
||||
In-house-OCR provides a evaluation of performance across multiple languages and text types. Our model demonstrates outstanding accuracy with the lowest edit distances in all evaluated scripts.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhouseocr.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Table
|
||||
|
||||
|
||||
**Comparison of In-house-Table Performance**
|
||||
|
||||
Our self-built evaluation set contains diverse types of table images, such as Chinese, English, mixed Chinese-English, and tables with various characteristics like full, partial, or no borders, book/manual formats, lists, academic papers, merged cells, as well as low-quality, watermarked, etc. PaddleOCR-VL achieves remarkable performance across all categories.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhousetable.png" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### Formula
|
||||
|
||||
**Comparison of In-house-Formula Performance**
|
||||
|
||||
|
||||
In-house-Formula evaluation set contains simple prints, complex prints, camera scans, and handwritten formulas. PaddleOCR-VL demonstrates the best performance in every category.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhouse-formula.png" width="500"/>
|
||||
</div>
|
||||
|
||||
#### Chart
|
||||
|
||||
**Comparison of In-house-Chart Performance**
|
||||
|
||||
The evaluation set is broadly categorized into 11 chart categories, including bar-line hybrid, pie, 100% stacked bar, area, bar, bubble, histogram, line, scatterplot, stacked area, and stacked bar. PaddleOCR-VL not only outperforms expert OCR VLMs but also surpasses some 72B-level multimodal language models.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhousechart.png" width="400"/>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
## 4、Inference and deployment Performance
|
||||
|
||||
To improve the inference performance of PaddleOCR-VL, we introduce multi-threading asynchronous execution into the inference workflow. The process is divided into three main stages—data loading (e.g., rendering PDF pages as images), layout model processing, and VLM inference—each running in a separate thread. Data is transferred between adjacent stages via queues, enabling concurrent execution for higher efficiency. We measured the end-to-end inference speed and GPU usage on the OmniDocBench v1.0 dataset, processing PDF files in batches of 512 on a single NVIDIA A100 GPU.
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inference.png" width="600"/>
|
||||
</div>
|
||||
|
||||
> **Notes:** <sup>†</sup> means vLLM backend，<sup>‡</sup> means sglang backend.
|
||||
|
||||
|
||||
## 5. Visualization
|
||||
|
||||
### Comprehensive Document Parsing
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview1.jpg" width="600"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview2.jpg" width="600"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview3.jpg" width="600"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview4.jpg" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
### Text Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/text_english_arabic.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/text_handwriting_02.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
|
||||
### Table Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/table_01.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/table_02.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
### Formula Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/formula_EN.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/formula_EN.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
|
||||
### Chart Recognition
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/chart_01.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/chart_02.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
|
||||
## 6. FAQ
|
||||
|
||||
**1.How to use PaddleOCR-VL for document parsing?**
|
||||
|
||||
Please refer to our usage documentation [PaddleOCR-VL Usage](../../pipeline_usage/PaddleOCR-VL.en.md).
|
||||
|
||||
**2. How to fine-tune the PaddleOCR-VL model?**
|
||||
|
||||
Currently, we do not support fine-tuning of the model, but it is a high-priority feature and will be released soon. Please stay tuned.
|
||||
|
||||
**3. Why was my chart not recognized and how can I use chart recognition?**
|
||||
|
||||
Because our default chart recognition function is turned off, it needs to be manually turned on. Please refer to [PaddleOCR-VL Usage](../../pipeline_usage/PaddleOCR-VL.en.md) and set the use_chart_recognition为True parameters to True turn it on.
|
||||
|
||||
**4. What are the 109 supported languages?**
|
||||
|
||||
Chinese, English, Korea, Japanese, Thai, Greek, Tamil, Telugu
|
||||
|
||||
Arabic: Arabic, Persian, Uyghur, Urdu, Pashto, Kurdish, Sindhi, Balochi
|
||||
|
||||
Latin: French, German, Afrikaans, Italian, Spanish, Bosnian, Portuguese, Czech, Welsh, Danish, Estonian, Irish, Croatian, Uzbek, Hungarian, Serbian (Latin), Indonesian, Occitan, Icelandic, Lithuanian, Maori, Malay, Dutch, Norwegian, Polish, Slovak, Slovenian, Albanian, Swedish, Swahili, Tagalog, Turkish, Latin, Azerbaijani, Kurdish, Latvian, Maltese, Pali, Romanian, Vietnamese, Finnish, Basque, Galician, Luxembourgish, Romansh, Catalan, Quechua
|
||||
|
||||
Cyrillic: Russian, Belarusian, Ukrainian, Serbian (Cyrillic), Bulgarian, Mongolian, Abkhazian, Adyghe, Kabardian, Avar, Dargin, Ingush, Chechen, Lak, Lezgin, Tabasaran, Kazakh, Kyrgyz, Tajik, Macedonian, Tatar, Chuvash, Bashkir, Malian, Moldovan, Udmurt, Komi, Ossetian, Buryat, Kalmyk, Tuvan, Sakha, Karakalpak
|
||||
|
||||
Devanagari: Hindi, Marathi, Nepali, Bihari, Maithili, Angika, Bhojpuri, Magahi, Santali, Newari, Konkani, Sanskrit, Haryanvi
|
||||
|
||||
**5. If the results of the layout check are not satisfactory, what solutions can be optimized?**
|
||||
|
||||
Since layout detection is mainly trained for various document scenarios, if your test data is non-standard documents such as license plates, tickets images or ID Cards and you want to do OCR recognition, you can directly use the PaddleOCR-VL-0.9B model and turn off the layout detection model by setting use_layout_detection to False. If you find any layout detection errors, you can directly try the effect of using PaddleOCR-VL-0.9B alone.
|
||||
We recommend using the [ERNIEKit toolkit](https://github.com/PaddlePaddle/ERNIE/tree/release/v1.4) to perform Supervised Fine-Tuning (SFT) on the PaddleOCR-VL-0.9B model. For detailed steps, please refer to the [ERNIEKit documentation](https://github.com/PaddlePaddle/ERNIE/blob/release/v1.4/docs/paddleocr_vl_sft.md).
|
||||
@@ -0,0 +1,200 @@
|
||||
## 一、PaddleOCR-VL简介
|
||||
|
||||
**PaddleOCR-VL** 是一款先进、高效的文档解析模型，专为文档中的元素识别设计。其核心组件为 PaddleOCR-VL-0.9B，这是一种紧凑而强大的视觉语言模型（VLM），它由 NaViT 风格的动态分辨率视觉编码器与 ERNIE-4.5-0.3B 语言模型组成，能够实现精准的元素识别。该模型支持 109 种语言，并在识别复杂元素（如文本、表格、公式和图表）方面表现出色，同时保持极低的资源消耗。通过在广泛使用的公开基准与内部基准上的全面评测，PaddleOCR-VL 在页级级文档解析与元素级识别均达到 SOTA 表现。它显著优于现有的基于Pipeline方案和文档解析多模态方案以及先进的通用多模态大模型，并具备更快的推理速度。这些优势使其非常适合在真实场景中落地部署。
|
||||
|
||||
### **关键指标:**
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/allmetric.png" width="800"/>
|
||||
</div>
|
||||
|
||||
### **核心特性:**
|
||||
|
||||
1. **紧凑而强大的视觉语言模型架构：** 我们提出了一种新的视觉语言模型，专为资源高效的推理而设计，在元素识别方面表现出色。通过将NaViT风格的动态高分辨率视觉编码器与轻量级的ERNIE-4.5-0.3B语言模型结合，我们显著增强了模型的识别能力和解码效率。这种集成在保持高准确率的同时降低了计算需求，使其非常适合高效且实用的文档处理应用。
|
||||
|
||||
2. **文档解析的SOTA性能：** PaddleOCR-VL在页面级文档解析和元素级识别中达到了最先进的性能。它显著优于现有的基于流水线的解决方案，并在文档解析中展现出与领先的视觉语言模型（VLMs）竞争的强劲实力。此外，它在识别复杂的文档元素（如文本、表格、公式和图表）方面表现出色，使其适用于包括手写文本和历史文献在内的各种具有挑战性的内容类型。这使得它具有高度的多功能性，适用于广泛的文档类型和场景。
|
||||
|
||||
3. **多语言支持：** PaddleOCR-VL支持109种语言，覆盖了主要的全球语言，包括但不限于中文、英文、日文、拉丁文和韩文，以及使用不同文字和结构的语言，如俄语（西里尔字母）、阿拉伯语、印地语（天城文）和泰语。这种广泛的语言覆盖大大增强了我们系统在多语言和全球化文档处理场景中的适用性。
|
||||
|
||||
|
||||
|
||||
|
||||
## 二、技术架构
|
||||
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/methods/paddleocrvl.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 三、 模型性能
|
||||
|
||||
### 页面级文档解析
|
||||
|
||||
#### 1. OmniDocBench v1.5
|
||||
|
||||
##### PaddleOCR-VL 在 OmniDocBench v1.5 上的整体、文本、公式、表格和阅读顺序中均达到最先进的性能。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/omni15.png" width="800"/>
|
||||
</div>
|
||||
|
||||
#### 2. OmniDocBench v1.0
|
||||
|
||||
##### PaddleOCR-VL 在 OmniDocBench v1.0 的整体、文本、公式、表格以及阅读顺序等几乎所有评估指标上均达到了 SOTA 性能。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/omni10.png" width="800"/>
|
||||
</div>
|
||||
|
||||
### 元素级识别
|
||||
|
||||
#### 文本
|
||||
|
||||
**OmniDocBench-OCR-block**
|
||||
|
||||
PaddleOCR-VL 在处理多样化文档类型方面展现出强大而灵活的能力，使其在 OmniDocBench-OCR-block 的性能评估中成为领先方法。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/omnibenchocr.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
**In-house-OCR-block**
|
||||
|
||||
我们自建的评测集评估了模型在多语言和多文本类型下的性能。我们的模型在所有评测文字体系中均表现出卓越的准确性，并取得了最低的编辑距离。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhouseocr.png" width="800"/>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
#### 表格
|
||||
|
||||
**In-house-Table**
|
||||
|
||||
我们自建的评测集包含多种类型的表格图像，例如中文、英文、中英混合表格，以及具有不同特征的表格类型，如完整边框、部分边框、无边框、书籍/手册格式、列表、学术论文表格、合并单元格等，还包括低质量和带水印的样本。PaddleOCR-VL 在所有类别中均展现出卓越的性能。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhousetable.png" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
#### 公式
|
||||
|
||||
**In-house-Formula**
|
||||
|
||||
我们自建的评测集包含简单印刷、复杂印刷、摄像扫描以及手写公式等多种类型。PaddleOCR-VL 在所有类别中均取得了最佳性能。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhouse-formula.png" width="500"/>
|
||||
</div>
|
||||
|
||||
#### 图表
|
||||
|
||||
**In-house-Chart 结果比较**
|
||||
|
||||
我们自建的评测集涵盖 11 种主要图表类型，包括柱线混合图、饼图、100% 堆叠柱状图、面积图、柱状图、气泡图、直方图、折线图、散点图、堆叠面积图和堆叠柱状图。PaddleOCR-VL 不仅优于专业 OCR VLM 模型，还超越了一些 72B 级别的多模态语言模型。
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inhousechart.png" width="400"/>
|
||||
</div>
|
||||
|
||||
|
||||
## 四、推理部署性能
|
||||
|
||||
为了提升PaddleOCR-VL的推理性能，我们在推理工作流程中引入了多线程异步执行。该过程分为三个主要阶段：数据加载（例如，将PDF页面渲染为图像）、布局模型处理和VLM推理——每个阶段都在一个单独的线程中运行。数据通过队列在相邻阶段之间传输，从而实现并发执行以提高效率。在OmniDocBench v1.0数据集上测量了端到端推理速度和GPU使用情况，以512个PDF文件的批次在单个NVIDIA A100 GPU上进行处理。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/metrics/inference.png" width="600"/>
|
||||
</div>
|
||||
|
||||
> **Notes:** <sup>†</sup> 表示vLLM后端，<sup>‡</sup> 表示SGLang后端
|
||||
|
||||
|
||||
|
||||
## 五、可视化
|
||||
|
||||
PaddleOCR-VL能够支持多种类型的文档解析，以下是一些预测案例的展示：
|
||||
|
||||
### 端到端文档解析
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview1.jpg" width="600"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview2.jpg" width="600"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview3.jpg" width="600"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/overview4.jpg" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
### 文本识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/text_english_arabic.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/text_handwriting_02.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
|
||||
### 表格识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/table_01.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/table_02.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
### 公式识别
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/formula_EN.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/formula_EN.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
### 图表识别
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/chart_01.jpg" width="300"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/chart_02.jpg" width="300"/>
|
||||
</div>
|
||||
|
||||
|
||||
## 六、FAQ
|
||||
|
||||
**1.如何使用 PaddleOCR-VL 做文档解析 ？**
|
||||
|
||||
参考我们的使用文档 [PaddleOCR-VL使用](../../pipeline_usage/PaddleOCR-VL.md)
|
||||
|
||||
**2.如何对 PaddleOCR-VL 模型进行微调 ？**
|
||||
|
||||
目前我们暂不支持模型的微调，但已经在高优的支持中，即将发布，请保持关注。
|
||||
|
||||
**3.为什么我的图表没有识别出来，如何使用图表识别 ？**
|
||||
|
||||
因为我们默认图表识别的功能是关闭的，需要手动开启，请参考 [PaddleOCR-VL使用](../../pipeline_usage/PaddleOCR-VL.md), 设置 use_chart_recognition为True 参数来开启。
|
||||
|
||||
**4.支持的109种语言有哪些？**
|
||||
|
||||
中文、英语、韩语、日语、泰语、希腊语、泰米尔语、泰卢固语
|
||||
|
||||
阿拉伯语：阿拉伯语、波斯语、维吾尔语、乌尔都语、普什图语、库尔德语、信德语、俾路支语
|
||||
|
||||
拉丁语：法语、德语、南非荷兰语、意大利语、西班牙语、波斯尼亚语、葡萄牙语、捷克语、威尔士语、丹麦语、爱沙尼亚语、爱尔兰语、克罗地亚语、乌兹别克语、匈牙利语、塞尔维亚语（拉丁语）、印度尼西亚语、奥克语、冰岛语、立陶宛语、毛利语、马来语、荷兰语、挪威语、波兰语、斯洛伐克语、斯洛文尼亚语、阿尔巴尼亚语、瑞典语、斯瓦希里语、他加禄语、土耳其语、拉丁语、阿塞拜疆语、库尔德语、拉脱维亚语、马耳他语、巴利语、罗马尼亚语、越南语、芬兰语、巴斯克语、加利西亚语、卢森堡语、罗曼什语、加泰罗尼亚语、盖丘亚语
|
||||
|
||||
西里尔文：俄语、白俄罗斯语、乌克兰语、塞尔维亚语（西里尔文）、保加利亚语、蒙古语、阿布哈兹语、阿迪杰语、卡巴尔达语、阿瓦尔语、达尔金语、印古什语、车臣语、拉克语、列兹金语、塔巴萨兰语、哈萨克语、吉尔吉斯语、塔吉克语、马其顿语、鞑靼语、楚瓦什语、巴什基尔语、马里语、摩尔多瓦语、乌德穆尔特语、科米语、奥塞梯语、布里亚特语、卡尔梅克语、图瓦语、萨哈语、卡拉卡尔帕克语
|
||||
|
||||
天城语：印地语、马拉地语、尼泊尔语、比哈里语、迈蒂利语、安吉卡语、博杰普里语、马基语、桑塔利语、纽瓦里语、康卡尼语、梵语、哈里亚维语
|
||||
|
||||
**5.如果版面检测的结果不理想，有什么方案可以优化？**
|
||||
|
||||
由于版面检测主要针对各种文档场景训练，所以您的测试数据如果是非标准文档，如车牌，火车票或身份证图像想做OCR识别，那可以直接使用PaddleOCR-VL-0.9B的模型，通过设置 use_layout_detection 为 False 关闭版面检测模型。如果您发现有任何版面检测的错误，都可以直接尝试一下单独使用PaddleOCR-VL-0.9B的效果。
|
||||
我们推荐使用 [ERNIEKit 套件](https://github.com/PaddlePaddle/ERNIE/tree/release/v1.4) 对 PaddleOCR-VL-0.9B 模型进行有监督微调（SFT）。具体操作步骤可参考 [ERNIEKit 官方文档](https://github.com/PaddlePaddle/ERNIE/blob/release/v1.4/docs/paddleocr_vl_sft_zh.md)。
|
||||
Reference in New Issue
Block a user