chore: import upstream snapshot with attribution

2026-07-13 11:59:26 +08:00
commit e904b667c6
2490 changed files with 596352 additions and 0 deletions
@@ -0,0 +1,64 @@
+# Introduction to PP-ChatOCRV4
+**PP-ChatOCRv4** is a unique document and image intelligent analysis solution from PaddlePaddle, combining LLM, MLLM, and OCR technologies to address complex document information extraction challenges such as layout analysis, rare characters, multi-page PDFs, tables, and seal recognition. Integrated with ERNIE Bot, it fuses massive data and knowledge, achieving high accuracy and wide applicability. This pipeline also provides flexible service deployment options, supporting deployment on various hardware. Furthermore, it offers custom development capabilities, allowing you to train and fine-tune models on your own datasets, with seamless integration of trained models.
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
+</div>
+
+# Key Metrics
+
+<div align="center">
+<table>
+ <thead>
+  <tr > 
+   <th class>Solution</td> 
+   <th class>Avg Recall</td> 
+  </tr> 
+<thead>
+ <tbody>
+  <tr> 
+   <td>GPT-4o</td> 
+   <td>63.47%</td> 
+  </tr>
+  <tr> 
+   <td>PP-ChatOCRv3</td> 
+   <td class>70.08%</td> 
+  </tr> 
+  <tr> 
+   <td>Qwen2.5-VL-72B</td> 
+   <td>80.26%</td> 
+  </tr> 
+  <tr> 
+   <td><b>PP-ChatOCRv4</b></td> 
+   <td><b>85.55%</b></td> 
+  </tr> 
+ </tbody>
+</table>
+</div>
+
+# Demo
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
+</div>
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
+</div>
+
+
+# FAQ
+
+1. Does support other multimodal models?
+
+Yes, only set on pipeline configuration.
+
+2. How to reduce latency and improve throughput?
+
+Use the High-performance inference plugin, and deploy multi instances.
+
+3. How to further improve accuracy?
+
+Firstly, it is necessary to check whether the extracted visual information is correct. If the visual information is incorrect, it is necessary to visualize the visual prediction results to determine which model performs poorly, and then fine-tune train the model with more data. If the visual information is correct but cannot extract the correct information, the prompt needs to be adjusted according to the analysing about the question and answer.
@@ -0,0 +1,65 @@
+# 一、PP-ChatOCRV4简介
+**PP-ChatOCRv4**是飞桨特色的文档图像智能分析解决方案，结合了 LLM、MLLM 和 OCR 等技术，一站式解决版面分析、生僻字识别、多页 PDF 文件批量解析、复杂表格识别、印章识别等常见的复杂文档信息抽取难点问题，结合文心大模型将海量数据和知识相融合，信息抽取准确率高且应用广泛。本产线同时提供了灵活的服务化部署方式，支持在多种硬件上部署。不仅如此，本产线也提供了二次开发的能力，您可以基于本产线在您自己的数据集上训练调优，训练后的模型也可以无缝集成。
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
+</div>
+
+# 二、关键指标
+
+<div align="center">
+<table>
+ <thead>
+  <tr > 
+   <th class>Solution</td> 
+   <th class>Avg Recall</td> 
+  </tr> 
+<thead>
+ <tbody>
+  <tr> 
+   <td>GPT-4o</td> 
+   <td>63.47%</td> 
+  </tr>
+  <tr> 
+   <td>PP-ChatOCRv3</td> 
+   <td class>70.08%</td> 
+  </tr> 
+  <tr> 
+   <td>Qwen2.5-VL-72B</td> 
+   <td>80.26%</td> 
+  </tr> 
+  <tr> 
+   <td><b>PP-ChatOCRv4</b></td> 
+   <td><b>85.55%</b></td> 
+  </tr> 
+ </tbody>
+</table>
+</div>
+
+# 三、PP-ChatOCRv4 Demo示例
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
+</div>
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
+</div>
+
+# 四、使用方法和常见问题
+
+1. 多模态大模型除了DocBee，是否支持其他多模态模型？
+
+支持，只需在配置文件中进行设置即可。
+
+2. 如何降低时延、提升吞吐？
+
+无论使用哪一种服务化部署方案，都可以通过启用高性能推理插件提升模型推理速度，从而降低处理时延。
+
+此外，对于高稳定性服务化部署方案，通过调整服务配置，设置多个实例，也可以充分利用部署机器的资源，有效提升吞吐。
+
+3. 如何进一步提升精度？
+
+首先需要检查提取的视觉信息是否正确，如果视觉信息有误，则需要通过可视化视觉预测结果，判断哪个模型效果较差，从而针对性地训练微调较差的模型；如果视觉信息无误，但无法抽取正确信息，则需要根据问答的具体情况调整Prompt。