chore: import upstream snapshot with attribution
Build/Publish Develop Docs / deploy (push) Failing after 1s
PaddleOCR Code Style Check / check-code-style (push) Failing after 1s
PaddleOCR PR Tests GPU / detect-changes (push) Failing after 1s
PaddleOCR PR Tests / detect-changes (push) Failing after 1s
PaddleOCR PR Tests GPU / test-pr-gpu (push) Has been cancelled
PaddleOCR PR Tests / test-pr (push) Has been cancelled
PaddleOCR PR Tests GPU / test-pr-gpu-impl (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.13) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.8) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.9) (push) Has been cancelled
Build/Publish Develop Docs / deploy (push) Failing after 1s
PaddleOCR Code Style Check / check-code-style (push) Failing after 1s
PaddleOCR PR Tests GPU / detect-changes (push) Failing after 1s
PaddleOCR PR Tests / detect-changes (push) Failing after 1s
PaddleOCR PR Tests GPU / test-pr-gpu (push) Has been cancelled
PaddleOCR PR Tests / test-pr (push) Has been cancelled
PaddleOCR PR Tests GPU / test-pr-gpu-impl (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.13) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.8) (push) Has been cancelled
PaddleOCR PR Tests / test-pr-python (3.9) (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,358 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Chart Parsing Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Multimodal chart parsing is a cutting-edge OCR technology that focuses on automatically converting various types of visual charts (such as bar charts, line charts, pie charts, etc.) into structured data tables with formatted output. Traditional methods rely on complex pipeline designs with chart keypoint detection models, which involve many prior assumptions and tend to lack robustness. The models in this module leverage the latest VLM (Vision-Language Model) techniques and are data-driven, learning robust features from vast real-world datasets. Application scenarios include financial analysis, academic research, business reporting, and more—for instance, quickly extracting growth trend data from financial reports, experimental comparison figures from research papers, or user distribution statistics from market surveys—empowering users to transition from “viewing charts” to “using data”.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Download Link</th>
|
||||
<th>Model Size (B)</th>
|
||||
<th>Storage Size (GB)</th>
|
||||
<th>Score</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-Chart2Table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-Chart2Table_infer.tar">Inference Model</a></td>
|
||||
<td>0.58</td>
|
||||
<td>1.4</td>
|
||||
<th>80.60</th>
|
||||
<td>PP-Chart2Table is a multimodal chart parsing model developed by the PaddlePaddle team. It demonstrates exceptional performance on both Chinese and English chart parsing tasks. The team designed a specialized “Shuffled Chart Data Retrieval” training task and adopted a carefully designed token masking strategy, significantly improving performance on chart-to-table conversion. Additionally, the team enhanced the model with a high-quality data synthesis process using seed data, RAG, and LLM persona-driven generation to diversify training data. To handle large amounts of out-of-distribution (OOD) unlabeled data, a two-stage large model distillation process was used to ensure excellent adaptability and generalization to diverse real-world data. In internal Chinese-English use case evaluations, PP-Chart2Table achieved state-of-the-art performance among models of similar size and reached accuracy comparable to 7B-parameter VLMs in key scenarios.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
**Note:** The scores above are based on internal evaluation on a test set of 1801 samples, covering various chart types (bar, line, pie, etc.) across scenarios such as financial reports, regulations, and contracts. There is currently no plan for public release.
|
||||
|
||||
> ❗ **Note:** The PP-Chart2Table model was upgraded on June 27, 2025. To use the previous version, please download it [here](https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-Chart2Table_infer.bak.tar)
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. Refer to the [Installation Guide](../installation.md) for details.
|
||||
|
||||
Run the following command to get started instantly:
|
||||
|
||||
```bash
|
||||
paddleocr chart_parsing -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png'}"
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_dynamic</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr chart_parsing -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png'}" \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr chart_parsing -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png'}" \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_dynamic` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
**Note:** By default, PaddleOCR retrieves models from HuggingFace. If HuggingFace access is restricted in your environment, you can switch the model source to BOS by setting the environment variable: `PADDLE_PDX_MODEL_SOURCE="BOS"`. Support for more mainstream sources is planned.
|
||||
|
||||
You can also integrate the inference of the vision-language model into your own project. Please download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png) locally before running the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import ChartParsing
|
||||
model = ChartParsing(model_name="PP-Chart2Table")
|
||||
results = model.predict(
|
||||
input={"image": "chart_parsing_02.png"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_dynamic</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import ChartParsing
|
||||
model = ChartParsing(
|
||||
model_name="PP-Chart2Table",
|
||||
engine="transformers",
|
||||
)
|
||||
results = model.predict(
|
||||
input={"image": "chart_parsing_02.png"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import ChartParsing
|
||||
model = ChartParsing(
|
||||
model_name="PP-Chart2Table",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
results = model.predict(
|
||||
input={"image": "chart_parsing_02.png"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_dynamic` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
The output result will be:
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'chart_parsing_02.png', 'result': 'Year | Avg Revenue per 5-star Hotel (Million CNY) | Avg Profit per 5-star Hotel (Million CNY)\n2018 | 104.22 | 9.87\n2019 | 99.11 | 7.47\n2020 | 57.87 | -3.87\n2021 | 68.99 | -2.9\n2022 | 56.29 | -9.48\n2023 | 87.99 | 5.96'}}
|
||||
```
|
||||
|
||||
Explanation of output parameters:
|
||||
<ul>
|
||||
<li><code>image</code>: The path to the input image</li>
|
||||
<li><code>result</code>: The model's prediction output</li>
|
||||
</ul>
|
||||
|
||||
The visualized result is:
|
||||
|
||||
```bash
|
||||
Year | Avg Revenue per 5-star Hotel (Million CNY) | Avg Profit per 5-star Hotel (Million CNY)
|
||||
2018 | 104.22 | 9.87
|
||||
2019 | 99.11 | 7.47
|
||||
2020 | 57.87 | -3.87
|
||||
2021 | 68.99 | -2.9
|
||||
2022 | 56.29 | -9.48
|
||||
2023 | 87.99 | 5.96
|
||||
```
|
||||
|
||||
Detailed explanation of related methods and parameters:
|
||||
|
||||
* Instantiate a vision-language model with <code>ChartParsing</code>. Parameters:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, defaults to <code>PP-Chart2Table</code>.</td>
|
||||
<td><code>str | None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning</b>Model storage path.</td>
|
||||
<td><code>str | None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b> Inference device.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Examples:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code><br/>
|
||||
Defaults to GPU 0 if available; otherwise falls back to CPU.
|
||||
</td>
|
||||
<td><code>str | None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_dynamic</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Use the model's <code>predict()</code> method for inference. This returns a list of results. The module also offers a <code>predict_iter()</code> method, which behaves identically in terms of inputs and outputs but returns a generator—ideal for large datasets or memory-sensitive scenarios. Choose based on your needs.
|
||||
|
||||
<code>predict()</code> method parameters:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b> Input data (required). <br/>
|
||||
<b>Description:</b>
|
||||
Input formats vary by model.<br/>
|
||||
<ul>
|
||||
<li>For PP-Chart2Table: <code>{'image': image_path}</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>N/A</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b> Batch size. <br/>
|
||||
<b>Description:</b>
|
||||
Any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Prediction results are returned as <code>Result</code> objects for each sample, with support for printing and saving to JSON:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Explanation</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Format output using JSON indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Indentation level for pretty-printed JSON. Only works when <code>format_json=True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to escape non-ASCII characters to Unicode. If <code>False</code>, keeps characters as-is.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save results to JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>File path to save. If a directory, file will use input name as filename.</td>
|
||||
<td>N/A</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Same as in <code>print()</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Same as in <code>print()</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* You can also access the result via properties:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Property</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>json</code></td>
|
||||
<td>Returns the result in JSON format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
Currently, this module supports inference only and does not yet support fine-tuning. Fine-tuning capabilities are planned for future releases.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="2">PP-Chart2Table</td>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>53.00</td>
|
||||
<td>17863.95</td>
|
||||
<td>0.30</td>
|
||||
<td>17917.78</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>23.95</td>
|
||||
<td>12217.37</td>
|
||||
<td>0.47</td>
|
||||
<td>12269.98</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,354 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 图表解析模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
多模态图表解析是一项OCR领域的前沿技术，专注于将各类可视化图表（如柱状图、折线图、饼图等）自动转化为底层数据表，并进行格式化输出。传统方法依赖于图表关键点检测等模型进行复杂串联编排，先验假设较多，鲁棒性较差，该模块中的模型使用最新的VLM技术，数据驱动，从海量的现实数据中学习鲁棒的特征。其应用场景覆盖金融分析、学术研究、商业报告等场景——例如快速提取财报中的增长趋势数据、科研论文中的实验对比数值，或市场调研中的用户分布统计，助力用户从“看图”转向“用数”。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>模型参数规模（B）</th>
|
||||
<th>模型存储大小（GB）</th>
|
||||
<th>模型分数 </th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-Chart2Table</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-Chart2Table_infer.tar">推理模型</a></td>
|
||||
<td>0.58</td>
|
||||
<td>1.4</td>
|
||||
<th>80.60</th>
|
||||
<td>PP-Chart2Table是飞桨团队自研的一款专注于图表解析的多模态模型，在中英文图表解析任务中展现出卓越性能。团队专为图表解析设计了Shuffled Chart Data Retrieval训练任务，并结合精心设计的令牌掩码策略，显著提升其在图表转数据表任务上的性能。此外，团队通过精心设计的数据合成流程增强了PP-Chart2Table的能力，该流程利用高质量的种子数据，并结合RAG和大语言模型人格设计，以生成更丰富多样化的数据。为了处理大量未标记的分布外 (OOD) 数据，团队采用了两阶段大模型蒸馏训练过程，确保模型在广泛的真实世界数据集中具有出色的适应性和泛化能力。在内部业务的中英文场景测试中，PP-Chart2Table不仅达到同参数量级模型中的SOTA水平，更在关键场景中实现了与7B参数量级VLM模型相当的精度。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>注：以上模型分数为内部评估集模型测试结果，共1801条数据，包括了各个场景（财报、法律法规、合同等）下的各种图表类型（柱状图、折线图、饼图等）的测试样本，暂时未有计划公开。</b>
|
||||
|
||||
> ❗ <b>注</b>：PP-Chart2Table模型于 2025.6.27 升级，如需使用升级前的模型权重，请点击<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-Chart2Table_infer.bak.tar">下载链接</a>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr chart_parsing -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png'}"
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_dynamic</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr chart_parsing -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png'}" \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr chart_parsing -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png'}" \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_dynamic` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将开放文档类视觉语言模型模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import ChartParsing
|
||||
model = ChartParsing(model_name="PP-Chart2Table")
|
||||
results = model.predict(
|
||||
input={"image": "chart_parsing_02.png"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_dynamic</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import ChartParsing
|
||||
model = ChartParsing(
|
||||
model_name="PP-Chart2Table",
|
||||
engine="transformers",
|
||||
)
|
||||
results = model.predict(
|
||||
input={"image": "chart_parsing_02.png"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import ChartParsing
|
||||
model = ChartParsing(
|
||||
model_name="PP-Chart2Table",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
results = model.predict(
|
||||
input={"image": "chart_parsing_02.png"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_dynamic` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'chart_parsing_02.png', 'result': '年份 | 单家五星级旅游饭店年平均营收 (百万元) | 单家五星级旅游饭店年平均利润 (百万元)\n2018 | 104.22 | 9.87\n2019 | 99.11 | 7.47\n2020 | 57.87 | -3.87\n2021 | 68.99 | -2.9\n2022 | 56.29 | -9.48\n2023 | 87.99 | 5.96'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><code>image</code>: 表示输入待预测图像的路径</li>
|
||||
<li><code>result</code>: 模型预测的结果信息</li>
|
||||
</ul>
|
||||
预测结果打印可视化如下：
|
||||
|
||||
```bash
|
||||
年份 | 单家五星级旅游饭店年平均营收 (百万元) | 单家五星级旅游饭店年平均利润 (百万元)
|
||||
2018 | 104.22 | 9.87
|
||||
2019 | 99.11 | 7.47
|
||||
2020 | 57.87 | -3.87
|
||||
2021 | 68.99 | -2.9
|
||||
2022 | 56.29 | -9.48
|
||||
2023 | 87.99 | 5.96
|
||||
```
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>ChartParsing</code>实例化文档类视觉语言模型，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-Chart2Table</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code></code>。
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_dynamic</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用图表解析模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 、 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，必填。<br/>
|
||||
<b>说明：</b>
|
||||
由于多模态模型对输入要求不同，请根据具体模型设定输入格式。<br/>
|
||||
<ul>
|
||||
<li>PP-Chart2Table的输入形式为<code>{'image': image_path}</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、推理引擎
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="2">PP-Chart2Table</td>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>53.00</td>
|
||||
<td>17863.95</td>
|
||||
<td>0.30</td>
|
||||
<td>17917.78</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>23.95</td>
|
||||
<td>12217.37</td>
|
||||
<td>0.47</td>
|
||||
<td>12269.98</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
## 六、FAQ
|
||||
@@ -0,0 +1,486 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Image Orientation Classification Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Document Image Orientation Classification Module is primarily designed to distinguish the orientation of document images and correct them through post-processing. During processes such as document scanning or ID photo capturing, the device might be rotated to achieve clearer images, resulting in images with various orientations. Standard OCR pipelines may not handle these images effectively. By leveraging image classification techniques, the orientation of documents or IDs containing text regions can be pre-determined and adjusted, thereby improving the accuracy of OCR processing.
|
||||
|
||||
## 2. Supported Models List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Top-1 Acc (%)</th>
|
||||
<th>GPU Inference Time (ms)<br>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td>A document image classification model based on PP-LCNet_x1_0, with four categories: 0°, 90°, 180°, and 270°.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Self-built multi-scenario dataset (1000 images, including ID/document scenarios)</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Optimal backend selected (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. For details, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference of the Document Image Orientation Classification Module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(model_name="PP-LCNet_x1_0_doc_ori")
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(
|
||||
model_name="PP-LCNet_x1_0_doc_ori",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(
|
||||
model_name="PP-LCNet_x1_0_doc_ori",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result will be:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'img_rot180_demo.jpg', 'page_index': None, 'class_ids': array([2], dtype=int32), 'scores': array([0.88164], dtype=float32), 'label_names': ['180']}}
|
||||
```
|
||||
|
||||
The meaning of the output parameters is as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>：Represents the path of the input image.</li>
|
||||
<li><code>class_ids</code>：Represents the predicted class ID, with four categories: 0°, 90°, 180°, and 270°.</li>
|
||||
<li><code>scores</code>：Represents the confidence level of the prediction result.</li>
|
||||
<li><code>label_names</code>：Represents the category names of the prediction results.</li>
|
||||
</ul>
|
||||
|
||||
Here is the visualization of the image:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/doc_img_ori_classification/img_rot180_demo_res.jpg">
|
||||
|
||||
The explanations of relevant methods and parameters are as follows:
|
||||
|
||||
* Instantiate the document image orientation classification model with <code>DocImgOrientationClassification</code> (taking <code>PP-LCNet_x1_0_doc_ori</code> as an example here). The specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning：</b>Model name. <br/>
|
||||
<b>Description：</b>
|
||||
If set to <code>None</code>, <code>PP-LCNet_x1_0_doc_ori</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning：</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning：</b>Device for inference.<br/>
|
||||
<b>Description：</b>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning：</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning：</b>Whether to use the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description：</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning：</b>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/>
|
||||
<b>Description：</b>
|
||||
<b>Options:</b><code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>Meaning：</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description：</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>Meaning：</b>MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning：</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the document image orientation classification model for inference prediction. This method will return a list of results. In addition, this module also provides the <code>predict_iter()</code> method. The two methods are completely consistent in terms of parameter acceptance and result return. The difference is that <code>predict_iter()</code> returns a <code>generator</code>, which can process and obtain prediction results step by step, suitable for scenarios where large datasets need to be processed or memory needs to be saved. You can choose either of these two methods according to your actual needs. The parameters of the <code>predict()</code> method are <code>input</code> and <code>batch_size</code>, and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning：</b>Input data to be predicted. Required. <br/>
|
||||
<b>Description：</b>
|
||||
Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>：Local image or PDF file path: <code>/root/data/img.jpg</code>；
|
||||
<b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
<b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning：</b>Batch size. <br/>
|
||||
<b>Description：</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is the corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data and make it more readable. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; when set to <code>False</code>, the original characters will be retained. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a file in <code>json</code> format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data and make it more readable. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; when set to <code>False</code>, the original characters will be retained. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining the visualization image with results and the prediction results through attributes. The specifics are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualization image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training functionality for document image orientation classification, if you need to train a document image orientation classification model, you can refer to the [PaddleX Secondary Development for Document Image Orientation Classification](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#iv-custom-development) section for training guidance. The trained model can be seamlessly integrated into PaddleOCR's API for inference purposes.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x1_0_doc_ori</td>
|
||||
<td>paddle_static</td>
|
||||
<td>2.21</td>
|
||||
<td>3.36</td>
|
||||
<td>0.06</td>
|
||||
<td>5.74</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>2.15</td>
|
||||
<td>7.54</td>
|
||||
<td>0.07</td>
|
||||
<td>9.87</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>4.46</td>
|
||||
<td>3.44</td>
|
||||
<td>0.14</td>
|
||||
<td>8.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>2.02</td>
|
||||
<td>0.87</td>
|
||||
<td>0.05</td>
|
||||
<td>3.03</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Text Image Orientation Classification Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,477 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档图像方向分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文档图像方向分类模块主要是将文档图像的方向区分出来，并使用后处理将其矫正。在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的图片也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断含文字区域的文档或证件的方向，并将其进行方向调整，从而提高OCR处理的准确性。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td>基于PP-LCNet_x1_0的文档图像分类模型，含有四个类别，即0度，90度，180度，270度</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建多场景数据集（1000张图片，含证件/文档等场景）</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
如需了解推理引擎的介绍、速度数据及权重转换等详细信息，请参考后文 [推理引擎](#五推理引擎) 章节。
|
||||
|
||||
您也可以将文档图像方向分类模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(model_name="PP-LCNet_x1_0_doc_ori")
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
如果希望使用 `transformers` 引擎进行推理，可以参考如下示例：
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(
|
||||
model_name="PP-LCNet_x1_0_doc_ori",
|
||||
engine="transformers",
|
||||
)
|
||||
```
|
||||
|
||||
如果希望使用 `onnxruntime` 引擎进行推理，可以参考如下示例：
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(
|
||||
model_name="PP-LCNet_x1_0_doc_ori",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
随后继续按上文方式调用 `predict()` 即可。
|
||||
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'img_rot180_demo.jpg', 'page_index': None, 'class_ids': array([2], dtype=int32), 'scores': array([0.88164], dtype=float32), 'label_names': ['180']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：表示输入图片的路径。</li>
|
||||
<li><code>class_ids</code>：表示预测结果的类别id，含有四个类别，即0度，90度，180度和270度。</li>
|
||||
<li><code>scores</code>：表示预测结果的置信度。</li>
|
||||
<li><code>label_names</code>：表示预测结果的类别名。</li>
|
||||
</ul>
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/doc_img_ori_classification/img_rot180_demo_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>DocImgOrientationClassification</code>实例化文档图像方向分类模型（此处以<code>PP-LCNet_x1_0_doc_ori</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>说明</th>
|
||||
<th>类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_doc_ori</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文档图像方向分类模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 和 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；
|
||||
<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；
|
||||
<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供文档图像方向分类的训练，因此，如果需要训练文档图像方向分类模型，可以参考 [PaddleX 文档图像方向分类二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x1_0_doc_ori</td>
|
||||
<td>paddle_static</td>
|
||||
<td>2.21</td>
|
||||
<td>3.36</td>
|
||||
<td>0.06</td>
|
||||
<td>5.74</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>2.15</td>
|
||||
<td>7.54</td>
|
||||
<td>0.07</td>
|
||||
<td>9.87</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>4.46</td>
|
||||
<td>3.44</td>
|
||||
<td>0.14</td>
|
||||
<td>8.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>2.02</td>
|
||||
<td>0.87</td>
|
||||
<td>0.05</td>
|
||||
<td>3.03</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 文本图像方向分类模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 六、FAQ
|
||||
@@ -0,0 +1,265 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Visual Language Model Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
Document visual language models are a cutting-edge multimodal processing technology aimed at addressing the limitations of traditional document processing methods. Traditional methods are often limited to processing document information in specific formats or predefined categories, whereas document visual language models can integrate visual and linguistic information to understand and handle diverse document content. By combining computer vision and natural language processing technologies, these models can recognize images, text, and their relationships within documents, and even understand semantic information within complex layout structures. This makes document processing more intelligent and flexible, with stronger generalization capabilities, showing broad application prospects in automated office work, information extraction, and other fields.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Model Storage Size (GB)</th>
|
||||
<th>Total Score</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">Inference Model</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee is a self-developed multimodal large model by the PaddlePaddle team, focusing on document understanding, and it performs excellently in Chinese document understanding tasks. The model is fine-tuned and optimized using nearly 5 million multimodal datasets for document understanding, including general VQA, OCR, charts, text-rich documents, mathematics and complex reasoning, synthetic data, and pure text data, with different training data ratios set. On several authoritative English document understanding evaluation lists in academia, PP-DocBee has basically achieved SOTA for models of the same parameter scale. In terms of internal business Chinese scenario indicators, PP-DocBee also outperforms the current popular open-source and closed-source models.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">Inference Model</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">Inference Model</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 is a self-developed multimodal large model by the PaddlePaddle team, further optimizing the base model on the foundation of PP-DocBee and introducing a new data optimization scheme to improve data quality. Using a small amount of 470,000 data generated by a self-developed data synthesis strategy, PP-DocBee2 performs better in Chinese document understanding tasks. In terms of internal business Chinese scenario indicators, PP-DocBee2 improves by about 11.4% compared to PP-DocBee, and also outperforms the current popular open-source and closed-source models of the same scale.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>Note: The total scores of the above models are test results from an internal evaluation set, where all images have a resolution (height, width) of (1680, 1204), with a total of 1196 data entries, covering scenarios such as financial reports, laws and regulations, scientific and technical papers, manuals, humanities papers, contracts, research reports, etc. There are no plans for public release at the moment.</b>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please install the PaddleOCR wheel package. For details, please refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one line of command:
|
||||
|
||||
```bash
|
||||
paddleocr doc_vlm -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出'}"
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_dynamic</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the open document visual language model module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import DocVLM
|
||||
model = DocVLM(model_name="PP-DocBee2-3B")
|
||||
results = model.predict(
|
||||
input={"image": "medal_table.png", "query": "识别这份表格的内容, 以markdown格式输出"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_dynamic</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
|
||||
The meaning of the result parameters is as follows:
|
||||
<ul>
|
||||
<li><b>image</b>: Indicates the path of the input image to be predicted</li>
|
||||
<li><b>query</b>: Represents the input text information to be predicted</li>
|
||||
<li><b>result</b>: Information of the model's prediction result</li>
|
||||
</ul>
|
||||
|
||||
The visualization of the prediction result is as follows:
|
||||
|
||||
```bash
|
||||
| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |
|
||||
| 2 | 美国（USA） | 36 | 39 | 37 | 112 |
|
||||
| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |
|
||||
| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |
|
||||
| 5 | 德国（GER） | 16 | 11 | 14 | 41 |
|
||||
| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |
|
||||
| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |
|
||||
| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |
|
||||
| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |
|
||||
| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |
|
||||
| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |
|
||||
| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |
|
||||
| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |
|
||||
| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |
|
||||
| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |
|
||||
```
|
||||
|
||||
Explanations of related methods, parameters, etc., are as follows:
|
||||
|
||||
* <code>DocVLM</code> instantiates the document visual language model (taking <code>PP-DocBee-2B</code> as an example), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-DocBee-2B</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b> Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, and <code>paddle_dynamic</code>. When left as <code>None</code>, local inference uses the <code>paddle_dynamic</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the document visual language model for inference prediction. This method will return a result list. Additionally, this module also provides the <code>predict_iter()</code> method. Both are completely consistent in terms of parameter acceptance and result return, the difference being that <code>predict_iter()</code> returns a <code>generator</code>, capable of gradually processing and obtaining prediction results, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods based on actual needs. The <code>predict()</code> method parameters include <code>input</code> , <code>batch_size</code>, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b> Input data. Required. <br/>
|
||||
<b>Description:</b>
|
||||
Since multimodal models have different input requirements, please refer to the specific model for the correct format.<br/>
|
||||
For example, for the PP-DocBee series models, the input format should be: <code>{'image': image_path, 'query': query_text}</code>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size.<br/>
|
||||
<b>Description:</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is the corresponding Result object, and it supports operations such as printing and saving as <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path of the file to be saved. When it is a directory, the naming of the saved file is consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, it also supports obtaining prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
The current module does not support fine-tuning training temporarily, only inference integration is supported. The fine-tuning training of this module is planned to be supported in the future.
|
||||
|
||||
## V. FAQ
|
||||
@@ -0,0 +1,268 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档类视觉语言模型模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文档类视觉语言模型是当前一种前沿的多模态处理技术，旨在解决传统文档处理方法的局限性。传统方法往往局限于处理特定格式或预定义类别的文档信息，而文档类视觉语言模型能够融合视觉与语言信息，理解并处理多样化的文档内容。通过结合计算机视觉与自然语言处理技术，模型可以识别文档中的图像、文本及其相互关系，甚至能理解复杂版面结构中的语义信息。这使得文档处理更加智能化、灵活化，具备更强的泛化能力，在自动化办公、信息提取等领域展现出广阔的应用前景。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>模型存储大小（GB）</th>
|
||||
<th>模型总分</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">推理模型</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee 是飞桨团队自研的一款专注于文档理解的多模态大模型，在中文文档理解任务上具有卓越表现。该模型通过近 500 万条文档理解类多模态数据集进行微调优化，各种数据集包括了通用VQA类、OCR类、图表类、text-rich文档类、数学和复杂推理类、合成数据类、纯文本数据等，并设置了不同训练数据配比。在学术界权威的几个英文文档理解评测榜单上，PP-DocBee基本都达到了同参数量级别模型的SOTA。在内部业务中文场景类的指标上，PP-DocBee也高于目前的热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">推理模型</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">推理模型</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 是飞桨团队自研的一款专注于文档理解的多模态大模型，在PP-DocBee的基础上进一步优化了基础模型，并引入了新的数据优化方案，提高了数据质量，使用自研数据合成策略生成的少量的47万数据便使得PP-DocBee2在中文文档理解任务上表现更佳。在内部业务中文场景类的指标上，PP-DocBee2相较于PP-DocBee提升了约11.4%，同时也高于目前的同规模热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>注：以上模型总分为内部评估集模型测试结果，内部评估集所有图像分辨率 (height, width) 为 (1680,1204)，共1196条数据，包括了财报、法律法规、理工科论文、说明书、文科论文、合同、研报等场景，暂时未有计划公开。</b>
|
||||
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr doc_vlm -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出'}"
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_dynamic</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将开放文档类视觉语言模型模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import DocVLM
|
||||
model = DocVLM(model_name="PP-DocBee2-3B")
|
||||
results = model.predict(
|
||||
input={"image": "medal_table.png", "query": "识别这份表格的内容, 以markdown格式输出"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_dynamic</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><b>image</b>: 表示输入待预测图像的路径</li>
|
||||
<li><b>query</b>: 表述输入待预测的文本信息</li>
|
||||
<li><b>result</b>: 模型预测的结果信息</li>
|
||||
</ul>
|
||||
|
||||
|
||||
预测结果打印可视化如下：
|
||||
|
||||
```bash
|
||||
| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |
|
||||
| 2 | 美国（USA） | 36 | 39 | 37 | 112 |
|
||||
| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |
|
||||
| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |
|
||||
| 5 | 德国（GER） | 16 | 11 | 14 | 41 |
|
||||
| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |
|
||||
| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |
|
||||
| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |
|
||||
| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |
|
||||
| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |
|
||||
| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |
|
||||
| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |
|
||||
| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |
|
||||
| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |
|
||||
| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |
|
||||
```
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>DocVLM</code>实例化文档类视觉语言模型（此处以<code>PP-DocBee-2B</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-DocBee-2B</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code></code>。
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_dynamic</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_dynamic</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文档类视觉语言模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 和 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，必填。<br/>
|
||||
<b>说明：</b>
|
||||
由于多模态模型对输入要求不同，请根据具体模型设定输入格式。<br/>
|
||||
例如：对于 PP-DocBee 系列模型，输入形式应为：<code>{'image': image_path, 'query': query_text}</code>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、FAQ
|
||||
@@ -0,0 +1,493 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Formula Recognition Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
The formula recognition module is a key component of an OCR (Optical Character Recognition) system, responsible for converting mathematical formulas in images into editable text or computer-readable formats. The performance of this module directly affects the accuracy and efficiency of the entire OCR system. The formula recognition module typically outputs LaTeX or MathML code of the mathematical formulas, which will be passed as input to the text understanding module for further processing.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">Training Model</a></td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td>UniMERNet is a formula recognition model developed by Shanghai AI Lab. It uses Donut Swin as the encoder and MBartDecoder as the decoder. The model is trained on a dataset of one million samples, including simple formulas, complex formulas, scanned formulas, and handwritten formulas, significantly improving the recognition accuracy of real-world formulas.</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td rowspan="2">PP-FormulaNet is an advanced formula recognition model developed by the Baidu PaddlePaddle Vision Team. The PP-FormulaNet-S version uses PP-HGNetV2-B4 as its backbone network. Through parallel masking and model distillation techniques, it significantly improves inference speed while maintaining high recognition accuracy, making it suitable for applications requiring fast inference. The PP-FormulaNet-L version, on the other hand, uses Vary_VIT_B as its backbone network and is trained on a large-scale formula dataset, showing significant improvements in recognizing complex formulas compared to PP-FormulaNet-S.</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td rowspan="3">PP-FormulaNet_plus is an enhanced version of the formula recognition model developed by the Baidu PaddlePaddle Vision Team, building upon the original PP-FormulaNet. Compared to the original version, PP-FormulaNet_plus utilizes a more diverse formula dataset during training, including sources such as Chinese dissertations, professional books, textbooks, exam papers, and mathematics journals. This expansion significantly improves the model’s recognition capabilities. Among the models, PP-FormulaNet_plus-M and PP-FormulaNet_plus-L have added support for Chinese formulas and increased the maximum number of predicted tokens for formulas from 1,024 to 2,560, greatly enhancing the recognition performance for complex formulas. Meanwhile, the PP-FormulaNet_plus-S model focuses on improving the recognition of English formulas. With these improvements, the PP-FormulaNet_plus series models perform exceptionally well in handling complex and diverse formula recognition tasks. </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">Training Model</a></td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td>LaTeX-OCR is a formula recognition algorithm based on an autoregressive large model. It uses Hybrid ViT as the backbone network and a transformer as the decoder, significantly improving the accuracy of formula recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> PaddleOCR internal custom formula recognition test set</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technique Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Standard Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of predefined precision type and acceleration strategy</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Optimal predefined backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. For details, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly try it out with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr formula_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png
|
||||
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the formula recognition module into your own project.Before running the code below, please download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import FormulaRecognition
|
||||
model = FormulaRecognition(model_name="PP-FormulaNet_plus-M")
|
||||
output = model.predict(input="general_formula_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
After running, the output is:
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_formula_rec_001.png', 'page_index': None, 'rec_formula': '\\zeta_{0}(\\nu)=-\\frac{\\nu\\varrho^{-2\\nu}}{\\pi}\\int_{\\mu}^{\\infty}d\\omega\\int_{C_{+}}d z\\frac{2z^{2}}{(z^{2}+\\omega^{2})^{\\nu+1}}\\breve{\\Psi}(\\omega;z)e^{i\\epsilon z}\\quad,'}}
|
||||
```
|
||||
Explanation of the result parameters:
|
||||
|
||||
- `input_path`： Indicates the path to the input formula image to be predicted
|
||||
- `page_index`： If the input is a PDF file, this represents the page number; otherwise, it is None
|
||||
- `rec_formula`：Indicates the predicted LaTeX source code of the formula image
|
||||
The visualization image is as follows. The left side is the input formula image, and the right side is the rendered formula from the prediction:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/formula_recog/general_formula_rec_001_res_paddleocr3.png">
|
||||
|
||||
<b>Note: If you need to visualize the formula recognition module, you must install the LaTeX rendering environment by running the following command. Currently, visualization is only supported on Ubuntu. Other environments are not supported for now. For complex formulas, the LaTeX result may contain advanced representations that may not render successfully in Markdown or similar environments:</b>
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install texlive texlive-latex-base texlive-xetex latex-cjk-all texlive-latex-extra -y
|
||||
```
|
||||
|
||||
Related methods and parameter descriptions are as follows:
|
||||
|
||||
* `FormulaRecognition` instantiates the formula recognition model (here using `PP-FormulaNet_plus-M` as an example), with detailed description as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-FormulaNet_plus-M</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, and <code>transformers</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b><code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the formula recognition model to perform inference, which returns a result list.
|
||||
Additionally, this module provides the `predict_iter()` method. Both accept the same parameters and return the same result format.
|
||||
The difference is that `predict_iter()` returns a `generator`, which can process and retrieve results step-by-step, suitable for large datasets or memory-efficient scenarios.
|
||||
|
||||
You can choose either method based on your actual needs. The `predict()` method takes parameters `input` and `batch_size`, described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The prediction results can be processed. Each result corresponds to a `Result` object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Details</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the <code>JSON</code> output; only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. If set to <code>True</code>, all non-ASCII characters are escaped; if <code>False</code>, original characters are kept. Only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json-formatted file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it is a directory, the saved file name will match the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the <code>JSON</code> output; only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. If set to <code>True</code>, all non-ASCII characters are escaped; if <code>False</code>, original characters are kept. Only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it is a directory, the saved file name will match the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, you can also access the visualized image and prediction result via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the models above do not perform well in your scenario, you can try the following steps for custom development.
|
||||
Here we take training `PP-FormulaNet_plus-M` as an example. For other models, just replace the corresponding config file. First, you need to prepare a formula recognition dataset. You can follow the format of the [formula recognition demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar). Once the data is ready, follow the steps below to train and export the model. After export, the model can be quickly integrated into the API described above. This example uses the demo dataset. Before training the model, please ensure you have installed all PaddleOCR dependencies as described in the [installation documentation](../installation.en.md).
|
||||
|
||||
### 4.1 Environment Setup
|
||||
|
||||
To train the formula recognition model, you need to install additional Python and Linux dependencies. Run the following commands:
|
||||
|
||||
```shell
|
||||
sudo apt-get update
|
||||
sudo apt-get install libmagickwand-dev
|
||||
pip install tokenizers==0.19.1 imagesize ftfy Wand
|
||||
```
|
||||
|
||||
### 4.2 Dataset and Pretrained Model Preparation
|
||||
|
||||
#### 4.2.1 Prepare the Dataset
|
||||
```shell
|
||||
# Download the demo dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
|
||||
tar -xf ocr_rec_latexocr_dataset_example.tar
|
||||
```
|
||||
|
||||
#### 4.2.2 Download the Pretrained Model
|
||||
```shell
|
||||
# Download the PP-FormulaNet_plus-M pre-trained model
|
||||
wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar
|
||||
tar -xf rec_ppformulanet_plus_m_train.tar
|
||||
```
|
||||
|
||||
### 4.3 Model Training
|
||||
PaddleOCR is modularized. To train the `PP-FormulaNet_plus-M` model, you need to use its [config file](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml).
|
||||
|
||||
Training commands are as follows:
|
||||
```bash
|
||||
# Single GPU training (default)
|
||||
python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU IDs with --gpus
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
|
||||
```
|
||||
**Note:**
|
||||
|
||||
- By default, evaluation is performed every 1 epoch.If you change the batch size or dataset, modify the following accordingly:
|
||||
```bash
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Evaluation
|
||||
You can evaluate trained weights, e.g., output/xxx/xxx.pdparams, or use the downloaded [model](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar ) with the following command:
|
||||
|
||||
```bash
|
||||
# Make sure pretrained_model is set to the local path.
|
||||
# For custom-trained models, modify the path and file name as {path/to/weights}/{model_name}
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.5 Model Export
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-FormulaNet_plus-M_infer/"
|
||||
```
|
||||
|
||||
After exporting, the static graph model will be saved in `./PP-FormulaNet_plus-M_infer/`, and you will see the following files:
|
||||
```
|
||||
./PP-FormulaNet_plus-M_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, the secondary development is complete. This static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## V. FAQ
|
||||
|
||||
**Q1: Which formula recognition model does PaddleOCR recommend?**
|
||||
|
||||
A1: It is recommended to use the PP-FormulaNet series.
|
||||
If your scenario is mainly in English and inference speed is not a concern, use PP-FormulaNet-L or PP-FormulaNet_plus-L.
|
||||
For mainly Chinese use cases, use PP-FormulaNet_plus-L or PP-FormulaNet_plus-M.
|
||||
If your device has limited computing power and you are working with English formulas, use PP-FormulaNet-S.
|
||||
|
||||
**Q2: Why does the inference report an error?**
|
||||
A2: The formula recognition model depends heavily on Paddle 3.0 official release.
|
||||
Please ensure the correct version is installed.
|
||||
|
||||
**Q3: Why is there no visualization image after prediction?**
|
||||
A3: This may be due to LaTeX not being installed.You need to refer to Section III and install the LaTeX rendering tools.
|
||||
@@ -0,0 +1,494 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 公式识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
公式识别模块是OCR（光学字符识别）系统中的关键组成部分，负责将图像中的数学公式转换为可编辑的文本或计算机可识别的格式。该模块的性能直接影响到整个OCR系统的准确性和效率。公式识别模块通常会输出数学公式的 LaTeX 或 MathML 代码，这些代码将作为输入传递给文本理解模块进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td>UniMERNet是由上海AI Lab研发的一款公式识别模型。该模型采用Donut Swin作为编码器，MBartDecoder作为解码器，并通过在包含简单公式、复杂公式、扫描捕捉公式和手写公式在内的一百万数据集上进行训练，大幅提升了模型对真实场景公式的识别准确率</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td rowspan="2">PP-FormulaNet 是由百度飞桨视觉团队开发的一款先进的公式识别模型，支持5万个常见LateX源码词汇的识别。PP-FormulaNet-S 版本采用了 PP-HGNetV2-B4 作为其骨干网络，通过并行掩码和模型蒸馏等技术，大幅提升了模型的推理速度，同时保持了较高的识别精度，适用于简单印刷公式、跨行简单印刷公式等场景。而 PP-FormulaNet-L 版本则基于 Vary_VIT_B 作为骨干网络，并在大规模公式数据集上进行了深入训练，在复杂公式的识别方面，相较于PP-FormulaNet-S表现出显著的提升，适用于简单印刷公式、复杂印刷公式、手写公式等场景。 </td>
|
||||
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td rowspan="3">PP-FormulaNet_plus 是百度飞桨视觉团队在 PP-FormulaNet 的基础上开发的增强版公式识别模型。与原版相比，PP-FormulaNet_plus 在训练中使用了更为丰富的公式数据集，包括中文学位论文、专业书籍、教材试卷以及数学期刊等多种来源。这一扩展显著提升了模型的识别能力。
|
||||
|
||||
其中，PP-FormulaNet_plus-M 和 PP-FormulaNet_plus-L 模型新增了对中文公式的支持，并将公式的最大预测 token 数从 1024 扩大至 2560，大幅提升了对复杂公式的识别性能。同时，PP-FormulaNet_plus-S 模型则专注于增强英文公式的识别能力。通过这些改进，PP-FormulaNet_plus 系列模型在处理复杂多样的公式识别任务时表现更加出色。 </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">训练模型</a></td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td>LaTeX-OCR是一种基于自回归大模型的公式识别算法，通过采用 Hybrid ViT 作为骨干网络，transformer作为解码器，显著提升了公式识别的准确性。</td>
|
||||
</tr>
|
||||
</table>
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR 内部自建公式识别测试集</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr formula_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将公式识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import FormulaRecognition
|
||||
model = FormulaRecognition(model_name="PP-FormulaNet_plus-M")
|
||||
output = model.predict(input="general_formula_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_formula_rec_001.png', 'page_index': None, 'rec_formula': '\\zeta_{0}(\\nu)=-\\frac{\\nu\\varrho^{-2\\nu}}{\\pi}\\int_{\\mu}^{\\infty}d\\omega\\int_{C_{+}}d z\\frac{2z^{2}}{(z^{2}+\\omega^{2})^{\\nu+1}}\\breve{\\Psi}(\\omega;z)e^{i\\epsilon z}\\quad,'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测公式图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `rec_formula`：表示公式图像的预测LaTeX源码
|
||||
|
||||
|
||||
可视化图片如下，左侧是待预测的公式图像，右边是预测的结果渲染后的公式图像：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/formula_recog/general_formula_rec_001_res_paddleocr3.png">
|
||||
|
||||
<b> 注：如果您需要对公式识别模块进行可视化，需要运行如下命令来对LaTeX渲染环境进行安装。目前公式识别模块可视化只支持Ubuntu环境，其他环境暂不支持。对于复杂公式，LaTeX 结果可能包含部分高级的表示，Markdown等环境中未必可以成功显示：</b>
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install texlive texlive-latex-base texlive-xetex latex-cjk-all texlive-latex-extra -y
|
||||
```
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `FormulaRecognition`实例化公式识别模型（此处以`PP-FormulaNet_plus-M`为例），具体说明如下：
|
||||
<table><thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr></thead><tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-FormulaNet_plus-M</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用公式识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
如果以上模型在您的场景下效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-FormulaNet-S` 举例，其他模型替换对应配置文件即可。首先，您需要准备公式识别的数据集，可以参考[公式识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述API中。此处以公式识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
### 4.1 环境配置
|
||||
|
||||
训练公式识别模型需要安装额外的Python依赖和linux依赖，执行如下命令安装：
|
||||
```shell
|
||||
sudo apt-get update
|
||||
sudo apt-get install libmagickwand-dev
|
||||
pip install tokenizers==0.19.1 imagesize ftfy Wand
|
||||
```
|
||||
|
||||
### 4.2 数据集、预训练模型准备
|
||||
|
||||
#### 4.2.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
|
||||
tar -xf ocr_rec_latexocr_dataset_example.tar
|
||||
```
|
||||
|
||||
#### 4.2.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-FormulaNet_plus-M 预训练模型
|
||||
wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar
|
||||
tar -xf rec_ppformulanet_plus_m_train.tar
|
||||
```
|
||||
|
||||
### 4.3 模型训练
|
||||
|
||||
PaddleOCR对代码进行了模块化，训练 `PP-FormulaNet_plus-M` 识别模型时需要使用 `PP-FormulaNet_plus-M` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
**注意：**
|
||||
|
||||
- 默认每训练 1个 epoch 进行 1 次评估，若您更改训练的 batch_size，或更换数据集，请在训练时作出如下修改
|
||||
```bash
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，也可以使用已经下载的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar)，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
|
||||
#注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
#demo 测试集评估
|
||||
python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.5 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-FormulaNet_plus-M_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-FormulaNet_plus-M_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-FormulaNet_plus-M_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
**Q1: PaddleOCR 更推荐哪个公式识别模型？**
|
||||
|
||||
A1: 更推荐使用 PP-FormulaNet 系列模型，如果是英文场景居多且不考虑推理耗时，则可以使用 PP-FormulaNet-L 或者 PP-FormulaNet_plus-L 模型，如果中文场景居多，则可以使用 PP-FormulaNet_plus-L 或者 PP-FormulaNet_plus-M，如果推理设备算力有限且是英文场景，则可以使用 PP-FormulaNet-S。
|
||||
|
||||
**Q2: 为什么推理报错？**
|
||||
|
||||
A2: 公式识别模型的推理强依赖于 Paddle 框架 3.0 正式版，请确保版本一致。
|
||||
|
||||
|
||||
**Q3: 为什么预测后没有可视化图像？**
|
||||
|
||||
A3: 可能是因为没有安装LaTeX导致，您需要参考第三节安装LaTeX渲染工具。
|
||||
@@ -0,0 +1,618 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
# Layout Analysis Module User Guide
|
||||
|
||||
|
||||
|
||||
## 1. Overview
|
||||
|
||||
|
||||
|
||||
Layout analysis is a crucial component in document parsing systems. Its goal is to parse the overall layout of a document, accurately detecting various elements it contains (such as text paragraphs, headings, images, tables, formulas, etc.), and restoring the correct reading order of these elements. The performance of this module directly impacts the overall accuracy and usability of the document parsing system.
|
||||
|
||||
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
|
||||
|
||||
Currently, this module only supports the PP-DocLayoutV2 model. Structurally, PP-DocLayoutV2 is based on the layout detection model [PP-DocLayout_plus-L](./layout_detection.en.md) (built upon the RT-DETR-L model) and cascades a lightweight pointer network with 6 Transformer layers. The PP-DocLayout_plus-L component continues to perform layout detection, identifying different elements in document images (such as text, charts, images, formulas, paragraphs, abstracts, references, etc.), classifying them into predefined categories, and determining their positions within the document. The detected bounding boxes and class labels are then fed into the subsequent pointer network to sort the layout elements and obtain the correct reading order.
|
||||
|
||||
|
||||
|
||||
<div align="center">
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/methods/PP-DocLayoutV2.png" width="600"/>
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
As shown in the figure above, PP-DocLayoutV2 embeds the targets detected by RT-DETR using absolute 2D positional encoding and class labels. Additionally, the pointer network’s attention mechanism incorporates the geometric bias mechanism from Relation-DETR to explicitly model pairwise geometric relationships between elements. The pairwise relation head linearly projects element representations into query and key vectors, then computes bilinear similarities to generate pairwise logits, ultimately producing an N×N matrix representing the relative order between every pair of elements. Finally, a deterministic "win-accumulation" decoding algorithm restores a topologically consistent reading order for the detected layout elements.
|
||||
|
||||
|
||||
|
||||
The following table only presents the layout detection accuracy of PP-DocLayoutV2. The evaluation dataset is a self-built layout region detection dataset, containing 1,000 images of various document types such as Chinese and English papers, magazines, newspapers, research reports, PPTs, exam papers, textbooks, etc., and covering 25 common layout element categories: document title, section header, text, vertical text, page number, abstract, table of contents, references, footnote, image caption, header, footer, header image, footer image, algorithm, inline formula, display formula, formula number, image, table, figure title (figure title, table title, chart title), seal, chart, aside text, and reference content.
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Standard" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
|
||||
<thead>
|
||||
|
||||
<tr>
|
||||
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
|
||||
<th>mAP(0.5) (%)</th>
|
||||
|
||||
<th>GPU Inference Time (ms)<br/>[Standard / High-Performance Mode]</th>
|
||||
|
||||
<th>CPU Inference Time (ms)<br/>[Standard / High-Performance Mode]</th>
|
||||
|
||||
<th>Model Size (MB)</th>
|
||||
|
||||
<th>Description</th>
|
||||
|
||||
</tr>
|
||||
|
||||
</thead>
|
||||
|
||||
<tbody>
|
||||
|
||||
<tr>
|
||||
|
||||
<td>PP-DocLayoutV2</td>
|
||||
|
||||
<td><a href="https://huggingface.co/PaddlePaddle/PP-DocLayoutV2">Inference Model</a></td>
|
||||
|
||||
<td>81.4</td>
|
||||
|
||||
<td> - </td>
|
||||
|
||||
<td> - </td>
|
||||
|
||||
<td>203.8</td>
|
||||
|
||||
<td>In-house layout analysis model trained on a diverse self-built dataset including Chinese and English academic papers, multi-column magazines, newspapers, PPTs, contracts, books, exam papers, research reports, ancient books, Japanese documents, and vertical text documents. Provides high-precision layout region localization and reading order recovery.</td>
|
||||
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
|
||||
</tbody>
|
||||
|
||||
</table>
|
||||
|
||||
## 3. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
You can quickly try it out with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg --model_name PP-DocLayoutV3
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg --model_name PP-DocLayoutV3 \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg --model_name PP-DocLayoutV3 \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable <code>PADDLE_PDX_MODEL_SOURCE="BOS"</code> to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayoutV2")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayoutV2",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayoutV2",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#42-weight-conversion) section in the [Inference Engine](#4-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 7, 'label': 'figure_title', 'score': 0.9764086604118347, 'coordinate': [34.227077, 19.217348, 362.48834, 80.243195]}, {'cls_id': 21, 'label': 'table', 'score': 0.9881672263145447, 'coordinate': [73.759026, 105.72034, 322.67398, 298.82642]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.9405199885368347, 'coordinate': [35.089825, 330.67575, 144.06467, 346.7406]}, {'cls_id': 22, 'label': 'text', 'score': 0.9889925122261047, 'coordinate': [33.540825, 349.37204, 363.5848, 615.04504]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.9430180788040161, 'coordinate': [35.032074, 627.2185, 188.24695, 643.66693]}, {'cls_id': 22, 'label': 'text', 'score': 0.988863468170166, 'coordinate': [33.72388, 646.5885, 363.05005, 852.60236]}, {'cls_id': 7, 'label': 'figure_title', 'score': 0.9749509692192078, 'coordinate': [385.14465, 19.341825, 715.4594, 78.739395]}, {'cls_id': 21, 'label': 'table', 'score': 0.9876241683959961, 'coordinate': [436.68512, 105.67539, 664.1041, 314.0018]}, {'cls_id': 22, 'label': 'text', 'score': 0.9878363013267517, 'coordinate': [385.06494, 345.74847, 715.173, 463.18677]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.951842725276947, 'coordinate': [386.3095, 476.34277, 703.0732, 493.00302]}, {'cls_id': 22, 'label': 'text', 'score': 0.9884033799171448, 'coordinate': [385.1237, 496.70123, 716.09717, 702.33386]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.940105676651001, 'coordinate': [386.66754, 715.5732, 527.5737, 731.77826]}, {'cls_id': 22, 'label': 'text', 'score': 0.9876392483711243, 'coordinate': [384.9688, 734.457, 715.63086, 853.71]}]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>：The path to the input image for prediction.</li>
|
||||
<li><code>page_index</code>：If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is <code>None</code>.</li>
|
||||
<li><code>boxes</code>：Predicted layout element information sorted in reading order, represented as a list of dictionaries. According to the reading order, each dictionary represents a detected layout element and contains the following information:
|
||||
<ol start="1" type="1">
|
||||
<li><code>cls_id</code>：Class ID, an integer.</li>
|
||||
<li><code>label</code>：Class label, a string.</li>
|
||||
<li><code>score</code>：Confidence score of the bounding box, a float.</li>
|
||||
<li><code>coordinate</code>：Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code></li>
|
||||
</ol>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
|
||||
Relevant methods, parameters, and explanations are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-DocLayout-L</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b> Device for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>"cpu"</code> will be used. <br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine. <br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the TensorRT subgraph engine in Paddle Inference. <br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td><b>Meaning:</b>Input image size.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640.</li>
|
||||
<li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to width 640 and height 512.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>Meaning:</b>Threshold for filtering low-confidence predictions.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>float</code> (threshold). For example, <code>{0: 0.45, 2: 0.48, 7: 0.4}</code> means class 0 uses threshold 0.45, class 2 uses 0.48, class 7 uses 0.4.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>Meaning:</b>Whether to use NMS post-processing to filter overlapping boxes.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>bool</b>: whether to use NMS for post-processing to filter overlapping boxes.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Scaling factor for the side length of the detection box.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>float</b>: A float greater than 0, e.g. <code>1.1</code>, expands width and height by 1.1 times.</li>
|
||||
<li><b>list</b>: e.g. <code>[1.2, 1.5]</code>, expands width by 1.2x and height by 1.5x.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>tuple</code> of two floats (width ratio, height ratio). For example, <code>{0: (1.1, 2.0)}</code> means for class 0, width is expanded by 1.1x and height by 2.0x.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>Meaning:</b>Merge mode for model output bounding boxes.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>"large"</b>: Only keep the largest outer box among overlapping boxes, remove inner boxes.</li>
|
||||
<li><b>"small"</b>: Only keep the smallest inner box among overlapping boxes, remove outer boxes.</li>
|
||||
<li><b>"union"</b>: Keep all boxes, no filtering.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>str</code> (mode). For example, <code>{0: "large", 2: "small"}</code> means class 0 uses "large" mode, class 2 uses "small" mode.</li>
|
||||
<li><b>None</b>: Use the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The <code>predict()</code> method of the target detection model is called for inference prediction. The parameters of the <code>predict()</code> method are <code>input</code>, <code>batch_size</code>, and <code>threshold</code>, which are explained as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b>Input data to be predicted. Required.<br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
<ul>
|
||||
<li>Local image or PDF file path: <code>/root/data/img.jpg</code>;</li>
|
||||
<li><b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;</li>
|
||||
<li><b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
</ul>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size<br/>
|
||||
<b>Description:</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results, with each sample's prediction result being the corresponding Result object, and supporting operations such as printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Parameter type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Do you want to use <code>JSON</code> indentation formatting for the output content</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non ASCII </code>characters will be escaped; <code>False</code> preserves the original characters and is only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a JSON format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non <code>ASCII</code> characters will be escaped; <code>False</code> preserves the original characters and is only valid when<code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* Additionally, it also supports obtaining the visualized image with results and the prediction results via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 4.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocLayoutV3</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.95</td>
|
||||
<td>47.99</td>
|
||||
<td>12.97</td>
|
||||
<td>72.33</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.33</td>
|
||||
<td>84.48</td>
|
||||
<td>1.31</td>
|
||||
<td>98.01</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>16.94</td>
|
||||
<td>47.11</td>
|
||||
<td>13.83</td>
|
||||
<td>78.97</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.26</td>
|
||||
<td>39.11</td>
|
||||
<td>10.23</td>
|
||||
<td>58.80</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocLayoutV2</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.48</td>
|
||||
<td>30.94</td>
|
||||
<td>1.33</td>
|
||||
<td>42.93</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.07</td>
|
||||
<td>86.38</td>
|
||||
<td>1.33</td>
|
||||
<td>99.80</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>16.76</td>
|
||||
<td>49.08</td>
|
||||
<td>2.43</td>
|
||||
<td>69.30</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.25</td>
|
||||
<td>15.13</td>
|
||||
<td>1.25</td>
|
||||
<td>25.82</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 4.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Layout Analysis Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_analysis.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
@@ -0,0 +1,567 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 版面分析模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
版面分析是文档解析系统中的重要组成环节，目标是对文档的整体布局进行解析，准确检测出其中所包含的各种元素（例如，文本段落、标题、图片、表格、公式等），并恢复这些元素正确的阅读顺序。该模块的性能直接影响到整个文档解析系统的准确性和可用性。
|
||||
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
该模块目前仅支持 PP-DocLayoutV2 一个模型。模型结构上，PP-DocLayoutV2 是在版面检测模型 [PP-DocLayout_plus-L](./layout_detection.md)（基于 RT-DETR-L 模型） 的基础上级联一个含 6 层 Transformer 层的轻量级指针网络（Pointer network）组成，原先 PP-DocLayout_plus-L 部分继续用于版面检测，识别文档图像中的不同元素（如文字、图表、图像、公式、段落、摘要、参考文献等），将其归类为预定义的类别并确定这些区域在文档中的位置。检测到的边界框和类别标签作为后续的指针网络的输入来对版面元素进行排序从而得到正确的阅读顺序。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl/methods/PP-DocLayoutV2.png" width="600"/>
|
||||
</div>
|
||||
|
||||
具体如上图所示，PP-DocLayoutV2 对来自 RT-DETR 检出的目标利用绝对二维位置编码和类别标签进行嵌入表示。此外，指针网络的注意力机制融合了 Relation-DETR中的几何偏置机制，以显式地建模元素之间的成对几何关系。成对关系头（pairwise relation head）将元素表示线性投影为查询（query）向量和键（key）向量，然后计算双线性相似度以生成成对的 logits，最终得到一个表示每对元素之间相对顺序的 N×N 矩阵。最后，一种确定性的“胜者累积”（win-accumulation）解码算法会为检测到的版面元素恢复出一个拓扑一致的阅读顺序。
|
||||
|
||||
下表仅给出 PP-DocLayoutV2 的版面检测精度。该精度指标的评估数据集是自建的版面区域检测数据集，包含了中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片，包含 25 类常见的版面元素：文档标题、段落标题、文本、竖排文本、页码、摘要、目录、参考文献、脚注、图像脚注、页眉、页脚、页眉图像、页脚图像、算法、行内公式、行间公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容。
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayoutV2</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayoutV2_infer.tar">推理模型</a></td>
|
||||
<td>81.4</td>
|
||||
<td> - </td>
|
||||
<td> - </td>
|
||||
<td>203.8</td>
|
||||
<td>自研的版面分析模型在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位和版面阅读顺序恢复模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg --model_name PP-DocLayoutV3
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg --model_name PP-DocLayoutV3 \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg --model_name PP-DocLayoutV3 \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
您可以将版面区域检测模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayoutV3")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayoutV3",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayoutV3",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#四推理引擎) 中的 [权重转换](#42-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 7, 'label': 'figure_title', 'score': 0.9764086604118347, 'coordinate': [34.227077, 19.217348, 362.48834, 80.243195]}, {'cls_id': 21, 'label': 'table', 'score': 0.9881672263145447, 'coordinate': [73.759026, 105.72034, 322.67398, 298.82642]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.9405199885368347, 'coordinate': [35.089825, 330.67575, 144.06467, 346.7406]}, {'cls_id': 22, 'label': 'text', 'score': 0.9889925122261047, 'coordinate': [33.540825, 349.37204, 363.5848, 615.04504]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.9430180788040161, 'coordinate': [35.032074, 627.2185, 188.24695, 643.66693]}, {'cls_id': 22, 'label': 'text', 'score': 0.988863468170166, 'coordinate': [33.72388, 646.5885, 363.05005, 852.60236]}, {'cls_id': 7, 'label': 'figure_title', 'score': 0.9749509692192078, 'coordinate': [385.14465, 19.341825, 715.4594, 78.739395]}, {'cls_id': 21, 'label': 'table', 'score': 0.9876241683959961, 'coordinate': [436.68512, 105.67539, 664.1041, 314.0018]}, {'cls_id': 22, 'label': 'text', 'score': 0.9878363013267517, 'coordinate': [385.06494, 345.74847, 715.173, 463.18677]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.951842725276947, 'coordinate': [386.3095, 476.34277, 703.0732, 493.00302]}, {'cls_id': 22, 'label': 'text', 'score': 0.9884033799171448, 'coordinate': [385.1237, 496.70123, 716.09717, 702.33386]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.940105676651001, 'coordinate': [386.66754, 715.5732, 527.5737, 731.77826]}, {'cls_id': 22, 'label': 'text', 'score': 0.9876392483711243, 'coordinate': [384.9688, 734.457, 715.63086, 853.71]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：输入的待预测图像的路径</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>boxes</code>：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
<ol start="1" type="1">
|
||||
<li><code>cls_id</code>：类别ID，一个整数</li>
|
||||
<li><code>label</code>：类别标签，一个字符串</li>
|
||||
<li><code>score</code>：目标框置信度，一个浮点数</li>
|
||||
<li><code>coordinate</code>：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code></li>
|
||||
</ol>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-DocLayout-L</code></td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td><b>含义：</b>输入图像大小。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640、高为512。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>含义：</b>用于过滤掉低置信度预测结果的阈值。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>float</b>：如<code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>float</code>类型阈值。如<code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对ID为0的类别应用阈值0.45、ID为1的类别应用阈值0.48、ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>含义：</b>是否使用NMS后处理，过滤重叠框。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>bool</b>表示使用/不使用NMS进行检测框的后处理过滤重叠框。</li>
|
||||
<li><b>None</b>使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>含义：</b>检测框的边长缩放倍数。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>float</b>：大于0的浮点数，如<code>1.1</code>，表示将模型输出的检测框中心不变，宽和高都扩张1.1倍。</li>
|
||||
<li><b>list</b>：如<code>[1.2, 1.5]</code>，表示将模型输出的检测框中心不变，宽度扩张1.2倍，高度扩张1.5倍。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>tuple</code>类型，如<code>{0: (1.1, 2.0)}</code>，表示将模型输出的第0类别检测框中心不变，宽度扩张1.1倍，高度扩张2.0倍。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>含义：</b>模型输出的检测框的合并处理模式。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>"large"</b>：设置为<code>"large"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留外部最大的框，删除重叠的内部框。</li>
|
||||
<li><b>"small"</b>：设置为<code>"small"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留内部被包含的小框，删除重叠的外部框。</li>
|
||||
<li><b>"union"</b>：不进行框的过滤处理，内外框都保留。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>str</code>类型, 如<code>{0: "large", 2: "small"}</code>, 表示对第0类别检测框使用<code>large</code>模式，对第2类别检测框使用<code>small</code>。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* 调用目标检测模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 、<code>batch_size</code>和<code>threshold</code>，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、推理引擎 {#四推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 4.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocLayoutV3</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.95</td>
|
||||
<td>47.99</td>
|
||||
<td>12.97</td>
|
||||
<td>72.33</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.33</td>
|
||||
<td>84.48</td>
|
||||
<td>1.31</td>
|
||||
<td>98.01</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>16.94</td>
|
||||
<td>47.11</td>
|
||||
<td>13.83</td>
|
||||
<td>78.97</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.26</td>
|
||||
<td>39.11</td>
|
||||
<td>10.23</td>
|
||||
<td>58.80</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocLayoutV2</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.48</td>
|
||||
<td>30.94</td>
|
||||
<td>1.33</td>
|
||||
<td>42.93</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.07</td>
|
||||
<td>86.38</td>
|
||||
<td>1.33</td>
|
||||
<td>99.80</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>16.76</td>
|
||||
<td>49.08</td>
|
||||
<td>2.43</td>
|
||||
<td>69.30</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.25</td>
|
||||
<td>15.13</td>
|
||||
<td>1.25</td>
|
||||
<td>25.82</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 4.2 权重转换 {#42-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 版面分析模块权重转换](https://paddlepaddle.github.io/PaddleX/main/module_usage/tutorials/ocr_modules/layout_analysis.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
@@ -0,0 +1,852 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Layout Detection Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The core task of structure analysis is to parse and segment the content of input document images. By identifying different elements in the image (such as text, charts, images, etc.), they are classified into predefined categories (e.g., pure text area, title area, table area, image area, list area, etc.), and the position and size of these regions in the document are determined.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
* <b>The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01</td>
|
||||
<td>A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 1 category: Block:</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td>A layout block localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, figure caption, table, table caption, seal, figure title, figure, header image, footer image, and sidebar text</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using RT-DETR-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td>A layout area localization model with balanced precision and efficiency, trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ The above list includes the <b>4 core models</b> that are key supported by the text recognition module. The module actually supports a total of <b>12 full models</b>, including several predefined models with different categories. The complete model list is as follows:
|
||||
|
||||
<details><summary> 👉 Details of Model List</summary>
|
||||
|
||||
* <b>Table Layout Detection Model</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset using PicoDet-1x, capable of detecting table regions.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3-Class Layout Detection Model, including Table, Image, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5-Class English Document Area Detection Model, including Text, Title, Table, Image, and List</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency English document layout area localization model trained on the PubLayNet dataset using PicoDet-1x.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong>
|
||||
<ul>
|
||||
<li>20 types of layout detection models: PaddleOCR's self built layout area detection dataset, including Chinese and English papers, magazines, newspapers, research papers PPT、 1300 images of document types such as test papers and textbooks. </li>
|
||||
<li>Type 1 version face region detection model: PaddleOCR's self built version face region detection dataset, including Chinese and English papers, magazines, newspapers, research reports PPT、 1000 document type images such as test papers and textbooks. </li>
|
||||
<li>23 categories Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, containing 500 common document type images such as Chinese and English papers, magazines, contracts, books, exam papers, and research reports.</li>
|
||||
<li>Table Layout Detection Model: A self-built table area detection dataset by PaddleOCR, including 7,835 Chinese and English paper document type images with tables.</li>
|
||||
<li> 3-Class Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, comprising 1,154 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
<li>5-Class English Document Area Detection Model: The evaluation dataset of <a href="https://developer.ibm.com/exchanges/data/all/publaynet">PubLayNet</a>, containing 11,245 images of English documents.</li>
|
||||
<li>17-Class Area Detection Model: A self-built layout area detection dataset by PaddleOCR, including 892 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
## 3. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable <code>PADDLE_PDX_MODEL_SOURCE="BOS"</code> to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayout_plus-L",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayout_plus-L",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>：The path to the input image for prediction.</li>
|
||||
<li><code>page_index</code>：If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is <code>None</code>.</li>
|
||||
<li><code>boxes</code>：Information about the predicted bounding boxes, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
<ol start="1" type="1">
|
||||
<li><code>cls_id</code>：Class ID, an integer.</li>
|
||||
<li><code>label</code>：Class label, a string.</li>
|
||||
<li><code>score</code>：Confidence score of the bounding box, a float.</li>
|
||||
<li><code>coordinate</code>：Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code>.</li>
|
||||
</ol>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res_plus.jpg"/>
|
||||
|
||||
Relevant methods, parameters, and explanations are as follows:
|
||||
|
||||
* <code>LayoutDetection</code> instantiates a target detection model (here, <code>PP-DocLayout_plus-L</code> is used as an example). The detailed explanation is as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-DocLayout-L</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b>Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td><b>Meaning:</b>Input image size.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640.</li>
|
||||
<li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to width 640 and height 512.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>Meaning:</b>Threshold for filtering low-confidence predictions.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>float</code> (threshold). For example, <code>{0: 0.45, 2: 0.48, 7: 0.4}</code> means class 0 uses threshold 0.45, class 2 uses 0.48, class 7 uses 0.4.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>Meaning:</b>Whether to use NMS post-processing to filter overlapping boxes.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>bool</b>: whether to use NMS for post-processing to filter overlapping boxes.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Scaling factor for the side length of the detection box.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>float</b>: A float greater than 0, e.g. <code>1.1</code>, expands width and height by 1.1 times.</li>
|
||||
<li><b>list</b>: e.g. <code>[1.2, 1.5]</code>, expands width by 1.2x and height by 1.5x.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>tuple</code> of two floats (width ratio, height ratio). For example, <code>{0: (1.1, 2.0)}</code> means for class 0, width is expanded by 1.1x and height by 2.0x.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>Meaning:</b>Merge mode for model output bounding boxes.<br/>
|
||||
<b>Description:</b>
|
||||
<ul>
|
||||
<li><b>"large"</b>: Only keep the largest outer box among overlapping boxes, remove inner boxes.</li>
|
||||
<li><b>"small"</b>: Only keep the smallest inner box among overlapping boxes, remove outer boxes.</li>
|
||||
<li><b>"union"</b>: Keep all boxes, no filtering.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>str</code> (mode). For example, <code>{0: "large", 2: "small"}</code> means class 0 uses "large" mode, class 2 uses "small" mode.</li>
|
||||
<li><b>None</b>: Use the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The <code>predict()</code> method of the target detection model is called for inference prediction. The parameters of the <code>predict()</code> method are <code>input</code> , <code>batch_size</code>, and <code>threshold</code>, which are explained as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b>Input data to be predicted. Required. <br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
<ul>
|
||||
<li>Local image or PDF file path: <code>/root/data/img.jpg</code>;</li>
|
||||
<li><b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;</li>
|
||||
<li><b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
</li>
|
||||
</ul>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size.<br/>
|
||||
<b>Description:</b>
|
||||
positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results, with each sample's prediction result being the corresponding Result object, and supporting operations such as printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Parameter type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Do you want to use <code>JSON</code> indentation formatting for the output content</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non ASCII </code>characters will be escaped; <code>False</code> preserves the original characters and is only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a JSON format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non <code>ASCII</code> characters will be escaped; <code>False</code> preserves the original characters and is only valid when<code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* Additionally, it also supports obtaining the visualized image with results and the prediction results via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the layout detection module, if you need to train the layout area detection model, you can refer to [PaddleX Layout Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_detection.html#iv-custom-development)Partially conduct training. The trained model can be seamlessly integrated into PaddleOCR's API for inference.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocLayout_plus-L</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.92</td>
|
||||
<td>26.11</td>
|
||||
<td>0.16</td>
|
||||
<td>37.38</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.09</td>
|
||||
<td>72.91</td>
|
||||
<td>0.16</td>
|
||||
<td>85.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>12.65</td>
|
||||
<td>37.91</td>
|
||||
<td>0.75</td>
|
||||
<td>52.24</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.14</td>
|
||||
<td>12.35</td>
|
||||
<td>0.14</td>
|
||||
<td>21.81</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocBlockLayout</td>
|
||||
<td>paddle_static</td>
|
||||
<td>9.51</td>
|
||||
<td>27.59</td>
|
||||
<td>0.08</td>
|
||||
<td>37.41</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>8.94</td>
|
||||
<td>70.77</td>
|
||||
<td>0.07</td>
|
||||
<td>80.73</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>11.37</td>
|
||||
<td>37.95</td>
|
||||
<td>0.75</td>
|
||||
<td>50.96</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>7.53</td>
|
||||
<td>9.22</td>
|
||||
<td>0.06</td>
|
||||
<td>16.97</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Layout Detection Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_detection.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,845 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 版面区域检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
版面区域检测任务的核心是对输入的文档图像进行内容解析和区域划分。通过识别图像中的不同元素（如文字、图表、图像、公式、段落、摘要、参考文献等），将其归类为预定义的类别，并确定这些区域在文档中的位置。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
* <b>版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</b>
|
||||
|
||||
* <b>文档图像版面子模块检测，包含1个 版面区域 类别，能检测多栏的报纸、杂志的每个子文章的文本区域：</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的文档图像版面子模块检测模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</b>
|
||||
|
||||
* <b>版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、杂志、合同、书本、试卷和研报等场景的自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">训练模型</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td>基于PicoDet-L在包含中英文论文、杂志、合同、书本、试卷和研报等场景的自建数据集训练的精度效率平衡的版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td>基于PicoDet-S在中英文论文、杂志、合同、书本、试卷和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、报纸、研报和试卷等 500 张文档类型图片。</b>
|
||||
|
||||
> ❗ 以上列出的是版面检测模块重点支持的<b>5个核心模型</b>，该模块总共支持<b>13个全量模型</b>，包含多个预定义了不同类别的模型，完整的模型列表如下：
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
|
||||
* <b>表格版面检测模型</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">训练模型</a></td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td>基于PicoDet-1x在自建数据集训练的高效率版面区域定位模型，可定位表格这1类区域</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3类版面检测模型，包含表格、图像、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td>基于PicoDet-S轻量模型在中英文论文、杂志和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td>基于PicoDet-L在中英文论文、杂志和研报等场景上自建数据集训练的效率精度均衡版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td>基于RT-DETR-H在中英文论文、杂志和研报等场景上自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5类英文文档区域检测模型，包含文字、标题、表格、图片以及列表</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">训练模型</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td>基于PicoDet-1x在PubLayNet数据集训练的高效率英文文档版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17类区域检测模型，包含17个版面常见类别，分别是：段落标题、图片、文本、数字、摘要、内容、图表标题、公式、表格、表格标题、参考文献、文档标题、脚注、页眉、算法、页脚、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td>基于PicoDet-S轻量模型在中英文论文、杂志和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td>基于PicoDet-L在中英文论文、杂志和研报等场景上自建数据集训练的效率精度均衡版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td>基于RT-DETR-H在中英文论文、杂志和研报等场景上自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>
|
||||
<ul>
|
||||
<li>20类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</li>
|
||||
<li>1类版面子区域检测模型： PaddleOCR 自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</li>
|
||||
<li>23类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。</li>
|
||||
<li>3类版面检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 1154 张文档类型图片。</li>
|
||||
<li>5类英文文档区域检测模型： <a href="https://developer.ibm.com/exchanges/data/all/publaynet" target="_blank">PubLayNet</a> 的评估数据集，包含英文文档的 11245 张图片。</li>
|
||||
<li>17类区域检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 892 张文档类型图片。</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</details>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将版面区域检测模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayout_plus-L",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(
|
||||
model_name="PP-DocLayout_plus-L",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：输入的待预测图像的路径</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>boxes</code>：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
<ol start="1" type="1">
|
||||
<li><code>cls_id</code>：类别ID，一个整数</li>
|
||||
<li><code>label</code>：类别标签，一个字符串</li>
|
||||
<li><code>score</code>：目标框置信度，一个浮点数</li>
|
||||
<li><code>coordinate</code>：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code></li>
|
||||
</ol>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res_plus.jpg"/>
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>LayoutDetection</code>实例化目标检测模型（此处以<code>PP-DocLayout_plus-L</code>为例），具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-DocLayout-L</code></td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td><b>含义：</b>输入图像大小。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640、高为512。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>含义：</b>用于过滤掉低置信度预测结果的阈值。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>float</b>：如<code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>float</code>类型阈值。如<code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对ID为0的类别应用阈值0.45、ID为1的类别应用阈值0.48、ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>含义：</b>是否使用NMS后处理，过滤重叠框。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>bool</b>表示使用/不使用NMS进行检测框的后处理过滤重叠框。</li>
|
||||
<li><b>None</b>使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>含义：</b>检测框的边长缩放倍数。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>float</b>：大于0的浮点数，如<code>1.1</code>，表示将模型输出的检测框中心不变，宽和高都扩张1.1倍。</li>
|
||||
<li><b>list</b>：如<code>[1.2, 1.5]</code>，表示将模型输出的检测框中心不变，宽度扩张1.2倍，高度扩张1.5倍。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>tuple</code>类型，如<code>{0: (1.1, 2.0)}</code>，表示将模型输出的第0类别检测框中心不变，宽度扩张1.1倍，高度扩张2.0倍。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>含义：</b>模型输出的检测框的合并处理模式。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>"large"</b>：设置为<code>"large"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留外部最大的框，删除重叠的内部框。</li>
|
||||
<li><b>"small"</b>：设置为<code>"small"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留内部被包含的小框，删除重叠的外部框。</li>
|
||||
<li><b>"union"</b>：不进行框的过滤处理，内外框都保留。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>str</code>类型, 如<code>{0: "large", 2: "small"}</code>, 表示对第0类别检测框使用<code>large</code>模式，对第2类别检测框使用<code>small</code>。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* 调用目标检测模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 、<code>batch_size</code>和<code>threshold</code>，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供版面区域检测模块的训练，因此，如果需要训练版面区域测模型，可以参考 [PaddleX 版面区域检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocLayout_plus-L</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.92</td>
|
||||
<td>26.11</td>
|
||||
<td>0.16</td>
|
||||
<td>37.38</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.09</td>
|
||||
<td>72.91</td>
|
||||
<td>0.16</td>
|
||||
<td>85.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>12.65</td>
|
||||
<td>37.91</td>
|
||||
<td>0.75</td>
|
||||
<td>52.24</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.14</td>
|
||||
<td>12.35</td>
|
||||
<td>0.14</td>
|
||||
<td>21.81</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-DocBlockLayout</td>
|
||||
<td>paddle_static</td>
|
||||
<td>9.51</td>
|
||||
<td>27.59</td>
|
||||
<td>0.08</td>
|
||||
<td>37.41</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>8.94</td>
|
||||
<td>70.77</td>
|
||||
<td>0.07</td>
|
||||
<td>80.73</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>11.37</td>
|
||||
<td>37.95</td>
|
||||
<td>0.75</td>
|
||||
<td>50.96</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>7.53</td>
|
||||
<td>9.22</td>
|
||||
<td>0.06</td>
|
||||
<td>16.97</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 版面区域检测模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 六、FAQ
|
||||
@@ -0,0 +1,3 @@
|
||||
# Module Overview
|
||||
|
||||
A module is the smallest unit that implements basic functionality. Modules typically use a single model to accomplish specific tasks, such as text detection, image classification, and other basic functions. As fundamental building blocks, modules provide the necessary functional support for more complex application scenarios. This design approach allows users to flexibly select and combine different modules according to their needs, thereby simplifying the development process and enhancing development flexibility and efficiency.
|
||||
@@ -0,0 +1,3 @@
|
||||
# 模块概述
|
||||
|
||||
模块是实现基本功能的最小单位。模块通常使用单个模型去完成特定的任务，比如文本检测、图像分类等基本功能。模块作为基础构建单元，为更复杂的应用场景提供了必要的功能支持。这种设计方式使得用户可以根据需要灵活选择和组合不同的模块，从而简化了开发流程，并提高了开发的灵活性和效率。
|
||||
@@ -0,0 +1,530 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Seal Text Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The seal text detection module typically outputs multi-point bounding boxes around text regions, which are then passed as inputs to the distortion correction and text recognition modules for subsequent processing to identify the textual content of the seal. Recognizing seal text is an integral part of document processing and finds applications in various scenarios such as contract comparison, inventory access auditing, and invoice reimbursement verification. The seal text detection module serves as a subtask within OCR (Optical Character Recognition), responsible for locating and marking the regions containing seal text within an image. The performance of this module directly impacts the accuracy and efficiency of the entire seal text OCR system.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model Name</th><th>Model Download Link</th>
|
||||
<th>Hmean（%）</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td>The server-side seal text detection model of PP-OCRv4 boasts higher accuracy and is suitable for deployment on better-equipped servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.6</td>
|
||||
<td>The mobile-side seal text detection model of PP-OCRv4, on the other hand, offers greater efficiency and is suitable for deployment on end devices.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> A Self-built Internal Dataset, Containing 500 Images of Circular Stamps.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
<ul>
|
||||
<li> <code>input_path</code>：represents the path of the input image to be predicted</li>
|
||||
<li> <code>dt_polys</code>：represents the predicted text detection boxes, where each text detection box contains multiple vertices of a polygon. Each vertex is a list of two elements, representing the x and y coordinates of the vertex respectively</li>
|
||||
<li> <code>dt_scores</code>：represents the confidence scores of the predicted text detection boxes</li>
|
||||
</ul>
|
||||
|
||||
The visualization image is as follows:
|
||||
|
||||
<img alt="Visualization Image" src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
The explanations of related methods and parameters are as follows:
|
||||
|
||||
* <code>SealTextDetection</code> instantiates a text detection model (here we take <code>PP-OCRv4_server_seal_det</code> as an example), and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b>Model name. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-OCRv4_mobile_seal_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b>Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, and <code>transformers</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine. <br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>Meaning:</b>Limit on the side length of the input image for detection. <br/>
|
||||
<b>Description:</b>
|
||||
<code>int</code> specifies the value. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>Meaning:</b>Type of image side length limitation. <br/>
|
||||
<b>Description:</b>
|
||||
<code>"min"</code> ensures the shortest side of the image is no less than <code>det_limit_side_len</code>; <code>"max"</code> ensures the longest side is no greater than <code>limit_side_len</code>. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>Meaning:</b>Pixel score threshold. <br/>
|
||||
<b>Description:</b>
|
||||
Pixels in the output probability map with scores greater than this threshold are considered text pixels. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>Meaning:</b>If the average score of all pixels inside the bounding box is greater than this threshold, the result is considered a text region. <br/>
|
||||
<b>Description:</b>
|
||||
Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Expansion ratio for the Vatti clipping algorithm, used to expand the text region. <br/>
|
||||
<b>Description:</b>Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td><b>Meaning:</b>Input image size for the model in the format <code>(C, H, W)</code>. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the model's default size will be used.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The <code>predict()</code> method of the seal text detection model is called for inference prediction. The parameters of the <code>predict()</code> method include <code>input</code> 、<code>batch_size</code>、 <code>limit_side_len</code>、 <code>limit_type</code>、 <code>thresh</code>、 <code>box_thresh</code>、 <code>max_candidates</code>、<code>unclip_ratio</code>. The specific descriptions are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b>Input data to be predicted. Required. <br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
<ul>
|
||||
<li>Local image or PDF file path: <code>/root/data/img.jpg</code>;</li>
|
||||
<li><b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;</li>
|
||||
<li><b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
</li>
|
||||
</ul>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size. <br/>
|
||||
<b>Description:</b>
|
||||
Can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results. Each sample's prediction result is a corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a file in JSON format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining visual images with results and prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visual image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the above model is still not performing well in your scenario, you can try the following steps for secondary development. Here, we'll use training `PP-OCRv4_server_seal_det` as an example; you can replace it with the corresponding configuration files for other models. First, you need to prepare a text detection dataset. You can refer to the format of the [seal text detection demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar) for preparation. Once prepared, you can follow the steps below for model training and export. After export, you can quickly integrate the model into the above API. This example uses a seal text detection demo dataset. Before training the model, please ensure that you have installed the dependencies required by PaddleOCR as per the [installation documentation](../installation.en.md).
|
||||
|
||||
### 4.1 Dataset and Pre-trained Model Preparation
|
||||
|
||||
#### 4.1.1 Preparing the Dataset
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
#### 4.1.1 Preparing the pre-trained model
|
||||
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR has modularized the code, and when training the `PP-OCRv4_server_seal_det` model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml) for `PP-OCRv4_server_seal_det`.
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single GPU training (default training method)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Train.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/train.txt \
|
||||
Eval.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Eval.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/val.txt
|
||||
|
||||
# Multi-GPU training, specify GPU ids using the --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Train.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/train.txt \
|
||||
Eval.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Eval.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/val.txt
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Make sure to set the pretrained_model path to the local path. If using a model that was trained and saved by yourself, be sure to modify the path and filename to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in the `./PP-OCRv4_server_seal_det_infer/` directory. In this directory, you will see the following files:
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
With this, the secondary development is complete, and the static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
## 5. FAQ
|
||||
@@ -0,0 +1,526 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 印章文本检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本检测模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4的服务端印章文本检测模型，精度更高，适合在较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4的移动端印章文本检测模型，效率更高，适合在端侧部署</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部数据集，包含500张圆形印章图像。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将印章文本检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li> <code>input_path</code>：表示输入待预测图像的路径</li>
|
||||
<li> <code>dt_polys</code>：表示预测的文本检测框，其中每个文本检测框包含一个多边形的多个顶点。其中每个顶点都是一个列表，分别表示该顶点的x坐标和y坐标</li>
|
||||
<li> <code>dt_scores</code>：表示预测的文本检测框的置信度</li>
|
||||
</ul>
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>SealTextDetection</code>实例化文本检测模型（此处以<code>PP-OCRv4_server_seal_det</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-OCRv4_mobile_seal_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>含义：</b>检测的图像边长限制：<code>int</code> 表示边长限制数值<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>int|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>含义：</b>检测的图像边长限制，检测的边长限制类型<br/>
|
||||
<b>说明：</b>
|
||||
<code>"min"</code> 表示保证图像最短边不小于 det_limit_side_len，<code>"max"</code> 表示保证图像最长边不大于 <code>limit_side_len</code>。如果设置为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>含义：</b>像素得分阈值。<br/>
|
||||
<b>说明：</b>
|
||||
输出概率图中得分大于该阈值的像素点被认为是文本像素。可选大于0的float任意浮点数。如果设置为<code>None</code>。将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>含义：</b>检测结果边框内，所有像素点的平均得分大于该阈值时，该结果会被认为是文字区域。<br/>
|
||||
<b>说明：</b>
|
||||
可选大于0的float任意浮点数。如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>含义：</b>Vatti clipping算法的扩张系数，使用该方法对文字区域进行扩张。<br/>
|
||||
<b>说明：</b>
|
||||
可选大于0的任意浮点数。如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td><b>含义：</b>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。<br/>
|
||||
<b>说明：</b>
|
||||
若为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>tuple|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用印章文本检测模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 、<code>batch_size</code>、 <code>limit_side_len</code>、 <code>limit_type</code>、 <code>thresh</code>、 <code>box_thresh</code>、 <code>max_candidates</code>、<code>unclip_ratio</code>，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 <code>PP-OCRv4_server_seal_det</code> 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[印章文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以印章文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv4_server_seal_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv4_server_seal_det` 模型时需要使用 `PP-OCRv4_server_seal_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Train.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/train.txt \
|
||||
Eval.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Eval.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/val.txt
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Train.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/train.txt \
|
||||
Eval.dataset.data_dir=./dataset/ocr_curve_det_dataset_examples Eval.dataset.label_file_list=./dataset/ocr_curve_det_dataset_examples/val.txt
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv4_server_seal_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
@@ -0,0 +1,537 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Table Cell Detection Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Table Cell Detection Module is a key component of the table recognition task, responsible for locating and marking each cell region in table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Cell Detection Module typically outputs bounding boxes for each cell region, which are then passed as input to the table recognition pipeline for further processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Regular Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td rowspan="2">RT-DETR is a real-time end-to-end object detection model. The Baidu PaddlePaddle Vision team pre-trained on a self-built table cell detection dataset based on the RT-DETR-L as the base model, achieving good performance in detecting both wired and wireless table cells.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">Training Model</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Internal evaluation set built by PaddleX.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate model inference from the table cell detection module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(model_name="RT-DETR-L_wired_table_cell_det")
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(
|
||||
model_name="RT-DETR-L_wired_table_cell_det",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(
|
||||
model_name="RT-DETR-L_wired_table_cell_det",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'boxes': [{'cls_id': 0, 'label': 'cell', 'score': 0.9698355197906494, 'coordinate': [2.3011515, 0, 546.29926, 30.530712]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9690820574760437, 'coordinate': [212.37508, 64.62493, 403.58868, 95.61413]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9668057560920715, 'coordinate': [212.46791, 30.311079, 403.7182, 64.62613]}, {'cls_id': 0, 'label': 'cell', 'score': 0.966505229473114, 'coordinate': [403.56082, 64.62544, 546.83215, 95.66117]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9662341475486755, 'coordinate': [109.48873, 64.66485, 212.5177, 95.631294]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9654079079627991, 'coordinate': [212.39197, 95.63037, 403.60852, 126.78792]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9653300642967224, 'coordinate': [2.2320926, 64.62229, 109.600494, 95.59732]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9639787673950195, 'coordinate': [403.5752, 30.562355, 546.98975, 64.61531]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9636150002479553, 'coordinate': [2.1537683, 30.410172, 109.568306, 64.62762]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631900191307068, 'coordinate': [2.0534437, 95.57448, 109.57601, 126.71458]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631181359291077, 'coordinate': [403.65976, 95.68139, 546.84766, 126.713394]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9614537358283997, 'coordinate': [109.56504, 30.391184, 212.65425, 64.6444]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9607433080673218, 'coordinate': [109.525795, 95.62622, 212.44917, 126.8258]}]}}
|
||||
```
|
||||
|
||||
The parameter meanings are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>: Path of the input image to be predicted</li>
|
||||
<li><code>page_index</code>: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`</li>
|
||||
<li><code>boxes</code>: Predicted bounding box information, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
<ol start = "1" type ="1">
|
||||
<li><code>cls_id</code>: Class ID, an integer</li>
|
||||
<li><code>label</code>: Class label, a string</li>
|
||||
<li><code>score</code>: Confidence of the bounding box, a float</li>
|
||||
<li><code>coordinate</code>: Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code></li>
|
||||
</ol>
|
||||
</li>
|
||||
</ul>
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_cells_detection/01.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* <code>TableCellsDetection</code> instantiates the table cell detection model (taking <code>RT-DETR-L_wired_table_cell_det</code> as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>RT-DETR-L_wired_table_cell_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b>Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td><b>Meaning:</b>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td><b>Meaning:</b>Input image size.<br/>
|
||||
<b>Description:</b>
|
||||
<ul><li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640</li><li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to 640 width and 512 height</li></ul></td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>Meaning:</b>Threshold for filtering out low-confidence prediction results.<br/>
|
||||
<b>Description:</b>
|
||||
<ul><li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li><li><b>dict</b>: keys are <code>int</code> (class id), values are <code>float</code> thresholds, e.g. <code>{0: 0.45, 2: 0.48, 7: 0.4}</code>, applies thresholds to specific classes.</li><li><b>None</b>: uses the model's default configuration.</li></ul></td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the table cell detection model for inference prediction. This method will return a result list. Additionally, this module also provides a <code>predict_iter()</code> method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that <code>predict_iter()</code> returns a <code>generator</code>, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The <code>predict()</code> method has parameters <code>input</code>, <code>batch_size</code>, and <code>threshold</code>, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Input data to be predicted. Required.<br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size.<br/>
|
||||
<b>Description:</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters.<br/>
|
||||
<b>Description:</b>If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the table cell detection module, if you need to train a table cell detection model, you can refer to the [PaddleX Table Cell Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_cells_detection.html#iv-secondary-development) section for training. The trained model can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">RT-DETR-L_wired_table_cell_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>3.59</td>
|
||||
<td>23.11</td>
|
||||
<td>0.14</td>
|
||||
<td>27.02</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>4.04</td>
|
||||
<td>70.38</td>
|
||||
<td>0.15</td>
|
||||
<td>75.49</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>3.69</td>
|
||||
<td>37.30</td>
|
||||
<td>0.71</td>
|
||||
<td>42.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>2.70</td>
|
||||
<td>9.10</td>
|
||||
<td>0.12</td>
|
||||
<td>12.07</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>3.77</td>
|
||||
<td>23.44</td>
|
||||
<td>0.14</td>
|
||||
<td>27.52</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>4.01</td>
|
||||
<td>69.97</td>
|
||||
<td>0.15</td>
|
||||
<td>75.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>3.69</td>
|
||||
<td>37.11</td>
|
||||
<td>0.71</td>
|
||||
<td>41.91</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>2.77</td>
|
||||
<td>9.11</td>
|
||||
<td>0.12</td>
|
||||
<td>12.14</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Table Cell Detection Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_cells_detection.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,548 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格单元格检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格单元格检测模块是表格识别任务的关键组成部分，负责在表格图像中定位和标记每个单元格区域，该模块的性能直接影响到整个表格识别过程的准确性和效率。表格单元格检测模块通常会输出各个单元格区域的边界框（Bounding Boxes），这些边界框将作为输入传递给表格识别相关产线进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td rowspan="2">RT-DETR 是一个实时的端到端目标检测模型。百度飞桨视觉团队基于 RT-DETR-L 作为基础模型，在自建表格单元格检测数据集上完成预训练，实现了对有线表格、无线表格均有较好性能的表格单元格检测。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部评测集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格单元格检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(model_name="RT-DETR-L_wired_table_cell_det")
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(
|
||||
model_name="RT-DETR-L_wired_table_cell_det",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(
|
||||
model_name="RT-DETR-L_wired_table_cell_det",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'boxes': [{'cls_id': 0, 'label': 'cell', 'score': 0.9698355197906494, 'coordinate': [2.3011515, 0, 546.29926, 30.530712]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9690820574760437, 'coordinate': [212.37508, 64.62493, 403.58868, 95.61413]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9668057560920715, 'coordinate': [212.46791, 30.311079, 403.7182, 64.62613]}, {'cls_id': 0, 'label': 'cell', 'score': 0.966505229473114, 'coordinate': [403.56082, 64.62544, 546.83215, 95.66117]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9662341475486755, 'coordinate': [109.48873, 64.66485, 212.5177, 95.631294]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9654079079627991, 'coordinate': [212.39197, 95.63037, 403.60852, 126.78792]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9653300642967224, 'coordinate': [2.2320926, 64.62229, 109.600494, 95.59732]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9639787673950195, 'coordinate': [403.5752, 30.562355, 546.98975, 64.61531]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9636150002479553, 'coordinate': [2.1537683, 30.410172, 109.568306, 64.62762]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631900191307068, 'coordinate': [2.0534437, 95.57448, 109.57601, 126.71458]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631181359291077, 'coordinate': [403.65976, 95.68139, 546.84766, 126.713394]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9614537358283997, 'coordinate': [109.56504, 30.391184, 212.65425, 64.6444]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9607433080673218, 'coordinate': [109.525795, 95.62622, 212.44917, 126.8258]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：输入的待预测图像的路径</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>boxes</code>：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
<ol start="1" type="1">
|
||||
<li><code>cls_id</code>：类别ID，一个整数</li>
|
||||
<li><code>label</code>：类别标签，一个字符串</li>
|
||||
<li><code>score</code>：目标框置信度，一个浮点数</li>
|
||||
<li><code>coordinate</code>：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code></li>
|
||||
</ol>
|
||||
</li>
|
||||
</ul>
|
||||
可视化图像如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_cells_detection/01.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>TableCellsDetection</code>实例化表格单元格检测模型（此处以<code>RT-DETR-L_wired_table_cell_det</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>RT-DETR-L_wired_table_cell_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td><b>含义：</b>输入图像大小。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640，高为512大小。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>含义：</b>用于过滤掉低置信度预测结果的阈值。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>float</b>：如 <code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。/li>
|
||||
<li><b>dict</b>：字典的键为 <code>int</code> 类型，代表类别ID；值为 <code>float</code> 类型阈值。如 <code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对类别ID为0的类别应用阈值0.45、类别ID为1的类别应用阈值0.48、类别ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认配置。
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 调用目标检测模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code>、<code>batch_size</code>和<code>threshold</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供表格单元格检测模块的训练，因此，如果需要训练表格单元格检测模型，可以参考 [PaddleX 表格单元格检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_cells_detection.html#_4)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">RT-DETR-L_wired_table_cell_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>3.59</td>
|
||||
<td>23.11</td>
|
||||
<td>0.14</td>
|
||||
<td>27.02</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>4.04</td>
|
||||
<td>70.38</td>
|
||||
<td>0.15</td>
|
||||
<td>75.49</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>3.69</td>
|
||||
<td>37.30</td>
|
||||
<td>0.71</td>
|
||||
<td>42.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>2.70</td>
|
||||
<td>9.10</td>
|
||||
<td>0.12</td>
|
||||
<td>12.07</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>3.77</td>
|
||||
<td>23.44</td>
|
||||
<td>0.14</td>
|
||||
<td>27.52</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>4.01</td>
|
||||
<td>69.97</td>
|
||||
<td>0.15</td>
|
||||
<td>75.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>3.69</td>
|
||||
<td>37.11</td>
|
||||
<td>0.71</td>
|
||||
<td>41.91</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>2.77</td>
|
||||
<td>9.11</td>
|
||||
<td>0.12</td>
|
||||
<td>12.14</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 表格单元格检测模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_cells_detection.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 六、FAQ
|
||||
@@ -0,0 +1,463 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Table Classification Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Table Classification Module is a key component in computer vision systems, responsible for classifying input table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Classification Module typically receives table images as input and, using deep learning algorithms, classifies them into predefined categories based on the characteristics and content of the images, such as wired and wireless tables. The classification results from the Table Classification Module serve as output for use in table recognition pipelines.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Regular Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Internal evaluation dataset built by PaddleX.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate model inference from the table classification module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(model_name="PP-LCNet_x1_0_table_cls")
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(
|
||||
model_name="PP-LCNet_x1_0_table_cls",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(
|
||||
model_name="PP-LCNet_x1_0_table_cls",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'class_ids': array([0, 1], dtype=int32), 'scores': array([0.84421, 0.15579], dtype=float32), 'label_names': ['wired_table', 'wireless_table']}}
|
||||
```
|
||||
|
||||
The parameter meanings are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>: Path of the input image</li>
|
||||
<li><code>page_index</code>: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is <code>None</code></li>
|
||||
<li><code>class_ids</code>: Class IDs of the prediction results</li>
|
||||
<li><code>scores</code>: Confidence scores of the prediction results</li>
|
||||
<li><code>label_names</code>: Class names of the prediction results</li>
|
||||
</ul>
|
||||
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_classification/01.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* <code>TableClassification</code> instantiates the table classification model (taking <code>PP-LCNet_x1_0_table_cls</code> as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b>Model name. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-LCNet_x1_0_table_cls</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b>Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine. <br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td><b>Meaning:</b>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the table classification model for inference prediction. This method will return a result list. Additionally, this module also provides a <code>predict_iter()</code> method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that <code>predict_iter()</code> returns a <code>generator</code>, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The <code>predict()</code> method has parameters <code>input</code> and <code>batch_size</code>, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Input data to be predicted. Required.<br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size.<br/>
|
||||
<b>Description:</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the table classification module, if you need to train a table classification model, you can refer to the [PaddleX Table Classification Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_classification.html#iv-secondary-development) section for training. The trained model can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x1_0_table_cls</td>
|
||||
<td>paddle_static</td>
|
||||
<td>3.56</td>
|
||||
<td>3.38</td>
|
||||
<td>0.06</td>
|
||||
<td>7.11</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>3.57</td>
|
||||
<td>7.77</td>
|
||||
<td>0.07</td>
|
||||
<td>11.52</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>9.30</td>
|
||||
<td>3.72</td>
|
||||
<td>0.15</td>
|
||||
<td>14.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>3.23</td>
|
||||
<td>0.88</td>
|
||||
<td>0.05</td>
|
||||
<td>4.25</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_image_classification_001.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Table Classification Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_classification.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,464 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格分类模块是计算机视觉系统中的关键组成部分，负责对输入的表格图像进行分类，该模块的性能直接影响到整个表格识别过程的准确性和效率。表格分类模块通常会接收表格图像作为输入，然后通过深度学习算法，根据图像的特性和内容，将其分类到预定义的类别中，例如有线表和无线表。表格分类模块的分类结果将作为输出，供表格识别相关产线使用。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部评测数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格分类的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(model_name="PP-LCNet_x1_0_table_cls")
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(
|
||||
model_name="PP-LCNet_x1_0_table_cls",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(
|
||||
model_name="PP-LCNet_x1_0_table_cls",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'class_ids': array([0, 1], dtype=int32), 'scores': array([0.84421, 0.15579], dtype=float32), 'label_names': ['wired_table', 'wireless_table']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：表示输入图片的路径</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>class_ids</code>：表示预测结果的类别id</li>
|
||||
<li><code>scores</code>：表示预测结果的置信度</li>
|
||||
<li><code>label_names</code>：表示预测结果的类别名</li>
|
||||
</ul>
|
||||
|
||||
可视化图像如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_classification/01.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>TableClassification</code>实例化表格分类模型（此处以<code>PP-LCNet_x1_0_table_cls</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_table_cls</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用表格分类模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 和 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供表格分类模块的训练，因此，如果需要训练表格分类模型，可以参考 [PaddleX 表格分类模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x1_0_table_cls</td>
|
||||
<td>paddle_static</td>
|
||||
<td>3.56</td>
|
||||
<td>3.38</td>
|
||||
<td>0.06</td>
|
||||
<td>7.11</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>3.57</td>
|
||||
<td>7.77</td>
|
||||
<td>0.07</td>
|
||||
<td>11.52</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>9.30</td>
|
||||
<td>3.72</td>
|
||||
<td>0.15</td>
|
||||
<td>14.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>3.23</td>
|
||||
<td>0.88</td>
|
||||
<td>0.05</td>
|
||||
<td>4.25</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_image_classification_001.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 表格分类模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_classification.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 六、FAQ
|
||||
@@ -0,0 +1,598 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Table Structure Recognition Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Table structure recognition is an important component of table recognition systems, capable of converting non-editable table images into editable table formats (such as HTML). The goal of table structure recognition is to identify the positions of rows, columns, and cells in tables. The performance of this module directly affects the accuracy and efficiency of the entire table recognition system. The table structure recognition module usually outputs HTML code for the table area, which is then passed as input to the table recognition pipeline for further processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Accuracy (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">Training Model</a></td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet is a table structure recognition model independently developed by Baidu PaddlePaddle Vision Team. By adopting a CPU-friendly lightweight backbone network PP-LCNet, high-low level feature fusion module CSP-PAN, and SLA Head, a feature decoding module aligning structure and position information, this model greatly improves the accuracy and inference speed of table structure recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">Training Model</a></td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet_plus is an enhanced version of the table structure recognition model SLANet independently developed by the Baidu PaddlePaddle Vision Team. Compared to SLANet, SLANet_plus has greatly improved the recognition ability for wireless and complex tables, and reduced the model's sensitivity to table positioning accuracy. Even if the table positioning is offset, it can still be accurately recognized.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">Training Model</a></td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td rowspan="2">The SLANeXt series is a new generation of table structure recognition models independently developed by the Baidu PaddlePaddle Vision Team. Compared to SLANet and SLANet_plus, SLANeXt focuses on table structure recognition, and trains dedicated weights for wired and wireless tables separately. The recognition ability for all types of tables has been significantly improved, especially for wired tables.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">Training Model</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> High-difficulty Chinese table recognition dataset.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Selects the prior optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. For details, please refer to the [Installation Tutorial](../installation.en.md).
|
||||
|
||||
Quickly experience with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference of the table structure recognition module into your own project. Before running the code below, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(model_name="SLANet")
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(
|
||||
model_name="SLANet",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(
|
||||
model_name="SLANet",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'bbox': [[42, 2, 390, 2, 388, 27, 40, 26], [11, 35, 89, 35, 87, 63, 11, 63], [113, 34, 192, 34, 186, 64, 109, 64], [219, 33, 399, 33, 393, 62, 212, 62], [413, 33, 544, 33, 544, 64, 407, 64], [12, 67, 98, 68, 96, 93, 12, 93], [115, 66, 205, 66, 200, 91, 111, 91], [234, 65, 390, 65, 385, 92, 227, 92], [414, 66, 537, 67, 537, 95, 409, 95], [7, 97, 106, 97, 104, 128, 7, 128], [113, 96, 206, 95, 201, 127, 109, 127], [236, 96, 386, 96, 381, 128, 230, 128], [413, 96, 534, 95, 533, 127, 408, 127]], 'structure': ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="4"', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '</table>', '</body>', '</html>'], 'structure_score': 0.99948007}}
|
||||
```
|
||||
|
||||
Parameter meanings are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>：The path of the input table image to be predicted</li>
|
||||
<li><code>page_index</code>：If the input is a PDF file, indicates the page number of the PDF; otherwise, it is <code>None</code></li>
|
||||
<li><code>boxes</code>： Predicted table cell information, a list consisting of the coordinates of predicted table cells. Notably, table cell predictions for the SLANeXt series models are invalid</li>
|
||||
<li><code>structure</code>：Predicted table structure HTML expressions, a list consisting of predicted HTML keywords in order</li>
|
||||
<li><code>structure_score</code>：Confidence of the predicted table structure</li>
|
||||
</ul>
|
||||
|
||||
Descriptions of related methods and parameters are as follows:
|
||||
|
||||
* <code>TableStructureRecognition</code> instantiates a table structure recognition model (using <code>SLANet</code> as an example). Details are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-LCNet_x1_0_table_cls</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b>Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td><b>Meaning:</b>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the table structure recognition model for inference prediction, which returns a result list. In addition, this module also provides the <code>predict_iter()</code> method. The two are completely consistent in parameter acceptance and result return. The difference is that <code>predict_iter()</code> returns a <code>generator</code>, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where you want to save memory. You can choose to use either method according to your actual needs. The <code>predict()</code> method has parameters <code>input</code> and <code>batch_size</code>, described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Data to be predicted. Required. <br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local path to an image or PDF file, e.g., <code>/root/data/img.jpg</code>; <b>URL</b>: Network URL to an image or PDF file, e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: A local directory containing images for prediction, e.g., <code>/root/data/</code> (Note: PDF files in directories are not supported; to predict a PDF, specify its file path directly)</li>
|
||||
<li><b>list</b>: A list of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size. <br/>
|
||||
<b>Description:</b>
|
||||
Can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
* For processing prediction results, the prediction result of each sample is the corresponding Result object, and supports printing and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to use <code>JSON</code> indentation formatting for the output</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> keeps the original characters. Effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save result as json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it's a directory, the saved file will be named the same as the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> keeps the original characters. Effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
If the above models are still not ideal for your scenario, you can try the following steps for secondary development. Here, training `SLANet_plus` is used as an example, and for other models, just replace the corresponding configuration file. First, you need to prepare a dataset for table structure recognition, which can be prepared with reference to the format of the [table structure recognition demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar). Once ready, you can train and export the model as follows. After exporting, you can quickly integrate the model into the above API. Here, the table structure recognition demo data is used as an example. Before training the model, please make sure you have installed the dependencies required by PaddleOCR according to the [installation documentation](../installation.en.md).
|
||||
|
||||
|
||||
## 4.1 Dataset and Pretrained Model Preparation
|
||||
|
||||
### 4.1.1 Prepare Dataset
|
||||
|
||||
```shell
|
||||
# Download sample dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar
|
||||
tar -xf table_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
### 4.1.2 Download Pretrained Model
|
||||
|
||||
```shell
|
||||
# Download SLANet_plus pretrained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR is modularized. When training the `SLANet_plus` recognition model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/table/SLANet_plus.yml) of `SLANet_plus`.
|
||||
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single card training (default training method)
|
||||
python3 tools/train.py -c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
|
||||
# Multi-card training, specify card numbers via --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Note to set the path of pretrained_model to the local path. If you use the model saved by your own training, please modify the path and file name to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./SLANet_plus_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in `./SLANet_plus_infer/` in the current directory. In this directory, you will see the following files:
|
||||
```
|
||||
./SLANet_plus_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, secondary development is complete, and this static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="1">SLANet</td>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.42</td>
|
||||
<td>22.02</td>
|
||||
<td>0.22</td>
|
||||
<td>23.81</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1">SLANet_plus</td>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.40</td>
|
||||
<td>21.64</td>
|
||||
<td>0.22</td>
|
||||
<td>23.42</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">SLANeXt_wired</td>
|
||||
<td>paddle_static</td>
|
||||
<td>1.50</td>
|
||||
<td>30.91</td>
|
||||
<td>0.23</td>
|
||||
<td>32.77</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>1.71</td>
|
||||
<td>57.44</td>
|
||||
<td>0.91</td>
|
||||
<td>60.23</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>4.03</td>
|
||||
<td>45.14</td>
|
||||
<td>0.74</td>
|
||||
<td>51.12</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.60</td>
|
||||
<td>30.96</td>
|
||||
<td>0.25</td>
|
||||
<td>32.99</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">SLANeXt_wireless</td>
|
||||
<td>paddle_static</td>
|
||||
<td>1.67</td>
|
||||
<td>30.49</td>
|
||||
<td>0.22</td>
|
||||
<td>32.51</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>1.68</td>
|
||||
<td>57.24</td>
|
||||
<td>0.96</td>
|
||||
<td>60.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>4.30</td>
|
||||
<td>45.51</td>
|
||||
<td>0.75</td>
|
||||
<td>51.76</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.62</td>
|
||||
<td>31.01</td>
|
||||
<td>0.26</td>
|
||||
<td>33.06</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Table Structure Recognition Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_structure_recognition.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,602 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格结构识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格结构识别是表格识别系统中的重要组成部分，能够将不可编辑表格图片转换为可编辑的表格形式（例如html）。表格结构识别的目标是对表格的行、列和单元格位置进行识别，该模块的性能直接影响到整个表格识别系统的准确性和效率。表格结构识别模块会输出表格区域的html代码，这些代码将作为输入传递给表格识别产线进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>精度（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">训练模型</a></td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet 是百度飞桨视觉团队自研的表格结构识别模型。该模型通过采用 CPU 友好型轻量级骨干网络 PP-LCNet、高低层特征融合模块 CSP-PAN、结构与位置信息对齐的特征解码模块 SLA Head，大幅提升了表格结构识别的精度和推理速度。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">训练模型</a></td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet_plus 是百度飞桨视觉团队自研的表格结构识别模型 SLANet 的增强版。相较于 SLANet，SLANet_plus 对无线表、复杂表格的识别能力得到了大幅提升，并降低了模型对表格定位准确性的敏感度，即使表格定位出现偏移，也能够较准确地进行识别。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">训练模型</a></td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td rowspan="2">SLANeXt 系列是百度飞桨视觉团队自研的新一代表格结构识别模型。相较于 SLANet 和 SLANet_plus，SLANeXt 专注于对表格结构进行识别，并且对有线表格(wired)和无线表格(wireless)的识别分别训练了专用的权重，对各类型表格的识别能力都得到了明显提高，特别是对有线表格的识别能力得到了大幅提升。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>内部自建的高难度中文表格识别数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格结构识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(model_name="SLANet")
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(
|
||||
model_name="SLANet",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(
|
||||
model_name="SLANet",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'bbox': [[42, 2, 390, 2, 388, 27, 40, 26], [11, 35, 89, 35, 87, 63, 11, 63], [113, 34, 192, 34, 186, 64, 109, 64], [219, 33, 399, 33, 393, 62, 212, 62], [413, 33, 544, 33, 544, 64, 407, 64], [12, 67, 98, 68, 96, 93, 12, 93], [115, 66, 205, 66, 200, 91, 111, 91], [234, 65, 390, 65, 385, 92, 227, 92], [414, 66, 537, 67, 537, 95, 409, 95], [7, 97, 106, 97, 104, 128, 7, 128], [113, 96, 206, 95, 201, 127, 109, 127], [236, 96, 386, 96, 381, 128, 230, 128], [413, 96, 534, 95, 533, 127, 408, 127]], 'structure': ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="4"', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '</table>', '</body>', '</html>'], 'structure_score': 0.99948007}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：输入的待预测表格图像的路径</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>boxes</code>：预测的表格单元格信息，一个列表，由预测的若干表格单元格坐标组成。特别地， SLANeXt 系列模型预测的表格单元格无效</li>
|
||||
<li><code>structure</code>：预测的表格结构Html表达式，一个列表，由预测的若干Html关键字按顺序组成</li>
|
||||
<li><code>structure_score</code>：预测表格结构的置信度</li>
|
||||
</ul>
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>TableStructureRecognition</code>实例化表格结构识别模型（此处以<code>SLANet</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_table_cls</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用表格结构识别模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 和 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小。<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `SLANet_plus` 举例，其他模型替换对应配置文件即可。首先，您需要准备表格结构识别的数据集，可以参考[表格结构识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以表格结构识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar
|
||||
tar -xf table_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 SLANet_plus 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `SLANet_plus` 识别模型时需要使用 `SLANet_plus` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/table/SLANet_plus.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./SLANet_plus_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./SLANet_plus_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./SLANet_plus_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="1">SLANet</td>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.42</td>
|
||||
<td>22.02</td>
|
||||
<td>0.22</td>
|
||||
<td>23.81</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1">SLANet_plus</td>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.40</td>
|
||||
<td>21.64</td>
|
||||
<td>0.22</td>
|
||||
<td>23.42</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">SLANeXt_wired</td>
|
||||
<td>paddle_static</td>
|
||||
<td>1.50</td>
|
||||
<td>30.91</td>
|
||||
<td>0.23</td>
|
||||
<td>32.77</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>1.71</td>
|
||||
<td>57.44</td>
|
||||
<td>0.91</td>
|
||||
<td>60.23</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>4.03</td>
|
||||
<td>45.14</td>
|
||||
<td>0.74</td>
|
||||
<td>51.12</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.60</td>
|
||||
<td>30.96</td>
|
||||
<td>0.25</td>
|
||||
<td>32.99</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">SLANeXt_wireless</td>
|
||||
<td>paddle_static</td>
|
||||
<td>1.67</td>
|
||||
<td>30.49</td>
|
||||
<td>0.22</td>
|
||||
<td>32.51</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>1.68</td>
|
||||
<td>57.24</td>
|
||||
<td>0.96</td>
|
||||
<td>60.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>4.30</td>
|
||||
<td>45.51</td>
|
||||
<td>0.75</td>
|
||||
<td>51.76</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>1.62</td>
|
||||
<td>31.01</td>
|
||||
<td>0.26</td>
|
||||
<td>33.06</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 表格结构识别模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_structure_recognition.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 六、FAQ
|
||||
@@ -0,0 +1,822 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Detection Module Usage Guide
|
||||
|
||||
## 1. Overview
|
||||
The text detection module is a critical component of OCR (Optical Character Recognition) systems, responsible for locating and marking text-containing regions in images. The performance of this module directly impacts the accuracy and efficiency of the entire OCR system. The text detection module typically outputs bounding boxes for text regions, which are then passed to the text recognition module for further processing.
|
||||
|
||||
## 2. Supported Models List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Standard Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Detection Hmean (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Standard Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Standard Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv6_medium_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv6_medium_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv6_medium_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>86.2*</td>
|
||||
<td>- / -</td>
|
||||
<td>- / -</td>
|
||||
<td>59.4</td>
|
||||
<td>PP-OCRv6 medium-scale text detection model based on PPLCNetV4 + RepLKFPN, highest accuracy, suitable for server deployment</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv6_small_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv6_small_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv6_small_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>84.1*</td>
|
||||
<td>- / -</td>
|
||||
<td>- / -</td>
|
||||
<td>9.6</td>
|
||||
<td>PP-OCRv6 small text detection model, balancing accuracy and efficiency, suitable for mobile deployment</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv6_tiny_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv6_tiny_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv6_tiny_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>80.6*</td>
|
||||
<td>- / -</td>
|
||||
<td>- / -</td>
|
||||
<td>1.9</td>
|
||||
<td>PP-OCRv6 ultra-lightweight text detection model (0.43M params), suitable for edge/IoT scenarios</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>84.3</td>
|
||||
<td>PP-OCRv5 server-side text detection model with higher accuracy, suitable for deployment on high-performance servers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv5 mobile-side text detection model with higher efficiency, suitable for deployment on edge devices</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>69.2</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4 server-side text detection model with higher accuracy, suitable for deployment on high-performance servers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4 mobile-side text detection model with higher efficiency, suitable for deployment on edge devices</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> *Note: PP-OCRv6 metrics are evaluated on an internal multi-scenario evaluation set, while PP-OCRv5/v4 metrics are based on a general evaluation set. As the evaluation sets differ, the metrics are not directly comparable.
|
||||
|
||||
<strong>Testing Environment:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Testing Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> PaddleOCR3.0 newly constructed multilingual dataset (including Chinese, Traditional Chinese, English, Japanese), covering street scenes, web images, documents, handwriting, blur, rotation, distortion, etc., totaling 2677 images.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Techniques</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Standard Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision types and acceleration strategies</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Optimal backend selection (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. Refer to the [Installation Guide](../installation.en.md) for details.
|
||||
|
||||
Use the following command for a quick experience:
|
||||
|
||||
```bash
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference into your project. Before running the following code, download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection()
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(engine="transformers")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(engine="onnxruntime")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
The output will be:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_001.png', 'page_index': None, 'dt_polys': array([[[ 75, 549],
|
||||
...,
|
||||
[ 77, 586]],
|
||||
|
||||
...,
|
||||
|
||||
[[ 31, 406],
|
||||
...,
|
||||
[ 34, 455]]], dtype=int16), 'dt_scores': [0.873949039891189, 0.8948166013613552, 0.8842595305917041, 0.876953790920377]}}
|
||||
```
|
||||
|
||||
Output parameter meanings:
|
||||
<ul>
|
||||
<li><code>input_path</code>：Path of the input image.</li>
|
||||
<li><code>page_index</code>：If the input is a PDF, this indicates the current page number; otherwise, it is <code>None</code></li>
|
||||
<li><code>dt_polys</code>：Predicted text detection boxes, where each box contains four vertices (x, y coordinates).</li>
|
||||
<li><code>dt_scores</code>：Confidence scores of the predicted text detection boxes.</li>
|
||||
</ul>
|
||||
|
||||
Visualization example:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_det/general_ocr_001_res.png"/>
|
||||
|
||||
Method and parameter descriptions:
|
||||
|
||||
* Instantiate the text detection model with <code>TextDetection</code>:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b>Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-OCRv6_medium_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b>Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b>Whether to use the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td><b>Meaning:</b>Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td><b>Meaning:</b>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>Meaning:</b>Limit on the side length of the input image for detection.
|
||||
<b>Description:</b>
|
||||
<code>int</code> specifies the value. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>Meaning:</b>Type of image side length limitation. <br/>
|
||||
<b>Description:</b>
|
||||
<code>"min"</code> ensures the shortest side of the image is no less than <code>det_limit_side_len</code>; <code>"max"</code> ensures the longest side is no greater than <code>limit_side_len</code>. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_side_limit</code></td>
|
||||
<td><b>Meaning:</b>Limit on the max length of the input image for detection. <br/>
|
||||
<b>Description:</b>
|
||||
<code>int</code> limits the longest side of the image for input detection model. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>Meaning:</b>Pixel score threshold. Pixels in the output probability map with scores greater than this threshold are considered text pixels. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>Meaning:</b>If the average score of all pixels inside the bounding box is greater than this threshold, the result is considered a text region. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Expansion ratio for the Vatti clipping algorithm, used to expand the text region.
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td><b>Meaning:</b>Input image size for the model in the format <code>(C, H, W)</code>.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The <code>predict()</code> method parameters:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>Input data to be predicted. Required.<br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python variable</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size<br/>
|
||||
<b>Description:</b>Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>Meaning:</b>Same meaning as the instantiation parameters. <br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Result processing methods:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Format output as JSON</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>JSON indentation level</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Escape non-ASCII characters</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save results as JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Output file path</td>
|
||||
<td>Required</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>JSON indentation level</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Escape non-ASCII characters</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save results as image</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Output file path</td>
|
||||
<td>Required</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additional attributes:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>json</code></td>
|
||||
<td>Get prediction results in JSON format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img</code></td>
|
||||
<td>Get visualization image as a dictionary</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
If the above models do not meet your requirements, follow these steps for custom development (using `PP-OCRv5_server_det` as an example). First, prepare a text detection dataset (refer to the [Demo Dataset](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar) format). After preparation, proceed with model training and export. The exported model can be integrated into the API. Ensure PaddleOCR dependencies are installed as per the [Installation Guide](../installation.en.md).
|
||||
|
||||
### 4.1 Dataset and Pretrained Model Preparation
|
||||
|
||||
#### 4.1.1 Prepare Dataset
|
||||
|
||||
```shell
|
||||
# Download example dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar
|
||||
tar -xf ocr_det_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 Download Pretrained Model
|
||||
|
||||
```shell
|
||||
# Download PP-OCRv5_server_det pretrained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR modularizes the code. To train the `PP-OCRv5_server_det` model, use its [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/det/PP-OCRv5/PP-OCRv5_server_det.yml).
|
||||
|
||||
Training command:
|
||||
|
||||
```bash
|
||||
# Single-GPU training (default)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
|
||||
# Multi-GPU training (specify GPUs with --gpus)
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate trained weights (e.g., `output/PP-OCRv5_server_det/best_accuracy.pdparams`) using the following command:
|
||||
|
||||
```bash
|
||||
# Note: Set pretrained_model to local path. For custom-trained models, modify the path and filename as {path/to/weights}/{model_name}.
|
||||
# Demo dataset evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml -o \
|
||||
Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_det_infer/"
|
||||
```
|
||||
|
||||
After export, the static graph model will be saved in `./PP-OCRv5_server_det_infer/` with the following files:
|
||||
```
|
||||
./PP-OCRv5_server_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
The custom development is now complete. This static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv5_mobile_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>11.43</td>
|
||||
<td>13.80</td>
|
||||
<td>2.15</td>
|
||||
<td>27.58</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.70</td>
|
||||
<td>48.36</td>
|
||||
<td>2.47</td>
|
||||
<td>62.71</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>14.05</td>
|
||||
<td>18.45</td>
|
||||
<td>3.98</td>
|
||||
<td>37.54</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.98</td>
|
||||
<td>5.70</td>
|
||||
<td>2.04</td>
|
||||
<td>17.90</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv5_server_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>13.24</td>
|
||||
<td>26.91</td>
|
||||
<td>2.63</td>
|
||||
<td>43.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.82</td>
|
||||
<td>45.56</td>
|
||||
<td>2.52</td>
|
||||
<td>60.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>14.56</td>
|
||||
<td>13.76</td>
|
||||
<td>7.44</td>
|
||||
<td>36.76</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>10.01</td>
|
||||
<td>13.76</td>
|
||||
<td>1.92</td>
|
||||
<td>25.86</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv6_medium_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>13.89</td>
|
||||
<td>16.02</td>
|
||||
<td>2.49</td>
|
||||
<td>33.14</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.42</td>
|
||||
<td>26.23</td>
|
||||
<td>2.30</td>
|
||||
<td>40.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>11.40</td>
|
||||
<td>8.57</td>
|
||||
<td>8.35</td>
|
||||
<td>29.57</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>10.80</td>
|
||||
<td>13.06</td>
|
||||
<td>2.19</td>
|
||||
<td>26.18</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv6_small_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.91</td>
|
||||
<td>10.97</td>
|
||||
<td>2.41</td>
|
||||
<td>24.45</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.56</td>
|
||||
<td>22.17</td>
|
||||
<td>2.66</td>
|
||||
<td>36.55</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>11.70</td>
|
||||
<td>7.34</td>
|
||||
<td>3.87</td>
|
||||
<td>23.89</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>11.32</td>
|
||||
<td>7.46</td>
|
||||
<td>2.54</td>
|
||||
<td>21.49</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv6_tiny_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>11.14</td>
|
||||
<td>10.71</td>
|
||||
<td>2.84</td>
|
||||
<td>24.85</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.52</td>
|
||||
<td>21.70</td>
|
||||
<td>2.94</td>
|
||||
<td>36.31</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>10.90</td>
|
||||
<td>6.99</td>
|
||||
<td>4.13</td>
|
||||
<td>23.00</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>11.19</td>
|
||||
<td>6.35</td>
|
||||
<td>2.79</td>
|
||||
<td>20.49</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Text Detection Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_detection.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 6. FAQ
|
||||
|
||||
- Use parameters `limit_type` and `limit_side_len` to constrain image dimensions.
|
||||
- `limit_type` options: [`max`, `min`]
|
||||
- `limit_side_len`: Positive integer (typically multiples of 32, e.g., 960).
|
||||
- For lower-resolution images, use `limit_type=min` and `limit_side_len=960` to balance computational efficiency and detection quality.
|
||||
- For higher-resolution images requiring larger detection scales, set `limit_side_len` to desired values (e.g., 1216).
|
||||
@@ -0,0 +1,936 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
文本检测模块是OCR（光学字符识别）系统中的关键组成部分，负责在图像中定位和标记出包含文本的区域。该模块的性能直接影响到整个OCR系统的准确性和效率。文本检测模块通常会输出文本区域的边界框（Bounding Boxes），这些边界框将作为输入传递给文本识别模块进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv6_medium_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv6_medium_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv6_medium_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.2*</td>
|
||||
<td>- / -</td>
|
||||
<td>- / -</td>
|
||||
<td>59.4</td>
|
||||
<td>PP-OCRv6 的中等规模文本检测模型，基于 PPLCNetV4 + RepLKFPN，精度最高，适合服务端部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv6_small_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv6_small_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv6_small_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>84.1*</td>
|
||||
<td>- / -</td>
|
||||
<td>- / -</td>
|
||||
<td>9.6</td>
|
||||
<td>PP-OCRv6 的小型文本检测模型，兼顾精度与效率，适合移动端部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv6_tiny_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv6_tiny_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv6_tiny_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>80.6*</td>
|
||||
<td>- / -</td>
|
||||
<td>- / -</td>
|
||||
<td>1.9</td>
|
||||
<td>PP-OCRv6 的超轻量文本检测模型（0.43M 参数），适合端侧/IoT 场景</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>84.3</td>
|
||||
<td>PP-OCRv5 的服务端文本检测模型，精度更高，适合在性能较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv5 的移动端文本检测模型，效率更高，适合在端侧设备部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>69.2</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4 的服务端文本检测模型，精度更高，适合在性能较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4 的移动端文本检测模型，效率更高，适合在端侧设备部署</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> *注：PP-OCRv6 指标基于内部多场景评估集测得，PP-OCRv5/v4 指标基于通用评估集测得，两者评估集不同，指标不可直接对比。
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR3.0 全新构建多语种（包含中、繁、英、日），覆盖街景、网图、文档、手写、模糊、旋转、扭曲等多个场景的文本检测数据集，包含2677 张图片。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
### 3.1 环境准备
|
||||
|
||||
#### 3.1.1 基础安装
|
||||
|
||||
```bash
|
||||
# 安装基础版本（仅包含文本检测功能）
|
||||
pip install paddleocr
|
||||
|
||||
# 安装完整版本（包含所有功能）
|
||||
pip install "paddleocr[all]"
|
||||
```
|
||||
|
||||
#### 3.1.2 环境验证
|
||||
|
||||
```python
|
||||
# 验证安装是否成功
|
||||
import paddleocr
|
||||
print(f"PaddleOCR版本: {paddleocr.__version__}")
|
||||
|
||||
# 验证GPU是否可用
|
||||
import paddle
|
||||
print(f"Paddle版本: {paddle.__version__}")
|
||||
print(f"GPU可用: {paddle.is_compiled_with_cuda()}")
|
||||
print(f"GPU数量: {paddle.device.cuda.device_count()}")
|
||||
```
|
||||
|
||||
### 3.2 命令行快速体验
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
# 使用默认模型进行文本检测
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png
|
||||
|
||||
# 指定模型
|
||||
paddleocr text_detection -i general_ocr_001.png --model_name PP-OCRv6_small_det
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
### 3.3 Python API 使用
|
||||
|
||||
您也可以将文本检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection()
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(engine="transformers")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(engine="onnxruntime")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_001.png', 'page_index': None, 'dt_polys': array([[[ 75, 549],
|
||||
...,
|
||||
[ 77, 586]],
|
||||
|
||||
...,
|
||||
|
||||
[[ 31, 406],
|
||||
...,
|
||||
[ 34, 455]]], dtype=int16), 'dt_scores': [0.873949039891189, 0.8948166013613552, 0.8842595305917041, 0.876953790920377]}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：表示输入待预测图像的路径</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>dt_polys</code>：表示预测的文本检测框，其中每个文本检测框包含一个四边形的四个顶点。其中每个顶点都是一个列表，分别表示该顶点的x坐标和y坐标</li>
|
||||
<li><code>dt_scores</code>：表示预测的文本检测框的置信度</li>
|
||||
</ul>
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_det/general_ocr_001_res.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>TextDetection</code>类实例化文本检测模型，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-OCRv6_medium_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>含义：</b>检测的图像边长限制：<code>int</code> 表示边长限制数值。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>含义：</b>检测的图像边长限制，检测的边长限制类型。<br/>
|
||||
<b>说明：</b>
|
||||
<code>"min"</code> 表示保证图像最短边不小于det_limit_side_len，<code>"max"</code>表示保证图像最长边不大于limit_side_len。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_side_limit</code></td>
|
||||
<td><b>含义：</b>检测的图像边长最大值限制：<code>int</code> 限制输入检测模型的图片最长边。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>含义：</b>像素得分阈值。输出概率图中得分大于该阈值的像素点被认为是文本像素。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>含义：</b>检测结果边框内，所有像素点的平均得分大于该阈值时，该结果会被认为是文字区域。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>含义：</b>Vatti clipping算法的扩张系数，使用该方法对文字区域进行扩张。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td><b>含义：</b>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* 调用文本检测模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 、<code>batch_size</code>、 <code>limit_side_len</code>、 <code>limit_type</code>、 <code>thresh</code>、 <code>box_thresh</code>、 <code>max_candidates</code>、<code>unclip_ratio</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td><b>含义:</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td><b>含义：</b>参数含义与实例化参数基本相同。<br/>
|
||||
<b>说明：</b>
|
||||
设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv5_server_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar
|
||||
tar -xf ocr_det_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv5_server_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_det` 识别模型时需要使用 `PP-OCRv5_server_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/det/PP-OCRv5/PP-OCRv5_server_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/PP-OCRv5_server_det/best_accuracy.pdprams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml -o \
|
||||
Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv5_server_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv5_server_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv5_mobile_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>11.43</td>
|
||||
<td>13.80</td>
|
||||
<td>2.15</td>
|
||||
<td>27.58</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.70</td>
|
||||
<td>48.36</td>
|
||||
<td>2.47</td>
|
||||
<td>62.71</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>14.05</td>
|
||||
<td>18.45</td>
|
||||
<td>3.98</td>
|
||||
<td>37.54</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>9.98</td>
|
||||
<td>5.70</td>
|
||||
<td>2.04</td>
|
||||
<td>17.90</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv5_server_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>13.24</td>
|
||||
<td>26.91</td>
|
||||
<td>2.63</td>
|
||||
<td>43.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.82</td>
|
||||
<td>45.56</td>
|
||||
<td>2.52</td>
|
||||
<td>60.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>14.56</td>
|
||||
<td>13.76</td>
|
||||
<td>7.44</td>
|
||||
<td>36.76</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>10.01</td>
|
||||
<td>13.76</td>
|
||||
<td>1.92</td>
|
||||
<td>25.86</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv6_medium_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>13.89</td>
|
||||
<td>16.02</td>
|
||||
<td>2.49</td>
|
||||
<td>33.14</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.42</td>
|
||||
<td>26.23</td>
|
||||
<td>2.30</td>
|
||||
<td>40.10</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>11.40</td>
|
||||
<td>8.57</td>
|
||||
<td>8.35</td>
|
||||
<td>29.57</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>10.80</td>
|
||||
<td>13.06</td>
|
||||
<td>2.19</td>
|
||||
<td>26.18</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv6_small_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>10.91</td>
|
||||
<td>10.97</td>
|
||||
<td>2.41</td>
|
||||
<td>24.45</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.56</td>
|
||||
<td>22.17</td>
|
||||
<td>2.66</td>
|
||||
<td>36.55</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>11.70</td>
|
||||
<td>7.34</td>
|
||||
<td>3.87</td>
|
||||
<td>23.89</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>11.32</td>
|
||||
<td>7.46</td>
|
||||
<td>2.54</td>
|
||||
<td>21.49</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-OCRv6_tiny_det</td>
|
||||
<td>paddle_static</td>
|
||||
<td>11.14</td>
|
||||
<td>10.71</td>
|
||||
<td>2.84</td>
|
||||
<td>24.85</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>11.52</td>
|
||||
<td>21.70</td>
|
||||
<td>2.94</td>
|
||||
<td>36.31</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>10.90</td>
|
||||
<td>6.99</td>
|
||||
<td>4.13</td>
|
||||
<td>23.00</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>11.19</td>
|
||||
<td>6.35</td>
|
||||
<td>2.79</td>
|
||||
<td>20.49</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 文本检测模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/text_detection.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 六、常见问题与解决方案
|
||||
|
||||
### 6.1 性能优化问题
|
||||
|
||||
#### Q: GPU推理速度慢怎么办？
|
||||
|
||||
**A**: 可以通过以下方式优化：
|
||||
（1）启用高性能推理：设置`enable_hpi=True`，自动选择最优加速策略
|
||||
（2）启用TensorRT加速：设置`use_tensorrt=True`，需要CUDA 11.8+和TensorRT 8.6+
|
||||
（3）使用半精度：设置`precision="fp16"`，可以显著提升速度
|
||||
（4）调整批处理大小：根据显存大小设置合适的`batch_size`
|
||||
（5）使用轻量模型：在精度要求不高时使用 `PP-OCRv6_small`/`PP-OCRv6_tiny` 等轻量级模型
|
||||
|
||||
#### Q: GPU内存不足（CUDA out of memory）怎么办？
|
||||
|
||||
**A**: 可以通过以下方式解决：
|
||||
（1）减小批处理大小：将`batch_size`设置为1
|
||||
（2）减小图像尺寸：设置`det_limit_side_len=640`
|
||||
（3）启用内存优化：设置`enable_memory_optim=True`
|
||||
（4）限制GPU内存使用：设置`gpu_mem=200`
|
||||
（5）使用轻量模型：切换到 `PP-OCRv6_small`/`PP-OCRv6_tiny` 等轻量级模型
|
||||
|
||||
### 6.2 检测精度问题
|
||||
|
||||
#### Q: 检测框不准确或漏检怎么办？
|
||||
|
||||
**A**: 可以通过以下方式优化：
|
||||
（1）调整检测参数：
|
||||
```python
|
||||
model = TextDetection(
|
||||
thresh=0.3, # 降低像素阈值
|
||||
box_thresh=0.5, # 降低检测框阈值
|
||||
unclip_ratio=2.0, # 增大扩张系数
|
||||
limit_side_len=1216 # 增大图像尺寸
|
||||
)
|
||||
```
|
||||
（2）使用更精确的后处理模式：设置`det_db_score_mode="slow"`
|
||||
（3）启用膨胀处理：设置`use_dilation=True`
|
||||
|
||||
### 6.3 模型选择建议
|
||||
|
||||
#### Q: 如何选择合适的模型？
|
||||
|
||||
**A**: 根据应用场景选择：
|
||||
- 服务器高精度场景：使用 `PP-OCRv6_medium_det`，精度最高
|
||||
- 移动端部署：使用 `PP-OCRv6_small_det`，兼顾精度与效率
|
||||
- 端侧/IoT：使用 `PP-OCRv6_tiny_det`，模型最小
|
||||
- 实时处理：使用 `PP-OCRv6_small_det` 或 `PP-OCRv6_tiny_det`，推理更快
|
||||
|
||||
### 6.4 参数调优建议
|
||||
|
||||
#### Q: 如何调优检测参数？
|
||||
|
||||
**A**: 通过参数`limit_type`和`limit_side_len`来对图片的尺寸进行限制，`limit_type`可选参数为[`max`, `min`]，`limit_side_len` 为正整数，一般设置为 32 的倍数，比如 960。
|
||||
|
||||
如果输入图形分辨率不大，建议使用`limit_type=min` 和 `limit_side_len=960` 节省计算资源的同时能获得最佳检测效果。如果输入图片的分辨率比较大，而且想使用更大的分辨率预测，可以设置 `limit_side_len` 为想要的值，比如 1216。
|
||||
|
||||
#### Q: 不同场景的参数配置建议？
|
||||
|
||||
**A**:
|
||||
- **高精度配置**：`limit_side_len=1216`, `thresh=0.3`, `box_thresh=0.5`, `unclip_ratio=1.5`
|
||||
- **高速度配置**：`limit_side_len=640`, `thresh=0.5`, `box_thresh=0.7`, `unclip_ratio=1.2`
|
||||
- **平衡配置**：`limit_side_len=960`, `thresh=0.4`, `box_thresh=0.6`, `unclip_ratio=1.5`
|
||||
|
||||
### 6.5 错误处理
|
||||
|
||||
#### Q: 模型加载失败怎么办？
|
||||
|
||||
**A**:
|
||||
（1）检查模型路径是否正确
|
||||
（2）确保模型文件完整（inference.pdmodel, inference.pdiparams, inference.json）
|
||||
（3）设置模型下载源：`os.environ['PADDLE_PDX_MODEL_SOURCE'] = 'BOS'`
|
||||
（4）使用本地模型：指定`model_dir`参数
|
||||
|
||||
#### Q: 输入数据格式错误怎么办？
|
||||
|
||||
**A**:
|
||||
（1）检查图像文件是否存在
|
||||
（2）确保图像格式正确（支持PNG、JPG、JPEG）
|
||||
（3）验证图像尺寸（最小10x10像素）
|
||||
（4）检查图像是否为3通道格式
|
||||
@@ -0,0 +1,478 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Text Image Rectification Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The primary purpose of text image rectification is to perform geometric transformations on images to correct distortions, inclinations, perspective deformations, etc., in the document images for more accurate subsequent text recognition.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>CER</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">Training Model</a></td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td>High-accuracy text image rectification model</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> <a href="https://www3.cs.stonybrook.edu/~cvl/docunet.html">DocUNet benchmark</a> dataset.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Choose the optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the image rectification module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(model_name="UVDoc")
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The example above uses the <code>paddle_static</code> inference engine by default. To run it, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(
|
||||
model_name="UVDoc",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(
|
||||
model_name="UVDoc",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'doc_test.jpg', 'page_index': None, 'doctr_img': '...'}}
|
||||
```
|
||||
|
||||
The meanings of the parameters in the result are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>：Indicates the path of the image to be rectified</li>
|
||||
<li><code>doctr_img</code>：Indicates the rectified image result. Due to the large amount of data, it is not convenient to print directly, so it is replaced here with<code>...</code>.You can use<code>res.save_to_img()</code>to save the prediction result as an image, and <code>res.save_to_json()</code> to save the prediction result as a json file.</li>
|
||||
</ul>
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/image_unwarp/doc_test_res.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* <code>TextImageUnwarping</code> instantiates the image rectification model (taking <code>UVDoc</code> as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Name of the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b> Model storage path</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b> Device(s) to use for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Examples:</b> <code>cpu</code>, <code>gpu</code>, <code>npu</code>, <code>gpu:0</code>, <code>gpu:0,1</code>.<br/>
|
||||
If multiple devices are specified, inference will be performed in parallel. Note that parallel inference is not always supported.<br/>
|
||||
By default, GPU 0 will be used if available; otherwise, the CPU will be used.
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b> Whether to use the high performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b> Whether to use the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b>Precision for TensorRT when using the Paddle Inference TensorRT subgraph engine.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>, etc.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b> Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b> Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the image rectification model for inference prediction. This method will return a result list. Additionally, this module also provides a <code>predict_iter()</code> method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that <code>predict_iter()</code> returns a <code>generator</code>, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The <code>predict()</code> method has parameters <code>input</code> and <code>batch_size</code>, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b> Input data to be predicted. Required.<br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
<ul>
|
||||
<li>Local image or PDF file path: <code>/root/data/img.jpg</code>;</li>
|
||||
<li><b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">example</a>;</li>
|
||||
<li><b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b> Batch size<br/>
|
||||
<b>Description:</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
The current module does not support fine-tuning training and only supports inference integration. Concerning fine-tuning training for this module, there are plans to support it in the future.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">UVDoc</td>
|
||||
<td>paddle_static</td>
|
||||
<td>14.96</td>
|
||||
<td>18.60</td>
|
||||
<td>1.93</td>
|
||||
<td>36.66</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>10.90</td>
|
||||
<td>27.59</td>
|
||||
<td>1.96</td>
|
||||
<td>40.94</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>13.54</td>
|
||||
<td>6.74</td>
|
||||
<td>0.91</td>
|
||||
<td>33.07</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>10.60</td>
|
||||
<td>8.44</td>
|
||||
<td>1.75</td>
|
||||
<td>21.30</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
## 6. FAQ
|
||||
@@ -0,0 +1,473 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本图像矫正模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本图像矫正的主要目的是针对图像进行几何变换，以纠正图像中的文档扭曲、倾斜、透视变形等问题，以供后续的文本识别进行更加准确。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>CER </th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td>高精度文本图像矫正模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong><a href="https://www3.cs.stonybrook.edu/~cvl/docunet.html">DocUNet benchmark</a>数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将图像矫正的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(model_name="UVDoc")
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(
|
||||
model_name="UVDoc",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(
|
||||
model_name="UVDoc",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'doc_test.jpg', 'page_index': None, 'doctr_img': '...'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：表示输入待矫正图像的路径</li>
|
||||
<li><code>doctr_img</code>：表示矫正后的图像结果，由于数据过多不便于直接print，所以此处用<code>...</code>替换，可以通过<code>res.save_to_img()</code>将预测结果保存为图片，通过<code>res.save_to_json()</code>将预测结果保存为json文件。</li>
|
||||
</ul>
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/image_unwarp/doc_test_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>TextImageUnwarping</code>实例化图像矫正模型（此处以<code>UVDoc</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>UVDoc</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。</br>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用图像矫正模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 和 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、推理引擎
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">UVDoc</td>
|
||||
<td>paddle_static</td>
|
||||
<td>14.96</td>
|
||||
<td>18.60</td>
|
||||
<td>1.93</td>
|
||||
<td>36.66</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>10.90</td>
|
||||
<td>27.59</td>
|
||||
<td>1.96</td>
|
||||
<td>40.94</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>13.54</td>
|
||||
<td>6.74</td>
|
||||
<td>0.91</td>
|
||||
<td>33.07</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>10.60</td>
|
||||
<td>8.44</td>
|
||||
<td>1.75</td>
|
||||
<td>21.30</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
## 六、FAQ
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,559 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Line Orientation Classification Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
The text line orientation classification module primarily distinguishes the orientation of text lines and corrects them using post-processing. In processes such as document scanning and license/certificate photography, to capture clearer images, the capture device may be rotated, resulting in text lines in various orientations. Standard OCR pipelines cannot handle such data well. By utilizing image classification technology, the orientation of text lines can be predetermined and adjusted, thereby enhancing the accuracy of OCR processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
> The inference time only includes the model inference time and does not include the time for pre- or post-processing. The "Normal Mode" values correspond to the local <code>paddle_static</code> inference engine.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Top-1 Accuracy (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x0_25_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x0_25_textline_ori_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.85</td>
|
||||
<td>2.16 / 0.41</td>
|
||||
<td>2.37 / 0.73</td>
|
||||
<td>0.96</td>
|
||||
<td>Text line classification model based on PP-LCNet_x0_25, with two classes: 0 degrees and 180 degrees</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_textline_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_textline_ori_pretrained.pdparams">Training Model</a></td>
|
||||
<td>99.42</td>
|
||||
<td>- / -</td>
|
||||
<td>2.98 / 2.98</td>
|
||||
<td>6.5</td>
|
||||
<td>Text line classification model based on PP-LCNet_x1_0, with two classes: 0 degrees and 180 degrees</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ **Note**: The text line orientation classification model was upgraded on May 26, 2025, and `PP-LCNet_x1_0_textline_ori` has been added. If you need to use the pre-upgrade model weights, please click the <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.bak.tar">download link</a>.
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> PaddleX Self-built Dataset, Covering Multiple Scenarios Such as Documents and Certificates, Containing 1000 Images.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Integration
|
||||
|
||||
> ❗ Before starting, please install the wheel package of PaddleOCR. For detailed instructions, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience the functionality with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg
|
||||
```
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the transformers engine for inference
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following command:
|
||||
|
||||
```bash
|
||||
# Use the onnxruntime engine for inference
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable <code>PADDLE_PDX_MODEL_SOURCE="BOS"</code> to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the text line orientation classification model into your project. Run the following code after downloading the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(model_name="PP-LCNet_x0_25_textline_ori")
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `transformers` as the inference engine, make sure the Transformers environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(
|
||||
model_name="PP-LCNet_x0_25_textline_ori",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
If you choose `onnxruntime` as the inference engine, make sure the ONNX Runtime environment is configured, and then run the following code:
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(
|
||||
model_name="PP-LCNet_x0_25_textline_ori",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
In most scenarios, the default `paddle_static` inference engine delivers better inference performance and is the recommended first choice.
|
||||
|
||||
If you want to use the trained model with the `paddle_dynamic` or `transformers` engine, refer to the [Weight Conversion](#52-weight-conversion) section in the [Inference Engine](#5-inference-engine) section below to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'textline_rot180_demo.jpg', 'page_index': None, 'class_ids': array([1], dtype=int32), 'scores': array([0.99864], dtype=float32), 'label_names': ['180_degree']}}
|
||||
```
|
||||
|
||||
The meanings of the running results parameters are as follows:
|
||||
<ul>
|
||||
<li><code>input_path</code>：Indicates the path of the input image.</li>
|
||||
<li><code>page_index</code>：If the input is a PDF file, it indicates the current page number of the PDF; otherwise, it is <code>None</code></li>
|
||||
<li><code>class_ids</code>：Indicates the class ID of the prediction result.</li>
|
||||
<li><code>scores</code>：Indicates the confidence score of the prediction result.</li>
|
||||
<li><code>label_names</code>：Indicates the class name of the prediction result.
|
||||
The visualization image is as follows:</li>
|
||||
</ul>
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/textline_ori_classification/textline_rot180_demo_res.jpg">
|
||||
|
||||
The explanations for the methods, parameters, etc., are as follows:
|
||||
|
||||
* <code>TextLineOrientationClassification</code> instantiates a textline classification model (here, <code>PP-LCNet_x0_25_textline_ori</code> is used as an example), and the specific explanations are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>Meaning:</b> Model name.<br/>
|
||||
<b>Description:</b>
|
||||
If set to <code>None</code>, <code>PP-LCNet_x0_25_textline_ori</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>Meaning:</b>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>Meaning:</b> Device for inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>Meaning:</b> Inference engine.<br/><b>Description:</b> Supports <code>None</code> (the default), <code>paddle</code>, <code>paddle_static</code>, <code>paddle_dynamic</code>, <code>transformers</code>, and <code>onnxruntime</code>. When left as <code>None</code>, local inference uses the <code>paddle_static</code> engine by default. For detailed descriptions, supported values, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>Meaning:</b> Inference-engine configuration.<br/><b>Description:</b> Recommended together with <code>engine</code>. For supported fields, compatibility rules, and examples, see <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration</a>.</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>Meaning:</b> Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>Meaning:</b> Whether to use the Paddle Inference TensorRT subgraph engine. <br/>
|
||||
<b>Description:</b>
|
||||
If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>Meaning:</b> Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/>
|
||||
<b>Description:</b>
|
||||
<b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b> Whether to enable MKL-DNN acceleration for inference. <br/>
|
||||
<b>Description:</b>
|
||||
If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>Meaning:</b>MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>Meaning:</b> Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Use the <code>predict()</code> method of the text line direction classification model to perform inference. This method returns a list of results. In addition, this module also provides the <code>predict_iter()</code> method. Both methods accept the same parameters and return the same result format. The difference is that <code>predict_iter()</code> returns a <code>generator</code>, which processes and retrieves prediction results step by step. It is suitable for handling large datasets or memory-efficient scenarios. You can choose either method based on your actual needs. The <code>predict()</code> method accepts the parameters <code>input</code> and <code>batch_size</code>, which are described in detail below:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b>Input data to be predicted. Required. <br/>
|
||||
<b>Description:</b>
|
||||
Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
<ul>
|
||||
<li>Local image or PDF file path: <code>/root/data/img.jpg</code>;</li>
|
||||
<li><b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">example</a>;</li>
|
||||
<li><b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size. <br/>
|
||||
<b>Description:</b>
|
||||
Positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Call the <code>predict()</code> method of the text line orientation classification model for inference. This method will return a list of results. In addition, this module also provides a <code>predict_iter()</code> method. Both methods accept the same parameters and return the same results, but <code>predict_iter()</code> returns a <code>generator</code>, which is more suitable for processing large datasets or when you want to save memory. You can choose either method according to your needs. The parameters of the <code>predict()</code> method are <code>input</code> and <code>batch_size</code>, as described below:
|
||||
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>Meaning:</b>Data to be predicted,<br/>
|
||||
<b>Description:</b>
|
||||
Supporting multiple input types</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>Python variable</b>, such as image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>File path</b>, such as the local path of an image file: <code>/root/data/img.jpg</code></li>
|
||||
<li><b>URL link</b>, such as the network URL of an image file: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg">Example</a></li>
|
||||
<li><b>Local directory</b>, the directory should contain data files to be predicted, such as the local path: <code>/root/data/</code></li>
|
||||
<li><b>list</b>, the elements of the list should be of the above-mentioned data types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"]</code>, <code>[\"/root/data1\", \"/root/data2\"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>Meaning:</b>Batch size</td>
|
||||
<td><code>int</code></td>
|
||||
<td>Any integer</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The prediction results are processed, and the prediction result for each sample is of type <code>dict</code>. It supports operations such as printing, saving as an image, and saving as a <code>json</code> file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the results to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. If set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the results as a JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. If it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. If set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. If it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, it supports obtaining the visualization image with results and the prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualization image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
Since PaddleOCR does not natively support training for text line orientation classification, refer to [PaddleX's Custom Development Guide](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#iv-custom-development) for training. Trained models can seamlessly integrate into PaddleOCR's API for inference.
|
||||
|
||||
If you want to use the `paddle_dynamic` or `transformers` engine with the trained model, please refer to the [Weight Conversion](#52-weight-conversion) section in [Inference Engine](#5-inference-engine) later in this document to convert the model from the `pdparams` format to the `safetensors` format using PaddleX.
|
||||
|
||||
## 5. Inference Engine
|
||||
|
||||
For detailed descriptions, values, compatibility rules, and examples of the inference engine, please refer to <a href="../inference_deployment/local_inference/inference_engine.en.md">Inference Engine and Configuration Description</a>.
|
||||
|
||||
### 5.1 Speed Data
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x0_25_textline_ori</td>
|
||||
<td>paddle_static</td>
|
||||
<td>0.30</td>
|
||||
<td>2.89</td>
|
||||
<td>0.06</td>
|
||||
<td>3.34</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>0.28</td>
|
||||
<td>6.52</td>
|
||||
<td>0.08</td>
|
||||
<td>6.98</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>1.30</td>
|
||||
<td>3.76</td>
|
||||
<td>0.15</td>
|
||||
<td>5.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>0.27</td>
|
||||
<td>0.76</td>
|
||||
<td>0.05</td>
|
||||
<td>1.16</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x1_0_textline_ori</td>
|
||||
<td>paddle_static</td>
|
||||
<td>0.33</td>
|
||||
<td>3.20</td>
|
||||
<td>0.06</td>
|
||||
<td>3.69</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>0.29</td>
|
||||
<td>7.60</td>
|
||||
<td>0.07</td>
|
||||
<td>8.06</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>1.28</td>
|
||||
<td>3.47</td>
|
||||
<td>0.14</td>
|
||||
<td>5.04</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>0.27</td>
|
||||
<td>0.77</td>
|
||||
<td>0.05</td>
|
||||
<td>1.16</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><strong>Test Data:</strong> <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg">Sample Image</a></li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA A100 40G</li>
|
||||
<li>CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 Weight Conversion
|
||||
|
||||
When using the inference engine, the system will automatically download the official pre-trained model. If you need to use a self-trained model with the `paddle_dynamic` or `transformers` engine, please refer to the [PaddleX Text Line Orientation Classification Module Weight Conversion](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#442) section to convert the model from the `pdparams` format to the `safetensors` format using PaddleX. This allows seamless integration into the PaddleOCR API for inference. If you need to use a self-trained model with the `onnxruntime` engine, refer to [PaddleX Obtain ONNX Models](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html) to obtain the ONNX model, so it can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
@@ -0,0 +1,523 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本行方向分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本行方向分类模块主要是将文本行的方向区分出来，并使用后处理将其矫正。在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的文本行也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断文本行方向，并将其进行方向调整，从而提高OCR处理的准确性。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
> 推理耗时仅包含模型推理耗时，不包含前后处理耗时。表格中的“常规模式”耗时对应本地 <code>paddle_static</code> 推理引擎。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>模型下载链接</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）</th>
|
||||
<th>CPU推理耗时 (ms)</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x0_25_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x0_25_textline_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.85</td>
|
||||
<td>2.16 / 0.41</td>
|
||||
<td>2.37 / 0.73</td>
|
||||
<td>0.96</td>
|
||||
<td>基于PP-LCNet_x0_25的文本行分类模型，含有两个类别，即0度，180度</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_textline_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_textline_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>99.42</td>
|
||||
<td>- / -</td>
|
||||
<td>2.98 / 2.98</td>
|
||||
<td>6.5</td>
|
||||
<td>基于PP-LCNet_x1_0的文本行分类模型，含有两个类别，即0度，180度</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ <b>注</b>：文本行方向分类模型于 2025.5.26 升级，并增加 `PP-LCNet_x1_0_textline_ori`，如需使用升级前的模型权重，请点击<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.bak.tar">下载链接</a>。
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR 自建的数据集，覆盖证件和文档等多个场景，包含 1000 张图片。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle-gpu 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 transformers 引擎进行推理
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg \
|
||||
--engine transformers
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下命令：
|
||||
|
||||
```bash
|
||||
# 使用 onnxruntime 引擎进行推理
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg \
|
||||
--engine onnxruntime
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本行方向分类模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(model_name="PP-LCNet_x0_25_textline_ori")
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
上述示例默认使用 <code>paddle_static</code> 推理引擎，请先按照[飞桨框架安装](../paddlepaddle_installation.md)完成 PaddlePaddle 安装。
|
||||
|
||||
如果选择 `transformers` 作为推理引擎，请确保已配置 Transformers 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(
|
||||
model_name="PP-LCNet_x0_25_textline_ori",
|
||||
engine="transformers",
|
||||
)
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
如果选择 `onnxruntime` 作为推理引擎，请确保已配置 ONNX Runtime 环境，然后执行如下代码：
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(
|
||||
model_name="PP-LCNet_x0_25_textline_ori",
|
||||
engine="onnxruntime",
|
||||
)
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
在大多数场景下，默认的 `paddle_static` 推理引擎通常具备更好的推理性能，建议优先使用。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'textline_rot180_demo.jpg', 'page_index': None, 'class_ids': array([1], dtype=int32), 'scores': array([0.99864], dtype=float32), 'label_names': ['180_degree']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
<ul>
|
||||
<li><code>input_path</code>：表示输入图片的路径。</li>
|
||||
<li><code>page_index</code>：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 <code>None</code></li>
|
||||
<li><code>class_ids</code>：表示预测结果的类别 id，含有两个类别，即0度和180度。</li>
|
||||
<li><code>scores</code>：表示预测结果的置信度。</li>
|
||||
<li><code>label_names</code>：表示预测结果的类别名。</li>
|
||||
</ul>
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/textline_ori_classification/textline_rot180_demo_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* <code>TextLineOrientationClassification</code>实例化文本行方向分类模型（此处以<code>PP-LCNet_x0_25_textline_ori</code>为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td><b>含义：</b>模型名称。<br/>
|
||||
<b>说明：</b>
|
||||
如果设置为<code>None</code>，则使用<code>PP-LCNet_x0_25_textline_ori</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td><b>含义：</b>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td><b>含义：</b>用于推理的设备。<br/>
|
||||
<b>说明：</b>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine</code></td>
|
||||
<td><b>含义：</b>推理引擎。<br><b>说明：</b>支持 <code>None</code>（默认值）、<code>paddle</code>、<code>paddle_static</code>、<code>paddle_dynamic</code>、<code>transformers</code>、<code>onnxruntime</code>。保持为默认值 <code>None</code> 时，本地推理默认使用 <code>paddle_static</code> 引擎。详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>engine_config</code></td>
|
||||
<td><b>含义：</b>推理引擎配置。<br><b>说明：</b>推荐与 <code>engine</code> 搭配使用。详细字段、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。</td>
|
||||
<td><code>dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td><b>含义：</b>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td><b>含义：</b>是否启用 Paddle Inference 的 TensorRT 子图引擎。<br/>
|
||||
<b>说明：</b>
|
||||
如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td><b>含义：</b>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/>
|
||||
<b>说明：</b>
|
||||
<b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
<b>含义：</b>是否启用 MKL-DNN 加速推理。<br/>
|
||||
<b>说明：</b>
|
||||
如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
<b>含义：</b>MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td><b>含义：</b>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本行方向分类模型的 <code>predict()</code> 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 <code>predict_iter()</code> 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 <code>predict_iter()</code> 返回的是一个 <code>generator</code>，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。<code>predict()</code> 方法参数有 <code>input</code> 和 <code>batch_size</code>，具体说明如下：
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td><b>含义：</b>待预测数据，支持多种输入类型，必填。<br/>
|
||||
<b>说明：</b>
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>List</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td><b>含义：</b>批大小<br/>
|
||||
<b>说明：</b>
|
||||
可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为<code>json</code>文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供文本行方向分类的训练，因此，如果需要训练文档图像方向分类模型，可以参考 [PaddleX 文本行方向分类二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
训练后的模型如果想使用 `paddle_dynamic` 或 `transformers` 引擎，请参考后文 [推理引擎](#五推理引擎) 中的 [权重转换](#52-权重转换) 部分将模型由 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式。
|
||||
|
||||
## 五、推理引擎 {#五推理引擎}
|
||||
|
||||
关于推理引擎的详细说明、取值、兼容性规则与示例请参见 <a href="../inference_deployment/local_inference/inference_engine.md">推理引擎与配置说明</a>。
|
||||
|
||||
### 5.1 速度数据
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>model</th>
|
||||
<th>engine</th>
|
||||
<th>Preprocessing (ms)</th>
|
||||
<th>Inference (ms)</th>
|
||||
<th>PostProcessing (ms)</th>
|
||||
<th>End-to-End (ms)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x0_25_textline_ori</td>
|
||||
<td>paddle_static</td>
|
||||
<td>0.30</td>
|
||||
<td>2.89</td>
|
||||
<td>0.06</td>
|
||||
<td>3.34</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>0.28</td>
|
||||
<td>6.52</td>
|
||||
<td>0.08</td>
|
||||
<td>6.98</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>1.30</td>
|
||||
<td>3.76</td>
|
||||
<td>0.15</td>
|
||||
<td>5.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>0.27</td>
|
||||
<td>0.76</td>
|
||||
<td>0.05</td>
|
||||
<td>1.16</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">PP-LCNet_x1_0_textline_ori</td>
|
||||
<td>paddle_static</td>
|
||||
<td>0.33</td>
|
||||
<td>3.20</td>
|
||||
<td>0.06</td>
|
||||
<td>3.69</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>paddle_dynamic</td>
|
||||
<td>0.29</td>
|
||||
<td>7.60</td>
|
||||
<td>0.07</td>
|
||||
<td>8.06</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>transformers</td>
|
||||
<td>1.28</td>
|
||||
<td>3.47</td>
|
||||
<td>0.14</td>
|
||||
<td>5.04</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnxruntime</td>
|
||||
<td>0.27</td>
|
||||
<td>0.77</td>
|
||||
<td>0.05</td>
|
||||
<td>1.16</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
<ul>
|
||||
<li><strong>测试数据：</strong><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg">示例图片</a></li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA A100 40G</li>
|
||||
<li>CPU：Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 22.04 / CUDA 12.6 / cuDNN 9.5</li>
|
||||
<li>paddlepaddle-gpu 3.2.1 / paddleocr 3.5 / transformers 5.4.0 / torch 2.10 / onnxruntime-gpu 1.23.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
### 5.2 权重转换 {#52-权重转换}
|
||||
|
||||
使用推理引擎时，系统会自动下载官方预训练模型。若需使用自训练模型配合 `paddle_dynamic` 或 `transformers` 引擎，请参考 [PaddleX 文本行方向分类模块权重转换](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#442) 部分，将 `pdparams` 格式通过 PaddleX 转换为 `safetensors` 格式，即可无缝集成到 PaddleOCR 的 API 中进行推理。若需使用自训练模型配合`onnxruntime`引擎，请参考[PaddleX 获取 ONNX 模型](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/paddle2onnx.html)获取onnx模型，即可无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
Reference in New Issue
Block a user