chore: import upstream snapshot with attribution

2026-07-13 11:59:26 +08:00
commit e904b667c6
2490 changed files with 596352 additions and 0 deletions
@@ -0,0 +1,99 @@
+---
+comments: true
+---
+
+## 1. Introduction
+
+Paper:
+> [When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition](https://arxiv.org/abs/2207.11463)
+> Bohan Li, Ye Yuan, Dingkang Liang, Xiao Liu, Zhilong Ji, Jinfeng Bai, Wenyu Liu, Xiang Bai
+> ECCV, 2022
+
+Using CROHME handwrittem mathematical expression recognition datasets for training, and evaluating on its test sets, the algorithm reproduction effect is as follows:
+
+|Model|Backbone|config|exprate|Download link|
+| --- | --- | --- | --- | --- |
+|CAN|DenseNet|[rec_d28_can.yml](https://github.com/PaddlePaddle/PaddleOCR/tree/{{PADDLEOCR_GITHUB_REF}}/configs/rec/rec_d28_can.yml)|51.72%|[trained model](https://paddleocr.bj.bcebos.com/contribution/rec_d28_can_train.tar)|
+
+## 2. Environment
+
+Please refer to ["Environment Preparation"](../../ppocr/environment.en.md) to configure the PaddleOCR environment, and refer to ["Project Clone"](../../ppocr/blog/clone.en.md)to clone the project code.
+
+## 3. Model Training / Evaluation / Prediction
+
+Please refer to [Text Recognition Tutorial](../../ppocr/model_train/recognition.en.md). PaddleOCR modularizes the code, and training different recognition models only requires **changing the configuration file**.
+
+### Training
+
+Specifically, after the data preparation is completed, the training can be started. The training command is as follows:
+
+```bash linenums="1"
+# Single GPU training (long training period, not recommended)
+python3 tools/train.py -c configs/rec/rec_d28_can.yml
+
+# Multi GPU training, specify the gpu number through the --gpus parameter
+python3 -m paddle.distributed.launch --gpus '0,1,2,3'  tools/train.py -c configs/rec/rec_d28_can.yml
+```
+
+### Evaluation
+
+```bash linenums="1"
+# GPU evaluation
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_d28_can.yml -o Global.pretrained_model=./rec_d28_can_train/best_accuracy.pdparams
+```
+
+Prediction:
+
+```bash linenums="1"
+# The configuration file used for prediction must match the training
+python3 tools/infer_rec.py -c configs/rec/rec_d28_can.yml -o Architecture.Head.attdecoder.is_train=False Global.infer_img='./doc/crohme_demo/hme_00.jpg' Global.pretrained_model=./rec_d28_can_train/best_accuracy.pdparams
+```
+
+## 4. Inference and Deployment
+
+### 4.1 Python Inference
+
+First, the model saved during the CAN handwritten mathematical expression recognition training process is converted into an inference model. you can use the following command to convert:
+
+```bash linenums="1"
+python3 tools/export_model.py -c configs/rec/rec_d28_can.yml -o Global.pretrained_model=./rec_d28_can_train/best_accuracy.pdparams Global.save_inference_dir=./inference/rec_d28_can/ Architecture.Head.attdecoder.is_train=False
+
+# The default output max length of the model is 36. If you need to predict a longer sequence, please specify its output sequence as an appropriate value when exporting the model, as: Architecture.Head.max_ text_ length=72
+```
+
+For CAN handwritten mathematical expression recognition model inference, the following commands can be executed:
+
+```bash linenums="1"
+python3 tools/infer/predict_rec.py --image_dir="./doc/datasets/crohme_demo/hme_00.jpg" --rec_algorithm="CAN" --rec_batch_num=1 --rec_model_dir="./inference/rec_d28_can/" --rec_char_dict_path="./ppocr/utils/dict/latex_symbol_dict.txt"
+
+# If you need to predict on a picture with black characters on a white background, please set: -- rec_ image_ inverse=False
+```
+
+### 4.2 C++ Inference
+
+Not supported
+
+### 4.3 Serving
+
+Not supported
+
+### 4.4 More
+
+Not supported
+
+## 5. FAQ
+
+## Citation
+
+```bibtex
+@misc{https://doi.org/10.48550/arxiv.2207.11463,
+  doi = {10.48550/ARXIV.2207.11463},
+  url = {https://arxiv.org/abs/2207.11463},
+  author = {Li, Bohan and Yuan, Ye and Liang, Dingkang and Liu, Xiao and Ji, Zhilong and Bai, Jinfeng and Liu, Wenyu and Bai, Xiang},
+  keywords = {Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  title = {When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition},
+  publisher = {arXiv},
+  year = {2022},
+  copyright = {arXiv.org perpetual, non-exclusive license}
+}
+```
@@ -0,0 +1,155 @@
+---
+typora-copy-images-to: images
+comments: true
+---
+
+# 手写数学公式识别算法-CAN
+
+## 1. 算法简介
+
+论文信息：
+> [When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition](https://arxiv.org/abs/2207.11463)
+> Bohan Li, Ye Yuan, Dingkang Liang, Xiao Liu, Zhilong Ji, Jinfeng Bai, Wenyu Liu, Xiang Bai
+> ECCV, 2022
+
+`CAN`使用CROHME手写公式数据集进行训练，在对应测试集上的精度如下：
+
+|模型    |骨干网络|配置文件|ExpRate|下载链接|
+| ----- | ----- | ----- | ----- | ----- |
+|CAN|DenseNet|[rec_d28_can.yml](https://github.com/PaddlePaddle/PaddleOCR/tree/{{PADDLEOCR_GITHUB_REF}}/configs/rec/rec_d28_can.yml)|51.72%|[训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_d28_can_train.tar)|
+
+## 2. 环境配置
+
+请先参考[《运行环境准备》](../../ppocr/environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](../../ppocr/blog/clone.md)克隆项目代码。
+
+## 3. 模型训练、评估、预测
+
+### 3.1 模型训练
+
+请参考[文本识别训练教程](../../ppocr/model_train/recognition.md)。PaddleOCR对代码进行了模块化，训练`CAN`识别模型时需要**更换配置文件**为`CAN`的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/tree/{{PADDLEOCR_GITHUB_REF}}/configs/rec/rec_d28_can.yml)。
+
+#### 启动训练
+
+具体地，在完成数据准备后，便可以启动训练，训练命令如下：
+
+```bash linenums="1"
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_d28_can.yml
+
+# 多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3'  tools/train.py -c configs/rec/rec_d28_can.yml
+```
+
+**注意：**
+
+- 我们提供的数据集，即[`CROHME数据集`](https://paddleocr.bj.bcebos.com/dataset/CROHME.tar)将手写公式存储为黑底白字的格式，若您自行准备的数据集与之相反，即以白底黑字模式存储，请在训练时做出如下修改
+
+  ```bash linenums="1"
+  python3 tools/train.py -c configs/rec/rec_d28_can.yml -o Train.dataset.transforms.GrayImageChannelFormat.inverse=False
+  ```
+
+- 默认每训练1个epoch（1105次iteration）进行1次评估，若您更改训练的batch_size，或更换数据集，请在训练时作出如下修改
+
+  ```bash linenums="1"
+  python3 tools/train.py -c configs/rec/rec_d28_can.yml -o Global.eval_batch_step=[0, {length_of_dataset//batch_size}]
+  ```
+
+### 3.2 评估
+
+可下载已训练完成的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_d28_can_train.tar)，使用如下命令进行评估：
+
+```bash linenums="1"
+# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_d28_can.yml -o Global.pretrained_model=./rec_d28_can_train/best_accuracy.pdparams
+```
+
+### 3.3 预测
+
+使用如下命令进行单张图片预测：
+
+```bash linenums="1"
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/infer_rec.py -c configs/rec/rec_d28_can.yml -o Architecture.Head.attdecoder.is_train=False Global.infer_img='./doc/datasets/crohme_demo/hme_00.jpg' Global.pretrained_model=./rec_d28_can_train/best_accuracy.pdparams
+
+# 预测文件夹下所有图像时，可修改infer_img为文件夹，如 Global.infer_img='./doc/datasets/crohme_demo/'。
+```
+
+## 4. 推理部署
+
+### 4.1 Python推理
+
+首先将训练得到best模型，转换成inference model。这里以训练完成的模型为例（[模型下载地址](https://paddleocr.bj.bcebos.com/contribution/rec_d28_can_train.tar) )，可以使用如下命令进行转换：
+
+```bash linenums="1"
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/export_model.py -c configs/rec/rec_d28_can.yml -o Global.pretrained_model=./rec_d28_can_train/best_accuracy.pdparams Global.save_inference_dir=./inference/rec_d28_can/ Architecture.Head.attdecoder.is_train=False
+
+# 目前的静态图模型默认的输出长度最大为36，如果您需要预测更长的序列，请在导出模型时指定其输出序列为合适的值，例如 Architecture.Head.max_text_length=72
+```
+
+**注意：**
+如果您是在自己的数据集上训练的模型，并且调整了字典文件，请注意修改配置文件中的`character_dict_path`是否是所需要的字典文件。
+
+转换成功后，在目录下有三个文件：
+
+```text linenums="1"
+/inference/rec_d28_can/
+    ├── inference.pdiparams         # 识别inference模型的参数文件
+    ├── inference.pdiparams.info    # 识别inference模型的参数信息，可忽略
+    └── inference.pdmodel           # 识别inference模型的program文件
+```
+
+执行如下命令进行模型推理：
+
+```bash linenums="1"
+python3 tools/infer/predict_rec.py --image_dir="./doc/datasets/crohme_demo/hme_00.jpg" --rec_algorithm="CAN" --rec_batch_num=1 --rec_model_dir="./inference/rec_d28_can/" --rec_char_dict_path="./ppocr/utils/dict/latex_symbol_dict.txt"
+
+# 预测文件夹下所有图像时，可修改image_dir为文件夹，如 --image_dir='./doc/datasets/crohme_demo/'。
+
+# 如果您需要在白底黑字的图片上进行预测，请设置 --rec_image_inverse=False
+```
+
+![测试图片样例](./images/hme_00.jpg)
+
+执行命令后，上面图像的预测结果（识别的文本）会打印到屏幕上，示例如下：
+
+```bash linenums="1"
+Predicts of ./doc/imgs_hme/hme_00.jpg:['x _ { k } x x _ { k } + y _ { k } y x _ { k }', []]
+```
+
+**注意**：
+
+- 需要注意预测图像为**黑底白字**，即手写公式部分为白色，背景为黑色的图片。
+- 在推理时需要设置参数`rec_char_dict_path`指定字典，如果您修改了字典，请修改该参数为您的字典文件。
+- 如果您修改了预处理方法，需修改`tools/infer/predict_rec.py`中CAN的预处理为您的预处理方法。
+
+### 4.2 C++推理部署
+
+由于C++预处理后处理还未支持CAN，所以暂未支持
+
+### 4.3 Serving服务化部署
+
+暂不支持
+
+### 4.4 更多推理部署
+
+暂不支持
+
+## 5. FAQ
+
+1. CROHME数据集来自于[CAN源repo](https://github.com/LBH1024/CAN) 。
+
+## 引用
+
+```bibtex
+@misc{https://doi.org/10.48550/arxiv.2207.11463,
+  doi = {10.48550/ARXIV.2207.11463},
+  url = {https://arxiv.org/abs/2207.11463},
+  author = {Li, Bohan and Yuan, Ye and Liang, Dingkang and Liu, Xiao and Ji, Zhilong and Bai, Jinfeng and Liu, Wenyu and Bai, Xiang},
+  keywords = {Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  title = {When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition},
+  publisher = {arXiv},
+  year = {2022},
+  copyright = {arXiv.org perpetual, non-exclusive license}
+}
+```
@@ -0,0 +1,105 @@
+# LaTeX-OCR
+
+## 1. Introduction
+
+Original Project:
+> [https://github.com/lukas-blecher/LaTeX-OCR](https://github.com/lukas-blecher/LaTeX-OCR)
+
+
+Using LaTeX-OCR printed mathematical expression recognition datasets for training, and evaluating on its test sets, the algorithm reproduction effect is as follows:
+
+| Model       | Backbone |config| BLEU score  | normed edit distance  |  ExpRate  |Download link|
+|-----------|----------| ---- |:-----------:|:---------------------:|:---------:| ----- |
+| LaTeX-OCR | Hybrid ViT |[LaTeX_OCR_rec.yaml](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/LaTeX_OCR_rec.yaml)|   0.8821    |        0.0823         |  40.01%   |[trained model](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_train.tar)|
+
+## 2. Environment
+Please refer to ["Environment Preparation"](../../ppocr/environment.en.md) to configure the PaddleOCR environment, and refer to ["Project Clone"](../../ppocr/blog/clone.en.md) to clone the project code.
+
+Furthermore, additional dependencies need to be installed:
+```shell
+pip install -r docs/algorithm/formula_recognition/requirements.txt
+```
+
+## 3. Model Training / Evaluation / Prediction
+
+Please refer to [Text Recognition Tutorial](../../ppocr/model_train/recognition.en.md). PaddleOCR modularizes the code, and training different recognition models only requires **changing the configuration file**.
+
+Pickle File Generation:
+
+Download formulae.zip and math.txt in [Google Drive](https://drive.google.com/drive/folders/13CA4vAmOmD_I_dSbvLp-Lf0s6KiaNfuO), and then use the following command to generate the pickle file.
+
+```shell
+# Create a LaTeX-OCR dataset directory
+mkdir -p train_data/LaTeXOCR
+# Unzip formulae.zip and copy math.txt
+unzip -d train_data/LaTeXOCR path/formulae.zip
+cp path/math.txt train_data/LaTeXOCR
+# Convert the original .txt file to a .pkl file to group images of different scales
+# Training set conversion
+python ppocr/utils/formula_utils/math_txt2pkl.py --image_dir=train_data/LaTeXOCR/train --mathtxt_path=train_data/LaTeXOCR/math.txt --output_dir=train_data/LaTeXOCR/
+# Validation set conversion
+python ppocr/utils/formula_utils/math_txt2pkl.py --image_dir=train_data/LaTeXOCR/val --mathtxt_path=train_data/LaTeXOCR/math.txt --output_dir=train_data/LaTeXOCR/
+# Test set conversion
+python ppocr/utils/formula_utils/math_txt2pkl.py --image_dir=train_data/LaTeXOCR/test --mathtxt_path=train_data/LaTeXOCR/math.txt --output_dir=train_data/LaTeXOCR/
+```
+
+
+Training:
+
+Specifically, after the data preparation is completed, the training can be started. The training command is as follows:
+
+```
+#Single GPU training (Default training method)
+python3 tools/train.py -c configs/rec/LaTeX_OCR_rec.yaml
+
+#Multi GPU training, specify the gpu number through the --gpus parameter
+python3 -m paddle.distributed.launch --gpus '0,1,2,3'  tools/train.py -c configs/rec/LaTeX_OCR_rec.yaml
+```
+
+Evaluation:
+
+```
+# GPU evaluation
+# Validation set evaluation
+python3 tools/eval.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams
+# Test set evaluation
+python3 tools/eval.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams Eval.dataset.data_dir=./train_data/LaTeXOCR/test Eval.dataset.data=./train_data/LaTeXOCR/latexocr_test.pkl
+```
+
+Prediction:
+
+```
+# The configuration file used for prediction must match the training
+python3 tools/infer_rec.py -c configs/rec/LaTeX_OCR_rec.yaml  -o  Global.infer_img='./docs/datasets/images/pme_demo/0000013.png' Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams
+```
+
+## 4. Inference and Deployment
+
+### 4.1 Python Inference
+First, the model saved during the LaTeX-OCR printed mathematical expression recognition training process is converted into an inference model. you can use the following command to convert:
+
+```
+python3 tools/export_model.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams Global.save_inference_dir=./inference/rec_latex_ocr_infer/ 
+
+# The default output max length of the model is 512.
+```
+
+For LaTeX-OCR printed mathematical expression recognition model inference, the following commands can be executed:
+
+```
+python3 tools/infer/predict_rec.py --image_dir='./docs/datasets/images/pme_demo/0000295.png' --rec_algorithm="LaTeXOCR" --rec_batch_num=1 --rec_model_dir="./inference/rec_latex_ocr_infer/"  --rec_char_dict_path="./ppocr/utils/dict/latex_ocr_tokenizer.json"
+```
+
+### 4.2 C++ Inference
+
+Not supported
+
+### 4.3 Serving
+
+Not supported
+
+### 4.4 More
+
+Not supported
+
+## 5. FAQ
@@ -0,0 +1,147 @@
+# 印刷数学公式识别算法-LaTeX-OCR
+
+## 1. 算法简介
+
+原始项目：
+> [https://github.com/lukas-blecher/LaTeX-OCR](https://github.com/lukas-blecher/LaTeX-OCR)
+
+
+`LaTeX-OCR`使用[`LaTeX-OCR印刷公式数据集`](https://drive.google.com/drive/folders/13CA4vAmOmD_I_dSbvLp-Lf0s6KiaNfuO)进行训练，在对应测试集上的精度如下：
+
+| 模型        | 骨干网络       |配置文件 | BLEU score  | normed edit distance  |  ExpRate  |下载链接|
+|-----------|------------| ----- |:-----------:|:---------------------:|:---------:| ----- |
+| LaTeX-OCR | Hybrid ViT |[LaTeX_OCR_rec.yaml](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/LaTeX_OCR_rec.yaml)|   0.8821    |        0.0823         |  40.01%   |[训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_train.tar)|
+
+## 2. 环境配置
+请先参考[《运行环境准备》](../../ppocr/environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](../../ppocr/blog/clone.md)克隆项目代码。
+
+此外，需要安装额外的依赖：
+```shell
+pip install -r docs/algorithm/formula_recognition/requirements.txt
+```
+
+## 3. 模型训练、评估、预测
+
+### 3.1 pickle 标签文件生成
+从[谷歌云盘](https://drive.google.com/drive/folders/13CA4vAmOmD_I_dSbvLp-Lf0s6KiaNfuO)中下载 formulae.zip 和 math.txt，之后，使用如下命令，生成 pickle 标签文件。
+
+```shell
+# 创建 LaTeX-OCR 数据集目录
+mkdir -p train_data/LaTeXOCR
+# 解压formulae.zip ，并拷贝math.txt
+unzip -d train_data/LaTeXOCR path/formulae.zip
+cp path/math.txt train_data/LaTeXOCR
+# 将原始的 .txt 文件转换为 .pkl 文件，从而对不同尺度的图像进行分组
+# 训练集转换
+python ppocr/utils/formula_utils/math_txt2pkl.py --image_dir=train_data/LaTeXOCR/train --mathtxt_path=train_data/LaTeXOCR/math.txt --output_dir=train_data/LaTeXOCR/
+# 验证集转换
+python ppocr/utils/formula_utils/math_txt2pkl.py --image_dir=train_data/LaTeXOCR/val --mathtxt_path=train_data/LaTeXOCR/math.txt --output_dir=train_data/LaTeXOCR/
+# 测试集转换
+python ppocr/utils/formula_utils/math_txt2pkl.py --image_dir=train_data/LaTeXOCR/test --mathtxt_path=train_data/LaTeXOCR/math.txt --output_dir=train_data/LaTeXOCR/
+```
+
+### 3.2 模型训练
+
+请参考[文本识别训练教程](../../ppocr/model_train/recognition.md)。PaddleOCR对代码进行了模块化，训练`LaTeX-OCR`识别模型时需要**更换配置文件**为`LaTeX-OCR`的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/LaTeX_OCR_rec.yaml)。
+
+#### 启动训练
+
+
+具体地，在完成数据准备后，便可以启动训练，训练命令如下：
+```shell
+#单卡训练 (默认训练方式)
+python3 tools/train.py -c configs/rec/LaTeX_OCR_rec.yaml
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3'  tools/train.py -c configs/rec/LaTeX_OCR_rec.yaml
+```
+
+**注意：**
+
+- 默认每训练22个epoch（60000次iteration）进行1次评估，若您更改训练的batch_size，或更换数据集，请在训练时作出如下修改
+```
+python3 tools/train.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.eval_batch_step=[0,{length_of_dataset//batch_size*22}]
+```
+
+### 3.3 评估
+
+可下载已训练完成的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_train.tar)，使用如下命令进行评估：
+
+```shell
+# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
+# 验证集评估
+python3 tools/eval.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams
+# 测试集评估
+python3 tools/eval.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams Eval.dataset.data_dir=./train_data/LaTeXOCR/test Eval.dataset.data=./train_data/LaTeXOCR/latexocr_test.pkl
+```
+
+### 3.4 预测
+
+使用如下命令进行单张图片预测：
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/infer_rec.py -c configs/rec/LaTeX_OCR_rec.yaml  -o  Global.infer_img='./docs/datasets/images/pme_demo/0000013.png' Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams
+# 预测文件夹下所有图像时，可修改infer_img为文件夹，如 Global.infer_img='./doc/datasets/pme_demo/'。
+```
+
+## 4. 推理部署
+
+### 4.1 Python推理
+首先将训练得到best模型，转换成inference model。这里以训练完成的模型为例（[模型下载地址](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_train.tar)），可以使用如下命令进行转换：
+
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/export_model.py -c configs/rec/LaTeX_OCR_rec.yaml -o Global.pretrained_model=./rec_latex_ocr_train/best_accuracy.pdparams Global.save_inference_dir=./inference/rec_latex_ocr_infer/ 
+
+# 目前的静态图模型支持的最大输出长度为512
+```
+**注意：**
+- 如果您是在自己的数据集上训练的模型，并且调整了字典文件，请检查配置文件中的`rec_char_dict_path`是否为所需要的字典文件。
+- [转换后模型下载地址](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_infer.tar)
+
+转换成功后，在目录下有三个文件：
+```
+/inference/rec_latex_ocr_infer/
+    ├── inference.pdiparams         # 识别inference模型的参数文件
+    ├── inference.pdiparams.info    # 识别inference模型的参数信息，可忽略
+    └── inference.pdmodel           # 识别inference模型的program文件
+```
+
+执行如下命令进行模型推理：
+
+```shell
+python3 tools/infer/predict_rec.py --image_dir='./docs/datasets/images/pme_demo/0000295.png' --rec_algorithm="LaTeXOCR" --rec_batch_num=1 --rec_model_dir="./inference/rec_latex_ocr_infer/"  --rec_char_dict_path="./ppocr/utils/dict/latex_ocr_tokenizer.json"
+
+# 预测文件夹下所有图像时，可修改image_dir为文件夹，如 --image_dir='./doc/datasets/pme_demo/'。
+```
+&nbsp;
+
+![测试图片样例](../../../datasets/images/pme_demo/0000295.png)
+
+执行命令后，上面图像的预测结果（识别的公式）会打印到屏幕上，示例如下：
+```shell
+Predicts of ./doc/datasets/pme_demo/0000295.png:\zeta_{0}(\nu)=-{\frac{\nu\varrho^{-2\nu}}{\pi}}\int_{\mu}^{\infty}d\omega\int_{C_{+}}d z{\frac{2z^{2}}{(z^{2}+\omega^{2})^{\nu+1}}}{\tilde{\Psi}}(\omega;z)e^{i\epsilon z}~~~,
+```
+
+
+**注意**：
+
+- 需要注意预测图像为**白底黑字**，即手写公式部分为黑色，背景为白色的图片。
+- 在推理时需要设置参数`rec_char_dict_path`指定字典，如果您修改了字典，请修改该参数为您的字典文件。
+- 如果您修改了预处理方法，需修改`tools/infer/predict_rec.py`中 LaTeX-OCR 的预处理为您的预处理方法。
+
+
+### 4.2 C++推理部署
+
+由于C++预处理后处理还未支持 LaTeX-OCR，所以暂未支持
+
+### 4.3 Serving服务化部署
+
+暂不支持
+
+### 4.4 更多推理部署
+
+暂不支持
+
+## 5. FAQ
+
+1. LaTeX-OCR 数据集来自于[LaTeXOCR源repo](https://github.com/lukas-blecher/LaTeX-OCR) 。
@@ -0,0 +1,95 @@
+# 印刷数学公式识别算法-PP-FormulaNet
+
+## 1. 算法简介
+
+`PP-FormulaNet` 是由百度飞桨视觉团队开发的一款先进的公式识别模型，支持识别 5 万个常见 LaTeX 源码词汇。其中，PP-FormulaNet-S 版本采用 PP-HGNetV2-B4 作为骨干网络，通过并行掩码和模型蒸馏等技术大幅提升推理速度并保持高识别精度，适用于简单印刷公式和跨行简单印刷公式等场景；而 PP-FormulaNet-L 版本基于 Vary_VIT_B 并经过大规模公式数据集的深入训练，在复杂公式识别方面表现显著提升，适用于简单印刷、复杂印刷和手写公式。
+
+上述模型在对应测试集上的精度如下：
+
+| 模型          | 骨干网络       | 配置文件                   | En-BLEU↑  |GPU推理耗时（ms）| 下载链接 |
+|-----------|--------|----------------------------------------|:-----------------:|:--------------:|:--------------:|
+| UniMERNet | Donut Swin | [UniMERNet.yaml](../../../../configs/rec/UniMERNet.yaml) |     85.91   | 2266.96 | [训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_unimernet_train.tar)|
+| PP-FormulaNet-S | PPHGNetV2_B4 | [PP-FormulaNet-S.yaml](../../../../configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml) |   87.00 | 202.25 |[训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar)|
+| PP-FormulaNet-L | Vary_VIT_B | [PP-FormulaNet-L.yaml](../../../../configs/rec/PP-FormuaNet/PP-FormulaNet-L.yaml) |    90.36    | 1976.52  |[训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_l_train.tar )|
+| LaTeX-OCR | Hybrid ViT |[LaTeX_OCR_rec.yaml](../../../../configs/rec/LaTeX_OCR_rec.yaml)|   74.55   | 	1244.61   |[训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_train.tar)|
+
+这里，英文公式评估集包含UniMERNet的简单和复杂公式，以及PaddleX内部自建的简单、中等和复杂公式。
+
+## 2. 环境配置
+请先参考[《运行环境准备》](../../ppocr/environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](../../ppocr/blog/clone.md)克隆项目代码。
+
+此外，需要安装额外的依赖：
+```shell
+sudo apt-get update
+sudo apt-get install libmagickwand-dev
+pip install -r docs/algorithm/formula_recognition/requirements.txt
+```
+
+## 3. 模型训练、评估、预测
+
+### 3.1 准备数据集
+
+```shell
+# 下载 PaddleX 官方示例数据集
+wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
+tar -xf ocr_rec_latexocr_dataset_example.tar
+```
+
+
+### 3.2 下载预训练模型
+
+```shell
+# 下载 PP-FormulaNet-S 预训练模型
+wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar 
+tar -xf rec_ppformulanet_s_train.tar
+```
+
+
+### 3.3 模型训练
+
+请参考[文本识别训练教程](../../ppocr/model_train/recognition.md)。PaddleOCR对代码进行了模块化，训练 `PP-FormulaNet-S` 识别模型时需要**更换配置文件**为 `PP-FormulaNet-S` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml)。
+
+#### 启动训练
+
+具体地，在完成数据准备后，便可以启动训练，训练命令如下：
+```shell
+#单卡训练 (默认训练方式)
+python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+   -o Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' --ips=127.0.0.1   tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+        -o Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+```
+
+**注意：**
+
+- 默认每训练 1个epoch（179 次iteration）进行1次评估，若您更改训练的batch_size，或更换数据集，请在训练时作出如下修改
+```
+python3  -m paddle.distributed.launch --gpus '0,1,2,3' --ips=127.0.0.1   tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+  -o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
+   Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+```
+
+### 3.4 评估
+
+可下载已训练完成的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar )，使用如下命令进行评估：
+
+```shell
+# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
+ # demo 测试集评估
+ python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml -o \
+ Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+```
+
+### 3.5 预测
+
+使用如下命令进行单张图片预测：
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/infer_rec.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+  -o  Global.infer_img='./docs/datasets/images/pme_demo/0000295.png'\
+   Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+# 预测文件夹下所有图像时，可修改infer_img为文件夹，如 Global.infer_img='./doc/datasets/pme_demo/'。
+```
+
+## 4. FAQ
@@ -0,0 +1,82 @@
+# PP-FormulaNet
+
+## 1. Introduction
+
+
+`PP-FormulaNet` is an advanced formula recognition model developed by Baidu’s PaddlePaddle Vision Team, supporting the recognition of 50,000 common LaTeX source terms. The PP-FormulaNet-S version uses PP-HGNetV2-B4 as its backbone network, leveraging techniques like parallel masking and model distillation to significantly enhance inference speed while maintaining high recognition accuracy, making it suitable for scenarios involving simple printed formulas and cross-line simple printed formulas. On the other hand, the PP-FormulaNet-L version is based on Vary_VIT_B and has undergone extensive training on a large-scale formula dataset, showing significant improvement in recognizing complex formulas, and is applicable to simple printed, complex printed, and handwritten formulas.
+
+The accuracy of the above models on the corresponding test sets is as follows:
+
+| Model           | Backbone       | config                  | En-BLEU↑  | GPU Inference Time (ms)| Download link |
+|-----------|--------|----------------------------------------|:-----------------:|:--------------:|:--------------:|
+| UniMERNet | Donut Swin | [UniMERNet.yaml](../../../../configs/rec/UniMERNet.yaml) |     85.91  | 2266.96 | [trained model](https://paddleocr.bj.bcebos.com/contribution/rec_unimernet_train.tar)|
+| PP-FormulaNet-S | PPHGNetV2_B4 | [PP-FormulaNet-S.yaml](../../../../configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml) |   87.00   | 202.25 |[trained model](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar)|
+| PP-FormulaNet-L | Vary_VIT_B | [PP-FormulaNet-L.yaml](../../../../configs/rec/PP-FormuaNet/PP-FormulaNet-L.yaml) |    90.36   | 1976.52  |[trained model](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_l_train.tar )|
+| LaTeX-OCR | Hybrid ViT |[LaTeX_OCR_rec.yaml](../../../../configs/rec/LaTeX_OCR_rec.yaml)|   74.55   | 	1244.61   |[trained model](https://paddleocr.bj.bcebos.com/contribution/rec_latex_ocr_train.tar)|
+
+
+The English formula evaluation set here contains both simple and complex formulas from UniMERNet, as well as simple, medium, and complex formulas independently created by PaddleX.
+
+
+## 2. Environment
+Please refer to ["Environment Preparation"](../../ppocr/environment.en.md) to configure the PaddleOCR environment, and refer to ["Project Clone"](../../ppocr/blog/clone.en.md) to clone the project code.
+
+Furthermore, additional dependencies need to be installed:
+```shell
+sudo apt-get update
+sudo apt-get install libmagickwand-dev
+pip install -r docs/algorithm/formula_recognition/requirements.txt
+```
+
+## 3. Model Training / Evaluation / Prediction
+
+Please refer to [Text Recognition Tutorial](../../ppocr/model_train/recognition.en.md). PaddleOCR modularizes the code, and training different recognition models only requires **changing the configuration file**.
+
+
+Dataset Preparation:
+
+```shell
+# download PaddleX official example dataset
+wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
+tar -xf ocr_rec_latexocr_dataset_example.tar
+```
+
+Download the Pre-trained Model:
+
+```shell
+# download the PP-FormulaNet-S pretrained model
+wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar 
+tar -xf rec_ppformulanet_s_train.tar
+```
+
+Training:
+
+Specifically, after the data preparation is completed, the training can be started. The training command is as follows:
+
+```shell
+#Single GPU training 
+python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+   -o Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+#Multi GPU training, specify the gpu number through the --gpus parameter
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' --ips=127.0.0.1   tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+        -o Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+```
+
+Evaluation:
+
+```shell
+# GPU evaluation
+ python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml -o \
+ Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+```
+
+Prediction:
+
+```shell
+# The configuration file used for prediction must match the training
+python3 tools/infer_rec.py -c configs/rec/PP-FormuaNet/PP-FormulaNet-S.yaml \
+  -o  Global.infer_img='./docs/datasets/images/pme_demo/0000295.png'\
+   Global.pretrained_model=./rec_ppformulanet_s_train/best_accuracy.pdparams
+```
+
+## 4. FAQ
@@ -0,0 +1,154 @@
+# 通用数学公式识别算法-UniMERNet
+
+## 1. 算法简介
+
+原始项目：
+> [https://github.com/opendatalab/UniMERNet](https://github.com/opendatalab/UniMERNet)
+
+
+`UniMERNet`使用[`UniMERNet通用公式识别数据集`](https://opendatalab.com/OpenDataLab/UniMER-Dataset)进行训练，在对应测试集上的精度如下：
+
+| 模型        | 骨干网络       | 配置文件                                                  | SPE-<br/>BLEU↑ | SPE-<br/>EditDis↓ | CPE-<br/>BLEU↑  |CPE-<br/>EditDis↓ | SCE-<br/>BLEU↑ | SCE-<br/>EditDis↓ | HWE-<br/>BLEU↑ | HWE-<br/>EditDis↓ | 下载链接 |
+|-----------|------------|-------------------------------------------------------|:--------------:|:-----------------:|:----------:|:----------------:|:---------:|:-----------------:|:--------------:|:-----------------:|-------|
+| UniMERNet | Donut Swin | [UniMERNet.yaml](../../../../configs/rec/UniMERNet.yaml) |     0.9187     |      0.0584       |  0.9252    |      0.0596      | 0.6068 |     0.2297        |   0.9157|     0.0546           |[训练模型](https://paddleocr.bj.bcebos.com/contribution/rec_unimernet_train.tar)|
+
+其中，SPE表示简单公式，CPE表示复杂公式，SCE表示扫描捕捉公式，HWE表示手写公式。每种类型的公式示例图如下：
+![unimernet_dataset](https://github.com/user-attachments/assets/fb801a36-5614-4031-8585-700bd9f8fb2e)
+
+## 2. 环境配置
+请先参考[《运行环境准备》](../../ppocr/environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](../../ppocr/blog/clone.md)克隆项目代码。
+
+此外，需要安装额外的依赖：
+```shell
+sudo apt-get update
+sudo apt-get install libmagickwand-dev
+pip install -r docs/algorithm/formula_recognition/requirements.txt
+```
+
+## 3. 模型训练、评估、预测
+
+### 3.1 准备数据集
+
+从 [OpenDataLab](https://opendatalab.com/OpenDataLab/UniMER-Dataset) 上下载 UniMER-1M.zip 和 UniMER-Test.zip。
+从 [好未来平台](https://ai.100tal.com/dataset) 下载 HME100K 数据集。之后， 使用如下命令创建数据集目录，并对数据集进行转换。
+
+```shell
+# 创建 UniMERNet 数据集目录
+mkdir -p train_data/UniMERNet
+# 解压 UniMERNet 、 UniMER-Test.zip 和 HME100K.zip
+unzip -d train_data/UniMERNet path/UniMER-1M.zip
+unzip -d train_data/UniMERNet path/UniMER-Test.zip
+unzip -d train_data/UniMERNet/HME100K train_data/UniMERNet/HME100K/train.zip
+unzip -d train_data/UniMERNet/HME100K train_data/UniMERNet/HME100K/test.zip
+# 训练集转换   
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet \
+     --datatype=unimernet_train \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-1M/train.txt \
+     --hme100k_txt_path=train_data/UniMERNet/HME100K/train_labels.txt \
+     --output_path=train_data/UniMERNet/train_unimernet_1M.txt
+# 测试集转换
+# SPE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/spe \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/spe.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_spe.txt
+# CPE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/cpe \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/cpe.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_cpe.txt
+# SCE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/sce \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/sce.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_sce.txt
+# HWE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/hwe \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/hwe.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_hwe.txt
+```
+
+
+### 3.2 下载预训练模型
+
+```shell
+# 下载 Texify 预训练模型
+wget -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/pretrained/texify.pdparams
+```
+
+
+### 3.3 模型训练
+
+请参考[文本识别训练教程](../../ppocr/model_train/recognition.md)。PaddleOCR对代码进行了模块化，训练`UniMERNet`识别模型时需要**更换配置文件**为`UniMERNet`的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/{{PADDLEOCR_GITHUB_REF}}/configs/rec/UniMERNet.yaml)。
+
+#### 启动训练
+
+
+具体地，在完成数据准备后，便可以启动训练，训练命令如下：
+```shell
+#单卡训练 (默认训练方式)
+python3 tools/train.py -c configs/rec/UniMERNet.yaml \
+   -o Global.pretrained_model=./pretrain_models/texify.pdparams
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' --ips=127.0.0.1   tools/train.py -c configs/rec/UniMERNet.yaml \
+        -o Global.pretrained_model=./pretrain_models/texify.pdparams
+```
+
+**注意：**
+
+- 默认每训练 1个epoch（37880 次iteration）进行1次评估，若您更改训练的batch_size，或更换数据集，请在训练时作出如下修改
+```
+python3  -m paddle.distributed.launch --gpus '0,1,2,3' --ips=127.0.0.1   tools/train.py -c configs/rec/UniMERNet.yaml \
+  -o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
+   Global.pretrained_model=./pretrain_models/texify.pdparams
+```
+
+### 3.4 评估
+
+可下载已训练完成的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_unimernet_train.tar)，使用如下命令进行评估：
+
+```shell
+# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
+ # SPE 测试集评估
+ python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/spe \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_spe.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+ # CPE 测试集评估
+ python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/cpe \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_cpe.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+ # SCE 测试集评估
+  python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/sce \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_sce.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+ # HWE 测试集评估
+ python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/hwe \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_hwe.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+
+```
+
+### 3.5 预测
+
+使用如下命令进行单张图片预测：
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/infer_rec.py -c configs/rec/UniMERNet.yaml \
+  -o  Global.infer_img='./docs/datasets/images/pme_demo/0000099.png'\
+   Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+# 预测文件夹下所有图像时，可修改infer_img为文件夹，如 Global.infer_img='./doc/datasets/pme_demo/'。
+```
+
+## 4. FAQ
+
+1. UniMERNet 数据集来自于[UniMERNet源repo](https://github.com/opendatalab/UniMERNet) 。
@@ -0,0 +1,136 @@
+# UniMERNet
+
+## 1. Introduction
+
+Original Project:
+> [https://github.com/opendatalab/UniMERNet](https://github.com/opendatalab/UniMERNet)
+
+
+Using UniMERNet general mathematical expression recognition datasets for training, and evaluating on its test sets, the algorithm reproduction effect is as follows:
+
+| Model           | Backbone       | config                                                  | SPE-<br/>BLEU↑ | SPE-<br/>EditDis↓ | CPE-<br/>BLEU↑  |CPE-<br/>EditDis↓ | SCE-<br/>BLEU↑ | SCE-<br/>EditDis↓ | HWE-<br/>BLEU↑ | HWE-<br/>EditDis↓ | Download link |
+|-----------|--------|---------------------------------------------------|:--------------:|:-----------------:|:----------:|:----------------:|:---------:|:-----------------:|:--------------:|:-----------------:|---|
+| UniMERNet | Donut Swin | [UniMERNet.yaml](../../../../configs/rec/UniMERNet.yaml) |     0.9187     |      0.0584       |  0.9252    |      0.0596      | 0.6068 |     0.2297        |   0.9157|     0.0546           |[trained model](https://paddleocr.bj.bcebos.com/contribution/rec_unimernet_train.tar)|
+
+SPE represents simple formulas, CPE represents complex formulas, SCE represents scanned captured formulas, and HWE represents handwritten formulas. Example images of each type of formula are shown below:
+
+![unimernet_dataset](https://github.com/user-attachments/assets/fb801a36-5614-4031-8585-700bd9f8fb2e)
+
+## 2. Environment
+Please refer to ["Environment Preparation"](../../ppocr/environment.en.md) to configure the PaddleOCR environment, and refer to ["Project Clone"](../../ppocr/blog/clone.en.md) to clone the project code.
+
+Furthermore, additional dependencies need to be installed:
+```shell
+sudo apt-get update
+sudo apt-get install libmagickwand-dev
+pip install -r docs/algorithm/formula_recognition/requirements.txt
+```
+
+## 3. Model Training / Evaluation / Prediction
+
+Please refer to [Text Recognition Tutorial](../../ppocr/model_train/recognition.en.md). PaddleOCR modularizes the code, and training different recognition models only requires **changing the configuration file**.
+
+
+Dataset Preparation:
+
+Download UniMER-1M.zip and UniMER-Test.zip from [OpenDataLab](https://opendatalab.com/OpenDataLab/UniMER-Dataset). Download the HME100K dataset from the [TAL AI Platform](https://ai.100tal.com/dataset). After that, use the following command to create a dataset directory and convert the dataset.
+
+```shell
+# create the UniMERNet dataset directory
+mkdir -p train_data/UniMERNet
+# unzip UniMERNet 、 UniMER-Test.zip and HME100K.zip
+unzip -d train_data/UniMERNet path/UniMER-1M.zip
+unzip -d train_data/UniMERNet path/UniMER-Test.zip
+unzip -d train_data/UniMERNet/HME100K train_data/UniMERNet/HME100K/train.zip
+unzip -d train_data/UniMERNet/HME100K train_data/UniMERNet/HME100K/test.zip
+# convert the training set 
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet \
+     --datatype=unimernet_train \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-1M/train.txt \
+     --hme100k_txt_path=train_data/UniMERNet/HME100K/train_labels.txt \
+     --output_path=train_data/UniMERNet/train_unimernet_1M.txt
+# convert the test set
+# SPE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/spe \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/spe.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_spe.txt
+# CPE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/cpe \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/cpe.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_cpe.txt
+# SCE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/sce \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/sce.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_sce.txt
+# HWE
+python ppocr/utils/formula_utils/unimernet_data_convert.py \
+     --image_dir=train_data/UniMERNet/UniMER-Test/hwe \
+     --datatype=unimernet_test \
+     --unimernet_txt_path=train_data/UniMERNet/UniMER-Test/hwe.txt \
+     --output_path=train_data/UniMERNet/test_unimernet_hwe.txt
+```
+
+Download the Pre-trained Model:
+
+```shell
+# download the Texify pre-trained model
+wget -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/pretrained/texify.pdparams
+```
+
+Training:
+
+Specifically, after the data preparation is completed, the training can be started. The training command is as follows:
+
+```shell
+#Single GPU training 
+python3 tools/train.py -c configs/rec/UniMERNet.yaml \
+   -o Global.pretrained_model=./pretrain_models/texify.pdparams
+#Multi GPU training, specify the gpu number through the --gpus parameter
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' --ips=127.0.0.1   tools/train.py -c configs/rec/UniMERNet.yaml \
+        -o Global.pretrained_model=./pretrain_models/texify.pdparams
+```
+
+Evaluation:
+
+```shell
+# GPU evaluation
+ # SPE test set evaluation
+ python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/spe \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_spe.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+ # CPE test set evaluation
+ python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/cpe \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_cpe.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+ # SCE test set evaluation
+  python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/sce \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_sce.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+ # HWE test set evaluation
+ python3 tools/eval.py -c configs/rec/UniMERNet.yaml -o \
+  Eval.dataset.data_dir=./train_data/UniMERNet/UniMER-Test/hwe \
+  Eval.dataset.label_file_list=["./train_data/UniMERNet/test_unimernet_hwe.txt"] \
+ Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+
+```
+
+Prediction:
+
+```shell
+# The configuration file used for prediction must match the training
+python3 tools/infer_rec.py -c configs/rec/UniMERNet.yaml \
+  -o  Global.infer_img='./docs/datasets/images/pme_demo/0000099.png'\
+   Global.pretrained_model=./rec_unimernet_train/best_accuracy.pdparams
+```
+
+## 4. FAQ
@@ -0,0 +1,4 @@
+tokenizers==0.19.1
+imagesize
+ftfy
+Wand