docs: make Chinese README the default
Main / Python 3.11 - Test (push) Has been cancelled
Main / Python 3.11 - Build (push) Has been cancelled
Main / Python 3.11 - Docs (push) Has been cancelled
Main / Python 3.11 - Lint (push) Has been cancelled
Main / Python 3.11 - Style (push) Has been cancelled
Main / GPU CI (push) Has been cancelled
Main / Release (push) Has been cancelled
Main / Build and Push Docker Images (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 11:09:13 +00:00
parent 80c362ad31
commit 4cf0739cf5
+116 -112
View File
@@ -1,3 +1,9 @@
<!-- WEHUB_ZH_README -->
> [!NOTE]
> 本文档由 WeHub 基于上游 README 翻译整理,属于社区翻译,非官方中文文档。
> [English](./README.en.md) · [原始项目](https://github.com/allenai/olmocr) · [上游 README](https://github.com/allenai/olmocr/blob/HEAD/README.md)
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
<div align="center">
<img width="350" alt="olmocr-2-full@2x" src="https://github.com/user-attachments/assets/24f1b596-4059-46f1-8130-5d72dcc0b02e" />
<hr/>
@@ -23,34 +29,33 @@
</a>
</p>
A toolkit for converting PDFs and other image-based document formats into clean, readable, plain text format.
一款将 PDF 及其他基于图像的文档格式转换为干净、可读纯文本格式的工具包。
Try the online demo: [https://olmocr.allenai.org/](https://olmocr.allenai.org/)
试用在线演示:[https://olmocr.allenai.org/](https://olmocr.allenai.org/)
Features:
- Convert PDF, PNG, and JPEG based documents into clean Markdown
- Support for equations, tables, handwriting, and complex formatting
- Automatically removes headers and footers
- Convert into text with a natural reading order, even in the presence of
figures, multi-column layouts, and insets
- Efficient, less than $200 USD per million pages converted
- (Based on a 7B parameter VLM, so it requires a GPU)
功能特性:
- 将基于 PDFPNG JPEG 的文档转换为干净的 Markdown
- 支持公式、表格、手写体及复杂排版
- 自动移除页眉和页脚
- 即使存在插图、多栏布局和嵌入内容,也能按自然阅读顺序转换为文本
- 高效,每转换一百万页成本低于 200 美元
- (基于 70 亿参数 VLMVision Language Model,视觉语言模型),因此需要 GPU)
### News
- October 21, 2025 - v0.4.0 - [New model release](https://huggingface.co/allenai/olmOCR-2-7B-1025-FP8), boosts olmOCR-bench score by ~4 points using synthetic data and introduces RL training.
- August 13, 2025 - v0.3.0 - [New model release](https://huggingface.co/allenai/olmOCR-7B-0825-FP8), fixes auto-rotation detection, and hallucinations on blank documents.
- July 24, 2025 - v0.2.1 - [New model release](https://huggingface.co/allenai/olmOCR-7B-0725-FP8), scores 3 points higher on [olmOCR-Bench](https://github.com/allenai/olmocr/tree/main/olmocr/bench), also runs significantly faster because it's default FP8, and needs much fewer retries per document.
- July 23, 2025 - v0.2.0 - New cleaned up [trainer code](https://github.com/allenai/olmocr/tree/main/olmocr/train), makes it much simpler to train olmOCR models yourself.
- June 17, 2025 - v0.1.75 - Switch from sglang to vllm based inference pipeline, updated docker image to CUDA 12.8.
- May 23, 2025 - v0.1.70 - Official docker support and images are now available! [See Docker usage](#using-docker)
- May 19, 2025 - v0.1.68 - [olmOCR-Bench](https://github.com/allenai/olmocr/tree/main/olmocr/bench) launch, scoring 77.4. Launch includes 2 point performance boost in olmOCR pipeline due to bug fixes with prompts.
- Mar 17, 2025 - v0.1.60 - Performance improvements due to better temperature selection in sampling.
- Feb 25, 2025 - v0.1.58 - Initial public launch and demo.
### 新闻
- 2025 年 10 月 21 日 - v0.4.0 - [新模型发布](https://huggingface.co/allenai/olmOCR-2-7B-1025-FP8), 通过合成数据将 olmOCR-bench 分数提升约 4 分,并引入 RLReinforcement Learning,强化学习)训练。
- 2025 年 8 月 13 日 - v0.3.0 - [新模型发布](https://huggingface.co/allenai/olmOCR-7B-0825-FP8), 修复了自动旋转检测及空白文档上的幻觉问题。
- 2025 年 7 月 24 日 - v0.2.1 - [新模型发布](https://huggingface.co/allenai/olmOCR-7B-0725-FP8), [olmOCR-Bench](https://github.com/allenai/olmocr/tree/main/olmocr/bench), 上得分提高 3 分,同时由于默认采用 FP8,运行速度显著加快,且每份文档所需重试次数大幅减少。
- 2025 年 7 月 23 日 - v0.2.0 - 全新整理的 [训练代码](https://github.com/allenai/olmocr/tree/main/olmocr/train), 使自行训练 olmOCR 模型变得更加简单。
- 2025 年 6 月 17 日 - v0.1.75 - 推理流水线从 sglang 切换为基于 vLLM 的实现,Docker 镜像更新至 CUDA 12.8
- 2025 年 5 月 23 日 - v0.1.70 - 官方 Docker 支持及镜像现已可用![查看 Docker 用法](#using-docker)
- 2025 年 5 月 19 日 - v0.1.68 - [olmOCR-Bench](https://github.com/allenai/olmocr/tree/main/olmocr/bench) 发布,得分 77.4。发布版本因提示词相关 bug 修复,olmOCR 流水线性能提升 2 分。
- 2025 年 3 月 17 日 - v0.1.60 - 通过改进采样温度选择提升性能。
- 2025 年 2 月 25 日 - v0.1.58 - 首次公开发布及演示。
### Benchmark
### 基准测试
[**olmOCR-Bench**](https://github.com/allenai/olmocr/tree/main/olmocr/bench):
We also ship a comprehensive benchmark suite covering over 7,000 test cases across 1,400 documents to help measure performance of OCR systems.
我们还提供一套全面的基准测试套件,涵盖 1,400 份文档中的 7,000 多个测试用例,用于衡量 OCR 系统的性能。
<table>
<thead>
@@ -183,45 +188,44 @@ We also ship a comprehensive benchmark suite covering over 7,000 test cases acro
</table>
### Installation
### 安装
#### System Dependencies
#### 系统依赖
You will need to install poppler-utils and additional fonts for rendering PDF images.
你需要安装 poppler-utils 以及用于渲染 PDF 图像的额外字体。
Install dependencies (Ubuntu/Debian):
安装依赖(Ubuntu/Debian):
```bash
sudo apt-get update
sudo apt-get install poppler-utils ttf-mscorefonts-installer msttcorefonts fonts-crosextra-caladea fonts-crosextra-carlito gsfonts lcdf-typetools
```
#### Python Installation
#### Python 安装
Set up a conda environment and install olmocr. The requirements for running olmOCR
are difficult to install in an existing python environment, so please do make a clean python environment to install into.
设置 conda 环境并安装 olmocr。运行 olmOCR 的依赖项在现有 Python 环境中较难安装,因此请务必创建一个干净的 Python 环境进行安装。
```bash
conda create -n olmocr python=3.11
conda activate olmocr
```
Choose the installation option that matches your use case:
选择与你的使用场景匹配的安装选项:
**Option 1: Remote Inference (Lightweight)**
**选项 1:远程推理(轻量)**
If you plan to use a remote vLLM server with the `--server` flag, install the base package:
如果你计划配合 `--server` 标志使用远程 vLLM 服务器,请安装基础包:
```bash
pip install olmocr
```
This avoids installing heavy GPU dependencies like PyTorch (~2GB+).
这可避免安装 PyTorch(约 2GB+)等重型 GPU 依赖。
**Option 2: Local GPU Inference**
**选项 2:本地 GPU 推理**
Requirements:
- Recent NVIDIA GPU (tested on RTX 4090, L40S, A100, H100) with at least 12 GB of GPU RAM
- 30GB of free disk space
要求:
- 较新的 NVIDIA GPU(已在 RTX 4090L40SA100H100 上测试),至少 12 GB GPU 显存
- 30GB 可用磁盘空间
For running inference with your own GPU:
使用自有 GPU 运行推理:
```bash
pip install olmocr[gpu] --extra-index-url https://download.pytorch.org/whl/cu128
@@ -229,23 +233,23 @@ pip install olmocr[gpu] --extra-index-url https://download.pytorch.org/whl/cu128
pip install https://download.pytorch.org/whl/cu128/flashinfer/flashinfer_python-0.2.5%2Bcu128torch2.7-cp38-abi3-linux_x86_64.whl
```
**Option 3: Beaker Cluster Execution**
**选项 3Beaker 集群执行**
For submitting jobs to Beaker clusters with the `--beaker` flag:
用于向 Beaker 集群提交作业,并配合 `--beaker` 标志:
```bash
pip install olmocr[beaker]
```
**Option 4: Benchmark Suite**
**选项 4:基准测试套件**
For running the olmOCR benchmark suite:
用于运行 olmOCR 基准测试套件:
```bash
pip install olmocr[bench]
```
**Combined Installation**
**组合安装**
You can combine multiple options:
你可以组合多个选项:
```bash
# GPU + Beaker support
pip install olmocr[gpu,beaker] --extra-index-url https://download.pytorch.org/whl/cu128
@@ -254,18 +258,18 @@ pip install olmocr[gpu,beaker] --extra-index-url https://download.pytorch.org/wh
pip install olmocr[gpu,bench] --extra-index-url https://download.pytorch.org/whl/cu128
```
**Troubleshooting**
**故障排查**
If you run into errors about `too many open files`, update your ulimit:
如果遇到与 `too many open files` 相关的错误,请更新 ulimit
```bash
ulimit -n 65536
```
### Usage Examples
### 使用示例
For quick testing, try the [web demo](https://olmocr.allen.ai/).
如需快速测试,可尝试 [web demo](https://olmocr.allen.ai/).
**Convert a Single PDF (Local GPU):**
**转换单个 PDF(本地 GPU):**
```bash
# Download a sample PDF
curl -o olmocr-sample.pdf https://olmocr.allenai.org/papers/olmocr_3pg_sample.pdf
@@ -274,28 +278,28 @@ curl -o olmocr-sample.pdf https://olmocr.allenai.org/papers/olmocr_3pg_sample.pd
olmocr ./localworkspace --markdown --pdfs olmocr-sample.pdf
```
**Convert an Image file:**
**转换图像文件:**
```bash
olmocr ./localworkspace --markdown --pdfs random_page.png
```
**Convert Multiple PDFs:**
**转换多个 PDF**
```bash
olmocr ./localworkspace --markdown --pdfs tests/gnarly_pdfs/*.pdf
```
**Use Remote Inference Server:**
**使用远程推理服务器:**
```bash
olmocr ./localworkspace --server http://remote-server:8000/v1 --model allenai/olmOCR-2-7B-1025-FP8 --markdown --pdfs *.pdf
```
With the `--markdown` flag, results will be stored as markdown files inside of `./localworkspace/markdown/`.
配合 `--markdown` 标志,结果将以 markdown 文件形式保存在 `./localworkspace/markdown/` 内。
> **Note:** You can also use `python -m olmocr.pipeline` instead of `olmocr` if you prefer.
> **注意:** 如果你愿意,也可以使用 `python -m olmocr.pipeline` 代替 `olmocr`。
#### Viewing Results
#### 查看结果
The `./localworkspace/` workspace folder will then have both [Dolma](https://github.com/allenai/dolma) and markdown files (if using `--markdown`).
随后,`./localworkspace/` 工作区文件夹将同时包含 [Dolma](https://github.com/allenai/dolma) markdown 文件(若使用 `--markdown`)。
```bash
@@ -307,32 +311,32 @@ olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models
...
```
### Using an Inference Provider or External Server
### 使用推理提供商或外部服务器
If you have a vLLM server already running elsewhere (or any inference platform implementing the OpenAI API), you can point olmOCR to use it instead of spawning a local instance.
如果你已在其他地方运行 vLLM 服务器(或任何实现 OpenAI API 的推理平台),可以让 olmOCR 指向它,而无需在本地启动实例。
**Installation for Remote Inference:**
**远程推理安装:**
```bash
# Lightweight installation - no GPU dependencies needed
pip install olmocr
```
**Using an External Server:**
**使用外部服务器:**
```bash
# Use external vLLM server instead of local one
olmocr ./localworkspace --server http://remote-server:8000/v1 --model allenai/olmOCR-2-7B-1025-FP8 --markdown --pdfs tests/gnarly_pdfs/*.pdf
```
The served model name in vLLM needs to match the value provided in `--model`.
vLLM 中提供的模型名称必须与 `--model` 中提供的值一致。
**Example vLLM Server Launch:**
**vLLM 服务器启动示例:**
```bash
vllm serve allenai/olmOCR-2-7B-1025-FP8 --max-model-len 16384
```
#### Verified External Providers
#### 已验证的外部提供商
We have tested `olmOCR-2-7B-1025-FP8` on these external model providers and confirmed that they work
我们已在以下外部模型提供商上测试 `olmOCR-2-7B-1025-FP8`,并确认其可用
| | $/1M Input tokens | $/1M Output tokens | Example Command |
|-----------------------------------------------------------------------------|-------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
@@ -341,65 +345,65 @@ We have tested `olmOCR-2-7B-1025-FP8` on these external model providers and conf
| [Parasail](https://www.saas.parasail.io/serverless?name=olmocr-7b-1025-fp8) | $0.10 | $0.20 | `olmocr ./workspace --server https://api.parasail.io/v1 --api_key psk-XXXXX --workers 1 --max_concurrent_requests 20 --model allenai/olmOCR-2-7B-1025 --pdfs tests/gnarly_pdfs/*.pdf` |
Notes on arguments
- `--server`: Defines the OpenAI-compatible endpoint: ex `https://api.deepinfra.com/v1/openai`
- `--api_key`: Your API key, bassed in via Authorization Bearer HTTP header
- `--max_concurrent_requests`: Max concurrent requests that will be in-flight to the inference provider at one time
- `--workers`: Max number of page groups that will be processed at once. You may want to set this to `1` so that you finish one group of stuff before moving on.
- `--pages_per_group`: You may want a smaller number of pages per group as many external provides have lower concurrent request limits
- `--model`: The model identifier, ex. `allenai/olmOCR-2-7B-1025`, different providers have different names, and if you run locally, you can use `olmocr`
- Other arguments work the same as with local inference
参数说明
- `--server`:定义 OpenAI 兼容端点,例如 `https://api.deepinfra.com/v1/openai`
- `--api_key`:你的 API 密钥,通过 Authorization Bearer HTTP 标头传入
- `--max_concurrent_requests`:同时发往推理提供商的在途请求最大并发数
- `--workers`:一次处理的最大页面组数量。你可能希望将其设为 `1`,以便在处理下一组之前先完成当前一组。
- `--pages_per_group`:你可能希望每组包含更少的页数,因为许多外部提供商的并发请求上限较低
- `--model`:模型标识符,例如 `allenai/olmOCR-2-7B-1025`;不同提供商的名称各不相同,若在本地运行,可使用 `olmocr`
- 其他参数与本地推理时的用法相同
### Multi-node / Cluster Usage
### 多节点 / 集群用法
If you want to convert millions of PDFs using multiple nodes running in parallel, olmOCR supports
reading PDFs from AWS S3 and coordinating work using an AWS S3 output bucket.
如果你想使用多个并行运行的节点将数百万份 PDF 进行转换,olmOCR 支持
从 AWS S3 读取 PDF,并使用 AWS S3 输出存储桶协调任务。
**Start the first worker node:**
**启动第一个工作节点:**
```bash
olmocr s3://my_s3_bucket/pdfworkspaces/exampleworkspace --pdfs s3://my_s3_bucket/jakep/gnarly_pdfs/*.pdf
```
This sets up a simple work queue in your AWS bucket and starts converting PDFs.
这会在你的 AWS 存储桶中建立一个简单的工作队列,并开始转换 PDF
**On subsequent worker nodes:**
**在后续工作节点上:**
```bash
olmocr s3://my_s3_bucket/pdfworkspaces/exampleworkspace
```
They will automatically start grabbing items from the same workspace queue.
它们会自动从同一工作区队列中获取任务。
#### Using Beaker for Cluster Execution
#### 使用 Beaker 进行集群执行
If you are at Ai2 and want to linearize millions of PDFs efficiently using [beaker](https://www.beaker.org), install with Beaker support:
如果你在 Ai2,并希望借助 [beaker](https://www.beaker.org), 高效地将数百万份 PDF 线性化,请安装带 Beaker 支持的版本:
```bash
pip install olmocr[gpu,beaker] --extra-index-url https://download.pytorch.org/whl/cu128
```
Then use the `--beaker` flag to prepare the workspace locally and launch N GPU workers in the cluster:
然后使用 `--beaker` 标志在本地准备工作区,并在集群中启动 N 个 GPU 工作节点:
```bash
olmocr s3://my_s3_bucket/pdfworkspaces/exampleworkspace --pdfs s3://my_s3_bucket/jakep/gnarly_pdfs/*.pdf --beaker --beaker_gpus 4
```
### Using Docker
### 使用 Docker
Pull the Docker image (large, includes the model, ~30GB):
拉取 Docker 镜像(体积较大,包含模型,约 30GB):
```bash
docker pull alleninstituteforai/olmocr:latest-with-model
```
For advanced users who want to manage their own model downloads, we also provide a base image without the model:
对于希望自行管理模型下载的高级用户,我们还提供不含模型的基础镜像:
```bash
docker pull alleninstituteforai/olmocr:latest
```
#### Quick Start - Process PDFs
#### 快速开始 - 处理 PDF
Process a single PDF in your current directory:
处理当前目录中的单个 PDF
```bash
docker run --gpus all \
-v $(pwd):/workspace \
@@ -407,7 +411,7 @@ docker run --gpus all \
-c "olmocr /workspace/output --markdown --pdfs /workspace/sample.pdf"
```
Process multiple PDFs:
处理多个 PDF
```bash
docker run --gpus all \
-v /path/to/pdfs:/input \
@@ -416,18 +420,18 @@ docker run --gpus all \
-c "olmocr /output --markdown --pdfs /input/*.pdf"
```
#### Interactive Mode
#### 交互模式
Run the container interactively for exploration and debugging:
以交互方式运行容器,便于探索和调试:
```bash
docker run -it --gpus all alleninstituteforai/olmocr:latest-with-model
```
> Visit our Docker repository on [Docker Hub](https://hub.docker.com/r/alleninstituteforai/olmocr) for more information.
> 访问我们在 [Docker Hub](https://hub.docker.com/r/alleninstituteforai/olmocr) 上的 Docker 仓库以了解更多信息。
### Full Documentation
### 完整文档
To see all available options:
要查看所有可用选项:
```bash
olmocr --help
usage: pipeline.py [-h] [--pdfs [PDFS ...]] [--model MODEL] [--workspace_profile WORKSPACE_PROFILE] [--pdf_profile PDF_PROFILE] [--pages_per_group PAGES_PER_GROUP] [--max_page_retries MAX_PAGE_RETRIES] [--max_page_error_rate MAX_PAGE_ERROR_RATE] [--workers WORKERS]
@@ -490,41 +494,41 @@ beaker/cluster execution:
Beaker priority level for the job
```
## Code overview
## 代码概览
There are some nice reusable pieces of the code that may be useful for your own projects:
- A prompting strategy to get really good natural text parsing using ChatGPT 4o - [buildsilver.py](https://github.com/allenai/olmocr/blob/main/olmocr/data/buildsilver.py)
- Basic filtering by language and SEO spam removal - [filter.py](https://github.com/allenai/olmocr/blob/main/olmocr/filter/filter.py)
- SFT Finetuning code for Qwen2.5-VL - [train.py](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py)
- GRPO RL Trainer - [grpo_train.py](https://github.com/allenai/olmocr/blob/main/olmocr/train/grpo_train.py)
- Synthetic data generation - [mine_html_templates.py](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py)
- Processing millions of PDFs through a finetuned model using VLLM - [pipeline.py](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py)
- Viewing [Dolma docs](https://github.com/allenai/dolma) created from PDFs - [dolmaviewer.py](https://github.com/allenai/olmocr/blob/main/olmocr/viewer/dolmaviewer.py)
其中有一些很好的可复用代码片段,也许对你自己的项目有帮助:
- 一种使用 ChatGPT 4o 获得高质量自然语言文本解析的提示策略(prompting strategy- [buildsilver.py](https://github.com/allenai/olmocr/blob/main/olmocr/data/buildsilver.py)
- 按语言进行基础过滤并去除 SEO 垃圾内容 - [filter.py](https://github.com/allenai/olmocr/blob/main/olmocr/filter/filter.py)
- Qwen2.5-VL 的 SFT 微调代码 - [train.py](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py)
- GRPO 强化学习(RL)训练器 - [grpo_train.py](https://github.com/allenai/olmocr/blob/main/olmocr/train/grpo_train.py)
- 合成数据生成 - [mine_html_templates.py](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py)
- 使用 VLLM 通过微调模型处理数百万份 PDF - [pipeline.py](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py)
- 查看由 PDF 生成的 [Dolma 文档](https://github.com/allenai/dolma) - [dolmaviewer.py](https://github.com/allenai/olmocr/blob/main/olmocr/viewer/dolmaviewer.py)
## Team
## 团队
<!-- start team -->
**olmOCR** is developed and maintained by the AllenNLP team, backed by [the Allen Institute for Artificial Intelligence (AI2)](https://allenai.org/).
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
To learn more about who specifically contributed to this codebase, see [our contributors](https://github.com/allenai/olmocr/graphs/contributors) page.
**olmOCR** 由 AllenNLP 团队开发并维护,得到 [艾伦人工智能研究所(Allen Institute for Artificial IntelligenceAI2](https://allenai.org/). 的支持
AI2 是一家非营利研究机构,使命是通过高影响力的 AI 研究与工程为人类作出贡献。
若要了解具体有哪些人为本代码库作出贡献,请参阅[我们的贡献者](https://github.com/allenai/olmocr/graphs/contributors) 页面。
<!-- end team -->
## License
## 许可证
<!-- start license -->
**olmOCR** is licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
A full copy of the license can be found [on GitHub](https://github.com/allenai/olmocr/blob/main/LICENSE).
**olmOCR** 采用 [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). 许可证
完整许可证副本可在 [GitHub](https://github.com/allenai/olmocr/blob/main/LICENSE). 上找到
<!-- end license -->
## Citing
## 引用
For olmOCR v1 and OlmOCR-bench:
引用 olmOCR v1 OlmOCR-bench
```bibtex
@misc{olmocrbench,
title={{olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models}},
@@ -537,7 +541,7 @@ For olmOCR v1 and OlmOCR-bench:
}
```
For olmOCR v2 Unit Testing Rewards with RL:
引用 olmOCR v2 Unit Testing Rewards with RL
```bibtex
@misc{olmocr2,
title={olmOCR 2: Unit Test Rewards for Document OCR},