---
comments: true
---

# Serving

Serving is a common deployment method in real-world production environments. By encapsulating inference capabilities as services, clients can access these services via network requests to obtain inference results. **The client-side code can be written in different programming languages and does not need to match the server-side code.** PaddleOCR recommends using [PaddleX](https://github.com/PaddlePaddle/PaddleX) for serving. Please refer to [Differences and Connections between PaddleOCR and PaddleX](../../paddleocr_and_paddlex.en.md#1-differences-and-connections-between-paddleocr-and-paddlex) to understand the relationship between PaddleOCR and PaddleX.

PaddleX provides the following serving solutions:

- **Basic Serving**: An easy-to-use serving solution with low development costs.
- **High-Stability Serving**: Built based on [NVIDIA Triton Inference Server](https://developer.nvidia.com/triton-inference-server). Compared to the basic serving, this solution offers higher stability and allows users to adjust configurations to optimize performance.

**It is recommended to first use the basic serving solution for quick validation**, and then evaluate whether to try more complex solutions based on actual needs.

## 1. Basic Serving

### 1.1 Install Dependencies

Run the following command to install the PaddleX serving plugin via PaddleX CLI:

```bash
paddlex --install serving
```

### 1.2 Run the Server

Run the server via PaddleX CLI:

```bash
paddlex --serve --pipeline {PaddleX pipeline registration name or pipeline configuration file path} [{other command-line options}]
```

Take the general OCR pipeline as an example:

```bash
paddlex --serve --pipeline OCR
```

You should see information similar to the following:

```text
INFO:     Started server process [63108]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
```

To adjust configurations (such as model path, batch size, deployment device, etc.), specify `--pipeline` as a custom configuration file. Refer to [PaddleOCR and PaddleX](../../paddleocr_and_paddlex.en.md) for the mapping between PaddleOCR pipelines and PaddleX pipeline registration names, as well as how to obtain and modify PaddleX pipeline configuration files.

The command-line options related to serving are as follows:

<table>
<thead>
<tr>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>--pipeline</code></td>
<td>PaddleX pipeline registration name or pipeline configuration file path.</td>
</tr>
<tr>
<td><code>--device</code></td>
<td>Deployment device for the pipeline. By default, a GPU will be used if available; otherwise, a CPU will be used."</td>
</tr>
<tr>
<td><code>--host</code></td>
<td>Hostname or IP address to which the server is bound. Defaults to <code>0.0.0.0</code>.</td>
</tr>
<tr>
<td><code>--port</code></td>
<td>Port number on which the server listens. Defaults to <code>8080</code>.</td>
</tr>
<tr>
<td><code>--use_hpip</code></td>
<td>If specified, uses high-performance inference. Refer to the High-Performance Inference documentation for more information.</td>
</tr>
<tr>
<td><code>--hpi_config</code></td>
<td>High-performance inference configuration. Refer to the High-Performance Inference documentation for more information.</td>
</tr>
</tbody>
</table>

### 1.3 Invoke the Service

The <b>"Development Integration/Deployment"</b> section in the PaddleOCR pipeline tutorial provides API references and multi-language invocation examples for the service.

## 2. High-Stability Serving

Please refer to the [PaddleX Serving Guide](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_deploy/serving.html#2). More information about PaddleX pipeline configuration files can be found in [Using PaddleX Pipeline Configuration Files](../../paddleocr_and_paddlex.en.md#3-using-paddlex-pipeline-configuration-files).

It should be noted that, due to the lack of fine-grained optimization and other reasons, the current high-stability serving deployment solution provided by PaddleOCR may not match the performance of the 2.x version based on PaddleServing. However, this new solution fully supports the PaddlePaddle 3.0 framework. We will continue to optimize it and consider introducing more performant deployment solutions in the future.

## 3. Returning Binary Content as URLs

By default, both basic serving and high-stability serving return images and other binary content in the response inline as Base64-encoded strings. When the response contains large images or a multi-page PDF, Base64 encoding can significantly inflate the payload; you can configure the service to return URLs instead. Enable it in the `Serving` section of the pipeline configuration file (`return_urls` is a top-level switch; object-storage settings live under `Serving.extra`) to return those fields as pre-signed URLs:

```yaml
Serving:
  return_urls: true
  extra:
    file_storage:
      type: bos
      endpoint: <BOS endpoint, e.g. https://bj.bcebos.com>
      ak: xxx
      sk: xxx
      bucket_name: <bucket name>
    url_expires_in: 3600  # Pre-signed URL lifetime in seconds; -1 means no expiry
```

- Basic serving: write the configuration to the pipeline config file passed to `paddlex --serve --pipeline`.
- High-stability serving: the same configuration applies — write it to `server/pipeline_config.yaml` inside the SDK and restart the container.

Currently, URL return is only supported by the `bos` (Baidu Intelligent Cloud object storage) backend. URL return is controlled by the top-level `Serving.return_urls` field, which applies to every Base64-inlined file field in the response (not just images). For the full configuration reference, notes, and use cases, see [PaddleX Serving Guide - Returning Binary Content as URLs](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_deploy/serving.html#3). See the [Baidu Intelligent Cloud documentation](https://cloud.baidu.com/doc/BOS/index.html) for AK/SK retrieval.