From 415bbf96dd46dea0b8e6dc25e1109511df3b9a22 Mon Sep 17 00:00:00 2001 From: wehub-resource-sync Date: Mon, 13 Jul 2026 10:47:48 +0000 Subject: [PATCH] docs: make Chinese README the default --- README.md | 124 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 65 insertions(+), 59 deletions(-) diff --git a/README.md b/README.md index 63c5059..9a30509 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,12 @@ + +> [!NOTE] +> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。 +> [English](./README.en.md) · [原始项目](https://github.com/bentoml/OpenLLM) · [上游 README](https://github.com/bentoml/OpenLLM/blob/HEAD/README.md) +> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。 +
-

🦾 OpenLLM: Self-Hosting LLMs Made Easy

+

🦾 OpenLLM：轻松自托管 LLM

[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202-green.svg)](https://github.com/bentoml/OpenLLM/blob/main/LICENSE) [![Releases](https://img.shields.io/pypi/v/openllm.svg?logo=pypi&label=PyPI&logoColor=gold)](https://pypi.org/project/openllm) @@ -10,13 +16,13 @@
-OpenLLM allows developers to run **any open-source LLMs** (Llama 3.3, Qwen2.5, Phi3 and [more](#supported-models)) or **custom models** as **OpenAI-compatible APIs** with a single command. It features a [built-in chat UI](#chat-ui), state-of-the-art inference backends, and a simplified workflow for creating enterprise-grade cloud deployment with Docker, Kubernetes, and [BentoCloud](#deploy-to-bentocloud). +OpenLLM 让开发者只需一条命令，即可将**任意开源 LLM**（Llama 3.3、Qwen2.5、Phi3 及[更多](#supported-models)）或**自定义模型**作为**兼容 OpenAI 的 API** 运行。它提供[内置聊天 UI](#chat-ui)、最先进的推理后端，以及通过 Docker、Kubernetes 和 [BentoCloud](#deploy-to-bentocloud) 创建企业级云部署的简化工作流。 -Understand the [design philosophy of OpenLLM](https://www.bentoml.com/blog/from-ollama-to-openllm-running-llms-in-the-cloud). +了解 [OpenLLM 的设计理念](https://www.bentoml.com/blog/from-ollama-to-openllm-running-llms-in-the-cloud). -## Get Started +## 快速开始 -Run the following commands to install OpenLLM and explore it interactively. +运行以下命令安装 OpenLLM 并交互式探索。 ```bash pip install openllm # or pip3 install openllm @@ -25,16 +31,16 @@ openllm hello ![hello](https://github.com/user-attachments/assets/5af19f23-1b34-4c45-b1e0-a6798b4586d1) -## Supported models +## 支持的模型 -OpenLLM supports a wide range of state-of-the-art open-source LLMs. You can also add a [model repository to run custom models](#set-up-a-custom-repository) with OpenLLM. +OpenLLM 支持广泛的最先进开源 LLM。你也可以添加[模型仓库以运行自定义模型](#set-up-a-custom-repository) 并在 OpenLLM 中使用。 - - - - + + + + @@ -128,18 +134,18 @@ OpenLLM supports a wide range of state-of-the-art open-source LLMs. You can also
ModelParametersRequired GPUStart a Server模型参数量所需 GPU启动服务
deepseek
-For the full model list, see the [OpenLLM models repository](https://github.com/bentoml/openllm-models). +完整模型列表请参阅 [OpenLLM 模型仓库](https://github.com/bentoml/openllm-models). -## Start an LLM server +## 启动 LLM 服务 -To start an LLM server locally, use the `openllm serve` command and specify the model version. +要在本地启动 LLM 服务，请使用 `openllm serve` 命令并指定模型版本。 > [!NOTE] -> OpenLLM does not store model weights. A Hugging Face token (HF_TOKEN) is required for gated models. +> OpenLLM 不存储模型权重。访问受限（gated）模型需要 Hugging Face 令牌（HF_TOKEN）。 > -> 1. Create your Hugging Face token [here](https://huggingface.co/settings/tokens). -> 2. Request access to the gated model, such as [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct). -> 3. Set your token as an environment variable by running: +> 1. 在[此处](https://huggingface.co/settings/tokens). 创建你的 Hugging Face 令牌 +> 2. 申请受限模型的访问权限，例如 [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct). +> 3. 通过运行以下命令将令牌设置为环境变量： > ```bash > export HF_TOKEN= > ``` @@ -148,17 +154,17 @@ To start an LLM server locally, use the `openllm serve` command and specify the openllm serve llama3.2:1b ``` -The server will be accessible at [http://localhost:3000](http://localhost:3000/), providing OpenAI-compatible APIs for interaction. You can call the endpoints with different frameworks and tools that support OpenAI-compatible APIs. Typically, you may need to specify the following: +服务将在 [http://localhost:3000](http://localhost:3000/), 处可用，提供兼容 OpenAI 的 API 用于交互。你可以使用支持兼容 OpenAI API 的不同框架和工具调用端点。通常，你可能需要指定以下内容： -- **The API host address**: By default, the LLM is hosted at [http://localhost:3000](http://localhost:3000/). -- **The model name:** The name can be different depending on the tool you use. -- **The API key**: The API key used for client authentication. This is optional. +- **API 主机地址**：默认情况下，LLM 托管于 [http://localhost:3000](http://localhost:3000/). +- **模型名称：** 名称可能因你使用的工具而异。 +- **API 密钥**：用于客户端身份验证的 API 密钥。此项为可选。 -Here are some examples: +以下是一些示例：
-OpenAI Python client +OpenAI Python 客户端 ```python from openai import OpenAI @@ -198,95 +204,95 @@ llm = OpenAI(api_bese="http://localhost:3000/v1", model="meta-llama/Llama-3.2-1B
-## Chat UI +## 聊天 UI -OpenLLM provides a chat UI at the `/chat` endpoint for the launched LLM server at http://localhost:3000/chat. +OpenLLM 在 `/chat` 端点为 http://localhost:3000/chat. 上启动的 LLM 服务提供聊天 UI。 openllm_ui -## Chat with a model in the CLI +## 在 CLI 中与模型对话 -To start a chat conversation in the CLI, use the `openllm run` command and specify the model version. +要在 CLI 中开始聊天对话，请使用 `openllm run` 命令并指定模型版本。 ```bash openllm run llama3:8b ``` -## Model repository +## 模型仓库 -A model repository in OpenLLM represents a catalog of available LLMs that you can run. OpenLLM provides a default model repository that includes the latest open-source LLMs like Llama 3, Mistral, and Qwen2, hosted at [this GitHub repository](https://github.com/bentoml/openllm-models). To see all available models from the default and any added repository, use: +OpenLLM 中的模型仓库表示你可运行的可用 LLM 目录。OpenLLM 提供默认模型仓库，包含 Llama 3、Mistral 和 Qwen2 等最新开源 LLM，托管于[此 GitHub 仓库](https://github.com/bentoml/openllm-models). 要查看默认仓库及任何已添加仓库中的所有可用模型，请运行： ```bash openllm model list ``` -To ensure your local list of models is synchronized with the latest updates from all connected repositories, run: +为确保本地模型列表与所有已连接仓库的最新更新同步，请运行： ```bash openllm repo update ``` -To review a model’s information, run: +要查看模型信息，请运行： ```bash openllm model get llama3.2:1b ``` -### Add a model to the default model repository +### 向默认模型仓库添加模型 -You can contribute to the default model repository by adding new models that others can use. This involves creating and submitting a Bento of the LLM. For more information, check out this [example pull request](https://github.com/bentoml/openllm-models/pull/1). +你可以通过添加他人可使用的新模型来为默认模型仓库做贡献。这涉及创建并提交 LLM 的 Bento。更多信息请参阅此[示例拉取请求](https://github.com/bentoml/openllm-models/pull/1). -### Set up a custom repository +### 设置自定义仓库 -You can add your own repository to OpenLLM with custom models. To do so, follow the format in the default OpenLLM model repository with a `bentos` directory to store custom LLMs. You need to [build your Bentos with BentoML](https://docs.bentoml.com/en/latest/guides/build-options.html) and submit them to your model repository. +你可以将包含自定义模型的自有仓库添加到 OpenLLM。为此，请遵循默认 OpenLLM 模型仓库的格式，使用 `bentos` 目录存储自定义 LLM。你需要[使用 BentoML 构建 Bentos](https://docs.bentoml.com/en/latest/guides/build-options.html) 并将其提交到你的模型仓库。 -First, prepare your custom models in a `bentos` directory following the guidelines provided by [BentoML to build Bentos](https://docs.bentoml.com/en/latest/guides/build-options.html). Check out the [default model repository](https://github.com/bentoml/openllm-repo) for an example and read the [Developer Guide](https://github.com/bentoml/OpenLLM/blob/main/DEVELOPMENT.md) for details. +首先，按照 [BentoML 构建 Bentos 指南](https://docs.bentoml.com/en/latest/guides/build-options.html). 在 `bentos` 目录中准备自定义模型。可参考[默认模型仓库](https://github.com/bentoml/openllm-repo) 示例，并阅读[开发者指南](https://github.com/bentoml/OpenLLM/blob/main/DEVELOPMENT.md) 了解详情。 -Then, register your custom model repository with OpenLLM: +接下来，在 OpenLLM 中注册你的自定义模型仓库： ```bash openllm repo add ``` -**Note**: Currently, OpenLLM only supports adding public repositories. +**注意**：目前，OpenLLM 仅支持添加公开仓库。 -## Deploy to BentoCloud +## 部署到 BentoCloud -OpenLLM supports LLM cloud deployment via BentoML, the unified model serving framework, and BentoCloud, an AI inference platform for enterprise AI teams. BentoCloud provides fully-managed infrastructure optimized for LLM inference with autoscaling, model orchestration, observability, and many more, allowing you to run any AI model in the cloud. +OpenLLM 支持通过 BentoML（统一模型服务框架）和 BentoCloud（面向企业 AI 团队的 AI 推理平台）进行 LLM 云端部署。BentoCloud 提供针对 LLM 推理优化的全托管基础设施，具备自动扩缩容、模型编排、可观测性等众多功能，让你能够在云端运行任意 AI 模型。 -[Sign up for BentoCloud](https://www.bentoml.com/) for free and [log in](https://docs.bentoml.com/en/latest/bentocloud/how-tos/manage-access-token.html). Then, run `openllm deploy` to deploy a model to BentoCloud: +[注册 BentoCloud](https://www.bentoml.com/) 免费账号，并[登录](https://docs.bentoml.com/en/latest/bentocloud/how-tos/manage-access-token.html). 然后运行 `openllm deploy` 将模型部署到 BentoCloud： ```bash openllm deploy llama3.2:1b --env HF_TOKEN ``` > [!NOTE] -> If you are deploying a gated model, make sure to set HF_TOKEN in enviroment variables. +> 如果你正在部署受限（gated）模型，请确保在环境变量中设置 HF_TOKEN。 -Once the deployment is complete, you can run model inference on the BentoCloud console: +部署完成后，你可以在 BentoCloud 控制台运行模型推理： bentocloud_ui -## Community +## 社区 -OpenLLM is actively maintained by the BentoML team. Feel free to reach out and join us in our pursuit to make LLMs more accessible and easy to use 👉 [Join our Slack community!](https://l.bentoml.com/join-slack) +OpenLLM 由 BentoML 团队积极维护。欢迎与我们联系，加入我们一起让 LLM 更易获取、更易使用的努力 👉 [加入我们的 Slack 社区！](https://l.bentoml.com/join-slack) -## Contributing +## 贡献 -As an open-source project, we welcome contributions of all kinds, such as new features, bug fixes, and documentation. Here are some of the ways to contribute: +作为开源项目，我们欢迎各类贡献，例如新功能、缺陷修复和文档改进。以下是一些贡献方式： -- Repost a bug by [creating a GitHub issue](https://github.com/bentoml/OpenLLM/issues/new/choose). -- [Submit a pull request](https://github.com/bentoml/OpenLLM/compare) or help review other developers’ [pull requests](https://github.com/bentoml/OpenLLM/pulls). -- Add an LLM to the OpenLLM default model repository so that other users can run your model. See the [pull request template](https://github.com/bentoml/openllm-models/pull/1). -- Check out the [Developer Guide](https://github.com/bentoml/OpenLLM/blob/main/DEVELOPMENT.md) to learn more. +- 通过[创建 GitHub issue](https://github.com/bentoml/OpenLLM/issues/new/choose). 报告缺陷 +- [提交 pull request](https://github.com/bentoml/OpenLLM/compare)，或帮助审查其他开发者的 [pull request](https://github.com/bentoml/OpenLLM/pulls). +- 将 LLM 添加到 OpenLLM 默认模型仓库，以便其他用户运行你的模型。请参阅 [pull request 模板](https://github.com/bentoml/openllm-models/pull/1). +- 查看[开发者指南](https://github.com/bentoml/OpenLLM/blob/main/DEVELOPMENT.md) 了解更多。 -## Acknowledgements +## 致谢 -This project uses the following open-source projects: +本项目使用了以下开源项目： -- [bentoml/bentoml](https://github.com/bentoml/bentoml) for production level model serving -- [vllm-project/vllm](https://github.com/vllm-project/vllm) for production level LLM backend -- [blrchen/chatgpt-lite](https://github.com/blrchen/chatgpt-lite) for a fancy Web Chat UI -- [astral-sh/uv](https://github.com/astral-sh/uv) for blazing fast model requirements installing +- [bentoml/bentoml](https://github.com/bentoml/bentoml) 用于生产级模型服务 +- [vllm-project/vllm](https://github.com/vllm-project/vllm) 用于生产级 LLM 后端 +- [blrchen/chatgpt-lite](https://github.com/blrchen/chatgpt-lite) 用于精美的 Web 聊天 UI +- [astral-sh/uv](https://github.com/astral-sh/uv) 用于极速安装模型依赖 -We are grateful to the developers and contributors of these projects for their hard work and dedication. +我们感谢这些项目的开发者和贡献者所付出的辛勤工作与奉献。