wehub/microsoft--jarvis
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

docs: make Chinese README the default

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 10:34:31 +00:00
parent a74e5b38b1
commit 27596097b8
1 changed files with 80 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+80 -75
View File
@@ -1,81 +1,87 @@
<!-- WEHUB_ZH_README -->
> [!NOTE]
> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
> [English](./README.en.md) · [原始项目](https://github.com/microsoft/JARVIS) · [上游 README](https://github.com/microsoft/JARVIS/blob/HEAD/README.md)
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
# JARVIS
[![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2303.17580)
[![Open in Spaces](https://img.shields.io/badge/%F0%9F%A4%97-Open%20in%20Spaces-blue)](https://huggingface.co/spaces/microsoft/HuggingGPT)
The mission of JARVIS is to explore artificial general intelligence (AGI) and deliver cutting-edge research to the whole community.
JARVIS 的使命是探索通用人工智能（AGI），并将前沿研究成果带给整个社区。
## What's New
## 最新动态
+ [2024.01.15] We release Easytool for easier tool usage.
+ The code and datasets are available at [EasyTool](/easytool).
+ The paper is available at [EasyTool: Enhancing LLM-based Agents with Concise Tool Instruction](https://arxiv.org/abs/2401.06201).
+ [2023.11.30] We release TaskBench for evaluating task automation capability of LLMs.
+ The code and datasets are available at [TaskBench](/taskbench).
+ The paper is available at [TaskBench: Benchmarking Large Language Models for Task Automation](https://arxiv.org/abs/2311.18760).
+ [2023.07.28] We are now in the process of planning evaluation and project rebuilding. We will release a new version of Jarvis in the near future.
+ [2023.07.24] We released a light langchain version of Jarvis. See <a href="https://github.com/langchain-ai/langchain/tree/master/libs/experimental/langchain_experimental/autonomous_agents/hugginggpt">here</a>.
+ [2023.04.16] Jarvis now supports the OpenAI service on the Azure platform and the GPT-4 model.
+ [2023.04.06] We added the Gradio demo and built the web API for `/tasks` and `/results` in `server` mode.
+ The Gradio demo is now hosted on Hugging Face Space. (Build with `inference_mode=hybrid` and `local_deployment=standard`)
+ The Web API `/tasks` and `/results` access intermediate results for `Stage #1`: task planning and `Stage #1-3`: model selection with execution results. See <a href="#Server">here</a>.
+ [2023.04.03] We added the CLI mode and provided parameters for configuring the scale of local endpoints.
+ You can enjoy a lightweight experience with Jarvis without deploying the models locally. See <a href="#Configuration">here</a>.
+ Just run `python awesome_chat.py --config configs/config.lite.yaml` to experience it.
+ [2023.04.01] We updated a version of code for building.
+ [2024.01.15] 我们发布了 Easytool，以便更轻松地使用工具。
+ 代码和数据集可在 [EasyTool](/easytool) 获取。
+ 论文可在 [EasyTool: Enhancing LLM-based Agents with Concise Tool Instruction](https://arxiv.org/abs/2401.06201). 获取。
+ [2023.11.30] 我们发布了 TaskBench，用于评估大语言模型（LLM）的任务自动化能力。
+ 代码和数据集可在 [TaskBench](/taskbench) 获取。
+ 论文可在 [TaskBench: Benchmarking Large Language Models for Task Automation](https://arxiv.org/abs/2311.18760). 获取。
+ [2023.07.28] 我们正在进行评估规划和项目重建。我们将在不久的将来发布 Jarvis 的新版本。
+ [2023.07.24] 我们发布了 Jarvis 的轻量级 langchain 版本。见 <a href="https://github.com/langchain-ai/langchain/tree/master/libs/experimental/langchain_experimental/autonomous_agents/hugginggpt">此处</a>。
+ [2023.04.16] Jarvis 现已支持 Azure 平台上的 OpenAI 服务以及 GPT-4 模型。
+ [2023.04.06] 我们添加了 Gradio 演示，并在 `server` 模式下为 `/tasks` 和 `/results` 构建了 Web API。
+ Gradio 演示现已托管在 Hugging Face Space 上。（使用 `inference_mode=hybrid` 和 `local_deployment=standard` 构建）
+ Web API `/tasks` 和 `/results` 可访问 `Stage #1` 的中间结果：任务规划，以及 `Stage #1-3`：带执行结果的模型选择。见 <a href="#Server">此处</a>。
+ [2023.04.03] 我们添加了 CLI 模式，并提供了用于配置本地端点规模的参数。
+ 无需在本地部署模型，即可享受 Jarvis 的轻量体验。见 <a href="#Configuration">此处</a>。
+ 只需运行 `python awesome_chat.py --config configs/config.lite.yaml` 即可体验。
+ [2023.04.01] 我们更新了用于构建的代码版本。
### Overview
### 概览
Language serves as an interface for LLMs to connect numerous AI models for solving complicated AI tasks!
语言充当接口，使 LLM 能够连接众多 AI 模型以解决复杂的 AI 任务！
<p align="center">
<img width="100%" alt="image" src="./hugginggpt/assets/intro.png">
</p>
See our paper: [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace](http://arxiv.org/abs/2303.17580), Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu and Yueting Zhuang (the first two authors contribute equally)
请参阅我们的论文：[HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace](http://arxiv.org/abs/2303.17580), Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu and Yueting Zhuang（前两位作者贡献相同）
We introduce a collaborative system that consists of **an LLM as the controller** and **numerous expert models as collaborative executors** (from HuggingFace Hub). The workflow of our system consists of four stages:
+ **Task Planning**: Using ChatGPT to analyze the requests of users to understand their intention, and disassemble them into possible solvable tasks.
+ **Model Selection**: To solve the planned tasks, ChatGPT selects expert models hosted on Hugging Face based on their descriptions.
+ **Task Execution**: Invokes and executes each selected model, and return the results to ChatGPT.
+ **Response Generation**: Finally, using ChatGPT to integrate the prediction of all models, and generate responses.
我们介绍了一个协作系统，由 **LLM 作为控制器** 和 **众多专家模型作为协作执行者**（来自 HuggingFace Hub）组成。我们系统的工作流程包含四个阶段：
+ **任务规划（Task Planning）**：使用 ChatGPT 分析用户请求以理解其意图，并将其拆解为可能可解决的任务。
+ **模型选择（Model Selection）**：为解决规划好的任务，ChatGPT 根据专家模型在 Hugging Face 上的描述进行选择。
+ **任务执行（Task Execution）**：调用并执行每个所选模型，并将结果返回给 ChatGPT。
+ **响应生成（Response Generation）**：最后，使用 ChatGPT 整合所有模型的预测结果并生成响应。
<p align="center"><img src="./hugginggpt/assets/overview.jpg"></p>
### System Requirements
### 系统要求
#### Default (Recommended)
#### 默认（推荐）
For `configs/config.default.yaml`:
适用于 `configs/config.default.yaml`：
+ Ubuntu 16.04 LTS
+ VRAM >= 24GB
+ RAM > 12GB (minimal), 16GB (standard), 80GB (full)
+ Disk > 284GB
+ 42GB for `damo-vilab/text-to-video-ms-1.7b`
+ 126GB for `ControlNet`
+ 66GB for `stable-diffusion-v1-5`
+ 50GB for others
+ RAM > 12GB（最低），16GB（标准），80GB（完整）
+ 磁盘 > 284GB
+ 42GB 用于 `damo-vilab/text-to-video-ms-1.7b`
+ 126GB 用于 `ControlNet`
+ 66GB 用于 `stable-diffusion-v1-5`
+ 50GB 用于其他
#### Minimum (Lite)
#### 最低（精简版）
For `configs/config.lite.yaml`:
适用于 `configs/config.lite.yaml`：
+ Ubuntu 16.04 LTS
+ Nothing else
+ 无其他要求
The configuration `configs/config.lite.yaml` does not require any expert models to be downloaded and deployed locally. However, it means that Jarvis is restricted to models running stably on HuggingFace Inference Endpoints.
配置 `configs/config.lite.yaml` 无需在本地下载和部署任何专家模型。不过，这意味着 Jarvis 仅限于在 HuggingFace Inference Endpoints 上稳定运行的模型。
### Quick Start
### 快速开始
First replace `openai.key` and `huggingface.token` in `server/configs/config.default.yaml` with **your personal OpenAI Key** and **your Hugging Face Token**, or put them in the environment variables `OPENAI_API_KEY` and `HUGGINGFACE_ACCESS_TOKEN` respectively. Then run the following commands:
请先在 `server/configs/config.default.yaml` 中将 `openai.key` 和 `huggingface.token` 替换为 **你的个人 OpenAI Key** 和 **你的 Hugging Face Token**，或分别将它们放入环境变量 `OPENAI_API_KEY` 和 `HUGGINGFACE_ACCESS_TOKEN`。然后运行以下命令：
<span id="Server"></span>
#### For Server:
#### 服务端：
```bash
# setup env
@@ -95,13 +101,13 @@ python models_server.py --config configs/config.default.yaml # required when `in
python awesome_chat.py --config configs/config.default.yaml --mode server # for text-davinci-003
```
Now you can access Jarvis' services by the Web API.
现在你可以通过 Web API 访问 Jarvis 的服务。
+ `/hugginggpt` --method `POST`, access the full service.
+ `/tasks` --method `POST`, access intermediate results for Stage #1.
+ `/results` --method `POST`, access intermediate results for Stage #1-3.
+ `/hugginggpt` --method `POST`，访问完整服务。
+ `/tasks` --method `POST`，访问第 1 阶段的中间结果。
+ `/results` --method `POST`，访问第 1-3 阶段的中间结果。
For example:
例如：
```bash
# request
@@ -121,14 +127,14 @@ curl --location 'http://localhost:8004/tasks' \
```
#### For Web:
#### Web 端：
We provide a user-friendly web page. After starting `awesome_chat.py` in a server mode, you can run the commands to communicate with Jarvis in your browser:
我们提供了一个用户友好的网页。在以服务端模式启动 `awesome_chat.py` 后，你可以运行以下命令在浏览器中与 Jarvis 交互：
- you need to install `nodejs` and `npm` first.
- [ IMPORTANT ] if you are running the web client on another machine, you need set `http://{LAN_IP_of_the_server}:{port}/` to `HUGGINGGPT_BASE_URL` of `web/src/config/index.ts`.
- if you want to use the video generation feature, you need to compile `ffmpeg` manually with H.264.
- you can switch to ChatGPT by `double click` on the setting icon!
- 你需要先安装 `nodejs` 和 `npm`。
- [ 重要 ] 如果你在另一台机器上运行 Web 客户端，需要将 `http://{LAN_IP_of_the_server}:{port}/` 设置为 `web/src/config/index.ts` 的 `HUGGINGGPT_BASE_URL`。
- 如果你想使用视频生成功能，需要手动使用 H.264 编译 `ffmpeg`。
- 你可以通过设置图标上的 `double click` 切换到 ChatGPT！
```bash
cd web
@@ -144,9 +150,9 @@ LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/ffmpeg -i input.mp4 -vcodec libx26
<span id="Gradio"></span>
#### For Gradio
#### Gradio
The Gradio demo is now hosted on Hugging Face Space. You can also run the following commands to start the demo locally:
Gradio 演示现已托管在 Hugging Face Space 上。你也可以运行以下命令在本地启动演示：
```bash
python models_server.py --config configs/config.gradio.yaml
@@ -156,16 +162,16 @@ python run_gradio_demo.py --config configs/config.gradio.yaml
docker run -it -p 7860:7860 --platform=linux/amd64 registry.hf.space/microsoft-hugginggpt:latest python app.py
```
#### For CLI:
#### CLI：
You can also run Jarvis more easily in CLI mode:
你也可以在 CLI 模式下更轻松地运行 Jarvis：
```bash
cd server
python awesome_chat.py --config configs/config.default.yaml --mode cli
```
Examples of CLI mode:
CLI 模式示例：
```
Welcome to Jarvis! A collaborative system that consists of an LLM as the controller and numerous expert models as collaborative executors. Jarvis can plan tasks, schedule Hugging Face models, generate friendly responses based on your requests, and help you with many things. Please enter your request (`exit` to exit).
@@ -177,38 +183,37 @@ Welcome to Jarvis! A collaborative system that consists of an LLM as the control
[ Jarvis ]: Sure. I have generated a canny image based on /examples/f.jpg. To do this, I first used the image-to-text model nlpconnect/vit-gpt2-image-captioning to generate the text description of the image, which is "a herd of giraffes and zebras grazing in a field". Then I used the canny-control model to generate the canny image of the original image, which is stored in the path "/images/1ea2.png". Finally, I used the canny-text-to-image model lllyasviel/sd-controlnet-canny to generate the canny image based on the text description and the canny image, which is stored in the path "/images/ba66.png". Do you need anything else?
```
### 配置
### Configuration
服务端配置文件为 `server/configs/config.default.yaml`，部分参数如下所示：
The server-side configuration file is `server/configs/config.default.yaml`, and some parameters are presented as follows:
+ `model`: LLM, currently supports `text-davinci-003`. We are working on integrating more open-source LLMs.
+ `inference_mode`: mode of inference endpoints
+ `local`: only use the local inference endpoints
+ `huggingface`: only use the Hugging Face Inference Endpoints **(free of local inference endpoints)**
+ `hybrid`: both of `local` and `huggingface`
+ `local_deployment`: scale of locally deployed models, works under `local` or `hybrid` inference mode:
+ `model`：LLM，当前支持 `text-davinci-003`。我们正在努力集成更多开源 LLM。
+ `inference_mode`：推理端点（inference endpoints）模式
+ `local`：仅使用本地推理端点
+ `huggingface`：仅使用 Hugging Face Inference Endpoints **（无需本地推理端点）**
+ `hybrid`：同时使用 `local` 和 `huggingface`
+ `local_deployment`：本地部署模型的规模，适用于 `local` 或 `hybrid` 推理模式：
+ `minimal` (RAM>12GB, ControlNet only)
+ `standard` (RAM>16GB, ControlNet + Standard Pipelines)
+ `full` (RAM>42GB, All registered models)
On a personal laptop, we recommend the configuration of `inference_mode: hybrid `and `local_deployment: minimal`. But the available models under this setting may be limited due to the instability of remote Hugging Face Inference Endpoints.
在个人笔记本电脑上，我们推荐使用 `inference_mode: hybrid ` 和 `local_deployment: minimal` 的配置。但由于远程 Hugging Face Inference Endpoints 的不稳定性，在此设置下可用模型可能会受到限制。
### NVIDIA Jetson Embedded Device Support
A [Dockerfile](./Dockerfile.jetson) is included that provides experimental support for [NVIDIA Jetson embedded devices](https://developer.nvidia.com/embedded-computing). This image provides accelerated ffmpeg, pytorch, torchaudio, and torchvision dependencies. To build the docker image, [ensure that the default docker runtime is set to 'nvidia'](https://github.com/NVIDIA/nvidia-docker/wiki/Advanced-topics#default-runtime). A pre-built image is provided at https://hub.docker.com/r/toolboc/nv-jarvis.
### NVIDIA Jetson 嵌入式设备支持
项目包含一个 [Dockerfile](./Dockerfile.jetson)，为 [NVIDIA Jetson 嵌入式设备](https://developer.nvidia.com/embedded-computing). 该镜像提供加速版 ffmpeg、pytorch、torchaudio 和 torchvision 依赖。要构建 Docker 镜像，请[确保默认 Docker 运行时设置为 'nvidia'](https://github.com/NVIDIA/nvidia-docker/wiki/Advanced-topics#default-runtime). 预构建镜像见 https://hub.docker.com/r/toolboc/nv-jarvis.
```bash
#Build the docker image
docker build --pull --rm -f "Dockerfile.jetson" -t toolboc/nv-jarvis:r35.2.1
```
Due to to memory requirements, JARVIS is required to run on Jetson AGX Orin family devices (64G on-board RAM device preferred) with config options set to:
由于内存要求，JARVIS 需要在 Jetson AGX Orin 系列设备上运行（优先选择板载 64G RAM 的设备），并将配置选项设置为：
* `inference_mode: local`
* `local_deployment: standard`
Models and configs are recommended to be provided through a volume mount from the host to the container as shown in the `docker run` step below. It is possible to uncomment the `# Download local models` section of the [Dockerfile](./Dockerfile.jetson) to build a container with models included.
建议通过从宿主机到容器的卷挂载提供模型和配置，如下方 `docker run` 步骤所示。你也可以取消注释 [Dockerfile](./Dockerfile.jetson) 中的 `# Download local models` 部分，以构建包含模型的容器。
#### Start the model server, awesomechat, and web app on Jetson Orin AGX
#### 在 Jetson Orin AGX 上启动模型服务器、awesomechat 和 Web 应用
```bash
# run the container which will automatically start the model server
@@ -223,7 +228,7 @@ docker exec jarvis python3 awesome_chat.py --config configs/config.default.yaml
docker exec jarvis npm run dev --prefix=/app/web
```
### Screenshots
### 截图
<p align="center"><img src="./hugginggpt/assets/screenshot_q.jpg"><img src="./hugginggpt/assets/screenshot_a.jpg"></p>
@@ -231,7 +236,7 @@ docker exec jarvis npm run dev --prefix=/app/web
## Citation
If you find this work useful in your method, you can cite the paper as below:
如果你在方法中觉得本工作有用，可以按如下方式引用论文：
@inproceedings{shen2023hugginggpt,
author = {Shen, Yongliang and Song, Kaitao and Tan, Xu and Li, Dongsheng and Lu, Weiming and Zhuang, Yueting},