docs: make Chinese README the default
This commit is contained in:
@@ -1,15 +1,20 @@
|
||||
<!-- WEHUB_ZH_README -->
|
||||
> [!NOTE]
|
||||
> 本文档由 WeHub 基于上游 README 翻译整理,属于社区翻译,非官方中文文档。
|
||||
> [English](./README.en.md) · [原始项目](https://github.com/TencentARC/Pixal3D) · [上游 README](https://github.com/TencentARC/Pixal3D/blob/HEAD/README.md)
|
||||
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
|
||||
|
||||
<div align="center">
|
||||
|
||||
# Pixal3D: Pixel-Aligned 3D Generation from Images
|
||||
# Pixal3D:从图像生成像素对齐的 3D(Pixel-Aligned 3D Generation from Images)
|
||||
|
||||
<h3>SIGGRAPH 2026</h3>
|
||||
|
||||
<small>[Dong-Yang Li](https://ldyang694.github.io/)¹ · [Wang Zhao](https://thuzhaowang.github.io/)²* · [Yuxin Chen](https://orcid.org/0000-0002-7854-1072)² · [Wenbo Hu](https://wbhu.github.io/)² · [Meng-Hao Guo](https://menghaoguo.github.io/)¹ · [Fang-Lue Zhang](https://fanglue.github.io/)³ · [Ying Shan](https://www.linkedin.com/in/YingShanProfile)² · [Shi-Min Hu](https://cg.cs.tsinghua.edu.cn/shimin.htm)¹✉</small>
|
||||
|
||||
¹Tsinghua University (BNRist) ²Tencent ARC Lab ³Victoria University of Wellington
|
||||
¹清华大学 (BNRist) ²腾讯 ARC 实验室 ³惠灵顿维多利亚大学
|
||||
|
||||
*Project lead ✉Corresponding author
|
||||
*项目负责人 ✉通讯作者
|
||||
|
||||
</div>
|
||||
|
||||
@@ -22,82 +27,82 @@
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="assets/teaser.png" alt="Teaser image of Pixal3D"/>
|
||||
<img src="assets/teaser.png" alt="Pixal3D 预览图"/>
|
||||
</div>
|
||||
|
||||
**Pixal3D** generates high-fidelity 3D assets from a single image. Unlike previous methods that loosely inject image features via attention, Pixal3D explicitly lifts pixel features into 3D through back-projection, establishing direct pixel-to-3D correspondences. This enables near-reconstruction-level fidelity with detailed geometry and PBR textures.
|
||||
**Pixal3D** 可从单张图像生成高保真 3D 资产。与此前通过注意力机制松散注入图像特征的方法不同,Pixal3D 通过反投影(back-projection)将像素特征显式提升到 3D 空间,建立直接的像素到 3D 对应关系。这使得其能够达到接近重建级别的保真度,并具备精细的几何与 PBR 纹理。
|
||||
|
||||
---
|
||||
|
||||
## ✨ News
|
||||
## ✨ 动态
|
||||
|
||||
- **May 2026**: Release training code and data preparation toolkit. 🔧
|
||||
- **May 2026**: Release the improved version based on [Trellis.2](https://github.com/microsoft/TRELLIS.2) backbone. 💪
|
||||
- **May 2026**: Release inference code and online demo. 🤗
|
||||
- **Apr 2026**: Our paper is accepted to SIGGRAPH 2026! 🎉
|
||||
- **2026 年 5 月**:发布训练代码与数据准备工具包。🔧
|
||||
- **2026 年 5 月**:发布基于 [Trellis.2](https://github.com/microsoft/TRELLIS.2) 骨干网络的改进版本。💪
|
||||
- **2026 年 5 月**:发布推理代码与在线演示。🤗
|
||||
- **2026 年 4 月**:论文被 SIGGRAPH 2026 接收!🎉
|
||||
|
||||
## 📌 Branches
|
||||
## 📌 分支
|
||||
|
||||
| Branch | Description |
|
||||
| 分支 | 说明 |
|
||||
|--------|-------------|
|
||||
| `main` | **Latest version** — improved implementation based on [Trellis.2](https://github.com/microsoft/TRELLIS.2) backbone with better performance. |
|
||||
| `paper` | **Paper version** — original implementation based on [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2), corresponding to results reported in our SIGGRAPH 2026 paper. |
|
||||
| `main` | **最新版本**——基于 [Trellis.2](https://github.com/microsoft/TRELLIS.2) 骨干网络的改进实现,性能更优。 |
|
||||
| `paper` | **论文版本**——基于 [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2), 的原始实现,对应我们 SIGGRAPH 2026 论文中报告的结果。 |
|
||||
|
||||
> If you want to reproduce the results in our paper, please switch to the `paper` branch.
|
||||
> 若要复现我们论文中的结果,请切换到 `paper` 分支。
|
||||
|
||||
## 🎮 Try It Online
|
||||
## 🎮 在线体验
|
||||
|
||||
You can try Pixal3D directly in your browser without any installation via our Hugging Face Gradio demo:
|
||||
您无需安装任何软件,即可通过我们的 Hugging Face Gradio 演示在浏览器中直接体验 Pixal3D:
|
||||
|
||||
👉 [**Launch Demo**](https://huggingface.co/spaces/TencentARC/Pixal3D)
|
||||
👉 [**启动演示**](https://huggingface.co/spaces/TencentARC/Pixal3D)
|
||||
|
||||
## 🚀 Getting Started
|
||||
## 🚀 快速开始
|
||||
|
||||
### Installation
|
||||
### 安装
|
||||
|
||||
#### Step 1: Follow TRELLIS.2 Installation
|
||||
#### 步骤 1:按 TRELLIS.2 安装指南配置
|
||||
|
||||
Please first follow the installation guide of [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) to set up the base environment.
|
||||
请先按照 [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) 的安装指南配置基础环境。
|
||||
|
||||
#### Step 2: Install Additional Dependencies
|
||||
#### 步骤 2:安装额外依赖
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
#### Step 3: Install natten
|
||||
#### 步骤 3:安装 natten
|
||||
|
||||
```bash
|
||||
NATTEN_CUDA_ARCH="xx" NATTEN_N_WORKERS=xx pip install natten==0.21.0 --no-build-isolation
|
||||
```
|
||||
|
||||
Please replace `xx` with the CUDA architecture and the number of build workers suitable for your machine.
|
||||
请将 `xx` 替换为您机器适用的 CUDA 架构与编译并行 worker 数量。
|
||||
|
||||
#### Step 4: Install utils3d
|
||||
#### 步骤 4:安装 utils3d
|
||||
|
||||
```bash
|
||||
pip install https://github.com/LDYang694/Storages/releases/download/20260430/utils3d-0.0.2-py3-none-any.whl
|
||||
```
|
||||
|
||||
> **Note**: `requirements-hfdemo.txt` is for the Hugging Face Spaces demo (H-series GPU architecture) and may not be compatible with other architectures.
|
||||
> **注意**:`requirements-hfdemo.txt` 适用于 Hugging Face Spaces 演示(H 系列 GPU 架构),可能与其他架构不兼容。
|
||||
|
||||
### Usage
|
||||
### 使用方法
|
||||
|
||||
#### Inference
|
||||
#### 推理
|
||||
|
||||
Generate a GLB mesh from a single image:
|
||||
从单张图像生成 GLB 网格:
|
||||
|
||||
```bash
|
||||
python inference.py --image assets/images/0_img.png --output ./output.glb
|
||||
```
|
||||
|
||||
**Low-VRAM mode** (reduces peak VRAM by loading models on-demand):
|
||||
**低显存模式**(通过按需加载模型降低峰值显存占用):
|
||||
|
||||
```bash
|
||||
python inference.py --image assets/images/0_img.png --output ./output.glb --low_vram
|
||||
```
|
||||
|
||||
By default, the pipeline resolution is **1536** (standard mode) or **1024** (low-VRAM mode). You can override this with `--resolution`:
|
||||
默认流水线分辨率为 **1536**(标准模式)或 **1024**(低显存模式)。可通过 `--resolution` 覆盖:
|
||||
|
||||
```bash
|
||||
# Force 1536 even in low-VRAM mode
|
||||
@@ -107,49 +112,49 @@ python inference.py --image assets/images/0_img.png --output ./output.glb --low_
|
||||
python inference.py --image assets/images/0_img.png --output ./output.glb --resolution 1024
|
||||
```
|
||||
|
||||
**Tip**: If you don't have `flash_attn` installed, you can use PyTorch's built-in SDPA backend instead:
|
||||
**提示**:若未安装 `flash_attn`,可改用 PyTorch 内置的 SDPA 后端:
|
||||
> ```bash
|
||||
> ATTN_BACKEND=sdpa python inference.py --image assets/images/0_img.png --output ./output.glb --low_vram
|
||||
> ```
|
||||
|
||||
### Web Demo
|
||||
### Web 演示
|
||||
|
||||
We provide a Gradio web demo for Pixal3D, which allows you to generate 3D meshes from images interactively.
|
||||
我们提供了 Pixal3D 的 Gradio Web 演示,支持以交互方式从图像生成 3D 网格。
|
||||
|
||||
```bash
|
||||
python app.py
|
||||
```
|
||||
|
||||
Low-VRAM mode is also available for the web demo. The frontend default resolution will automatically switch to 1024 in low-VRAM mode (1536 otherwise), but can be changed manually in the UI.
|
||||
Web 演示同样支持低显存模式。前端默认分辨率在低显存模式下会自动切换为 1024(否则为 1536),也可在 UI 中手动修改。
|
||||
|
||||
```bash
|
||||
python app.py --low_vram
|
||||
# or via environment variable:
|
||||
LOW_VRAM=1 python app.py
|
||||
```
|
||||
## 🔧 Training
|
||||
## 🔧 训练
|
||||
|
||||
We provide the full training codebase for reproducing Pixal3D from scratch.
|
||||
我们提供完整的训练代码库,用于从零复现 Pixal3D。
|
||||
|
||||
### Data Preparation
|
||||
### 数据准备
|
||||
|
||||
Prepare view-aligned O-Voxel data and rendered condition images by following the data toolkit instructions:
|
||||
请按照数据工具包说明,准备视角对齐的 O-Voxel 数据与渲染条件图像:
|
||||
|
||||
> 📂 **[data_toolkit/README.md](data_toolkit/README.md)**
|
||||
|
||||
### Overview
|
||||
### 概览
|
||||
|
||||
Pixal3D is trained as a three-stage cascade, each progressively increasing resolution:
|
||||
Pixal3D 采用三阶段级联训练,每一阶段逐步提高分辨率:
|
||||
|
||||
| Stage | Model | Resolutions | Config Prefix |
|
||||
| 阶段 | 模型 | 分辨率 | 配置前缀 |
|
||||
|-------|-------|-------------|---------------|
|
||||
| 1 | Sparse Structure | 32 → 64 | `ss_flow_img_dit_*_proj_finetune` |
|
||||
| 2 | Shape | 256 → 512 → 1024 | `slat_flow_img2shape_*_proj_finetune` |
|
||||
| 3 | Texture | 256 → 512 → 1024 | `slat_flow_imgshape2tex_*_proj_finetune` |
|
||||
|
||||
All stages use **pixel-aligned projection conditioning** and **view-aligned latents** (2 views by default). Within each stage, start from the lowest resolution and progressively fine-tune to higher resolutions by setting `finetune_ckpt` in the config.
|
||||
所有阶段均使用 **像素对齐投影条件(pixel-aligned projection conditioning)** 与 **视角对齐潜变量(view-aligned latents)**(默认 2 个视角)。在每个阶段内,从最低分辨率开始,通过在配置中设置 `finetune_ckpt`,逐步微调至更高分辨率。
|
||||
|
||||
### Quick Start
|
||||
### 快速开始
|
||||
|
||||
```sh
|
||||
python train.py \
|
||||
@@ -158,20 +163,20 @@ python train.py \
|
||||
--data_dir '<DATA_DIR_JSON>'
|
||||
```
|
||||
|
||||
`--data_dir` is a JSON string describing the dataset layout. Different stages require different keys:
|
||||
`--data_dir` 是描述数据集布局的 JSON 字符串。不同阶段需要不同的键:
|
||||
|
||||
| Stage | Required keys |
|
||||
| 阶段 | 必需键 |
|
||||
|-------|---------------|
|
||||
| Sparse Structure | `base`, `ss_latent`, `render_cond` |
|
||||
| Shape | `base`, `shape_latent`, `render_cond` |
|
||||
| Texture | `base`, `shape_latent`, `pbr_latent`, `render_cond` |
|
||||
|
||||
### Example: Training All Three Stages
|
||||
### 示例:训练全部三个阶段
|
||||
|
||||
Below we show the full training sequence using ObjaverseXL as an example. Each higher-resolution step requires updating `finetune_ckpt` in its config JSON to point to the previous checkpoint.
|
||||
以下以 ObjaverseXL 为例展示完整训练流程。每个更高分辨率步骤都需要在其配置 JSON 中更新 `finetune_ckpt`,使其指向上一个检查点。
|
||||
|
||||
<details>
|
||||
<summary><b>Stage 1: Sparse Structure (32 → 64)</b></summary>
|
||||
<summary><b>阶段 1:稀疏结构(32 → 64)</b></summary>
|
||||
|
||||
```sh
|
||||
# Resolution 32
|
||||
@@ -189,7 +194,7 @@ python train.py \
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Stage 2: Shape (256 → 512 → 1024)</b></summary>
|
||||
<summary><b>阶段 2:形状(256 → 512 → 1024)</b></summary>
|
||||
|
||||
```sh
|
||||
# Resolution 256
|
||||
@@ -213,7 +218,7 @@ python train.py \
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Stage 3: Texture (256 → 512 → 1024)</b></summary>
|
||||
<summary><b>阶段 3:纹理(256 → 512 → 1024)</b></summary>
|
||||
|
||||
```sh
|
||||
# Resolution 256
|
||||
@@ -236,52 +241,52 @@ python train.py \
|
||||
```
|
||||
</details>
|
||||
|
||||
### Additional Options
|
||||
### 其他选项
|
||||
|
||||
<details>
|
||||
<summary><b>All command-line arguments</b></summary>
|
||||
<summary><b>所有命令行参数</b></summary>
|
||||
|
||||
| Argument | Description | Default |
|
||||
| 参数 | 说明 | 默认值 |
|
||||
|----------|-------------|---------|
|
||||
| `--config` | Config JSON path | *required* |
|
||||
| `--output_dir` | Output directory | *required* |
|
||||
| `--data_dir` | Dataset JSON string | `./data/` |
|
||||
| `--load_dir` | Checkpoint load directory | `output_dir` |
|
||||
| `--ckpt` | Resume from step | `latest` |
|
||||
| `--auto_retry` | Retries on failure | `3` |
|
||||
| `--tryrun` | Dry run | `false` |
|
||||
| `--profile` | Profiling | `false` |
|
||||
| `--num_nodes` | Number of nodes | `1` |
|
||||
| `--node_rank` | Current node rank | `0` |
|
||||
| `--num_gpus` | GPUs per node | all |
|
||||
| `--master_addr` | Master address | `localhost` |
|
||||
| `--master_port` | Master port | `12666` |
|
||||
| `--use_wandb` | Enable W&B logging | `false` |
|
||||
| `--wandb_project` | W&B project | `trellis2-training` |
|
||||
| `--wandb_name` | W&B run name | basename of `output_dir` |
|
||||
| `--wandb_id` | W&B run ID (resume) | — |
|
||||
| `--config` | 配置文件 JSON 路径 | *必填* |
|
||||
| `--output_dir` | 输出目录 | *必填* |
|
||||
| `--data_dir` | 数据集 JSON 字符串 | `./data/` |
|
||||
| `--load_dir` | 检查点加载目录 | `output_dir` |
|
||||
| `--ckpt` | 从指定步骤恢复 | `latest` |
|
||||
| `--auto_retry` | 失败重试次数 | `3` |
|
||||
| `--tryrun` | 试运行 | `false` |
|
||||
| `--profile` | 性能分析 | `false` |
|
||||
| `--num_nodes` | 节点数量 | `1` |
|
||||
| `--node_rank` | 当前节点 rank | `0` |
|
||||
| `--num_gpus` | 每节点 GPU 数量 | all |
|
||||
| `--master_addr` | 主节点地址 | `localhost` |
|
||||
| `--master_port` | 主节点端口 | `12666` |
|
||||
| `--use_wandb` | 启用 W&B 日志记录 | `false` |
|
||||
| `--wandb_project` | W&B 项目 | `trellis2-training` |
|
||||
| `--wandb_name` | W&B 运行名称 | `output_dir` 的 basename |
|
||||
| `--wandb_id` | W&B 运行 ID(恢复) | — |
|
||||
|
||||
</details>
|
||||
|
||||
## 🌐 Community Projects
|
||||
## 🌐 社区项目
|
||||
|
||||
We thank the community for building extensions and deployment guides for Pixal3D!
|
||||
我们感谢社区为 Pixal3D 构建扩展与部署指南!
|
||||
|
||||
- [Pixal3D-ComfyUI](https://github.com/Saganaki22/Pixal3D-ComfyUI) — ComfyUI integration with deployment guides for Windows, WSL, and more.
|
||||
- [Pixal3D-ComfyUI](https://github.com/Saganaki22/Pixal3D-ComfyUI) — ComfyUI 集成,包含适用于 Windows、WSL 等环境的部署指南。
|
||||
|
||||
## 🤗 Acknowledgements
|
||||
## 🤗 致谢
|
||||
|
||||
This project is heavily built upon [Trellis.2](https://github.com/microsoft/TRELLIS.2) and [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2). We sincerely thank the authors for their outstanding work on scalable 3D generation , which serves as the foundation of our codebase and model architecture.
|
||||
本项目大量借鉴了 [Trellis.2](https://github.com/microsoft/TRELLIS.2) 与 [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2).。我们衷心感谢作者在可扩展 3D 生成(scalable 3D generation)方面的杰出工作,其为我们的代码库与模型架构奠定了基础。
|
||||
|
||||
We also thank the following repos for their great contributions:
|
||||
我们也感谢以下仓库的杰出贡献:
|
||||
|
||||
- [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2)
|
||||
- [Trellis](https://github.com/microsoft/TRELLIS)
|
||||
- [Trellis.2](https://github.com/microsoft/TRELLIS.2)
|
||||
|
||||
## 📄 Citation
|
||||
## 📄 引用
|
||||
|
||||
If you find this work useful, please consider citing:
|
||||
若您觉得本工作有帮助,欢迎引用:
|
||||
|
||||
```bibtex
|
||||
@article{li2026pixal3d,
|
||||
@@ -292,7 +297,6 @@ If you find this work useful, please consider citing:
|
||||
}
|
||||
```
|
||||
|
||||
## 📜 License
|
||||
|
||||
This project is released under the [MIT License](LICENSE). The third-party components included in this project remain licensed under their respective original terms; see [NOTICE](NOTICE) for the full list of dependencies and their licenses.
|
||||
## 📜 许可证
|
||||
|
||||
本项目基于 [MIT License](LICENSE) 发布。本项目中包含的第三方组件仍按其各自原始条款授权;完整依赖列表及其许可证请参阅 [NOTICE](NOTICE)。
|
||||
|
||||
Reference in New Issue
Block a user