diff --git a/README.md b/README.md index 09d7884..9ad8116 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,20 @@ + +> [!NOTE] +> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。 +> [English](./README.en.md) · [原始项目](https://github.com/TencentARC/Pixal3D) · [上游 README](https://github.com/TencentARC/Pixal3D/blob/HEAD/README.md) +> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
-# Pixal3D: Pixel-Aligned 3D Generation from Images +# Pixal3D：从图像生成像素对齐的 3D（Pixel-Aligned 3D Generation from Images）

SIGGRAPH 2026

[Dong-Yang Li](https://ldyang694.github.io/)¹ · [Wang Zhao](https://thuzhaowang.github.io/)²* · [Yuxin Chen](https://orcid.org/0000-0002-7854-1072)² · [Wenbo Hu](https://wbhu.github.io/)² · [Meng-Hao Guo](https://menghaoguo.github.io/)¹ · [Fang-Lue Zhang](https://fanglue.github.io/)³ · [Ying Shan](https://www.linkedin.com/in/YingShanProfile)² · [Shi-Min Hu](https://cg.cs.tsinghua.edu.cn/shimin.htm)¹✉ -¹Tsinghua University (BNRist)    ²Tencent ARC Lab    ³Victoria University of Wellington +¹清华大学 (BNRist)    ²腾讯 ARC 实验室    ³惠灵顿维多利亚大学 -*Project lead    ✉Corresponding author +*项目负责人    ✉通讯作者
@@ -22,82 +27,82 @@
- Teaser image of Pixal3D + Pixal3D 预览图
-**Pixal3D** generates high-fidelity 3D assets from a single image. Unlike previous methods that loosely inject image features via attention, Pixal3D explicitly lifts pixel features into 3D through back-projection, establishing direct pixel-to-3D correspondences. This enables near-reconstruction-level fidelity with detailed geometry and PBR textures. +**Pixal3D** 可从单张图像生成高保真 3D 资产。与此前通过注意力机制松散注入图像特征的方法不同，Pixal3D 通过反投影（back-projection）将像素特征显式提升到 3D 空间，建立直接的像素到 3D 对应关系。这使得其能够达到接近重建级别的保真度，并具备精细的几何与 PBR 纹理。 --- -## ✨ News +## ✨ 动态 -- **May 2026**: Release training code and data preparation toolkit. 🔧 -- **May 2026**: Release the improved version based on [Trellis.2](https://github.com/microsoft/TRELLIS.2) backbone. 💪 -- **May 2026**: Release inference code and online demo. 🤗 -- **Apr 2026**: Our paper is accepted to SIGGRAPH 2026! 🎉 +- **2026 年 5 月**：发布训练代码与数据准备工具包。🔧 +- **2026 年 5 月**：发布基于 [Trellis.2](https://github.com/microsoft/TRELLIS.2) 骨干网络的改进版本。💪 +- **2026 年 5 月**：发布推理代码与在线演示。🤗 +- **2026 年 4 月**：论文被 SIGGRAPH 2026 接收！🎉 -## 📌 Branches +## 📌 分支 -| Branch | Description | +| 分支 | 说明 | |--------|-------------| -| `main` | **Latest version** — improved implementation based on [Trellis.2](https://github.com/microsoft/TRELLIS.2) backbone with better performance. | -| `paper` | **Paper version** — original implementation based on [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2), corresponding to results reported in our SIGGRAPH 2026 paper. | +| `main` | **最新版本**——基于 [Trellis.2](https://github.com/microsoft/TRELLIS.2) 骨干网络的改进实现，性能更优。 | +| `paper` | **论文版本**——基于 [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2), 的原始实现，对应我们 SIGGRAPH 2026 论文中报告的结果。 | -> If you want to reproduce the results in our paper, please switch to the `paper` branch. +> 若要复现我们论文中的结果，请切换到 `paper` 分支。 -## 🎮 Try It Online +## 🎮 在线体验 -You can try Pixal3D directly in your browser without any installation via our Hugging Face Gradio demo: +您无需安装任何软件，即可通过我们的 Hugging Face Gradio 演示在浏览器中直接体验 Pixal3D： -👉 [**Launch Demo**](https://huggingface.co/spaces/TencentARC/Pixal3D) +👉 [**启动演示**](https://huggingface.co/spaces/TencentARC/Pixal3D) -## 🚀 Getting Started +## 🚀 快速开始 -### Installation +### 安装 -#### Step 1: Follow TRELLIS.2 Installation +#### 步骤 1：按 TRELLIS.2 安装指南配置 -Please first follow the installation guide of [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) to set up the base environment. +请先按照 [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) 的安装指南配置基础环境。 -#### Step 2: Install Additional Dependencies +#### 步骤 2：安装额外依赖 ```bash pip install -r requirements.txt ``` -#### Step 3: Install natten +#### 步骤 3：安装 natten ```bash NATTEN_CUDA_ARCH="xx" NATTEN_N_WORKERS=xx pip install natten==0.21.0 --no-build-isolation ``` -Please replace `xx` with the CUDA architecture and the number of build workers suitable for your machine. +请将 `xx` 替换为您机器适用的 CUDA 架构与编译并行 worker 数量。 -#### Step 4: Install utils3d +#### 步骤 4：安装 utils3d ```bash pip install https://github.com/LDYang694/Storages/releases/download/20260430/utils3d-0.0.2-py3-none-any.whl ``` -> **Note**: `requirements-hfdemo.txt` is for the Hugging Face Spaces demo (H-series GPU architecture) and may not be compatible with other architectures. +> **注意**：`requirements-hfdemo.txt` 适用于 Hugging Face Spaces 演示（H 系列 GPU 架构），可能与其他架构不兼容。 -### Usage +### 使用方法 -#### Inference +#### 推理 -Generate a GLB mesh from a single image: +从单张图像生成 GLB 网格： ```bash python inference.py --image assets/images/0_img.png --output ./output.glb ``` -**Low-VRAM mode** (reduces peak VRAM by loading models on-demand): +**低显存模式**（通过按需加载模型降低峰值显存占用）： ```bash python inference.py --image assets/images/0_img.png --output ./output.glb --low_vram ``` -By default, the pipeline resolution is **1536** (standard mode) or **1024** (low-VRAM mode). You can override this with `--resolution`: +默认流水线分辨率为 **1536**（标准模式）或 **1024**（低显存模式）。可通过 `--resolution` 覆盖： ```bash # Force 1536 even in low-VRAM mode @@ -107,49 +112,49 @@ python inference.py --image assets/images/0_img.png --output ./output.glb --low_ python inference.py --image assets/images/0_img.png --output ./output.glb --resolution 1024 ``` -**Tip**: If you don't have `flash_attn` installed, you can use PyTorch's built-in SDPA backend instead: +**提示**：若未安装 `flash_attn`，可改用 PyTorch 内置的 SDPA 后端： > ```bash > ATTN_BACKEND=sdpa python inference.py --image assets/images/0_img.png --output ./output.glb --low_vram > ``` -### Web Demo +### Web 演示 -We provide a Gradio web demo for Pixal3D, which allows you to generate 3D meshes from images interactively. +我们提供了 Pixal3D 的 Gradio Web 演示，支持以交互方式从图像生成 3D 网格。 ```bash python app.py ``` -Low-VRAM mode is also available for the web demo. The frontend default resolution will automatically switch to 1024 in low-VRAM mode (1536 otherwise), but can be changed manually in the UI. +Web 演示同样支持低显存模式。前端默认分辨率在低显存模式下会自动切换为 1024（否则为 1536），也可在 UI 中手动修改。 ```bash python app.py --low_vram # or via environment variable: LOW_VRAM=1 python app.py ``` -## 🔧 Training +## 🔧 训练 -We provide the full training codebase for reproducing Pixal3D from scratch. +我们提供完整的训练代码库，用于从零复现 Pixal3D。 -### Data Preparation +### 数据准备 -Prepare view-aligned O-Voxel data and rendered condition images by following the data toolkit instructions: +请按照数据工具包说明，准备视角对齐的 O-Voxel 数据与渲染条件图像： > 📂 **[data_toolkit/README.md](data_toolkit/README.md)** -### Overview +### 概览 -Pixal3D is trained as a three-stage cascade, each progressively increasing resolution: +Pixal3D 采用三阶段级联训练，每一阶段逐步提高分辨率： -| Stage | Model | Resolutions | Config Prefix | +| 阶段 | 模型 | 分辨率 | 配置前缀 | |-------|-------|-------------|---------------| | 1 | Sparse Structure | 32 → 64 | `ss_flow_img_dit_*_proj_finetune` | | 2 | Shape | 256 → 512 → 1024 | `slat_flow_img2shape_*_proj_finetune` | | 3 | Texture | 256 → 512 → 1024 | `slat_flow_imgshape2tex_*_proj_finetune` | -All stages use **pixel-aligned projection conditioning** and **view-aligned latents** (2 views by default). Within each stage, start from the lowest resolution and progressively fine-tune to higher resolutions by setting `finetune_ckpt` in the config. +所有阶段均使用 **像素对齐投影条件（pixel-aligned projection conditioning）** 与 **视角对齐潜变量（view-aligned latents）**（默认 2 个视角）。在每个阶段内，从最低分辨率开始，通过在配置中设置 `finetune_ckpt`，逐步微调至更高分辨率。 -### Quick Start +### 快速开始 ```sh python train.py \ @@ -158,20 +163,20 @@ python train.py \ --data_dir '' ``` -`--data_dir` is a JSON string describing the dataset layout. Different stages require different keys: +`--data_dir` 是描述数据集布局的 JSON 字符串。不同阶段需要不同的键： -| Stage | Required keys | +| 阶段 | 必需键 | |-------|---------------| | Sparse Structure | `base`, `ss_latent`, `render_cond` | | Shape | `base`, `shape_latent`, `render_cond` | | Texture | `base`, `shape_latent`, `pbr_latent`, `render_cond` | -### Example: Training All Three Stages +### 示例：训练全部三个阶段 -Below we show the full training sequence using ObjaverseXL as an example. Each higher-resolution step requires updating `finetune_ckpt` in its config JSON to point to the previous checkpoint. +以下以 ObjaverseXL 为例展示完整训练流程。每个更高分辨率步骤都需要在其配置 JSON 中更新 `finetune_ckpt`，使其指向上一个检查点。
-Stage 1: Sparse Structure (32 → 64) +阶段 1：稀疏结构（32 → 64） ```sh # Resolution 32 @@ -189,7 +194,7 @@ python train.py \
-Stage 2: Shape (256 → 512 → 1024) +阶段 2：形状（256 → 512 → 1024） ```sh # Resolution 256 @@ -213,7 +218,7 @@ python train.py \
-Stage 3: Texture (256 → 512 → 1024) +阶段 3：纹理（256 → 512 → 1024） ```sh # Resolution 256 @@ -236,52 +241,52 @@ python train.py \ ```
-### Additional Options +### 其他选项
-All command-line arguments +所有命令行参数 -| Argument | Description | Default | +| 参数 | 说明 | 默认值 | |----------|-------------|---------| -| `--config` | Config JSON path | *required* | -| `--output_dir` | Output directory | *required* | -| `--data_dir` | Dataset JSON string | `./data/` | -| `--load_dir` | Checkpoint load directory | `output_dir` | -| `--ckpt` | Resume from step | `latest` | -| `--auto_retry` | Retries on failure | `3` | -| `--tryrun` | Dry run | `false` | -| `--profile` | Profiling | `false` | -| `--num_nodes` | Number of nodes | `1` | -| `--node_rank` | Current node rank | `0` | -| `--num_gpus` | GPUs per node | all | -| `--master_addr` | Master address | `localhost` | -| `--master_port` | Master port | `12666` | -| `--use_wandb` | Enable W&B logging | `false` | -| `--wandb_project` | W&B project | `trellis2-training` | -| `--wandb_name` | W&B run name | basename of `output_dir` | -| `--wandb_id` | W&B run ID (resume) | — | +| `--config` | 配置文件 JSON 路径 | *必填* | +| `--output_dir` | 输出目录 | *必填* | +| `--data_dir` | 数据集 JSON 字符串 | `./data/` | +| `--load_dir` | 检查点加载目录 | `output_dir` | +| `--ckpt` | 从指定步骤恢复 | `latest` | +| `--auto_retry` | 失败重试次数 | `3` | +| `--tryrun` | 试运行 | `false` | +| `--profile` | 性能分析 | `false` | +| `--num_nodes` | 节点数量 | `1` | +| `--node_rank` | 当前节点 rank | `0` | +| `--num_gpus` | 每节点 GPU 数量 | all | +| `--master_addr` | 主节点地址 | `localhost` | +| `--master_port` | 主节点端口 | `12666` | +| `--use_wandb` | 启用 W&B 日志记录 | `false` | +| `--wandb_project` | W&B 项目 | `trellis2-training` | +| `--wandb_name` | W&B 运行名称 | `output_dir` 的 basename | +| `--wandb_id` | W&B 运行 ID（恢复） | — |
-## 🌐 Community Projects +## 🌐 社区项目 -We thank the community for building extensions and deployment guides for Pixal3D! +我们感谢社区为 Pixal3D 构建扩展与部署指南！ -- [Pixal3D-ComfyUI](https://github.com/Saganaki22/Pixal3D-ComfyUI) — ComfyUI integration with deployment guides for Windows, WSL, and more. +- [Pixal3D-ComfyUI](https://github.com/Saganaki22/Pixal3D-ComfyUI) — ComfyUI 集成，包含适用于 Windows、WSL 等环境的部署指南。 -## 🤗 Acknowledgements +## 🤗 致谢 -This project is heavily built upon [Trellis.2](https://github.com/microsoft/TRELLIS.2) and [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2). We sincerely thank the authors for their outstanding work on scalable 3D generation , which serves as the foundation of our codebase and model architecture. +本项目大量借鉴了 [Trellis.2](https://github.com/microsoft/TRELLIS.2) 与 [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2).。我们衷心感谢作者在可扩展 3D 生成（scalable 3D generation）方面的杰出工作，其为我们的代码库与模型架构奠定了基础。 -We also thank the following repos for their great contributions: +我们也感谢以下仓库的杰出贡献： - [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2) - [Trellis](https://github.com/microsoft/TRELLIS) - [Trellis.2](https://github.com/microsoft/TRELLIS.2) -## 📄 Citation +## 📄 引用 -If you find this work useful, please consider citing: +若您觉得本工作有帮助，欢迎引用： ```bibtex @article{li2026pixal3d, @@ -292,7 +297,6 @@ If you find this work useful, please consider citing: } ``` -## 📜 License - -This project is released under the [MIT License](LICENSE). The third-party components included in this project remain licensed under their respective original terms; see [NOTICE](NOTICE) for the full list of dependencies and their licenses. +## 📜 许可证 +本项目基于 [MIT License](LICENSE) 发布。本项目中包含的第三方组件仍按其各自原始条款授权；完整依赖列表及其许可证请参阅 [NOTICE](NOTICE)。