wehub-resource-sync
181 lines
11 KiB
Markdown
181 lines
11 KiB
Markdown
# SWIFT Installation
|
|
|
|
## Wheel Packages Installation
|
|
|
|
You can install it using pip:
|
|
|
|
```shell
|
|
# recommend
|
|
pip install 'ms-swift' -U
|
|
# Install additional Megatron dependencies
|
|
pip install 'ms-swift[megatron]' -U
|
|
# Install additional evaluation dependencies
|
|
pip install 'ms-swift[eval]' -U
|
|
# Full capabilities
|
|
pip install 'ms-swift[all]' -U
|
|
|
|
# Using uv
|
|
pip install uv
|
|
uv pip install 'ms-swift' --torch-backend=auto
|
|
```
|
|
|
|
## Source Code Installation
|
|
The current main branch is for swift 4.x version.
|
|
```shell
|
|
# pip install git+https://github.com/modelscope/ms-swift.git
|
|
|
|
# Full capabilities
|
|
# pip install "ms-swift[all]@git+https://github.com/modelscope/ms-swift.git"
|
|
|
|
git clone https://github.com/modelscope/ms-swift.git
|
|
cd ms-swift
|
|
pip install -e .
|
|
|
|
# Full capabilities
|
|
# pip install -e '.[all]'
|
|
```
|
|
|
|
Installing swift 3.x:
|
|
```shell
|
|
# pip install "git+https://github.com/modelscope/ms-swift.git@release/3.12"
|
|
|
|
# Full capabilities
|
|
# pip install "ms-swift[all]@git+https://github.com/modelscope/ms-swift.git@release/3.12"
|
|
|
|
git clone -b release/3.12 https://github.com/modelscope/ms-swift.git
|
|
cd ms-swift
|
|
pip install -e .
|
|
|
|
# Using uv
|
|
uv pip install -e . --torch-backend=auto
|
|
|
|
# Full capabilities
|
|
# pip install -e '.[all]'
|
|
```
|
|
|
|
## Mirror
|
|
|
|
You can check Docker [here](https://github.com/modelscope/modelscope/blob/build_swift_image/docker/build_image.py#L392).
|
|
```
|
|
# swift4.3.2
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda13.0.3-py312-torch2.11.0-vllm0.23.0-modelscope1.37.1-swift4.3.2
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda13.0.3-py312-torch2.11.0-vllm0.23.0-modelscope1.37.1-swift4.3.2
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda13.0.3-py312-torch2.11.0-vllm0.23.0-modelscope1.37.1-swift4.3.2
|
|
|
|
# swift4.2.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda13.0.3-py312-torch2.11.0-vllm0.21.0-modelscope1.36.3-swift4.2.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda13.0.3-py312-torch2.11.0-vllm0.21.0-modelscope1.36.3-swift4.2.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda13.0.3-py312-torch2.11.0-vllm0.21.0-modelscope1.36.3-swift4.2.3
|
|
|
|
# swift4.1.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3
|
|
|
|
# swift4.0.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.10.0-vllm0.17.1-modelscope1.34.0-swift4.0.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.10.0-vllm0.17.1-modelscope1.34.0-swift4.0.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.10.0-vllm0.17.1-modelscope1.34.0-swift4.0.3
|
|
```
|
|
|
|
<details><summary>Historical Mirrors</summary>
|
|
|
|
```
|
|
# swift3.12.5
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.9.0-vllm0.13.0-modelscope1.33.0-swift3.12.5
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.9.0-vllm0.13.0-modelscope1.33.0-swift3.12.5
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.9.0-vllm0.13.0-modelscope1.33.0-swift3.12.5
|
|
|
|
# swift3.11.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py311-torch2.8.0-vllm0.11.0-modelscope1.32.0-swift3.11.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py311-torch2.8.0-vllm0.11.0-modelscope1.32.0-swift3.11.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py311-torch2.8.0-vllm0.11.0-modelscope1.32.0-swift3.11.3
|
|
|
|
# swift3.10.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
|
|
|
|
# swift3.9.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.9.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.9.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.9.3
|
|
|
|
# swift3.8.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.1.1-modelscope1.29.2-swift3.8.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.1.1-modelscope1.29.2-swift3.8.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.1.1-modelscope1.29.2-swift3.8.3
|
|
|
|
# swift3.7.2
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.0-modelscope1.28.2-swift3.7.2
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.0-modelscope1.28.2-swift3.7.2
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.0-modelscope1.28.2-swift3.7.2
|
|
|
|
# swift3.6.4
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.28.1-swift3.6.4
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.28.1-swift3.6.4
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.28.1-swift3.6.4
|
|
|
|
# swift3.5.3
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.27.1-swift3.5.3
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.27.1-swift3.5.3
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.27.1-swift3.5.3
|
|
|
|
# swift3.4.1.post1
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.5.post1-modelscope1.26.0-swift3.4.1.post1
|
|
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.5.post1-modelscope1.26.0-swift3.4.1.post1
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.5.post1-modelscope1.26.0-swift3.4.1.post1
|
|
|
|
# swift3.3.0.post1
|
|
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.3-modelscope1.25.0-swift3.3.0.post1
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.3-modelscope1.25.0-swift3.3.0.post1
|
|
|
|
# swift3.2.2
|
|
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.5.1-modelscope1.25.0-swift3.2.2
|
|
```
|
|
</details>
|
|
|
|
More images can be found [here](https://modelscope.cn/docs/intro/environment-setup#%E6%9C%80%E6%96%B0%E9%95%9C%E5%83%8F).
|
|
|
|
## Supported Hardware
|
|
|
|
| Hardware Environment | Remarks |
|
|
| -------------------- | ------------------------------------------------------ |
|
|
| A10/A100/H100 | |
|
|
| RTX 20/30/40 Series | |
|
|
| T4/V100 | Some models may encounter NAN |
|
|
| Ascend NPU | Some models may encounter NAN or unsupported operators |
|
|
| MPS | Refer to [issue 4572](https://github.com/modelscope/ms-swift/issues/4572) |
|
|
| CPU | |
|
|
|
|
|
|
## Running Environment
|
|
|
|
| | Range | Recommended | Notes |
|
|
|--------------|--------------|---------------------|-------------------------------------------|
|
|
| python | >=3.10 | 3.12 | |
|
|
| cuda | | cuda12.8/13.0 | No need to install if using CPU, NPU, MPS |
|
|
| torch | >=2.0 | 2.8.0/2.11.0 | |
|
|
| transformers | >=4.33 | 4.57.6/5.12.1 | |
|
|
| modelscope | >=1.23 | | |
|
|
| datasets | >=3.0,<4.8.5 | 3.6.0/4.8.4 | |
|
|
| peft | >=0.11,<0.20 | | |
|
|
| flash_attn | | 2.8.3/4.0.0b15 | |
|
|
| trl | >=0.15,<1.0 | 0.29.1 | RLHF |
|
|
| deepspeed | >=0.14 | 0.18.9 | Training |
|
|
| vllm | >=0.5.1 | 0.11.0/0.23.0 | Inference/Deployment |
|
|
| sglang | >=0.4.6 | | Inference/Deployment |
|
|
| evalscope | >=1.0 | | Evaluation |
|
|
| gradio | | 5.32.1 | Web-UI/App |
|
|
|
|
|
|
For more optional dependencies, you can refer to [here](https://github.com/modelscope/ms-swift/blob/main/requirements/install_all.sh).
|
|
|
|
## Notebook Environment
|
|
|
|
Most models that Swift supports for training can be used on A10 GPUs. Users can take advantage of the free GPU resources offered by ModelScope:
|
|
|
|
1. Visit the [ModelScope](https://www.modelscope.cn) official website and log in.
|
|
2. Click on `My Notebook` on the left and start a free GPU instance.
|
|
3. Enjoy utilizing the A10 GPU resources.
|