paddlepaddle--paddlenlp/llm/server/docs/static_models.md

## 静态图模型下载支持

* 静态图模型已支持 Deepseek 系列、Qwen 系列、llama 系列等模型 详细支持列表如下：

### DeepSeekV2

|模型名称|静态图下载 model_name|
|:------|:-:|
| deepseek-ai/DeepSeek-V2-Chat              | 🚧 |
| deepseek-ai/DeepSeek-V2-Lite-Chat         | 🚧 |

### DeepSeekV3

|模型名称|静态图下载 model_name|
|:------|:-:|
| deepseek-ai/DeepSeek-V3                   | 🚧 |

### DeepSeekR1
部署硬件要求：
- 除MTP模型，Fp8模型之外支持的最低版本是SM80 (机器：A100 / A800) 要求CUDA 11.8 以上
- DeepSeek-R1-MTP 与 Fp8 模型 支持的最低版本是SM90 (机器：H800) 要求CUDA 12.4 以上
- Append-Attn：
  - 支持的最低版本是SM80 (机器：A100 / A800)
  - 要求CUDA 11.8 以上
- Block-Attn:
  - 支持的最低版本是SM70 (机器：V100)
  - 要求CUDA 11.8 以上

|模型名称|精度|MTP|节点数|静态图下载 model_name|
|:------|:-:|:-:|:-:|:-:|
| deepseek-ai/DeepSeek-R1  |weight_only_int4|否|1| deepseek-ai/DeepSeek-R1/weight_only_int4 |
| deepseek-ai/DeepSeek-R1  |weight_only_int4|是|1| deepseek-ai/DeepSeek-R1-MTP/weight_only_int4 |
| deepseek-ai/DeepSeek-R1  |weight_only_int8|否|2| deepseek-ai/DeepSeek-R1-2nodes/weight_only_int8 |
| deepseek-ai/DeepSeek-R1  |weight_only_int8|是|2| deepseek-ai/DeepSeek-R1-MTP-2nodes/weight_only_int8 |
| deepseek-ai/DeepSeek-R1  |a8w8_fp8|否|2| deepseek-ai/DeepSeek-R1-2nodes/a8w8_fp8|
| deepseek-ai/DeepSeek-R1  |a8w8_fp8|是|2| deepseek-ai/DeepSeek-R1-MTP-2nodes/a8w8_fp8|
| deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B |weight_only_int8|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/weight_only_int8,deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B-Block-Attn/weight_only_int8  |
| deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B |fp16|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B-Block-Attn/float16|
| deepseek-ai/DeepSeek-R1-Distill-Qwen-7B   |weight_only_int8|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-7B/weight_only_int8,deepseek-ai/DeepSeek-R1-Distill-Qwen-7B-Block-Attn/weight_only_int8 |
| deepseek-ai/DeepSeek-R1-Distill-Qwen-7B   |fp16|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-7B-Block-Attn/float16 |
| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B  |weight_only_int8|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/weight_only_int8,deepseek-ai/DeepSeek-R1-Distill-Qwen-14B-Block-Attn/weight_only_int8 |
| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B  |fp16|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-14B-Block-Attn/float16 |
| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B  |weight_only_int8|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-32B/weight_only_int8,deepseek-ai/DeepSeek-R1-Distill-Qwen-32B-Block-Attn/weight_only_int8 |
| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B  |fp16|-|-|deepseek-ai/DeepSeek-R1-Distill-Qwen-32B-Block-Attn/float16 |
| deepseek-ai/DeepSeek-R1-Distill-Llama-8B  |weight_only_int8|-|-|deepseek-ai/DeepSeek-R1-Distill-Llama-8B/weight_only_int8,deepseek-ai/DeepSeek-R1-Distill-Llama-8B-Block-Attn/weight_only_int8 |
| deepseek-ai/DeepSeek-R1-Distill-Llama-8B  |fp16|-|-|deepseek-ai/DeepSeek-R1-Distill-Llama-8B-Block-Attn/float16 |
| deepseek-ai/DeepSeek-R1-Distill-Llama-70B |weight_only_int8|-|-|deepseek-ai/DeepSeek-R1-Distill-Llama-70B/weight_only_int8 |

### QWQ

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/QwQ-32B | Qwen/QwQ-32B-Append-Attn/weight_only_int8 |
| Qwen/QwQ-32B-Preview | 🚧 |

### LLaMA

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| facebook/llama-7b | 🚧 |
| facebook/llama-13b | 🚧 |
| facebook/llama-30b | 🚧 |
| facebook/llama-65b | 🚧 |

### Llama2

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| meta-llama/Llama-2-7b | 🚧 |
| meta-llama/Llama-2-7b-chat | 🚧 |
| meta-llama/Llama-2-13b | 🚧 |
| meta-llama/Llama-2-13b-chat | 🚧 |
| meta-llama/Llama-2-70b | 🚧 |
| meta-llama/Llama-2-70b-chat | 🚧 |

### Llama3

部署硬件要求：
- Append-Attn：
  - 支持的最低版本是SM80 (机器：A100 / A800)
  - 要求CUDA 11.8 以上
- Block-Attn:
  - 支持的最低版本是SM70 (机器：V100)
  - 要求CUDA 11.8 以上


| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| meta-llama/Meta-Llama-3-8B | 🚧 |
| meta-llama/Meta-Llama-3-8B-Instruct |meta-llama/Meta-Llama-3-8B-Instruct-Append-Attn/bfloat16,meta-llama/Meta-Llama-3-8B-Instruct-Block-Attn/float16|
| meta-llama/Meta-Llama-3-70B | 🚧 |
| meta-llama/Meta-Llama-3-70B-Instruct | 🚧 |

### Llama3.1

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| meta-llama/Meta-Llama-3.1-8B | 🚧 |
| meta-llama/Meta-Llama-3.1-8B-Instruct | 🚧 |
| meta-llama/Meta-Llama-3.1-70B | 🚧 |
| meta-llama/Meta-Llama-3.1-70B-Instruct | 🚧 |
| meta-llama/Meta-Llama-3.1-405B | 🚧 |
| meta-llama/Meta-Llama-3.1-405B-Instruct | 🚧 |
| meta-llama/Llama-Guard-3-8B | 🚧 |

### Llama3.2

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| meta-llama/Llama-3.2-1B | 🚧 |
| meta-llama/Llama-3.2-1B-Instruct | 🚧 |
| meta-llama/Llama-3.2-3B | 🚧 |
| meta-llama/Llama-3.2-3B-Instruct | 🚧 |
| meta-llama/Llama-Guard-3-1B | 🚧 |

### Llama3.3

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| meta-llama/Llama-3.3-70B-Instruct | 🚧 |


### Mixtral

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| mistralai/Mixtral-8x7B-Instruct-v0.1 | 🚧 |

### Qwen

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| qwen/qwen-7b | 🚧 |
| qwen/qwen-7b-chat | 🚧 |
| qwen/qwen-14b | 🚧 |
| qwen/qwen-14b-chat | 🚧 |
| qwen/qwen-72b | 🚧 |
| qwen/qwen-72b-chat | 🚧 |

### Qwen1.5
部署硬件要求：
- Block-Attn:
  - 支持的最低版本是SM70 (机器：V100)
  - 要求CUDA 11.8 以上

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/Qwen1.5-0.5B | Qwen/Qwen1.5-0.5B-Block-Attn/bfloat16,Qwen/Qwen1.5-0.5B-Block-Attn/float16 |
| Qwen/Qwen1.5-0.5B-Chat | 🚧 |
| Qwen/Qwen1.5-1.8B | 🚧 |
| Qwen/Qwen1.5-1.8B-Chat | 🚧 |
| Qwen/Qwen1.5-4B | 🚧 |
| Qwen/Qwen1.5-4B-Chat | 🚧 |
| Qwen/Qwen1.5-7B | 🚧 |
| Qwen/Qwen1.5-7B-Chat | 🚧 |
| Qwen/Qwen1.5-14B | 🚧 |
| Qwen/Qwen1.5-14B-Chat | 🚧 |
| Qwen/Qwen1.5-32B | 🚧 |
| Qwen/Qwen1.5-32B-Chat | 🚧 |
| Qwen/Qwen1.5-72B | 🚧 |
| Qwen/Qwen1.5-72B-Chat | 🚧 |
| Qwen/Qwen1.5-110B | 🚧 |
| Qwen/Qwen1.5-110B-Chat | 🚧 |
| Qwen/Qwen1.5-MoE-A2.7B | 🚧 |
| Qwen/Qwen1.5-MoE-A2.7B-Chat | 🚧 |

### Qwen2
部署硬件要求：
- Append-Attn：
  - 支持的最低版本是SM80 (机器：A100 / A800)
  - 要求CUDA 11.8 以上
- Block-Attn:
  - 支持的最低版本是SM70 (机器：V100)
  - 要求CUDA 11.8 以上

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/Qwen2-0.5B | 🚧 |
| Qwen/Qwen2-0.5B-Instruct | 🚧 |
| Qwen/Qwen2-1.5B | 🚧 |
| Qwen/Qwen2-1.5B-Instruct | Qwen/Qwen2-1.5B-Instruct-Append-Attn/bfloat16, Qwen/Qwen2-1.5B-Instruct-Block-Attn/float16|
| Qwen/Qwen2-7B | 🚧 |
| Qwen/Qwen2-7B-Instruct | 🚧 |
| Qwen/Qwen2-72B | 🚧 |
| Qwen/Qwen2-72B-Instruct | 🚧 |
| Qwen/Qwen2-57B-A14B | 🚧 |
| Qwen/Qwen2-57B-A14B-Instruct | 🚧 |

### Qwen2-Math

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/Qwen2-Math-1.5B | 🚧 |
| Qwen/Qwen2-Math-1.5B-Instruct | 🚧 |
| Qwen/Qwen2-Math-7B | 🚧 |
| Qwen/Qwen2-Math-7B-Instruct | 🚧 |
| Qwen/Qwen2-Math-72B | 🚧 |
| Qwen/Qwen2-Math-72B-Instruct | 🚧 |
| Qwen/Qwen2-Math-RM-72B | 🚧 |

### Qwen2.5

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/Qwen2.5-0.5B | 🚧 |
| Qwen/Qwen2.5-0.5B-Instruct | 🚧 |
| Qwen/Qwen2.5-1.5B | 🚧 |
| Qwen/Qwen2.5-1.5B-Instruct | 🚧 |
| Qwen/Qwen2.5-3B | 🚧 |
| Qwen/Qwen2.5-3B-Instruct | 🚧 |
| Qwen/Qwen2.5-7B | 🚧 |
| Qwen/Qwen2.5-7B-Instruct | 🚧 |
| Qwen/Qwen2.5-14B | 🚧 |
| Qwen/Qwen2.5-14B-Instruct | 🚧 |
| Qwen/Qwen2.5-32B | 🚧 |
| Qwen/Qwen2.5-32B-Instruct | 🚧 |
| Qwen/Qwen2.5-72B | 🚧 |
| Qwen/Qwen2.5-72B-Instruct | 🚧 |

### Qwen2.5-Math

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/Qwen2.5-Math-1.5B | 🚧 |
| Qwen/Qwen2.5-Math-1.5B-Instruct | 🚧 |
| Qwen/Qwen2.5-Math-7B | 🚧 |
| Qwen/Qwen2.5-Math-7B-Instruct | 🚧 |
| Qwen/Qwen2.5-Math-72B | 🚧 |
| Qwen/Qwen2.5-Math-72B-Instruct | 🚧 |
| Qwen/Qwen2.5-Math-RM-72B | 🚧 |

### Qwen2.5-Coder

| 模型名称 | 静态图下载 model_name |
|:------|:-:|
| Qwen/Qwen2.5-Coder-1.5B | 🚧 |
| Qwen/Qwen2.5-Coder-1.5B-Instruct | 🚧 |
| Qwen/Qwen2.5-Coder-7B | 🚧 |
| Qwen/Qwen2.5-Coder-7B-Instruct | 🚧 |