chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,63 @@
|
||||
# CPU - Intel® Xeon®
|
||||
|
||||
!!! note "AMD Zen CPUs"
|
||||
On AMD Zen 4 / Zen 5 CPUs, AMD Zen optimizations are auto-enabled when the [`zentorch`](https://github.com/amd/ZenDNN-pytorch-plugin) package is installed. All models supported by vLLM on CPU are supported on AMD Zen as well; model compatibility does not change. This page reflects the current CPU reference validation matrix on Intel systems. See [AMD Zen optimizations](../../getting_started/installation/cpu.md#amd-zen-optimizations) for details.
|
||||
|
||||
## Validated Hardware
|
||||
|
||||
| Hardware |
|
||||
| -------- |
|
||||
| [Intel® Xeon® 6 Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html) |
|
||||
| [Intel® Xeon® 5 Processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) |
|
||||
|
||||
## Recommended Models
|
||||
|
||||
### Text-only Language Models
|
||||
|
||||
| Model | Architecture | Supported |
|
||||
| ------------------------------------ | ---------------------------------------- | --------- |
|
||||
| unsloth/gpt-oss-20b | GptOssForCausalLM | ✅ |
|
||||
| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ |
|
||||
| meta-llama/Llama-3.2-1B | LlamaForCausalLM | ✅ |
|
||||
| meta-llama/Llama-3.2-3B-Instruct | LlamaForCausalLM | ✅ |
|
||||
| meta-llama/Llama-3.3-70B-Instruct | LlamaForCausalLM | ✅ |
|
||||
| RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8 | LlamaForCausalLM | ✅ |
|
||||
| RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ |
|
||||
| RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ |
|
||||
| RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ |
|
||||
| RedHatAI/DeepSeek-R1-Distill-Llama-70B-quantized.w8a8 | LlamaForCausalLM | ✅ |
|
||||
| hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 | LlamaForCausalLM | ✅ |
|
||||
| AMead10/Llama-3.2-1B-Instruct-AWQ | LlamaForCausalLM | ✅ |
|
||||
| AMead10/Llama-3.2-3B-Instruct-AWQ | LlamaForCausalLM | ✅ |
|
||||
| TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ | LlamaForCausalLM | ✅ |
|
||||
| TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ | LlamaForCausalLM | ✅ |
|
||||
| ibm-granite/granite-3.2-2b-instruct | GraniteForCausalLM | ✅ |
|
||||
| Qwen/Qwen3-1.7B | Qwen3ForCausalLM | ✅ |
|
||||
| Qwen/Qwen3-4B | Qwen3ForCausalLM | ✅ |
|
||||
| Qwen/Qwen3-8B | Qwen3ForCausalLM | ✅ |
|
||||
| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ |
|
||||
| Qwen/Qwen3-14B-AWQ | Qwen3ForCausalLM | ✅ |
|
||||
| Qwen/Qwen3-30B-A3B | Qwen3MoeForCausalLM | ✅ |
|
||||
| Qwen/QwQ-32B-AWQ | Qwen2ForCausalLM | ✅ |
|
||||
| Qwen/Qwen1.5-0.5B-Chat-GPTQ-Int4 | Qwen2ForCausalLM | ✅ |
|
||||
| RedHatAI/QwQ-32B-quantized.w8a8 | Qwen2ForCausalLM | ✅ |
|
||||
| zai-org/glm-4-9b-hf | GLMForCausalLM | ✅ |
|
||||
| google/gemma-7b | GemmaForCausalLM | ✅ |
|
||||
| microsoft/Phi-4-reasoning | Phi3ForCausalLM | ✅ |
|
||||
| TheBloke/Mistral-7B-Instruct-v0.2-AWQ | MistralForCausalLM | ✅ |
|
||||
|
||||
### Multimodal Language Models
|
||||
|
||||
| Model | Architecture | Supported |
|
||||
| ------------------------------------ | ---------------------------------------- | --------- |
|
||||
| meta-llama/Llama-4-Scout-17B-16E-Instruct | Llama4ForConditionalGeneration | ✅ |
|
||||
| google/gemma-3-4b-it | Gemma3ForConditionalGeneration | ✅ |
|
||||
| google/gemma-3-12b-it | Gemma3ForConditionalGeneration | ✅ |
|
||||
| google/gemma-4-E4B-it | Gemma4ForConditionalGeneration | ✅ |
|
||||
| google/gemma-4-E2B-it | Gemma4ForConditionalGeneration | ✅ |
|
||||
| google/gemma-4-26B-A4B-it | Gemma4ForConditionalGeneration | ✅ |
|
||||
| microsoft/Phi-4-multimodal-instruct | Phi4MMForCausalLM | ✅ |
|
||||
| Qwen/Qwen2.5-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ |
|
||||
| openai/whisper-large-v3 | WhisperForConditionalGeneration | ✅ |
|
||||
|
||||
✅ Runs and optimized.
|
||||
@@ -0,0 +1,64 @@
|
||||
# XPU - Intel® GPUs
|
||||
|
||||
## Validated Hardware
|
||||
|
||||
| Hardware |
|
||||
| -------- |
|
||||
| [Intel® Arc™ Pro B-Series Graphics](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/arc/workstations/b-series/overview.html) |
|
||||
|
||||
## Recommended Models
|
||||
|
||||
### Text-only Language Models
|
||||
|
||||
| Model | Architecture | BF16/FP16/Dynamic FP8 | Compressed_tensors FP8 | MXFP4 |
|
||||
| -------------------------------------------------- | ------------------------------------------------ | --------------------- | ---------------------- | ----- |
|
||||
| openai/gpt-oss-20b | GPTForCausalLM | | | ✅ |
|
||||
| openai/gpt-oss-120b | GPTForCausalLM | | | ✅ |
|
||||
| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | LlamaForCausalLM | ✅ | | |
|
||||
| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | QwenForCausalLM | ✅ | | |
|
||||
| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | QwenForCausalLM | ✅ | | |
|
||||
| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | LlamaForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen2.5-72B-Instruct | Qwen2ForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen3-32B | Qwen3ForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen3-30B-A3B | Qwen3ForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen3-30B-A3B-GPTQ-Int4 | Qwen3ForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen3-coder-30B-A3B-Instruct | Qwen3ForCausalLM | ✅ | | |
|
||||
| Qwen/QwQ-32B | QwenForCausalLM | ✅ | | |
|
||||
| deepseek-ai/DeepSeek-V2-Lite | DeepSeekForCausalLM | ✅ | | |
|
||||
| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | | |
|
||||
| THUDM/GLM-4-9B-chat | GLMForCausalLM | ✅ | | |
|
||||
| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
|
||||
| chuhac/TeleChat2-35B | LlamaForCausalLM (TeleChat2 based on Llama arch) | ✅ | | |
|
||||
| 01-ai/Yi1.5-34B-Chat | YiForCausalLM | ✅ | | |
|
||||
| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
|
||||
| deepseek-ai/DeepSeek-Coder-33B-base | DeepSeekCoderForCausalLM | ✅ | | |
|
||||
| meta-llama/Llama-2-13b-chat-hf | LlamaForCausalLM | ✅ | | |
|
||||
| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen1.5-14B-Chat | QwenForCausalLM | ✅ | | |
|
||||
| Qwen/Qwen1.5-32B-Chat | QwenForCausalLM | ✅ | | |
|
||||
| RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic | LlamaForCausalLM | | ✅ | |
|
||||
|
||||
### Multimodal Language Models
|
||||
|
||||
| Model | Architecture | BF16 | Dynamic FP8 | MXFP4 |
|
||||
| ---------------------------- | -------------------------------- | ---- | ----------- | ----- |
|
||||
| OpenGVLab/InternVL3_5-8B | InternVLForConditionalGeneration | ✅ | ✅ | |
|
||||
| OpenGVLab/InternVL3_5-14B | InternVLForConditionalGeneration | ✅ | ✅ | |
|
||||
| OpenGVLab/InternVL3_5-38B | InternVLForConditionalGeneration | ✅ | ✅ | |
|
||||
| Qwen/Qwen2-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
|
||||
| Qwen/Qwen2.5-VL-72B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
|
||||
| Qwen/Qwen2.5-VL-32B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
|
||||
| THUDM/GLM-4v-9B | GLM4vForConditionalGeneration | ✅ | ✅ | |
|
||||
| openbmb/MiniCPM-V-4 | MiniCPMVForConditionalGeneration | ✅ | ✅ | |
|
||||
|
||||
### Embedding and Reranker Language Models
|
||||
|
||||
| Model | Architecture | BF16 | Dynamic FP8 | MXFP4 |
|
||||
| ----------------------- | ------------------------------ | ---- | ----------- | ----- |
|
||||
| Qwen/Qwen3-Embedding-8B | Qwen3ForTextEmbedding | ✅ | ✅ | |
|
||||
| Qwen/Qwen3-Reranker-8B | Qwen3ForSequenceClassification | ✅ | ✅ | |
|
||||
|
||||
✅ Runs and optimized.
|
||||
🟨 Runs and correct but not optimized to green yet.
|
||||
❌ Does not pass accuracy test or does not run.
|
||||
Reference in New Issue
Block a user