docs: make Chinese README the default
This commit is contained in:
@@ -1,3 +1,9 @@
|
||||
<!-- WEHUB_ZH_README -->
|
||||
> [!NOTE]
|
||||
> 本文档由 WeHub 基于上游 README 翻译整理,属于社区翻译,非官方中文文档。
|
||||
> [English](./README.en.md) · [原始项目](https://github.com/Andyyyy64/whichllm) · [上游 README](https://github.com/Andyyyy64/whichllm/blob/HEAD/README.md)
|
||||
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
|
||||
|
||||
# whichllm
|
||||
|
||||
[](https://pypi.org/project/whichllm/)
|
||||
@@ -10,65 +16,62 @@
|
||||
<a href="https://trendshift.io/repositories/30336" target="_blank"><img src="https://trendshift.io/api/badge/repositories/30336" alt="Andyyyy64%2Fwhichllm | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
</p>
|
||||
|
||||
**Find the best local LLM that actually runs on your hardware.**
|
||||
**找到真正能在你硬件上运行的最佳本地 LLM。**
|
||||
|
||||
Auto-detects your GPU/CPU/RAM and ranks the top models from HuggingFace that fit your system.
|
||||
自动检测你的 GPU/CPU/RAM,并从 HuggingFace 中按适配度对适合你系统的顶级模型进行排名。
|
||||
|
||||
[日本語版はこちら](docs/README.ja.md)
|
||||
|
||||
## Quick start
|
||||
## 快速开始
|
||||
|
||||
Run the recommendation command once, with no project setup.
|
||||
无需项目配置,运行一次推荐命令即可。
|
||||
|
||||
```bash
|
||||
uvx whichllm@latest
|
||||
```
|
||||
|
||||
Simulate a GPU before you buy hardware.
|
||||
在购买硬件之前模拟 GPU。
|
||||
|
||||
```bash
|
||||
uvx whichllm@latest --gpu "RTX 4090"
|
||||
```
|
||||
|
||||
Install it when you use it often.
|
||||
若经常使用,可安装到本地。
|
||||
|
||||
```bash
|
||||
uv tool install whichllm
|
||||
uv tool upgrade whichllm # update an existing install
|
||||
```
|
||||
|
||||
Other install paths.
|
||||
其他安装方式。
|
||||
|
||||
```bash
|
||||
brew install andyyyy64/whichllm/whichllm
|
||||
pip install whichllm
|
||||
```
|
||||
|
||||
## Want a safer pick?
|
||||
## 想要更稳妥的选择?
|
||||
|
||||
By default, whichllm is ambitious. It ranks the best model that looks runnable
|
||||
on your machine, including partial RAM offload and near-edge VRAM fits when
|
||||
they seem usable.
|
||||
默认情况下,whichllm 较为激进。它会对你机器上看起来可运行的最佳模型进行排名,包括部分 RAM 卸载(offload)以及看似可用的接近显存上限的 VRAM 配置。
|
||||
|
||||
If you want a more comfortable LM Studio-style recommendation, start with:
|
||||
若你想要更接近 LM Studio 风格的舒适推荐,可从以下方式开始:
|
||||
|
||||
```bash
|
||||
uvx whichllm@latest --gpu-only --speed usable --vram-headroom 1GB
|
||||
```
|
||||
|
||||
This keeps only models that fit fully in GPU VRAM, filters out slow estimates,
|
||||
and leaves extra VRAM for runtime overhead.
|
||||
这样只会保留完全装入 GPU VRAM 的模型,过滤掉速度较慢的估算结果,并预留额外 VRAM 以应对运行时开销。
|
||||
|
||||
If LM Studio still says the model is slightly too large, increase the headroom:
|
||||
若 LM Studio 仍提示模型略大,可增加余量:
|
||||
|
||||
```bash
|
||||
uvx whichllm@latest --gpu-only --speed usable --vram-headroom 1.5GB
|
||||
```
|
||||
|
||||
## Common workflows
|
||||
## 常见工作流
|
||||
|
||||
After install, run `whichllm` directly. For one-off runs, replace `whichllm`
|
||||
with `uvx whichllm@latest`.
|
||||
安装后,直接运行 `whichllm`。若只需一次性运行,将 `whichllm`
|
||||
替换为 `uvx whichllm@latest`。
|
||||
|
||||
```bash
|
||||
# Best models for this machine
|
||||
@@ -112,7 +115,7 @@ whichllm --top 1 --json
|
||||
|
||||

|
||||
|
||||
## See it
|
||||
## 眼见为实
|
||||
|
||||
```text
|
||||
$ whichllm --gpu "RTX 4090"
|
||||
@@ -122,16 +125,14 @@ $ whichllm --gpu "RTX 4090"
|
||||
#3 Qwen/Qwen3-30B-A3B 30.0B Q5_K_M score 82.7 102 t/s
|
||||
```
|
||||
|
||||
The 32B model **fits your card fine** — whichllm still ranks the 27B #1,
|
||||
because it scores higher on real benchmarks and is a newer generation.
|
||||
A size-only "what fits?" tool would hand you the bigger one. That gap is
|
||||
the whole point of whichllm. (Note #3: a MoE model at 102 t/s — speed is
|
||||
ranked on *active* params, quality on *total*.)
|
||||
32B 模型**完全适配你的显卡**——whichllm 仍将 27B 排在第 1 位,
|
||||
因为它在真实基准测试中得分更高,且属于更新一代。
|
||||
仅按尺寸判断「能跑什么?」的工具会把更大的模型推荐给你。这种差距正是 whichllm 的价值所在。(注 #3:MoE 模型达 102 t/s——速度按*活跃*参数量排名,质量按*总*参数量评估。)
|
||||
|
||||
## What can I run?
|
||||
## 我能运行什么?
|
||||
|
||||
Real top picks (snapshot 2026-05 — your results track **live** HuggingFace
|
||||
data, this is not a static list):
|
||||
真实精选推荐(快照 2026-05——你的结果会跟踪 **实时** HuggingFace
|
||||
数据,这不是静态列表):
|
||||
|
||||
| Hardware | VRAM | Top pick | Speed |
|
||||
|---|---|---|---|
|
||||
@@ -141,68 +142,55 @@ data, this is not a static list):
|
||||
| Apple M3 Max | 36 GB | `Qwen3.6-27B` · Q5_K_M · score 89.4 | ~9 t/s |
|
||||
| CPU only | — | `gpt-oss-20b` (MoE) · Q4_K_M · score 45.2 | ~6 t/s |
|
||||
|
||||
`whichllm --gpu "<your card>"` simulates any of these before you buy.
|
||||
By default, rankings include full-GPU, partial-offload, and CPU-only
|
||||
candidates when they are usable. Use `--gpu-only` or `--fit full-gpu` when
|
||||
you only want models that fit entirely in GPU VRAM.
|
||||
The default table shows memory, estimated generation speed, fit type, and
|
||||
published date. Speed is colored by practical usability: under 4 tok/s is red,
|
||||
4-10 is yellow, 10-30 is green, and 30+ is bright green. `~` / `?` still mark
|
||||
estimate confidence.
|
||||
`whichllm --gpu "<your card>"` 可在购买前模拟上述任意配置。
|
||||
默认情况下,排名会包含全 GPU、部分卸载和仅 CPU 的候选(在可用时)。若你只想选择完全装入 GPU VRAM 的模型,请使用 `--gpu-only` 或 `--fit full-gpu`。
|
||||
默认表格会显示内存、预估生成速度、适配类型和发布日期。速度按实际可用性着色:低于 4 tok/s 为红色,
|
||||
4-10 为黄色,10-30 为绿色,30+ 为亮绿色。`~` / `?` 仍会标注
|
||||
估算置信度。
|
||||
|
||||
## Why whichllm?
|
||||
## 为什么选择 whichllm?
|
||||
|
||||
Fitting a model into your VRAM is the easy part. The hard part is knowing
|
||||
**which of the models that fit is actually the best** — and that is what
|
||||
whichllm is built to get right.
|
||||
把模型塞进 VRAM 并不难。难的是知道**在能跑的模型里,哪一个才是真正最好的**——而这正是
|
||||
whichllm 要解决的问题。
|
||||
|
||||
- **Evidence-based ranking, not a size heuristic** — The top pick is
|
||||
chosen from merged real benchmarks (LiveBench, Artificial Analysis,
|
||||
Aider, multimodal/vision, Chatbot Arena ELO, Open LLM Leaderboard) —
|
||||
never "the biggest model that happens to fit."
|
||||
- **Recency-aware** — Stale leaderboards are demoted along each model's
|
||||
lineage, so a 2024 model can't outrank a current-generation one on an
|
||||
outdated score. The benchmark snapshot date is printed under every
|
||||
ranking, so a stale recommendation is self-evident instead of silently
|
||||
trusted.
|
||||
- **Evidence-graded and guarded** — Every score is tagged
|
||||
`direct` / `variant` / `base` / `interpolated` / `self-reported` and
|
||||
discounted by confidence. Fabricated uploader claims and cross-family
|
||||
inheritance (a small fork borrowing its much larger base's score) are
|
||||
actively rejected.
|
||||
- **Architecture-aware estimates** — VRAM = weights + GQA KV cache +
|
||||
activation + overhead; speed is bandwidth-bound with per-quant
|
||||
efficiency, per-backend factors, MoE active-vs-total split, and
|
||||
unified-memory vs discrete-PCIe partial-offload modeling.
|
||||
- **One command, scriptable** — `whichllm` prints the answer; add
|
||||
`--json | jq` for pipelines. No TUI, no keybindings to memorize.
|
||||
- **Live data** — Models fetched directly from the HuggingFace API, with
|
||||
curated frozen fallbacks for offline or rate-limited use.
|
||||
- **基于证据的排名,而非尺寸启发式** —— 首选来自合并的真实基准测试(LiveBench、Artificial Analysis、
|
||||
Aider、多模态/视觉、Chatbot Arena ELO、Open LLM Leaderboard)——
|
||||
绝不是「碰巧能塞进去的最大模型」。
|
||||
- **重视时效性** —— 过时的排行榜会沿各模型谱系降权,因此 2024 年的模型不能凭过时分数压过当前一代。
|
||||
每次排名下方都会打印基准快照日期,过时推荐一目了然,而非被默默采信。
|
||||
- **证据分级与防护** —— 每个分数都会标注
|
||||
`direct` / `variant` / `base` / `interpolated` / `self-reported`,并
|
||||
按置信度折减。上传者的虚假声明和跨系列继承(小分支借用大得多的基座分数)会被主动拒绝。
|
||||
- **架构感知的估算** —— VRAM = 权重 + GQA KV cache +
|
||||
激活 + 开销;速度受带宽限制,并考虑各量化效率、各后端因子、MoE 活跃/总参数拆分,以及统一内存与离散 PCIe 部分卸载建模。
|
||||
- **一条命令,可脚本化** —— `whichllm` 直接输出答案;添加
|
||||
`--json | jq` 即可接入流水线。无 TUI,无需记忆快捷键。
|
||||
- **实时数据** —— 模型直接从 HuggingFace API 获取,并为离线或限流场景提供精选的冻结回退数据。
|
||||
|
||||
## Features
|
||||
## 功能特性
|
||||
|
||||
- **Auto-detect hardware** — NVIDIA, AMD, Intel, Apple Silicon, CPU-only
|
||||
- **Smart ranking** — Scores models by VRAM fit, speed, and benchmark quality
|
||||
- **One-command chat** — `whichllm run` downloads and starts a chat session instantly
|
||||
- **Code snippets** — `whichllm snippet` prints ready-to-run Python for any model
|
||||
- **Live data** — Fetches models directly from HuggingFace (cached for performance)
|
||||
- **Benchmark-aware** — Integrates real eval scores with confidence-based dampening
|
||||
- **Task profiles** — Filter by general, coding, vision, or math use cases
|
||||
- **GPU simulation** — Test with any GPU: `whichllm --gpu "RTX 4090"`
|
||||
- **Multi-GPU simulation** — Repeat `--gpu`, use commas, or write `2x RTX 4090`
|
||||
- **Full-GPU filter** — `--gpu-only` / `--fit full-gpu` hides offload candidates
|
||||
- **Speed-aware filtering** — `--speed usable|fast` hides slow rows by threshold
|
||||
- **Markdown output** — `--markdown` / `-m` prints pasteable GFM tables
|
||||
- **Runtime memory budgets** — `--vram-headroom` and `--ram-budget` avoid edge fits
|
||||
- **Hardware planning** — Reverse lookup: `whichllm plan "llama 3 70b"`
|
||||
- **Upgrade planning** — Compare your current machine with candidate GPUs
|
||||
- **JSON output** — Pipe-friendly: `whichllm --json`
|
||||
- **自动检测硬件** —— NVIDIA、AMD、Intel、Apple Silicon、仅 CPU
|
||||
- **智能排名** —— 按 VRAM 适配、速度和基准质量为模型打分
|
||||
- **一条命令聊天** —— `whichllm run` 即时下载并启动聊天会话
|
||||
- **代码片段** —— `whichllm snippet` 为任意模型输出可直接运行的 Python 代码
|
||||
- **实时数据** —— 直接从 HuggingFace 获取模型(带缓存以提升性能)
|
||||
- **基准感知** —— 整合真实评测分数,并按置信度衰减
|
||||
- **任务配置** —— 按通用、编程、视觉或数学场景筛选
|
||||
- **GPU 模拟** —— 用任意 GPU 测试:`whichllm --gpu "RTX 4090"`
|
||||
- **多 GPU 模拟** —— 重复 `--gpu`、使用逗号分隔,或编写 `2x RTX 4090`
|
||||
- **全 GPU 筛选** —— `--gpu-only` / `--fit full-gpu` 隐藏卸载候选
|
||||
- **速度感知筛选** —— `--speed usable|fast` 按阈值隐藏较慢行
|
||||
- **Markdown 输出** —— `--markdown` / `-m` 输出可粘贴的 GFM 表格
|
||||
- **运行时内存预算** —— `--vram-headroom` 和 `--ram-budget` 避免临界适配
|
||||
- **硬件规划** —— 反向查询:`whichllm plan "llama 3 70b"`
|
||||
- **升级规划** —— 将当前机器与候选 GPU 对比
|
||||
- **JSON 输出** —— 便于管道处理:`whichllm --json`
|
||||
|
||||
## Run & Snippet
|
||||
## 运行与代码片段
|
||||
|
||||
Try any model with a single command. No manual installs needed — whichllm
|
||||
creates an isolated environment via `uv`, installs dependencies, downloads the
|
||||
model, and starts an interactive chat.
|
||||
用一条命令试用任意模型。无需手动安装——whichllm
|
||||
通过 `uv` 创建隔离环境,安装依赖,下载
|
||||
模型,并启动交互式聊天。
|
||||
|
||||

|
||||
|
||||
@@ -217,12 +205,12 @@ whichllm run
|
||||
whichllm run "phi 3 mini gguf" --cpu-only
|
||||
```
|
||||
|
||||
Works with **all model formats**:
|
||||
- **GGUF** — via `llama-cpp-python` (lightweight, fast)
|
||||
- **AWQ / GPTQ** — via `transformers` + `autoawq` / `auto-gptq`
|
||||
- **FP16 / BF16** — via `transformers`
|
||||
支持**所有模型格式**:
|
||||
- **GGUF** — 通过 `llama-cpp-python`(轻量、快速)
|
||||
- **AWQ / GPTQ** — 通过 `transformers` + `autoawq` / `auto-gptq`
|
||||
- **FP16 / BF16** — 通过 `transformers`
|
||||
|
||||
Get a **copy-paste Python snippet** instead:
|
||||
也可直接获取**可复制粘贴的 Python 代码片段**:
|
||||
|
||||
```bash
|
||||
whichllm snippet "qwen 7b"
|
||||
@@ -245,7 +233,7 @@ output = llm.create_chat_completion(
|
||||
print(output["choices"][0]["message"]["content"])
|
||||
```
|
||||
|
||||
## Usage
|
||||
## 用法
|
||||
|
||||
```bash
|
||||
# Auto-detect hardware and show best models
|
||||
@@ -318,26 +306,25 @@ whichllm snippet "qwen 7b"
|
||||
whichllm snippet "llama 3 8b gguf" --quant Q5_K_M
|
||||
```
|
||||
|
||||
Markdown output is intended for GitHub issues, READMEs, Slack, Discord, and
|
||||
blog posts:
|
||||
Markdown 输出适用于 GitHub issues、README、Slack、Discord 和博客文章:
|
||||
|
||||
```bash
|
||||
whichllm --markdown
|
||||
whichllm -m --top 5 --gpu "RTX 4090"
|
||||
```
|
||||
|
||||
JSON model rows include `fit_type`, `vram_required_bytes`,
|
||||
`vram_available_bytes`, `uses_multi_gpu`, `multi_gpu_effective_vram_bytes`,
|
||||
`estimated_tok_per_sec`, `speed_confidence`, `speed_range_tok_per_sec`,
|
||||
`speed_notes`, `benchmark_source`, and `benchmark_confidence`. The speed range
|
||||
is a planning range, not a live benchmark.
|
||||
JSON 模型行包含 `fit_type`、`vram_required_bytes`、
|
||||
`vram_available_bytes`、`uses_multi_gpu`、`multi_gpu_effective_vram_bytes`、
|
||||
`estimated_tok_per_sec`、`speed_confidence`、`speed_range_tok_per_sec`、
|
||||
`speed_notes`、`benchmark_source` 以及 `benchmark_confidence`。速度范围
|
||||
是规划参考区间,而非实时基准测试结果。
|
||||
|
||||
## Integrations
|
||||
## 集成
|
||||
|
||||
### Ollama
|
||||
|
||||
Use JSON output to feed scripts that map HuggingFace IDs to your local Ollama
|
||||
model names:
|
||||
使用 JSON 输出为脚本提供数据,将 HuggingFace ID 映射到本地 Ollama
|
||||
模型名称:
|
||||
|
||||
```bash
|
||||
# Pick the top HuggingFace model ID
|
||||
@@ -347,95 +334,90 @@ whichllm --top 1 --json | jq -r '.models[0].model_id'
|
||||
whichllm --profile coding --top 1 --json | jq -r '.models[0].model_id'
|
||||
```
|
||||
|
||||
Ollama model names do not always match HuggingFace repo IDs, so a small mapping
|
||||
step is usually needed before `ollama run`.
|
||||
Ollama 模型名称并不总与 HuggingFace 仓库 ID 一致,因此在执行 `ollama run` 之前通常需要一步简单映射。
|
||||
|
||||
### Shell alias
|
||||
### Shell 别名
|
||||
|
||||
Add to your `.bashrc` / `.zshrc`:
|
||||
添加到你的 `.bashrc` / `.zshrc`:
|
||||
|
||||
```bash
|
||||
alias bestllm='whichllm --top 1 --json | jq -r ".models[0].model_id"'
|
||||
# Usage: ollama run $(bestllm)
|
||||
```
|
||||
|
||||
## Scoring
|
||||
## 评分
|
||||
|
||||
Each model gets a 0-100 score. Benchmark quality and size form the core;
|
||||
evidence confidence and runtime fit then scale it, with speed, source
|
||||
trust, and popularity as adjustments.
|
||||
每个模型获得 0–100 分。基准测试质量与模型规模构成核心;
|
||||
证据置信度与运行时适配随后进行缩放,速度、来源可信度与流行度作为调整项。
|
||||
|
||||
| Factor | Effect | Description |
|
||||
| 因素 | 作用 | 说明 |
|
||||
|--------|--------|-------------|
|
||||
| Benchmark quality | core | Merged LiveBench / Artificial Analysis / Aider / Vision / Arena ELO / Open LLM Leaderboard, weighted by source confidence |
|
||||
| Model size | up to 35 | `log2`-scaled world-knowledge proxy (MoE uses total params) |
|
||||
| Quantization | × penalty | Lower-bit quants discounted multiplicatively |
|
||||
| Evidence confidence | ×0.55–1.0 | none / self-reported ×0.55, inherited ×0.78, direct full |
|
||||
| Runtime fit | ×0.50–1.0 | partial-offload ×0.72, CPU-only ×0.50 |
|
||||
| Speed | -8 to +8 | Usability gate vs a fit-dependent tok/s floor; reported with confidence and range metadata |
|
||||
| Source trust | -5 to +5 | Official-org bonus, known-repackager penalty |
|
||||
| Popularity | tie-breaker | Downloads/likes; weight shrinks as evidence strengthens |
|
||||
| 基准测试质量 | 核心 | 合并 LiveBench / Artificial Analysis / Aider / Vision / Arena ELO / Open LLM Leaderboard,按来源置信度加权 |
|
||||
| 模型规模 | 最高 35 | 经 `log2` 缩放的世界知识代理指标(MoE 使用总参数量) |
|
||||
| 量化 | × 惩罚 | 低位数量化按乘法折减 |
|
||||
| 证据置信度 | ×0.55–1.0 | 无 / 自报 ×0.55,继承 ×0.78,直接完整 |
|
||||
| 运行时适配 | ×0.50–1.0 | 部分卸载(partial-offload)×0.72,仅 CPU ×0.50 |
|
||||
| 速度 | -8 至 +8 | 可用性门槛对比与适配类型相关的 tok/s 下限;附带置信度与区间元数据报告 |
|
||||
| 来源可信度 | -5 至 +5 | 官方机构加分,已知重打包者减分 |
|
||||
| 流行度 | 决胜项 | 下载量/点赞数;证据越强权重越小 |
|
||||
|
||||
Score markers:
|
||||
- **`~`** (yellow) — No direct benchmark; score inherited/interpolated from the model family
|
||||
- **`!sr`** (bright yellow) — Uploader-reported benchmark only, not independently verified
|
||||
- **`?`** (red) — No benchmark data available
|
||||
评分标记:
|
||||
- **`~`**(黄色)— 无直接基准测试;分数从模型家族继承/插值
|
||||
- **`!sr`**(亮黄色)— 仅有上传者报告的基准测试,未经独立验证
|
||||
- **`?`**(红色)— 无可用基准测试数据
|
||||
|
||||
Speed display:
|
||||
- **red** — Slow generation speed (`<4 tok/s`)
|
||||
- **yellow** — Marginal generation speed (`4-10 tok/s`)
|
||||
- **green** — Usable generation speed (`10-30 tok/s`)
|
||||
- **bright green** — Fast local generation speed (`>=30 tok/s`)
|
||||
- **`~`** (yellow) — Estimated tok/s range is available
|
||||
- **`?`** (red) — Low-confidence speed estimate; backend/runtime sensitivity is high
|
||||
速度显示:
|
||||
- **red** — 生成速度较慢(`<4 tok/s`)
|
||||
- **yellow** — 生成速度勉强可用(`4-10 tok/s`)
|
||||
- **green** — 生成速度可用(`10-30 tok/s`)
|
||||
- **bright green** — 本地生成速度快(`>=30 tok/s`)
|
||||
- **`~`**(黄色)— 提供估算的 tok/s 区间
|
||||
- **`?`**(红色)— 低置信度速度估算;对后端/运行时敏感度高
|
||||
|
||||
## Documentation
|
||||
## 文档
|
||||
|
||||
- [CLI reference](docs/cli.md)
|
||||
- [How it works](docs/how-it-works.md)
|
||||
- [Scoring](docs/scoring.md)
|
||||
- [Hardware detection and simulation](docs/hardware.md)
|
||||
- [Run and snippet](docs/run-snippet.md)
|
||||
- [Troubleshooting](docs/troubleshooting.md)
|
||||
- [CLI 参考](docs/cli.md)
|
||||
- [工作原理](docs/how-it-works.md)
|
||||
- [评分](docs/scoring.md)
|
||||
- [硬件检测与模拟](docs/hardware.md)
|
||||
- [运行与代码片段](docs/run-snippet.md)
|
||||
- [故障排除](docs/troubleshooting.md)
|
||||
|
||||
## How it works
|
||||
## 工作原理
|
||||
|
||||
### Data pipeline
|
||||
### 数据流水线
|
||||
|
||||
1. **Model fetching** — Fetches popular models from HuggingFace API:
|
||||
- Text-generation (downloads + recently updated)
|
||||
- GGUF-filtered (separate query for coverage)
|
||||
- Vision models (`image-text-to-text`) when `--profile vision` or `any`
|
||||
2. **Benchmark sources** — *Current tier* (LiveBench, Artificial Analysis
|
||||
Index, Aider) merged live when reachable, plus a curated multimodal /
|
||||
vision index; *frozen tier* (Open LLM Leaderboard v2, Chatbot Arena
|
||||
ELO). Tiers have separate caps and lineage-aware recency demotion so
|
||||
stale leaderboards stop over-rewarding older generations.
|
||||
3. **Benchmark evidence** — Five resolution levels, increasingly discounted:
|
||||
- `direct` — Exact model ID match
|
||||
- `variant` — Suffix-stripped or -Instruct variant
|
||||
- `base_model` — Base model from cardData
|
||||
- `line_interp` — Size-aware interpolation within model family
|
||||
- `self_reported` — Uploader-claimed eval (heavily discounted)
|
||||
1. **模型获取** — 从 HuggingFace API 获取热门模型:
|
||||
- 文本生成(按下载量 + 最近更新)
|
||||
- GGUF 过滤(单独查询以覆盖)
|
||||
- 视觉模型(`image-text-to-text`),当 `--profile vision` 或 `any` 时
|
||||
2. **基准测试来源** — *当前层级*(LiveBench、Artificial Analysis
|
||||
Index、Aider)在可访问时实时合并,外加精选的多模态 /
|
||||
视觉指数;*冻结层级*(Open LLM Leaderboard v2、Chatbot Arena
|
||||
ELO)。各层级设有独立上限,并配合谱系感知的时效性降权,使过时排行榜不再过度奖励旧代模型。
|
||||
3. **基准测试证据** — 五个解析层级,折扣递增:
|
||||
- `direct` — 精确模型 ID 匹配
|
||||
- `variant` — 去除后缀或 -Instruct 变体
|
||||
- `base_model` — 来自 cardData 的基础模型
|
||||
- `line_interp` — 模型家族内按规模感知的插值
|
||||
- `self_reported` — 上传者声称的评测(大幅折减)
|
||||
|
||||
Inheritance is rejected when a model's params diverge more than 2× from
|
||||
its family's dominant member, catching draft / MTP / abliterated forks
|
||||
that share a `family_id` with a much larger base.
|
||||
4. **Cache** — normally `~/.cache/whichllm/`, or `$XDG_CACHE_HOME/whichllm/`
|
||||
when `XDG_CACHE_HOME` is set to an absolute path:
|
||||
当模型参数量与其家族主导成员相差超过 2× 时,拒绝继承,
|
||||
以捕获与更大基础模型共享 `family_id` 的 draft / MTP / abliterated 分支。
|
||||
4. **缓存** — 通常为 `~/.cache/whichllm/`,或将 `XDG_CACHE_HOME` 设为绝对路径时使用 `$XDG_CACHE_HOME/whichllm/`:
|
||||
- `models.json` — 6h TTL
|
||||
- `benchmark.json` — 24h TTL
|
||||
|
||||
### Ranking engine
|
||||
### 排序引擎
|
||||
|
||||
1. **Hardware detection** — NVIDIA (nvidia-ml-py), AMD (ROCm/dbgpu), Intel, Apple Silicon (Metal), CPU cores, RAM, disk
|
||||
2. **VRAM estimation** — Weights + KV cache + activation + framework overhead (~500MB)
|
||||
3. **Compatibility** — Full GPU / Partial Offload / CPU-only; compute capability and OS checks
|
||||
4. **Speed** — tok/s from GPU memory bandwidth, quantization, backend, fit type, and MoE active parameters
|
||||
5. **Scoring** — Benchmark (with confidence dampening), size, quantization penalty, fit type, speed, popularity, source trust (official vs repackager)
|
||||
6. **Backend filter** — Apple Silicon and CPU-only restrict to GGUF for stability; Linux+NVIDIA allows AWQ/GPTQ
|
||||
1. **硬件检测** — NVIDIA(nvidia-ml-py)、AMD(ROCm/dbgpu)、Intel、Apple Silicon(Metal)、CPU 核心数、RAM、磁盘
|
||||
2. **VRAM 估算** — 权重 + KV cache + 激活值 + 框架开销(约 500MB)
|
||||
3. **兼容性** — 全 GPU / 部分卸载(Partial Offload)/ 仅 CPU;计算能力与操作系统检查
|
||||
4. **速度** — 根据 GPU 内存带宽、量化、后端、适配类型与 MoE 活跃参数量计算 tok/s
|
||||
5. **评分** — 基准测试(含置信度衰减)、规模、量化惩罚、适配类型、速度、流行度、来源可信度(官方 vs 重打包者)
|
||||
6. **后端过滤** — Apple Silicon 与仅 CPU 为稳定性限制为 GGUF;Linux+NVIDIA 允许 AWQ/GPTQ
|
||||
|
||||
### Project structure
|
||||
### 项目结构
|
||||
|
||||
```
|
||||
src/whichllm/
|
||||
@@ -484,19 +466,18 @@ uv run pytest
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
||||
欢迎贡献!请参阅 [CONTRIBUTING.md](CONTRIBUTING.md) 了解指南。
|
||||
|
||||
## Support
|
||||
|
||||
If whichllm helped you find a model or avoid a bad hardware guess,
|
||||
sponsoring is appreciated. It helps keep the project maintained: hardware
|
||||
reports, packaging, test fixtures, benchmark updates, and support for more
|
||||
machines.
|
||||
如果 whichllm 帮你找到了合适的模型,或避免了一次糟糕的硬件猜测,
|
||||
欢迎赞助。这有助于维持项目:硬件报告、打包、测试夹具(test fixtures)、基准测试更新,
|
||||
以及对更多设备的支持。
|
||||
|
||||
whichllm will stay open-source either way. Issues and PRs are always welcome.
|
||||
无论是否赞助,whichllm 都会保持开源。Issues 和 PR 始终欢迎。
|
||||
|
||||
Useful? A GitHub star helps other people find it, and I'd genuinely like to
|
||||
know what it picked for your rig. Drop it in [Issues](https://github.com/Andyyyy64/whichllm/issues).
|
||||
觉得有用?GitHub star 能帮助其他人发现它,我也很想知道
|
||||
它为你的配置选了什么。欢迎在 [Issues](https://github.com/Andyyyy64/whichllm/issues).
|
||||
|
||||
## Star History
|
||||
|
||||
@@ -505,8 +486,8 @@ know what it picked for your rig. Drop it in [Issues](https://github.com/Andyyyy
|
||||
## Requirements
|
||||
|
||||
- Python 3.11+
|
||||
- NVIDIA GPU detection via `nvidia-ml-py` (included by default)
|
||||
- AMD / Apple Silicon detected automatically
|
||||
- 通过 `nvidia-ml-py` 检测 NVIDIA GPU(默认已包含)
|
||||
- AMD / Apple Silicon 可自动检测
|
||||
|
||||
## License
|
||||
|
||||
|
||||
Reference in New Issue
Block a user