# Hardware detection and simulation

whichllm detects the current machine and can also simulate hardware for
purchase planning.

The source of truth is the `hardware/` package plus curated registry data in
`data/gpu.py`. `constants.py` remains as a compatibility export layer for older
imports.

## Detected data

The ranker receives a `HardwareInfo` object with:

- GPU list
- CPU name
- physical CPU cores
- AVX2 and AVX-512 support
- total RAM
- free disk space
- OS name

Each GPU is represented as `GPUInfo`:

- name
- vendor
- VRAM bytes
- usable VRAM bytes, when a runtime headroom is active
- NVIDIA compute capability, when known
- CUDA or ROCm version, when known
- memory bandwidth estimate
- whether the GPU uses shared memory

## NVIDIA

NVIDIA detection tries `nvidia-ml-py` first. If NVML is unavailable, fails to
initialize, or returns no devices, whichllm falls back to:

```bash
nvidia-smi --query-gpu=name,memory.total,clocks.max.memory --format=csv,noheader,nounits
```

If a driver rejects `clocks.max.memory`, whichllm retries the older
`name,memory.total` query.

For known cards, curated data and strict `dbgpu` lookups provide:

- memory bandwidth
- compute capability

The max memory clock is used when a marketing name covers multiple memory
types. For example, GTX 1650 GDDR5 and GDDR6 cards share the same broad driver
name, so whichllm uses the reported memory clock when available and falls back
to the conservative bandwidth when it is not.

DGX Spark / NVIDIA GB10 uses unified system memory. When the driver reports
`memory.total` as unavailable, whichllm treats GB10 as shared memory and uses
system RAM for fit checks.

Compute capability is used to warn when a card is below the minimum expected by
common local inference tools.

## AMD

On Linux, AMD detection tries `rocm-smi` first:

- product name
- VRAM
- ROCm driver version

If `rocm-smi` is unavailable, it falls back to `lspci` and then
`/sys/class/drm`.

On Windows, whichllm uses `Win32_VideoController` as a fallback for AMD GPUs.
When possible, it also reads the 64-bit dedicated-memory value from the
`Control\Video` registry path because `AdapterRAM` is a 32-bit field and can
cap larger cards around 4 GB.

AMD shared-memory APUs are treated differently from discrete GPUs. Names such
as Strix Halo, Ryzen AI MAX, Radeon 8050S, Radeon 8060S, Radeon 890M, and
Radeon 780M are modeled as shared-memory systems. If the reported VRAM is just
a small aperture, whichllm uses the system memory pool for fit checks instead
of treating it as a tiny discrete GPU.

## Intel

Intel integrated GPUs are detected on Linux through `lspci` or sysfs, and on
Windows through `Win32_VideoController`. They do not normally report dedicated
VRAM, so whichllm records them with `0` dedicated VRAM and labels them as
shared memory.

Discrete Intel Arc cards are kept as dedicated-memory GPUs when the device name
and memory report look like a discrete adapter.

The Intel backend factor is lower than NVIDIA, AMD, and Apple because local LLM
GPU inference support is less mature.

## Apple Silicon

On macOS, whichllm uses:

```bash
system_profiler SPHardwareDataType -json
```

Apple Silicon uses unified memory, so the detected chip memory is treated as
available GPU memory. Memory bandwidth is looked up by chip family when known.

Partial offload on Apple Silicon is not penalized like discrete PCIe offload.
Weights still live in unified memory, so the speed penalty is milder.

## CPU and memory

CPU detection reads:

- `/proc/cpuinfo` on Linux
- `sysctl` on macOS
- `wmic` on Windows, then PowerShell / CIM when `wmic` is unavailable or only
  returns a header

Physical core count comes from `psutil`, with a Linux `/proc/cpuinfo` fallback.

RAM comes from `psutil.virtual_memory()`. Disk free space is checked under the
user's home directory by default.

## GPU simulation

Use `--gpu` to simulate a GPU:

```bash
whichllm --gpu "RTX 4090"
whichllm hardware --gpu "Apple M3 Max"
whichllm upgrade "RTX 4090" "RTX 5090" "H100"
```

Simulation uses the `dbgpu` package for a TechPowerUp-backed GPU database.
whichllm adds extra handling for common aliases and Apple Silicon chips because
those are not covered by dbgpu.

Use `--vram` when a GPU name is ambiguous, unknown, or has multiple variants:

```bash
whichllm --gpu "RTX 5060 Ti" --vram 16
whichllm hardware --gpu "Unknown GPU" --vram 24
```

For detected integrated or unified-memory GPUs, use `--vram` and
`--bandwidth` / `--ram-bandwidth` to override the automatically detected usable
capacity and memory bandwidth:

```bash
whichllm --vram 8 --ram-bandwidth 68
whichllm hardware --vram 8 --bandwidth 68
```

If multiple GPUs are detected, pass `--gpu-index` to choose the GPU shown by
`whichllm hardware`:

```bash
whichllm --gpu-index 1 --vram 8 --ram-bandwidth 68
```

By default, whichllm applies a small automatic VRAM headroom before fit checks.
This avoids recommending models that only fit on paper but overflow in runtimes
that need extra graph buffers or loader overhead. Tune it with:

```bash
whichllm --vram-headroom 1.5GB
whichllm --vram-headroom 10%
whichllm --vram-headroom none
```

Multi-GPU simulation accepts repeated flags, comma-separated values, and count
shorthand:

```bash
whichllm --gpu "2x RTX 4090"
whichllm --gpu "RTX 4090" --gpu "RTX 3090"
whichllm --gpu "RTX 4090, RTX 3090"
```

`--vram` is only supported for a single simulated GPU. For multi-GPU
simulation, use known GPU names so whichllm can resolve each card's VRAM from
the GPU database.

## Fit types

Compatibility checks classify a candidate into one of three fit types:

| Fit | Meaning |
| --- | --- |
| `full_gpu` | Required memory fits in available GPU memory |
| `partial_offload` | GPU plus usable system RAM can hold the model |
| `cpu_only` | Usable system RAM can hold the model without GPU |

If neither GPU memory nor usable RAM can hold the model, the candidate is not
ranked.

whichllm keeps a bounded system-RAM reserve for the OS and other processes.
Use `--ram-budget available` to cap partial-offload planning to the current
available RAM reported by the OS, or pass a fixed budget such as
`--ram-budget 8GB`.

## Multiple GPUs

For fit checks, whichllm uses a conservative multi-GPU budget rather than
pretending all VRAM is one perfect device. It starts from raw total VRAM, applies
a small per-GPU overhead, and then applies a utilization factor. Homogeneous
sets receive a less severe reduction than heterogeneous sets.

If a dedicated GPU is present, low-aperture shared-memory integrated GPUs are
not added to the fit pool. This avoids treating unrelated system RAM and
dedicated VRAM as one full-GPU target.

For speed estimates, whichllm uses the largest detected GPU as the
representative device and marks multi-GPU speed as low-confidence. This avoids
claiming ideal scaling when real performance depends on backend split mode,
PCIe/NVLink bandwidth, NCCL/RCCL support, batch size, and model architecture.

This is a practical fit approximation. It does not model every tensor-parallel
or pipeline-parallel runtime configuration.

## Disk checks

The compatibility check also compares estimated model weight size with free
disk space. If the model cannot be downloaded, it is marked unrunnable.

## Known limitations

- GPU bandwidth is a lookup or database estimate, not a live benchmark.
- Speed estimates are planning numbers. The default table and JSON fields such
  as `speed_confidence` and `speed_range_tok_per_sec` show uncertainty.
- Driver, runtime, batch size, prompt length, and thermal limits can change real
  performance.
- Multi-GPU runtime behavior depends on the inference backend and is only
  approximated.
- Apple and shared-memory APU behavior is modeled as unified-memory style, but
  real results still depend on OS pressure and memory bandwidth.