elizaos--eliza/plugins/plugin-local-inference/native/verify/PLATFORM_MATRIX.md

# Eliza-1 platform matrix — build · verify · bench, one command each

> Single reference for every entry in `SUPPORTED_TARGETS`
> (`packages/app-core/scripts/build-llama-cpp-mtp.mjs`). For each target:
> the one-command build, the one-command kernel verify, the one-command
> bench, the current status, and the exact prerequisite if it is not done
> here. The tracked hardware/readiness view is
> [`../../../../docs/eliza-1-pipeline/06-test-matrix.md`](../../../../docs/eliza-1-pipeline/06-test-matrix.md);
> the enforceable contract is [`kernel-contract.json`](kernel-contract.json)
> (checked by `make -C packages/inference/verify kernel-contract`); the
> bundle plan is
> [`../../../../docs/ELIZA_1_GGUF_PLATFORM_PLAN.json`](../../../../docs/ELIZA_1_GGUF_PLATFORM_PLAN.json).

## Verify status as of 2026-05-12 (post multi-agent wave)

### 2026-07-04 Stage 6 contract update — Gemma runtime path

The 8/8 QJL / PolarQuant / `turbo3_tcq` fixture gates remain valuable
regression coverage for legacy head_dim=128 KV-cache routes and shared FFI
symbols, but they are **not** evidence that the shipped Gemma 4 product graph
runs those KV kernels. The Gemma path is now represented separately in the
machine-readable contract (`kernel-contract.json` `gemmaRuntimeDispatch`) and
in code by
`plugins/plugin-local-inference/src/services/active-model.ts::assertGemmaRuntimeDispatchContract`:
managed Eliza-1 bundles must resolve to `turboquant_q4` in the manifest,
flash-attention on, stock KV (`q8_0` / f16 headroom upgrade), and drafter-backed
`draft-mtp` whenever the manifest/catalog claims MTP.

Current MTP artifact coverage is partial: `eliza-1-2b` and `eliza-1-4b` host
`mtp/drafter-<tier>.gguf`; `9b`, `27b`, and `27b-256k` still need hosted
Gemma drafter GGUFs plus non-zero acceptance evidence. `runMtpDoctor()` now
fails the "Gemma MTP drafter coverage" check until all five tiers are hosted.
Android Adreno/Mali, iOS weight-backed, native Windows, ROCm, linux-aarch64,
and LiteRT NPU rows remain real-device blockers until recordable JSON evidence
with `passRecordable: true` is produced on those targets.

Re-ran the full integration verify matrix on this box (Intel Arrow Lake CPU +
Intel ARL/ANV Vulkan + RTX 5080 / sm_120 CUDA, with a full-corpus SFT job
holding ~12 GB VRAM concurrently — no OOM contention on the short verify runs):

| Target | Result |
| --- | --- |
| `make kernel-contract` | PASS — `OK kernels=6 targets=26 manifestNames=6` |
| `make reference-test` | PASS — C reference clean; `gen_fixture --self-test` finite (fused-attn + TBQ V-cache parity OK) |
| `make cpu-bench` | PASS (nothing to rebuild; harness in place) |
| `make cpu-dispatch-smoke` | PASS — `ATTN_SCORE_QJL` + `FUSED_ATTN_QJL_TBQ` MT-vs-ST bit-identical, no NaN (compiles + runs `qjl_mt_check.c`) |
| `make cpu-qjl-polar-attn-smoke` | PASS — `flash_attn_ext` over the dequant-hopped QJL1_256 K / Q4_POLAR V path emits 65536 finite outputs (nan=0 inf=0 maxabs=0.0300), guards fork commit `cb700767` / tag `v1.1.1-eliza`. (Was an orphan `cpu_qjl_polar_attn_smoke.c` wired into no Makefile target until 2026-06-25; now built+run by this target. Re-verified on Apple M4 Max against a stock llama.cpp ggml build, 2026-06-25.) |
| `make vulkan-dispatch-smoke` | PASS — Intel ARL: `GGML_OP_ATTN_SCORE_QJL` 32 outs max 2.7e-7, `GGML_OP_FUSED_ATTN_QJL_TBQ` 512 outs max 4.5e-8 |
| `make vulkan-verify` | PASS — 8/8 (turbo3/turbo4/turbo3_tcq/qjl/polar incl. polar pre-Hadamard, both residual modes) |
| `make vulkan-verify-multiblock` | PASS — 8/8 across 1/2/4/8 blocks-per-workgroup |
| `make vulkan-verify-fused` | PASS — 1920/1920 outputs (4 cases) on Intel ARL ANV, max diff ≤ 7.2e-7 |
| `make cuda-verify` | PASS — 8/8 each kernel + 1920/1920 fused on RTX 5080 (sm_120), max diff ≤ 9.5e-6 |
| `make cuda-verify-fused` | PASS — 1920/1920 fused QJL-K/TBQ-V on RTX 5080, max diff 4.47e-7 |
| `gen_fixture --self-test` on cvd-1 (`android-x86_64-cpu`) | PASS — bit-identical to host across all 6 required kernels + fused-attn + TBQ V-cache; cross-compiled with NDK r29 Android Clang 20.0.0, run on live Cuttlefish (cvd 1.53.0) under KVM (`evidence/platform/android-x86_64-cpu.json`). The cross-built fork `llama-server` loads on the cvd with the full Eliza-1 KV-cache-type whitelist (`tbq3_0, tbq4_0, qjl1_256, q4_polar, tbq3_tcq`). |
| `vulkan_verify` on cvd-1 SwiftShader (`android-x86_64-vulkan`) | DIAGNOSTIC-ONLY — 8/8 SPIR-V fixture cases pass with max diff < 1e-5, but the cvd virtio-gpu ICD is SwiftShader (software, vendor 0x1ae0); per the fail-closed software-ICD rule, this is **not** recordable runtime-ready evidence (`evidence/platform/android-x86_64-vulkan.json`). |

Nothing regressed in this wave. `bun run typecheck` for `packages/app-core` is
clean; `bun test packages/app-core/src/services/local-inference/` is 603 pass /
17 fail where all 17 failures are the known test-isolation flakes (downloader ×6
— passes 7/7 alone — plus `cache-restart-corruption` / `cache-multi-model` /
`cache-thrash` / `cache-stress` shared-mock-state, and the 2 `fused llama-server`
tests that need the fused binary built); `…/voice/` is 217/218 + 28/28 green;
`python3 -m pytest packages/training/scripts/{eval,publish,manifest,wakeword}
packages/training/benchmarks` is 140 passed / 1 skipped.

## Status vocabulary

| Status | Meaning |
| --- | --- |
| **verified-here** | A real hardware run happened on the machine that wrote this doc (Intel Arrow Lake / Mesa ANV Linux for CPU + Vulkan; **NVIDIA RTX 5080 Mobile / sm_120 for CUDA**; Apple M4 Max for Metal/MoltenVK from prior passes; iPhone 15 Pro for the iOS device smoke). |
| **authored-pending-hardware** | Source + build plumbing + a fail-closed runner exist; no real run on the matching device class yet. |
| **needs-operator** | The build/run needs `sudo` or a toolkit install the agent cannot do. (CUDA 12.8 for native sm_120 SASS is now installed at `/usr/local/cuda-12.8`; the build hook auto-pins it.) |
| **needs-bigger-box** | The build itself OOMs / is too slow on the 31 GB / 24-core dev box (the CUDA-fused build is ~30 GB peak RAM, ~2 h); use the cloud runner (`packages/app-core/scripts/cloud/run-on-cloud.sh`, or `packages/training/scripts/cloud/`). |

## How the three "one commands" map

- **Build:** `node packages/app-core/scripts/build-llama-cpp-mtp.mjs --target <triple>` (prepend `ELIZA_MTP_SKIP_SERVER_STRUCTURED_OUTPUT=1` while the structured-output server patch is still being fixed; the iOS targets emit a `.a` for the xcframework patch, the `-fused` targets emit `libelizainference` + the fused server, everything else emits `llama-server` + `llama-cli` + `llama-speculative-simple` + `llama-bench` + `llama-completion`). The fork build (`packages/inference/llama.cpp` submodule, or the `~/.cache/eliza-mtp` clone) `git reset --hard`s on each run — do source edits first, build last, retry on clobber. **Serialize fork builds; never two CUDA builds at once on the 31 GB box.**
- **Kernel verify** (synthetic fixtures, fast — minutes): `make -C packages/inference/verify <backend>-verify` (`metal-verify` / `vulkan-verify` / `cuda-verify`; add `-multiblock` / `-fused` for the extra coverage). These are the AGENTS.md §8 8/8-PASS gates. They do **not** need the bundle bytes.
- **Built-fork graph dispatch** (proves a real llama.cpp graph route selects the kernel): `make -C packages/inference/verify vulkan-dispatch-smoke` (Vulkan), `metal dispatch-smoke` (Metal), the C++ `vulkan_dispatch_smoke` / `dispatch_smoke.mm` harnesses. CUDA's equivalent is `cuda-verify` (fixture-parity `__device__` kernels) + the `cuda_runner.sh` graph smoke (now via `llama-bench` / `llama-completion`, not `llama-cli` — the fork's `llama-cli` is conversation-only and busy-loops on stdin EOF).
- **Bench:** `make -C packages/inference/verify <backend>-bench` (`metal-bench` / `vulkan-bench` / `cpu-bench`) for the standalone-kernel perf harness; `llama-bench -m <gguf> -ngl 99 -p … -n … -fa 1 --cache-type-k …` for the model-graph throughput (the verify runners do this); `verify/e2e_loop_bench.mjs` / `verify/thirty_turn_endurance_harness.mjs` for the end-to-end voice loop. The fork ships `llama-bench` + `llama-completion` next to `llama-server` (as of this commit), so the bench path exists on every built target.

## CPU baseline — runnable anywhere

| Target | Build | Kernel verify | Bench | Status | Prereq if not done |
| --- | --- | --- | --- | --- | --- |
| `linux-x64-cpu` | `node …/build-llama-cpp-mtp.mjs --target linux-x64-cpu` | `make -C …/verify reference-test` (C-reference round-trip); the CPU score/decode ops are the C references themselves; `make cpu-dispatch-smoke` (graph picks `GGML_OP_ATTN_SCORE_QJL` + `GGML_OP_FUSED_ATTN_QJL_TBQ` on the CPU backend and asserts MT-vs-ST bit-identical, no NaN — `verify/qjl_mt_check.c`; `CPU_BIN_DIR`/`GGML_INC_DIR` default to the in-repo fork build/checkout, no env vars) | `make -C …/verify cpu-bench cpu-simd-bench`; `llama-bench` on the staged text GGUF | **verified-here** (`reference-test` clean; `cpu-dispatch-smoke` PASS — MT-vs-ST bit-identical; AVX-VNNI int8-QJL 5.25× / fp32-QJL LUT-gather ~2.5–8× — `bench_results/cpu_avxvnni_2026-05-11.json`, `bench_results/cpu_kopt_2026-05-11.json`). `kernel-contract.json` `runtimeStatus.cpu` = `runtime-ready` for `qjl` + `fusedAttn` (`verify/cpu-runtime-dispatch-evidence.json`); `reference-only` for TBQ/Polar standalone score (no public CPU graph op — validated by `reference-test`). The §3 CPU kernel-completeness build gate still fails by design (turbo3_tcq/polarquant not CPU-buildable). | verify-on-device against the staged bundle bytes (`verifyBundleOnDevice`); wire `probeKernels()` to read `cpu-runtime-dispatch-evidence.json` so a fresh `linux-x64-cpu` build's `CAPABILITIES.json` reports `qjl_full` runtime-ready. |
| `linux-aarch64-cpu` | `--target linux-aarch64-cpu` (needs an arm64 Linux host or a sysroot+cross-toolchain — no aarch64-cross wiring on x64 here) | `make reference-test` + `cpu-dispatch-smoke` on the arm64 host | `cpu-bench cpu-simd-bench` (NEON dotprod paths) | **authored-pending-hardware** | An arm64 Linux box (Ampere Altra / Graviton / Snapdragon-Linux). |
| `windows-x64-cpu` | `--target windows-x64-cpu` (mingw cross-build) | `pwsh -File verify/windows_runner.ps1 -Backend cpu -Model C:\models\eliza-1-smoke.gguf` on a real Windows box (now drives `llama-bench` + `llama-completion`, not `llama-cli`) | `windows_runner.ps1` (above) | **authored-pending-hardware** (cross-built exe is not counted) | A native Windows x64 host. |
| `windows-arm64-cpu` | `--target windows-arm64-cpu` (needs an MSVC arm64 cross-toolchain or a native Windows-arm64 host — no mingw arm64 wiring here) | `windows_runner.ps1 -Backend cpu` on a Snapdragon X box | `windows_runner.ps1` | **authored-pending-hardware** | A Snapdragon X Elite / Copilot+ PC. |
| `android-arm64-cpu` | `node packages/app-core/scripts/aosp/compile-libllama.mjs` (NDK cross-build) | CPU/NEON parity via `adb` on a physical Android device | `adb`-pushed `cpu_bench` / `llama-bench` | **authored-pending-hardware** | A physical Android device + NDK. |
| `android-x86_64-cpu` | `ANDROID_NDK_HOME=… node …/build-llama-cpp-mtp.mjs --target android-x86_64-cpu` (NDK cross-build, `-DANDROID_ABI=x86_64`, forces AVX/AVX2/FMA/F16C — the x86_64 Android ABI baseline is SSE4.2; the QJL/Polar CPU kernels need AVX2) | (a) **kernel C-reference parity on cvd (NEW 2026-05-12, verified-here):** cross-compile `gen_fixture` with NDK r29 (Android Clang 20.0.0, target x86_64-unknown-linux-android24) and `adb push` to cvd-1; run `gen_fixture_android_x86_64 --self-test` → bit-identical to host across all six required kernels + fused-attn + tbq V-cache parity (`turbo3=-2.501480 / turbo4=-23.721790 / turbo3_tcq=-4.822659 / qjl=3.696591 / polar=-1.994053 / polar_qjl=-1.438744`); the cross-built fork `llama-server` loads on the cvd and exposes the full Eliza-1 KV-cache-type whitelist (tbq3_0, tbq4_0, qjl1_256, q4_polar, tbq3_tcq) with banner `built with Clang 20.0.0 for Android x86_64`, fork commit `536ff214`. Evidence: [`evidence/platform/android-x86_64-cpu.json`](evidence/platform/android-x86_64-cpu.json). (b) The orthogonal 8-step Cuttlefish (`cvd`) chat-completion smoke `node packages/app-core/scripts/aosp/smoke-cuttlefish.mjs`: 5/6 infra steps PASS on the live cvd (cvd reachable, APK installed abi=x86_64, ElizaAgentService start, /api/health agentState=running runtime=ok, bearer token); step 6 chat completion failed — no model staged in the release APK on that cvd. See [`../reports/porting/2026-05-12/cuttlefish-x86_64-smoke.md`](../reports/porting/2026-05-12/cuttlefish-x86_64-smoke.md). | `adb`-pushed `cpu_bench` / `llama-bench`; `e2e_loop_bench.mjs` on the cvd | **kernel C-reference parity verified-here on Cuttlefish** + **build verified-here** (real x86_64 Android ELF — `interpreter /system/bin/linker64` — + libs, fork commit `536ff214`; `CAPABILITIES.json` `qjl_full`/`polarquant` true), Cuttlefish cvd 8-step infra smoke 5/6 PASS. `kernel-contract.json` `platformTargets.android-x86_64-cpu` = `runtime-ready` / `runtime-ready` / `verified`. | A `build-aosp.mjs --launch` rebuild staging the new `android-x86_64-cpu` libllama + a bundled eliza-1-smoke GGUF in the privileged APK → 8/8 chat smoke; Vulkan-on-cvd is gfxstream/SwiftShader (software → not recordable). |
| `android-x86_64-vulkan` | `…/build-llama-cpp-mtp.mjs --target android-x86_64-vulkan` (NDK + Vulkan headers, `-DANDROID_ABI=x86_64`) | standalone `vulkan_verify` fixtures pass on the host ANV iGPU AND on the cvd SwiftShader ICD (8/8 cases, max diff < 1e-5, [`evidence/platform/android-x86_64-vulkan.json`](evidence/platform/android-x86_64-vulkan.json)) — **DIAGNOSTIC-ONLY** under cvd because SwiftShader is software (vendor 0x1ae0 LLVM 16); graph dispatch needs real ChromeOS x86_64 GPU (Adreno/Mali under ARCVM) — cvd virtio-gpu Vulkan is gfxstream/SwiftShader (software → no recordable evidence per fail-closed) | `adb`-pushed `vulkan_bench` | **authored-pending-hardware** for graph dispatch (ChromeOS GPU); cross-build + SPIR-V fixture pass + Android Vulkan loader path validated under SwiftShader on cvd | Real ChromeOS x86_64 GPU silicon (Adreno/Mali under ARCVM) or a passed-through host GPU via crosvm gfxstream + real-GPU host. |
| `linux-x64-cpu-fused` | `ELIZA_MTP_SKIP_SERVER_STRUCTURED_OUTPUT=1 …/build-llama-cpp-mtp.mjs --target linux-x64-cpu-fused` | `OMNIVOICE_FUSE_VERIFY.json` `ok=true` + `verifyFusedSymbols` (abi/omnivoice/llama-reexport counts); **`llama-server --cache-type-k qjl1_256 --cache-type-v q4_polar` boots healthy, `/completion` returns tokens (NEW 2026-05-12, fork commit `cb700767` / tag `v1.1.1-eliza`)** — guarded by the `make cpu-qjl-polar-attn-smoke` regression test (`cpu_qjl_polar_attn_smoke.c`) | `FFI runtime-fused.integration.test.ts` (spawns the fused `llama-server`, hits `/completion` + `/v1/audio/speech` same-PID); `llama-bench`/`llama-completion` for text | **verified-here** for the merged HTTP route + symbol-verify + the QJL/Polar KV-cache warmup-no-segfault path; exit-1 is the §3 CPU-backend kernel-completeness gate (turbo3_tcq/qjl_full/polarquant aren't CPU-graph-dispatch caps), `CAPABILITIES.json` `publishable: false`. | A weight-backed `/v1/audio/speech` smoke against a real `tts/omnivoice-*.gguf` (the dev stand-in bundle has no `tts/`); voice:duet end-to-end (the QJL/Polar segfault is fixed; remaining duet block is an SQL/embeddings-dim runtime-bootstrap concern, separate from the kernel). |

## CUDA — verified-here on the RTX 5080 (sm_120, **native SASS** via CUDA 12.8)

| Target | Build | Kernel verify | Bench | Status | Prereq if not done |
| --- | --- | --- | --- | --- | --- |
| `linux-x64-cuda` | `ELIZA_MTP_SKIP_SERVER_STRUCTURED_OUTPUT=1 …/build-llama-cpp-mtp.mjs --target linux-x64-cuda` (~1.5–2 h, ~30 GB peak — serialize; check `free -m`/`uptime` first). CUDA 12.8 is installed at `/usr/local/cuda-12.8`; pass `CUDACXX=/usr/local/cuda-12.8/bin/nvcc PATH=/usr/local/cuda-12.8/bin:$PATH` so the build hook's `cudaArchListFlag()` sees the 12.8 nvcc and appends `100;120` to the arch list (the system `/usr/bin/nvcc` is 12.0 and would silently downgrade to `80;86;89;90;90a`). **Full integration build now installed (2026-05-12)** — `~/.eliza/local-inference/bin/mtp/linux-x64-cuda/`, forkCommit `a61c93aaa5` (v1.2.0-eliza), builtAt `2026-05-12T17:16:58Z`, `libggml-cuda.so.0.9.7` 473 MB (real sm_120a SASS), all binaries (`llama-bench`/`llama-cli`/`llama-completion`/`llama-server`/`llama-speculative-simple`) ldd-clean via `$ORIGIN` rpath. | `make -C …/verify cuda-verify cuda-verify-fused` (self-contained nvcc compile of `cuda_verify.cu`; **8/8 + 1920/1920 PASS on the RTX 5080**, max diff ≤ 9.5e-6 / 4.47e-7; `cuda-verify-fused` exercises the warp-cooperative kernel mirroring the production `cuda/fused-attn-qjl-tbq.cu`; the harness builds a native `sm_120.cubin` under 12.8). **Re-verified 2026-05-12** against the installed real-build. | `verify/cuda_runner.sh --report …` (builds the fork, `cuda-verify`, then `runtime_graph_smoke.sh --gen-check` → `llama-bench --cache-type-k tbq3_0 -ngl 99` + `llama-completion`); `bench_results/cuda_e2e_2026-05-11.json` (text pp ~2.3–6.7k t/s, tg ~40–55 t/s; ASR `eliza-1-asr.gguf` → arch `qwen3vl 1.7B`, pp16 ~1023, pp128 ~4561, tg32 ~62 t/s); nsys: DP4A `qjl_score_dp4a_kernel` ~2.27× faster than fp32 `qjl_score_kernel`. **New `llama-bench` numbers (2026-05-12, real install, freed GPU):** eliza-1-0_6b bundle pp512/tg128 d=0 **19932 / 345.5 t/s**, d=16000 **1956 / 108.5 t/s**; eliza-1-1_7b bundle pp512/tg128 d=0 **11931 / 194.7 t/s**, d=16000 **1797 / 84.9 t/s**; base Qwen3-0.6B-Q8_0 d=0 **20979 / 356 t/s**, base Qwen3-1.7B-Q8_0 d=0 **12414 / 159 t/s**. **`llama-server` smoke verified**: 4 GPU slots, `/health → ok`, `POST /completion` 32-token decode at **420.57 tps decode / 1092.66 tps prefill** on the 0_6b bundle. | **verified-here** for `cuda-verify` / `cuda-verify-fused` / **full ggml-cuda integration build** (NEW 2026-05-12) / text + ASR `llama-bench` / `llama-server /completion` / native-`sm_120`-SASS-compile. `kernel-contract.json` `runtimeStatus.cuda` + `fusedAttn.runtimeStatus.cuda` = `runtime-ready`. `runtime_graph_smoke.sh --gen-check` errors with "no cache-type alias for turbo3" — expected on the non-fused build (`CAPABILITIES.json` reports `mtp: false`, `missingRequiredKernels: ["mtp"]`); the cache-type aliases are added by the `mtp`/fused-build patch path. | verify-on-device against the staged bundle bytes (`verifyBundleOnDevice`); wire `probeKernels()` to read the install's `CAPABILITIES.json`. |
| `linux-x64-cuda-fused` | `ELIZA_MTP_SKIP_SERVER_STRUCTURED_OUTPUT=1 CUDACXX=/usr/local/cuda-12.8/bin/nvcc PATH=/usr/local/cuda-12.8/bin:$PATH …/build-llama-cpp-mtp.mjs --target linux-x64-cuda-fused --jobs 3` — the **big** build: full ggml-cuda + the omnivoice-core graft, ~30 GB peak RAM under `-j 6` (the 31 GB dev box OOM-killed the parent build script under `-j 6` mid-build at the `fattn.cu` long-pole). **Built 2026-05-12** on this box at `-j 3` from a clean dir against fork commit `a61c93aaa5` (v1.0.0-eliza) + omnivoice pin `38f824023d12`. `OMNIVOICE_FUSE_VERIFY.json` `ok=true` (llama=0, omnivoice=10, abi=23 symbols). | `cuda-verify cuda-verify-fused` + `OMNIVOICE_FUSE_VERIFY.json` — **1920/1920 PASS on RTX 5080 sm_120, max diff 5.07e-07** (`logs/cuda-verify-fused-fusedbuild-rtx5080-2026-05-12.log`). `make cuda-hardware` against the install: 6/6 fixture-set PASS (`logs/cuda-hardware-fusedbuild-rtx5080-2026-05-12.log`); graph-smoke gated on `llama-bench` (not in fused-target list — non-blocking tooling gap). | `verify/e2e_loop_bench.mjs --backend cuda --tier 0_6b --turns 1` against the fused install — **voice_rtf 0.4255 (PASS ≤ 0.5)**, tg 64.82 tok/s, first_token 43.3 ms, mtp 12/12, peak RSS 2340 MB. `packages/inference/reports/porting/2026-05-12/e2e-loop-cuda-2026-05-12.json`. | **verified-here on RTX 5080 Laptop (sm_120, CUDA 12.8)** — `CAPABILITIES.json` reports `publishable: true`, `missingRequiredKernels: []`, `mtp + turbo3 + turbo4 + turbo3_tcq + qjl_full + polarquant + lookahead + ngramDraft = all true`. | Re-run on additional sm classes (sm_89 / sm_90 / sm_100 datacenter) to confirm no arch regression in the CMAKE_CUDA_ARCHITECTURES list. |
| `linux-aarch64-cuda` | `--target linux-aarch64-cuda` on an arm64 Linux + Hopper/Blackwell host (GH200 = aarch64 host + H100/H200/GB200 GPU) | `make cuda-verify cuda-verify-fused` on that host; `verify/gh200_runner.sh --report …` (refuses non-aarch64 / non-Hopper-9.x) | `gh200_runner.sh`; `llama-bench` on the `27b-256k` / `27b-256k` tier GGUFs | **authored-pending-hardware** | A GH200 / H100-aarch64 / GB200 host. Use the cloud runner. |
| `windows-x64-cuda` | `--target windows-x64-cuda` (MSVC + CUDA Toolkit on Windows) | `pwsh -File verify/windows_runner.ps1 -Backend cuda -Model C:\models\eliza-1-smoke.gguf` on NVIDIA hardware (drives `llama-bench` + `llama-completion`) | `windows_runner.ps1` (above) | **authored-pending-hardware** (cross-built exe not counted) | A native Windows + NVIDIA box. |
| `windows-x64-cuda-fused` | `--target windows-x64-cuda-fused` | `windows_runner.ps1 -Backend cuda` + `OMNIVOICE_FUSE_VERIFY.json` | the fused Windows `llama-server`'s `/v1/audio/speech` | **authored-pending-hardware** | The Windows-CUDA hardware runner first, then the fused build on that host. |

## Vulkan — verified-here on Intel Arc/Xe Mesa ANV + NVIDIA RTX 5080 (two device classes)

| Target | Build | Kernel verify | Bench | Status | Prereq if not done |
| --- | --- | --- | --- | --- | --- |
| `linux-x64-vulkan` | `…/build-llama-cpp-mtp.mjs --target linux-x64-vulkan` | `make -C …/verify vulkan-verify vulkan-verify-multiblock vulkan-verify-fused` (**8/8 + 8/8 + 1920/1920 PASS on Intel ANV**, max diff ≤ 7.6e-6 / 6.3e-7); `make vulkan-native-smoke` / `vulkan-dispatch-smoke` (**built-fork graph routes PASS on Intel ARL/ANV** — the harness drives the two fused attention ops the fork pin declares in `ggml.h`: `GGML_OP_ATTN_SCORE_QJL` 32 outs max 2.7e-7 + `GGML_OP_FUSED_ATTN_QJL_TBQ` 512 outs max 4.5e-8 — `vulkan-runtime-dispatch-evidence.json` + `hardware-results/linux-vulkan-smoke-*.log`. The standalone TBQ/Polar score kernels are covered by `vulkan-verify`; their built-fork graph entries in the evidence file are from a prior full-patched-build run.) | `make vulkan-bench`; `llama-bench -ngl 99` (the dispatch smoke does this) | **verified-here on Intel ARL/ANV AND NVIDIA RTX 5080** — `kernel-contract.json` `runtimeStatus.vulkan` = `runtime-ready` for the 5 score kernels + fused_attn. **NEW 2026-06-23 (verified-here, RTX 5080 Laptop / sm_120, Vulkan api 1.4.329):** `make vulkan-verify` → **8/8 PASS on the 5080** for turbo3 / turbo4 / turbo3_tcq / qjl / polar incl. polar pre-Hadamard + both residual modes, max diff ≤ 7.6e-6 (`reports/vulkan-verify-rtx5080-2026-06-23.txt`); plus a full model-graph run — gemma-4-E2B-Q8 `llama-bench -ngl 99` pp512 **1486** / tg128 **123** t/s with FA engaged (`flash_attn = enabled`, no V-cache padding, output correctness-checked via `llama-cli`). Built with the Android-NDK host `glslc` (shaderc v2022.3, **no coopmat → scalar path**) + system spirv-headers. Two device classes now (Intel ANV + NVIDIA). | Native AMD (RADV) Vulkan; **coopmat/tensor-core Vulkan** (needs a coopmat-capable glslc — the NDK one is too old); verify-on-device against the staged bundle bytes. |
| `linux-x64-vulkan-fused` | `…/build-llama-cpp-mtp.mjs --target linux-x64-vulkan-fused --jobs 4` — Vulkan ggml + the omnivoice-core graft (much lighter than the CUDA-fused build — ~3 min on this box vs. ~85 min for CUDA-fused). **Built 2026-05-12** from a clean dir against fork commit `a61c93aaa5` + omnivoice pin `38f824023d12`. `OMNIVOICE_FUSE_VERIFY.json` `ok=true` (llama=0, omnivoice=10, abi=23). Fused SPIR-V baked into `libggml-vulkan.so` (`eliza_fused_attn_qjl_tbq_data` + `eliza_fused_attn_qjl_polar_data` symbols, _len pair). | `vulkan-verify vulkan-verify-fused` + `OMNIVOICE_FUSE_VERIFY.json` — **all PASS on Intel ARL iGPU (Mesa ANV 25.2.8)**: `fused_attn_qjl_tbq` 1920/1920 + 1536/1536 causal + `fused_attn_qjl_polar` 1920/1920 + 1536/1536 causal = 6912 outputs total, max diff 6.26e-07 (`logs/vulkan-verify-fused-fusedbuild-anv-2026-05-12.log`). | `verify/e2e_loop_bench.mjs --backend vulkan --tier 0_6b --turns 1` against the fused install — runs end-to-end (`e2eOk: true`, mtp 31/31), but iGPU performance keeps voice_rtf at **1.7269** (FAIL ≤ 0.5; the gate targets discrete-GPU class). tg 12.13 tok/s, first_token 493 ms, peak RSS 1370 MB. `packages/inference/reports/porting/2026-05-12/e2e-loop-vulkan-2026-05-12.json`. | **verified-here for kernel parity + e2e functionality on Intel ARL iGPU**; `CAPABILITIES.json` `publishable: true`, `missingRequiredKernels: []`. Publish-gate (`voice_rtf ≤ 0.5`) FAIL on iGPU — gate is a discrete-GPU target. | Re-run `e2e_loop_bench` on a discrete Vulkan-mode card (RDNA3 / Ada in pure-Vulkan / Intel BMG) for a Vulkan voice-rtf number under the discrete-GPU class. |
| `windows-x64-vulkan` | `--target windows-x64-vulkan` (mingw + Khronos Vulkan-Headers cross-build) | `pwsh -File verify/windows_runner.ps1 -Backend vulkan -Model C:\models\eliza-1-smoke.gguf` on native Windows Vulkan | `windows_runner.ps1` | **authored-pending-hardware** | A native Windows + GPU box. |
| `windows-arm64-vulkan` | `--target windows-arm64-vulkan` (MSVC arm64 cross-toolchain) | `windows_runner.ps1 -Backend vulkan` on a Snapdragon X box (Adreno X1 = Vulkan 1.3) | `windows_runner.ps1` | **authored-pending-hardware** | A Snapdragon X Elite / Copilot+ PC. |
| `android-arm64-vulkan` | `node packages/app-core/scripts/aosp/compile-libllama.mjs` (NDK cross-build) | `make -C …/verify android-vulkan-smoke` — standalone fixtures **6/6 PASS on Pixel 6a / Mali-G78** (`hardware-results/android-vulkan-smoke-*.log`); built-fork graph dispatch evidence (`ELIZA_ANDROID_VULKAN_GRAPH_EVIDENCE`) still open; Adreno not yet run | `adb`-pushed `vulkan_bench` / `llama-bench` | **authored-pending-hardware** for graph dispatch (standalone fixtures verified-here on Mali) | A built-fork/app graph-dispatch report on one Adreno + one Mali device. |

## Metal / Apple — verified-here on Apple M4 Max (re-verified 2026-06-23 for the Gemma 4 cutover)

> **2026-06-23 (Gemma 4 cutover Metal validation).** Re-ran the full §8 kernel
> gate on an Apple **M4 Max** (macOS 26.2, 128 GB): `metal-verify` **8/8**,
> `metal-verify-multiblock` **8/8**, and `metal-verify-fused` now **PASS
> 1536/1536** fused-attention outputs across 2 cases (`n_kv_heads=2` GQA/MQA —
> exactly Gemma 4's shared-KV shape), max diff ≤ 9.5e-7. That **resolves the prior
> "fails by design (no `metal_verify` cases-array path)" item.** Gemma 4's only
> Metal-specific lever is flash-attention at `head_dim_global = 512`:
> **confirmed supported** by the Metal FA gate (`ggml-metal-device.m` supported
> head-dim set `{112,128,192,256,320,512,576}`). End-to-end generation of the
> **real `google/gemma-4-E2B` eliza-1 base** (Q8_0, 4.65 B; loader-reported
> `gemma4` arch, `head_dim 512`, MQA `head_count_kv=1`, SWA window 512,
> logit-softcap 30, 128k ctx) passes with `flash_attn` auto→enabled — correct
> output ("…Paris… Eiffel Tower and the Louvre Museum.") plus `llama-bench`
> **pp512 636 / tg128 23 t/s** (`-ngl 99 -fa 1`). This supersedes the earlier
> `gemma-3-1b` (head_dim 256) proxy — the head_dim-512 FA path is now exercised
> directly. Records:
> [`evidence/platform/darwin-arm64-metal.json`](evidence/platform/darwin-arm64-metal.json)
> (+ `darwin-arm64-metal-verify.log`, `darwin-arm64-metal-gemma-gen.log`).
>
> **2026-06-25 — per-tier Metal throughput matrix (#9580).** The single
> gemma-4-E2B point above is one model; the full **per-tier** sweep (the
> Qwen3.5-era Eliza-1 bundles staged on the M4 Max — `0_6b`→`9b`, pp512
> 9307→724 / tg128 103→41 t/s) + the §8 kernel gate re-verified 8/8 today live
> in [`metal-per-tier-perf-matrix.md`](metal-per-tier-perf-matrix.md), reproducible
> via [`metal-perf-matrix.mjs`](metal-perf-matrix.mjs). Re-run once the Gemma-4
> bundles are staged for per-tier Gemma numbers.

| Target | Build | Kernel verify | Bench | Status | Prereq if not done |
| --- | --- | --- | --- | --- | --- |
| `darwin-arm64-metal` | `…/build-llama-cpp-mtp.mjs --target darwin-arm64-metal` (macOS host, builds + embeds `default.metallib`) | `make -C …/verify metal-verify metal-verify-multiblock metal-verify-fused` (**8/8 + 8/8 + 1536/1536 on M4 Max, 2026-06-23**); `make dispatch-smoke` (built-fork graph dispatch for `GGML_OP_ATTN_SCORE_{QJL,TBQ×3,POLAR}` + pre-Hadamard Polar — all PASS). Gemma-4 FA head_dim=512 **supported** by the Metal FA gate. | `make metal-bench metal-bench-batched metal-bench-multiblock`; `llama-bench -ngl 99` | **verified-here on M4 Max** — §8 kernel gate 40/40 PASS (incl. fused-attention); `kernel-contract.json` `runtimeStatus.metal` = `runtime-ready`; **real `google/gemma-4-E2B` (head_dim 512, MQA, SWA) generation + `llama-bench` pp512 636 / tg128 23 t/s (FA=1) verified on the Metal FA path** (2026-06-23). | Full text+**MTP**+voice latency/RSS/thermal gate against a release-shaped Gemma-4 bundle. The text base is Metal-verified (left); the **separate MTP drafter** weights are now **sourced + staged** (`SeatownSin/gemma-4-E4B-mtp-drafter` LiteRT extraction, 77M params / 42 tensors). Remaining: the `safetensors → mtp-draft GGUF` conversion + a `darwin-arm64-metal-fused` build for the on-Metal `--spec-type draft-mtp` gate — full runbook in [`docs/gemma4-mtp-drafter-conversion.md`](../../docs/gemma4-mtp-drafter-conversion.md). |
| `darwin-arm64-metal-fused` | `…/build-llama-cpp-mtp.mjs --target darwin-arm64-metal-fused --jobs 10` — links `omnivoice-core` + `libelizainference.dylib` + `llama-omnivoice-server` + `libmtmd` + `default.metallib`; `verify-symbols.mjs` (`omnivoice=10 abi=8`) | `metal-verify metal-verify-multiblock dispatch-smoke` (same as above) + `verify-symbols.mjs` | Bun FFI smoke against `~/.eliza/local-inference/models/eliza-1-1_7b.bundle` (loads real OmniVoice Q4_K_M + Qwen3-ASR for TTS + ASR — `reports/local-e2e/2026-05-11/fused-voice-ffi-smoke.json`); `e2e_loop_bench.mjs` | **verified-here on macOS Metal** for the fused dylib FFI smoke (real GGUF-backed TTS + ASR in one fused process) | Built-fork graph-dispatch smoke + full latency/RSS/thermal gates; the fused `llama-server` route on macOS (currently the macOS evidence is the FFI path, not the HTTP route). |
| `ios-arm64-metal` | `…/build-llama-cpp-mtp.mjs --target ios-arm64-metal` (macOS+Xcode, emits `.a` + headers + `default.metallib` → `build-xcframework.mjs --verify` glues the `LlamaCpp.xcframework`) | `build-xcframework.mjs --verify` (kernel-symbol + runtime-symbol + structure audits — PASS); `run-physical-device-smoke.mjs` (**3/3 XCTest cases PASS on iPhone 15 Pro / iOS 26.3.1**, `--skip-voice-abi=false` — `hardware-results/ios-device-smoke-2026-05-11.json`) | the iOS XCTest harness; (no `llama-bench` on iOS — the runtime is the static lib + `eliza_inference_*` ABI) | **verified-here on iPhone 15 Pro** for the symbol/structure audits + the runtime-symbol XCTest; the §3 P0 blocker is a weight-backed Eliza-1 bundle smoke from the Capacitor app shell (first token / first audio / peak RSS / thermal). | A real Eliza-1 bundle smoke from the Capacitor app shell. |
| `ios-arm64-simulator-metal` | `…/build-llama-cpp-mtp.mjs --target ios-arm64-simulator-metal` | `build-xcframework.mjs --verify` (simulator slice — PASS); simulator smoke against the embedded metallib + `GGML_OP_ATTN_SCORE_TBQ` Turbo4 route | the iOS simulator XCTest | **authored-pending-hardware** (symbol/structure audits pass; no simulator weight-backed run) | Simulator smoke against the embedded metallib. |

(`darwin-x64-metal` is **not** a supported target — Apple Silicon `darwin-arm64-metal` only.)

## ROCm — runner exists, no AMD host here

| Target | Build | Kernel verify | Bench | Status | Prereq if not done |
| --- | --- | --- | --- | --- | --- |
| `linux-x64-rocm` | `…/build-llama-cpp-mtp.mjs --target linux-x64-rocm` (needs `hipcc` + ROCm) | `make -C …/verify hip-verify` — the standalone fixture-parity harness (NEW this wave): `hip_verify.cu` is a thin shim that `#include`s `cuda_verify.cu` (which now guards its backend headers on `__HIP_PLATFORM_AMD__` and aliases the `cuda*` runtime calls to `hip*`), so it runs the EXACT same ~25 device kernels + fixture loader + reference cross-check the NVIDIA `cuda-verify` does, compiled by `hipcc` against a `gfx*` GPU. Plus `verify/rocm_runner.sh --report …` (refuses without `hipcc` + `rocminfo` `gfx*` agent + a smoke GGUF; builds the fork, then `runtime_graph_smoke.sh --gen-check` → `llama-bench` + `llama-completion` on the HIP backend). | `make hip-verify`; `rocm_runner.sh`; `llama-bench -ngl 99` on the HIP backend | **authored-pending-hardware** — `hip_verify.cu` + the `hip-verify` Makefile target are authored + buildable (no `hipcc` on the authoring box → clean "install ROCm / see rocm_runner.sh" message); the fork's *production* `.cu` kernels (turboquant.cuh/qjl.cu/polarquant.cu/turbo-tcq.cu) are not yet `__HIP_PLATFORM_AMD__`-clean — until that lands the ROCm runtime story is the `hip-verify` numeric gate + the documented reduced-optimization local mode (`ELIZA_LOCAL_ALLOW_STOCK_KV=1`, loud warning, not publishable) for production inference. | An AMD ROCm host (RDNA2/RDNA3 or CDNA, `gfx*` agent — e.g. a vast.ai MI300 box). |

## LiteRT-LM (Android NPU) — dispatcher code exists, no hardware verify yet

The `litert` backend is real in-process dispatcher code — the `.litertlm`
single-file loader (`services/engine.ts` `stagedLitertModelPath`,
`services/backend.ts` selection, `litertBackendSupported`, the `litert`
lifecycle component in `local-model-lifecycle-matrix.ts`, and the manifest
`litert-lm` runtime in `manifest/schema.ts`). It is the compiled-in NPU path
described in §11 of the AGENTS.md contract (an owned backend behind the same
FFI, NOT a subprocess). It has **zero rows anywhere else in this matrix** and no
hardware run — recorded here honestly instead of silently omitted.

| Target | Build | Kernel verify | Bench | Status | Prereq if not done |
| --- | --- | --- | --- | --- | --- |
| `android-arm64-litertlm` | LiteRT-LM runtime compiled into `libelizainference` for the Android NPU path (AICore/QNN-class delegate); staged as a `.litertlm` bundle file. | No LiteRT parity harness under `verify/` yet — the numeric-parity gate for the `.litertlm` forward is unwritten. | `adb`-pushed decode/latency on a physical NPU-class device (e.g. Pixel Tensor / Snapdragon Hexagon). | **authored-pending-hardware** — dispatcher + loader + manifest runtime + lifecycle component exist and are unit-tested (`backend-selector.precedence.test.ts`, `engine-direct-bundle.test.ts`), but no real NPU device has ever run a `.litertlm` bundle, and there is no kernel-parity gate. | A physical Android NPU device + a converted `.litertlm` bundle + a `verify/`-side parity harness. |

## Quick "one command for everything I can run here" line

```bash
# From the repo root, on this box (CPU + Intel-ANV Vulkan + RTX 5080 CUDA):
make -C packages/inference/verify kernel-contract reference-test cuda-verify cuda-verify-fused
make -C packages/inference/verify vulkan-verify vulkan-verify-multiblock vulkan-verify-fused
make -C packages/inference/verify vulkan-dispatch-smoke   # built-fork Vulkan graph routes (needs the linux-x64-vulkan build)
# Bench (CUDA text + ASR, RTX 5080):
~/.cache/eliza-mtp/eliza-llama-cpp/build-cuda/bin/llama-bench \
  -m ~/.eliza/local-inference/models/eliza-1-1_7b.bundle/text/eliza-1-1_7b-32k.gguf -ngl 99 -p 16,512 -n 32 -fa 1
```

## Not in `SUPPORTED_TARGETS` — runtime-side / explicitly-out-of-scope notes

### MLX (`mlx_lm.server`) — REMOVED

The Apple-Silicon `mlx_lm.server` spawn-and-route path was removed in commit
`20d50d7553` (P1 consolidation). It violated the local-inference invariant
of no subprocesses + no TCP loopback. No production callsite ever invoked
it, and `MLX_IN_PROCESS_PLAN.md` documents the in-process unblock plan if
MLX becomes a real requirement. The stub `mlx-server.ts` itself has been
deleted; see `services/index.ts` for the cleaned export surface.

### TPU / NPU — not a target this wave (verdict, documented)

**No.** The eliza-1 text backbone (0.6B smallest, fp16/Q4) does not fit a Coral
Edge TPU's 8 MB on-chip SRAM, isn't int8-only quantizable to the Coral's
constraints, and KV-cache attention is not an Edge-TPU workload. The Pixel
Tensor TPU could in principle run a small int8 transformer but there is no
public delegate API to target it from a third-party app, and NNAPI is
deprecated by Google in favour of per-vendor delegates. The Android GPU
(Mali/Adreno via Vulkan) is the right on-device accelerator for the text model
— which the `android-arm64-vulkan` / `android-x86_64-vulkan` targets already
cover. The sidecars (Silero VAD, Qwen3-ASR-0.6B, Qwen3-Embedding-0.6B) don't
win enough on an NPU to justify the conversion work, and OmniVoice TTS is fused
into the llama.cpp build (one GGML pin) — pulling it onto a separate NPU breaks
the fusion contract (§4: one process, one build). The one open angle: a
`ELIZA_VAD_QNN_DELEGATE=1` flag that, when `onnxruntime-mobile` is built with
the Qualcomm QNN EP, runs Silero VAD on the Hexagon NPU island while the CPU
sleeps — that is a **battery** optimization for always-listening wake-word
mode, not a latency one, and is a stretch, not core. No `plugin-coral` /
`plugin-qnn` is added.