chore: import upstream snapshot with attribution

2026-07-13 12:31:33 +08:00
commit 9e8f1bbeed
1156 changed files with 235330 additions and 0 deletions
@@ -0,0 +1,179 @@
+# ODS Validation Matrix
+
+Last updated: 2026-05-25
+
+This page describes the layered coverage used to validate ODS between
+changes. It is intentionally sanitized: it publishes hardware classes,
+operating systems, GPU paths, and test phases without private hostnames, LAN
+addresses, usernames, or filesystem paths.
+
+## Layered Test Surface
+
+ODS uses four standing validation layers:
+
+| Layer | Where it runs | Coverage | What it proves |
+|---|---|---|---|
+| CI matrix | GitHub Actions containers | Installer, shell, Python, PowerShell, dashboard, environment, and distro smoke checks | Fast pull-request safety for syntax, parsing, and cheap regressions |
+| Zero-prereq bootstrap lab | Release harness distro containers | Bare Ubuntu, Debian, Fedora, Rocky, Arch, and openSUSE images with no assumed Git, jq, Python, Docker, or Compose | The public `curl` bootstrap path can provision its own prerequisites on clean Linux systems |
+| Fleet distro lab | Private lab Docker containers + Incus VMs | 10 container distros plus VMs for Ubuntu, Fedora, Rocky, Arch, and openSUSE | Fast distro breadth plus real systemd, network, Docker daemon, Compose, and installer dry-run with Docker enabled |
+| Real hardware fleet | Private local machines | Linux NVIDIA, Linux AMD, Linux ARM NVIDIA, macOS Apple Silicon, and optional Windows laptop targets | Fresh installs, GPU runtime, dashboard, Hermes, UI, lifecycle, ODS Talk, and agent capabilities |
+
+Containers are breadth, not user-experience proof. Incus VMs add systemd,
+kernel, and Docker-daemon realism, but they still do not prove GPU runtime.
+Physical fleet machines remain the release gate for accelerator and product
+behavior.
+
+## Hardware Surface
+
+| Test surface | OS family | Architecture | Accelerator path | Memory class | Fleet role |
+|---|---|---:|---|---:|---|
+| Linux NVIDIA workstation | Ubuntu 24.04 | x86_64 | High-memory CUDA GPUs | 90 GB+ VRAM per GPU | Primary CUDA install, dashboard, UI, and capability target |
+| Linux AMD unified-memory workstation | Ubuntu 24.04 | x86_64 | AMD Strix Halo / ROCm-Lemonade path | 120 GB+ unified | Primary AMD install/runtime validation target |
+| Linux NVIDIA unified-memory appliance | NVIDIA Ubuntu derivative | aarch64 | Grace Blackwell / CUDA path | 120 GB+ unified | ARM Linux + NVIDIA appliance validation target |
+| macOS constrained Apple Silicon | macOS | arm64 | Native Metal inference + Docker services | 16 GB unified | Smoke gate and tight-memory macOS validation target |
+| macOS high-memory Apple Silicon | macOS | arm64 | Native Metal inference + Docker services | 120 GB+ unified | Large-model macOS validation target |
+| Windows hybrid GPU laptop | Windows 11 + Docker Desktop/WSL2 | x86_64 | NVIDIA mobile GPU plus Intel integrated GPU | 32 GB+ system RAM | Windows installer, Docker Desktop, WSL2, and mobile-GPU validation target when enabled for a run |
+
+This standing hardware fleet is the repeatable release surface for GPU and
+product behavior: it can run in parallel whenever installer, bootstrap,
+dashboard, agent, model, or extension code changes. The CI matrix, zero-prereq
+bootstrap lab, and distro lab add repeatable distro evidence between hardware
+fleet runs. Community and volunteer testers add broader coverage on other GPUs,
+distros, operating-system versions, storage layouts, and network environments,
+but those reports are complementary evidence rather than the always-on release
+gate.
+
+## Fleet Phases
+
+The private fleet harness runs these phases and records structured artifacts for
+each host where the phase is applicable.
+
+| Phase | What it proves | Normal cadence |
+|---|---|---|
+| Zero-prereq bootstrap | The public installer can bootstrap clean distro containers without preinstalled developer tools or Docker | Every release-grade run |
+| Regression replay | Previously fixed fleet bugs have not returned | Every full fleet run |
+| Smoke gate | A constrained Apple Silicon target can fresh-install and pass core health before the larger fleet starts | Every full fleet run |
+| Preflight | OS, RAM, disk, Docker, firewall, port conflicts, prior install state | Every install run |
+| Fresh install | The public bootstrap path can nuke prior state and install non-interactively | Every full fleet run |
+| Core verify | Dashboard API, dashboard UI, llama-server models/chat, and Hermes proxy are reachable | Every post-install run |
+| Cloud-mode contracts | Cloud and hybrid modes do not accidentally require local llama-server and still render required configs | Every release-grade run |
+| Dashboard API flows | Model listing, model download/switch, and extension install state transitions | Every post-install run |
+| Hermes auth/chat | Magic-link session auth, gated Hermes access, and seed echo through the agent path | Every post-install run |
+| Browser UI | Dashboard navigation, model/extension surfaces, and Open WebUI model proxy behavior | Default UI target every run; scheduled wider UI sweeps |
+| Capability probes | Chat coherence, web search, file write/read, code execution, skills list, loaded-model identity, context, and ODS Talk/owner-portal surfaces where enabled | Every post-install run, with LLM probes deferred while bootstrap is still active |
+| Lifecycle | Idempotent reinstall, `ods restart`, and `ods doctor` after state changes | Every release-grade run; optional for lighter development runs |
+| Release confidence report | The run is summarized into product, capability, lifecycle, and user-facing gates | Every release-grade run |
+
+`--phase all` is intended for a faster full-product development sweep. The
+private release-grade path adds zero-prereq bootstrap and lifecycle gates so a
+green release run means more than "fresh install worked once."
+
+Run the release-grade fleet after operational code changes: installer phases,
+bootstrap, compose stack generation, service wiring, dashboard/API behavior,
+Hermes, model routing, GPU detection, lifecycle commands, or anything else that
+can affect a user's install or running stack. Docs-only and cosmetic changes can
+usually rely on CI and focused documentation checks.
+
+## Release Confidence Gates
+
+Release-grade runs render a machine-readable confidence summary plus a human
+report. The top-level gates are:
+
+| Gate | Pass signal |
+|---|---|
+| Zero-prereq bootstrap | Bare distro containers can fetch the public installer and provision prerequisites |
+| Install Green | Every enabled hardware host completed a fresh public-path install |
+| Product Green | Core services, cloud contracts, dashboard flows, Hermes auth/chat, and UI checks passed |
+| Capability Green | Full-model capability probes passed, or were explicitly deferred/skipped for documented reasons |
+| Lifecycle Green | Idempotent reinstall, restart, and doctor checks passed on enabled hosts |
+| User Green | The combined release-readiness gate: zero-prereq, install, product, capability, and lifecycle evidence are all clean or explicitly accounted for |
+
+This is why a green release fleet pass is stronger than ordinary service
+health. It exercises the user-visible setup path, first-run behavior,
+post-install actions, real agent work, and lifecycle recovery after state has
+changed.
+
+## Evidence Receipts
+
+Release notes should cite sanitized evidence from the current candidate, not a
+private run directory. A useful receipt includes:
+
+- the ODS commit, tag, or release candidate;
+- the run date;
+- sanitized hardware classes covered;
+- distro lab breadth;
+- regression replay result;
+- install, verify, dashboard, Hermes, UI, capability, and lifecycle summaries;
+- skipped, deferred, blocked-by-environment, or not-run phases;
+- any known gaps that should not be read as supported behavior.
+
+Private artifacts keep detailed logs, JSON events, and host-specific evidence
+inside the lab. Public docs should quote the sanitized summary only, especially
+when a run includes local hostnames, LAN addresses, usernames, or filesystem
+paths.
+
+Treat Windows evidence as release-relevant only when the Windows target produces
+preflight, install, verify, dashboard, and UI artifacts for the candidate being
+claimed. A supported platform can have installer and code support even when it
+is not included in every default private release-fleet run; release notes should
+say which hardware classes actually ran.
+
+## What This Proves
+
+- Installer OS and package-manager logic is exercised across the major Linux
+  package-manager families.
+- The public `curl` bootstrap path is tested from clean Linux images that do
+  not assume developer tools or Docker are already present.
+- Systemd, network, Docker daemon, Docker Compose, and installer dry-run
+  behavior are exercised in disposable Incus VMs for the major Linux families.
+- The installer is repeatedly exercised on real machines, not only CI
+  containers and VMs.
+- The release path covers heterogeneous GPU vendors, memory sizes, operating
+  systems, and CPU architectures.
+- The harness records environment state before install so firewall, Docker,
+  disk, DNS, and port issues can be separated from product bugs.
+- The user-facing path is tested beyond service liveness: dashboard actions,
+  model switching, Hermes auth, agent capabilities, ODS Talk surfaces, and
+  regression fixtures are part of the gate.
+- Lifecycle behavior is part of release-grade confidence: reinstall, restart,
+  and `ods doctor` must recover cleanly after state has changed.
+
+## What This Does Not Claim
+
+- Every Linux distribution is exhaustively installed on real hardware for every
+  change. CI containers and the distro lab cover broad distro logic and
+  systemd/Docker VM behavior; the physical fleet covers the high-value hardware
+  paths.
+- The Incus VM matrix is not GPU validation. GPU runtime claims require real
+  NVIDIA, AMD, Intel, or Apple hardware evidence.
+- OS and distro rotation is periodic because reprovisioning real machines is
+  intentionally slower than running the standing fleet. Release notes should
+  call out any rotated distro or OS image that was included for that candidate.
+- Intel Arc is still experimental unless a release cites a successful Arc fleet
+  run for that release candidate.
+- ODS Talk, LAN, AP-mode, packaged appliance handoff, router, Wi-Fi, mDNS,
+  and client-device behavior require target-mode validation because home
+  networks vary.
+- A fresh fleet pass is not a long-term soak test. Bench, thermal, and
+  overnight stability runs are separate evidence.
+- A green fleet pass is a strong release signal, not a promise that every
+  unsupported driver, storage layout, firewall, network, or optional hardware
+  combination will work perfectly.
+
+## Release Readiness Receipt
+
+Before a release is described as ready, the release notes should cite:
+
+- the ODS version and matching Git tag or release;
+- the fleet run date and sanitized hardware classes covered;
+- regression replay result;
+- zero-prereq bootstrap and distro-lab result summary;
+- install/verify/dashboard/Hermes/UI/capability/lifecycle result summary;
+- any skipped, deferred, blocked-by-environment, or not-run phases;
+- known gaps that should not be read as supported behavior.
+
+The version signal should be internally consistent before publication:
+`manifest.json`, the changelog section, the Git tag/release, and any release
+notes should all name the same version. If a candidate has not been tagged yet,
+describe it as unreleased or release-candidate evidence rather than as a shipped
+stable release.