# Ruflo Verification System A cross-platform, cryptographically-signed regression-protection toolkit. Every documented fix in ruflo gets attested by a SHA-256 + marker substring + Ed25519 signature. Anyone with the same git commit can re-derive the public key and verify it independently. > **Three-layer regression protection:** > 1. **Behavioral smoke tests** (CI) — exercise user-visible failure modes > 2. **Cryptographic witness manifest** (this folder) — attest fix presence > 3. **Temporal history** (`/history.jsonl`) — bisect when a regression was introduced > > See [docs/validation/README.md](../docs/validation/README.md) for the full architecture, [ADR-102](../v3/docs/adr/ADR-102-plugin-hook-cli-flag-regression-ci-guard.md) for the smoke-harness pattern, and [ADR-103](../v3/docs/adr/ADR-103-witness-temporal-history.md) for the temporal-history layer. --- ## Why this exists Three regressions filed on 2026-05-08 (#1859, #1862, #1867) all passed unit tests on the broken commits but broke for users on first install. Unit tests verify code paths; users hit subprocess flag parsers, fresh `npm install` resolution, plugin/CLI version drift. **The witness layer attests that every documented fix's load-bearing code is still present** — even when the fix has no dedicated smoke test. A regression that deletes the load-bearing line of a fix flips its `markerVerified` to `false`. The CI's `witness-verify` job blocks publish. --- ## Folder layout ``` verification/ README.md ← you are here witness-fixes.json ← input config (OS-independent) mcp-tool-baseline.json ← ADR-112 — monotone-decreasing tool-discoverability baseline results.md ← human-readable last-run report inventory.json ← reserved for capability inventory linux/ manifest.md.json ← signed by Linux CI runner history.jsonl ← Linux temporal log macos/ manifest.md.json ← signed by macOS CI runner history.jsonl ← macOS temporal log windows/ manifest.md.json ← signed by Windows CI runner history.jsonl ← Windows temporal log ``` **Why per-OS?** File hashes drift across platforms (LF vs CRLF normalization, path separators, prebuilt binary differences). Each OS produces its own truthful snapshot. The `witness-verify` CI job runs once per OS and writes/verifies that OS's bundle, catching platform-specific marker drift before it reaches users on that platform. **Why JSONL inside each OS folder?** The witness use case is structured key-value retrieval (snapshot N's commit + fix statuses), not vector similarity search. JSONL is git-diff-friendly, append-cheap, and human-readable — the better fit. The folder layout adopts the RVF-style cognitive container concept (per-OS bundle of manifest + history + metadata); the file format inside is JSONL. --- ## Tutorial — adopt the witness in your own project ### 1. Bootstrap The witness toolkit lives in `plugins/ruflo-core/scripts/witness/` — copy it into your project (or install the ruflo-core plugin). ```bash # Initialize per-OS folders + sample fix template node plugins/ruflo-core/scripts/witness/init.mjs --root . # Single runtime dependency npm i @noble/ed25519 ``` Outputs: ``` verification/ witness-fixes.json # template — edit with your fixes / manifest.md.json # empty seed history.jsonl # empty seed ``` ### 2. Register your first fix Open `verification/witness-fixes.json`: ```json { "fixes": [ { "id": "MY-001", "desc": "Race condition in token refresh — early-return on in-flight refresh", "file": "dist/auth.js", "marker": "if (this._refreshing) return this._refreshing;" } ] } ``` **Choosing a marker** is the load-bearing skill: | Bad marker | Why it fails | |---|---| | `'function'` | False-positives in any file with a function | | `'TODO'` | Flaps as TODOs come and go | | `'fix'` | Too generic | | Good marker | Why it works | |---|---| | `(await import('better-sqlite3')).default` | Distinctive AND specific to the fix mechanism | | `(ctx.flags.file as string) \|\| ctx.args[0]` | The exact swap that fixes the bug | | `import * as bcrypt from 'bcryptjs'` | Proves the bcrypt → bcryptjs migration is in dist | Pick a unique substring from the diff that's specifically created by the fix. ### 3. Generate the signed manifest In ruflo's monorepo there's a wrapper that hard-codes paths: ```bash node scripts/regen-witness.mjs ``` For external projects, call the plugin script with explicit paths: ```bash node plugins/ruflo-core/scripts/witness/regen.mjs \ --manifest verification/macos/manifest.md.json \ --history verification/macos/history.jsonl \ --fixes verification/witness-fixes.json ``` Output: ``` witness regen summary ───────────────────── gitCommit: abc123def456… branch: main issuedAt: 2026-05-09T00:00:00.000Z total fixes: 1 (was 0) verified: 1 missing: 0 new entries: MY-001 written: verification/macos/manifest.md.json appended: verification/macos/history.jsonl ``` `verified: N/N` confirms every marker is present in its cited file. ### 4. Verify ```bash node plugins/ruflo-core/scripts/witness/verify.mjs \ --manifest verification/macos/manifest.md.json ``` Output: ``` Manifest signature: hash matches: yes public key reproducible: yes Ed25519 signature valid: yes Summary: pass=1 drift=0 regressed=0 missing=0 ``` If `regressed > 0` or `signatureValid: no`, the manifest has been tampered with or a documented fix's marker has been deleted. ### 5. Investigate a regression When CI reports `F12 regressed`: ```bash node plugins/ruflo-core/scripts/witness/history.mjs \ --history verification/macos/history.jsonl regressions ``` Output: ``` F12 last pass: a1b2c3d4 2026-05-07T14:23:11.000Z regressed at: 9f8e7d6c 2026-05-08T09:14:55.000Z ``` Then `git log lastPass..regressedAt -- ` collapses bisect from "many commits" to "the few in this 18-hour window." ### 6. Wire into CI Add to `.github/workflows/.yml`: ```yaml witness-verify: strategy: matrix: os: [ubuntu-latest, macos-latest, windows-latest] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '22' } - run: npm i @noble/ed25519 - shell: bash run: | case "$RUNNER_OS" in Linux) OS_DIR=linux ;; macOS) OS_DIR=macos ;; Windows) OS_DIR=windows ;; esac node plugins/ruflo-core/scripts/witness/verify.mjs \ --manifest "verification/$OS_DIR/manifest.md.json" \ --json > /tmp/result.json node -e "if (!require('/tmp/result.json').ok) process.exit(1)" ``` Block publish on failure. Every commit on every OS gets attested. --- ## Features | Feature | Mechanism | What it catches | |---|---|---| | **SHA-256 fingerprint per fix** | Per-file hash recorded at issuance | Any change to the file | | **Marker substring** | Distinctive code substring required to be present | Refactors that delete the fix | | **Ed25519 signature** | Deterministic seed `sha256(gitCommit + ':ruflo-witness/v1')` | Manifest tampering, key-derivation drift | | **Reproducible public key** | Anyone with the git commit re-derives the same pubkey | No committed private key needed | | **Per-OS bundles** | `verification/{linux,macos,windows}/` | Platform-specific marker drift (CRLF, path separators) | | **Temporal history (JSONL)** | One snapshot appended per regen | When a regression was introduced | | **Drift vs regression** | sha256 changed but marker present → drift; marker missing → regression | False alarms suppressed; real regressions flagged | | **Status timeline per fix** | `history.mjs timeline --id F12` | Flapping markers (too brittle) and persistent drift (too unstable) | --- ## Daily workflow cheat sheet ```bash # Add a new fix $EDITOR verification/witness-fixes.json node scripts/regen-witness.mjs # Verify the current tree node plugins/ruflo-core/scripts/witness/verify.mjs \ --manifest verification/macos/manifest.md.json # Diff against the previous snapshot node plugins/ruflo-core/scripts/witness/history.mjs \ --history verification/macos/history.jsonl summary # Find when a regression was introduced node plugins/ruflo-core/scripts/witness/history.mjs \ --history verification/macos/history.jsonl regressions # Status timeline for one fix node plugins/ruflo-core/scripts/witness/history.mjs \ --history verification/macos/history.jsonl timeline --id F12 ``` --- ## Schema ### `/manifest.md.json` ```jsonc { "manifest": { "schema": "ruflo-witness/v1", "issuedAt": "2026-05-09T00:00:00.000Z", "gitCommit": "", "branch": "main", "os": "macos", // matches the folder "releases": { "ruflo": "3.7.0-alpha.21", "@claude-flow/plugin-agent-federation": "1.0.0-alpha.15" }, "summary": { "totalFixes": 102, "verified": 102, "missing": 0 }, "fixes": [ { "id": "F1", "desc": "...", "file": "v3/@claude-flow/cli/dist/...", "sha256": "<64 hex>", "marker": "", "markerVerified": true } ] }, "integrity": { "manifestHashAlgo": "sha256", "manifestHash": "<64 hex of canonical manifest>", "signature": "<64-byte hex of Ed25519 signature over canonical manifest>", "signatureAlgo": "ed25519", "publicKey": "<32-byte hex>", "signature": "<64-byte hex>", "seedDerivation": "sha256(gitCommit + ':ruflo-witness/v1')" } } ``` ### `/history.jsonl` (one entry per line) ```jsonc { "v": 1, "commit": "", "issuedAt": "2026-05-09T00:00:00.000Z", "branch": "main", "os": "macos", "manifestHash": "", "summary": { "totalFixes": 82, "verified": 82, "missing": 0 }, "fixes": { "F1": { "sha256": "...", "markerVerified": true }, "#1867": { "sha256": "...", "markerVerified": true } } } ``` ### `witness-fixes.json` (input, OS-independent) ```jsonc { "fixes": [ { "id": "MY-001", "desc": "human-readable description", "file": "path/relative/to/repo/root.js", "marker": "distinctive substring that must remain in the file" } ] } ``` --- ## Anti-patterns - **Hand-editing `manifest.md.json`** — breaks the signature. Always regenerate via `regen.mjs`. - **Markers that match generic patterns** (`'function'`, `'import'`, `'TODO'`) — false positives or flapping. - **Skipping the history append** — without `--history`, you lose Layer 3 (regression introduction time). - **Committing the manifest without the history** — they belong in the same commit. - **Cross-OS manifest editing** — the manifest is signed for the OS that generated it. Don't copy a Linux manifest into the macOS folder. --- ## How it integrates with the rest of ruflo's regression stack | Layer | Where | What it catches | |---|---|---| | Layer 1 — install smoke | `v3/@claude-flow/memory/scripts/smoke-no-bsqlite.mjs` + CI `smoke-install-no-bsqlite` | `npm install` failures on platforms without prebuilds | | Layer 1 — hook smoke | `plugins/ruflo-core/scripts/test-hooks.mjs` + CI `plugin-hooks-smoke` | Plugin/CLI flag drift, parser ambiguity | | Layer 1 — MCP protocol smoke | `plugins/ruflo-core/scripts/test-mcp-protocol.mjs` + CI `mcp-protocol-smoke` | HTTP MCP wire-format compliance | | Layer 1 — memory-import smoke | `plugins/ruflo-core/scripts/test-memory-import.mjs` + CI `memory-import-smoke` | Memory_import_claude WSL path + key sanitization regressions | | Layer 1 — tool-description audit | `scripts/audit-tool-descriptions.mjs` + CI `tool-descriptions-audit` (ADR-112) | Every MCP tool description must have "Use when … is wrong because …" guidance + length ≥ 80 + unique. Baseline at `verification/mcp-tool-baseline.json` monotone-decreasing | | **Layer 2 — witness manifest** | **`verification//manifest.md.json`** + CI `witness-verify` | **Documented fix marker disappearing** | | Layer 3 — temporal history | `verification//history.jsonl` + `history.mjs` | When a regression was introduced | All CI jobs gate `publish`. A regression in any layer blocks the release on the platform where it was caught. --- ## References - [docs/validation/README.md](../docs/validation/README.md) — full architecture overview (in-repo) - [Public gist](https://gist.github.com/ruvnet/ee7763c36f7a9a1c1886da783abc872b) — external-friendly version - [ADR-102](../v3/docs/adr/ADR-102-plugin-hook-cli-flag-regression-ci-guard.md) — smoke harness pattern + flag-priority CLI convention - [ADR-103](../v3/docs/adr/ADR-103-witness-temporal-history.md) — JSONL history layer + plugin distribution - [`plugins/ruflo-core/skills/witness/SKILL.md`](../plugins/ruflo-core/skills/witness/SKILL.md) — Claude Code skill workflow - [`plugins/ruflo-core/agents/witness-curator.md`](../plugins/ruflo-core/agents/witness-curator.md) — curator agent definition - [`.github/workflows/v3-ci.yml`](../.github/workflows/v3-ci.yml) — `smoke-install-no-bsqlite`, `plugin-hooks-smoke`, `witness-verify` jobs