chore: import upstream snapshot with attribution
Firmware QEMU Tests (ADR-061) / QEMU Test (edge-tier1) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (full-adr060) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (tdm-3node) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / Swarm Test (ADR-062) (push) Has been skipped
npm packages / tools/ruview-mcp (node 22) (push) Failing after 1s
nvsim-server → ghcr.io / build-and-publish (push) Failing after 1s
ruview-swarm CI guard / tests (full+train) (push) Failing after 2s
Bench Regression Guard / bench compile-verify (--no-run) (push) Failing after 0s
Bench Regression Guard / bench fast-run (informational, non-gating) (push) Has been skipped
Firmware CI / Verify version.txt matches release tag (push) Has been skipped
Dashboard a11y + cross-browser / a11y (push) Failing after 0s
nvsim Dashboard → GitHub Pages / build-and-deploy (push) Failing after 2s
Firmware CI / Build firmware (esp32s3 / 4mb) (push) Failing after 15s
Firmware QEMU Tests (ADR-061) / Build Espressif QEMU (push) Failing after 1s
Firmware QEMU Tests (ADR-061) / Fuzz Testing (ADR-061 Layer 6) (push) Failing after 1s
Continuous Deployment / Pre-deployment Checks (push) Has been skipped
Firmware CI / Build firmware (esp32c6 / c6-4mb) (push) Failing after 15s
Firmware CI / Build firmware (esp32s3 / 8mb) (push) Failing after 15s
Firmware QEMU Tests (ADR-061) / QEMU Test (boundary-max) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (boundary-min) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (default) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (edge-tier0) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / NVS Matrix Generation (push) Failing after 1s
Security Scanning / Security Policy Compliance (push) Failing after 0s
Security Scanning / Dependency Vulnerability Scan (push) Failing after 0s
Security Scanning / Static Application Security Testing (push) Failing after 1s
Security Scanning / Infrastructure Security Scan (push) Failing after 1s
Security Scanning / Secret Scanning (push) Failing after 1s
npm packages / harness/ruview (node 22) (push) Failing after 17s
Security Scanning / License Compliance Scan (push) Failing after 1s
Security Scanning / Container Security Scan (push) Failing after 4s
three.js demos → GitHub Pages / build-and-deploy (push) Failing after 1s
Verify Pipeline Determinism / Verify Pipeline Determinism (3.11) (push) Failing after 1s
Fix-Marker Regression Guard / Verify fix markers (push) Failing after 1s
ADR-115 MQTT integration tests / mqtt-integration (push) Failing after 1s
npm packages / harness/ruview (node 20) (push) Failing after 1s
npm packages / tools/ruview-mcp (node 20) (push) Failing after 1s
npm packages / tools/ruview-cli (node 20) (push) Failing after 1s
npm packages / tools/ruview-cli (node 22) (push) Failing after 1s
BFLD MQTT Integration / cargo test --features mqtt (live mosquitto) (push) Failing after 29s
ruview-swarm CI guard / build train_marl bin (push) Failing after 2s
ruview-swarm CI guard / clippy (-D warnings, --no-deps) (push) Failing after 3s
ruview-swarm CI guard / tests (ruflo) (push) Failing after 1s
ruview-swarm CI guard / tests (train) (push) Failing after 2s
ruview-swarm CI guard / tests (default) (push) Failing after 2s
Point Cloud Viewer → GitHub Pages / build-and-deploy (push) Failing after 8s
ruview-swarm CI guard / ITAR / publish guard (push) Failing after 0s
wifi-densepose sensing-server → Docker Hub + ghcr.io / build · push · smoke-test (push) Failing after 1s
Continuous Deployment / Deploy to Production (push) Has been cancelled
Continuous Deployment / Rollback Deployment (push) Has been cancelled
Continuous Deployment / Post-deployment Monitoring (push) Has been cancelled
Continuous Deployment / Notify Deployment Status (push) Has been cancelled
Continuous Deployment / Deploy to Staging (push) Has been cancelled
Security Scanning / Security Report (push) Has been cancelled
Firmware QEMU Tests (ADR-061) / QEMU Test (edge-tier1) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (full-adr060) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (tdm-3node) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / Swarm Test (ADR-062) (push) Has been skipped
npm packages / tools/ruview-mcp (node 22) (push) Failing after 1s
nvsim-server → ghcr.io / build-and-publish (push) Failing after 1s
ruview-swarm CI guard / tests (full+train) (push) Failing after 2s
Bench Regression Guard / bench compile-verify (--no-run) (push) Failing after 0s
Bench Regression Guard / bench fast-run (informational, non-gating) (push) Has been skipped
Firmware CI / Verify version.txt matches release tag (push) Has been skipped
Dashboard a11y + cross-browser / a11y (push) Failing after 0s
nvsim Dashboard → GitHub Pages / build-and-deploy (push) Failing after 2s
Firmware CI / Build firmware (esp32s3 / 4mb) (push) Failing after 15s
Firmware QEMU Tests (ADR-061) / Build Espressif QEMU (push) Failing after 1s
Firmware QEMU Tests (ADR-061) / Fuzz Testing (ADR-061 Layer 6) (push) Failing after 1s
Continuous Deployment / Pre-deployment Checks (push) Has been skipped
Firmware CI / Build firmware (esp32c6 / c6-4mb) (push) Failing after 15s
Firmware CI / Build firmware (esp32s3 / 8mb) (push) Failing after 15s
Firmware QEMU Tests (ADR-061) / QEMU Test (boundary-max) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (boundary-min) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (default) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / QEMU Test (edge-tier0) (push) Has been skipped
Firmware QEMU Tests (ADR-061) / NVS Matrix Generation (push) Failing after 1s
Security Scanning / Security Policy Compliance (push) Failing after 0s
Security Scanning / Dependency Vulnerability Scan (push) Failing after 0s
Security Scanning / Static Application Security Testing (push) Failing after 1s
Security Scanning / Infrastructure Security Scan (push) Failing after 1s
Security Scanning / Secret Scanning (push) Failing after 1s
npm packages / harness/ruview (node 22) (push) Failing after 17s
Security Scanning / License Compliance Scan (push) Failing after 1s
Security Scanning / Container Security Scan (push) Failing after 4s
three.js demos → GitHub Pages / build-and-deploy (push) Failing after 1s
Verify Pipeline Determinism / Verify Pipeline Determinism (3.11) (push) Failing after 1s
Fix-Marker Regression Guard / Verify fix markers (push) Failing after 1s
ADR-115 MQTT integration tests / mqtt-integration (push) Failing after 1s
npm packages / harness/ruview (node 20) (push) Failing after 1s
npm packages / tools/ruview-mcp (node 20) (push) Failing after 1s
npm packages / tools/ruview-cli (node 20) (push) Failing after 1s
npm packages / tools/ruview-cli (node 22) (push) Failing after 1s
BFLD MQTT Integration / cargo test --features mqtt (live mosquitto) (push) Failing after 29s
ruview-swarm CI guard / build train_marl bin (push) Failing after 2s
ruview-swarm CI guard / clippy (-D warnings, --no-deps) (push) Failing after 3s
ruview-swarm CI guard / tests (ruflo) (push) Failing after 1s
ruview-swarm CI guard / tests (train) (push) Failing after 2s
ruview-swarm CI guard / tests (default) (push) Failing after 2s
Point Cloud Viewer → GitHub Pages / build-and-deploy (push) Failing after 8s
ruview-swarm CI guard / ITAR / publish guard (push) Failing after 0s
wifi-densepose sensing-server → Docker Hub + ghcr.io / build · push · smoke-test (push) Failing after 1s
Continuous Deployment / Deploy to Production (push) Has been cancelled
Continuous Deployment / Rollback Deployment (push) Has been cancelled
Continuous Deployment / Post-deployment Monitoring (push) Has been cancelled
Continuous Deployment / Notify Deployment Status (push) Has been cancelled
Continuous Deployment / Deploy to Staging (push) Has been cancelled
Security Scanning / Security Report (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,96 @@
|
||||
name: AetherArena harness gate (ADR-149)
|
||||
|
||||
# Runs the AetherArena scoring harness as a PR build gate. Every PR that touches
|
||||
# the scorer, the metrics, or the benchmark scaffold must keep the deterministic
|
||||
# score hash stable (ADR-149 §2.5 determinism_gate). If the scoring maths changes,
|
||||
# the hash moves and this gate fails until `expected_score.sha256` is regenerated
|
||||
# and reviewed — so scorer drift can never land silently.
|
||||
#
|
||||
# This is the "a PR that runs the harness as part of the build process" requirement.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'v2/crates/wifi-densepose-train/src/ruview_metrics.rs'
|
||||
- 'v2/crates/wifi-densepose-train/src/ablation.rs'
|
||||
- 'v2/crates/wifi-densepose-train/src/bin/aa_score_runner.rs'
|
||||
- 'aether-arena/**'
|
||||
- '.github/workflows/aether-arena-harness.yml'
|
||||
push:
|
||||
branches: ['feat/adr-149-aether-arena']
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
harness-gate:
|
||||
name: Run AA scorer harness (determinism gate)
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: v2
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install Rust toolchain
|
||||
run: rustup show && rustc --version
|
||||
|
||||
- name: Cache cargo
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: aa-harness-${{ runner.os }}-${{ hashFiles('v2/Cargo.lock') }}
|
||||
|
||||
# 1. Build the pure-Rust scorer (no torch / no GPU → fast PR gate).
|
||||
- name: Build AA score runner
|
||||
run: cargo build -p wifi-densepose-train --bin aa_score_runner --no-default-features
|
||||
|
||||
# 2. Determinism gate: the committed expected hash must still match. A
|
||||
# non-zero exit here fails the PR.
|
||||
- name: Run determinism gate
|
||||
run: cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features
|
||||
|
||||
# 3. Repeatability analysis (witness chain): the harness must produce one
|
||||
# identical proof hash across many runs — any nondeterminism fails here.
|
||||
- name: Repeatability analysis (16 runs)
|
||||
run: cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --repeat 16
|
||||
|
||||
# 4. Real-scoring smoke: score a sample prediction against the public smoke
|
||||
# split, exercising the actual model-scoring path (not just the fixture).
|
||||
- name: Real-scoring smoke test
|
||||
run: |
|
||||
cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- \
|
||||
--split ../aether-arena/fixtures/smoke_split.json \
|
||||
--pred ../aether-arena/fixtures/smoke_pred.json --json
|
||||
|
||||
# 5. Witness ledger chain integrity: the append-only results ledger must
|
||||
# verify (every prev_hash link + row_hash intact = no silent edits).
|
||||
- name: Verify witness ledger chain
|
||||
working-directory: aether-arena/ledger
|
||||
run: python3 ledger_tools.py verify
|
||||
|
||||
# 6. Emit the witness row + repeatability into the PR run summary.
|
||||
- name: Witness row → job summary
|
||||
if: always()
|
||||
run: |
|
||||
ROW=$(cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --json)
|
||||
REP=$(cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --repeat 16)
|
||||
{
|
||||
echo "## AetherArena harness gate (witness chain)"
|
||||
echo ""
|
||||
echo "Deterministic witness (ADR-149 §2.2 / proof + repeatability):"
|
||||
echo '```json'
|
||||
echo "$ROW"
|
||||
echo "$REP"
|
||||
echo '```'
|
||||
echo ""
|
||||
echo "If the determinism gate failed, the scoring maths changed: regenerate with"
|
||||
echo '`cargo run -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --generate-hash > aether-arena/fixtures/expected_score.sha256` and review the diff.'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -0,0 +1,199 @@
|
||||
name: Bench Regression Guard
|
||||
|
||||
# Sub-deliverable 8.3 of the benchmark/optimization milestone.
|
||||
#
|
||||
# HONEST SCOPE (read this before assuming this gates on timing):
|
||||
# * The `bench-compile` job is a REAL, HARD-FAILING regression gate. It runs
|
||||
# `cargo bench --no-default-features --no-run`, which type-checks and links
|
||||
# EVERY criterion bench in the v2/ workspace without running a single
|
||||
# measurement. Benches are not part of `cargo test`, so they silently
|
||||
# bit-rot when a public API they call changes — this job catches that the
|
||||
# moment it happens. This is the part of this workflow that can fail a PR.
|
||||
#
|
||||
# * The `bench-fast-run` job runs a small, curated subset of pure-CPU benches
|
||||
# in criterion "quick mode" (short warm-up / measurement / 10 samples) and
|
||||
# is INFORMATIONAL ONLY (`continue-on-error: true`). It does NOT gate on
|
||||
# timing. Wall-clock timings on shared GitHub-hosted runners vary by
|
||||
# 2-3x run-to-run (noisy neighbours, CPU throttling, no pinned frequency),
|
||||
# so a hard ">X ms" threshold here would flake constantly and teach
|
||||
# everyone to ignore it. We deliberately do not pretend to do timing
|
||||
# regression-gating we cannot deliver reliably. The numbers are surfaced in
|
||||
# the job log + uploaded as an artifact for humans to eyeball trends.
|
||||
#
|
||||
# WHY NO criterion --baseline COMPARE GATE:
|
||||
# criterion's `--save-baseline` / `--baseline` compare is the textbook
|
||||
# regression mechanism, but it only produces a trustworthy verdict when the
|
||||
# baseline and the candidate were measured on the SAME hardware under the SAME
|
||||
# conditions. GitHub-hosted runners give neither (the baseline commit and the
|
||||
# PR commit land on different physical machines). Committing a baseline JSON
|
||||
# measured on one runner and comparing a different runner against it would
|
||||
# manufacture false regressions. If/when these benches run on a dedicated,
|
||||
# frequency-pinned self-hosted runner, a `--baseline` compare with a generous
|
||||
# (>2x) noise floor becomes honest and can be added then. Until then,
|
||||
# compile-verify + informational-run is the honest gate.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, develop, 'feat/*' ]
|
||||
paths:
|
||||
- 'v2/crates/**/benches/**'
|
||||
- 'v2/crates/**/Cargo.toml'
|
||||
- 'v2/crates/**/src/**'
|
||||
- 'v2/Cargo.toml'
|
||||
- 'v2/Cargo.lock'
|
||||
- '.github/workflows/bench-regression.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'v2/crates/**/benches/**'
|
||||
- 'v2/crates/**/Cargo.toml'
|
||||
- 'v2/crates/**/src/**'
|
||||
- 'v2/Cargo.toml'
|
||||
- 'v2/Cargo.lock'
|
||||
- '.github/workflows/bench-regression.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
# Debuginfo is useless in CI and the 38-crate workspace target dir otherwise
|
||||
# exhausts the runner disk (mirrors ci.yml's rust-tests job). The bench
|
||||
# profile inherits release + debug = true (v2/Cargo.toml [profile.bench]);
|
||||
# force it off so the link step does not run out of space.
|
||||
CARGO_PROFILE_BENCH_DEBUG: "0"
|
||||
CARGO_PROFILE_RELEASE_DEBUG: "0"
|
||||
|
||||
jobs:
|
||||
# ── HARD GATE: every bench must still compile + link ─────────────────────
|
||||
bench-compile:
|
||||
name: bench compile-verify (--no-run)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout (recursive — wifi-densepose-rufield path-deps vendor/rufield)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
# The workspace includes `wifi-densepose-rufield`, which path-deps the
|
||||
# `vendor/rufield` submodule crates. Without a recursive checkout the
|
||||
# whole workspace fails to resolve before any bench is built.
|
||||
submodules: recursive
|
||||
|
||||
# The workspace pulls in `wifi-densepose-desktop` (Tauri v2) whose -sys
|
||||
# crates need the GTK/WebKit/serial dev libraries via pkg-config, exactly
|
||||
# as ci.yml's rust-tests job documents. A `--workspace` bench build links
|
||||
# the whole graph, so these are required here too.
|
||||
- name: Install Tauri / GTK / serial system dev libraries
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y --no-install-recommends \
|
||||
libglib2.0-dev \
|
||||
libgtk-3-dev \
|
||||
libsoup-3.0-dev \
|
||||
libjavascriptcoregtk-4.1-dev \
|
||||
libwebkit2gtk-4.1-dev \
|
||||
libayatana-appindicator3-dev \
|
||||
librsvg2-dev \
|
||||
libxdo-dev \
|
||||
libudev-dev \
|
||||
libdbus-1-dev \
|
||||
libssl-dev \
|
||||
pkg-config
|
||||
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Cache cargo (Swatinem/rust-cache)
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: v2
|
||||
# Distinct cache scope from ci.yml's rust-tests so the bench profile
|
||||
# artifacts (release+opt) do not evict the test profile cache.
|
||||
key: bench-regression
|
||||
|
||||
# The core regression guard. `--no-run` compiles + links every bench
|
||||
# target in the workspace's DEFAULT feature set but runs no measurement,
|
||||
# so it is deterministic and fast-ish (build only). A bench that no longer
|
||||
# compiles — because a type/signature it calls changed and nobody updated
|
||||
# the bench — fails the build here. `--no-default-features` is the
|
||||
# workspace's standard gate flag (openblas/tch/ort/onnx stay opt-out).
|
||||
- name: Compile all workspace benches (default features)
|
||||
working-directory: v2
|
||||
run: cargo bench --workspace --no-default-features --no-run
|
||||
|
||||
# Feature-gated benches are skipped by the default build above because
|
||||
# their `[[bench]]` entries carry `required-features`. Compile the ones we
|
||||
# can guard so they are also covered against bit-rot.
|
||||
# * cir → wifi-densepose-signal/benches/cir_bench.rs (ADR-134). The
|
||||
# `cir` feature is pure-Rust (`cir = []`), so it builds on the stock
|
||||
# runner and is a real, hard-failing guard like the step above.
|
||||
#
|
||||
# NOT guarded here (honest scope):
|
||||
# * crv → wifi-densepose-ruvector/benches/crv_bench.rs. The `crv` feature
|
||||
# pulls the crates.io dependency `ruvector-crv 0.1.1`, which currently
|
||||
# FAILS to compile on stable (E0308 type mismatch in its own
|
||||
# `stage_iii.rs` — an UPSTREAM bug, unrelated to bench bit-rot).
|
||||
# Adding a hard `--features crv` compile step would make this workflow
|
||||
# red for a reason this gate is not meant to police. Re-add this step
|
||||
# once `ruvector-crv` ships a fixed release. (mqtt/onnx benches are
|
||||
# likewise left to their own crate workflows.)
|
||||
- name: Compile feature-gated benches (cir)
|
||||
working-directory: v2
|
||||
run: cargo bench -p wifi-densepose-signal --no-default-features --features cir --bench cir_bench --no-run
|
||||
|
||||
# ── INFORMATIONAL: run a curated fast subset (never gates) ───────────────
|
||||
bench-fast-run:
|
||||
name: bench fast-run (informational, non-gating)
|
||||
runs-on: ubuntu-latest
|
||||
# NEVER fail the workflow on this job — timings are noise-prone on shared
|
||||
# runners (see header). It exists to surface trends for humans, not to gate.
|
||||
continue-on-error: true
|
||||
needs: [bench-compile]
|
||||
steps:
|
||||
- name: Checkout (recursive)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Cache cargo (Swatinem/rust-cache)
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: v2
|
||||
key: bench-regression
|
||||
|
||||
# Curated subset = pure-CPU, fast, dependency-light criterion benches that
|
||||
# finish in seconds under quick-mode flags. Each is targeted by `--bench`
|
||||
# (NOT a bare `cargo bench -p`) because the crates' lib targets use the
|
||||
# libtest harness, which rejects criterion's CLI flags (--warm-up-time
|
||||
# etc.) and aborts the run. Quick-mode: 1s warm-up, 2s measure, 10 samples.
|
||||
- name: nvsim pipeline_throughput (quick)
|
||||
working-directory: v2
|
||||
run: |
|
||||
mkdir -p ../bench-out
|
||||
cargo bench -p nvsim --no-default-features --bench pipeline_throughput -- \
|
||||
--warm-up-time 1 --measurement-time 2 --sample-size 10 \
|
||||
| tee ../bench-out/nvsim_pipeline_throughput.txt
|
||||
|
||||
- name: ruvector sketch_bench (quick)
|
||||
working-directory: v2
|
||||
run: |
|
||||
cargo bench -p wifi-densepose-ruvector --no-default-features --bench sketch_bench -- \
|
||||
--warm-up-time 1 --measurement-time 2 --sample-size 10 \
|
||||
| tee ../bench-out/ruvector_sketch_bench.txt
|
||||
|
||||
- name: ruvector fusion_bench (quick)
|
||||
working-directory: v2
|
||||
run: |
|
||||
cargo bench -p wifi-densepose-ruvector --no-default-features --bench fusion_bench -- \
|
||||
--warm-up-time 1 --measurement-time 2 --sample-size 10 \
|
||||
| tee ../bench-out/ruvector_fusion_bench.txt
|
||||
|
||||
- name: Upload informational bench logs
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: bench-fast-run-logs
|
||||
path: bench-out/
|
||||
if-no-files-found: warn
|
||||
@@ -0,0 +1,101 @@
|
||||
name: BFLD MQTT Integration
|
||||
|
||||
# Runs the env-gated mosquitto integration tests from iters 24 + 29 of the
|
||||
# BFLD rollout (ADR-118 / ADR-122 §2.2). Spins up an eclipse-mosquitto:2
|
||||
# service container, exports BFLD_MQTT_BROKER, runs `cargo test --features
|
||||
# mqtt`. Local developers can reproduce with:
|
||||
#
|
||||
# scoop install mosquitto # Windows
|
||||
# # or: docker run -p 1883:1883 eclipse-mosquitto:2
|
||||
# BFLD_MQTT_BROKER=tcp://localhost:1883 \
|
||||
# cargo test -p wifi-densepose-bfld --features mqtt
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- 'feat/adr-118-*'
|
||||
- 'feat/bfld-*'
|
||||
paths:
|
||||
- 'v2/crates/wifi-densepose-bfld/**'
|
||||
- '.github/workflows/bfld-mqtt-integration.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'v2/crates/wifi-densepose-bfld/**'
|
||||
- '.github/workflows/bfld-mqtt-integration.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
mqtt-live-broker:
|
||||
name: cargo test --features mqtt (live mosquitto)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
|
||||
services:
|
||||
mosquitto:
|
||||
image: eclipse-mosquitto:2
|
||||
ports:
|
||||
- 1883:1883
|
||||
# Allow anonymous connections — local-only CI broker, no exposure
|
||||
# to the public internet, never touches production credentials.
|
||||
options: >-
|
||||
--health-cmd "mosquitto_pub -h localhost -t healthcheck -m ping || exit 1"
|
||||
--health-interval 5s
|
||||
--health-timeout 3s
|
||||
--health-retries 10
|
||||
|
||||
env:
|
||||
BFLD_MQTT_BROKER: tcp://localhost:1883
|
||||
CARGO_TERM_COLOR: always
|
||||
CARGO_INCREMENTAL: 0
|
||||
RUSTFLAGS: -D warnings
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
components: clippy
|
||||
|
||||
- name: Cache cargo registry + target
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: bfld-mqtt-${{ runner.os }}-${{ hashFiles('v2/Cargo.lock') }}
|
||||
|
||||
- name: Wait for mosquitto to be ready
|
||||
run: |
|
||||
for i in {1..20}; do
|
||||
if nc -z localhost 1883; then
|
||||
echo "mosquitto reachable on port 1883 (attempt $i)"
|
||||
exit 0
|
||||
fi
|
||||
echo "waiting for mosquitto ($i/20)..."
|
||||
sleep 1
|
||||
done
|
||||
echo "mosquitto never became reachable" >&2
|
||||
exit 1
|
||||
|
||||
- name: cargo test --no-default-features (baseline regression)
|
||||
working-directory: v2
|
||||
run: cargo test -p wifi-densepose-bfld --no-default-features
|
||||
|
||||
- name: cargo test (default features)
|
||||
working-directory: v2
|
||||
run: cargo test -p wifi-densepose-bfld
|
||||
|
||||
- name: cargo test --features mqtt (incl. live mosquitto roundtrip)
|
||||
working-directory: v2
|
||||
run: cargo test -p wifi-densepose-bfld --features mqtt
|
||||
|
||||
- name: cargo clippy --features mqtt (lint gate)
|
||||
working-directory: v2
|
||||
run: cargo clippy -p wifi-densepose-bfld --features mqtt --all-targets -- -D warnings
|
||||
continue-on-error: true
|
||||
@@ -0,0 +1,358 @@
|
||||
name: Continuous Deployment
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main ]
|
||||
tags: [ 'v*' ]
|
||||
workflow_run:
|
||||
workflows: ["Continuous Integration"]
|
||||
types:
|
||||
- completed
|
||||
branches: [ main ]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
environment:
|
||||
description: 'Deployment environment'
|
||||
required: true
|
||||
default: 'staging'
|
||||
type: choice
|
||||
options:
|
||||
- staging
|
||||
- production
|
||||
force_deploy:
|
||||
description: 'Force deployment (skip checks)'
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
KUBE_CONFIG_DATA: ${{ secrets.KUBE_CONFIG_DATA }}
|
||||
|
||||
jobs:
|
||||
# Pre-deployment checks
|
||||
pre-deployment:
|
||||
name: Pre-deployment Checks
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event.workflow_run.conclusion == 'success' || github.event_name == 'workflow_dispatch'
|
||||
outputs:
|
||||
deploy_env: ${{ steps.determine-env.outputs.environment }}
|
||||
image_tag: ${{ steps.determine-tag.outputs.tag }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Determine deployment environment
|
||||
id: determine-env
|
||||
env:
|
||||
# Use environment variable to prevent shell injection
|
||||
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
||||
GITHUB_REF: ${{ github.ref }}
|
||||
GITHUB_INPUT_ENVIRONMENT: ${{ github.event.inputs.environment }}
|
||||
run: |
|
||||
if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" ]]; then
|
||||
echo "environment=$GITHUB_INPUT_ENVIRONMENT" >> $GITHUB_OUTPUT
|
||||
elif [[ "$GITHUB_REF" == "refs/heads/main" ]]; then
|
||||
echo "environment=staging" >> $GITHUB_OUTPUT
|
||||
elif [[ "$GITHUB_REF" == refs/tags/v* ]]; then
|
||||
echo "environment=production" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "environment=staging" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Determine image tag
|
||||
id: determine-tag
|
||||
run: |
|
||||
if [[ "${{ github.ref }}" == refs/tags/v* ]]; then
|
||||
echo "tag=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "tag=${{ github.sha }}" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Verify image exists
|
||||
run: |
|
||||
docker manifest inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.determine-tag.outputs.tag }}
|
||||
|
||||
# Deploy to staging
|
||||
deploy-staging:
|
||||
name: Deploy to Staging
|
||||
runs-on: ubuntu-latest
|
||||
needs: [pre-deployment]
|
||||
if: needs.pre-deployment.outputs.deploy_env == 'staging'
|
||||
environment:
|
||||
name: staging
|
||||
url: https://staging.wifi-densepose.com
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up kubectl
|
||||
uses: azure/setup-kubectl@v3
|
||||
with:
|
||||
version: 'v1.28.0'
|
||||
|
||||
- name: Configure kubectl
|
||||
run: |
|
||||
echo "${{ secrets.KUBE_CONFIG_DATA_STAGING }}" | base64 -d > kubeconfig
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
- name: Deploy to staging namespace
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Update image tag in deployment
|
||||
kubectl set image deployment/wifi-densepose wifi-densepose=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.pre-deployment.outputs.image_tag }} -n wifi-densepose-staging
|
||||
|
||||
# Wait for rollout to complete
|
||||
kubectl rollout status deployment/wifi-densepose -n wifi-densepose-staging --timeout=600s
|
||||
|
||||
# Verify deployment
|
||||
kubectl get pods -n wifi-densepose-staging -l app=wifi-densepose
|
||||
|
||||
- name: Run smoke tests
|
||||
run: |
|
||||
sleep 30
|
||||
curl -f https://staging.wifi-densepose.com/health || exit 1
|
||||
curl -f https://staging.wifi-densepose.com/api/v1/info || exit 1
|
||||
|
||||
- name: Run integration tests against staging
|
||||
run: |
|
||||
python -m pytest tests/integration/ --base-url=https://staging.wifi-densepose.com -v
|
||||
|
||||
# Deploy to production
|
||||
deploy-production:
|
||||
name: Deploy to Production
|
||||
runs-on: ubuntu-latest
|
||||
needs: [pre-deployment, deploy-staging]
|
||||
if: needs.pre-deployment.outputs.deploy_env == 'production' || (github.ref == 'refs/tags/v*' && needs.deploy-staging.result == 'success')
|
||||
environment:
|
||||
name: production
|
||||
url: https://wifi-densepose.com
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up kubectl
|
||||
uses: azure/setup-kubectl@v3
|
||||
with:
|
||||
version: 'v1.28.0'
|
||||
|
||||
- name: Configure kubectl
|
||||
run: |
|
||||
echo "${{ secrets.KUBE_CONFIG_DATA_PRODUCTION }}" | base64 -d > kubeconfig
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
- name: Pre-deployment backup
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Backup current deployment
|
||||
kubectl get deployment wifi-densepose -n wifi-densepose -o yaml > backup-deployment.yaml
|
||||
|
||||
# Backup database
|
||||
kubectl exec -n wifi-densepose deployment/postgres -- pg_dump -U wifi_user wifi_densepose > backup-db.sql
|
||||
|
||||
- name: Blue-Green Deployment
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Create green deployment
|
||||
kubectl patch deployment wifi-densepose -n wifi-densepose -p '{"spec":{"template":{"metadata":{"labels":{"version":"green"}}}}}'
|
||||
kubectl set image deployment/wifi-densepose wifi-densepose=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.pre-deployment.outputs.image_tag }} -n wifi-densepose
|
||||
|
||||
# Wait for green deployment to be ready
|
||||
kubectl rollout status deployment/wifi-densepose -n wifi-densepose --timeout=600s
|
||||
|
||||
# Verify green deployment health
|
||||
kubectl wait --for=condition=ready pod -l app=wifi-densepose,version=green -n wifi-densepose --timeout=300s
|
||||
|
||||
- name: Traffic switching validation
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Get green pod IP for direct testing
|
||||
GREEN_POD=$(kubectl get pods -n wifi-densepose -l app=wifi-densepose,version=green -o jsonpath='{.items[0].metadata.name}')
|
||||
|
||||
# Test green deployment directly
|
||||
kubectl exec -n wifi-densepose $GREEN_POD -- curl -f http://localhost:8000/health
|
||||
kubectl exec -n wifi-densepose $GREEN_POD -- curl -f http://localhost:8000/api/v1/info
|
||||
|
||||
- name: Switch traffic to green
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Update service selector to point to green
|
||||
kubectl patch service wifi-densepose-service -n wifi-densepose -p '{"spec":{"selector":{"version":"green"}}}'
|
||||
|
||||
# Wait for traffic switch
|
||||
sleep 30
|
||||
|
||||
- name: Production smoke tests
|
||||
run: |
|
||||
curl -f https://wifi-densepose.com/health || exit 1
|
||||
curl -f https://wifi-densepose.com/api/v1/info || exit 1
|
||||
|
||||
- name: Cleanup old deployment
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Remove blue version label from old pods
|
||||
kubectl label pods -n wifi-densepose -l app=wifi-densepose,version!=green version-
|
||||
|
||||
# Scale down old replica set (optional)
|
||||
# kubectl scale rs -n wifi-densepose -l app=wifi-densepose,version!=green --replicas=0
|
||||
|
||||
- name: Upload deployment artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: production-deployment-${{ github.run_number }}
|
||||
path: |
|
||||
backup-deployment.yaml
|
||||
backup-db.sql
|
||||
|
||||
# Rollback capability
|
||||
rollback:
|
||||
name: Rollback Deployment
|
||||
runs-on: ubuntu-latest
|
||||
if: failure() && (needs.deploy-staging.result == 'failure' || needs.deploy-production.result == 'failure')
|
||||
needs: [pre-deployment, deploy-staging, deploy-production]
|
||||
environment:
|
||||
name: ${{ needs.pre-deployment.outputs.deploy_env }}
|
||||
steps:
|
||||
- name: Set up kubectl
|
||||
uses: azure/setup-kubectl@v3
|
||||
with:
|
||||
version: 'v1.28.0'
|
||||
|
||||
- name: Configure kubectl
|
||||
run: |
|
||||
if [[ "${{ needs.pre-deployment.outputs.deploy_env }}" == "production" ]]; then
|
||||
echo "${{ secrets.KUBE_CONFIG_DATA_PRODUCTION }}" | base64 -d > kubeconfig
|
||||
NAMESPACE="wifi-densepose"
|
||||
else
|
||||
echo "${{ secrets.KUBE_CONFIG_DATA_STAGING }}" | base64 -d > kubeconfig
|
||||
NAMESPACE="wifi-densepose-staging"
|
||||
fi
|
||||
export KUBECONFIG=kubeconfig
|
||||
echo "NAMESPACE=$NAMESPACE" >> $GITHUB_ENV
|
||||
|
||||
- name: Rollback deployment
|
||||
run: |
|
||||
export KUBECONFIG=kubeconfig
|
||||
|
||||
# Rollback to previous version
|
||||
kubectl rollout undo deployment/wifi-densepose -n ${{ env.NAMESPACE }}
|
||||
|
||||
# Wait for rollback to complete
|
||||
kubectl rollout status deployment/wifi-densepose -n ${{ env.NAMESPACE }} --timeout=600s
|
||||
|
||||
# Verify rollback
|
||||
kubectl get pods -n ${{ env.NAMESPACE }} -l app=wifi-densepose
|
||||
|
||||
# Post-deployment monitoring
|
||||
post-deployment:
|
||||
name: Post-deployment Monitoring
|
||||
runs-on: ubuntu-latest
|
||||
needs: [deploy-staging, deploy-production]
|
||||
if: always() && (needs.deploy-staging.result == 'success' || needs.deploy-production.result == 'success')
|
||||
steps:
|
||||
- name: Monitor deployment health
|
||||
run: |
|
||||
ENV="${{ needs.pre-deployment.outputs.deploy_env }}"
|
||||
if [[ "$ENV" == "production" ]]; then
|
||||
BASE_URL="https://wifi-densepose.com"
|
||||
else
|
||||
BASE_URL="https://staging.wifi-densepose.com"
|
||||
fi
|
||||
|
||||
# Monitor for 5 minutes
|
||||
for i in {1..10}; do
|
||||
echo "Health check $i/10"
|
||||
curl -f $BASE_URL/health || exit 1
|
||||
curl -f $BASE_URL/api/v1/status || exit 1
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Update deployment status
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
script: |
|
||||
const deployEnv = '${{ needs.pre-deployment.outputs.deploy_env }}';
|
||||
const environmentUrl = deployEnv === 'production' ? 'https://wifi-densepose.com' : 'https://staging.wifi-densepose.com';
|
||||
|
||||
const { data: deployment } = await github.rest.repos.createDeploymentStatus({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
deployment_id: context.payload.deployment.id,
|
||||
state: 'success',
|
||||
environment_url: environmentUrl,
|
||||
description: 'Deployment completed successfully'
|
||||
});
|
||||
|
||||
# Notification
|
||||
notify:
|
||||
name: Notify Deployment Status
|
||||
runs-on: ubuntu-latest
|
||||
needs: [deploy-staging, deploy-production, post-deployment]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Notify Slack on success
|
||||
if: needs.deploy-production.result == 'success' || needs.deploy-staging.result == 'success'
|
||||
uses: 8398a7/action-slack@v3
|
||||
with:
|
||||
status: success
|
||||
channel: '#deployments'
|
||||
text: |
|
||||
🚀 Deployment successful!
|
||||
Environment: ${{ needs.pre-deployment.outputs.deploy_env }}
|
||||
Image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.pre-deployment.outputs.image_tag }}
|
||||
URL: https://${{ needs.pre-deployment.outputs.deploy_env == 'production' && 'wifi-densepose.com' || 'staging.wifi-densepose.com' }}
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
|
||||
|
||||
- name: Notify Slack on failure
|
||||
if: needs.deploy-production.result == 'failure' || needs.deploy-staging.result == 'failure'
|
||||
uses: 8398a7/action-slack@v3
|
||||
with:
|
||||
status: failure
|
||||
channel: '#deployments'
|
||||
text: |
|
||||
❌ Deployment failed!
|
||||
Environment: ${{ needs.pre-deployment.outputs.deploy_env }}
|
||||
Please check the logs and consider rollback if necessary.
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
|
||||
|
||||
- name: Create deployment issue on failure
|
||||
if: needs.deploy-production.result == 'failure'
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
script: |
|
||||
github.rest.issues.create({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
title: `Production Deployment Failed - ${new Date().toISOString()}`,
|
||||
body: `
|
||||
## Deployment Failure Report
|
||||
|
||||
**Environment:** Production
|
||||
**Image Tag:** ${{ needs.pre-deployment.outputs.image_tag }}
|
||||
**Workflow Run:** ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
|
||||
**Action Required:**
|
||||
- [ ] Investigate deployment failure
|
||||
- [ ] Consider rollback if necessary
|
||||
- [ ] Fix underlying issues
|
||||
- [ ] Re-deploy when ready
|
||||
|
||||
**Logs:** Check the workflow run for detailed error messages.
|
||||
`,
|
||||
labels: ['deployment', 'production', 'urgent']
|
||||
})
|
||||
@@ -0,0 +1,503 @@
|
||||
name: Continuous Integration
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, develop, 'feature/*', 'feat/*', 'hotfix/*' ]
|
||||
pull_request:
|
||||
branches: [ main, develop ]
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
PYTHON_VERSION: '3.11'
|
||||
NODE_VERSION: '20' # ADR-265: all Node packages in this repo declare engines >= 20
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
jobs:
|
||||
# Code Quality and Security Checks
|
||||
# The Python codebase moved to `archive/v1/` when the runtime was rewritten in
|
||||
# Rust under `v2/`. The lint/format/type/scan checks below still run against
|
||||
# the archive for hygiene, but with `continue-on-error: true` everywhere — the
|
||||
# archive is frozen reference code, not active development, so a stale lint
|
||||
# rule shouldn't gate PRs to the Rust workspace.
|
||||
code-quality:
|
||||
name: Code Quality & Security
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
continue-on-error: true
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
continue-on-error: true
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install black flake8 mypy bandit safety
|
||||
|
||||
- name: Code formatting check (Black)
|
||||
continue-on-error: true
|
||||
run: black --check --diff archive/v1/src archive/v1/tests
|
||||
|
||||
- name: Linting (Flake8)
|
||||
continue-on-error: true
|
||||
run: flake8 archive/v1/src archive/v1/tests --max-line-length=88 --extend-ignore=E203,W503
|
||||
|
||||
- name: Type checking (MyPy)
|
||||
continue-on-error: true
|
||||
run: mypy archive/v1/src --ignore-missing-imports
|
||||
|
||||
- name: Security scan (Bandit)
|
||||
run: bandit -r archive/v1/src -f json -o bandit-report.json
|
||||
continue-on-error: true
|
||||
|
||||
- name: Dependency vulnerability scan (Safety)
|
||||
run: safety check --json --output safety-report.json
|
||||
continue-on-error: true
|
||||
|
||||
- name: Upload security reports
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: security-reports
|
||||
path: |
|
||||
bandit-report.json
|
||||
safety-report.json
|
||||
|
||||
# Rust Workspace Tests
|
||||
rust-tests:
|
||||
name: Rust Workspace Tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
# ADR-262 P1: `wifi-densepose-rufield` path-deps the `vendor/rufield`
|
||||
# submodule. Without a recursive checkout the workspace build fails to
|
||||
# resolve those path deps in CI even though it passes locally.
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
# `wifi-densepose-desktop` is a Tauri v2 app — `glib-sys`, `gtk-sys`,
|
||||
# `webkit2gtk-sys`, etc. need the Linux dev libraries via pkg-config or the
|
||||
# workspace test fails at the build step before any test runs (every recent
|
||||
# main CI run has been red on this for exactly this reason). Install the
|
||||
# standard Tauri-on-Ubuntu set.
|
||||
- name: Install Tauri / GTK / serial system dev libraries
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y --no-install-recommends \
|
||||
libglib2.0-dev \
|
||||
libgtk-3-dev \
|
||||
libsoup-3.0-dev \
|
||||
libjavascriptcoregtk-4.1-dev \
|
||||
libwebkit2gtk-4.1-dev \
|
||||
libayatana-appindicator3-dev \
|
||||
librsvg2-dev \
|
||||
libxdo-dev \
|
||||
libudev-dev \
|
||||
libdbus-1-dev \
|
||||
libssl-dev \
|
||||
pkg-config
|
||||
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
# Swatinem/rust-cache replaces a naive `actions/cache` of the whole
|
||||
# `v2/target`. That manual cache of a 38-crate target dir (multi-GB) was an
|
||||
# intermittent failure source — several CI runs this cycle died at the
|
||||
# cache/setup step (after toolchain install, before "Run Rust tests"),
|
||||
# needing a rerun. rust-cache is purpose-built for Rust: it caches the
|
||||
# registry + git + a pruned target, evicts stale deps, and restores far more
|
||||
# reliably (and faster) on large workspaces. `workspaces: v2` points it at
|
||||
# the v2/ cargo workspace (keys on v2/Cargo.lock, caches v2/target).
|
||||
- name: Cache cargo (Swatinem/rust-cache)
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: v2
|
||||
|
||||
# The 38-crate workspace debug build exhausts the runner's disk when built
|
||||
# with full debuginfo (observed: "final link failed: No space left on
|
||||
# device" once the engine/benchmark crates landed; the same tree's local
|
||||
# debug target measured 151 GB). Debuginfo is useless in CI — tests either
|
||||
# pass or print their failure — so build without it; target shrinks ~5-10x.
|
||||
- name: Run Rust tests
|
||||
working-directory: v2
|
||||
env:
|
||||
CARGO_PROFILE_DEV_DEBUG: "0"
|
||||
CARGO_PROFILE_TEST_DEBUG: "0"
|
||||
run: cargo test --workspace --no-default-features
|
||||
|
||||
- name: Run ADR-147 worldmodel tests
|
||||
working-directory: v2
|
||||
env:
|
||||
CARGO_PROFILE_DEV_DEBUG: "0"
|
||||
CARGO_PROFILE_TEST_DEBUG: "0"
|
||||
run: cargo test -p wifi-densepose-worldmodel --no-default-features
|
||||
|
||||
# ADR-134 CIR tests are behind the `cir` feature so the bench dependency
|
||||
# (Criterion) only pulls when actually exercised. Run them as a separate
|
||||
# step so a CIR-only regression is unambiguously attributable.
|
||||
- name: Run ADR-134 CIR tests
|
||||
working-directory: v2
|
||||
run: cargo test -p wifi-densepose-signal --no-default-features --features cir --tests
|
||||
|
||||
# ADR-134 + ADR-028 witness guard. The CIR proof runner produces a
|
||||
# bit-deterministic SHA-256 over CirEstimator output on the synthetic
|
||||
# reference signal. Any algorithmic regression — changes to ISTA
|
||||
# convergence, sensing matrix construction, soft-thresholding, or input
|
||||
# padding — breaks the hash and fails the build. To regenerate after an
|
||||
# *intentional* change:
|
||||
# cd v2 && cargo run -p wifi-densepose-signal --bin cir_proof_runner \
|
||||
# --release --no-default-features -- --generate-hash \
|
||||
# > ../archive/v1/data/proof/expected_cir_features.sha256
|
||||
- name: ADR-134 CIR witness proof (determinism guard)
|
||||
run: bash scripts/verify-cir-proof.sh
|
||||
|
||||
- name: ADR-135 calibration witness proof (determinism guard)
|
||||
run: bash scripts/verify-calibration-proof.sh
|
||||
|
||||
# Unit and Integration Tests
|
||||
# Python pytest matrix — runs against the archived v1 Python tree.
|
||||
# `continue-on-error: true` for the same reason as code-quality above:
|
||||
# the archive is frozen reference, not blocking the Rust workspace PRs.
|
||||
test:
|
||||
name: Tests
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ['3.10', '3.11', '3.12']
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:15
|
||||
env:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_DB: test_wifi_densepose
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 5432:5432
|
||||
|
||||
redis:
|
||||
image: redis:7
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 6379:6379
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
continue-on-error: true
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
continue-on-error: true
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install pytest-cov pytest-xdist
|
||||
|
||||
- name: Run unit tests
|
||||
continue-on-error: true
|
||||
env:
|
||||
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test_wifi_densepose
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
ENVIRONMENT: test
|
||||
run: |
|
||||
pytest archive/v1/tests/unit/ -v --cov=archive/v1/src --cov-report=xml --cov-report=html --junitxml=junit.xml
|
||||
|
||||
- name: Run integration tests
|
||||
continue-on-error: true
|
||||
env:
|
||||
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test_wifi_densepose
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
ENVIRONMENT: test
|
||||
run: |
|
||||
pytest archive/v1/tests/integration/ -v --junitxml=integration-junit.xml
|
||||
|
||||
- name: Upload coverage reports
|
||||
continue-on-error: true
|
||||
uses: codecov/codecov-action@v6
|
||||
with:
|
||||
file: ./coverage.xml
|
||||
flags: unittests
|
||||
name: codecov-umbrella
|
||||
|
||||
- name: Upload test results
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: test-results-${{ matrix.python-version }}
|
||||
path: |
|
||||
junit.xml
|
||||
integration-junit.xml
|
||||
htmlcov/
|
||||
|
||||
# Performance and Load Tests
|
||||
# NOTE: tests/performance/locustfile.py and the src.api.main app path both
|
||||
# predate the v1→archive/v1 reorganisation. continue-on-error: true until a
|
||||
# proper locust suite is added under archive/v1/tests/performance/.
|
||||
performance-test:
|
||||
name: Performance Tests
|
||||
runs-on: ubuntu-latest
|
||||
needs: [test]
|
||||
continue-on-error: true
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install pytest # the perf suite is pytest, not locust
|
||||
|
||||
# No "Start application" step: the gated test (test_frame_budget.py) drives
|
||||
# the CSIProcessor pipeline in-process and makes no HTTP calls, so the old
|
||||
# uvicorn server + `sleep 10` were dead weight — they only existed for the
|
||||
# now-excluded api_throughput/inference_speed tests, and on every run dumped
|
||||
# ~50 misleading "router requires hardware setup" ERROR lines for a server
|
||||
# no test touched. MOCK_POSE_DATA is server-only and unused here.
|
||||
|
||||
- name: Run performance tests
|
||||
working-directory: archive/v1
|
||||
run: |
|
||||
# Gate only on the genuine, deterministic perf guard:
|
||||
# test_frame_budget.py times the *real* CSIProcessor pipeline against
|
||||
# the ADR 50 ms per-frame budget (single-frame, p95 over 100 frames,
|
||||
# +Doppler) — a true regression signal.
|
||||
#
|
||||
# test_api_throughput.py / test_inference_speed.py are excluded: every
|
||||
# test there is a TDD red-phase stub (suffix `_should_fail_initially`)
|
||||
# that times a *mock that sleeps* — meaningless as a perf signal, with
|
||||
# machine-dependent wall-clock asserts (e.g. `actual_rps >= 40`,
|
||||
# `batch_time < individual_time`) that are inherently flaky on shared
|
||||
# CI runners, plus a cross-class fixture-scope bug. Forcing them green
|
||||
# would be manufacturing a false signal; they stay in-repo for local
|
||||
# TDD but do not gate CI until the underlying features are implemented.
|
||||
#
|
||||
# `python -m pytest` (not the bare `pytest` script) puts the cwd
|
||||
# (archive/v1) on sys.path so `from src.core...` resolves — the bare
|
||||
# script omits cwd and raises ModuleNotFoundError: No module named 'src'.
|
||||
# -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100.
|
||||
python -m pytest tests/performance/test_frame_budget.py \
|
||||
-o addopts="" -v --junitxml=perf-junit.xml
|
||||
|
||||
- name: Upload performance results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: performance-results
|
||||
path: archive/v1/perf-junit.xml
|
||||
|
||||
# Docker Build and Test
|
||||
# NOTE: the canonical Docker build for the sensing-server is now
|
||||
# `.github/workflows/sensing-server-docker.yml` (multi-registry push, asset
|
||||
# smoke tests, bearer-auth smoke tests — #520/#514/#443). This job predates
|
||||
# that workflow, points at a non-existent root `Dockerfile` with a
|
||||
# non-existent `target: production`, and pushes to a mis-cased image name —
|
||||
# `continue-on-error: true` until it's deleted or rewired to call the new
|
||||
# workflow, so it doesn't gate the rest of the pipeline.
|
||||
docker-build:
|
||||
name: Docker Build & Test
|
||||
runs-on: ubuntu-latest
|
||||
needs: [code-quality, test, rust-tests]
|
||||
continue-on-error: true
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
continue-on-error: true
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Log in to Container Registry
|
||||
continue-on-error: true
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata
|
||||
continue-on-error: true
|
||||
id: meta
|
||||
uses: docker/metadata-action@v6
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
type=sha,prefix={{branch}}-
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
- name: Build and push Docker image
|
||||
continue-on-error: true
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
target: production
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Test Docker image
|
||||
continue-on-error: true
|
||||
run: |
|
||||
docker run --rm -d --name test-container -p 8000:8000 ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
sleep 10
|
||||
curl -f http://localhost:8000/health || exit 1
|
||||
docker stop test-container
|
||||
|
||||
- name: Run container security scan
|
||||
continue-on-error: true
|
||||
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
|
||||
with:
|
||||
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
format: 'sarif'
|
||||
output: 'trivy-results.sarif'
|
||||
|
||||
- name: Upload Trivy scan results
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: 'trivy-results.sarif'
|
||||
|
||||
# API Documentation
|
||||
docs:
|
||||
name: API Documentation
|
||||
runs-on: ubuntu-latest
|
||||
needs: [docker-build]
|
||||
if: github.ref == 'refs/heads/main'
|
||||
permissions:
|
||||
contents: write # gh-pages deploy needs write (GITHUB_TOKEN is read-only by default -> 403)
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
|
||||
- name: Generate OpenAPI spec
|
||||
working-directory: archive/v1
|
||||
env:
|
||||
MOCK_POSE_DATA: "true" # no CSI hardware in CI
|
||||
run: |
|
||||
python -c "
|
||||
from src.api.main import app
|
||||
import json
|
||||
with open('openapi.json', 'w') as f:
|
||||
json.dump(app.openapi(), f, indent=2)
|
||||
"
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
uses: peaceiris/actions-gh-pages@v4
|
||||
continue-on-error: true # openapi generation above is the real validation; deploy is best-effort (Pages may be disabled)
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_dir: ./docs
|
||||
destination_dir: api-docs
|
||||
|
||||
# Notification
|
||||
notify:
|
||||
name: Notify
|
||||
runs-on: ubuntu-latest
|
||||
needs: [code-quality, test, rust-tests, performance-test, docker-build, docs]
|
||||
if: always()
|
||||
permissions:
|
||||
contents: write # required by softprops/action-gh-release
|
||||
# GitHub Actions does not allow `secrets.X` directly in step-level `if:`
|
||||
# expressions — only `env.X`. Promote the secret to env at job scope so
|
||||
# the gating expression below is parseable.
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
|
||||
steps:
|
||||
- name: Notify Slack on success
|
||||
if: ${{ env.SLACK_WEBHOOK_URL != '' && needs.code-quality.result == 'success' && needs.test.result == 'success' && needs.docker-build.result == 'success' }}
|
||||
uses: 8398a7/action-slack@v3
|
||||
with:
|
||||
status: success
|
||||
channel: '#ci-cd'
|
||||
text: '✅ CI pipeline completed successfully for ${{ github.ref }}'
|
||||
|
||||
- name: Notify Slack on failure
|
||||
if: ${{ env.SLACK_WEBHOOK_URL != '' && (needs.code-quality.result == 'failure' || needs.test.result == 'failure' || needs.docker-build.result == 'failure') }}
|
||||
uses: 8398a7/action-slack@v3
|
||||
with:
|
||||
status: failure
|
||||
channel: '#ci-cd'
|
||||
text: '❌ CI pipeline failed for ${{ github.ref }}'
|
||||
|
||||
- name: Create GitHub Release
|
||||
if: github.ref == 'refs/heads/main' && needs.docker-build.result == 'success'
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
tag_name: v${{ github.run_number }}
|
||||
name: Release v${{ github.run_number }}
|
||||
body: |
|
||||
Automated release from CI pipeline
|
||||
|
||||
**Changes:**
|
||||
${{ github.event.head_commit.message }}
|
||||
|
||||
**Docker Image:**
|
||||
`${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}`
|
||||
draft: false
|
||||
prerelease: false
|
||||
@@ -0,0 +1,151 @@
|
||||
name: GitHub Clone Tracking → data/clone-data.rvf
|
||||
|
||||
# Persists rolling 14-day clone-traffic snapshots to data/clone-data.rvf in
|
||||
# the ruvector JSONL RVF format. GitHub's /traffic/clones endpoint only
|
||||
# retains the last 14 days server-side, so without this scheduled scrape
|
||||
# the data is gone forever the moment it falls outside the window.
|
||||
#
|
||||
# Format: JSONL RVF
|
||||
# - line 1 is a `metadata` segment that initializes the file
|
||||
# - each subsequent run appends one `clone_snapshot` segment carrying the
|
||||
# 14-day rollup PLUS per-day breakdown
|
||||
# - file is idempotent: per-day entries are keyed by `timestamp` so a
|
||||
# downstream reader can dedupe across overlapping snapshot windows
|
||||
#
|
||||
# Schedule: every 14 days (1st + 15th of each month, ~14-day cadence in
|
||||
# practice). Workflow can also be dispatched manually for backfill or test.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 01:23 UTC on the 1st and 15th of every month — close to 14-day cadence
|
||||
# without cron's "every 14 days" monthly-reset weirdness. Picking :23
|
||||
# avoids the cron herd on :00.
|
||||
- cron: '23 1 1,15 * *'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
concurrency:
|
||||
group: clone-tracking
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
snapshot:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Fetch /traffic/clones + /traffic/views from GitHub
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
mkdir -p data
|
||||
gh api repos/${{ github.repository }}/traffic/clones > /tmp/clones.json
|
||||
gh api repos/${{ github.repository }}/traffic/views > /tmp/views.json
|
||||
echo "--- clones rollup ---"
|
||||
jq '{count, uniques, days: (.clones | length)}' /tmp/clones.json
|
||||
echo "--- views rollup ---"
|
||||
jq '{count, uniques, days: (.views | length)}' /tmp/views.json
|
||||
|
||||
- name: Append snapshot to data/clone-data.rvf
|
||||
env:
|
||||
REPO: ${{ github.repository }}
|
||||
run: |
|
||||
set -e
|
||||
RVF="data/clone-data.rvf"
|
||||
FETCHED_AT=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
# Initialize the file with a metadata segment on first run.
|
||||
if [ ! -f "$RVF" ]; then
|
||||
echo "Initializing $RVF with metadata segment"
|
||||
jq -n --arg repo "$REPO" --arg ts "$FETCHED_AT" '{
|
||||
type: "metadata",
|
||||
name: "ruview-clone-traffic-history",
|
||||
version: "1.0.0",
|
||||
schema: "ruvector.rvf.jsonl/v1",
|
||||
format: "github-traffic-snapshots",
|
||||
repo: $repo,
|
||||
source: "GitHub Traffic API /repos/{repo}/traffic/{clones,views}",
|
||||
policy: "GitHub retains only 14 days server-side; this file is the long-term record.",
|
||||
segments: ["metadata", "clone_snapshot", "view_snapshot"],
|
||||
created_at: $ts,
|
||||
custom: {
|
||||
cadence: "twice monthly (1st and 15th, ~14-day intervals)",
|
||||
idempotency_key: "timestamp (per-day records de-duplicate across overlapping snapshot windows)"
|
||||
}
|
||||
}' >> "$RVF"
|
||||
fi
|
||||
|
||||
# Append the clone snapshot.
|
||||
jq --arg ts "$FETCHED_AT" '{
|
||||
type: "clone_snapshot",
|
||||
fetched_at: $ts,
|
||||
window_count: .count,
|
||||
window_uniques: .uniques,
|
||||
per_day: .clones
|
||||
}' /tmp/clones.json >> "$RVF"
|
||||
|
||||
# Append the views snapshot (free with the same auth).
|
||||
jq --arg ts "$FETCHED_AT" '{
|
||||
type: "view_snapshot",
|
||||
fetched_at: $ts,
|
||||
window_count: .count,
|
||||
window_uniques: .uniques,
|
||||
per_day: .views
|
||||
}' /tmp/views.json >> "$RVF"
|
||||
|
||||
echo "--- RVF tail (last 4 lines) ---"
|
||||
tail -4 "$RVF" | jq -c '{type, fetched_at, window_count, window_uniques}' || true
|
||||
echo "--- file size ---"
|
||||
wc -l "$RVF"
|
||||
|
||||
- name: Compute aggregates for the commit summary
|
||||
id: agg
|
||||
run: |
|
||||
# Count distinct per-day entries across all snapshots so we can
|
||||
# show "cumulative observed clones" in the commit message.
|
||||
python3 - <<'PY'
|
||||
import json, os
|
||||
path = "data/clone-data.rvf"
|
||||
per_day_clones = {}
|
||||
per_day_views = {}
|
||||
with open(path, encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if not line.strip():
|
||||
continue
|
||||
d = json.loads(line)
|
||||
if d.get("type") == "clone_snapshot":
|
||||
for entry in d.get("per_day", []):
|
||||
per_day_clones[entry["timestamp"]] = entry
|
||||
elif d.get("type") == "view_snapshot":
|
||||
for entry in d.get("per_day", []):
|
||||
per_day_views[entry["timestamp"]] = entry
|
||||
|
||||
tot_clones = sum(e.get("count", 0) for e in per_day_clones.values())
|
||||
tot_uniq_clones = sum(e.get("uniques", 0) for e in per_day_clones.values())
|
||||
tot_views = sum(e.get("count", 0) for e in per_day_views.values())
|
||||
tot_uniq_views = sum(e.get("uniques", 0) for e in per_day_views.values())
|
||||
print(f"clone days observed: {len(per_day_clones)} total clones: {tot_clones:,} total unique cloners: {tot_uniq_clones:,}")
|
||||
print(f"view days observed: {len(per_day_views)} total views: {tot_views:,} total unique viewers: {tot_uniq_views:,}")
|
||||
|
||||
with open(os.environ["GITHUB_OUTPUT"], "a") as out:
|
||||
out.write(f"clones={tot_clones}\n")
|
||||
out.write(f"clone_days={len(per_day_clones)}\n")
|
||||
out.write(f"views={tot_views}\n")
|
||||
out.write(f"view_days={len(per_day_views)}\n")
|
||||
PY
|
||||
|
||||
- name: Commit + push if changed
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
if git diff --quiet data/clone-data.rvf; then
|
||||
echo "no changes to commit"
|
||||
exit 0
|
||||
fi
|
||||
git add data/clone-data.rvf
|
||||
git commit -m "chore(traffic): clone snapshot — ${{ steps.agg.outputs.clone_days }} days observed → ${{ steps.agg.outputs.clones }} clones, ${{ steps.agg.outputs.view_days }} view-days → ${{ steps.agg.outputs.views }} views"
|
||||
git push
|
||||
@@ -0,0 +1,206 @@
|
||||
name: Cog HA-Matter Release
|
||||
|
||||
# ADR-116 P8 — Build + sign + bundle the cog-ha-matter cog on a
|
||||
# version tag. Upload to gs://cognitum-apps/ runs only when the
|
||||
# GCP_CREDENTIALS + COGNITUM_OWNER_SIGNING_KEY secrets are set, so
|
||||
# this workflow is safe to merge before the production credentials
|
||||
# land — it'll bundle release artifacts to the workflow run page
|
||||
# either way.
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'cog-ha-matter-v*'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
dry_run:
|
||||
description: 'Build + sign + bundle but skip GCS upload'
|
||||
required: false
|
||||
default: 'true'
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
CRATE: cog-ha-matter
|
||||
|
||||
jobs:
|
||||
build-x86_64:
|
||||
name: Build x86_64
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup Rust
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
targets: x86_64-unknown-linux-gnu
|
||||
|
||||
- name: Cache cargo registry
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: cog-ha-matter-x86_64-${{ hashFiles('v2/Cargo.lock') }}
|
||||
|
||||
- name: Build release binary
|
||||
working-directory: v2/crates/cog-ha-matter/cog
|
||||
run: make build-x86_64
|
||||
|
||||
- name: Compute SHA-256
|
||||
working-directory: v2/crates/cog-ha-matter/cog
|
||||
run: make sign-x86_64
|
||||
|
||||
- name: Sign with Ed25519 (gated)
|
||||
if: ${{ env.SIGNING_KEY != '' }}
|
||||
env:
|
||||
SIGNING_KEY: ${{ secrets.COGNITUM_OWNER_SIGNING_KEY }}
|
||||
working-directory: v2/crates/cog-ha-matter/cog
|
||||
run: |
|
||||
printf '%s' "$SIGNING_KEY" \
|
||||
| openssl pkeyutl -sign -inkey /dev/stdin -rawin \
|
||||
-in dist/cog-ha-matter-x86_64.sha256 \
|
||||
| base64 -w0 > dist/cog-ha-matter-x86_64.sig
|
||||
echo "Signed cog-ha-matter-x86_64 ($(wc -c < dist/cog-ha-matter-x86_64.sig) bytes)"
|
||||
|
||||
- name: Upload workflow artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: cog-ha-matter-x86_64
|
||||
path: |
|
||||
v2/crates/cog-ha-matter/cog/dist/cog-ha-matter-x86_64
|
||||
v2/crates/cog-ha-matter/cog/dist/cog-ha-matter-x86_64.sha256
|
||||
v2/crates/cog-ha-matter/cog/dist/cog-ha-matter-x86_64.sig
|
||||
if-no-files-found: warn
|
||||
|
||||
build-arm:
|
||||
name: Build aarch64 (arm)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup Rust
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
targets: aarch64-unknown-linux-gnu
|
||||
|
||||
- name: Install cross-compiler
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y gcc-aarch64-linux-gnu
|
||||
|
||||
- name: Cache cargo registry
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: cog-ha-matter-arm-${{ hashFiles('v2/Cargo.lock') }}
|
||||
|
||||
- name: Build release binary
|
||||
working-directory: v2
|
||||
env:
|
||||
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
|
||||
run: |
|
||||
cargo build -p cog-ha-matter --release --target aarch64-unknown-linux-gnu
|
||||
mkdir -p crates/cog-ha-matter/cog/dist
|
||||
cp target/aarch64-unknown-linux-gnu/release/cog-ha-matter \
|
||||
crates/cog-ha-matter/cog/dist/cog-ha-matter-arm
|
||||
# ^ matches Makefile's `dist/$(CRATE)-arm` so `make sign-arm` finds it
|
||||
|
||||
- name: Compute SHA-256
|
||||
working-directory: v2/crates/cog-ha-matter/cog
|
||||
run: make sign-arm
|
||||
|
||||
- name: Sign with Ed25519 (gated)
|
||||
if: ${{ env.SIGNING_KEY != '' }}
|
||||
env:
|
||||
SIGNING_KEY: ${{ secrets.COGNITUM_OWNER_SIGNING_KEY }}
|
||||
working-directory: v2/crates/cog-ha-matter/cog
|
||||
run: |
|
||||
printf '%s' "$SIGNING_KEY" \
|
||||
| openssl pkeyutl -sign -inkey /dev/stdin -rawin \
|
||||
-in dist/cog-ha-matter-arm.sha256 \
|
||||
| base64 -w0 > dist/cog-ha-matter-arm.sig
|
||||
echo "Signed cog-ha-matter-arm ($(wc -c < dist/cog-ha-matter-arm.sig) bytes)"
|
||||
|
||||
- name: Upload workflow artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: cog-ha-matter-arm
|
||||
path: |
|
||||
v2/crates/cog-ha-matter/cog/dist/cog-ha-matter-arm
|
||||
v2/crates/cog-ha-matter/cog/dist/cog-ha-matter-arm.sha256
|
||||
v2/crates/cog-ha-matter/cog/dist/cog-ha-matter-arm.sig
|
||||
if-no-files-found: warn
|
||||
|
||||
publish-gcs:
|
||||
name: Upload to GCS (gated)
|
||||
needs: [build-x86_64, build-arm]
|
||||
runs-on: ubuntu-latest
|
||||
# Skip on dry-run dispatch; skip on tags when GCP_CREDENTIALS unset.
|
||||
if: >
|
||||
github.event_name == 'push' &&
|
||||
vars.HAS_GCP_CREDENTIALS == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Download x86_64 artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: cog-ha-matter-x86_64
|
||||
path: dist/
|
||||
|
||||
- name: Download arm artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: cog-ha-matter-arm
|
||||
path: dist/
|
||||
|
||||
- name: Auth to GCP
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
credentials_json: ${{ secrets.GCP_CREDENTIALS }}
|
||||
|
||||
- name: Set up gcloud
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
|
||||
- name: Upload binaries + sidecars
|
||||
run: |
|
||||
gsutil cp dist/cog-ha-matter-x86_64 gs://cognitum-apps/cogs/x86_64/cog-ha-matter-x86_64
|
||||
gsutil cp dist/cog-ha-matter-x86_64.sha256 gs://cognitum-apps/cogs/x86_64/cog-ha-matter-x86_64.sha256
|
||||
gsutil cp dist/cog-ha-matter-arm gs://cognitum-apps/cogs/arm/cog-ha-matter-arm
|
||||
gsutil cp dist/cog-ha-matter-arm.sha256 gs://cognitum-apps/cogs/arm/cog-ha-matter-arm.sha256
|
||||
if [ -f dist/cog-ha-matter-x86_64.sig ]; then
|
||||
gsutil cp dist/cog-ha-matter-x86_64.sig gs://cognitum-apps/cogs/x86_64/cog-ha-matter-x86_64.sig
|
||||
fi
|
||||
if [ -f dist/cog-ha-matter-arm.sig ]; then
|
||||
gsutil cp dist/cog-ha-matter-arm.sig gs://cognitum-apps/cogs/arm/cog-ha-matter-arm.sig
|
||||
fi
|
||||
|
||||
- name: Print app-registry.json snippet for the cognitum-one PR
|
||||
run: |
|
||||
for arch in arm x86_64; do
|
||||
sha=$(cat dist/cog-cog-ha-matter-$arch.sha256)
|
||||
sig=$([ -f dist/cog-cog-ha-matter-$arch.sig ] && cat dist/cog-cog-ha-matter-$arch.sig || echo "")
|
||||
cat <<EOF
|
||||
--- $arch ---
|
||||
{
|
||||
"id": "ha-matter",
|
||||
"version": "${GITHUB_REF_NAME#cog-ha-matter-v}",
|
||||
"binary_url": "https://storage.googleapis.com/cognitum-apps/cogs/$arch/cog-cog-ha-matter-$arch",
|
||||
"binary_sha256": "$sha",
|
||||
"binary_signature": "$sig",
|
||||
"description": "Home Assistant + Matter Cognitum Seed cog (mDNS + witness chain)",
|
||||
"min_seed_version": "0.6.0",
|
||||
"installable_on": ["$arch"]
|
||||
}
|
||||
EOF
|
||||
done
|
||||
@@ -0,0 +1,48 @@
|
||||
name: Dashboard a11y + cross-browser
|
||||
|
||||
# Runs axe-core a11y assertions on the built dashboard across
|
||||
# Chromium, Firefox, and WebKit. Closes ADR-092 §11.5 (axe-core)
|
||||
# and §11.8 (cross-browser).
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ['dashboard/**', 'v2/crates/nvsim/**']
|
||||
pull_request:
|
||||
paths: ['dashboard/**']
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
a11y:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
with: { targets: wasm32-unknown-unknown }
|
||||
|
||||
- name: Install wasm-pack
|
||||
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
|
||||
|
||||
- name: Build nvsim WASM
|
||||
working-directory: v2
|
||||
run: |
|
||||
wasm-pack build crates/nvsim --target web \
|
||||
--out-dir ../../dashboard/public/nvsim-pkg \
|
||||
--release -- --no-default-features --features wasm
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with: { node-version: 20, cache: npm, cache-dependency-path: dashboard/package-lock.json }
|
||||
|
||||
- working-directory: dashboard
|
||||
run: |
|
||||
npm ci
|
||||
npm install --save-dev @playwright/test @axe-core/playwright
|
||||
npx playwright install --with-deps
|
||||
npm run build
|
||||
npx playwright test
|
||||
@@ -0,0 +1,89 @@
|
||||
name: nvsim Dashboard → GitHub Pages
|
||||
|
||||
# Deploys the nvsim Vite/Lit dashboard to gh-pages/nvsim/ — preserving
|
||||
# the existing observatory/, pose-fusion/, and root index.html demos
|
||||
# already published from gh-pages. ADR-092 §9.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'v2/crates/nvsim/**'
|
||||
- 'dashboard/**'
|
||||
- '.github/workflows/dashboard-pages.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
concurrency:
|
||||
group: dashboard-pages
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install Rust + wasm32 target
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
targets: wasm32-unknown-unknown
|
||||
|
||||
- name: Cache cargo registry
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: ${{ runner.os }}-cargo-nvsim-${{ hashFiles('v2/Cargo.lock') }}
|
||||
restore-keys: ${{ runner.os }}-cargo-nvsim-
|
||||
|
||||
- name: Install wasm-pack
|
||||
run: |
|
||||
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
|
||||
which wasm-pack
|
||||
|
||||
- name: Build nvsim WASM
|
||||
working-directory: v2
|
||||
run: |
|
||||
wasm-pack build crates/nvsim \
|
||||
--target web \
|
||||
--out-dir ../../dashboard/public/nvsim-pkg \
|
||||
--release \
|
||||
-- --no-default-features --features wasm
|
||||
|
||||
- name: Setup Node 20
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
cache-dependency-path: dashboard/package-lock.json
|
||||
|
||||
- name: Install dashboard deps
|
||||
working-directory: dashboard
|
||||
run: npm ci
|
||||
|
||||
- name: Build dashboard
|
||||
working-directory: dashboard
|
||||
env:
|
||||
NVSIM_BASE: /RuView/nvsim/
|
||||
run: npm run build
|
||||
|
||||
- name: Deploy to gh-pages/nvsim/
|
||||
uses: peaceiris/actions-gh-pages@v4
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_dir: ./dashboard/dist
|
||||
destination_dir: nvsim
|
||||
# CRITICAL: preserves observatory/, pose-fusion/, root index.html
|
||||
# and any other RuView demos already on gh-pages.
|
||||
keep_files: true
|
||||
commit_message: 'deploy(nvsim): ${{ github.sha }}'
|
||||
user_name: 'github-actions[bot]'
|
||||
user_email: 'github-actions[bot]@users.noreply.github.com'
|
||||
@@ -0,0 +1,190 @@
|
||||
name: Desktop Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'desktop-v*'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Version to release (e.g., 0.4.0)'
|
||||
required: true
|
||||
default: '0.4.0'
|
||||
attach_to_existing:
|
||||
description: 'Attach to existing release tag (leave empty to create new)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
|
||||
jobs:
|
||||
build-macos:
|
||||
name: Build macOS
|
||||
runs-on: macos-latest
|
||||
strategy:
|
||||
matrix:
|
||||
target: [aarch64-apple-darwin, x86_64-apple-darwin]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Setup Rust
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
targets: ${{ matrix.target }}
|
||||
|
||||
- name: Install frontend dependencies
|
||||
working-directory: v2/crates/wifi-densepose-desktop/ui
|
||||
run: npm ci
|
||||
|
||||
- name: Build frontend
|
||||
working-directory: v2/crates/wifi-densepose-desktop/ui
|
||||
run: npm run build
|
||||
|
||||
- name: Install Tauri CLI
|
||||
run: cargo install tauri-cli --version "^2.0.0"
|
||||
|
||||
- name: Build Tauri app
|
||||
working-directory: v2/crates/wifi-densepose-desktop
|
||||
run: cargo tauri build --target ${{ matrix.target }}
|
||||
env:
|
||||
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
|
||||
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
|
||||
|
||||
- name: Get architecture name
|
||||
id: arch
|
||||
run: |
|
||||
if [ "${{ matrix.target }}" = "aarch64-apple-darwin" ]; then
|
||||
echo "arch=arm64" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "arch=x64" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Package macOS app
|
||||
run: |
|
||||
cd v2/target/${{ matrix.target }}/release/bundle/macos
|
||||
zip -r "RuView-Desktop-${{ github.event.inputs.version || '0.4.0' }}-macos-${{ steps.arch.outputs.arch }}.zip" "RuView Desktop.app"
|
||||
|
||||
- name: Upload macOS artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ruview-macos-${{ steps.arch.outputs.arch }}
|
||||
path: v2/target/${{ matrix.target }}/release/bundle/macos/*.zip
|
||||
|
||||
build-windows:
|
||||
name: Build Windows
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Setup Rust
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Install frontend dependencies
|
||||
working-directory: v2/crates/wifi-densepose-desktop/ui
|
||||
run: npm ci
|
||||
|
||||
- name: Build frontend
|
||||
working-directory: v2/crates/wifi-densepose-desktop/ui
|
||||
run: npm run build
|
||||
|
||||
- name: Install Tauri CLI
|
||||
run: cargo install tauri-cli --version "^2.0.0"
|
||||
|
||||
- name: Build Tauri app
|
||||
working-directory: v2/crates/wifi-densepose-desktop
|
||||
run: cargo tauri build
|
||||
env:
|
||||
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
|
||||
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
|
||||
|
||||
- name: Upload Windows MSI artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ruview-windows-msi
|
||||
path: v2/target/release/bundle/msi/*.msi
|
||||
|
||||
- name: Upload Windows NSIS artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ruview-windows-nsis
|
||||
path: v2/target/release/bundle/nsis/*.exe
|
||||
|
||||
create-release:
|
||||
name: Create Release
|
||||
needs: [build-macos, build-windows]
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Download all artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: artifacts
|
||||
|
||||
- name: List artifacts
|
||||
run: find artifacts -type f
|
||||
|
||||
- name: Create or Update Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
name: RuView Desktop v${{ github.event.inputs.version || '0.4.0' }}
|
||||
tag_name: ${{ github.event.inputs.attach_to_existing || format('desktop-v{0}', github.event.inputs.version || '0.4.0') }}
|
||||
draft: false
|
||||
prerelease: false
|
||||
generate_release_notes: ${{ github.event.inputs.attach_to_existing == '' }}
|
||||
files: |
|
||||
artifacts/**/*.zip
|
||||
artifacts/**/*.msi
|
||||
artifacts/**/*.exe
|
||||
artifacts/**/*.dmg
|
||||
body: |
|
||||
## RuView Desktop v${{ github.event.inputs.version || '0.4.0' }}
|
||||
|
||||
WiFi-based human pose estimation desktop application.
|
||||
|
||||
### Downloads
|
||||
|
||||
| Platform | Architecture | Download |
|
||||
|----------|--------------|----------|
|
||||
| macOS | Apple Silicon (M1/M2/M3) | `RuView-Desktop-*-macos-arm64.zip` |
|
||||
| macOS | Intel | `RuView-Desktop-*-macos-x64.zip` |
|
||||
| Windows | x64 | `RuView-Desktop-*.msi` or `RuView-Desktop-*.exe` |
|
||||
|
||||
### Installation
|
||||
|
||||
**macOS:**
|
||||
1. Download the appropriate `.zip` file for your Mac
|
||||
2. Extract the zip file
|
||||
3. Move `RuView Desktop.app` to your Applications folder
|
||||
4. Right-click and select "Open" (first time only, to bypass Gatekeeper)
|
||||
|
||||
**Windows:**
|
||||
1. Download the `.msi` installer
|
||||
2. Run the installer
|
||||
3. Launch RuView Desktop from the Start menu
|
||||
|
||||
### Requirements
|
||||
- macOS 11.0+ (Big Sur or later)
|
||||
- Windows 10/11 (64-bit)
|
||||
@@ -0,0 +1,174 @@
|
||||
name: Firmware CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- '**'
|
||||
tags:
|
||||
# ESP32 firmware release tags — build + version-consistency guard (RuView#505).
|
||||
- 'v*-esp32'
|
||||
paths:
|
||||
- 'firmware/**'
|
||||
- '.github/workflows/firmware-ci.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'firmware/**'
|
||||
- '.github/workflows/firmware-ci.yml'
|
||||
|
||||
jobs:
|
||||
version-guard:
|
||||
name: Verify version.txt matches release tag
|
||||
runs-on: ubuntu-latest
|
||||
if: github.ref_type == 'tag'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
- name: Check firmware version.txt == tag
|
||||
run: |
|
||||
# Tag form: vX.Y.Z-esp32 → expect version.txt to contain X.Y.Z
|
||||
TAG="${GITHUB_REF_NAME}"
|
||||
EXPECTED="${TAG#v}"
|
||||
EXPECTED="${EXPECTED%-esp32}"
|
||||
ACTUAL="$(tr -d '[:space:]' < firmware/esp32-csi-node/version.txt)"
|
||||
echo "Tag: $TAG → expected version.txt: $EXPECTED | actual: $ACTUAL"
|
||||
if [ "$EXPECTED" != "$ACTUAL" ]; then
|
||||
echo "::error::firmware/esp32-csi-node/version.txt is '$ACTUAL' but tag '$TAG' expects '$EXPECTED'."
|
||||
echo "::error::Bump version.txt and re-tag so esp_app_get_description()->version is correct (RuView#505)."
|
||||
exit 1
|
||||
fi
|
||||
echo "version.txt matches the release tag."
|
||||
|
||||
build:
|
||||
name: Build firmware (${{ matrix.target }} / ${{ matrix.variant }})
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: espressif/idf:v5.4
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- variant: 8mb
|
||||
target: esp32s3
|
||||
sdkconfig: sdkconfig.defaults
|
||||
partition_table_name: partitions_display.csv
|
||||
size_limit_kb: 1100
|
||||
artifact_app: esp32-csi-node.bin
|
||||
artifact_pt: partition-table.bin
|
||||
- variant: 4mb
|
||||
target: esp32s3
|
||||
sdkconfig: sdkconfig.defaults.4mb
|
||||
partition_table_name: partitions_4mb.csv
|
||||
size_limit_kb: 1100
|
||||
artifact_app: esp32-csi-node-4mb.bin
|
||||
artifact_pt: partition-table-4mb.bin
|
||||
# ADR-110: ESP32-C6 research target (Wi-Fi 6 / 802.15.4 / TWT / LP-core)
|
||||
- variant: c6-4mb
|
||||
target: esp32c6
|
||||
sdkconfig: sdkconfig.defaults
|
||||
partition_table_name: partitions_4mb.csv
|
||||
size_limit_kb: 1100
|
||||
artifact_app: esp32-csi-node-c6.bin
|
||||
artifact_pt: partition-table-c6.bin
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Build firmware (${{ matrix.variant }})
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
# 4mb variant supplies its own sdkconfig.defaults overlay.
|
||||
# c6-4mb variant relies on the auto-applied sdkconfig.defaults.esp32c6
|
||||
# overlay (ESP-IDF auto-loads sdkconfig.defaults.$TARGET when present).
|
||||
if [ "${{ matrix.variant }}" = "4mb" ]; then
|
||||
cp "${{ matrix.sdkconfig }}" sdkconfig.defaults
|
||||
fi
|
||||
idf.py set-target ${{ matrix.target }}
|
||||
idf.py build
|
||||
|
||||
- name: Build and run host-side ADR-110 unit tests
|
||||
if: matrix.variant == 'c6-4mb'
|
||||
working-directory: firmware/esp32-csi-node/test
|
||||
run: |
|
||||
make test_adr110
|
||||
./test_adr110
|
||||
|
||||
- name: Verify binary size (< ${{ matrix.size_limit_kb }} KB gate)
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
BIN=build/esp32-csi-node.bin
|
||||
SIZE=$(stat -c%s "$BIN")
|
||||
MAX=$((${{ matrix.size_limit_kb }} * 1024))
|
||||
echo "Binary size: $SIZE bytes ($(( SIZE / 1024 )) KB)"
|
||||
echo "Size limit: $MAX bytes (${{ matrix.size_limit_kb }} KB)"
|
||||
if [ "$SIZE" -gt "$MAX" ]; then
|
||||
echo "::error::Firmware binary exceeds ${{ matrix.size_limit_kb }} KB size gate ($SIZE > $MAX)"
|
||||
exit 1
|
||||
fi
|
||||
echo "Binary size OK: $SIZE <= $MAX"
|
||||
|
||||
- name: Verify flash image integrity
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
ERRORS=0
|
||||
BIN=build/esp32-csi-node.bin
|
||||
|
||||
if [ ! -s "$BIN" ]; then
|
||||
echo "::error::Binary not found or empty"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PT=build/partition_table/partition-table.bin
|
||||
if [ -f "$PT" ]; then
|
||||
MAGIC=$(od -A n -t x1 -N 2 "$PT" | tr -d ' ')
|
||||
if [ "$MAGIC" != "aa50" ]; then
|
||||
echo "::warning::Partition table magic mismatch: $MAGIC (expected aa50)"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
fi
|
||||
|
||||
BL=build/bootloader/bootloader.bin
|
||||
if [ ! -s "$BL" ]; then
|
||||
echo "::warning::Bootloader binary missing or empty"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
|
||||
NONZERO=$(od -A n -t x1 -N 1024 "$BIN" | tr -d ' f\n' | wc -c)
|
||||
if [ "$NONZERO" -lt 100 ]; then
|
||||
echo "::error::Binary appears to be mostly padding (non-zero chars: $NONZERO)"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
|
||||
if [ "$ERRORS" -gt 0 ]; then
|
||||
echo "::warning::Flash image verification completed with $ERRORS warning(s)"
|
||||
else
|
||||
echo "Flash image integrity verified"
|
||||
fi
|
||||
|
||||
- name: Stage release binaries with variant-specific names
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
mkdir -p release-staging
|
||||
cp build/esp32-csi-node.bin release-staging/${{ matrix.artifact_app }}
|
||||
cp build/partition_table/partition-table.bin release-staging/${{ matrix.artifact_pt }}
|
||||
if [ "${{ matrix.variant }}" = "8mb" ]; then
|
||||
cp build/bootloader/bootloader.bin release-staging/bootloader.bin
|
||||
cp build/ota_data_initial.bin release-staging/ota_data_initial.bin
|
||||
fi
|
||||
ls -la release-staging/
|
||||
|
||||
- name: Check QEMU ESP32-S3 support status
|
||||
run: |
|
||||
echo "::notice::ESP32-S3 QEMU support is experimental in ESP-IDF v5.4. "
|
||||
echo "Full smoke testing requires QEMU 8.2+ with xtensa-esp32s3 target."
|
||||
echo "See: https://github.com/espressif/qemu/wiki"
|
||||
|
||||
- name: Upload firmware artifact (${{ matrix.variant }})
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: esp32-csi-node-firmware-${{ matrix.variant }}
|
||||
path: firmware/esp32-csi-node/release-staging/
|
||||
retention-days: 90
|
||||
@@ -0,0 +1,378 @@
|
||||
name: Firmware QEMU Tests (ADR-061)
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- 'firmware/**'
|
||||
- 'scripts/qemu-esp32s3-test.sh'
|
||||
- 'scripts/validate_qemu_output.py'
|
||||
- 'scripts/generate_nvs_matrix.py'
|
||||
- 'scripts/qemu_swarm.py'
|
||||
- 'scripts/swarm_health.py'
|
||||
- 'scripts/swarm_presets/**'
|
||||
- '.github/workflows/firmware-qemu.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'firmware/**'
|
||||
- 'scripts/qemu-esp32s3-test.sh'
|
||||
- 'scripts/validate_qemu_output.py'
|
||||
- 'scripts/generate_nvs_matrix.py'
|
||||
- 'scripts/qemu_swarm.py'
|
||||
- 'scripts/swarm_health.py'
|
||||
- 'scripts/swarm_presets/**'
|
||||
- '.github/workflows/firmware-qemu.yml'
|
||||
|
||||
env:
|
||||
IDF_VERSION: "v5.4"
|
||||
QEMU_REPO: "https://github.com/espressif/qemu.git"
|
||||
QEMU_BRANCH: "esp-develop"
|
||||
|
||||
jobs:
|
||||
build-qemu:
|
||||
name: Build Espressif QEMU
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Cache QEMU build
|
||||
id: cache-qemu
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: /opt/qemu-esp32
|
||||
# Include date component so cache refreshes monthly when branch updates
|
||||
key: qemu-esp32s3-${{ env.QEMU_BRANCH }}-v5
|
||||
restore-keys: |
|
||||
qemu-esp32s3-${{ env.QEMU_BRANCH }}-
|
||||
|
||||
- name: Install QEMU build dependencies
|
||||
if: steps.cache-qemu.outputs.cache-hit != 'true'
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
git build-essential ninja-build pkg-config \
|
||||
libglib2.0-dev libpixman-1-dev libslirp-dev \
|
||||
libgcrypt20-dev \
|
||||
python3 python3-venv
|
||||
|
||||
- name: Clone and build Espressif QEMU
|
||||
if: steps.cache-qemu.outputs.cache-hit != 'true'
|
||||
run: |
|
||||
git clone --depth 1 -b "$QEMU_BRANCH" "$QEMU_REPO" /tmp/qemu-esp
|
||||
cd /tmp/qemu-esp
|
||||
mkdir build && cd build
|
||||
../configure \
|
||||
--target-list=xtensa-softmmu \
|
||||
--prefix=/opt/qemu-esp32 \
|
||||
--enable-slirp \
|
||||
--disable-werror
|
||||
ninja -j$(nproc)
|
||||
ninja install
|
||||
|
||||
- name: Verify QEMU binary
|
||||
run: |
|
||||
file_size() { stat -c%s "$1" 2>/dev/null || stat -f%z "$1" 2>/dev/null || wc -c < "$1"; }
|
||||
/opt/qemu-esp32/bin/qemu-system-xtensa --version
|
||||
echo "QEMU binary size: $(file_size /opt/qemu-esp32/bin/qemu-system-xtensa) bytes"
|
||||
|
||||
- name: Upload QEMU artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: qemu-esp32
|
||||
path: /opt/qemu-esp32/
|
||||
retention-days: 7
|
||||
|
||||
qemu-test:
|
||||
name: QEMU Test (${{ matrix.nvs_config }})
|
||||
needs: build-qemu
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: espressif/idf:v5.4
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
nvs_config:
|
||||
- default
|
||||
- full-adr060
|
||||
- edge-tier0
|
||||
- edge-tier1
|
||||
- tdm-3node
|
||||
- boundary-max
|
||||
- boundary-min
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Download QEMU artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: qemu-esp32
|
||||
path: /opt/qemu-esp32
|
||||
|
||||
- name: Make QEMU executable
|
||||
run: chmod +x /opt/qemu-esp32/bin/qemu-system-xtensa
|
||||
|
||||
- name: Verify QEMU works
|
||||
run: /opt/qemu-esp32/bin/qemu-system-xtensa --version
|
||||
|
||||
- name: Install Python dependencies
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
pip install esptool esp-idf-nvs-partition-gen
|
||||
|
||||
- name: Set target ESP32-S3
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
idf.py set-target esp32s3
|
||||
|
||||
- name: Build firmware (mock CSI mode)
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
idf.py \
|
||||
-D SDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.qemu" \
|
||||
build
|
||||
|
||||
- name: Generate NVS matrix
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
python3 scripts/generate_nvs_matrix.py \
|
||||
--output-dir firmware/esp32-csi-node/build/nvs_matrix \
|
||||
--only ${{ matrix.nvs_config }}
|
||||
|
||||
- name: Create merged flash image
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
|
||||
# Determine merge_bin arguments
|
||||
OTA_ARGS=""
|
||||
if [ -f build/ota_data_initial.bin ]; then
|
||||
OTA_ARGS="0xf000 build/ota_data_initial.bin"
|
||||
fi
|
||||
|
||||
python3 -m esptool --chip esp32s3 merge_bin \
|
||||
-o build/qemu_flash.bin \
|
||||
--flash_mode dio --flash_freq 80m --flash_size 8MB \
|
||||
--fill-flash-size 8MB \
|
||||
0x0 build/bootloader/bootloader.bin \
|
||||
0x8000 build/partition_table/partition-table.bin \
|
||||
$OTA_ARGS \
|
||||
0x20000 build/esp32-csi-node.bin
|
||||
|
||||
file_size() { stat -c%s "$1" 2>/dev/null || stat -f%z "$1" 2>/dev/null || wc -c < "$1"; }
|
||||
echo "Flash image size: $(file_size build/qemu_flash.bin) bytes"
|
||||
|
||||
- name: Inject NVS partition
|
||||
if: matrix.nvs_config != 'default'
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
NVS_BIN="build/nvs_matrix/nvs_${{ matrix.nvs_config }}.bin"
|
||||
if [ -f "$NVS_BIN" ]; then
|
||||
file_size() { stat -c%s "$1" 2>/dev/null || stat -f%z "$1" 2>/dev/null || wc -c < "$1"; }
|
||||
echo "Injecting NVS: $NVS_BIN ($(file_size "$NVS_BIN") bytes)"
|
||||
dd if="$NVS_BIN" of=build/qemu_flash.bin \
|
||||
bs=1 seek=$((0x9000)) conv=notrunc 2>/dev/null
|
||||
else
|
||||
echo "WARNING: NVS binary not found: $NVS_BIN"
|
||||
fi
|
||||
|
||||
- name: Run QEMU smoke test
|
||||
env:
|
||||
QEMU_PATH: /opt/qemu-esp32/bin/qemu-system-xtensa
|
||||
QEMU_TIMEOUT: "90"
|
||||
run: |
|
||||
echo "Starting QEMU (timeout: ${QEMU_TIMEOUT}s)..."
|
||||
|
||||
timeout "$QEMU_TIMEOUT" "$QEMU_PATH" \
|
||||
-machine esp32s3 \
|
||||
-nographic \
|
||||
-drive file=firmware/esp32-csi-node/build/qemu_flash.bin,if=mtd,format=raw \
|
||||
-serial mon:stdio \
|
||||
-nic user,model=open_eth,net=10.0.2.0/24 \
|
||||
-no-reboot \
|
||||
2>&1 | tee firmware/esp32-csi-node/build/qemu_output.log || true
|
||||
|
||||
echo "QEMU finished. Log size: $(wc -l < firmware/esp32-csi-node/build/qemu_output.log) lines"
|
||||
|
||||
- name: Validate QEMU output
|
||||
run: |
|
||||
python3 scripts/validate_qemu_output.py \
|
||||
firmware/esp32-csi-node/build/qemu_output.log
|
||||
|
||||
- name: Upload test logs
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: qemu-logs-${{ matrix.nvs_config }}
|
||||
path: |
|
||||
firmware/esp32-csi-node/build/qemu_output.log
|
||||
firmware/esp32-csi-node/build/nvs_matrix/
|
||||
retention-days: 14
|
||||
|
||||
fuzz-test:
|
||||
name: Fuzz Testing (ADR-061 Layer 6)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install clang
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y clang
|
||||
|
||||
- name: Build fuzz targets
|
||||
working-directory: firmware/esp32-csi-node/test
|
||||
run: make all CC=clang
|
||||
|
||||
- name: Run serialize fuzzer (60s)
|
||||
working-directory: firmware/esp32-csi-node/test
|
||||
run: make run_serialize FUZZ_DURATION=60 || echo "FUZZER_CRASH=serialize" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Run edge enqueue fuzzer (60s)
|
||||
working-directory: firmware/esp32-csi-node/test
|
||||
run: make run_edge FUZZ_DURATION=60 || echo "FUZZER_CRASH=edge" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Run NVS config fuzzer (60s)
|
||||
working-directory: firmware/esp32-csi-node/test
|
||||
run: make run_nvs FUZZ_DURATION=60 || echo "FUZZER_CRASH=nvs" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Check for crashes
|
||||
working-directory: firmware/esp32-csi-node/test
|
||||
run: |
|
||||
CRASHES=$(find . -type f \( -name "crash-*" -o -name "oom-*" -o -name "timeout-*" \) 2>/dev/null | wc -l)
|
||||
echo "Crash artifacts found: $CRASHES"
|
||||
if [ "$CRASHES" -gt 0 ] || [ -n "${FUZZER_CRASH:-}" ]; then
|
||||
echo "::error::Fuzzer found $CRASHES crash/oom/timeout artifacts. FUZZER_CRASH=${FUZZER_CRASH:-none}"
|
||||
ls -la crash-* oom-* timeout-* 2>/dev/null
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Upload fuzz artifacts
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: fuzz-crashes
|
||||
path: |
|
||||
firmware/esp32-csi-node/test/crash-*
|
||||
firmware/esp32-csi-node/test/oom-*
|
||||
firmware/esp32-csi-node/test/timeout-*
|
||||
retention-days: 30
|
||||
|
||||
nvs-matrix-validate:
|
||||
name: NVS Matrix Generation
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install NVS generator
|
||||
run: pip install esp-idf-nvs-partition-gen
|
||||
|
||||
- name: Generate all 14 NVS configs
|
||||
run: |
|
||||
python3 scripts/generate_nvs_matrix.py \
|
||||
--output-dir build/nvs_matrix
|
||||
|
||||
- name: Verify all binaries generated
|
||||
run: |
|
||||
EXPECTED=14
|
||||
ACTUAL=$(find build/nvs_matrix -type f -name "nvs_*.bin" 2>/dev/null | wc -l)
|
||||
echo "Generated $ACTUAL / $EXPECTED NVS binaries"
|
||||
ls -la build/nvs_matrix/
|
||||
|
||||
if [ "$ACTUAL" -lt "$EXPECTED" ]; then
|
||||
echo "::error::Only $ACTUAL of $EXPECTED NVS binaries generated"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Verify binary sizes
|
||||
run: |
|
||||
file_size() { stat -c%s "$1" 2>/dev/null || stat -f%z "$1" 2>/dev/null || wc -c < "$1"; }
|
||||
for f in build/nvs_matrix/nvs_*.bin; do
|
||||
SIZE=$(file_size "$f")
|
||||
if [ "$SIZE" -ne 24576 ]; then
|
||||
echo "::error::$f has unexpected size $SIZE (expected 24576)"
|
||||
exit 1
|
||||
fi
|
||||
echo " OK: $(basename $f) ($SIZE bytes)"
|
||||
done
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-062: QEMU Swarm Configurator Test
|
||||
#
|
||||
# Runs a lightweight 3-node swarm (ci_matrix preset) under QEMU to validate
|
||||
# multi-node orchestration, TDM slot coordination, and swarm-level health
|
||||
# assertions. Uses the pre-built QEMU binary from the build-qemu job and the
|
||||
# firmware built by qemu-test.
|
||||
#
|
||||
# The CI runner is non-root, so TAP bridge networking is unavailable.
|
||||
# The orchestrator (qemu_swarm.py) detects this and falls back to SLIRP
|
||||
# user-mode networking, which is sufficient for the ci_matrix preset.
|
||||
# ---------------------------------------------------------------------------
|
||||
swarm-test:
|
||||
name: Swarm Test (ADR-062)
|
||||
needs: [build-qemu]
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: espressif/idf:v5.4
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Download QEMU artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: qemu-esp32
|
||||
path: /opt/qemu-esp32
|
||||
|
||||
- name: Make QEMU executable
|
||||
run: chmod +x /opt/qemu-esp32/bin/qemu-system-xtensa
|
||||
|
||||
- name: Install Python dependencies
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
pip install pyyaml esptool esp-idf-nvs-partition-gen
|
||||
|
||||
- name: Build firmware for swarm
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
idf.py set-target esp32s3
|
||||
idf.py -D SDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.qemu" build
|
||||
python3 -m esptool --chip esp32s3 merge_bin \
|
||||
-o build/qemu_flash.bin \
|
||||
--flash_mode dio --flash_freq 80m --flash_size 8MB \
|
||||
--fill-flash-size 8MB \
|
||||
0x0 build/bootloader/bootloader.bin \
|
||||
0x8000 build/partition_table/partition-table.bin \
|
||||
0x20000 build/esp32-csi-node.bin
|
||||
|
||||
- name: Run swarm smoke test
|
||||
run: |
|
||||
. $IDF_PATH/export.sh
|
||||
EXIT_CODE=0
|
||||
python3 scripts/qemu_swarm.py --preset ci_matrix \
|
||||
--qemu-path /opt/qemu-esp32/bin/qemu-system-xtensa \
|
||||
--output-dir build/swarm-results || EXIT_CODE=$?
|
||||
# Exit 0=PASS, 1=WARN (acceptable in CI without real hardware)
|
||||
if [ "$EXIT_CODE" -gt 1 ]; then
|
||||
echo "Swarm test failed with exit code $EXIT_CODE"
|
||||
exit "$EXIT_CODE"
|
||||
fi
|
||||
timeout-minutes: 10
|
||||
|
||||
- name: Upload swarm results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: swarm-results
|
||||
path: |
|
||||
build/swarm-results/
|
||||
retention-days: 14
|
||||
@@ -0,0 +1,56 @@
|
||||
name: Fix-Marker Regression Guard
|
||||
|
||||
# Asserts that previously-shipped fixes are still present in the tree.
|
||||
# Manifest: scripts/fix-markers.json Checker: scripts/check_fix_markers.py
|
||||
# Run locally: python scripts/check_fix_markers.py (also --list / --json)
|
||||
#
|
||||
# This complements the heavyweight checks (firmware build, deterministic
|
||||
# pipeline proof, witness bundle) with a fast per-PR "did someone revert a
|
||||
# known fix?" gate — the CI analogue of the ruflo witness fix-marker system.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- master
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
fix-markers:
|
||||
name: Verify fix markers
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Validate the manifest is well-formed JSON
|
||||
run: python -c "import json; json.load(open('scripts/fix-markers.json')); print('manifest OK')"
|
||||
|
||||
- name: Check fix markers
|
||||
run: python scripts/check_fix_markers.py
|
||||
|
||||
- name: Emit machine-readable result (for the run summary)
|
||||
if: always()
|
||||
run: |
|
||||
python scripts/check_fix_markers.py --json > fix-markers-result.json || true
|
||||
{
|
||||
echo '### Fix-marker regression guard'
|
||||
echo ''
|
||||
echo '```'
|
||||
python scripts/check_fix_markers.py || true
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Upload result artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: fix-markers-result
|
||||
path: fix-markers-result.json
|
||||
retention-days: 30
|
||||
@@ -0,0 +1,112 @@
|
||||
name: ADR-115 MQTT integration tests
|
||||
|
||||
# Runs the Mosquitto-broker-backed integration tests for ADR-115's MQTT
|
||||
# publisher. These prove the publisher reaches a real broker, emits the
|
||||
# expected HA-discovery topic shape, and honours --privacy-mode at the
|
||||
# wire boundary (not just in unit-test logic).
|
||||
#
|
||||
# Default `cargo test --workspace` does not run these tests because they
|
||||
# require a broker and pull rumqttc into the build. This workflow opts
|
||||
# into both by setting --features mqtt and RUVIEW_RUN_INTEGRATION=1.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'v2/crates/wifi-densepose-sensing-server/src/mqtt/**'
|
||||
- 'v2/crates/wifi-densepose-sensing-server/tests/mqtt_integration.rs'
|
||||
- 'v2/crates/wifi-densepose-sensing-server/Cargo.toml'
|
||||
- '.github/workflows/mqtt-integration.yml'
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'v2/crates/wifi-densepose-sensing-server/src/mqtt/**'
|
||||
workflow_dispatch: {}
|
||||
|
||||
jobs:
|
||||
mqtt-integration:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
|
||||
# NB: we don't use a `services:` mosquitto container here because the
|
||||
# eclipse-mosquitto:2.x image rejects anonymous connections by default
|
||||
# and GH Actions `services` doesn't easily support mounting a custom
|
||||
# config file. We start mosquitto manually in a step below with an
|
||||
# inline `allow_anonymous true` config.
|
||||
|
||||
env:
|
||||
RUVIEW_RUN_INTEGRATION: "1"
|
||||
RUVIEW_TEST_MQTT_PORT: "11883"
|
||||
CARGO_TERM_COLOR: always
|
||||
RUST_BACKTRACE: 1
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Install mosquitto + clients and start with allow_anonymous
|
||||
run: |
|
||||
sudo apt-get update -qq
|
||||
sudo apt-get install -y mosquitto mosquitto-clients
|
||||
sudo systemctl stop mosquitto || true
|
||||
# Inline config: anon listener on 11883 only — no TLS, no auth,
|
||||
# OK for CI because we test the wire shape, not security.
|
||||
# Production deployments enable mTLS per ADR-115 §3.9.
|
||||
cat > /tmp/mosquitto-ci.conf <<'EOF'
|
||||
listener 11883
|
||||
allow_anonymous true
|
||||
persistence false
|
||||
log_dest stdout
|
||||
EOF
|
||||
mosquitto -c /tmp/mosquitto-ci.conf -d
|
||||
for i in {1..20}; do
|
||||
if mosquitto_pub -h 127.0.0.1 -p 11883 -t healthcheck -m ok -q 0 2>/dev/null; then
|
||||
echo "mosquitto reachable on 11883"; exit 0
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
echo "mosquitto never became reachable" >&2
|
||||
tail -50 /var/log/mosquitto/*.log 2>/dev/null || true
|
||||
exit 1
|
||||
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
toolchain: stable
|
||||
|
||||
- name: Cache cargo registry + build
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: v2 -> target
|
||||
|
||||
- name: Validate HA Blueprints
|
||||
run: |
|
||||
python -m pip install --quiet pyyaml
|
||||
python scripts/validate-ha-blueprints.py
|
||||
|
||||
- name: Verify unit tests still pass under --features mqtt
|
||||
working-directory: v2
|
||||
# `cargo test` accepts a single TESTNAME filter, so we run the
|
||||
# whole --lib suite here. That gives us the full 410-test green
|
||||
# bar under --features mqtt (which is more reassuring than
|
||||
# filtering anyway).
|
||||
run: >-
|
||||
cargo test -p wifi-densepose-sensing-server
|
||||
--features mqtt --no-default-features
|
||||
--lib
|
||||
--no-fail-fast
|
||||
|
||||
- name: Run integration tests against mosquitto
|
||||
working-directory: v2
|
||||
run: >-
|
||||
cargo test -p wifi-densepose-sensing-server
|
||||
--features mqtt --no-default-features
|
||||
--test mqtt_integration
|
||||
--no-fail-fast
|
||||
-- --test-threads=1 --nocapture
|
||||
|
||||
- name: Dump broker logs on failure
|
||||
if: failure()
|
||||
run: |
|
||||
docker ps -a
|
||||
docker logs $(docker ps -aqf "ancestor=eclipse-mosquitto:2.0.18") || true
|
||||
@@ -0,0 +1,148 @@
|
||||
# ADR-265 D1 — the npm-package gate.
|
||||
#
|
||||
# Every Node package in this repo (published or private) gets: install, build,
|
||||
# tests, a version-literal gate (D3 — package.json is the only place a version
|
||||
# lives), a pack-content gate (no source maps, unpacked-size budget), a
|
||||
# tarball-install smoke test (would have caught ADR-264 F1's broken `require`
|
||||
# export), and the claim-check honesty lint on the README (D4).
|
||||
|
||||
name: npm packages
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'harness/ruview/**'
|
||||
- 'tools/ruview-mcp/**'
|
||||
- 'tools/ruview-cli/**'
|
||||
- '.github/workflows/npm-packages.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'harness/ruview/**'
|
||||
- 'tools/ruview-mcp/**'
|
||||
- 'tools/ruview-cli/**'
|
||||
- '.github/workflows/npm-packages.yml'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
gate:
|
||||
name: ${{ matrix.package.dir }} (node ${{ matrix.node }})
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
node: ['20', '22']
|
||||
package:
|
||||
- dir: harness/ruview
|
||||
build: false
|
||||
publishable: true
|
||||
# ADR-263: dependency-free harness; budget guards against dep creep.
|
||||
unpacked_budget: 65536
|
||||
- dir: tools/ruview-mcp
|
||||
build: true
|
||||
publishable: true
|
||||
# ADR-264 O2: map-free tarball (was 188 kB with maps).
|
||||
unpacked_budget: 140000
|
||||
- dir: tools/ruview-cli
|
||||
build: true
|
||||
publishable: false
|
||||
unpacked_budget: 0
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ matrix.package.dir }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ matrix.node }}
|
||||
|
||||
# Repo policy gitignores lockfiles under harness/ (the harness is
|
||||
# dependency-free anyway); the TS packages commit theirs.
|
||||
- name: Install
|
||||
run: |
|
||||
if [ -f package-lock.json ]; then npm ci; else npm install --no-fund --no-audit; fi
|
||||
|
||||
- name: Build
|
||||
if: ${{ matrix.package.build }}
|
||||
run: npm run build
|
||||
|
||||
- name: Test
|
||||
run: npm test --if-present
|
||||
|
||||
# ADR-265 D3 — package.json is the only place a version string lives.
|
||||
- name: Version-literal gate
|
||||
run: |
|
||||
set -euo pipefail
|
||||
hits=""
|
||||
for d in src bin; do
|
||||
if [ -d "$d" ]; then
|
||||
hits+=$(grep -rEn '\b[0-9]+\.[0-9]+\.[0-9]+\b' "$d" | grep -vE '127\.0\.0\.1|0\.0\.0\.0' || true)
|
||||
fi
|
||||
done
|
||||
if [ -n "$hits" ]; then
|
||||
echo "Hardcoded version-like literals found (read package.json instead — ADR-265 D3):"
|
||||
echo "$hits"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ADR-265 D1.3 — pack-content gate: no maps, size budget enforced.
|
||||
- name: Pack gate
|
||||
if: ${{ matrix.package.publishable }}
|
||||
run: |
|
||||
npm pack --dry-run --json 2>/dev/null | node -e "
|
||||
const [info] = JSON.parse(require('fs').readFileSync(0, 'utf8'));
|
||||
const budget = Number(process.env.UNPACKED_BUDGET);
|
||||
const maps = info.files.filter((f) => f.path.endsWith('.map'));
|
||||
if (maps.length > 0) {
|
||||
console.error('Tarball contains source maps (ADR-264 F2):', maps.map((m) => m.path));
|
||||
process.exit(1);
|
||||
}
|
||||
if (info.unpackedSize > budget) {
|
||||
console.error(\`Unpacked size \${info.unpackedSize} B exceeds budget \${budget} B\`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(\`pack gate OK: \${info.files.length} files, \${info.unpackedSize} B unpacked (budget \${budget} B), 0 maps\`);
|
||||
"
|
||||
env:
|
||||
UNPACKED_BUDGET: ${{ matrix.package.unpacked_budget }}
|
||||
|
||||
# ADR-265 D1.4 — install the real tarball and drive each bin/export.
|
||||
- name: Tarball smoke test
|
||||
if: ${{ matrix.package.publishable }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TGZ="$PWD/$(npm pack --silent 2>/dev/null | tail -1)"
|
||||
SMOKE="$(mktemp -d)"
|
||||
cd "$SMOKE"
|
||||
npm init -y > /dev/null
|
||||
npm i --no-fund --no-audit "$TGZ"
|
||||
case "${{ matrix.package.dir }}" in
|
||||
harness/ruview)
|
||||
./node_modules/.bin/ruview --version
|
||||
./node_modules/.bin/ruview doctor
|
||||
# the honesty gate must fail closed on empty input (ADR-263 F1)
|
||||
if ./node_modules/.bin/ruview claim-check; then
|
||||
echo 'claim-check passed with no input — fail-open regression'; exit 1
|
||||
fi
|
||||
node --input-type=module -e "const m = await import('@ruvnet/ruview'); if (!m.TOOLS) process.exit(1);"
|
||||
;;
|
||||
tools/ruview-mcp)
|
||||
# initialize over stdio; server must answer and exit 0 on EOF
|
||||
printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"ci","version":"0"}}}\n' \
|
||||
| timeout 30 ./node_modules/.bin/rvagent | grep -q '"serverInfo"'
|
||||
# the ESM export must resolve from the installed tarball (ADR-264 F1)
|
||||
timeout 30 node --input-type=module -e "await import('@ruvnet/rvagent');" < /dev/null
|
||||
;;
|
||||
esac
|
||||
|
||||
# ADR-265 D4 — package READMEs must pass the project's own honesty lint.
|
||||
- name: Claim-check README
|
||||
run: |
|
||||
if [ -f README.md ]; then
|
||||
node "$GITHUB_WORKSPACE/harness/ruview/bin/cli.js" claim-check --file README.md
|
||||
else
|
||||
echo "no README.md — skipping"
|
||||
fi
|
||||
@@ -0,0 +1,71 @@
|
||||
name: nvsim-server → ghcr.io
|
||||
|
||||
# Builds and publishes the nvsim-server Docker image to ghcr.io on:
|
||||
# - push to main affecting nvsim-server or nvsim
|
||||
# - tag push matching nvsim-server-v*
|
||||
# - manual workflow_dispatch
|
||||
#
|
||||
# ADR-092 §6.2 + §9.4.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'v2/crates/nvsim-server/**'
|
||||
- 'v2/crates/nvsim/**'
|
||||
- '.github/workflows/nvsim-server-docker.yml'
|
||||
tags: ['nvsim-server-v*']
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
jobs:
|
||||
build-and-publish:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata
|
||||
id: meta
|
||||
uses: docker/metadata-action@v6
|
||||
with:
|
||||
images: ghcr.io/ruvnet/nvsim-server
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=tag
|
||||
type=sha,format=short
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
- name: Build + push
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: v2
|
||||
file: v2/crates/nvsim-server/Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Smoke-test the image
|
||||
run: |
|
||||
docker pull ghcr.io/ruvnet/nvsim-server:sha-${GITHUB_SHA::7} || \
|
||||
docker pull ghcr.io/ruvnet/nvsim-server:latest
|
||||
docker run --rm -d --name nvsim-test -p 7878:7878 \
|
||||
ghcr.io/ruvnet/nvsim-server:latest
|
||||
sleep 4
|
||||
curl -fsS http://localhost:7878/api/health
|
||||
docker stop nvsim-test
|
||||
@@ -0,0 +1,292 @@
|
||||
# ADR-117 P5 — cibuildwheel + PyPI publish workflow for `wifi-densepose`
|
||||
#
|
||||
# This workflow is **explicitly NOT** triggered on every push. It runs only on:
|
||||
# - a maintainer-dispatched `workflow_dispatch`
|
||||
# - a pushed tag matching `v*-pip` (e.g. `v2.0.0-pip`)
|
||||
#
|
||||
# The reason for the `-pip` tag suffix is that the repo already cuts
|
||||
# `v0.X.Y-esp32` tags for firmware releases (see CLAUDE.md). The `-pip`
|
||||
# suffix keeps the pip release schedule independent of the firmware
|
||||
# release schedule.
|
||||
#
|
||||
# Sequencing on release day (per ADR-117 §7.3):
|
||||
# 1. cut tag `v1.99.0-pip` → publishes the tombstone wheel first
|
||||
# 2. cut tag `v2.0.0-pip` → publishes the PyO3 v2 wheel matrix
|
||||
#
|
||||
# Publishes via the `PYPI_API_TOKEN` GitHub Actions secret. The
|
||||
# token-refresh runbook (GCP Secret Manager → gh secret set) lives in
|
||||
# docs/integrations/pypi-release.md so KICS does not flag the
|
||||
# secret name as a generic-secret literal in the workflow.
|
||||
#
|
||||
# Q3 (witness hash v2 — open in ADR-117 §11.3) MUST be resolved
|
||||
# before the first v2.0.0 publish. When v2 lands, add a parallel
|
||||
# step that verifies the v2 hash against the Rust pipeline.
|
||||
|
||||
name: pip-release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
target:
|
||||
description: "Which package to release"
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- v2-wheels
|
||||
- v1-99-tombstone
|
||||
publish_to:
|
||||
description: "Where to publish"
|
||||
required: true
|
||||
default: testpypi
|
||||
type: choice
|
||||
options:
|
||||
- testpypi # dry-run target
|
||||
- pypi # production
|
||||
push:
|
||||
tags:
|
||||
- "v*-pip"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
# v2.0.0 — cibuildwheel matrix (5 wheels + sdist)
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
|
||||
build-wheels:
|
||||
name: Build ${{ matrix.os }} ${{ matrix.arch }}
|
||||
if: |
|
||||
github.event_name == 'workflow_dispatch' && inputs.target == 'v2-wheels' ||
|
||||
startsWith(github.ref, 'refs/tags/v2.')
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
arch: x86_64
|
||||
- os: ubuntu-latest
|
||||
arch: aarch64
|
||||
- os: macos-13 # x86_64 runner
|
||||
arch: x86_64
|
||||
- os: macos-14 # arm64 runner
|
||||
arch: arm64
|
||||
- os: windows-latest
|
||||
arch: AMD64
|
||||
runs-on: ${{ matrix.os }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
# Linux aarch64 needs QEMU for cross-build on x86_64 runners.
|
||||
- name: Set up QEMU
|
||||
if: matrix.os == 'ubuntu-latest' && matrix.arch == 'aarch64'
|
||||
uses: docker/setup-qemu-action@v3
|
||||
|
||||
# ADR-117 §5.4: abi3-py310 — one binary per OS/arch covers all
|
||||
# Python minor versions ≥ 3.10. Build only cp310 wheels.
|
||||
- name: Build wheels (cibuildwheel)
|
||||
uses: pypa/cibuildwheel@v2.21
|
||||
env:
|
||||
CIBW_BUILD: "cp310-*"
|
||||
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
|
||||
CIBW_ARCHS_MACOS: ${{ matrix.arch }}
|
||||
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
|
||||
CIBW_BUILD_FRONTEND: "build"
|
||||
CIBW_BEFORE_BUILD: "pip install maturin>=1.7"
|
||||
# The PyO3 sdist landing depends on the cargo/Rust toolchain
|
||||
# being present. cibuildwheel images carry rustup on Linux
|
||||
# but we also pin a known-good version for reproducibility.
|
||||
CIBW_BEFORE_ALL_LINUX: "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.82"
|
||||
CIBW_ENVIRONMENT_LINUX: 'PATH="$HOME/.cargo/bin:$PATH"'
|
||||
# Smoke-test every built wheel before accepting it. Catches
|
||||
# the case where the wheel imports but the compiled symbols
|
||||
# are missing.
|
||||
CIBW_TEST_REQUIRES: "pytest>=8.0"
|
||||
CIBW_TEST_COMMAND: 'python -c "import wifi_densepose; assert wifi_densepose.hello() == \"ok\"; print(wifi_densepose.__build_features__)"'
|
||||
with:
|
||||
package-dir: python
|
||||
output-dir: wheelhouse
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: wheels-${{ matrix.os }}-${{ matrix.arch }}
|
||||
path: wheelhouse/*.whl
|
||||
if-no-files-found: error
|
||||
|
||||
build-sdist:
|
||||
name: Build v2 sdist
|
||||
if: |
|
||||
github.event_name == 'workflow_dispatch' && inputs.target == 'v2-wheels' ||
|
||||
startsWith(github.ref, 'refs/tags/v2.')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
- name: Install maturin
|
||||
run: pip install maturin>=1.7
|
||||
- name: Build sdist
|
||||
working-directory: python
|
||||
run: maturin sdist --out ../sdist
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: sdist
|
||||
path: sdist/*.tar.gz
|
||||
if-no-files-found: error
|
||||
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
# v1.99.0 — tombstone wheel (pure Python, single sdist + wheel)
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
|
||||
build-tombstone:
|
||||
name: Build v1.99.0 tombstone
|
||||
if: |
|
||||
github.event_name == 'workflow_dispatch' && inputs.target == 'v1-99-tombstone' ||
|
||||
startsWith(github.ref, 'refs/tags/v1.99')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
- name: Install build backend
|
||||
run: python -m pip install --upgrade pip build>=1.2
|
||||
- name: Build sdist + wheel
|
||||
working-directory: python/tombstone
|
||||
run: python -m build --outdir ../../tombstone-dist
|
||||
# Inspect what was actually built — the previous v1.99.0-pip run
|
||||
# showed an `import wifi_densepose` that returned cleanly instead
|
||||
# of raising, even though build logs said `adding 'wifi_densepose/__init__.py'`.
|
||||
# Print the wheel manifest + the __init__.py content so any
|
||||
# future regression is debuggable from the run log alone.
|
||||
- name: Inspect wheel contents
|
||||
run: |
|
||||
set -e
|
||||
WHL=tombstone-dist/wifi_densepose-1.99.0-py3-none-any.whl
|
||||
echo "--- wheel listing ---"
|
||||
python -m zipfile -l "$WHL"
|
||||
echo "--- wifi_densepose/__init__.py inside the wheel ---"
|
||||
python -m zipfile -e "$WHL" /tmp/tomb-inspect
|
||||
cat /tmp/tomb-inspect/wifi_densepose/__init__.py
|
||||
echo "--- size in bytes ---"
|
||||
wc -c /tmp/tomb-inspect/wifi_densepose/__init__.py
|
||||
# Smoke-test in an ISOLATED venv. The previous run's failure
|
||||
# mode was that the ubuntu-latest runner's system `python` had
|
||||
# site-packages picking up something other than the user-installed
|
||||
# wheel, so the import resolved to a different module. A clean
|
||||
# venv removes any ambiguity about which wifi_densepose is loaded.
|
||||
- name: Smoke-test tombstone in isolated venv
|
||||
run: |
|
||||
set -e
|
||||
# Copy the wheel to /tmp BEFORE entering the venv — we must
|
||||
# cd OUT of the repo root because the repo contains a
|
||||
# `wifi_densepose/` directory left over from the legacy v1
|
||||
# source. Python puts cwd at sys.path[0], so an import from
|
||||
# the repo root would resolve to the legacy directory and
|
||||
# bypass the freshly-installed wheel entirely (this was the
|
||||
# silent failure mode of the previous two run attempts).
|
||||
cp tombstone-dist/wifi_densepose-1.99.0-py3-none-any.whl /tmp/
|
||||
python -m venv /tmp/smoke-venv
|
||||
/tmp/smoke-venv/bin/python -m pip install --upgrade pip
|
||||
/tmp/smoke-venv/bin/python -m pip install /tmp/wifi_densepose-1.99.0-py3-none-any.whl
|
||||
cd /tmp # away from the repo root's stray wifi_densepose/
|
||||
/tmp/smoke-venv/bin/python -c "import importlib.util as u; s = u.find_spec('wifi_densepose'); print('Resolved to:', s.origin); print('--- file content ---'); print(open(s.origin).read())"
|
||||
set +e
|
||||
/tmp/smoke-venv/bin/python -c "import wifi_densepose" 2> import-output.txt
|
||||
rc=$?
|
||||
set -e
|
||||
if [ "$rc" -eq 0 ]; then
|
||||
echo "ERROR: tombstone import succeeded — should have raised ImportError"
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q "github.com/ruvnet/RuView" import-output.txt; then
|
||||
echo "ERROR: tombstone ImportError missing migration URL"
|
||||
cat import-output.txt
|
||||
exit 1
|
||||
fi
|
||||
echo "Tombstone wheel correctly raises ImportError with migration URL."
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: tombstone
|
||||
path: tombstone-dist/*
|
||||
if-no-files-found: error
|
||||
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
# Publish — gated by manual dispatch OR by the tag form
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
|
||||
publish-v2:
|
||||
name: Publish v2 wheels
|
||||
needs: [build-wheels, build-sdist]
|
||||
if: |
|
||||
always() &&
|
||||
needs.build-wheels.result == 'success' &&
|
||||
needs.build-sdist.result == 'success' &&
|
||||
(
|
||||
github.event_name == 'workflow_dispatch' && inputs.target == 'v2-wheels' ||
|
||||
startsWith(github.ref, 'refs/tags/v2.')
|
||||
)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Gather all artifacts into dist/
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: dist-staging
|
||||
- name: Flatten artifacts
|
||||
run: |
|
||||
mkdir -p dist
|
||||
find dist-staging -type f \( -name '*.whl' -o -name '*.tar.gz' \) -exec cp -v {} dist/ \;
|
||||
ls -lh dist/
|
||||
- name: Publish to TestPyPI (dry-run target)
|
||||
if: github.event_name == 'workflow_dispatch' && inputs.publish_to == 'testpypi'
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
repository-url: https://test.pypi.org/legacy/
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
packages-dir: dist
|
||||
skip-existing: true
|
||||
- name: Publish to PyPI
|
||||
if: |
|
||||
startsWith(github.ref, 'refs/tags/v2.') ||
|
||||
(github.event_name == 'workflow_dispatch' && inputs.publish_to == 'pypi')
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
packages-dir: dist
|
||||
|
||||
publish-tombstone:
|
||||
name: Publish v1.99 tombstone
|
||||
needs: [build-tombstone]
|
||||
if: |
|
||||
always() &&
|
||||
needs.build-tombstone.result == 'success' &&
|
||||
(
|
||||
github.event_name == 'workflow_dispatch' && inputs.target == 'v1-99-tombstone' ||
|
||||
startsWith(github.ref, 'refs/tags/v1.99')
|
||||
)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: tombstone
|
||||
path: dist
|
||||
- name: Publish to TestPyPI (dry-run target)
|
||||
if: github.event_name == 'workflow_dispatch' && inputs.publish_to == 'testpypi'
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
repository-url: https://test.pypi.org/legacy/
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
packages-dir: dist
|
||||
skip-existing: true
|
||||
- name: Publish to PyPI
|
||||
if: |
|
||||
startsWith(github.ref, 'refs/tags/v1.99') ||
|
||||
(github.event_name == 'workflow_dispatch' && inputs.publish_to == 'pypi')
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
packages-dir: dist
|
||||
@@ -0,0 +1,76 @@
|
||||
name: Point Cloud Viewer → GitHub Pages
|
||||
|
||||
# Publishes the live 3D point cloud viewer to gh-pages/pointcloud/.
|
||||
# The viewer defaults to a synthetic in-browser demo; users can append
|
||||
# ?backend=<url> or ?backend=auto to point it at a real ruview-pointcloud
|
||||
# server (CORS-permitting host required). See ADR-094.
|
||||
#
|
||||
# Uses keep_files: true to preserve the existing observatory/, pose-fusion/,
|
||||
# nvsim/, and root index.html demos already on gh-pages.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'v2/crates/wifi-densepose-pointcloud/src/viewer.html'
|
||||
- '.github/workflows/pointcloud-pages.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
concurrency:
|
||||
group: pointcloud-pages
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Stage viewer for Pages
|
||||
run: |
|
||||
mkdir -p _site/pointcloud
|
||||
cp v2/crates/wifi-densepose-pointcloud/src/viewer.html _site/pointcloud/index.html
|
||||
# Drop a tiny README so direct browsers of the directory get context.
|
||||
cat > _site/pointcloud/README.md <<'EOF'
|
||||
# RuView — Live 3D Point Cloud Viewer
|
||||
|
||||
Hosted at: https://ruvnet.github.io/RuView/pointcloud/
|
||||
|
||||
## Modes
|
||||
|
||||
- Default — synthetic in-browser demo (no backend, no network calls).
|
||||
- `?backend=auto` — fetch from `/api/splats` on the same origin
|
||||
(only works when the viewer is served by `ruview-pointcloud serve`).
|
||||
- `?backend=<url>` — fetch from `<url>/api/splats`. The intended
|
||||
local-ESP32 use is `?backend=http://127.0.0.1:9880`: run
|
||||
`ruview-pointcloud serve --bind 127.0.0.1:9880` on the same
|
||||
machine with your ESP32 streaming CSI to UDP port 3333, then
|
||||
visit the URL above. The local server's CorsLayer permits
|
||||
requests from `https://ruvnet.github.io`, and modern browsers
|
||||
permit HTTPS→127.0.0.1 mixed-content as a trustworthy origin.
|
||||
The "📡 Connect ESP32" button in the viewer prompts for this
|
||||
URL and persists it in localStorage.
|
||||
- `?live=1` — require a live backend; show an offline message instead
|
||||
of falling back to the synthetic demo.
|
||||
|
||||
See ADR-094 for the deployment design.
|
||||
EOF
|
||||
|
||||
- name: Deploy to gh-pages/pointcloud/
|
||||
uses: peaceiris/actions-gh-pages@v4
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_dir: ./_site/pointcloud
|
||||
destination_dir: pointcloud
|
||||
# CRITICAL: preserves observatory/, pose-fusion/, nvsim/, and root
|
||||
# index.html already on gh-pages.
|
||||
keep_files: true
|
||||
commit_message: 'deploy(pointcloud): ${{ github.sha }}'
|
||||
user_name: 'github-actions[bot]'
|
||||
user_email: 'github-actions[bot]@users.noreply.github.com'
|
||||
@@ -0,0 +1,137 @@
|
||||
# ADR-265 D2 — publish only from CI, with provenance.
|
||||
#
|
||||
# Manual `npm publish` from laptops stops: this workflow re-runs the ADR-265 D1
|
||||
# gate for the selected package and then publishes with npm provenance
|
||||
# attestations (OIDC), tying every published version to a public commit +
|
||||
# workflow run — the npm-side analogue of the ADR-028 witness bundle.
|
||||
#
|
||||
# Requires: NPM_TOKEN repo secret (an npm automation token), or npm Trusted
|
||||
# Publishing configured for the package (in which case the token is unused).
|
||||
|
||||
name: ruview npm release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
package:
|
||||
description: 'Package directory to publish'
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- harness/ruview
|
||||
- tools/ruview-mcp
|
||||
dist_tag:
|
||||
description: 'npm dist-tag'
|
||||
required: false
|
||||
default: 'latest'
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # npm --provenance
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ inputs.package }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
registry-url: 'https://registry.npmjs.org'
|
||||
|
||||
- name: Install
|
||||
run: |
|
||||
if [ -f package-lock.json ]; then npm ci; else npm install --no-fund --no-audit; fi
|
||||
|
||||
- name: Build (if present)
|
||||
run: npm run build --if-present
|
||||
|
||||
- name: Test
|
||||
run: npm test --if-present
|
||||
|
||||
# ADR-265 D3 — package.json is the only place a version string lives.
|
||||
- name: Version-literal gate
|
||||
run: |
|
||||
set -euo pipefail
|
||||
hits=""
|
||||
for d in src bin; do
|
||||
if [ -d "$d" ]; then
|
||||
hits+=$(grep -rEn '\b[0-9]+\.[0-9]+\.[0-9]+\b' "$d" | grep -vE '127\.0\.0\.1|0\.0\.0\.0' || true)
|
||||
fi
|
||||
done
|
||||
if [ -n "$hits" ]; then
|
||||
echo "Hardcoded version-like literals found (read package.json instead — ADR-265 D3):"
|
||||
echo "$hits"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ADR-265 D1.3 — pack-content gate: no maps AND the per-package
|
||||
# unpacked-size budget (the budgets that npm-packages.yml enforces).
|
||||
- name: Pack gate (no maps + size budget)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
case "${{ inputs.package }}" in
|
||||
# ADR-263: dependency-free harness; budget guards against dep creep.
|
||||
harness/ruview) export UNPACKED_BUDGET=65536 ;;
|
||||
# ADR-264 O2: map-free tarball (was 188 kB with maps).
|
||||
tools/ruview-mcp) export UNPACKED_BUDGET=140000 ;;
|
||||
*) echo "Unknown package '${{ inputs.package }}' — no budget defined"; exit 1 ;;
|
||||
esac
|
||||
npm pack --dry-run --json 2>/dev/null | node -e "
|
||||
const [info] = JSON.parse(require('fs').readFileSync(0, 'utf8'));
|
||||
const budget = Number(process.env.UNPACKED_BUDGET);
|
||||
const maps = info.files.filter((f) => f.path.endsWith('.map'));
|
||||
if (maps.length > 0) {
|
||||
console.error('Tarball contains source maps (ADR-264 F2):', maps.map((m) => m.path));
|
||||
process.exit(1);
|
||||
}
|
||||
if (info.unpackedSize > budget) {
|
||||
console.error(\`Unpacked size \${info.unpackedSize} B exceeds budget \${budget} B\`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(\`pack gate OK: \${info.files.length} files, \${info.unpackedSize} B unpacked (budget \${budget} B), 0 maps\`);
|
||||
"
|
||||
|
||||
# ADR-265 D1.4 — install the real tarball and drive each bin/export.
|
||||
- name: Tarball smoke test
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TGZ="$PWD/$(npm pack --silent 2>/dev/null | tail -1)"
|
||||
SMOKE="$(mktemp -d)"
|
||||
cd "$SMOKE"
|
||||
npm init -y > /dev/null
|
||||
npm i --no-fund --no-audit "$TGZ"
|
||||
case "${{ inputs.package }}" in
|
||||
harness/ruview)
|
||||
./node_modules/.bin/ruview --version
|
||||
./node_modules/.bin/ruview doctor
|
||||
# the honesty gate must fail closed on empty input (ADR-263 F1)
|
||||
if ./node_modules/.bin/ruview claim-check; then
|
||||
echo 'claim-check passed with no input — fail-open regression'; exit 1
|
||||
fi
|
||||
node --input-type=module -e "const m = await import('@ruvnet/ruview'); if (!m.TOOLS) process.exit(1);"
|
||||
;;
|
||||
tools/ruview-mcp)
|
||||
# initialize over stdio; server must answer and exit 0 on EOF
|
||||
printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"ci","version":"0"}}}\n' \
|
||||
| timeout 30 ./node_modules/.bin/rvagent | grep -q '"serverInfo"'
|
||||
# the ESM export must resolve from the installed tarball (ADR-264 F1)
|
||||
timeout 30 node --input-type=module -e "await import('@ruvnet/rvagent');" < /dev/null
|
||||
;;
|
||||
esac
|
||||
|
||||
- name: Claim-check README
|
||||
run: |
|
||||
if [ -f README.md ]; then
|
||||
node "$GITHUB_WORKSPACE/harness/ruview/bin/cli.js" claim-check --file README.md
|
||||
fi
|
||||
|
||||
- name: Publish (with provenance)
|
||||
run: npm publish --provenance --access public --tag "${{ inputs.dist_tag }}"
|
||||
env:
|
||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
||||
@@ -0,0 +1,157 @@
|
||||
name: ruview-swarm CI guard
|
||||
|
||||
# Dedicated guard for the ADR-148 drone swarm crate (`v2/crates/ruview-swarm`).
|
||||
# The main ci.yml runs `cargo test --workspace --no-default-features`, which
|
||||
# only exercises ruview-swarm's DEFAULT feature set. This guard additionally:
|
||||
# - tests every feature combination (train / ruflo+itar / full)
|
||||
# - fails on ANY clippy warning in the crate's own code (--no-deps)
|
||||
# - asserts the ITAR + publish guards stay in place (USML Cat VIII(h)(12))
|
||||
# - builds the GPU training binary under the `train` feature
|
||||
#
|
||||
# Path-scoped so it only runs when the crate or this workflow changes.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, 'feat/*' ]
|
||||
paths:
|
||||
- 'v2/crates/ruview-swarm/**'
|
||||
- '.github/workflows/ruview-swarm-ci.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'v2/crates/ruview-swarm/**'
|
||||
- '.github/workflows/ruview-swarm-ci.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
|
||||
jobs:
|
||||
# ── Feature-matrix tests ─────────────────────────────────────────────────
|
||||
tests:
|
||||
name: tests (${{ matrix.features.label }})
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
features:
|
||||
- { label: 'default', flags: '--no-default-features' }
|
||||
- { label: 'train', flags: '--features train' }
|
||||
- { label: 'ruflo', flags: '--features ruflo' }
|
||||
- { label: 'full+train', flags: '--features full,train' }
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- name: Cache cargo
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: ${{ runner.os }}-ruview-swarm-${{ hashFiles('v2/Cargo.lock') }}
|
||||
restore-keys: ${{ runner.os }}-ruview-swarm-
|
||||
- name: cargo test -p ruview-swarm ${{ matrix.features.flags }}
|
||||
working-directory: v2
|
||||
run: cargo test -p ruview-swarm ${{ matrix.features.flags }} --lib
|
||||
|
||||
# ── Clippy: zero warnings in the crate's own code ────────────────────────
|
||||
clippy:
|
||||
name: clippy (-D warnings, --no-deps)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
# v2/rust-toolchain.toml pins channel "1.89" with profile "minimal" (no
|
||||
# clippy). dtolnay@stable installs clippy on the floating "stable"
|
||||
# toolchain, but the override makes cargo use the separate "1.89"
|
||||
# toolchain — so `cargo clippy` errors "cargo-clippy is not installed for
|
||||
# 1.89". Install clippy on the pinned toolchain that cargo actually uses.
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
toolchain: "1.89"
|
||||
components: clippy
|
||||
- name: Cache cargo
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: ${{ runner.os }}-ruview-swarm-clippy-${{ hashFiles('v2/Cargo.lock') }}
|
||||
restore-keys: ${{ runner.os }}-ruview-swarm-clippy-
|
||||
# --no-deps confines linting to ruview-swarm's own source, so pre-existing
|
||||
# warnings in dependency crates don't gate this PR.
|
||||
- name: clippy (default)
|
||||
working-directory: v2
|
||||
run: cargo clippy -p ruview-swarm --no-default-features --no-deps -- -D warnings
|
||||
- name: clippy (full,train)
|
||||
working-directory: v2
|
||||
run: cargo clippy -p ruview-swarm --features full,train --no-deps -- -D warnings
|
||||
|
||||
# ── Build the GPU training binary (train feature) ────────────────────────
|
||||
train-bin:
|
||||
name: build train_marl bin
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- name: Cache cargo
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
v2/target
|
||||
key: ${{ runner.os }}-ruview-swarm-bin-${{ hashFiles('v2/Cargo.lock') }}
|
||||
restore-keys: ${{ runner.os }}-ruview-swarm-bin-
|
||||
- name: cargo build --bin train_marl --features train
|
||||
working-directory: v2
|
||||
run: cargo build -p ruview-swarm --features train --bin train_marl
|
||||
- name: train_marl is excluded from the default build
|
||||
working-directory: v2
|
||||
run: |
|
||||
# The training binary requires the `train` feature; a default `--bins`
|
||||
# build must NOT produce it (keeps default/CI builds light + Candle-free).
|
||||
# Remove any prior artifact first so this checks what the DEFAULT build
|
||||
# produces, not a leftover from the train-feature build above.
|
||||
rm -f target/debug/train_marl
|
||||
cargo build -p ruview-swarm --no-default-features --bins
|
||||
if [ -f target/debug/train_marl ]; then
|
||||
echo "ERROR: train_marl built without the 'train' feature" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: train_marl correctly gated behind the 'train' feature"
|
||||
|
||||
# ── ITAR + publish guards ────────────────────────────────────────────────
|
||||
export-control-guard:
|
||||
name: ITAR / publish guard
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
- name: publish = false is present (no accidental crates.io publish)
|
||||
run: |
|
||||
CARGO=v2/crates/ruview-swarm/Cargo.toml
|
||||
if ! grep -qE '^\s*publish\s*=\s*false' "$CARGO"; then
|
||||
echo "ERROR: ruview-swarm Cargo.toml must keep 'publish = false' until" >&2
|
||||
echo " PR merge + dependency publish + ITAR export sign-off." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: publish = false present"
|
||||
- name: default feature set does NOT enable itar-unrestricted
|
||||
run: |
|
||||
CARGO=v2/crates/ruview-swarm/Cargo.toml
|
||||
# USML Cat VIII(h)(12): swarming coordination must be opt-in, never default.
|
||||
DEFAULT_LINE=$(grep -E '^\s*default\s*=' "$CARGO" || true)
|
||||
echo "default = $DEFAULT_LINE"
|
||||
if echo "$DEFAULT_LINE" | grep -q 'itar-unrestricted'; then
|
||||
echo "ERROR: 'itar-unrestricted' must NOT be in the default feature set" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: ITAR-gated coordination features are opt-in, not default"
|
||||
@@ -0,0 +1,521 @@
|
||||
name: Security Scanning
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, develop, 'feat/*' ]
|
||||
pull_request:
|
||||
branches: [ main, develop ]
|
||||
schedule:
|
||||
# Run security scans daily at 2 AM UTC
|
||||
- cron: '0 2 * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
PYTHON_VERSION: '3.11'
|
||||
|
||||
jobs:
|
||||
# Static Application Security Testing (SAST)
|
||||
sast:
|
||||
name: Static Application Security Testing
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
permissions:
|
||||
security-events: write
|
||||
actions: read
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
continue-on-error: true
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
continue-on-error: true
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install bandit semgrep safety
|
||||
|
||||
- name: Run Bandit security scan
|
||||
run: |
|
||||
# The Python codebase lives under archive/v1/src (it moved there when
|
||||
# the runtime was rewritten in Rust). Scanning `src/` matched nothing,
|
||||
# so this SAST step was a silent no-op.
|
||||
bandit -r archive/v1/src/ -f sarif -o bandit-results.sarif
|
||||
continue-on-error: true
|
||||
|
||||
- name: Upload Bandit results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: bandit-results.sarif
|
||||
category: bandit
|
||||
|
||||
# Removed the deprecated `returntocorp/semgrep-action@v1` step: it was
|
||||
# redundant (the pip `semgrep --sarif` below is what feeds GitHub Security;
|
||||
# the action only pushed to the Semgrep cloud app via SEMGREP_APP_TOKEN) and
|
||||
# it pulled `returntocorp/semgrep-agent:v1` from Docker Hub on every run,
|
||||
# which intermittently timed out and turned this check red. The pip semgrep
|
||||
# (installed above) needs no Docker pull. The action's `p/docker` +
|
||||
# `p/kubernetes` rulesets are folded into the command below so coverage is
|
||||
# preserved.
|
||||
- name: Run Semgrep + generate SARIF
|
||||
run: |
|
||||
semgrep \
|
||||
--config=p/security-audit --config=p/secrets --config=p/python \
|
||||
--config=p/docker --config=p/kubernetes \
|
||||
--sarif --output=semgrep.sarif archive/v1/src/
|
||||
continue-on-error: true
|
||||
|
||||
- name: Upload Semgrep results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: semgrep.sarif
|
||||
category: semgrep
|
||||
|
||||
# Dependency vulnerability scanning
|
||||
dependency-scan:
|
||||
name: Dependency Vulnerability Scan
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
permissions:
|
||||
security-events: write
|
||||
actions: read
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Python
|
||||
continue-on-error: true
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
continue-on-error: true
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install safety pip-audit
|
||||
|
||||
- name: Run Safety check
|
||||
run: |
|
||||
safety check --json --output safety-report.json
|
||||
continue-on-error: true
|
||||
|
||||
- name: Run pip-audit
|
||||
run: |
|
||||
pip-audit --format=json --output=pip-audit-report.json
|
||||
continue-on-error: true
|
||||
|
||||
- name: Run Snyk vulnerability scan
|
||||
uses: snyk/actions/python@9adf32b1121593767fc3c057af55b55db032dc04 # v1.0.0
|
||||
env:
|
||||
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
|
||||
with:
|
||||
args: --sarif-file-output=snyk-results.sarif
|
||||
continue-on-error: true
|
||||
|
||||
- name: Upload Snyk results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: snyk-results.sarif
|
||||
category: snyk
|
||||
|
||||
- name: Upload vulnerability reports
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: vulnerability-reports
|
||||
path: |
|
||||
safety-report.json
|
||||
pip-audit-report.json
|
||||
snyk-results.sarif
|
||||
|
||||
# Container security scanning
|
||||
container-scan:
|
||||
name: Container Security Scan
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
needs: []
|
||||
if: github.event_name == 'push' || github.event_name == 'schedule'
|
||||
permissions:
|
||||
security-events: write
|
||||
actions: read
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
continue-on-error: true
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Build Docker image for scanning
|
||||
continue-on-error: true
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
target: production
|
||||
load: true
|
||||
tags: wifi-densepose:scan
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Run Trivy vulnerability scanner
|
||||
continue-on-error: true
|
||||
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
|
||||
with:
|
||||
image-ref: 'wifi-densepose:scan'
|
||||
format: 'sarif'
|
||||
output: 'trivy-results.sarif'
|
||||
|
||||
- name: Upload Trivy results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: 'trivy-results.sarif'
|
||||
category: trivy
|
||||
|
||||
- name: Run Grype vulnerability scanner
|
||||
continue-on-error: true
|
||||
uses: anchore/scan-action@v7
|
||||
id: grype-scan
|
||||
with:
|
||||
image: 'wifi-densepose:scan'
|
||||
fail-build: false
|
||||
severity-cutoff: high
|
||||
output-format: sarif
|
||||
|
||||
- name: Upload Grype results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: ${{ steps.grype-scan.outputs.sarif }}
|
||||
category: grype
|
||||
|
||||
- name: Run Docker Scout
|
||||
continue-on-error: true
|
||||
uses: docker/scout-action@v1
|
||||
if: always()
|
||||
with:
|
||||
command: cves
|
||||
image: wifi-densepose:scan
|
||||
sarif-file: scout-results.sarif
|
||||
summary: true
|
||||
|
||||
- name: Upload Docker Scout results
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: scout-results.sarif
|
||||
category: docker-scout
|
||||
|
||||
# Infrastructure as Code security scanning
|
||||
iac-scan:
|
||||
name: Infrastructure Security Scan
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
permissions:
|
||||
security-events: write
|
||||
actions: read
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Run Checkov IaC scan
|
||||
continue-on-error: true
|
||||
uses: bridgecrewio/checkov-action@99bb2caf247dfd9f03cf984373bc6043d4e32ebf # v12.1347.0
|
||||
with:
|
||||
directory: .
|
||||
framework: kubernetes,dockerfile,terraform,ansible
|
||||
output_format: sarif
|
||||
output_file_path: checkov-results.sarif
|
||||
quiet: true
|
||||
soft_fail: true
|
||||
|
||||
- name: Upload Checkov results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: checkov-results.sarif
|
||||
category: checkov
|
||||
|
||||
- name: Run Terrascan IaC scan
|
||||
continue-on-error: true
|
||||
uses: tenable/terrascan-action@3a6e87da8e244513bd77b631e624552643f794c6 # v1.4.1
|
||||
with:
|
||||
iac_type: 'k8s'
|
||||
iac_version: 'v1'
|
||||
policy_type: 'k8s'
|
||||
only_warn: true
|
||||
sarif_upload: true
|
||||
|
||||
- name: Run KICS IaC scan
|
||||
continue-on-error: true
|
||||
uses: checkmarx/kics-github-action@05aa5eb70eede1355220f4ca5238d96b397e30a6 # v2.1.20
|
||||
with:
|
||||
path: '.'
|
||||
output_path: kics-results
|
||||
output_formats: 'sarif'
|
||||
exclude_paths: '.git,node_modules'
|
||||
exclude_queries: 'a7ef1e8c-fbf8-4ac1-b8c7-2c3b0e6c6c6c'
|
||||
|
||||
- name: Upload KICS results to GitHub Security
|
||||
continue-on-error: true
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: kics-results/results.sarif
|
||||
category: kics
|
||||
|
||||
# Secret scanning
|
||||
secret-scan:
|
||||
name: Secret Scanning
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
permissions:
|
||||
security-events: write
|
||||
actions: read
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Run TruffleHog secret scan
|
||||
continue-on-error: true
|
||||
uses: trufflesecurity/trufflehog@17456f8c7d042d8c82c9a8ca9e937231f9f42e26 # v3.95.2
|
||||
with:
|
||||
path: ./
|
||||
base: main
|
||||
head: HEAD
|
||||
extra_args: --debug --only-verified
|
||||
|
||||
- name: Run GitLeaks secret scan
|
||||
continue-on-error: true
|
||||
uses: gitleaks/gitleaks-action@v2
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }}
|
||||
|
||||
- name: Run detect-secrets
|
||||
run: |
|
||||
pip install detect-secrets
|
||||
detect-secrets scan --all-files --baseline .secrets.baseline
|
||||
detect-secrets audit .secrets.baseline
|
||||
continue-on-error: true
|
||||
|
||||
# License compliance scanning
|
||||
license-scan:
|
||||
name: License Compliance Scan
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Python
|
||||
continue-on-error: true
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
continue-on-error: true
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
pip install pip-licenses licensecheck
|
||||
|
||||
- name: Run license check
|
||||
continue-on-error: true
|
||||
run: |
|
||||
pip-licenses --format=json --output-file=licenses.json
|
||||
licensecheck --zero
|
||||
|
||||
- name: Upload license report
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: license-report
|
||||
path: licenses.json
|
||||
|
||||
# Security policy compliance
|
||||
compliance-check:
|
||||
name: Security Policy Compliance
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
steps:
|
||||
- name: Checkout code
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Check security policy files
|
||||
continue-on-error: true
|
||||
run: |
|
||||
# Check for required security files
|
||||
files=("SECURITY.md" ".github/SECURITY.md" "docs/SECURITY.md")
|
||||
found=false
|
||||
for file in "${files[@]}"; do
|
||||
if [[ -f "$file" ]]; then
|
||||
echo "✅ Found security policy: $file"
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [[ "$found" == false ]]; then
|
||||
echo "❌ No security policy found. Please create SECURITY.md"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Check for security headers in code
|
||||
continue-on-error: true
|
||||
run: |
|
||||
# Check for security-related configurations
|
||||
grep -r "X-Frame-Options\|X-Content-Type-Options\|X-XSS-Protection\|Content-Security-Policy" src/ || echo "⚠️ Consider adding security headers"
|
||||
|
||||
- name: Validate Kubernetes security contexts
|
||||
continue-on-error: true
|
||||
run: |
|
||||
# Check for security contexts in Kubernetes manifests
|
||||
if [[ -d "k8s" ]]; then
|
||||
if find k8s/ -name "*.yaml" -exec grep -l "securityContext" {} \; | wc -l | grep -q "^0$"; then
|
||||
echo "⚠️ No security contexts found in Kubernetes manifests"
|
||||
else
|
||||
echo "✅ Security contexts found in Kubernetes manifests"
|
||||
fi
|
||||
else
|
||||
echo "ℹ️ No k8s/ directory found — skipping Kubernetes security context check"
|
||||
fi
|
||||
|
||||
# Notification and reporting
|
||||
security-report:
|
||||
name: Security Report
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true # third-party scanners are flaky / SARIF uploads can 403; don't gate the PR
|
||||
needs: [sast, dependency-scan, container-scan, iac-scan, secret-scan, license-scan, compliance-check]
|
||||
if: always()
|
||||
# Promote secret to env-scope so the gating `if:` on the Slack-notify
|
||||
# step below is parseable (GitHub Actions rejects `secrets.X` in
|
||||
# step-level `if:` expressions).
|
||||
env:
|
||||
SECURITY_SLACK_WEBHOOK_URL: ${{ secrets.SECURITY_SLACK_WEBHOOK_URL }}
|
||||
steps:
|
||||
- name: Download all artifacts
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v4
|
||||
|
||||
- name: Generate security summary
|
||||
continue-on-error: true
|
||||
run: |
|
||||
echo "# Security Scan Summary" > security-summary.md
|
||||
echo "" >> security-summary.md
|
||||
echo "## Scan Results" >> security-summary.md
|
||||
echo "- SAST: ${{ needs.sast.result }}" >> security-summary.md
|
||||
echo "- Dependency Scan: ${{ needs.dependency-scan.result }}" >> security-summary.md
|
||||
echo "- Container Scan: ${{ needs.container-scan.result }}" >> security-summary.md
|
||||
echo "- IaC Scan: ${{ needs.iac-scan.result }}" >> security-summary.md
|
||||
echo "- Secret Scan: ${{ needs.secret-scan.result }}" >> security-summary.md
|
||||
echo "- License Scan: ${{ needs.license-scan.result }}" >> security-summary.md
|
||||
echo "- Compliance Check: ${{ needs.compliance-check.result }}" >> security-summary.md
|
||||
echo "" >> security-summary.md
|
||||
echo "Generated on: $(date)" >> security-summary.md
|
||||
|
||||
- name: Upload security summary
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: security-summary
|
||||
path: security-summary.md
|
||||
|
||||
# GitHub Actions does not allow `secrets.X` in step-level `if:` —
|
||||
# use env.X instead. Inherits SECURITY_SLACK_WEBHOOK_URL from the
|
||||
# job-level env block (added below).
|
||||
- name: Notify security team on critical findings
|
||||
continue-on-error: true
|
||||
if: ${{ env.SECURITY_SLACK_WEBHOOK_URL != '' && (needs.sast.result == 'failure' || needs.dependency-scan.result == 'failure' || needs.container-scan.result == 'failure') }}
|
||||
uses: 8398a7/action-slack@v3
|
||||
with:
|
||||
status: failure
|
||||
channel: '#security'
|
||||
text: |
|
||||
🚨 Critical security findings detected!
|
||||
Repository: ${{ github.repository }}
|
||||
Branch: ${{ github.ref }}
|
||||
Workflow: ${{ github.workflow }}
|
||||
Please review the security scan results immediately.
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ env.SECURITY_SLACK_WEBHOOK_URL }}
|
||||
|
||||
- name: Create security issue on critical findings
|
||||
continue-on-error: true
|
||||
if: needs.sast.result == 'failure' || needs.dependency-scan.result == 'failure'
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
script: |
|
||||
github.rest.issues.create({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
title: `Security Scan Failures - ${new Date().toISOString()}`,
|
||||
body: `
|
||||
## Security Scan Failures Detected
|
||||
|
||||
**Workflow Run:** ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
**Branch:** ${{ github.ref }}
|
||||
|
||||
**Failed Scans:**
|
||||
- SAST: ${{ needs.sast.result }}
|
||||
- Dependency Scan: ${{ needs.dependency-scan.result }}
|
||||
- Container Scan: ${{ needs.container-scan.result }}
|
||||
|
||||
**Action Required:**
|
||||
- [ ] Review security scan results
|
||||
- [ ] Address critical vulnerabilities
|
||||
- [ ] Update dependencies if needed
|
||||
- [ ] Re-run security scans
|
||||
|
||||
**Security Dashboard:** Check the Security tab for detailed findings.
|
||||
`,
|
||||
labels: ['security', 'vulnerability', 'urgent']
|
||||
})
|
||||
@@ -0,0 +1,181 @@
|
||||
name: wifi-densepose sensing-server → Docker Hub + ghcr.io
|
||||
|
||||
# Build + publish the `wifi-densepose` sensing-server image to both Docker Hub
|
||||
# (`ruvnet/wifi-densepose`) and ghcr.io (`ghcr.io/ruvnet/wifi-densepose`) on:
|
||||
# - push to main affecting the Dockerfile, the server crate, the UI assets,
|
||||
# or this workflow itself,
|
||||
# - tag push matching v* (release builds),
|
||||
# - manual workflow_dispatch.
|
||||
#
|
||||
# Closes #520 and #514: the stale `:latest` is rebuilt and pushed automatically
|
||||
# whenever the surface that produces it changes, and the Dockerfile fails the
|
||||
# build if the observatory/pose-fusion UI assets ever go missing again.
|
||||
#
|
||||
# Secrets:
|
||||
# DOCKERHUB_USERNAME — `ruvnet` (Docker Hub login name)
|
||||
# DOCKERHUB_TOKEN — Docker Hub access token with read/write/delete scope
|
||||
# (ghcr.io uses the workflow's GITHUB_TOKEN — no secret needed.)
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'docker/Dockerfile.rust'
|
||||
- 'docker/docker-entrypoint.sh'
|
||||
- 'v2/crates/wifi-densepose-sensing-server/**'
|
||||
- 'v2/crates/wifi-densepose-signal/**'
|
||||
- 'v2/crates/wifi-densepose-vitals/**'
|
||||
- 'v2/crates/wifi-densepose-wifiscan/**'
|
||||
- 'v2/crates/wifi-densepose-bfld/**'
|
||||
- 'v2/crates/cog-ha-matter/**'
|
||||
- 'v2/Cargo.toml'
|
||||
- 'v2/Cargo.lock'
|
||||
- 'ui/**'
|
||||
- '.github/workflows/sensing-server-docker.yml'
|
||||
tags: ['v*']
|
||||
workflow_dispatch: {}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
concurrency:
|
||||
group: sensing-server-docker-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-and-publish:
|
||||
name: build · push · smoke-test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
# QEMU is required so the amd64 GitHub runner can cross-build the
|
||||
# linux/arm64 layer below (Dockerfile.rust is arch-agnostic — no `--target`
|
||||
# flag — so buildx + QEMU is all that's needed; arm64 builds are emulated
|
||||
# by the runner, not built on a separate arm64 host).
|
||||
- uses: docker/setup-qemu-action@v3
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
# Bypassing docker/login-action@v3: the action kept emitting
|
||||
# "malformed HTTP Authorization header" against a known-good
|
||||
# dckr_pat_* token (verified by direct curl against the Hub API).
|
||||
# `docker login --password-stdin` is the documented credential
|
||||
# path and avoids whatever encoding step the action injects.
|
||||
env:
|
||||
DH_USER: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DH_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
run: |
|
||||
printf '%s' "$DH_TOKEN" | docker login docker.io -u "$DH_USER" --password-stdin
|
||||
|
||||
- name: Log in to ghcr.io
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Compute tags
|
||||
id: meta
|
||||
uses: docker/metadata-action@v6
|
||||
with:
|
||||
images: |
|
||||
docker.io/ruvnet/wifi-densepose
|
||||
ghcr.io/ruvnet/wifi-densepose
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=tag
|
||||
type=sha,format=short
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
- name: Build + push
|
||||
id: build
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: docker/Dockerfile.rust
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
# README badge advertises `amd64 + arm64`, and #547 promised multi-arch
|
||||
# as part of the docker publish refresh; arm64 was never actually wired
|
||||
# in, so Apple Silicon Macs hit `no matching manifest for linux/arm64/v8`
|
||||
# on `docker pull ruvnet/wifi-densepose:latest` (#136, #625). Build both.
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
# ---------------------------------------------------------------------
|
||||
# Smoke-test the freshly-pushed image:
|
||||
# 1. UI assets that closed #520 are inside `/app/ui` (the Dockerfile's
|
||||
# RUN guard catches missing ones at build time, this re-checks the
|
||||
# pushed artifact post-hoc as belt-and-braces).
|
||||
# 2. /health is up.
|
||||
# 3. /api/v1/info returns 200 with no auth (LAN-mode default).
|
||||
# 4. With RUVIEW_API_TOKEN set, /api/v1/info returns 401 without a
|
||||
# Bearer header, 200 with the correct one (the #443 auth middleware).
|
||||
# ---------------------------------------------------------------------
|
||||
- name: Smoke-test image assets + LAN-mode HTTP
|
||||
run: |
|
||||
set -euo pipefail
|
||||
IMAGE="ghcr.io/ruvnet/wifi-densepose:sha-${GITHUB_SHA::7}"
|
||||
docker pull "$IMAGE"
|
||||
docker run --rm "$IMAGE" sh -c \
|
||||
'ls /app/ui/observatory.html /app/ui/pose-fusion.html /app/ui/index.html /app/ui/viz.html >/dev/null'
|
||||
docker run --rm "$IMAGE" sh -c 'ls -d /app/ui/observatory /app/ui/pose-fusion >/dev/null'
|
||||
|
||||
docker run -d --name sm -p 3000:3000 -e CSI_SOURCE=simulated "$IMAGE"
|
||||
# Wait up to 30 s for /health.
|
||||
for _ in $(seq 1 30); do
|
||||
if curl -fsS http://127.0.0.1:3000/health >/dev/null 2>&1; then break; fi
|
||||
sleep 1
|
||||
done
|
||||
curl -fsS http://127.0.0.1:3000/health
|
||||
curl -fsS http://127.0.0.1:3000/api/v1/info >/dev/null
|
||||
curl -fsS http://127.0.0.1:3000/ui/observatory.html >/dev/null
|
||||
curl -fsS http://127.0.0.1:3000/ui/pose-fusion.html >/dev/null
|
||||
docker stop sm
|
||||
|
||||
- name: Smoke-test the bearer-token auth path
|
||||
run: |
|
||||
set -euo pipefail
|
||||
IMAGE="ghcr.io/ruvnet/wifi-densepose:sha-${GITHUB_SHA::7}"
|
||||
docker run -d --name auth \
|
||||
-p 3000:3000 \
|
||||
-e CSI_SOURCE=simulated \
|
||||
-e RUVIEW_API_TOKEN=smoke-test-token-do-not-use \
|
||||
"$IMAGE"
|
||||
for _ in $(seq 1 30); do
|
||||
if curl -fsS http://127.0.0.1:3000/health >/dev/null 2>&1; then break; fi
|
||||
sleep 1
|
||||
done
|
||||
# /health stays unauthenticated.
|
||||
curl -fsS http://127.0.0.1:3000/health >/dev/null
|
||||
# /api/v1/info without a bearer → 401.
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:3000/api/v1/info)
|
||||
test "$code" = "401" || { echo "expected 401, got $code"; exit 1; }
|
||||
# Wrong bearer → 401.
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Authorization: Bearer wrong' http://127.0.0.1:3000/api/v1/info)
|
||||
test "$code" = "401" || { echo "expected 401 (wrong token), got $code"; exit 1; }
|
||||
# Correct bearer → 200.
|
||||
curl -fsS -H 'Authorization: Bearer smoke-test-token-do-not-use' http://127.0.0.1:3000/api/v1/info >/dev/null
|
||||
docker stop auth
|
||||
|
||||
- name: Summary
|
||||
if: always()
|
||||
run: |
|
||||
{
|
||||
echo "## sensing-server image published"
|
||||
echo
|
||||
echo "Tags:"
|
||||
echo '```'
|
||||
echo "${{ steps.meta.outputs.tags }}"
|
||||
echo '```'
|
||||
echo
|
||||
echo "Closes #520 (missing observatory/pose-fusion UI assets) and #514 (stale `:latest` for the v0.6+ packet format)."
|
||||
echo "The Dockerfile fails the build if those UI assets ever disappear again, and this workflow rebuilds + pushes automatically on every change to the surface."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -0,0 +1,72 @@
|
||||
name: three.js demos → GitHub Pages
|
||||
|
||||
# Publishes the ADR-097 three.js demos under gh-pages/three.js/.
|
||||
# Uses keep_files: true so the existing observatory/, pose-fusion/,
|
||||
# pointcloud/, nvsim/, and root index.html demos are preserved.
|
||||
#
|
||||
# Demos 04 and 05 require a Mixamo "X Bot.fbx" placed in assets/.
|
||||
# That file is intentionally gitignored (license boundary), so this
|
||||
# workflow does NOT ship it. Demos 01-03 work standalone; the index
|
||||
# page documents the FBX requirement honestly.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'examples/three.js/**'
|
||||
- '.github/workflows/threejs-pages.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
concurrency:
|
||||
group: threejs-pages
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Stage demos for Pages
|
||||
run: |
|
||||
mkdir -p _site/three.js
|
||||
# Copy everything except the local Python server (CI doesn't need it)
|
||||
# and any stray scratch screenshots.
|
||||
cp -r examples/three.js/demos _site/three.js/demos
|
||||
cp -r examples/three.js/screenshots _site/three.js/screenshots
|
||||
cp examples/three.js/README.md _site/three.js/README.md
|
||||
# An index.html that lists the 5 demos with the FBX caveat.
|
||||
cp examples/three.js/index.html _site/three.js/index.html
|
||||
# Mixamo FBX is gitignored — assets dir won't exist in CI.
|
||||
# Drop an empty placeholder so the relative path 'assets/' resolves
|
||||
# to a directory listing (404 on missing file) instead of an opaque
|
||||
# network error. Browsers showing the 404 path makes the failure
|
||||
# visible to anyone trying demos 04/05 without their own FBX.
|
||||
mkdir -p _site/three.js/assets
|
||||
cat > _site/three.js/assets/README.txt <<'EOF'
|
||||
The Mixamo "X Bot.fbx" required by demos 04-skinned-fbx.html and
|
||||
05-skinned-realtime.html is intentionally not redistributed here.
|
||||
|
||||
Download your own from https://mixamo.com (FBX Binary, T-Pose,
|
||||
Without Skin) and place it here as "X Bot.fbx" if you want to
|
||||
run those demos locally. See examples/three.js/README.md in the
|
||||
repo for context.
|
||||
EOF
|
||||
echo "Staged contents:"
|
||||
ls -R _site/three.js/ | head -30
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
uses: peaceiris/actions-gh-pages@v3
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_dir: _site
|
||||
# Critical: preserve observatory/, pose-fusion/, pointcloud/, nvsim/
|
||||
# and the root index.html already on gh-pages.
|
||||
keep_files: true
|
||||
commit_message: 'three.js demos: ${{ github.event.head_commit.message }}'
|
||||
@@ -0,0 +1,67 @@
|
||||
name: Update vendor submodules
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 */6 * * *' # Every 6 hours
|
||||
workflow_dispatch: # Manual trigger
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
update:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# Identity must be set BEFORE any operation that can create a commit.
|
||||
# `git submodule update --remote --merge` used to fail here with
|
||||
# "Committer identity unknown" because the merge inside vendor/ruvector
|
||||
# needs an author when the pinned commit isn't a fast-forward of upstream.
|
||||
- name: Configure git identity
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
# Use a plain `--remote` checkout (detached HEAD at each submodule's
|
||||
# configured `branch` tip from .gitmodules) rather than `--merge`. We only
|
||||
# want to bump the superproject's gitlink to the latest upstream commit;
|
||||
# there's no reason to create merge commits inside the vendored repos, and
|
||||
# `--merge` breaks whenever the current pin has diverged from that branch.
|
||||
- name: Update submodules to latest tracked branch
|
||||
run: |
|
||||
git submodule sync --recursive
|
||||
git submodule update --remote --recursive
|
||||
|
||||
- name: Check for changes
|
||||
id: check
|
||||
run: |
|
||||
if git diff --quiet; then
|
||||
echo "changed=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "changed=true" >> "$GITHUB_OUTPUT"
|
||||
echo "--- submodule pointer changes ---"
|
||||
git submodule status --recursive || true
|
||||
git diff --submodule=log -- vendor/ || true
|
||||
fi
|
||||
|
||||
- name: Create PR with updates
|
||||
if: steps.check.outputs.changed == 'true'
|
||||
run: |
|
||||
BRANCH="chore/update-submodules-$(date +%Y%m%d-%H%M%S)"
|
||||
git checkout -b "$BRANCH"
|
||||
git add vendor/
|
||||
git commit -m "chore: update vendor submodules to latest upstream"
|
||||
git push origin "$BRANCH"
|
||||
gh pr create \
|
||||
--title "chore: update vendor submodules" \
|
||||
--body "Automated submodule update to the latest upstream commit on each submodule's tracked branch (see \`.gitmodules\`). Review the pointer diff before merging." \
|
||||
--base main \
|
||||
--head "$BRANCH"
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -0,0 +1,126 @@
|
||||
name: Verify Pipeline Determinism
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, master, 'claude/**' ]
|
||||
paths:
|
||||
- 'archive/v1/src/core/**'
|
||||
- 'archive/v1/src/hardware/**'
|
||||
- 'archive/v1/data/proof/**'
|
||||
- 'archive/v1/requirements-lock.txt'
|
||||
- '.github/workflows/verify-pipeline.yml'
|
||||
pull_request:
|
||||
branches: [ main, master ]
|
||||
paths:
|
||||
- 'archive/v1/src/core/**'
|
||||
- 'archive/v1/src/hardware/**'
|
||||
- 'archive/v1/data/proof/**'
|
||||
- 'archive/v1/requirements-lock.txt'
|
||||
- '.github/workflows/verify-pipeline.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
verify-determinism:
|
||||
name: Verify Pipeline Determinism
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ['3.11']
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install pinned dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r archive/v1/requirements-lock.txt
|
||||
|
||||
- name: Verify reference signal is reproducible
|
||||
run: |
|
||||
echo "=== Regenerating reference signal ==="
|
||||
python archive/v1/data/proof/generate_reference_signal.py
|
||||
echo ""
|
||||
echo "=== Checking data file matches committed version ==="
|
||||
# The regenerated file should be identical to the committed one
|
||||
# (We compare the metadata file since data file is large)
|
||||
python -c "
|
||||
import json, hashlib
|
||||
with open('archive/v1/data/proof/sample_csi_meta.json') as f:
|
||||
meta = json.load(f)
|
||||
assert meta['is_synthetic'] == True, 'Metadata must mark signal as synthetic'
|
||||
assert meta['numpy_seed'] == 42, 'Seed must be 42'
|
||||
print('Reference signal metadata validated.')
|
||||
"
|
||||
|
||||
- name: Run pipeline verification
|
||||
working-directory: archive/v1
|
||||
env:
|
||||
# Pin thread count for scipy.fft / BLAS — multi-threaded reduction
|
||||
# order is otherwise non-deterministic across CI runs (issue #560
|
||||
# follow-up: 9- and 6-decimal quantization were not enough because
|
||||
# the divergence is from threading order, not SIMD reordering).
|
||||
# Single-threaded keeps the proof reproducible at a ~2-3x slowdown.
|
||||
OMP_NUM_THREADS: "1"
|
||||
OPENBLAS_NUM_THREADS: "1"
|
||||
MKL_NUM_THREADS: "1"
|
||||
VECLIB_MAXIMUM_THREADS: "1"
|
||||
NUMEXPR_NUM_THREADS: "1"
|
||||
run: |
|
||||
echo "=== Running pipeline verification ==="
|
||||
python data/proof/verify.py
|
||||
echo ""
|
||||
echo "Pipeline verification PASSED."
|
||||
|
||||
- name: Run verification twice to confirm determinism
|
||||
working-directory: archive/v1
|
||||
env:
|
||||
OMP_NUM_THREADS: "1"
|
||||
OPENBLAS_NUM_THREADS: "1"
|
||||
MKL_NUM_THREADS: "1"
|
||||
VECLIB_MAXIMUM_THREADS: "1"
|
||||
NUMEXPR_NUM_THREADS: "1"
|
||||
run: |
|
||||
echo "=== Second run for determinism confirmation ==="
|
||||
python data/proof/verify.py
|
||||
echo "Determinism confirmed across multiple runs."
|
||||
|
||||
- name: Check for unseeded np.random in production code
|
||||
run: |
|
||||
echo "=== Scanning for unseeded np.random usage in production code ==="
|
||||
# Search for np.random calls without a seed in production code
|
||||
# Exclude test files, proof data generators, and known parser placeholders
|
||||
VIOLATIONS=$(grep -rn "np\.random\." archive/v1/src/ \
|
||||
--include="*.py" \
|
||||
--exclude-dir="__pycache__" \
|
||||
| grep -v "np\.random\.RandomState" \
|
||||
| grep -v "np\.random\.seed" \
|
||||
| grep -v "np\.random\.default_rng" \
|
||||
| grep -v "# placeholder" \
|
||||
| grep -v "# mock" \
|
||||
| grep -v "# test" \
|
||||
|| true)
|
||||
|
||||
if [ -n "$VIOLATIONS" ]; then
|
||||
echo ""
|
||||
echo "WARNING: Found potential unseeded np.random usage in production code:"
|
||||
echo "$VIOLATIONS"
|
||||
echo ""
|
||||
echo "Each np.random call should either:"
|
||||
echo " 1. Use np.random.RandomState(seed) or np.random.default_rng(seed)"
|
||||
echo " 2. Be in a test/mock context (add '# placeholder' comment)"
|
||||
echo ""
|
||||
# Note: This is a warning, not a failure, because some existing
|
||||
# placeholder code in parsers uses np.random for mock data.
|
||||
# Once hardware integration is complete, these should be removed.
|
||||
echo "WARNING: Review the above usages. Existing parser placeholders are expected."
|
||||
else
|
||||
echo "No unseeded np.random usage found in production code."
|
||||
fi
|
||||
Reference in New Issue
Block a user