wehub-resource-sync
426e9eeabd
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
ci / test (push) Has been cancelled
ci / lint-and-format (push) Has been cancelled
ci / build (push) Has been cancelled
ci / dev-startup (push) Has been cancelled
gitleaks / gitleaks (push) Has been cancelled
Markdown Links / Relative Markdown Links (push) Has been cancelled
Quality (Extended) / Homepage Build (PR smoke) (push) Has been cancelled
Quality (Extended) / Comment-only diff guard (push) Has been cancelled
Quality (Extended) / Format + Type Safety Ratchet (push) Has been cancelled
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Has been cancelled
Quality (Extended) / Develop Gate (lint) (push) Has been cancelled
Chat shell gestures / Chat shell gesture + parity e2e (push) Has been cancelled
Cloud Gateway Discord / Test (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Has been cancelled
Build Agent Image / build-and-push (push) Has been cancelled
Dev Smoke / bun run dev onboarding chat (push) Has been cancelled
Dev Smoke / Vite HMR dependency-level smoke (push) Has been cancelled
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Has been cancelled
Publish @elizaos/example-code / check_npm (push) Has been cancelled
Publish @elizaos/example-code / publish_npm (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / verify_version (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / publish_npm (push) Has been cancelled
Sandbox Live Smoke / Sandbox live smoke (push) Has been cancelled
Snap Build & Test / Build Snap (amd64) (push) Has been cancelled
Snap Build & Test / Build Snap (arm64) (push) Has been cancelled
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Has been cancelled
Cloud Gateway Webhook / Test (push) Has been cancelled
Cloud Tests / lint-and-types (push) Has been cancelled
Cloud Tests / unit-tests (push) Has been cancelled
Cloud Tests / integration-tests (push) Has been cancelled
Cloud Tests / e2e-tests (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Deploy Apps Worker (Product 2) / Determine environment (push) Has been cancelled
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Has been cancelled
Deploy Eliza Provisioning Worker / Determine environment (push) Has been cancelled
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Has been cancelled
Dev Smoke / Classify changed paths (push) Has been cancelled
supply-chain / sbom (push) Has been cancelled
supply-chain / vulnerability-scan (push) Has been cancelled
Build, Push & Deploy to Phala Cloud / build-and-push (push) Has been cancelled
Test Packaging / Validate Packaging Configs (push) Has been cancelled
Test Packaging / Build & Test PyPI Package (push) Has been cancelled
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Has been cancelled
Test Packaging / Pack & Test JS Tarballs (push) Has been cancelled
UI Fixture E2E / ui-fixture-e2e (push) Has been cancelled
UI Fixture E2E / fixture-e2e (push) Has been cancelled
UI Story Gate / story-gate (push) Has been cancelled
vault-ci / test (macos-latest) (push) Has been cancelled
vault-ci / test (ubuntu-latest) (push) Has been cancelled
vault-ci / test (windows-latest) (push) Has been cancelled
vault-ci / app-core wiring tests (push) Has been cancelled
verify-patches / verify patches/CHECKSUMS.sha256 (push) Has been cancelled
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Has been cancelled
Voice Benchmark Smoke / voice bench smoke summary (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Has been cancelled
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Has been cancelled
1410 lines
65 KiB
YAML
1410 lines
65 KiB
YAML
name: Tests
|
|
|
|
# Comprehensive test orchestrator. Runs the full eliza test suite plus
|
|
# focused server/client/plugin slices and the database security guard.
|
|
# This is a peer of `ci.yaml` — `ci.yaml` is the canonical lint+typecheck+
|
|
# build+core-test gate; this workflow adds the broader integration surface.
|
|
|
|
on:
|
|
# Post-merge integration surface. PRs to `develop` run the lightweight
|
|
# lint/typecheck/build gate (develop-pr.yml); the full suite runs here once
|
|
# changes land on `develop`, plus nightly and on demand.
|
|
push:
|
|
branches: [develop]
|
|
workflow_dispatch:
|
|
schedule:
|
|
# Nightly live smoke for container-backed remote capability plugins.
|
|
- cron: "17 9 * * *"
|
|
|
|
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
|
|
# github.run_id on push, so every develop push got a unique group and
|
|
# nothing ever superseded. A 37-merge wave queued thousands of push runs
|
|
# and starved the self-hosted fleet. Fall back to github.ref (all develop
|
|
# pushes share a group) and cancel superseded runs on push too. PR behavior
|
|
# is unchanged (pull_request.number still wins the key); merge_group and
|
|
# schedule events are not in the cancel set, so the merge queue and nightly
|
|
# runs always complete.
|
|
concurrency:
|
|
group: test-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
|
|
|
|
env:
|
|
CI: "true"
|
|
# pinned: floating canary hangs `bun install` on hosted CI runners and writes
|
|
# lockfileVersion 2 which breaks --frozen-lockfile (#11184/#9454). This env
|
|
# drives the integration/unit/lint/cloud-live-e2e jobs' bun-version input.
|
|
BUN_VERSION: "1.3.14"
|
|
NODE_VERSION: "24.15.0"
|
|
NODE_NO_WARNINGS: "1"
|
|
NODE_OPTIONS: "--max-old-space-size=8192"
|
|
# Baseline PR jobs are secret-free. Dedicated live jobs inject provider keys
|
|
# locally and remain opt-in/scheduled where external services are involved.
|
|
|
|
# Default to least privilege. Override per-job where needed.
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
stale-base-preflight:
|
|
name: PR stale-base preflight
|
|
# PR-only in EFFECT — every step gates on github.event_name ==
|
|
# 'pull_request' — but the JOB itself runs (and no-op-succeeds) on every
|
|
# event. It used to be job-level `if: github.event_name == 'pull_request'`,
|
|
# which SKIPPED it on push; because `changes` needs this job and GitHub
|
|
# evaluates the implicit success() over the TRANSITIVE needs chain
|
|
# (actions/runner#491), that one skipped ancestor silently skip-cascaded
|
|
# every push-event lane downstream of `changes` whose `if` carries no
|
|
# status-check function — develop push runs went vacuously red from the
|
|
# moment the needs edge landed. Step-level gating keeps the PR-only
|
|
# contract (ci-merge-gate-contract.mjs) and the `changes` ordering while
|
|
# letting push/schedule/dispatch fan-outs actually run.
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 10
|
|
env:
|
|
BASE_REF: ${{ github.event.pull_request.base.ref }}
|
|
HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }}
|
|
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
ACK: ${{ contains(github.event.pull_request.labels.*.name, 'stale-base-ack') && '1' || '' }}
|
|
steps:
|
|
- name: Checkout target branch (guard logic comes from the base, not the PR)
|
|
if: github.event_name == 'pull_request'
|
|
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
ref: ${{ github.event.pull_request.base.ref }}
|
|
# The guard disables lazy promisor fetches while it runs. Fetch the
|
|
# same bounded history window up front so merge commits can resolve
|
|
# their parents without network fallback.
|
|
fetch-depth: 1500
|
|
filter: blob:none
|
|
submodules: false
|
|
|
|
- name: Setup Node.js for stale-base guard
|
|
if: github.event_name == 'pull_request'
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Fetch PR head + bounded history (blobless)
|
|
if: github.event_name == 'pull_request'
|
|
run: |
|
|
set -euo pipefail
|
|
git fetch --quiet --no-tags --filter=blob:none --depth=1500 \
|
|
"https://github.com/${HEAD_REPO}.git" "$HEAD_SHA"
|
|
git fetch --quiet --no-tags --filter=blob:none --depth=1500 origin \
|
|
"+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}"
|
|
|
|
- name: Self-test the guard on fixture repos
|
|
if: github.event_name == 'pull_request'
|
|
run: node packages/scripts/stale-base-guard.self-test.mjs
|
|
|
|
- name: Detect silent reverts + stale base before fanning out tests
|
|
if: github.event_name == 'pull_request'
|
|
env:
|
|
GIT_NO_LAZY_FETCH: "1"
|
|
run: |
|
|
set -euo pipefail
|
|
node packages/scripts/stale-base-guard.mjs \
|
|
--base "refs/remotes/origin/${BASE_REF}" \
|
|
--head "$HEAD_SHA" \
|
|
--window 1200 \
|
|
--max-behind-commits 200 \
|
|
--max-behind-hours 72 \
|
|
${ACK:+--ack} \
|
|
--github \
|
|
--summary "$GITHUB_STEP_SUMMARY"
|
|
|
|
changes:
|
|
name: Classify changed paths
|
|
needs: stale-base-preflight
|
|
if: always() && (github.event_name != 'pull_request' || needs.stale-base-preflight.result == 'success')
|
|
# Route with the same fleet-aware conditional as the test lanes. GitHub-hosted
|
|
# runners are billing-frozen (#13481), so a hardcoded ubuntu-24.04 here queued
|
|
# forever and — since every lane needs this classifier — stalled the entire
|
|
# ci-ok chain while the self-hosted fleet sat idle. Self-hosted when online;
|
|
# hosted only when an operator flags the fleet down (and billing is restored).
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 10
|
|
outputs:
|
|
server: ${{ steps.filter.outputs.server }}
|
|
client: ${{ steps.filter.outputs.client }}
|
|
plugins: ${{ steps.filter.outputs.plugins }}
|
|
desktop: ${{ steps.filter.outputs.desktop }}
|
|
zero_key: ${{ steps.filter.outputs.zero_key }}
|
|
cloud: ${{ steps.filter.outputs.cloud }}
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
fetch-depth: 0
|
|
submodules: false
|
|
|
|
- name: Setup Node.js for path gate
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Validate CI path gate
|
|
run: node packages/scripts/ci-path-gate.self-test.mjs
|
|
|
|
- name: Validate CI workflow dedup contract
|
|
run: node packages/scripts/ci-workflow-dedup-contract.mjs
|
|
|
|
- name: Validate GitHub-native turbo cache contract
|
|
run: node packages/scripts/ci-turbo-cache-contract.mjs
|
|
|
|
- name: Validate CI Bun version pin contract
|
|
run: node packages/scripts/ci-bun-version-contract.mjs
|
|
|
|
- name: Validate zero-key command ownership contract
|
|
run: node packages/scripts/ci-zero-key-command-ownership-contract.mjs
|
|
|
|
- name: Validate Windows command coverage contract
|
|
run: node packages/scripts/ci-windows-command-coverage-contract.mjs
|
|
|
|
- name: Install alias-read guard parser dependency
|
|
run: npm install --prefix "$RUNNER_TEMP/alias-read-guard-deps" --no-save --no-package-lock --ignore-scripts typescript@6.0.3
|
|
|
|
- name: Validate brand-env alias-read guard self-test
|
|
env:
|
|
NODE_PATH: ${{ runner.temp }}/alias-read-guard-deps/node_modules
|
|
run: node packages/scripts/alias-read-guard.mjs --self-test
|
|
|
|
- name: Validate merge-gate robustness contract self-test
|
|
run: node packages/scripts/ci-merge-gate-contract.mjs --self-test
|
|
|
|
- name: Validate merge-gate robustness contract
|
|
run: node packages/scripts/ci-merge-gate-contract.mjs
|
|
|
|
- name: Determine affected test lanes
|
|
id: filter
|
|
shell: bash
|
|
env:
|
|
PR_LABELS: ${{ github.event_name == 'pull_request' && join(github.event.pull_request.labels.*.name, ',') || '' }}
|
|
BASE_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || '' }}
|
|
HEAD_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || '' }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "${GITHUB_EVENT_NAME}" = "merge_group" ]; then
|
|
{
|
|
echo "server=true"
|
|
echo "client=true"
|
|
echo "plugins=true"
|
|
echo "desktop=true"
|
|
echo "zero_key=true"
|
|
echo "cloud=true"
|
|
} >> "$GITHUB_OUTPUT"
|
|
{
|
|
echo "### Tests path gate"
|
|
echo
|
|
echo "Merge queue runs validate the synthesized queue SHA, so path"
|
|
echo "diffs are deliberately not used. Running the full required"
|
|
echo "test suite for the \`ci-ok\` aggregate check."
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
exit 0
|
|
fi
|
|
node packages/scripts/ci-path-gate.mjs \
|
|
--config test \
|
|
--event "${{ github.event_name }}" \
|
|
--base "$BASE_SHA" \
|
|
--head "$HEAD_SHA" \
|
|
--labels "$PR_LABELS" \
|
|
--output "$GITHUB_OUTPUT" \
|
|
--summary "$GITHUB_STEP_SUMMARY"
|
|
|
|
test-runner-vacuous-green-guard:
|
|
name: Test-runner vacuous-green guard
|
|
needs: changes
|
|
# #13620/#12342: this guard protects the ordinary PR path from runner lanes
|
|
# that collect no real tests or swallow a failure as "no tests found".
|
|
# Keep it outside path gates and outside the opt-in `ci:full` heavy family.
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 15
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Bun
|
|
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
|
|
- name: Test-runner vacuous-green guard
|
|
run: bun test packages/scripts/__tests__/run-all-tests-vacuous-green-guard.test.ts
|
|
|
|
server-tests:
|
|
name: Server Tests
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.server == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 60
|
|
env:
|
|
PGLITE_WASM_MODE: node
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Build view bundles
|
|
run: bun run build:views
|
|
|
|
- name: Views system tests
|
|
run: bunx vitest run packages/agent/src/__tests__/views-system-smoke.test.ts packages/agent/src/__tests__/views-registry-integration.test.ts packages/agent/src/__tests__/plugin-tui-view-coverage.test.ts
|
|
|
|
- name: Remote capability router tests
|
|
run: bun run test:remote-capabilities
|
|
|
|
- name: Remote capability plugin surface audit
|
|
run: bun run test:remote-capabilities:surface-audit
|
|
|
|
- name: Remote capability live CI audit
|
|
run: bun run test:remote-capabilities:live-ci-audit
|
|
|
|
- name: Remote capability live CI audit self-test
|
|
run: bun run test:remote-capabilities:live-ci-audit:self-test
|
|
|
|
- name: Remote capability naming audit
|
|
run: bun run test:remote-capabilities:naming-audit
|
|
|
|
- name: Remote capability naming audit self-test
|
|
run: bun run test:remote-capabilities:naming-audit:self-test
|
|
|
|
- name: Remote capability source-build smoke
|
|
run: bun run test:remote-capabilities:source-build
|
|
|
|
- name: Remote capability fixture server smoke
|
|
run: bun run test:remote-capabilities:fixture-server
|
|
|
|
- name: Remote capability live report validator self-test
|
|
run: bun run test:remote-capabilities:validate-live-reports:self-test
|
|
|
|
- name: Remote capability GitHub live evidence self-test
|
|
run: bun run test:remote-capabilities:github-live-evidence:self-test
|
|
|
|
- name: Remote capability GitHub live artifact self-test
|
|
run: bun run test:remote-capabilities:github-live-artifacts:self-test
|
|
|
|
- name: Remote capability Docker smoke
|
|
run: bun run test:remote-capabilities:docker
|
|
|
|
- name: Run server tests
|
|
run: bun run test:server
|
|
|
|
client-tests:
|
|
name: Client Tests
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.client == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 60
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Run client tests
|
|
run: bun run test:client
|
|
|
|
- name: Install Playwright Chromium
|
|
run: .github/scripts/install-playwright-browsers.sh chromium
|
|
|
|
- name: Remote capability UI smoke
|
|
run: bun run test:remote-capabilities:ui
|
|
|
|
# Perf gate (#9954): drive REAL overlay thread-scroll + pull-to-maximize /
|
|
# top-pull-restore in headless chromium, feed REAL rAF/longtask/layout-shift
|
|
# entries through the shared frame-budget + layout-stability detectors,
|
|
# HARD-FAIL on dropped frames, p95 frame time, or CLS. The gate first proves
|
|
# its CLS detector has teeth by injecting a real non-transient shift and
|
|
# asserting it is flagged (#14333), so a true CLS of 0.0000 cannot be a
|
|
# vacuous pass from a dead observer or an over-broad transient marker.
|
|
- name: Perf gate e2e (#9954)
|
|
run: bun run --cwd packages/ui test:perf-gate-e2e
|
|
|
|
- name: Upload perf gate e2e artifacts
|
|
if: always()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
|
|
with:
|
|
name: perf-gate-e2e-artifacts
|
|
path: packages/ui/src/components/shell/__e2e__/output-perf-gate-e2e/
|
|
if-no-files-found: ignore
|
|
|
|
plugin-tests:
|
|
name: Plugin Tests (${{ matrix.shard }}/4)
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.plugins == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 95
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard: [1, 2, 3, 4]
|
|
env:
|
|
TEST_SHARD: ${{ matrix.shard }}/4
|
|
# Share compiled rust artifacts across all plugin crates so the
|
|
# `elizaos = "2.0.0"` crate (and its sqlx/hyper/etc transitive deps)
|
|
# only compiles once, not once per plugin. Without this, each
|
|
# plugin-{discord,groq,openai,sql,whatsapp} rebuilds the same heavy
|
|
# dep tree from scratch, blowing past the step timeout.
|
|
CARGO_TARGET_DIR: ${{ github.workspace }}/.cargo-shared-target
|
|
CARGO_INCREMENTAL: "0"
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
fetch-depth: 1
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Cache cargo registry and shared target dir
|
|
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
${{ github.workspace }}/.cargo-shared-target
|
|
key: ${{ runner.os }}-cargo-plugin-tests-${{ hashFiles('plugins/*/rust/Cargo.lock', 'plugins/*/rust/Cargo.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-cargo-plugin-tests-
|
|
|
|
# The view-bundle coverage tests assert every view plugin has a built
|
|
# dist/views/bundle.js; build them before running plugin tests.
|
|
- name: Build view bundles
|
|
run: bun run build:views
|
|
|
|
- name: Run plugin tests
|
|
# Cap step below the 95m job budget so a flaky plugin
|
|
# (most recently plugin-shell rust PTY tests in run 25234887145)
|
|
# fails fast instead of burning the whole job's wall clock.
|
|
timeout-minutes: 85
|
|
run: bun run test:plugins
|
|
|
|
- name: Prune cargo cache payload
|
|
if: always()
|
|
run: |
|
|
node packages/scripts/rm-path-recursive.mjs \
|
|
"${CARGO_TARGET_DIR}/debug/incremental" \
|
|
"${CARGO_TARGET_DIR}/debug/.fingerprint" \
|
|
"${CARGO_TARGET_DIR}/debug/build" \
|
|
"${CARGO_TARGET_DIR}/debug/examples" \
|
|
"${CARGO_TARGET_DIR}/release/incremental" \
|
|
"${CARGO_TARGET_DIR}/release/.fingerprint" \
|
|
"${CARGO_TARGET_DIR}/release/build" \
|
|
"${CARGO_TARGET_DIR}/release/examples"
|
|
find "${CARGO_TARGET_DIR}" -type f \( -name '*.d' -o -name '*.rlib' -o -name '*.rmeta' \) -delete 2>/dev/null || true
|
|
|
|
plugin-tests-status:
|
|
name: Plugin Tests
|
|
needs:
|
|
- changes
|
|
- plugin-tests
|
|
if: ${{ !cancelled() && always() }}
|
|
# Fleet-aware (see `changes`): hardcoded hosted froze this ci-ok dependency.
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 10
|
|
steps:
|
|
- name: Check plugin shard results
|
|
shell: bash
|
|
run: |
|
|
set -u
|
|
result="${{ needs.plugin-tests.result }}"
|
|
echo "plugin-tests matrix: ${result}"
|
|
if [ "$result" = "success" ]; then
|
|
exit 0
|
|
fi
|
|
if [ "$result" = "skipped" ] && [ "${{ github.event_name }}" = "pull_request" ] && [ "${{ needs.changes.outputs.plugins }}" != "true" ]; then
|
|
echo "::notice title=Plugin Tests::Path gate skipped plugin tests for this PR context."
|
|
exit 0
|
|
fi
|
|
echo "::error title=Plugin Tests::Plugin shard matrix finished with '${result}'"
|
|
exit 1
|
|
|
|
# First CI consumer of the repo-level integration lane
|
|
# (packages/test/vitest/integration.config.ts). The lane was dead in CI:
|
|
# nothing invoked the config, its @elizaos/core alias broke the /node
|
|
# subpath, and its eliza/-prefixed paths matched zero files in a flat
|
|
# checkout (#11047). Runs plugin-personal-assistant's test:integration —
|
|
# the src-integration lane (scheduler/recurrence/global-pause real-DB
|
|
# suites) plus the repo config over the package's named test/ files —
|
|
# keyless. Burn down the remaining plugins/*/test integration suites from
|
|
# here.
|
|
integration-tests:
|
|
name: Integration Lane (personal-assistant)
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.plugins == 'true' || needs.changes.outputs.server == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 45
|
|
env:
|
|
PGLITE_WASM_MODE: node
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
# The personal-assistant barrel imports sibling plugin dists (e.g.
|
|
# plugin-calendly pulls "@elizaos/core/node") straight from
|
|
# node_modules, so the lane needs the package's workspace dependency
|
|
# graph built. Turbo replays warm runs from cache.
|
|
- name: Build personal-assistant dependency graph
|
|
run: node packages/scripts/run-turbo.mjs run build --filter='@elizaos/plugin-personal-assistant...'
|
|
|
|
- name: Run personal-assistant integration lanes
|
|
timeout-minutes: 35
|
|
run: bun run --cwd plugins/plugin-personal-assistant test:integration
|
|
|
|
desktop-contract:
|
|
name: Electrobun Desktop Contract
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.desktop == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
|
|
- name: Build workspace packages for Vitest resolution
|
|
run: bun run build:core
|
|
|
|
- name: Run Electrobun contract tests
|
|
working-directory: packages/app-core/platforms/electrobun
|
|
run: bun run test
|
|
|
|
zero-key-unit-coverage:
|
|
name: Zero-Key unit + UI coverage
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.zero_key == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 30
|
|
env:
|
|
TEST_LANE: "pr"
|
|
ELIZA_LIVE_TEST: "0"
|
|
SCENARIO_USE_LLM_PROXY: "1"
|
|
CEREBRAS_API_KEY: ""
|
|
OPENAI_API_KEY: ""
|
|
ANTHROPIC_API_KEY: ""
|
|
GROQ_API_KEY: ""
|
|
OPENROUTER_API_KEY: ""
|
|
GOOGLE_GENERATIVE_AI_API_KEY: ""
|
|
XAI_API_KEY: ""
|
|
NVIDIA_API_KEY: ""
|
|
ELIZAOS_CLOUD_API_KEY: ""
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install --ignore-scripts --no-frozen-lockfile
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
# The scenario CLI imports @elizaos/core from source, which imports the
|
|
# gitignored ./generated/validation-keyword-data.ts. --ignore-scripts skips
|
|
# postinstall, so generate it explicitly before any step resolves core.
|
|
- name: Ensure generated shared i18n data
|
|
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
|
|
|
|
- name: Scenario runner unit coverage
|
|
run: bun run --cwd packages/scenario-runner test -- src/executor.test.ts src/runtime-factory.test.ts src/scenario-pr-workflow.test.ts
|
|
|
|
- name: Deterministic LLM proxy unit coverage
|
|
working-directory: packages
|
|
run: bunx vitest run --config test/mocks/vitest.config.ts test/mocks/__tests__/llm-proxy-plugin.test.ts
|
|
|
|
- name: UI voice interaction coverage
|
|
run: bun run --cwd packages/ui test -- src/hooks/useVoiceChat.bidirectional.test.tsx src/hooks/useContinuousChat.test.tsx
|
|
|
|
- name: UI dynamic view interaction coverage
|
|
run: bun run --cwd packages/ui test -- src/App.navigate-view-wiring.test.tsx src/app-navigate-view.test.ts src/components/shell/__tests__/shell-assistant-flow.test.tsx src/components/views/DynamicViewLoader.test.tsx src/hooks/useAvailableViews.test.tsx src/hooks/useDesktopTabs.test.tsx src/state/startup-phase-hydrate.view-interact.test.ts
|
|
|
|
- name: App route + ui-smoke spec coverage gates
|
|
run: bun run --cwd packages/app test -- test/route-coverage.test.ts test/ui-smoke-coverage.test.ts test/view-interaction-coverage.test.ts
|
|
|
|
zero-key-browser:
|
|
name: Zero-Key app browser
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.zero_key == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 35
|
|
env:
|
|
TEST_LANE: "pr"
|
|
ELIZA_LIVE_TEST: "0"
|
|
SCENARIO_USE_LLM_PROXY: "1"
|
|
CEREBRAS_API_KEY: ""
|
|
OPENAI_API_KEY: ""
|
|
ANTHROPIC_API_KEY: ""
|
|
GROQ_API_KEY: ""
|
|
OPENROUTER_API_KEY: ""
|
|
GOOGLE_GENERATIVE_AI_API_KEY: ""
|
|
XAI_API_KEY: ""
|
|
NVIDIA_API_KEY: ""
|
|
ELIZAOS_CLOUD_API_KEY: ""
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install --ignore-scripts --no-frozen-lockfile
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Ensure generated shared i18n data
|
|
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
|
|
|
|
- name: Install Playwright Chromium + WebKit
|
|
run: .github/scripts/install-playwright-browsers.sh chromium webkit
|
|
|
|
- name: Actual app assistant pill browser coverage
|
|
run: bun run --cwd packages/app test:e2e test/ui-smoke/assistant-home-flow.spec.ts --project=chromium -g "captures first-run, assistant home, chat suppression, and view pill states"
|
|
|
|
- name: Actual app assistant voice browser coverage
|
|
run: bun run --cwd packages/app test:e2e test/ui-smoke/assistant-home-flow.spec.ts --project=chromium -g "drives the assistant home voice path with a scripted browser STT turn"
|
|
|
|
- name: Actual app TTS and STT browser coverage
|
|
run: bun run --cwd packages/app test:e2e test/ui-smoke/tts-stt-e2e.spec.ts --project=chromium
|
|
|
|
- name: Actual app plugin view browser coverage
|
|
run: bun run --cwd packages/app test:e2e test/ui-smoke/plugin-views-visual.spec.ts test/ui-smoke/plugin-views-interaction.spec.ts test/ui-smoke/orchestrator-gui-workbench.spec.ts test/ui-smoke/screenshare-gui-interactions.spec.ts test/ui-smoke/task-coordinator-gui-interactions.spec.ts --project=chromium
|
|
|
|
# WebKit cross-engine pointer/focus lane (#10722): the playwright config's
|
|
# opt-in `webkit` project (Desktop Safari) over the keyless chat
|
|
# pointer/focus/composer specs. Without this step the lane never runs
|
|
# anywhere — Chromium-only coverage ships WebKit-specific regressions to
|
|
# iOS/iPadOS Safari unseen.
|
|
- name: Actual app WebKit pointer/focus coverage
|
|
run: PLAYWRIGHT_WEBKIT=1 bun run --cwd packages/app test:e2e --project=webkit
|
|
|
|
- name: Upload zero-key browser artifacts
|
|
if: always()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
|
|
with:
|
|
name: tests-zero-key-browser
|
|
path: |
|
|
packages/app/playwright-report/
|
|
packages/app/test-results/
|
|
if-no-files-found: ignore
|
|
retention-days: 3
|
|
|
|
zero-key-diagnostics:
|
|
name: Zero-Key diagnostics
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.zero_key == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 30
|
|
env:
|
|
TEST_LANE: "pr"
|
|
ELIZA_LIVE_TEST: "0"
|
|
SCENARIO_USE_LLM_PROXY: "1"
|
|
CEREBRAS_API_KEY: ""
|
|
OPENAI_API_KEY: ""
|
|
ANTHROPIC_API_KEY: ""
|
|
GROQ_API_KEY: ""
|
|
OPENROUTER_API_KEY: ""
|
|
GOOGLE_GENERATIVE_AI_API_KEY: ""
|
|
XAI_API_KEY: ""
|
|
NVIDIA_API_KEY: ""
|
|
ELIZAOS_CLOUD_API_KEY: ""
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install --ignore-scripts --no-frozen-lockfile
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Ensure generated shared i18n data
|
|
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
|
|
|
|
- name: Electrobun window and dynamic-view coverage
|
|
run: bun run --cwd packages/app-core/platforms/electrobun test src/native/desktop-window.test.ts src/rpc-handlers.test.ts src/dynamic-view-rpc-schema.test.ts src/surface-windows.test.ts src/dynamic-views/host.test.ts src/application-menu.test.ts
|
|
|
|
- name: View management action coverage
|
|
run: bun run --cwd plugins/plugin-app-control test -- src/actions/views-management.test.ts
|
|
|
|
- name: App verification diagnostics coverage
|
|
run: bun run --cwd plugins/plugin-app-control test -- src/services/__tests__/app-verification.integration.test.ts src/services/__tests__/verification-room-bridge.test.ts
|
|
|
|
- name: Computer-use screenshot quality diagnostics coverage
|
|
run: bun run --cwd plugins/plugin-computeruse test -- test/helpers/screenshot-quality.test.ts src/__tests__/browser-auto-open.test.ts
|
|
|
|
- name: App screenshot quality diagnostics coverage
|
|
run: bun run --cwd packages/app test -- test/screenshot-quality.test.ts
|
|
|
|
- name: App-core screenshot quality diagnostics coverage
|
|
run: bun run --cwd packages/app-core test -- test/app/screenshot-quality.test.ts
|
|
|
|
zero-key-scenario-runner:
|
|
name: Zero-Key scenario runner
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.zero_key == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 30
|
|
env:
|
|
TEST_LANE: "pr"
|
|
ELIZA_LIVE_TEST: "0"
|
|
SCENARIO_USE_LLM_PROXY: "1"
|
|
CEREBRAS_API_KEY: ""
|
|
OPENAI_API_KEY: ""
|
|
ANTHROPIC_API_KEY: ""
|
|
GROQ_API_KEY: ""
|
|
OPENROUTER_API_KEY: ""
|
|
GOOGLE_GENERATIVE_AI_API_KEY: ""
|
|
XAI_API_KEY: ""
|
|
NVIDIA_API_KEY: ""
|
|
ELIZAOS_CLOUD_API_KEY: ""
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install --ignore-scripts --no-frozen-lockfile
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
# The scenario CLI imports @elizaos/core from source, which imports the
|
|
# gitignored ./generated/validation-keyword-data.ts. --ignore-scripts skips
|
|
# postinstall, so generate it explicitly before any step resolves core.
|
|
- name: Ensure generated shared i18n data
|
|
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
|
|
|
|
- name: Scenario catalog coverage gate
|
|
run: node packages/scripts/check-scenario-workflow-coverage.mjs --report-dir reports/scenarios/catalog-inventory
|
|
|
|
- name: Persona catalog coverage gate
|
|
run: node packages/scripts/check-lifeops-persona-catalog-coverage.mjs
|
|
|
|
- name: Build workspace packages for scenario resolution
|
|
run: bunx turbo run build --filter=@elizaos/scenario-runner...
|
|
|
|
- name: Zero-cost scenario e2e
|
|
run: bun run --cwd packages/scenario-runner test:pr:e2e
|
|
|
|
- name: Upload deterministic E2E reports
|
|
if: always()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
|
|
with:
|
|
name: tests-zero-key-e2e-report
|
|
path: |
|
|
reports/scenarios/pr-deterministic/
|
|
reports/scenarios/pr-deterministic-corpus/
|
|
reports/scenarios/pr-deterministic-orchestrator/
|
|
reports/scenarios/pr-deterministic-lifeops/
|
|
reports/scenarios/catalog-inventory/
|
|
if-no-files-found: ignore
|
|
retention-days: 3
|
|
|
|
- name: Prune deterministic E2E scratch output
|
|
if: always()
|
|
run: node packages/scripts/rm-path-recursive.mjs reports/scenarios/pr-deterministic reports/scenarios/pr-deterministic-corpus reports/scenarios/pr-deterministic-orchestrator reports/scenarios/pr-deterministic-lifeops reports/scenarios/catalog-inventory packages/app/playwright-report packages/app/test-results
|
|
|
|
zero-key-harness-e2e:
|
|
name: Zero-Key harness E2E
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.zero_key == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 15
|
|
env:
|
|
TEST_LANE: "pr"
|
|
ELIZA_LIVE_TEST: "0"
|
|
SCENARIO_USE_LLM_PROXY: "1"
|
|
CEREBRAS_API_KEY: ""
|
|
OPENAI_API_KEY: ""
|
|
ANTHROPIC_API_KEY: ""
|
|
GROQ_API_KEY: ""
|
|
OPENROUTER_API_KEY: ""
|
|
GOOGLE_GENERATIVE_AI_API_KEY: ""
|
|
XAI_API_KEY: ""
|
|
NVIDIA_API_KEY: ""
|
|
ELIZAOS_CLOUD_API_KEY: ""
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install --ignore-scripts --no-frozen-lockfile
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Ensure generated shared i18n data
|
|
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
|
|
|
|
- name: Keyless harness e2e (mock-LLM + recorded fixtures)
|
|
working-directory: packages
|
|
run: bunx vitest run --config test/mocks/vitest.config.ts test/mocks/__tests__/
|
|
|
|
- name: Per-plugin keyless harness adoption proofs
|
|
run: |
|
|
bun run --cwd plugins/plugin-anthropic test:harness
|
|
bun run --cwd plugins/plugin-discord test:harness
|
|
|
|
zero-key-e2e:
|
|
name: Zero-Key Deterministic E2E
|
|
needs:
|
|
- changes
|
|
- zero-key-unit-coverage
|
|
- zero-key-browser
|
|
- zero-key-diagnostics
|
|
- zero-key-scenario-runner
|
|
- zero-key-harness-e2e
|
|
if: ${{ !cancelled() && always() && (github.event_name != 'pull_request' || needs.changes.outputs.zero_key == 'true') }}
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
steps:
|
|
- name: Check zero-key slices
|
|
shell: bash
|
|
run: |
|
|
set -u
|
|
bad=0
|
|
for pair in \
|
|
"zero-key-unit-coverage:${{ needs.zero-key-unit-coverage.result }}" \
|
|
"zero-key-browser:${{ needs.zero-key-browser.result }}" \
|
|
"zero-key-diagnostics:${{ needs.zero-key-diagnostics.result }}" \
|
|
"zero-key-scenario-runner:${{ needs.zero-key-scenario-runner.result }}" \
|
|
"zero-key-harness-e2e:${{ needs.zero-key-harness-e2e.result }}"; do
|
|
name="${pair%%:*}"
|
|
result="${pair##*:}"
|
|
echo "$name: $result"
|
|
if [ "$result" != "success" ]; then
|
|
echo "::error title=Tests zero-key::$name finished with $result"
|
|
bad=1
|
|
fi
|
|
done
|
|
if [ "$bad" -ne 0 ]; then
|
|
exit 1
|
|
fi
|
|
echo "All zero-key slices passed."
|
|
|
|
cloud-live-e2e:
|
|
name: Cloud Live E2E (Eliza Cloud)
|
|
needs: changes
|
|
if: github.event_name != 'pull_request' || needs.changes.outputs.cloud == 'true'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 60
|
|
outputs:
|
|
skip: ${{ steps.cloud.outputs.skip }}
|
|
capability_skip: ${{ steps.cloud.outputs.capability_skip }}
|
|
steps:
|
|
- name: Check cloud secrets
|
|
id: cloud
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
ELIZAOS_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY != '' && secrets.ELIZAOS_CLOUD_API_KEY || secrets.ELIZACLOUD_API_KEY }}
|
|
ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED: ${{ vars.ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED != '' && vars.ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED || secrets.ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED }}
|
|
run: |
|
|
strict_context=false
|
|
case "${EVENT_NAME}" in
|
|
workflow_dispatch|schedule) strict_context=true ;;
|
|
esac
|
|
|
|
if [ -z "${ELIZAOS_CLOUD_API_KEY}" ]; then
|
|
message="No Eliza Cloud API key configured"
|
|
if [ "${strict_context}" = "true" ]; then
|
|
echo "::error::${message}; strict ${EVENT_NAME} live lanes must fail instead of greening by skip."
|
|
exit 1
|
|
fi
|
|
echo "::warning::${message} - skipping optional cloud live E2E for ${EVENT_NAME}."
|
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
|
echo "capability_skip=true" >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
else
|
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
if [ "${ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED}" = "1" ]; then
|
|
echo "capability_skip=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
if [ "${EVENT_NAME}" = "workflow_dispatch" ] || [ "${EVENT_NAME}" = "schedule" ]; then
|
|
echo "::error::Remote capability cloud sandbox live smoke is required for ${EVENT_NAME}; set ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED=1 instead of producing a green skip."
|
|
exit 1
|
|
fi
|
|
echo "::notice::Remote capability cloud sandbox live smoke skipped because ELIZA_REMOTE_CAPABILITY_CLOUD_LIVE_ENABLED is not configured."
|
|
echo "capability_skip=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Checkout
|
|
if: steps.cloud.outputs.skip != 'true'
|
|
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
if: steps.cloud.outputs.skip != 'true'
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
if: steps.cloud.outputs.skip != 'true'
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Run Cloud tests
|
|
if: steps.cloud.outputs.skip != 'true'
|
|
shell: bash
|
|
run: |
|
|
set -o pipefail
|
|
log_file="$(mktemp)"
|
|
|
|
if bun run --cwd packages/cloud/sdk test:e2e 2>&1 | tee "$log_file"; then
|
|
exit 0
|
|
fi
|
|
|
|
# Soft-fail ONLY on genuine provider-side unavailability (HTTP 429/5xx, quota
|
|
# exhaustion, or network errors) that a contributor can't control. Client-side
|
|
# failures must redden the job so real regressions aren't masked: 4xx auth/permission
|
|
# (e.g. 403 access_denied / "missing required permission"), assertion failures and
|
|
# timeouts all fall through to exit 1. Patterns are anchored to HTTP status codes and
|
|
# error tokens so they can't collide with a benign test title (the old allowlist
|
|
# matched "credit balance" from the test name "gets credit balance and summary",
|
|
# swallowing every failure on every run).
|
|
if grep -Eiq '(status code|statusCode|HTTP)[: ]*429|too many requests|rate.?limit|(status code|statusCode|HTTP)[: ]*5[0-9][0-9]|service unavailable|internal server error|temporarily unavailable|overloaded|exceeded your current quota|quota exceeded|insufficient[_ -]?(quota|balance|funds|credits?)|please add credits|top[_ -]?up your credits|ECONNREFUSED|ENOTFOUND|ETIMEDOUT|ECONNRESET|socket hang up' "$log_file"; then
|
|
echo "::warning::Cloud Live E2E soft-failed: external provider unavailable (HTTP 429/5xx, quota, or network error). Genuine test failures are NOT suppressed."
|
|
exit 0
|
|
fi
|
|
|
|
exit 1
|
|
env:
|
|
ELIZA_LIVE_TEST: "1"
|
|
ELIZAOS_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY != '' && secrets.ELIZAOS_CLOUD_API_KEY || secrets.ELIZACLOUD_API_KEY }}
|
|
ELIZA_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY != '' && secrets.ELIZAOS_CLOUD_API_KEY || secrets.ELIZACLOUD_API_KEY }}
|
|
ELIZAOS_CLOUD_BASE_URL: ${{ secrets.ELIZAOS_CLOUD_BASE_URL }}
|
|
ELIZA_CLOUD_BASE_URL: ${{ secrets.ELIZAOS_CLOUD_BASE_URL }}
|
|
ELIZA_CLOUD_API_BASE_URL: ${{ secrets.ELIZAOS_CLOUD_API_BASE_URL }}
|
|
|
|
- name: Prepare remote capability cloud live report directory
|
|
if: steps.cloud.outputs.capability_skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: |
|
|
node packages/scripts/rm-path-recursive.mjs reports/remote-capabilities/cloud
|
|
mkdir -p reports/remote-capabilities/cloud
|
|
|
|
- name: Remote capability cloud sandbox live smoke
|
|
if: steps.cloud.outputs.capability_skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: bun run --cwd packages/agent test:remote-capabilities:cloud-live
|
|
env:
|
|
ELIZA_LIVE_TEST: "1"
|
|
ELIZA_REMOTE_CAPABILITY_LIVE_REPORT_DIR: reports/remote-capabilities/cloud
|
|
ELIZAOS_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY != '' && secrets.ELIZAOS_CLOUD_API_KEY || secrets.ELIZACLOUD_API_KEY }}
|
|
ELIZAOS_CLOUD_BASE_URL: ${{ secrets.ELIZAOS_CLOUD_BASE_URL }}
|
|
ELIZA_CLOUD_BASE_URL: ${{ secrets.ELIZAOS_CLOUD_BASE_URL }}
|
|
|
|
- name: Validate remote capability cloud live report
|
|
if: steps.cloud.outputs.capability_skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: bun run test:remote-capabilities:validate-live-reports --kind cloud --expect-count 1 --max-age-minutes 90 --max-future-minutes 5 --require-ci --require-file-identity --match-github-env reports/remote-capabilities/cloud
|
|
|
|
- name: Upload remote capability cloud live report
|
|
if: steps.cloud.outputs.capability_skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
|
|
with:
|
|
name: remote-capability-cloud-live-report
|
|
path: reports/remote-capabilities/cloud/*.json
|
|
if-no-files-found: error
|
|
retention-days: 7
|
|
|
|
- name: Prune remote capability cloud report
|
|
if: always() && steps.cloud.outputs.capability_skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: node packages/scripts/rm-path-recursive.mjs reports/remote-capabilities/cloud
|
|
|
|
- name: Note remote capability cloud live smoke not observed
|
|
if: steps.cloud.outputs.skip != 'true' && github.event_name != 'workflow_dispatch' && github.event_name != 'schedule'
|
|
run: |
|
|
echo "::notice::Remote capability cloud sandbox live smoke is only observed on workflow_dispatch or schedule events."
|
|
{
|
|
echo "### Remote capability cloud live smoke"
|
|
echo ""
|
|
echo "Not observed for event \`${{ github.event_name }}\`. The full-surface cloud sandbox smoke runs only on \`workflow_dispatch\` and \`schedule\`."
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
provider-live-e2e:
|
|
name: Remote Capability Provider Live E2E
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 45
|
|
outputs:
|
|
skip: ${{ steps.providers.outputs.skip }}
|
|
steps:
|
|
- name: Check provider endpoint secrets
|
|
id: providers
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
E2B_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_E2B_URL }}
|
|
HOME_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_URL }}
|
|
MOBILE_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_URL }}
|
|
DESKTOP_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_URL }}
|
|
run: |
|
|
strict_context=false
|
|
case "${EVENT_NAME}" in
|
|
workflow_dispatch|schedule) strict_context=true ;;
|
|
esac
|
|
|
|
if [ -z "${E2B_URL}${HOME_URL}${MOBILE_URL}${DESKTOP_URL}" ]; then
|
|
message="No remote capability provider endpoints configured"
|
|
if [ "${strict_context}" = "true" ]; then
|
|
echo "::error::${message}; strict ${EVENT_NAME} live lanes must fail instead of greening by skip."
|
|
exit 1
|
|
fi
|
|
echo "::warning::${message} - skipping optional provider live E2E."
|
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
|
|
missing_required=()
|
|
if [ -z "${E2B_URL}" ]; then
|
|
missing_required+=("ELIZA_REMOTE_CAPABILITY_E2B_URL")
|
|
fi
|
|
if [ -z "${HOME_URL}" ]; then
|
|
missing_required+=("ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_URL")
|
|
fi
|
|
if [ -z "${MOBILE_URL}" ]; then
|
|
missing_required+=("ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_URL")
|
|
fi
|
|
|
|
if [ "${#missing_required[@]}" -gt 0 ]; then
|
|
message="Missing required remote capability provider endpoint secrets: ${missing_required[*]}"
|
|
if [ "${strict_context}" = "true" ]; then
|
|
echo "::error::${message}; strict ${EVENT_NAME} live lanes must fail instead of greening by skip."
|
|
exit 1
|
|
fi
|
|
echo "::warning::${message}. Skipping optional provider live E2E for ${EVENT_NAME}."
|
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Checkout
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Prepare remote capability provider live report directory
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: |
|
|
node packages/scripts/rm-path-recursive.mjs reports/remote-capabilities/providers
|
|
mkdir -p reports/remote-capabilities/providers
|
|
|
|
- name: Remote capability URL-backed provider live smoke
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: bun run --cwd packages/agent test:remote-capabilities:provider-live
|
|
env:
|
|
ELIZA_LIVE_TEST: "1"
|
|
ELIZA_REMOTE_CAPABILITY_LIVE_REPORT_DIR: reports/remote-capabilities/providers
|
|
ELIZA_REMOTE_CAPABILITY_E2B_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_E2B_URL }}
|
|
ELIZA_REMOTE_CAPABILITY_E2B_TOKEN: ${{ secrets.ELIZA_REMOTE_CAPABILITY_E2B_TOKEN }}
|
|
ELIZA_REMOTE_CAPABILITY_E2B_ENDPOINT_ID: ${{ secrets.ELIZA_REMOTE_CAPABILITY_E2B_ENDPOINT_ID }}
|
|
ELIZA_REMOTE_CAPABILITY_E2B_MODULES: ${{ secrets.ELIZA_REMOTE_CAPABILITY_E2B_MODULES }}
|
|
ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_URL }}
|
|
ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_TOKEN: ${{ secrets.ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_TOKEN }}
|
|
ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_ENDPOINT_ID: ${{ secrets.ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_ENDPOINT_ID }}
|
|
ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_MODULES: ${{ secrets.ELIZA_REMOTE_CAPABILITY_HOME_MACHINE_MODULES }}
|
|
ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_URL }}
|
|
ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_TOKEN: ${{ secrets.ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_TOKEN }}
|
|
ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_ENDPOINT_ID: ${{ secrets.ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_ENDPOINT_ID }}
|
|
ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_MODULES: ${{ secrets.ELIZA_REMOTE_CAPABILITY_MOBILE_COMPANION_MODULES }}
|
|
ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_URL: ${{ secrets.ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_URL }}
|
|
ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_TOKEN: ${{ secrets.ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_TOKEN }}
|
|
ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_ENDPOINT_ID: ${{ secrets.ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_ENDPOINT_ID }}
|
|
ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_MODULES: ${{ secrets.ELIZA_REMOTE_CAPABILITY_DESKTOP_COMPANION_MODULES }}
|
|
|
|
- name: Validate remote capability provider live reports
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: bun run test:remote-capabilities:validate-live-reports --kind provider --expect-count 3..4 --max-age-minutes 90 --max-future-minutes 5 --allowed-providers e2b,home-machine,mobile-companion,desktop-companion --require-providers e2b,home-machine,mobile-companion --require-ci --require-file-identity --match-github-env reports/remote-capabilities/providers
|
|
|
|
- name: Upload remote capability provider live report
|
|
if: steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
|
|
with:
|
|
name: remote-capability-provider-live-report
|
|
path: reports/remote-capabilities/providers/*.json
|
|
if-no-files-found: error
|
|
retention-days: 7
|
|
|
|
- name: Prune remote capability provider reports
|
|
if: always() && steps.providers.outputs.skip != 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
|
|
run: node packages/scripts/rm-path-recursive.mjs reports/remote-capabilities/providers
|
|
|
|
- name: Note provider live smoke not observed
|
|
if: steps.providers.outputs.skip != 'true' && github.event_name != 'workflow_dispatch' && github.event_name != 'schedule'
|
|
run: |
|
|
echo "::notice::Remote capability provider live smoke is only observed on workflow_dispatch or schedule events."
|
|
{
|
|
echo "### Remote capability provider live smoke"
|
|
echo ""
|
|
echo "Endpoint secrets are configured, but the full-surface provider live smoke was not observed for event \`${{ github.event_name }}\`. It runs only on \`workflow_dispatch\` and \`schedule\`."
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
github-live-artifact-validate:
|
|
name: Remote Capability GitHub Live Artifact Validator
|
|
needs:
|
|
- cloud-live-e2e
|
|
- provider-live-e2e
|
|
if: always() && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule') && (needs.cloud-live-e2e.outputs.capability_skip != 'true' || needs.provider-live-e2e.outputs.skip != 'true')
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 15
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
submodules: false
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
install-command: bun install
|
|
install-native-deps: "false"
|
|
skip-avatar-clone: "true"
|
|
no-vision-deps: "true"
|
|
|
|
- name: Download cloud + provider live report artifacts
|
|
run: |
|
|
mkdir -p reports/remote-capabilities/cloud reports/remote-capabilities/providers
|
|
gh run download "$GITHUB_RUN_ID" \
|
|
--repo "$GITHUB_REPOSITORY" \
|
|
--name remote-capability-cloud-live-report \
|
|
--dir reports/remote-capabilities/cloud || echo "::notice::remote-capability-cloud-live-report artifact not present"
|
|
gh run download "$GITHUB_RUN_ID" \
|
|
--repo "$GITHUB_REPOSITORY" \
|
|
--name remote-capability-provider-live-report \
|
|
--dir reports/remote-capabilities/providers || echo "::notice::remote-capability-provider-live-report artifact not present"
|
|
|
|
- name: Re-validate cloud live report from downloaded artifact
|
|
if: needs.cloud-live-e2e.result == 'success' && needs.cloud-live-e2e.outputs.capability_skip != 'true'
|
|
run: bun run test:remote-capabilities:validate-live-reports --kind cloud --expect-count 1 --max-age-minutes 120 --max-future-minutes 5 --require-ci --require-file-identity reports/remote-capabilities/cloud
|
|
|
|
- name: Re-validate provider live reports from downloaded artifacts
|
|
if: needs.provider-live-e2e.result == 'success' && needs.provider-live-e2e.outputs.skip != 'true'
|
|
run: bun run test:remote-capabilities:validate-live-reports --kind provider --expect-count 3..4 --max-age-minutes 120 --max-future-minutes 5 --allowed-providers e2b,home-machine,mobile-companion,desktop-companion --require-providers e2b,home-machine,mobile-companion --require-ci --require-file-identity reports/remote-capabilities/providers
|
|
|
|
# The merge queue's sole required context is `ci-ok`, and until now `ci-ok`
|
|
# needed only the test lanes — so a change that breaks repo-wide typecheck,
|
|
# fails biome lint, is unformatted, or commits a secret could still merge to
|
|
# develop as long as tests passed (all of these are required on `main` but were
|
|
# advisory on develop; #13617 F2). This job restores parity: it runs the same
|
|
# lint / format / typecheck / secret-scan gates that guard `main`, and `ci-ok`
|
|
# needs it. It runs anywhere `ci-ok` treats skipped jobs as failures (merge
|
|
# queue, develop push, manual dispatch, and nightly schedule); on PRs
|
|
# quality.yml already covers this surface, so the job self-skips and `ci-ok`'s
|
|
# lenient PR path lets the skip pass. Runner: fleet-aware like the test lanes —
|
|
# it was pinned to GitHub-hosted for independence from the self-hosted fleet,
|
|
# but with hosted runners billing-frozen (#13481) that pin froze this ci-ok
|
|
# dependency; route to self-hosted while the fleet is online.
|
|
merge-quality-gate:
|
|
name: Merge Queue Quality Gate
|
|
needs: changes
|
|
if: >-
|
|
github.event_name == 'merge_group' ||
|
|
github.event_name == 'push' ||
|
|
github.event_name == 'workflow_dispatch' ||
|
|
github.event_name == 'schedule'
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 40
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
|
with:
|
|
fetch-depth: 0
|
|
submodules: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Setup workspace dependencies
|
|
uses: ./.github/actions/setup-bun-workspace
|
|
with:
|
|
bun-version: ${{ env.BUN_VERSION }}
|
|
|
|
- name: Repo-wide biome lint
|
|
run: bun run lint
|
|
|
|
- name: Repo-wide biome format check
|
|
run: bun run format:check
|
|
|
|
- name: Repo-wide typecheck
|
|
run: bun run typecheck
|
|
|
|
- name: Stale base guard
|
|
env:
|
|
CURRENT_SHA: ${{ github.sha }}
|
|
run: |
|
|
set -euo pipefail
|
|
if git rev-parse -q --verify "${CURRENT_SHA}^" >/dev/null 2>&1; then
|
|
BASE_SHA="$(git rev-parse "${CURRENT_SHA}^1")"
|
|
node packages/scripts/stale-base-guard.self-test.mjs
|
|
node packages/scripts/stale-base-guard.mjs \
|
|
--base "$BASE_SHA" \
|
|
--head "$CURRENT_SHA" \
|
|
--merge-base "$BASE_SHA" \
|
|
--window 1200 \
|
|
--max-behind-commits 200 \
|
|
--max-behind-hours 72 \
|
|
--github \
|
|
--summary "$GITHUB_STEP_SUMMARY"
|
|
else
|
|
echo "::notice::Current SHA has no parent; stale-base guard has no target branch parent to compare."
|
|
fi
|
|
|
|
- name: Install gitleaks (OSS binary — no license required)
|
|
run: |
|
|
set -euo pipefail
|
|
GITLEAKS_VERSION=8.30.1
|
|
curl -sSL -o /tmp/gitleaks.tar.gz \
|
|
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
|
|
tar -xzf /tmp/gitleaks.tar.gz -C /tmp gitleaks
|
|
mkdir -p "$HOME/.local/bin"
|
|
install -m 0755 /tmp/gitleaks "$HOME/.local/bin/gitleaks"
|
|
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
|
"$HOME/.local/bin/gitleaks" version
|
|
|
|
- name: Secret scan (gitleaks)
|
|
env:
|
|
CURRENT_SHA: ${{ github.sha }}
|
|
run: |
|
|
set -euo pipefail
|
|
# Scan the queue/tip commit only. New content is what the merge
|
|
# introduces; a full-history re-scan re-flags long-known vendored
|
|
# artifacts (see gitleaks.yml for the same rationale). Merge queue
|
|
# SHAs are merge commits, and `git log -p -1 <merge>` emits no patch
|
|
# by default, so use `-m -p` whenever the tip has multiple parents.
|
|
parent_count="$(git rev-list --parents -n 1 "$CURRENT_SHA" | wc -w | tr -d ' ')"
|
|
if [ "$parent_count" -gt 2 ]; then
|
|
RANGE_ARGS=(--log-opts "-m -p -1 ${CURRENT_SHA}")
|
|
elif [ "$parent_count" -eq 2 ]; then
|
|
RANGE_ARGS=(--log-opts "-1 ${CURRENT_SHA}")
|
|
else
|
|
echo "gitleaks: tip ${CURRENT_SHA} is an orphan/root baseline commit (no parent);"
|
|
echo "its content was already scanned incrementally on develop. Skipping full-tree re-scan."
|
|
exit 0
|
|
fi
|
|
gitleaks detect \
|
|
--config .gitleaks.toml \
|
|
--source . \
|
|
--verbose \
|
|
--redact \
|
|
--report-format sarif \
|
|
--report-path gitleaks.sarif \
|
|
--no-banner \
|
|
"${RANGE_ARGS[@]}"
|
|
|
|
ci-ok:
|
|
name: ci-ok
|
|
if: ${{ !cancelled() && always() }}
|
|
needs:
|
|
- stale-base-preflight
|
|
- changes
|
|
- test-runner-vacuous-green-guard
|
|
- merge-quality-gate
|
|
- server-tests
|
|
- client-tests
|
|
- plugin-tests-status
|
|
- integration-tests
|
|
- desktop-contract
|
|
- zero-key-e2e
|
|
- cloud-live-e2e
|
|
- provider-live-e2e
|
|
- github-live-artifact-validate
|
|
# Fleet-aware (see `changes`): the aggregate gate itself must run to report a
|
|
# conclusion — a hardcoded hosted pin froze `ci-ok` on the billing-frozen
|
|
# fleet (#13481), so the required check never posted and nothing could merge.
|
|
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
|
|
timeout-minutes: 10
|
|
steps:
|
|
- name: Check test results
|
|
shell: bash
|
|
run: |
|
|
set -u
|
|
echo "=== Required test job results ==="
|
|
echo "stale-base-preflight:${{ needs.stale-base-preflight.result }}"
|
|
echo "changes: ${{ needs.changes.result }}"
|
|
echo "runner guard: ${{ needs.test-runner-vacuous-green-guard.result }}"
|
|
echo "merge-quality-gate:${{ needs.merge-quality-gate.result }}"
|
|
echo "server-tests: ${{ needs.server-tests.result }}"
|
|
echo "client-tests: ${{ needs.client-tests.result }}"
|
|
echo "plugin-tests: ${{ needs.plugin-tests-status.result }}"
|
|
echo "integration-tests:${{ needs.integration-tests.result }}"
|
|
echo "desktop-contract: ${{ needs.desktop-contract.result }}"
|
|
echo "zero-key-e2e: ${{ needs.zero-key-e2e.result }}"
|
|
echo "cloud-live-e2e: ${{ needs.cloud-live-e2e.result }}"
|
|
echo "provider-live-e2e:${{ needs.provider-live-e2e.result }}"
|
|
echo "github-live-artifact-validate:${{ needs.github-live-artifact-validate.result }}"
|
|
echo "==================================="
|
|
strict_results="${{ github.event_name == 'push' || github.event_name == 'merge_group' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' }}"
|
|
bad=0
|
|
for pair in \
|
|
"stale-base-preflight:${{ needs.stale-base-preflight.result }}" \
|
|
"changes:${{ needs.changes.result }}" \
|
|
"test-runner-vacuous-green-guard:${{ needs.test-runner-vacuous-green-guard.result }}" \
|
|
"merge-quality-gate:${{ needs.merge-quality-gate.result }}" \
|
|
"server-tests:${{ needs.server-tests.result }}" \
|
|
"client-tests:${{ needs.client-tests.result }}" \
|
|
"plugin-tests:${{ needs.plugin-tests-status.result }}" \
|
|
"integration-tests:${{ needs.integration-tests.result }}" \
|
|
"desktop-contract:${{ needs.desktop-contract.result }}" \
|
|
"zero-key-e2e:${{ needs.zero-key-e2e.result }}" \
|
|
"cloud-live-e2e:${{ needs.cloud-live-e2e.result }}" \
|
|
"provider-live-e2e:${{ needs.provider-live-e2e.result }}" \
|
|
"github-live-artifact-validate:${{ needs.github-live-artifact-validate.result }}"; do
|
|
name="${pair%%:*}"
|
|
result="${pair##*:}"
|
|
if [ "$name" = "stale-base-preflight" ] && [ "$result" = "skipped" ] && [ "${{ github.event_name }}" != "pull_request" ]; then
|
|
echo "::notice title=Test gate::PR stale-base preflight is skipped for ${{ github.event_name }}; merge-quality-gate runs the strict stale-base guard for this event."
|
|
continue
|
|
fi
|
|
if [ "$name" = "github-live-artifact-validate" ] && [ "$result" = "skipped" ] && [ "${{ github.event_name }}" != "workflow_dispatch" ] && [ "${{ github.event_name }}" != "schedule" ]; then
|
|
echo "::notice title=Test gate::Live artifact validation is observed only on workflow_dispatch/schedule, so its skip is allowed for ${{ github.event_name }}."
|
|
continue
|
|
fi
|
|
if [ "$name" = "github-live-artifact-validate" ] && [ "$result" = "skipped" ] && [ "${{ needs.cloud-live-e2e.outputs.capability_skip }}" = "true" ] && [ "${{ needs.provider-live-e2e.outputs.skip }}" = "true" ]; then
|
|
echo "::notice title=Test gate::Live artifact validation skipped because no live report producers were configured."
|
|
continue
|
|
fi
|
|
if [ "$result" = "skipped" ] && [ "$strict_results" != "true" ]; then
|
|
echo "::notice title=Test gate::Path-gated job '${name}' was skipped for this PR context."
|
|
continue
|
|
fi
|
|
if [ "$result" != "success" ]; then
|
|
echo "::error title=Test gate::Required job '${name}' finished with '${result}' (expected success)"
|
|
bad=1
|
|
fi
|
|
done
|
|
if [ "$bad" -ne 0 ]; then
|
|
exit 1
|
|
fi
|
|
echo "All required test jobs passed. Required develop ruleset check: ci-ok."
|