chore: import upstream snapshot with attribution
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
ci / test (push) Has been cancelled
ci / lint-and-format (push) Has been cancelled
ci / build (push) Has been cancelled
ci / dev-startup (push) Has been cancelled
gitleaks / gitleaks (push) Has been cancelled
Markdown Links / Relative Markdown Links (push) Has been cancelled
Quality (Extended) / Homepage Build (PR smoke) (push) Has been cancelled
Quality (Extended) / Comment-only diff guard (push) Has been cancelled
Quality (Extended) / Format + Type Safety Ratchet (push) Has been cancelled
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Has been cancelled
Quality (Extended) / Develop Gate (lint) (push) Has been cancelled
Chat shell gestures / Chat shell gesture + parity e2e (push) Has been cancelled
Cloud Gateway Discord / Test (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Has been cancelled
Build Agent Image / build-and-push (push) Has been cancelled
Dev Smoke / bun run dev onboarding chat (push) Has been cancelled
Dev Smoke / Vite HMR dependency-level smoke (push) Has been cancelled
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Has been cancelled
Publish @elizaos/example-code / check_npm (push) Has been cancelled
Publish @elizaos/example-code / publish_npm (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / verify_version (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / publish_npm (push) Has been cancelled
Sandbox Live Smoke / Sandbox live smoke (push) Has been cancelled
Snap Build & Test / Build Snap (amd64) (push) Has been cancelled
Snap Build & Test / Build Snap (arm64) (push) Has been cancelled
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Has been cancelled
Cloud Gateway Webhook / Test (push) Has been cancelled
Cloud Tests / lint-and-types (push) Has been cancelled
Cloud Tests / unit-tests (push) Has been cancelled
Cloud Tests / integration-tests (push) Has been cancelled
Cloud Tests / e2e-tests (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Deploy Apps Worker (Product 2) / Determine environment (push) Has been cancelled
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Has been cancelled
Deploy Eliza Provisioning Worker / Determine environment (push) Has been cancelled
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Has been cancelled
Dev Smoke / Classify changed paths (push) Has been cancelled
supply-chain / sbom (push) Has been cancelled
supply-chain / vulnerability-scan (push) Has been cancelled
Build, Push & Deploy to Phala Cloud / build-and-push (push) Has been cancelled
Test Packaging / Validate Packaging Configs (push) Has been cancelled
Test Packaging / Build & Test PyPI Package (push) Has been cancelled
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Has been cancelled
Test Packaging / Pack & Test JS Tarballs (push) Has been cancelled
UI Fixture E2E / ui-fixture-e2e (push) Has been cancelled
UI Fixture E2E / fixture-e2e (push) Has been cancelled
UI Story Gate / story-gate (push) Has been cancelled
vault-ci / test (macos-latest) (push) Has been cancelled
vault-ci / test (ubuntu-latest) (push) Has been cancelled
vault-ci / test (windows-latest) (push) Has been cancelled
vault-ci / app-core wiring tests (push) Has been cancelled
verify-patches / verify patches/CHECKSUMS.sha256 (push) Has been cancelled
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Has been cancelled
Voice Benchmark Smoke / voice bench smoke summary (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Has been cancelled
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:43:05 +08:00
commit 426e9eeabd
41828 changed files with 9656266 additions and 0 deletions
+159
View File
@@ -0,0 +1,159 @@
# CI Workflow Changelog
This changelog tracks meaningful CI policy and workflow-architecture changes.
It is intentionally scoped to `.github/workflows` so product/package changelogs
do not have to carry CI-only history.
## 2026-07-04
### Changed
- Migrated the whole repo off the Vercel SaaS Turbo remote cache onto the
GitHub-native cache (#12341, epic #12191 phase 4). Removed every
`TURBO_TOKEN` / `TURBO_TEAM` / `TURBO_CACHE: remote:rw` env from all
workflows (108 lines across 11 files) and routed `setup-bun-workspace`'s
Turbo cache through the pinned `turbo-cache-github` shim, whose single
per-hash key replaces the former per-job-name key that fragmented the cache
across ~30 job names.
Why: project guidance is GitHub-native caching only; the SaaS env was
redundant with the `.turbo` `actions/cache` layer that already ran in the
shared setup action. `ci-workflow-dedup-contract.mjs` now **forbids** the
SaaS env anywhere and **requires** the GitHub-native cache on the publish
paths (nightly/release) — the inverse of its previous assertion, which
pinned the SaaS wiring.
- PR-lane `typecheck` (`ci.yaml`) and `lint` (`quality.yml`) now run through
`turbo --affected` scoped to the PR merge base (`TURBO_SCM_BASE`), with full
history fetched so scoping resolves; `push`/`merge_group` still run the full
graph. A shallow clone degrades to running everything, so scoping never
under-checks.
### Added
- `develop-exhaustive.yml` (#12342, epic #12191 phase 5): an un-cancellable
scheduled orchestrator (06:00/18:00 UTC, dedicated concurrency group) that
invokes every platform lane test.yml's schedule does not cover — Windows,
mobile, scenario, the three UI gates, keyless harness, docker, dev
onboarding, electrobun/desktop — via `workflow_call`, then runs the matrix
proof. A skipped or failed reusable lane fails the run (a coverage gap is not
a pass). Added a bare `workflow_call:` trigger to each of those 10 workflows
so they are reusable.
Why: the scheduled exhaustive lane is the "prove develop runs the full
matrix" DoD. `ci-full-matrix-proof.mjs` + `ci-lane-manifest.json` gained a
`reusableWorkflows` check so dropping a lane's `uses:` or a workflow's
`workflow_call` trigger fails the proof statically, before the run.
## 2026-06-29
### Changed
- Split automatic branch ownership between `ci.yaml` and `test.yml`:
`ci.yaml` remains the main-branch gate, while `test.yml` now runs
automatically on develop plus manual/scheduled invocations.
Why: both workflows were auto-running overlapping build/test work on `main`.
The zero-key PR gate for main remains covered by `scenario-pr.yml`, and
develop keeps the broader `Tests` orchestrator.
- Switched the root `test:plugins` script from a direct Turbo sweep to the
shard-aware cross-package test runner at concurrency 3.
Why: `TEST_SHARD` is implemented in `run-all-tests.mjs`. Routing the plugin
lane through that runner makes the matrix real instead of cosmetic, while
preserving the existing bounded concurrency policy.
### Added
- Added `packages/scripts/ci-workflow-dedup-contract.mjs`.
Why: this locks the branch-trigger split and verifies nightly/release keep the
Turbo remote-cache environment required by #10096.
- Added develop/main quality gates for the prompt-secret scan, UI determinism
audit, and build-enabled lint.
Why: `ci.yaml` still runs only for `main`, while the default PR target is
`develop`. These checks now run through `quality.yml`, which already covers
develop PRs.
- Split the plugin test lane into a four-way `TEST_SHARD` matrix with a stable
aggregate `Plugin Tests` check.
Why: the cross-package runner already had deterministic shard membership, but
no workflow used it. The plugin lane is the long pole in `test.yml`; sharding
it cuts wall-clock without reducing package coverage.
## 2026-06-14
### Added
- Added `packages/scripts/ci-path-gate.mjs` as the shared PR path classifier for
expensive workflows.
Why: repeated inline shell classifiers made it too easy for workflows to drift
apart, and reviewers could not see a consistent explanation for why a lane ran
or skipped.
- Added path-gate summaries to show changed files, selected lanes, and the path
or label reason for each lane.
Why: fast CI is only useful if contributors and maintainers trust the skip
decision. The summary turns the decision into reviewable evidence.
- Added force labels including `ci:full`, `ci:e2e`, `ci:zero-key`,
`ci:server`, `ci:client`, `ci:plugins`, `ci:cloud`, `ci:docker`,
`ci:mobile`, `ci:ios`, `ci:android`, `ci:desktop`, `ci:windows`, and
`ci:dev-smoke`.
Why: maintainers need a no-code way to request broader coverage when a change
is risky, cross-cutting, or ambiguous.
- Added `packages/scripts/ci-path-gate.self-test.mjs` and run it before the
`Tests` workflow consumes classifier outputs.
Why: the classifier is now part of the quality gate. Testing it in CI prevents
a future edit from silently skipping coverage that should have run.
### Changed
- Replaced inline path filters in `test.yml` and `scenario-pr.yml` with the
shared classifier.
Why: one implementation is easier to audit, document, and extend than several
workflow-local shell snippets.
- Added classifier jobs to Docker, mobile, dev smoke, Windows dev smoke, and
Windows desktop preload smoke workflows.
Why: these lanes are valuable but expensive. Running them only for relevant
PRs keeps feedback fast while still preserving push/manual coverage.
- Split deterministic zero-key E2E work into named parallel slices while keeping
the visible `Zero-Key Deterministic E2E` aggregate check.
Why: a single serial E2E log made failures slow to reach and hard to triage.
Parallel slices shorten wall-clock time and make the failing surface obvious
without removing the aggregate gate reviewers already understand.
- Moved `coverage-gate` dependency setup behind changed-test detection and
switched it to the shared Bun workspace setup.
Why: the coverage gate is advisory for changed Bun-native tests. Docs-only and
no-test PRs should not install the whole workspace or fail on unrelated
registry/install noise, while test-bearing PRs should use the same
lockfile-validating setup path as the rest of CI.
### Preserved
- Push, scheduled, and manual runs keep broad/default behavior.
Why: those runs protect branch health, release readiness, and periodic
confidence. PR path gates optimize contributor feedback, not the repository's
deeper safety net.
- The split E2E jobs keep the previous substantive commands.
Why: this change is a CI ergonomics and parallelism improvement, not a
reduction in coverage.
+289
View File
@@ -0,0 +1,289 @@
# CI/CD Workflows
This directory contains GitHub Actions workflows for the elizaOS project (v2.0.0).
## Workflow Overview
| Workflow | Trigger | Purpose |
|----------|---------|---------|
| `ci.yaml` | Push/PR to main | Main-specific CI - typecheck, tests, lint, build, dev startup |
| `test.yml` | Push/PR to develop, manual, schedule | Broader develop tests plus required zero-key deterministic E2E; live jobs are separate |
| `quality.yml` | Push/PR to main/develop, manual | Develop/main quality gates: format, type-safety ratchet, prompt-secret scan, UI determinism, lint |
| `scenario-pr.yml` | PR to main/develop, manual | Secret-free deterministic scenario/browser E2E gate |
| `scenario-matrix.yml` | Develop/manual opt-in | Real-service scenario matrix; not a PR gate |
| `pr.yaml` | PR opened/edited | PR title validation |
| `release.yaml` | Push to main, Release | NPM beta/production package releases |
| `claude.yml` | @claude mentions | Interactive Claude assistance |
| `claude-code-review.yml` | PR opened | Automated code review |
| `claude-security-review.yml` | PR opened | Security-focused review |
| `codeql.yml` | Push/PR to main, Weekly | Static security analysis |
| `docs-ci.yml` | PR (docs paths), Manual | Documentation quality checks |
| `build-agent-image.yml` | Push develop/main, Release, Manual | Docker image builds (`:develop`, `:stable`, `:latest`, release tags) |
| `tee-build-deploy.yml` | Push to main, Manual | TEE deployment to Phala Cloud |
| `weekly-maintenance.yml` | Weekly, Manual | Dependency/security audits |
| `jsdoc-automation.yml` | Manual | JSDoc generation |
## Release Workflows
### Alpha Tags
Alpha version tags are tags only. They do not publish NPM packages, run packaging
CI, or create GitHub Release entries.
### NPM Beta/Production Packages (`release.yaml`)
Publishes TypeScript/JavaScript packages to NPM.
**Triggers:**
- Push to `main` → Beta release (`@beta` tag)
- GitHub Release created → Production release (`@latest` tag)
**Packages:** All `@elizaos/*` packages in the monorepo
## Test Workflows
### Linux Runner Policy
The heavy develop **test lanes** in `test.yml` run on the self-hosted
`self-hosted, hetzner-robot` pool (GitHub-hosted minutes are billing-frozen for
this org, #13481). Everything the **merge gate** depends on to *reach a
conclusion* stays GitHub-hosted so a drained fleet can never wedge develop:
- **Path classifiers** (`Classify changed paths`) across `test.yml`,
`scenario-pr.yml`, `dev-smoke.yml`, `docker-ci-smoke.yml`,
`mobile-build-smoke.yml`, `windows-dev-smoke.yml`, and
`windows-desktop-preload-smoke.yml` run on `ubuntu-24.04`. They are git-diff +
node scripts with no self-hosted needs; pinning them to the fleet (#8501) once
left every downstream job queued indefinitely and gridlocked develop.
- **`ci-ok`** (the merge queue's sole required context), its
`plugin-tests-status` roll-up, and the hosted **`merge-quality-gate`** all run
on `ubuntu-24.04`.
Two SPOF guards, enforced by `packages/scripts/ci-merge-gate-contract.mjs` (run
in the `changes` job, #13617):
1. **Fleet-drain toggle.** Every self-hosted lane in `test.yml` reads
`runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}`.
Unset/anything-but-`false` keeps the current self-hosted placement; there is
no way to probe fleet health from a `runs-on:` expression, so during an
outage an admin sets repo **variable** `HETZNER_FLEET_ONLINE=false` once and
the whole workflow falls back to hosted — one flip unblocks the entire queue
instead of per-PR admin-bypass. Keep the runner-agnostic step hardening (no
`sudo`-only install/cleanup) so lanes run on either runner type.
2. **Hosted quality parity.** `merge-quality-gate` runs the same lint /
`format:check` / repo-wide `typecheck` / gitleaks secret scan that guard
`main`, and `ci-ok` needs it — so a lint, type, format, or committed-secret
regression is refused by the merge queue on develop, not just on `main`. It
runs on `merge_group` + develop `push`. The lightweight `develop-pr.yml`
lint job also runs `format:check`, so formatting fails on the PR even when a
busy push wave supersedes post-merge quality runs (#15959).
CodeQL is a separate exception: trusted push, scheduled, and manual CodeQL runs
use `self-hosted, Linux, X64, hetzner-robot` because full JavaScript analysis is
disk-bound and has exhausted GitHub-hosted runners during the `PolynomialReDoS`
dataflow query. Pull-request CodeQL remains GitHub-hosted so forked code never
executes on self-hosted machines. Keep the full CodeQL query surface intact;
move capacity around rather than weakening security coverage. The CodeQL config
may ignore deliberately invalid negative-test fixtures, but not real source
files; those fixtures should stay covered by their owning tests.
GPU / KVM / macOS jobs (labels `gpu-cuda-12.6`, `kvm`, `eliza-e2e-macos`) are a
separate purpose-built fleet and are unaffected by this policy.
### PR Path Gates
PR workflows use `packages/scripts/ci-path-gate.mjs` to keep expensive lanes
targeted. Each classifier job writes a GitHub step summary showing:
- which files changed
- which lanes will run
- which path or label caused each lane to run
Maintainers can force specific lanes with labels:
| Label | Effect |
|-------|--------|
| `ci:full` | Run every path-gated lane in workflows that honor the shared gate |
| `ci:e2e` / `ci:zero-key` | Run deterministic zero-key E2E lanes |
| `ci:scenario` | Run `scenario-pr.yml` deterministic scenario/browser E2E |
| `ci:server` | Run server tests |
| `ci:client` | Run client tests |
| `ci:plugins` | Run plugin tests |
| `ci:cloud` | Run cloud live E2E where secrets are configured |
| `ci:docker` | Run Docker CI smoke |
| `ci:mobile` / `ci:ios` / `ci:android` | Run mobile smoke, or one mobile platform |
| `ci:desktop` / `ci:windows` | Run desktop and Windows smoke lanes |
| `ci:dev-smoke` | Run the `bun run dev` onboarding smoke |
Push, scheduled, and manual runs keep their broader/default behavior; the path
gate mainly keeps PR feedback fast and explainable.
Why this exists:
- OSS contributors should get useful feedback quickly without waiting on
unrelated mobile, Docker, desktop, Windows, or browser-heavy lanes.
- Maintainers should be able to see why a lane ran or skipped from the job
summary, without reverse-engineering shell conditionals.
- The quality gate should stay equivalent for affected code. Path gates decide
which surface is relevant; they do not replace the tests for that surface.
- Push, scheduled, and manual runs remain broad because they protect branch
health, release readiness, and nightly confidence rather than one PR diff.
Quality contract:
- Any path-gated lane must be forced by `ci:full`.
- Every expensive lane needs a matching force label so maintainers can request
coverage without pushing a no-op commit.
- Workflow, shared setup, toolchain, lockfile, and classifier changes should run
the affected expensive lanes because they can change CI behavior even when
product code did not move.
- The `Tests` workflow runs the classifier self-test before consuming classifier
outputs. That self-test covers representative path matches and label forcing
so a future edit cannot silently weaken the broadest PR test gate.
- When splitting a long lane, keep the same substantive commands unless the PR
explicitly documents the safety reason for removing one.
Long deterministic E2E gates are split into named parallel slices for unit/UI
coverage, browser coverage, diagnostics, and scenario execution. The visible
`Zero-Key Deterministic E2E` check is an aggregate status over those slices, so
reviewers can see the failing surface without opening one giant serial log.
Plugin tests are also split across `TEST_SHARD=1/4` through `4/4` in the
`Tests` workflow. The root `test:plugins` script uses the cross-package runner
so shard membership is deterministic by package path, while the visible
`Plugin Tests` check remains an aggregate over the shard matrix.
Why the aggregate stays:
- Branch protection and reviewer muscle memory can keep using one stable check.
- The underlying slices can run in parallel and fail with precise names.
- Manual review becomes easier because a browser failure, diagnostics failure,
or scenario-runner failure points at the relevant log immediately.
Related CI docs:
- `CHANGELOG.md` records workflow policy changes and the reason they happened.
- `ROADMAP.md` tracks future CI performance work that should preserve gate
quality.
### Main CI (`ci.yaml`)
Runs on PRs and pushes to main:
- Typecheck + core/plugin tests
- Linting and formatting checks
- Build verification
- Dev startup + HMR propagation
- Interop TypeScript tests (`packages/interop`)
The broader `test.yml` orchestrator runs automatically on `develop` only to
avoid duplicating the main-branch CI gate. Secret-free deterministic zero-key
coverage for PRs to either protected branch is handled by `scenario-pr.yml`;
`test.yml` keeps the broader develop push/PR, manual, and scheduled coverage.
### Live E2E
PR E2E does not require `CEREBRAS_API_KEY`, `OPENAI_API_KEY`, or any other paid
provider key. Live/provider-key coverage belongs to the dedicated live jobs and
workflows (`cloud-live-e2e`, `provider-live-e2e`, `live-scenarios.yml`,
`scenario-matrix.yml`) where missing-key behavior is documented per lane.
## Code Review Workflows
### Claude Code Review (`claude-code-review.yml`)
Automated PR review using Claude. Checks for:
- Security issues (hardcoded keys, SQL injection, XSS)
- Test coverage
- TypeScript types (no `any`)
- Correct tooling (bun, vitest)
### Claude Security Review (`claude-security-review.yml`)
Dedicated security-focused review for code changes.
### Claude Interactive (`claude.yml`)
Responds to `@claude` mentions in issues and PRs.
## Documentation Workflows
### Docs CI (`docs-ci.yml`)
Documentation quality workflow:
- **Dead Link Checking:** Scans for broken internal/external links
- **Quality Checks:** Double headers, missing frontmatter, heading hierarchy
Automatically creates PRs with fixes when issues are found.
### JSDoc Automation (`jsdoc-automation.yml`)
Manual workflow for generating JSDoc documentation.
## Manual Release Process
### 1. Create a GitHub Release
1. Go to Releases → Create new release
2. Create a new tag: `v2.0.0` (follows semver)
3. Add release notes
4. Publish release
### 2. Automated Publishing
The release will trigger:
- `release.yaml` → NPM packages
### 3. Manual publishing
Use `bunx lerna publish` from the repo root when automation is not sufficient (see `release.yaml`).
## Setting Up Secrets
### Required Secrets
| Secret | Purpose | How to Get |
|--------|---------|------------|
| `NPM_TOKEN` | NPM publishing | [npmjs.com/settings/~/tokens](https://www.npmjs.com/settings/~/tokens) |
| `ANTHROPIC_API_KEY` | Claude workflows | [console.anthropic.com](https://console.anthropic.com) |
| `OPENAI_API_KEY` | Opt-in live/provider-key lanes | [platform.openai.com](https://platform.openai.com) |
### Optional Secrets
| Secret | Purpose |
|--------|---------|
| `PHALA_CLOUD_API_KEY` | TEE deployment |
| `GH_PAT` | Cross-repo operations |
Turbo caching is GitHub-native (`.github/actions/turbo-cache-github` via
`setup-bun-workspace`) — no Vercel SaaS remote cache, so `TURBO_TOKEN` /
`TURBO_TEAM` are no longer used and are banned by
`ci-workflow-dedup-contract.mjs` (#12341).
## Package dependencies
NPM packages are ordered by the monorepo graph; `release.yaml` / Lerna handle publish ordering for `@elizaos/*` packages.
## Troubleshooting
### CI Failures
1. Check if tests pass locally: `bun run test`
2. Check formatting: `bun run format:check`
3. Check linting: `bun run lint`
### Release Failures
1. Verify secrets are configured
2. Check workflow logs for specific errors
3. For NPM: ensure package versions are unique
### Claude Workflow Issues
1. Verify `ANTHROPIC_API_KEY` is set
2. Check rate limits on Anthropic API
3. Review Claude's output in workflow logs
+79
View File
@@ -0,0 +1,79 @@
# CI Workflow Roadmap
This roadmap is for making CI faster and friendlier without weakening the
quality gate.
## Principles
- Prefer affected-surface execution over blanket PR execution.
Why: contributors should not wait on unrelated platform builds, but affected
code still needs the same depth of verification.
- Keep broad verification on push, schedule, and manual dispatch.
Why: selective PR checks optimize review latency; broad branch checks catch
integration issues that only appear after many changes compose.
- Make every skip explainable and overridable.
Why: maintainers need confidence that a skipped lane was intentionally out of
scope, and they need a label escape hatch when judgment says to run it anyway.
- Split long serial lanes before removing coverage.
Why: parallelism usually improves developer experience without trading away
signal.
## Near Term
- Record workflow-level and job-level duration trends for PR and `develop`
pushes.
Why: the current long pole can move as workflows are split. Duration data lets
us optimize the real bottleneck instead of guessing from one run.
- Continue splitting long jobs into independently named slices with aggregate
status checks.
Why: aggregate checks preserve branch-protection simplicity while named slices
improve failure triage and parallelism.
- Expand classifier self-tests when adding a new path-gated workflow.
Why: every new gate adds skip logic. The test suite should grow with the blast
radius of that logic.
## Next
- Move repeated setup into reusable actions where it does not hide important
workflow behavior.
Why: duplicated setup makes split jobs noisy, but overly opaque setup makes CI
harder to debug. Reusable actions should reduce repetition while keeping logs
readable.
- Add a documented owner map for path-gate rules.
Why: package owners can judge whether a path belongs in server, client,
plugin, cloud, mobile, desktop, Docker, or E2E coverage better than a generic
workflow edit can.
- Evaluate matrix shaping for Windows CI.
Why: recent completed `develop` runs show Windows CI as the workflow-level
long pole. More parallelism or tighter affected-surface routing there may
produce the biggest wall-clock win.
## Later
- Publish a compact CI timing summary on PRs.
Why: contributors should see whether the slowest check is queue time, setup,
test execution, artifact upload, or a platform-specific lane.
- Add a periodic full-gate audit that compares path-gated PR behavior against a
broad run for representative diffs.
Why: selective CI needs periodic calibration so the faster path remains as
trustworthy as the older all-in path.
@@ -0,0 +1,412 @@
name: Actions Zombie Janitor
# Reap zombie workflow runs (dead-runner `in_progress`, wedged `queued`,
# production deploy `pending` with zero jobs, stale `waiting`) that freeze org
# concurrency or silently block deploys.
#
# WHY: when a self-hosted runner box dies, its runs stay `in_progress` until
# the job timeout (6h default, up to 12h here) — and each one keeps holding
# org concurrency slots. With enough zombies, NO new job can start repo-wide,
# including GitHub-hosted ones (observed 2026-07-01 ~21:3322:30 UTC: ~40
# zombies from a 17:4x runner death froze every queue — PR checks, gitleaks,
# and the production deploy dispatches; see #10839 updates 912 and #11045).
# Cancelling QUEUED runs does not free slots — the slots are held by the dead
# in-progress runs — but a WEDGED queued run is still poison: it holds its
# concurrency group and GitHub replaces-and-cancels every newer pending run in
# that group.
#
# WHY `waiting` RUNS TOO (2026-07-06 incident, #15147): a run `waiting` on an
# environment approval that never comes holds its concurrency group until
# GitHub's 30-day auto-expiry — with `cancel-in-progress: false` every later
# run of that group sits `pending` with zero jobs and dies silently. Two runs
# waiting since 2026-06-18 deadlocked the production backend + provisioning
# worker deploy lanes for 18 days; production ran June-18 code the whole time.
# Nobody approves a production deploy two days late — reaping the gate frees
# the group so the next dispatch deploys tip.
#
# WHY THE JANITOR ITSELF MUST BE FREEZE-PROOF (2026-07-02/03 incident): the
# previous single-lane design (one ubuntu-latest job + one shared concurrency
# group) had zero successful executions from 2026-07-02 14:15 UTC through
# 2026-07-03 while a fresh zombie batch (born 07-02 23:3123:50, hetzner-robot
# death) froze the queues ~3x in 24h and ~34 runs had to be force-cancelled BY
# HAND. Two failure modes compounded:
# 1. SLOT STARVATION — the janitor needs a runner slot from the very pool
# the zombies exhaust, so during a full freeze its scheduled runs starve
# in `queued` and never execute.
# 2. CONCURRENCY-GROUP SELF-BRICK — one wedged queued janitor run (e.g.
# 28635685728, live-verified `queued` for 13+ hours while other
# ubuntu-latest jobs started fine) held the shared group forever; GitHub
# then replaced-and-cancelled every newer pending tick (10 consecutive
# janitor runs cancelled unexecuted, 28628876713…28666171315).
# Fixes here:
# - TWO INDEPENDENT LANES (matrix): a GitHub-hosted job and a self-hosted
# robot-fleet job. A freeze that exhausts one pool leaves the other lane
# runnable; either lane alone fully reaps. Lanes never touch each other.
# - NO CONCURRENCY GROUP. Overlapping janitors are harmless (cancels are
# idempotent; every candidate is re-verified live), and any queued backlog
# purges itself: each lane cancels superseded queued janitor ticks.
# - Wedged-queued reaping: own ticks queued >60 min, anything queued >12 h.
# - Production Cloud CF Deploy pending/zero-job reaping: a prod deploy that
# never instantiates a job is neither an approval gate nor a runner backlog;
# after 15 minutes the janitor force-cancels it and redispatches once
# (#15503/#15499).
# - actions/github-script instead of gh/jq — self-hosted boxes only need
# the runner-bundled Node.
#
# WHAT COUNTS AS A ZOMBIE `in_progress` RUN — both must hold:
# 1. the current attempt is older than MAX_AGE_MINUTES (default 120), AND
# 2. no job OR STEP of the run started or completed within IDLE_MINUTES
# (default 90).
# The idle check is load-bearing: right after a freeze clears, hours-old runs
# legitimately start executing their backlogged jobs — age alone would kill
# healthy recovering runs; age+idle reaps only runs nothing is working on.
# Step timestamps (new) keep long multi-step jobs alive while their steps
# progress; a dead runner freezes job AND step timestamps, so zombies still
# trip the check. IDLE_MINUTES must stay above the longest legitimately-silent
# single-step stretch; 90 min clears every current suite. The two 12h KVM/AOSP
# suites run single steps silent for longer than any sane threshold, so they
# are exempt by name (their zombies fall to their own job timeout).
#
# Tuning (repo variables, no code change needed):
# ACTIONS_JANITOR_DISABLED=true — kill switch (both lanes)
# ACTIONS_JANITOR_ROBOT_LANE_DISABLED — =true remaps the robot lane onto
# ubuntu-latest (a harmless idempotent duplicate; `matrix` is not a legal
# context in a job-level `if`, so the lane is disabled by re-homing it)
# ACTIONS_JANITOR_ROBOT_RUNNER_JSON — robot lane runs-on JSON array
# ACTIONS_JANITOR_MAX_AGE_MINUTES — default 120
# (legacy ACTIONS_JANITOR_MAX_AGE_HOURS honored if the minutes var is unset)
# ACTIONS_JANITOR_IDLE_MINUTES — default 90
# ACTIONS_JANITOR_QUEUED_MAX_AGE_HOURS — default 24 (wedged-queued reap;
# deliberately at GitHub's own queued-job expiry so the sweep only catches
# runs GitHub failed to fail — hours-old queued runs can be legitimate
# freeze backlog that still executes once slots free, and cancelling a
# required PR check strands the PR red; tune down mid-incident if needed)
# ACTIONS_JANITOR_PENDING_ZERO_JOB_MINUTES — default 15 (production Cloud CF
# Deploy pending/zero-job zombie reap + one redispatch)
# ACTIONS_JANITOR_SELF_QUEUED_MINUTES — default 60 (own stale ticks)
# ACTIONS_JANITOR_WAITING_MAX_AGE_HOURS — default 48 (stale environment-gate
# reap; far above any sane approval latency, far below GitHub's 30-day
# expiry that let #15147 fester)
# ACTIONS_JANITOR_EXEMPT_WORKFLOWS — comma-separated workflow names
on:
schedule:
- cron: "*/15 * * * *"
workflow_dispatch:
inputs:
dry_run:
description: Log what would be cancelled without cancelling
required: false
default: false
type: boolean
permissions:
actions: write
jobs:
reap:
name: Reap zombie runs (${{ matrix.lane }})
strategy:
fail-fast: false
matrix:
include:
- lane: github-hosted
runner: '["ubuntu-latest"]'
- lane: robot-fleet
runner: ${{ vars.ACTIONS_JANITOR_ROBOT_LANE_DISABLED == 'true' && '["ubuntu-latest"]' || vars.ACTIONS_JANITOR_ROBOT_RUNNER_JSON || '["self-hosted","Linux","X64","hetzner-robot"]' }}
runs-on: ${{ fromJSON(matrix.runner) }}
if: github.repository == 'elizaOS/eliza' && vars.ACTIONS_JANITOR_DISABLED != 'true'
timeout-minutes: 10
steps:
- name: Cancel zombie in_progress + wedged queued + pending deploy + stale waiting runs
uses: actions/github-script@v7
env:
MAX_AGE_MINUTES: ${{ vars.ACTIONS_JANITOR_MAX_AGE_MINUTES || '' }}
LEGACY_MAX_AGE_HOURS: ${{ vars.ACTIONS_JANITOR_MAX_AGE_HOURS || '' }}
IDLE_MINUTES: ${{ vars.ACTIONS_JANITOR_IDLE_MINUTES || '90' }}
QUEUED_MAX_AGE_HOURS: ${{ vars.ACTIONS_JANITOR_QUEUED_MAX_AGE_HOURS || '24' }}
PENDING_ZERO_JOB_MINUTES: ${{ vars.ACTIONS_JANITOR_PENDING_ZERO_JOB_MINUTES || '15' }}
SELF_QUEUED_MINUTES: ${{ vars.ACTIONS_JANITOR_SELF_QUEUED_MINUTES || '60' }}
WAITING_MAX_AGE_HOURS: ${{ vars.ACTIONS_JANITOR_WAITING_MAX_AGE_HOURS || '48' }}
EXEMPT_WORKFLOWS: ${{ vars.ACTIONS_JANITOR_EXEMPT_WORKFLOWS || 'ElizaOS Cuttlefish,ElizaOS OpenAgent E1 (RISC-V AI SoC)' }}
DRY_RUN: ${{ inputs.dry_run == true && 'true' || 'false' }}
LANE: ${{ matrix.lane }}
with:
script: |
const cfg = {
maxAgeMin:
Number(process.env.MAX_AGE_MINUTES) ||
Number(process.env.LEGACY_MAX_AGE_HOURS) * 60 ||
120,
idleMin: Number(process.env.IDLE_MINUTES) || 90,
queuedMaxAgeMin: (Number(process.env.QUEUED_MAX_AGE_HOURS) || 24) * 60,
pendingZeroJobMin: Number(process.env.PENDING_ZERO_JOB_MINUTES) || 15,
selfQueuedMin: Number(process.env.SELF_QUEUED_MINUTES) || 60,
waitingMaxAgeMin: (Number(process.env.WAITING_MAX_AGE_HOURS) || 48) * 60,
// Janitor lane jobs have timeout-minutes:10; an own-workflow run
// in_progress far past that means its runner died mid-reap.
selfInProgressMin: 60,
exempt: (process.env.EXEMPT_WORKFLOWS || '')
.split(',')
.map((s) => s.trim())
.filter(Boolean),
dryRun: process.env.DRY_RUN === 'true',
lane: process.env.LANE || 'unknown',
};
const { owner, repo } = context.repo;
const now = Date.now();
const minutesSince = (iso) => Math.floor((now - Date.parse(iso)) / 60000);
const rows = [];
const reap = async (run, ageMin, idleMin, why) => {
const label = `${run.id} ${run.name} (age ${ageMin}m, idle ${idleMin}m, ${why})`;
if (cfg.dryRun) {
core.info(`would cancel: ${label}`);
rows.push([`${run.id}`, run.name, `${ageMin}`, `${idleMin}`, `would cancel (dry run) — ${why}`]);
return;
}
// FORCE-cancel: a plain cancel silently fails to reap runs whose
// runner died (they sit in cancel-requested limbo until the job
// timeout — exactly the runs this janitor exists for). The
// force-cancel endpoint kills them immediately; fall back to the
// polite cancel for anything the force path rejects.
try {
await github.request('POST /repos/{owner}/{repo}/actions/runs/{run_id}/force-cancel', {
owner, repo, run_id: run.id,
});
core.info(`force-cancelled: ${label}`);
rows.push([`${run.id}`, run.name, `${ageMin}`, `${idleMin}`, `force-cancelled — ${why}`]);
} catch (e) {
try {
await github.rest.actions.cancelWorkflowRun({ owner, repo, run_id: run.id });
core.info(`cancelled: ${label}`);
rows.push([`${run.id}`, run.name, `${ageMin}`, `${idleMin}`, `cancelled — ${why}`]);
} catch (e2) {
core.info(`cancel failed (likely already completed / raced with sibling lane): ${label}: ${e2.status ?? e2.message}`);
}
}
};
// The list endpoint serves STALE entries for force-killed runs —
// re-verify each candidate individually before judging it.
const liveStatus = async (runId) => {
try {
return (await github.rest.actions.getWorkflowRun({ owner, repo, run_id: runId })).data.status;
} catch {
return null;
}
};
const liveRun = async (runId) => {
try {
return (await github.rest.actions.getWorkflowRun({ owner, repo, run_id: runId })).data;
} catch {
return null;
}
};
// Both listings are pre-filtered server-side to runs CREATED before
// the smallest relevant threshold: the previous `--limit 100`
// newest-first listing could drop the oldest zombies — the exact
// runs to reap — once the queue saturated (observed: >1000 queued
// runs during the 2026-07-03 freeze put the 13h-wedged run beyond
// any sane pagination depth). A run created recently cannot have
// started earlier, so nothing eligible is ever excluded; re-run
// attempts (old created_at, fresh run_started_at) stay listed and
// are age-gated per attempt below.
const createdBefore = (minutes) =>
`<${new Date(now - minutes * 60000).toISOString()}`;
// ---- 1. zombie in_progress runs (these hold the concurrency slots) ----
const inProgress = await github.paginate(github.rest.actions.listWorkflowRunsForRepo, {
owner, repo, status: 'in_progress', per_page: 100,
created: createdBefore(Math.min(cfg.maxAgeMin, cfg.selfInProgressMin)),
});
inProgress.sort(
(a, b) => Date.parse(a.run_started_at || a.created_at) - Date.parse(b.run_started_at || b.created_at),
);
core.info(`[${cfg.lane}] in_progress listed: ${inProgress.length}`);
for (const run of inProgress) {
if (run.id === context.runId) continue;
// Age of the CURRENT attempt (run_started_at resets on re-run);
// created_at would overstate the age of re-run attempts.
const ageMin = minutesSince(run.run_started_at || run.created_at);
if (run.name === context.workflow) {
// Never touch the live sibling lane; only reap a janitor run
// whose own runner died mid-reap.
if (ageMin < cfg.selfInProgressMin) continue;
if ((await liveStatus(run.id)) !== 'in_progress') continue;
await reap(run, ageMin, ageMin, 'dead janitor run');
continue;
}
if (cfg.exempt.includes(run.name)) {
core.info(`exempt: ${run.id} ${run.name}`);
continue;
}
if (ageMin < cfg.maxAgeMin) continue;
const status = await liveStatus(run.id);
if (status !== 'in_progress') {
core.info(`skip (list-stale, actually '${status}'): ${run.id} ${run.name}`);
continue;
}
// Newest job OR step activity across up to 100 jobs; a run with
// no timestamps at all idles from its attempt start.
let latest = 0;
try {
const jobs = await github.paginate(github.rest.actions.listJobsForWorkflowRun, {
owner, repo, run_id: run.id, filter: 'latest', per_page: 100,
});
for (const j of jobs) {
for (const t of [j.started_at, j.completed_at]) {
if (t) latest = Math.max(latest, Date.parse(t));
}
for (const s of j.steps ?? []) {
for (const t of [s.started_at, s.completed_at]) {
if (t) latest = Math.max(latest, Date.parse(t));
}
}
}
} catch (e) {
core.warning(`jobs fetch failed for ${run.id}: ${e.status ?? e.message}`);
}
const idleMin = latest ? Math.floor((now - latest) / 60000) : ageMin;
if (idleMin < cfg.idleMin) {
core.info(`alive: ${run.id} ${run.name} (age ${ageMin}m, idle ${idleMin}m)`);
continue;
}
await reap(run, ageMin, idleMin, 'zombie in_progress');
}
// ---- 2. wedged queued runs (these brick concurrency groups) ----
// Cancelling queued runs does not free slots, but a run stuck in
// `queued` for hours holds its concurrency group and gets every
// newer pending run of that group replaced-and-cancelled (this is
// what bricked the janitor itself on 2026-07-03). Own ticks are
// superseded after selfQueuedMin (a later tick reaps strictly
// better). Other workflows only past queuedMaxAgeMin (24h —
// GitHub's own queued expiry): an hours-old queued run can be
// legitimate freeze backlog that still executes once slots free,
// and cancelling a required PR check strands the PR red.
// Two tightly-scoped listings instead of one walk of the whole
// queued backlog (which exceeds 1000 runs mid-freeze and would eat
// the API budget): own ticks via the workflow-scoped endpoint, and
// a repo-wide sweep only past the (12h) wedge threshold.
const selfQueued = await github.paginate(github.rest.actions.listWorkflowRuns, {
owner, repo, workflow_id: 'actions-zombie-janitor.yml', status: 'queued', per_page: 100,
created: createdBefore(cfg.selfQueuedMin),
});
const wedgedQueued = await github.paginate(github.rest.actions.listWorkflowRunsForRepo, {
owner, repo, status: 'queued', per_page: 100,
created: createdBefore(cfg.queuedMaxAgeMin),
});
const queued = new Map();
for (const run of [...selfQueued, ...wedgedQueued]) queued.set(run.id, run);
core.info(`[${cfg.lane}] stale queued listed: ${queued.size} (self ${selfQueued.length}, wedged ${wedgedQueued.length})`);
for (const run of queued.values()) {
if (run.id === context.runId) continue;
const ageMin = minutesSince(run.created_at);
const isSelf = run.name === context.workflow;
if (ageMin < (isSelf ? cfg.selfQueuedMin : cfg.queuedMaxAgeMin)) continue;
if ((await liveStatus(run.id)) !== 'queued') continue;
await reap(run, ageMin, ageMin, isSelf ? 'superseded janitor tick' : 'wedged queued');
}
// ---- 3. production deploy pending/zero-job zombies ----
// These runs never instantiate a job, so no environment approval,
// runner assignment, or job-level concurrency record exists for a
// human to inspect. Scope this to main->production Cloud CF Deploy:
// staging can legitimately sit behind active deploy work, while a
// production run with zero jobs and no active same-branch sibling is
// silent release blockage (#15503/#15499).
const activeCloudDeploy = new Map();
for (const status of ['in_progress', 'waiting']) {
const activeRuns = await github.paginate(github.rest.actions.listWorkflowRuns, {
owner, repo, workflow_id: 'cloud-cf-deploy.yml', status, per_page: 100,
});
for (const run of activeRuns) {
activeCloudDeploy.set(`${run.head_branch || ''}:${status}`, run);
}
}
const pendingCloudDeploy = await github.paginate(github.rest.actions.listWorkflowRuns, {
owner, repo, workflow_id: 'cloud-cf-deploy.yml', status: 'pending', per_page: 100,
created: createdBefore(cfg.pendingZeroJobMin),
});
core.info(`[${cfg.lane}] pending Cloud CF Deploy listed: ${pendingCloudDeploy.length}`);
for (const run of pendingCloudDeploy) {
if (run.id === context.runId) continue;
if (run.head_branch !== 'main') continue;
if (activeCloudDeploy.has('main:in_progress') || activeCloudDeploy.has('main:waiting')) {
core.info(`skip pending prod deploy ${run.id}: active same-branch deploy exists`);
continue;
}
const ageMin = minutesSince(run.created_at);
if (ageMin < cfg.pendingZeroJobMin) continue;
const live = await liveRun(run.id);
if (live?.status !== 'pending') {
core.info(`skip (list-stale, actually '${live?.status ?? 'missing'}'): ${run.id} ${run.name}`);
continue;
}
const jobs = await github.paginate(github.rest.actions.listJobsForWorkflowRun, {
owner, repo, run_id: run.id, filter: 'latest', per_page: 100,
});
if (jobs.length > 0) {
core.info(`skip pending prod deploy ${run.id}: ${jobs.length} job(s) exist`);
continue;
}
await reap(run, ageMin, ageMin, 'production deploy pending with zero jobs');
if (run.actor?.login === 'github-actions[bot]') {
core.warning(`cancelled bot-dispatched pending prod deploy ${run.id}; not redispatching again`);
continue;
}
if (cfg.dryRun) {
core.info(`would redispatch Cloud CF Deploy production at ${run.head_branch}`);
rows.push([`${run.id}`, run.name, `${ageMin}`, `${ageMin}`, 'would redispatch production deploy (dry run)']);
continue;
}
await github.rest.actions.createWorkflowDispatch({
owner, repo, workflow_id: 'cloud-cf-deploy.yml', ref: 'main',
inputs: { environment: 'production', force: 'false' },
});
core.warning(`redispatched Cloud CF Deploy production after cancelling pending/zero-job run ${run.id}`);
rows.push([`${run.id}`, run.name, `${ageMin}`, `${ageMin}`, 'redispatched production deploy once']);
}
// ---- 4. stale waiting runs (unapproved environment gates) ----
// A run `waiting` on an environment approval that never comes
// holds its concurrency group until GitHub's 30-day auto-expiry;
// with cancel-in-progress:false every later run of the group sits
// `pending` with zero jobs and dies silently (#15147: 18-day
// deadlock of both production deploy lanes). 48h is far above any
// sane approval latency — reap the gate so the group frees.
const staleWaiting = await github.paginate(github.rest.actions.listWorkflowRunsForRepo, {
owner, repo, status: 'waiting', per_page: 100,
created: createdBefore(cfg.waitingMaxAgeMin),
});
core.info(`[${cfg.lane}] stale waiting listed: ${staleWaiting.length}`);
for (const run of staleWaiting) {
// Attempt age: a re-run of an old run keeps created_at (which
// the server-side filter uses, as a superset) but resets
// run_started_at — gate on the current attempt.
const ageMin = minutesSince(run.run_started_at || run.created_at);
if (ageMin < cfg.waitingMaxAgeMin) continue;
if ((await liveStatus(run.id)) !== 'waiting') continue;
await reap(run, ageMin, ageMin, 'stale waiting (unapproved environment gate)');
}
// ---- summary ----
core.summary.addHeading(`Zombie janitor [${cfg.lane}] — ${new Date(now).toISOString()}`, 2);
if (rows.length === 0) {
core.summary.addRaw('No zombies found.', true);
} else {
core.summary.addTable([
[
{ data: 'run', header: true },
{ data: 'workflow', header: true },
{ data: 'age (min)', header: true },
{ data: 'idle (min)', header: true },
{ data: 'action', header: true },
],
...rows,
]);
}
await core.summary.write();
+548
View File
@@ -0,0 +1,548 @@
name: android-device-e2e
# Real-device Android e2e: builds the WebView-debuggable APK, boots an emulator,
# and drives the real app via Playwright's Android driver + the on-device agent
# smoke. A weekly host-backed schedule gives the lane unattended cadence without
# burning an emulator on every PR; workflow_dispatch still exposes the full
# backend selector for manual cloud/local investigations.
#
# IMPORTANT coverage note: the embedded on-device agent (bun + llama) runs on
# real arm64 hardware but SIGSEGVs on a stock x86_64 emulator, so the LOCAL
# route's full chat round-trip only goes green on a self-hosted arm64 device
# runner. On a GitHub-hosted x86_64 emulator this lane proves: build + install +
# WebView attach + route navigation, and runs the local smoke as a (currently
# expected-to-surface) signal. For green route coverage on the emulator, supply
# an Eliza Cloud token (ELIZA_CLOUD_AUTH_TOKEN secret) so the app onboards to a
# real cloud agent.
#
# iOS onboarding-smoke parity (#13580): mobile-build-smoke.yml runs
# ios-onboarding-smoke.mjs on every path-gated PR, but the Android
# onboarding->home smoke (test:e2e:android:onboarding) is intentionally NOT
# per-PR. The iOS smoke boots a simulator on a macOS runner cheaply; the
# Android smoke needs a KVM-accelerated x86_64 emulator on a Linux runner,
# which is too heavy to boot on every PR (issue #9943). Android
# onboarding->home instead runs on three cadences: the label-gated
# (ci:device) pr-device-smoke job below, the weekly host-backed schedule,
# and workflow_dispatch -- so an onboarding regression is caught on a PR
# opt-in or within one weekly cycle rather than never. This is the
# documented divergence the parity checkbox in #13580 asks for.
on:
schedule:
# Weekly hosted x86_64 emulator run, backend=host by default below. The
# self-hosted arm64 LOCAL route remains separate because bun+llama SIGSEGVs
# on the stock hosted x86_64 emulator (see coverage note above).
- cron: "17 9 * * 1"
workflow_dispatch:
inputs:
backend:
description: "WebView backend for route coverage (cloud|host|local)"
type: choice
options: [cloud, host, local]
default: cloud
api-level:
description: "Android emulator API level"
default: "34"
# Reachable on normal develop PRs and release-promotion PRs, but only when
# explicitly opted in with the `ci:device` label (issue #9943) — booting an
# emulator is slow/heavy, so we never burn a runner on every PR. The
# pr-device-smoke job below is gated on that label and runs a MINIMAL slice
# (build + install + WebView attach + onboarding→home + one host-backed chat
# turn); the full dispatch matrix above is unchanged.
pull_request:
branches: [develop, main]
types: [opened, synchronize, reopened, labeled]
concurrency:
group: android-device-e2e-${{ github.ref }}
cancel-in-progress: true
jobs:
android-e2e:
# Full lane runs on manual dispatch and on the weekly scheduled host-backed
# cadence. Never on PR; PRs use the label-gated minimal smoke below.
if: github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
permissions:
contents: read
env:
ELIZA_MOBILE_REPO_ROOT: ${{ github.workspace }}
ELIZA_WEBVIEW_DEBUG: "1"
ELIZA_BUN_RISCV64_OPTIONAL: "1"
ELIZA_ANDROID_SKIP_FORK_LLAMA_LIB: "1"
ELIZA_ANDROID_BACKEND: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.backend || 'host' }}
ELIZA_CLOUD_AUTH_TOKEN: ${{ secrets.ELIZA_CLOUD_AUTH_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3.14" # pinned: repo packageManager is bun@1.4.0, an unpublished version (GH+npm 404) that makes bare setup-bun fail here; match .github/ci-bun-version.json (#15071)
- uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "21"
- uses: android-actions/setup-android@v4
- name: Install workspace
run: bun install
- name: Build WebView-debuggable APK
run: bun run --cwd packages/app build:android
# KVM enables a usable emulator on GitHub ubuntu runners.
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \
| sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules && sudo udevadm trigger --name-match=kvm
- name: Route coverage on emulator
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.api-level || '34' }}
arch: x86_64
ram-size: 6144M
# avx2/fma passthrough so the (x86) on-device agent doesn't SIGILL.
emulator-options: -no-window -no-snapshot -no-boot-anim -gpu swiftshader_indirect -qemu -cpu qemu64,+avx,+avx2,+f16c,+fma
script: |
set -euo pipefail
adb root || true
adb shell setenforce 0 || true
bun run --cwd packages/app build # ensure web dist
mkdir -p packages/app/test-results/android-onboarding-to-home
ELIZA_API_PORT=31337 \
ELIZA_PAIRING_DISABLED=1 \
node packages/app-core/scripts/run-node-tsx.mjs \
packages/app-core/scripts/serve-real-local-agent.ts \
> packages/app/test-results/android-onboarding-to-home/host-agent.log 2>&1 &
HOST_AGENT_PID=$!
trap 'kill "$HOST_AGENT_PID" 2>/dev/null || true' EXIT
for i in $(seq 1 90); do
if curl -fsS \
-H 'X-ElizaOS-Client-Id: android-onboarding-ci' \
http://127.0.0.1:31337/api/health >/tmp/android-host-agent-health.json; then
cat /tmp/android-host-agent-health.json
break
fi
if ! kill -0 "$HOST_AGENT_PID" 2>/dev/null; then
echo "Host agent exited before becoming healthy"
cat packages/app/test-results/android-onboarding-to-home/host-agent.log
exit 1
fi
sleep 2
done
curl -fsS \
-H 'X-ElizaOS-Client-Id: android-onboarding-ci' \
http://127.0.0.1:31337/api/health >/dev/null
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:onboarding
if [ "${ELIZA_ANDROID_BACKEND}" = "local" ]; then
# LOCAL route is still signal-only on hosted x86_64: the embedded
# bun+llama agent is known to require a self-hosted arm64 device
# runner. Keep surfacing the artifact, but do not make the hosted
# emulator falsely red for that known platform gap.
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:routes || true
else
# Host/cloud route coverage is the scheduled unattended health
# signal. Fail red here instead of hiding regressions behind a
# permanent signal-only `|| true` leg.
ELIZA_ANDROID_REQUIRE_AGENT=0 \
bun run --cwd packages/app test:e2e:android:routes
fi
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:native-plugin-view
# Local on-device agent smoke (surfaces failure loudly; expected to
# fail on stock x86 emulator — see coverage note above).
bun run --cwd packages/app test:sim:local-chat:android:live || true
# Seeded launcher gesture loop (#12377): ≥200 real device swipes/taps
# with per-action rail invariants (data-page + AX probe + inertness +
# focus), chunked screenrecord + logcat. Signal-only on the x86
# emulator until proven emulator-green; the seed prints for replay.
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:launcher-loop || true
- name: Cloud provisioning probe
run: |
if [ -z "${ELIZA_CLOUD_AUTH_TOKEN:-}" ]; then
{
echo "### Cloud provisioning probe"
echo ""
echo "skipped: missing ELIZA_CLOUD_AUTH_TOKEN"
} >> "$GITHUB_STEP_SUMMARY"
echo "Cloud provisioning probe skipped: missing ELIZA_CLOUD_AUTH_TOKEN"
exit 0
fi
bun run --cwd packages/app test:e2e:android:cloud
- uses: actions/upload-artifact@v7
if: always()
with:
name: android-e2e-artifacts
path: |
packages/app/test-results/**
packages/app/playwright-report/**
if-no-files-found: ignore
notify-scheduled-failure:
needs: android-e2e
if: ${{ always() && github.event_name == 'schedule' && needs.android-e2e.result != 'success' }}
runs-on: ubuntu-24.04
permissions:
contents: read
issues: write
env:
ANDROID_E2E_RESULT: ${{ needs.android-e2e.result }}
ELIZA_ANDROID_BACKEND: host
steps:
- name: Open / update scheduled failure issue
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
with:
script: |
const title = "Scheduled Android device e2e is failing (#13580)";
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const artifactsUrl = `${runUrl}#artifacts`;
const runDate = new Date().toISOString();
const result = process.env.ANDROID_E2E_RESULT || "unknown";
const body = [
"The weekly host-backed Android device e2e cadence failed.",
"",
`Run: ${runUrl}`,
`Artifacts: ${artifactsUrl}`,
`Workflow: \`${context.workflow}\``,
`Android job result: \`${result}\``,
`Backend: \`${process.env.ELIZA_ANDROID_BACKEND || "host"}\``,
`Observed: ${runDate}`,
"",
"This issue is auto-maintained by `.github/workflows/android-device-e2e.yml` so scheduled failures are visible outside Actions history.",
"",
"Known #13580 residuals still tracked separately:",
"- LOCAL arm64 on-device route needs a self-hosted arm64 runner.",
"- Signal-only PR legs should be promoted only after emulator-green evidence exists.",
].join("\n");
const openIssues = await github.paginate(github.rest.issues.listForRepo, {
owner: context.repo.owner,
repo: context.repo.repo,
state: "open",
per_page: 100,
});
const existing = openIssues.find((issue) => !issue.pull_request && issue.title === title);
if (existing) {
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body,
});
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body: `Scheduled Android device e2e failed again (${result}): ${runUrl}`,
});
core.info(`Updated scheduled failure issue #${existing.number}`);
} else {
const { data: created } = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title,
body,
});
core.info(`Opened scheduled failure issue #${created.number}`);
}
# Minimal, label-gated PR device smoke (issue #9943). Reachable on a PR only
# when the `ci:device` label is applied, so it never burns an emulator runner
# on every PR. Runs the proven-green x86 slice: build the WebView-debuggable
# APK, boot the emulator, attach, and drive onboarding→home plus a chat turn
# against the host-run agent (the host backend runs the agent on the runner, so
# it works on a stock x86 emulator where the embedded on-device agent SIGSEGVs).
pr-device-smoke:
if: ${{ github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'ci:device') }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
env:
ELIZA_MOBILE_REPO_ROOT: ${{ github.workspace }}
ELIZA_WEBVIEW_DEBUG: "1"
ELIZA_BUN_RISCV64_OPTIONAL: "1"
ELIZA_ANDROID_SKIP_FORK_LLAMA_LIB: "1"
ELIZA_ANDROID_BACKEND: "host"
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3.14" # pinned: repo packageManager is bun@1.4.0, an unpublished version (GH+npm 404) that makes bare setup-bun fail here; match .github/ci-bun-version.json (#15071)
- uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "21"
- uses: android-actions/setup-android@v4
- name: Install workspace
run: bun install
- name: Build WebView-debuggable APK
run: bun run --cwd packages/app build:android
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \
| sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules && sudo udevadm trigger --name-match=kvm
- name: Onboarding→home + chat turn on emulator (host agent)
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: "34"
arch: x86_64
ram-size: 6144M
emulator-options: -no-window -no-snapshot -no-boot-anim -gpu swiftshader_indirect -qemu -cpu qemu64,+avx,+avx2,+f16c,+fma
script: |
set -euo pipefail
adb root || true
adb shell setenforce 0 || true
bun run --cwd packages/app build # ensure web dist
mkdir -p packages/app/test-results/android-onboarding-to-home
ELIZA_API_PORT=31337 \
ELIZA_PAIRING_DISABLED=1 \
node packages/app-core/scripts/run-node-tsx.mjs \
packages/app-core/scripts/serve-real-local-agent.ts \
> packages/app/test-results/android-onboarding-to-home/host-agent.log 2>&1 &
HOST_AGENT_PID=$!
trap 'kill "$HOST_AGENT_PID" 2>/dev/null || true' EXIT
for i in $(seq 1 90); do
if curl -fsS \
-H 'X-ElizaOS-Client-Id: android-pr-device-smoke' \
http://127.0.0.1:31337/api/health >/tmp/android-host-agent-health.json; then
cat /tmp/android-host-agent-health.json
break
fi
if ! kill -0 "$HOST_AGENT_PID" 2>/dev/null; then
echo "Host agent exited before becoming healthy"
cat packages/app/test-results/android-onboarding-to-home/host-agent.log
exit 1
fi
sleep 2
done
curl -fsS \
-H 'X-ElizaOS-Client-Id: android-pr-device-smoke' \
http://127.0.0.1:31337/api/health >/dev/null
# Onboarding→home + first chat turn, asserted against the host agent.
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:onboarding
# Native plugin x WebView smoke: proves Capacitor JS calls cross
# into Android Kotlin rather than the desktop Chromium bridge shim.
# Signal-only (|| true) on the PR lane until it's proven green on the
# x86_64 API-34 emulator this job uses — the hard assertion runs in
# the workflow_dispatch `android-e2e` job (validated on a real arm64
# device). Do not make it a hard PR gate on unproven-on-emulator
# evidence; flip to a hard gate once an emulator-green run is attached.
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:native-plugin-view || true
# Real OS-level touch swipe on the chat-sheet grabber (#9943): proves
# the home→launcher pager gesture fires with real Android touch input
# (pointerMouseCount===0), not desktop mouse emulation. Signal-only on
# the PR lane for the same reason as above — flip to a hard gate once
# emulator-green evidence is attached.
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
ELIZA_ANDROID_CLEAR_APP_DATA=1 \
bun run --cwd packages/app test:e2e:android:touch-gesture || true
# View-runtime telemetry soak (#10196): activates every registered
# view through the real eliza:navigate:view channel and asserts
# bounded render/heap/cache behavior. Signal-only on the PR lane.
ELIZA_ANDROID_BACKEND=host \
ELIZA_ANDROID_REQUIRE_AGENT=1 \
bun run --cwd packages/app test:e2e:android:view-runtime-soak || true
- uses: actions/upload-artifact@v7
if: always()
with:
name: android-pr-device-smoke-artifacts
path: |
packages/app/test-results/**
packages/app/playwright-report/**
if-no-files-found: ignore
# On-device instrumented (androidTest) gate for the native Capacitor plugins
# (issue #9967 "wire into CI" + issue #9943 "on-device never gates a PR"). Until
# this lane, the native-plugin Kotlin ran on no test, on no device in CI — the
# only coverage was a mocked Capacitor bridge in desktop Chromium. This builds
# every native plugin's androidTest, boots a KVM emulator, and runs the real
# on-device reads (RoleManager / AudioManager / WifiManager / TelecomManager /
# CameraManager / ContactsProvider / content://sms / UsageStatsManager /
# FusedLocationProvider) via `connectedDebugAndroidTest` — gradle fails the job
# on any test failure. Label-gated (`ci:device`) or dispatch, like the smokes
# above, because booting an emulator is heavy.
native-plugin-androidtest:
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'pull_request' &&
contains(github.event.pull_request.labels.*.name, 'ci:device'))
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
env:
ELIZA_BUN_RISCV64_OPTIONAL: "1"
ELIZA_ANDROID_SKIP_FORK_LLAMA_LIB: "1"
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3.14" # pinned: repo packageManager is bun@1.4.0, an unpublished version (GH+npm 404) that makes bare setup-bun fail here; match .github/ci-bun-version.json (#15071)
- uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "21"
- uses: android-actions/setup-android@v4
- name: Install workspace
run: bun install
# Sync the Capacitor Android project so capacitor.settings.gradle points at
# this install's @capacitor/android and every plugin module is wired.
- name: Prepare Android platform
run: bun run --cwd packages/app build:android
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \
| sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules && sudo udevadm trigger --name-match=kvm
- name: Native-plugin androidTest on emulator
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: "34"
arch: x86_64
ram-size: 4096M
emulator-options: -no-window -no-snapshot -no-boot-anim -gpu swiftshader_indirect
script: |
set -euo pipefail
adb wait-for-device
ANDROID_DIR="$GITHUB_WORKSPACE/packages/app-core/platforms/android"
GRADLEW="$ANDROID_DIR/gradlew"
chmod +x "$GRADLEW"
# Device-state orchestration (survives the test-APK install):
# - messages: inject a marker SMS so the SMS read-back asserts positively
# (it Assume-skips without the marker, so this is what makes it green).
# - location: high-accuracy mode (the fused fetch still Assume-skips on a
# headless GNSS HAL; the priority-mapping assertion runs regardless).
adb emu sms send 15558675309 "probe Eliza-9967-SMS-roundtrip ci" || true
adb shell settings put secure location_mode 3 || true
# The gate: build + install + run every native-plugin androidTest on the
# emulator in one invocation. gradle fails the job on any test failure.
# `:app:connectedDebugAndroidTest` adds the app-level assistant/IME/
# assist-surface instrumented tests (#13581) — the RETAIL-path
# ElizaAssistantSurfaceInstrumentedTest runs on the plain debug APK
# (not assumeSystemEliza-gated), so this app run is not vacuously
# green off-AOSP.
"$GRADLEW" -p "$ANDROID_DIR" --no-daemon \
:app:connectedDebugAndroidTest \
:elizaos-capacitor-system:connectedDebugAndroidTest \
:elizaos-capacitor-wifi:connectedDebugAndroidTest \
:elizaos-capacitor-phone:connectedDebugAndroidTest \
:elizaos-capacitor-camera:connectedDebugAndroidTest \
:elizaos-capacitor-contacts:connectedDebugAndroidTest \
:elizaos-capacitor-messages:connectedDebugAndroidTest \
:elizaos-capacitor-mobile-signals:connectedDebugAndroidTest \
:elizaos-capacitor-location:connectedDebugAndroidTest
# mobile-signals PACKAGE_USAGE_STATS is special-access (no runtime dialog)
# and the grant does NOT survive the gate's reinstall, so its usage reads
# Assume-skip above. For a positive on-device read: re-install the test APK
# the gate built (the gate uninstalls it after running), grant
# GET_USAGE_STATS, and re-run the usage tests via am instrument. Require a
# clean `OK (N tests)` completion and fail the job on any error/failure.
MS_APK="$GITHUB_WORKSPACE/plugins/plugin-native-mobile-signals/android/build/outputs/apk/androidTest/debug/elizaos-capacitor-mobile-signals-debug-androidTest.apk"
adb install -r -t "$MS_APK"
adb shell appops set ai.eliza.plugins.mobilesignals.test android:get_usage_stats allow
MS_OUT="$(adb shell am instrument -w -r \
-e class ai.eliza.plugins.mobilesignals.UsageStatsReaderInstrumentedTest \
ai.eliza.plugins.mobilesignals.test/androidx.test.runner.AndroidJUnitRunner 2>&1)"
printf '%s\n' "$MS_OUT"
if printf '%s\n' "$MS_OUT" | grep -qE 'FAILURES!!!|INSTRUMENTATION_FAILED|INSTRUMENTATION_RESULT: shortMsg'; then
echo "mobile-signals UsageStats assertions failed under granted PACKAGE_USAGE_STATS"
exit 1
fi
printf '%s\n' "$MS_OUT" | grep -qE 'OK \([0-9]+ test' || {
echo "mobile-signals UsageStats run did not complete cleanly"
exit 1
}
# Assistant-role / voice-IME / assist-key adb verification lane
# (#13581). The gradle gate above uninstalled its test APKs, so
# install the app APK the Capacitor build produced, then drive the
# RUNTIME surface: re-apply the role + IME that `adb install -r`
# clears, assert the secure settings, fire `cmd voiceinteraction
# show` / `KEYCODE_ASSIST` / the IME deep-link, and assert they land
# in MainActivity. `--require-device` makes a missing device fatal
# (it is present here, so this only guards against a silent skip).
# `--require-engine` is intentionally NOT passed: the emulator carries
# no on-device ASR engine, so the IME ASR round-trip legitimately
# resolves to the designed ENGINE_OFF state — the lane asserts that
# state rather than requiring a transcript. All other assertions
# (registration, role, IME selection, deep-link landing) still gate
# the job.
APP_APK="$GITHUB_WORKSPACE/packages/app-core/platforms/android/app/build/outputs/apk/debug/app-debug.apk"
if [ -f "$APP_APK" ]; then
adb install -r -t "$APP_APK"
else
echo "app-debug.apk not found at $APP_APK — build:android should have produced it"
exit 1
fi
ASSISTANT_ARTIFACT_DIR="$GITHUB_WORKSPACE/packages/app/test-results/android-assistant-verify"
mkdir -p "$ASSISTANT_ARTIFACT_DIR"
VERIFY_STATUS=0
node "$GITHUB_WORKSPACE/packages/app/scripts/android-assistant-verify.mjs" \
--require-device --json \
| tee "$ASSISTANT_ARTIFACT_DIR/verdict.json" || VERIFY_STATUS=$?
adb shell settings get secure voice_interaction_service \
> "$ASSISTANT_ARTIFACT_DIR/voice_interaction_service.txt" || true
adb shell settings get secure default_input_method \
> "$ASSISTANT_ARTIFACT_DIR/default_input_method.txt" || true
adb shell ime list -s \
> "$ASSISTANT_ARTIFACT_DIR/enabled_imes.txt" || true
adb shell cmd role holders android.app.role.ASSISTANT \
> "$ASSISTANT_ARTIFACT_DIR/assistant_role_holders.txt" || true
adb shell dumpsys activity activities \
> "$ASSISTANT_ARTIFACT_DIR/dumpsys-activity.txt" || true
adb logcat -d -v brief \
> "$ASSISTANT_ARTIFACT_DIR/logcat.txt" || true
exit "$VERIFY_STATUS"
- uses: actions/upload-artifact@v7
if: always()
with:
name: native-plugin-androidtest-reports
path: |
plugins/plugin-native-*/android/build/reports/androidTests/**
plugins/plugin-native-*/android/build/outputs/androidTest-results/**
packages/app/test-results/android-assistant-verify/**
if-no-files-found: ignore
+396
View File
@@ -0,0 +1,396 @@
name: Android Build & Publish
on:
workflow_call:
inputs:
track:
description: "Play Store track"
required: false
type: string
default: internal
version_name:
description: "Version name (e.g. 2.0.0-beta.0)"
required: true
type: string
secrets:
# ANDROID_KEYSTORE_* are downgraded to `required: false` so the
# reusable-workflow secret contract does not block startup when
# called from release-all.yml / release-orchestrator.yml with
# `secrets: inherit`. The job body's signing step (gated on
# ANDROID_KEYSTORE_BASE64 being non-empty) skips with a clear
# warning when the secrets are absent, instead of aborting the
# whole release pipeline at workflow startup with no logs. Same
# pattern as publish-apt-repo.yml (PR #7976) and update-homebrew.yml.
ANDROID_KEYSTORE_BASE64:
required: false
ANDROID_KEYSTORE_PASSWORD:
required: false
ANDROID_KEY_ALIAS:
required: false
ANDROID_KEY_PASSWORD:
required: false
PLAY_STORE_SERVICE_ACCOUNT_JSON:
required: false
workflow_dispatch:
inputs:
track:
description: "Play Store track"
type: choice
options:
- internal
- beta
- production
default: internal
version_name:
description: "Version name (e.g. 2.0.0-beta.0)"
required: false
type: string
concurrency:
group: android-release-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: write
env:
JAVA_VERSION: "21"
NODE_VERSION: "24.15.0"
jobs:
build-aab:
name: Build Signed AAB
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
version_name: ${{ steps.version.outputs.name }}
version_code: ${{ steps.version.outputs.code }}
play_store_ready: ${{ steps.play_store_credentials.outputs.ready }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Java
uses: actions/setup-java@ad2b38190b15e4d6bdf0c97fb4fca8412226d287
with:
distribution: "temurin"
java-version: ${{ env.JAVA_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Determine version
id: version
run: |
if [[ -n "${{ inputs.version_name }}" ]]; then
VERSION_NAME="${{ inputs.version_name }}"
else
VERSION_NAME=$(node -p "require('./package.json').version")
fi
# Generate a monotonic Android version code from semver.
# 2.0.0-alpha.82 -> 200001082, 2.0.0-beta.0 -> 200003000,
# 2.0.0 -> 200009000, 2.1.3 -> 201039000.
VERSION_CODE=$(node - "$VERSION_NAME" <<'NODE'
const version = process.argv[2];
const match = version.match(/^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z-]+)(?:\.(\d+))?)?$/);
if (!match) {
console.error(`Invalid semver version: ${version}`);
process.exit(1);
}
const [, majorRaw, minorRaw, patchRaw, prerelease, prereleaseRaw = "0"] =
match;
const major = Number(majorRaw);
const minor = Number(minorRaw);
const patch = Number(patchRaw);
const prereleaseNumber = Number(prereleaseRaw);
if (
!Number.isInteger(major) ||
!Number.isInteger(minor) ||
!Number.isInteger(patch) ||
!Number.isInteger(prereleaseNumber) ||
prereleaseNumber < 0 ||
prereleaseNumber > 999
) {
console.error(`Invalid Android version component in ${version}`);
process.exit(1);
}
const channelOffsets = {
alpha: 1000,
beta: 3000,
rc: 5000,
nightly: 7000,
};
const channelOffset = prerelease
? (channelOffsets[prerelease] ?? 8000)
: 9000;
const code =
major * 100000000 +
minor * 1000000 +
patch * 10000 +
channelOffset +
prereleaseNumber;
if (code > 2100000000) {
console.error(`Android versionCode ${code} exceeds Play Store limit`);
process.exit(1);
}
console.log(code);
NODE
)
echo "name=$VERSION_NAME" >> "$GITHUB_OUTPUT"
echo "code=$VERSION_CODE" >> "$GITHUB_OUTPUT"
echo "Version: $VERSION_NAME (code: $VERSION_CODE)"
- name: Check Play Store credentials
id: play_store_credentials
env:
PLAY_STORE_SERVICE_ACCOUNT_JSON: ${{ secrets.PLAY_STORE_SERVICE_ACCOUNT_JSON }}
run: |
if [[ -n "$PLAY_STORE_SERVICE_ACCOUNT_JSON" ]]; then
echo "ready=true" >> "$GITHUB_OUTPUT"
else
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "PLAY_STORE_SERVICE_ACCOUNT_JSON is not configured; signed APK/AAB artifacts will still attach to GitHub Release, but Play Store upload will be skipped."
fi
- name: Install dependencies
run: bun install
- name: Build workspace web runtime packages
run: |
bun run --cwd packages/core build
bun run --cwd packages/shared build
bun run --cwd packages/vault build
bun run --cwd packages/cloud/sdk build
bun run --cwd packages/app-core build:dist
- name: Prepare Android platform
working-directory: packages/app
run: bun run build:android:cloud
- name: Decode keystore
run: |
echo "${{ secrets.ANDROID_KEYSTORE_BASE64 }}" | base64 -d > /tmp/elizaos-upload.jks
env:
ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }}
- name: Run Android signing preflight
working-directory: packages/app
env:
ELIZAOS_KEYSTORE_PATH: /tmp/elizaos-upload.jks
ELIZAOS_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
ELIZAOS_KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }}
ELIZAOS_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD }}
run: |
bun run preflight:android:sideload
missing=()
[[ ! -s "$ELIZAOS_KEYSTORE_PATH" ]] && missing+=("ANDROID_KEYSTORE_BASE64")
[[ -z "$ELIZAOS_KEYSTORE_PASSWORD" ]] && missing+=("ANDROID_KEYSTORE_PASSWORD")
[[ -z "$ELIZAOS_KEY_ALIAS" ]] && missing+=("ANDROID_KEY_ALIAS")
[[ -z "$ELIZAOS_KEY_PASSWORD" ]] && missing+=("ANDROID_KEY_PASSWORD")
if [[ ${#missing[@]} -gt 0 ]]; then
echo "::error::Missing Android release signing inputs: ${missing[*]}"
exit 1
fi
- name: Run Play Store preflight
if: steps.play_store_credentials.outputs.ready == 'true'
working-directory: packages/app
env:
ELIZAOS_KEYSTORE_PATH: /tmp/elizaos-upload.jks
ELIZAOS_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
ELIZAOS_KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }}
ELIZAOS_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD }}
PLAY_STORE_SERVICE_ACCOUNT_JSON: ${{ secrets.PLAY_STORE_SERVICE_ACCOUNT_JSON }}
run: bun run preflight:android:store
- name: Build signed AAB
working-directory: packages/app-core/platforms/android
env:
ELIZAOS_VERSION_NAME: ${{ steps.version.outputs.name }}
ELIZAOS_VERSION_CODE: ${{ steps.version.outputs.code }}
ELIZAOS_KEYSTORE_PATH: /tmp/elizaos-upload.jks
ELIZAOS_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
ELIZAOS_KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }}
ELIZAOS_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD }}
run: |
chmod +x gradlew
./gradlew -PelizaCloudBuild=true -PelizaStripAgentAssets=true bundleRelease
- name: Build signed APK
working-directory: packages/app-core/platforms/android
env:
ELIZAOS_VERSION_NAME: ${{ steps.version.outputs.name }}
ELIZAOS_VERSION_CODE: ${{ steps.version.outputs.code }}
ELIZAOS_KEYSTORE_PATH: /tmp/elizaos-upload.jks
ELIZAOS_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
ELIZAOS_KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }}
ELIZAOS_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD }}
run: |
chmod +x gradlew
./gradlew -PelizaCloudBuild=true -PelizaStripAgentAssets=true assembleRelease
- name: Verify Android release artifacts
run: |
AAB_PATH="packages/app-core/platforms/android/app/build/outputs/bundle/release/app-release.aab"
APK_PATH="packages/app-core/platforms/android/app/build/outputs/apk/release/app-release.apk"
if [[ ! -f "$AAB_PATH" ]]; then
echo "ERROR: AAB not found at $AAB_PATH"
ls -la packages/app-core/platforms/android/app/build/outputs/bundle/release/ 2>/dev/null || echo "Release dir not found"
exit 1
fi
if [[ ! -f "$APK_PATH" ]]; then
echo "ERROR: APK not found at $APK_PATH"
ls -la packages/app-core/platforms/android/app/build/outputs/apk/release/ 2>/dev/null || echo "Release dir not found"
exit 1
fi
echo "AAB size: $(du -h "$AAB_PATH" | cut -f1)"
echo "APK size: $(du -h "$APK_PATH" | cut -f1)"
file "$AAB_PATH"
file "$APK_PATH"
sha256sum "$AAB_PATH" "$APK_PATH" > packages/app-core/platforms/android/app/build/outputs/release-SHA256SUMS.txt
cat packages/app-core/platforms/android/app/build/outputs/release-SHA256SUMS.txt
- name: Stage sideload APK with canonical name
run: |
VERSION="${{ steps.version.outputs.name }}"
APK_SRC="packages/app-core/platforms/android/app/build/outputs/apk/release/app-release.apk"
SIDELOAD_APK="elizaos-android-${VERSION}-release.apk"
cp "$APK_SRC" "$SIDELOAD_APK"
sha256sum "$SIDELOAD_APK" > "${SIDELOAD_APK}.sha256"
echo "Sideload APK: $SIDELOAD_APK"
echo "SHA256: $(cat "${SIDELOAD_APK}.sha256")"
- name: Upload Android release artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: eliza-android-release
path: |
packages/app-core/platforms/android/app/build/outputs/bundle/release/app-release.aab
packages/app-core/platforms/android/app/build/outputs/apk/release/app-release.apk
packages/app-core/platforms/android/app/build/outputs/release-SHA256SUMS.txt
elizaos-android-${{ steps.version.outputs.name }}-release.apk
elizaos-android-${{ steps.version.outputs.name }}-release.apk.sha256
retention-days: 90
- name: Attach Android artifacts to GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ steps.version.outputs.name }}"
TAG="v${VERSION}"
SIDELOAD_APK="elizaos-android-${VERSION}-release.apk"
if ! gh release view "$TAG" >/dev/null 2>&1; then
echo "::error::No GitHub release found for $TAG; cannot attach AAB."
exit 1
fi
cp packages/app-core/platforms/android/app/build/outputs/bundle/release/app-release.aab \
"Eliza-${VERSION}.aab"
cp packages/app-core/platforms/android/app/build/outputs/apk/release/app-release.apk \
"Eliza-${VERSION}.apk"
cp packages/app-core/platforms/android/app/build/outputs/release-SHA256SUMS.txt \
"Eliza-${VERSION}.android.SHA256SUMS.txt"
gh release upload "$TAG" \
"Eliza-${VERSION}.aab" \
"Eliza-${VERSION}.apk" \
"Eliza-${VERSION}.android.SHA256SUMS.txt" \
"${SIDELOAD_APK}" \
"${SIDELOAD_APK}.sha256" \
--clobber
- name: Clean up keystore
if: always()
run: rm -f /tmp/elizaos-upload.jks
publish-play-store:
name: Publish to Play Store
needs: build-aab
if: needs.build-aab.outputs.play_store_ready == 'true'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Download AAB
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: eliza-android-release
path: aab/
- name: Setup Ruby
uses: ruby/setup-ruby@d45b1a4e94b71acab930e56e79c6aa188764e7f9
with:
ruby-version: "4.0.3"
bundler-cache: true
working-directory: packages/app-core/platforms/android
- name: Decode Play Store service account key
run: |
echo "${{ secrets.PLAY_STORE_SERVICE_ACCOUNT_JSON }}" | base64 -d > /tmp/play-store-key.json
- name: Determine track
id: track
run: |
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
echo "track=${{ inputs.track }}" >> "$GITHUB_OUTPUT"
elif echo "${{ needs.build-aab.outputs.version_name }}" | grep -qE '(beta|rc)'; then
echo "track=internal" >> "$GITHUB_OUTPUT"
else
echo "track=production" >> "$GITHUB_OUTPUT"
fi
- name: Upload to Play Store
working-directory: packages/app-core/platforms/android
env:
PLAY_STORE_JSON_KEY: /tmp/play-store-key.json
run: |
bundle exec fastlane supply \
--aab "$GITHUB_WORKSPACE/aab/app-release.aab" \
--track "${{ steps.track.outputs.track }}" \
--json_key "$PLAY_STORE_JSON_KEY" \
--package_name "app.eliza"
- name: Clean up credentials
if: always()
run: rm -f /tmp/play-store-key.json
summary:
name: Release Summary
needs: [build-aab, publish-play-store]
if: always()
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Summary
run: |
{
echo "## Android Release Summary"
echo ""
echo "**Version:** ${{ needs.build-aab.outputs.version_name }} (code: ${{ needs.build-aab.outputs.version_code }})"
echo ""
echo "| Step | Status |"
echo "|------|--------|"
echo "| Build AAB | ${{ needs.build-aab.result }} |"
echo "| Publish to Play Store | ${{ needs.publish-play-store.result }} |"
} >> "$GITHUB_STEP_SUMMARY"
- name: Require Android release succeeded
run: |
failed=0
if [[ "${{ needs.build-aab.result }}" != "success" ]]; then
echo "::error::Signed AAB build result was '${{ needs.build-aab.result }}', expected success"
failed=1
fi
if [[ "${{ needs.build-aab.outputs.play_store_ready }}" == "true" && "${{ needs.publish-play-store.result }}" != "success" ]]; then
echo "::error::Play Store publish result was '${{ needs.publish-play-store.result }}', expected success"
failed=1
fi
exit "$failed"
+131
View File
@@ -0,0 +1,131 @@
name: App Aesthetic Audit
# Walk EVERY app view (built-in tabs + plugin view bundles) at desktop + mobile
# against the deterministic keyless stub stack, capture rest + hover screenshots,
# run the blank/one-color analyzer + brand-color (no-blue / orange-hover) checks,
# and write per-view verdicts + a contact sheet + report.json.
#
# This is the agent app's equivalent of cloud-frontend's `audit:cloud`. Until
# this workflow existed the audit ran in NO CI lane (honor-system per the PR
# template, #9304). It runs the strict verifier and uploads all artifacts, but
# remains advisory for PR throughput (#14051): the required `ci-ok` gate owns
# merge blocking, while this lane supplies visual evidence for reviewers.
on:
pull_request:
branches: [main, develop]
paths:
- "packages/app/**"
- "packages/ui/**"
- "packages/shared/**"
- "plugins/**"
- ".github/workflows/app-aesthetic-audit.yml"
workflow_dispatch:
concurrency:
group: app-aesthetic-audit-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
jobs:
aesthetic-audit:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
continue-on-error: true
env:
# Keyless: the ui-smoke stub is forced when CI=true (shouldForceStubStack).
ELIZA_UI_SMOKE_FORCE_STUB: "1"
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: .github/scripts/install-playwright-browsers.sh chromium
- name: Run all-views aesthetic audit
# Strict verifier, advisory check (#14051): failures stay visible in
# logs/artifacts without blocking the required ci-ok merge gate.
continue-on-error: true
env:
ELIZA_AUDIT_APP_STRICT: "1"
ELIZA_AUDIT_APP_STRICT_NEEDS_WORK: "1"
run: bun run --cwd packages/app audit:app:capture
- name: Upload aesthetic-audit artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: app-aesthetic-audit-output
path: packages/app/aesthetic-audit-output
retention-days: 14
if-no-files-found: ignore
ocr-content-audit:
needs: aesthetic-audit
runs-on: macos-15
timeout-minutes: 20
continue-on-error: true
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Download aesthetic-audit artifacts
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: app-aesthetic-audit-output
path: packages/app/aesthetic-audit-output
- name: Ensure OCR audit output directory exists
run: mkdir -p packages/app/aesthetic-audit-output
- name: Run pixel OCR content audit
continue-on-error: true
env:
ELIZA_MVP_OCR_ENGINE: packaged
run: bun run --cwd packages/app audit:ocr
- name: Upload OCR audit artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: app-ocr-audit-output
path: packages/app/aesthetic-audit-output/ocr-triage.json
retention-days: 14
if-no-files-found: ignore
+445
View File
@@ -0,0 +1,445 @@
name: App Live E2E
# Gated, NON-blocking live lane for the packages/app e2e suite — the home for
# REAL (not stub/mock) coverage of the dimensions the keyless PR lanes can only
# fixture-test.
#
# Why the keyless lanes can't do this: every PR-gating lane (scenario-pr, ci,
# test) forces the deterministic stub API (CI=true => FORCE_STUB_STACK, see the
# tested shouldForceStubStack helper) and blanks all provider/cloud keys, because
# secrets cannot be exposed to fork PRs and real model/cloud turns cost money and
# add network flake. So real coverage MUST live in a secret-gated, non-PR lane.
#
# The keystone that makes this possible: ELIZA_UI_SMOKE_LIVE_STACK=1 overrides the
# CI stub-force, so the live stack boots the real @elizaos/app-core runtime and
# the live-only specs (test.skip'd in keyless CI) un-skip.
#
# Runs nightly + on-demand; never on pull_request.
on:
schedule:
# 07:00 UTC daily — after the nightly build lanes settle.
- cron: "0 7 * * *"
workflow_dispatch:
inputs:
run_android_local_chat:
description: "Also run the on-device Android local-chat smoke (boots an emulator + builds/installs the APK + stages a GGUF)."
type: boolean
default: false
concurrency:
group: app-live-e2e-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CI: "true"
# The keystone: opt the live stack out of the CI-based stub force so the real
# app-core runtime boots. Without this, CI=true re-forces the stub even with a
# provider key present and every "live" test self-skips.
ELIZA_UI_SMOKE_LIVE_STACK: "1"
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
NODE_NO_WARNINGS: "1"
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
jobs:
# Real LOCAL chat turn through the live stack: a real app-core runtime + a real
# provider model, asserting an exact model-produced marker (APP_LIVE_AGENT_OK).
# UI-driven counterpart to dev-smoke; proves chat send/stream reaches a real
# agent from the UI.
app-live-chat:
name: app live chat (real local runtime)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
env:
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Free disk space for browser smoke
run: |
set -euxo pipefail
df -h /
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
df -h /
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
- name: Generate shared i18n assets
run: bun run --cwd packages/shared build:i18n
# Vite loads package exports before renderer aliases apply; fresh CI
# checkouts need these dist bundles before packages/app can resolve
# @elizaos/core and @elizaos/shared subpaths.
- name: Build @elizaos/core + @elizaos/shared browser bundles
run: bunx turbo run build --filter=@elizaos/core --filter=@elizaos/shared
# Prebuild the renderer dist so the live stack reuses a fresh bundle
# instead of a cold in-harness vite build (viteRendererBuildNeeded check).
- name: Build app renderer bundle
run: bun run --cwd packages/app build:web
- name: Run live agent chat (real model turn)
run: bun run --cwd packages/app test:e2e test/ui-smoke/live-agent-chat.spec.ts --project=chromium -g "live agent"
# Deep settings/vault round-trips that the keyless stub structurally
# cannot serve (write→reload→read-back against the real on-disk vault +
# runtime). LIVE_STACK=1 is set job-wide, so the test.skip(!LIVE_STACK)
# guards in these specs open. The two *-interactions specs also run keyless
# in scenario-pr.yml (skipping their deep blocks); here they run them.
- name: Run settings + vault live-stack deep e2e
run: bun run --cwd packages/app test:e2e test/ui-smoke/vault-routing.spec.ts test/ui-smoke/wallet-keys.spec.ts test/ui-smoke/provider-config.spec.ts test/ui-smoke/vault-modal-interactions.spec.ts test/ui-smoke/settings-sections-interactions.spec.ts --project=chromium
- name: Upload live-chat artifacts
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: app-live-e2e-chat
path: |
packages/app/playwright-report/
packages/app/test-results/
if-no-files-found: ignore
retention-days: 7
# Full continuous walkthrough against the REAL backend agent + model: cold
# launch → in-chat onboarding (greeting → runtime CHOICE → provider → one real
# POST /api/first-run) → all 8 tutorial frames → help/settings/wallet → a real
# chat round-trip (trajectory captured) → view switching → settings edit. Runs
# WITHOUT --skip-review so the AI vision reviewer scores every NN-step.png and
# writes WALKTHROUGH_VERDICTS.md (the PR mock lane is keyless and can't).
walkthrough-live:
name: full walkthrough (real backend + model + vision verdicts)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
env:
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Free disk space for browser smoke
run: |
set -euxo pipefail
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
- name: Install ffmpeg (recording stitch)
run: sudo apt-get update && sudo apt-get install -y ffmpeg
- name: Generate shared i18n assets
run: bun run --cwd packages/shared build:i18n
# Vite resolves linked workspace package exports while loading the app
# config, before renderer aliases can redirect to source files.
- name: Build @elizaos/core + @elizaos/shared browser bundles
run: bunx turbo run build --filter=@elizaos/core --filter=@elizaos/shared
# Prebuild the renderer dist so the live stack reuses a fresh bundle
# instead of a cold in-harness vite build.
- name: Build app renderer bundle
run: bun run --cwd packages/app build:web
- name: Drive + record + vision-review the full walkthrough (live)
run: bun run --cwd packages/app test:e2e:walkthrough:live
- name: Upload walkthrough screenshots + recording + verdicts
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: app-live-e2e-walkthrough
path: |
reports/walkthrough/
e2e-recordings/app/walkthrough/
packages/app/test/ui-smoke/walkthrough/WALKTHROUGH_VERDICTS.md
if-no-files-found: ignore
retention-days: 7
# Real CLOUD login + provisioning + chat through the app UI against real Eliza
# Cloud. cloud-live.spec.ts mocks nothing; ELIZA_UI_SMOKE_CLOUD_LIVE=1 leaves
# first-run open so the spec drives cloud onboarding, and ELIZAOS_CLOUD_API_KEY
# authenticates. The secret-gated job fails before setup when neither accepted
# repository secret is configured; the spec remains safely skippable outside
# this workflow so keyless PR lanes cannot spend Cloud credits.
cloud-live:
name: cloud login + provision + chat (real Eliza Cloud)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
env:
ELIZA_UI_SMOKE_CLOUD_LIVE: "1"
# Prefer the variable named for the runtime contract, but retain the
# repository's established ELIZACLOUD_API_KEY as the canonical fallback.
ELIZAOS_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY || secrets.ELIZACLOUD_API_KEY }}
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Require real Cloud credential
shell: bash
run: |
set -euo pipefail
cloud_key_without_whitespace="${ELIZAOS_CLOUD_API_KEY:-}"
cloud_key_without_whitespace="${cloud_key_without_whitespace//[[:space:]]/}"
if [[ -z "$cloud_key_without_whitespace" ]]; then
echo "::error::App Live E2E requires ELIZAOS_CLOUD_API_KEY or ELIZACLOUD_API_KEY; refusing a green-by-skip Cloud job."
exit 1
fi
unset cloud_key_without_whitespace
- name: Free disk space for browser smoke
run: |
set -euxo pipefail
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
- name: Generate shared i18n assets
run: bun run --cwd packages/shared build:i18n
# Cloud-live still builds the same renderer bundle, so it needs the same
# workspace dists before Vite can load packages/app/vite.config.ts.
- name: Build @elizaos/core + @elizaos/shared browser bundles
run: bunx turbo run build --filter=@elizaos/core --filter=@elizaos/shared
- name: Build app renderer bundle
run: bun run --cwd packages/app build:web
- name: Run real cloud login + provision + chat
run: bun run --cwd packages/app test:e2e test/ui-smoke/cloud-live.spec.ts --project=chromium
- name: Upload cloud-live artifacts
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: app-live-e2e-cloud
path: |
packages/app/playwright-report/
packages/app/test-results/
if-no-files-found: ignore
retention-days: 7
# Real on-device LOCAL provisioning + chat on Android: boots an emulator, builds
# + installs the debuggable APK, stages the smoke GGUF, starts the native
# ElizaAgentService local runtime, and asserts a real streamed on-device model
# reply (test:sim:local-chat:android:live). The ONLY lane that exercises Android
# local provisioning for real. Opt-in (workflow_dispatch input) because it is
# heavy. android-emulator-runner is the de-facto standard emulator action.
android-local-chat:
name: android local chat (on-device GGUF)
if: ${{ github.event_name == 'workflow_dispatch' && inputs.run_android_local_chat }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Enable KVM for the Android emulator
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \
| sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup JDK 21
uses: actions/setup-java@ad2b38190b15e4d6bdf0c97fb4fca8412226d287
with:
distribution: temurin
java-version: "21"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
# android-emulator-runner provides the booted AVD + adb; the script builds
# and installs the APK, then runs the live on-device local-chat smoke, which
# stages the GGUF and asserts a real model reply.
- name: Build, install, and run Android on-device local-chat smoke
uses: reactivecircus/android-emulator-runner@62dbb605bba737720e10b196cb4220d374026a6d
with:
api-level: 34
arch: x86_64
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim
disable-animations: true
script: |
bun run --cwd packages/app build:android
bun run --cwd packages/app install:android:adb
bun run --cwd packages/app test:sim:local-chat:android:live
# Desktop (Electrobun) packaged e2e. The build + agent boot run on a standard
# runner and catch real packaged-app bugs (e.g. a missing JSON import attribute
# that breaks the wallet plugin on boot). The renderer-driving assertions need a
# real GPU display (the app uses WebGPU); on GPU-less runners the build + launch
# still validate, so the renderer step is best-effort.
desktop-packaged:
name: desktop packaged (build + boot + renderer)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
env:
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Install WebKitGTK + xvfb for the Electrobun webview
# libayatana-appindicator3-1: libNativeWrapper.so (tray support) links
# against it; without it every packaged spec dies at launch with
# ERR_DLOPEN_FAILED before the test bridge comes up (the lane was red
# for 5+ days with "Packaged app exited before the desktop test bridge
# became ready").
# scrot/imagemagick/x11-apps/ffmpeg: ScreenCaptureManager tries, in
# order, scrot, ImageMagick import, ffmpeg x11grab, then xwd+ffmpeg for
# /main-window/screenshot. ffmpeg also stitches the walkthrough MP4.
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
xvfb libwebkit2gtk-4.1-0 libgtk-3-0 mesa-utils libgl1-mesa-dri \
libayatana-appindicator3-1 scrot imagemagick x11-apps ffmpeg
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright runner
run: bunx playwright install --with-deps chromium
# Build the packaged desktop app (catches build/packaging breaks). Run from
# the repo root so desktop-build.mjs resolves packages/app.
- name: Build the Electrobun desktop app
run: node packages/app-core/scripts/desktop-build.mjs build
# Launch + drive the packaged app under a virtual display + software GL.
- name: Run packaged desktop e2e (xvfb)
run: xvfb-run -a --server-args="-screen 0 1920x1080x24" bun run --cwd packages/app test:desktop:packaged
- name: Upload desktop-packaged artifacts
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: app-live-e2e-desktop
path: |
packages/app/playwright-report/
packages/app/test-results/
if-no-files-found: ignore
retention-days: 7
# A nightly that is always red is indistinguishable from no lane: without an
# alert, nobody notices the desktop loop has no green unattended cadence and
# regressions accumulate silently (issue #13681). This job turns a red
# scheduled run into an actionable signal by commenting on the tracking issue
# with the per-job results and a link to the run. Scheduled runs only —
# manual workflow_dispatch runs stay quiet to avoid noise.
notify-on-failure:
name: notify on red nightly
needs: [app-live-chat, walkthrough-live, cloud-live, desktop-packaged]
if: ${{ always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure') }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
permissions:
issues: write
steps:
- name: Comment red-nightly diagnostic on tracking issue #13681
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
R_CHAT: ${{ needs.app-live-chat.result }}
R_WALK: ${{ needs.walkthrough-live.result }}
R_CLOUD: ${{ needs.cloud-live.result }}
R_DESKTOP: ${{ needs.desktop-packaged.result }}
run: |
set -euo pipefail
{
printf '## 🌙 Nightly App Live E2E failed (scheduled run)\n\n'
printf 'The only scheduled desktop-packaged lane went red. Per-job results:\n\n'
printf -- '- app live chat: `%s`\n' "$R_CHAT"
printf -- '- full walkthrough: `%s`\n' "$R_WALK"
printf -- '- cloud login+provision+chat: `%s`\n' "$R_CLOUD"
printf -- '- desktop packaged (build+boot+renderer): `%s`\n' "$R_DESKTOP"
printf '\nRun log + failure artifacts: %s\n\n' "$RUN_URL"
printf '_Auto-posted by app-live-e2e.yml so a red nightly notifies someone (Done-when, #13681)._\n'
} > "$RUNNER_TEMP/nightly-red.md"
gh issue comment 13681 --repo "$REPO" --body-file "$RUNNER_TEMP/nightly-red.md"
+91
View File
@@ -0,0 +1,91 @@
name: App Real E2E (browser-driven)
# Gated, NON-blocking lane for the browser-driven REAL app e2e suite that lived
# in no workflow until now (it was dark): test/app/*.{real,live}.e2e.test.ts —
# - qa-checklist.real.e2e.test.ts (onboarding + doc upload + chat +
# settings pages + voice + char switch)
# - memory-relationships.real.e2e.test.ts(real /memories + /relationships views)
# - streaming-visible-text.live.e2e.test.ts (real SSE token streaming)
#
# These drive a real renderer (puppeteer/playwright) against a real app-core
# runtime + a real model provider. Each self-skips (describeIf / CAN_RUN) unless
# ELIZA_LIVE_TEST=1 AND a provider is resolvable, so a key-less run is a clean
# no-op rather than a failure. The default vitest config EXCLUDES these files and
# only scans src/, so they need the dedicated vitest.app-real-e2e.config.ts +
# the test:app-real-e2e script (added alongside this workflow).
#
# Runs nightly + on-demand; never on pull_request.
on:
schedule:
# 09:00 UTC daily — after app-live-e2e (07:00) and voice-live-e2e (08:30).
- cron: "0 9 * * *"
workflow_dispatch: {}
concurrency:
group: app-real-e2e-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CI: "true"
NODE_VERSION: "24"
BUN_VERSION: "canary"
ELIZA_LIVE_TEST: "1"
# Provider keys (the suite picks the first resolvable via selectLiveProvider();
# absent secrets => the suite self-skips, never fails).
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
ELEVENLABS_API_KEY: ${{ secrets.ELEVENLABS_API_KEY }}
jobs:
app-real-e2e:
name: Browser-driven real app e2e (nightly)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
continue-on-error: true
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Install browser drivers
run: |
set -euo pipefail
bunx playwright install --with-deps chromium
- name: Run browser-driven real app e2e
run: |
set -euo pipefail
# streaming-visible-text.live.e2e launches a real browser via
# playwright-core's chromium and honors ELIZA_CHROME_PATH, which
# defaults to a macOS Google Chrome path that never exists on ubuntu —
# so without this the live streaming suite self-skips forever. Point it
# at the chromium the "Install browser drivers" step installed, resolved
# from the same playwright-core the test imports.
ELIZA_CHROME_PATH="$(cd packages/app-core && node -e "console.log(require('playwright-core').chromium.executablePath())")"
test -x "$ELIZA_CHROME_PATH"
echo "Using ELIZA_CHROME_PATH=$ELIZA_CHROME_PATH"
export ELIZA_CHROME_PATH
bun run --cwd packages/app-core test:app-real-e2e
+551
View File
@@ -0,0 +1,551 @@
name: Apple Store Build & Publish
on:
workflow_call:
inputs:
platform:
description: "Platform to build"
required: false
type: string
default: both
track:
description: "Release track"
required: false
type: string
default: testflight
version:
description: "Version override (e.g. 2.0.0-beta.0)"
required: true
type: string
secrets:
# APPLE_ID, APPLE_TEAM_ID, APPLE_APP_SPECIFIC_PASSWORD are
# downgraded to `required: false` so this reusable workflow's
# secret contract does not startup_failure the parent
# (release-all.yml / release-orchestrator.yml) when called via
# `secrets: inherit` and the secrets are absent. The body's
# iOS-signing preflight checks for missing iOS secrets and
# exits with a clear ::error::, so the job still fails loudly
# when it can't sign — just inside the job, not at the parent
# workflow's startup. Same pattern as publish-apt-repo.yml
# (PR #7976), android-release.yml, update-homebrew.yml.
APPLE_ID:
required: false
APPLE_TEAM_ID:
required: false
ITC_TEAM_ID:
required: false
APP_STORE_APP_ID:
required: false
MATCH_PASSWORD:
required: false
MATCH_GIT_URL:
required: false
MATCH_GIT_BASIC_AUTHORIZATION:
required: false
APPLE_APP_SPECIFIC_PASSWORD:
required: false
MAS_CSC_LINK:
required: false
MAS_CSC_KEY_PASSWORD:
required: false
MAS_INSTALLER_CERT:
required: false
MAS_INSTALLER_KEY_PASSWORD:
required: false
APP_STORE_API_KEY_ID:
required: false
APP_STORE_API_ISSUER_ID:
required: false
APP_STORE_API_KEY_P8:
required: false
workflow_dispatch:
inputs:
platform:
description: "Platform to build"
type: choice
options:
- both
- ios
- macos
default: both
track:
description: "Release track"
type: choice
options:
- testflight
- app-store
default: testflight
version:
description: "Version override (e.g. 2.0.0-beta.0)"
required: false
type: string
concurrency:
group: apple-release-${{ github.ref }}
cancel-in-progress: false
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
LANG: en_US.UTF-8
LC_ALL: en_US.UTF-8
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
prepare:
name: Prepare
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
version: ${{ steps.ver.outputs.version }}
build_number: ${{ steps.ver.outputs.build_number }}
build_ios: ${{ steps.targets.outputs.ios }}
build_macos: ${{ steps.targets.outputs.macos }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Determine version
id: ver
run: |
if [[ -n "${{ inputs.version }}" ]]; then
VERSION="${{ inputs.version }}"
else
VERSION=$(node -p "require('./package.json').version")
fi
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
echo "build_number=$(date +%s)" >> "$GITHUB_OUTPUT"
echo "Version: $VERSION"
- name: Determine build targets
id: targets
run: |
PLATFORM="${{ inputs.platform || 'both' }}"
if [[ "$PLATFORM" == "both" || "$PLATFORM" == "ios" ]]; then
echo "ios=true" >> "$GITHUB_OUTPUT"
else
echo "ios=false" >> "$GITHUB_OUTPUT"
fi
if [[ "$PLATFORM" == "both" || "$PLATFORM" == "macos" ]]; then
echo "macos=true" >> "$GITHUB_OUTPUT"
else
echo "macos=false" >> "$GITHUB_OUTPUT"
fi
# ── iOS App Store ────────────────────────────────────────────────────
build-ios:
name: Build & Submit iOS
needs: prepare
if: needs.prepare.outputs.build_ios == 'true'
runs-on: macos-15
# Embedding the no-JIT Bun engine + CocoaPods adds build time over a thin
# cloud-only client, so allow more headroom than the bare-IPA budget.
timeout-minutes: 90
env:
# Make this split build (web + cap sync + ios-overlay + fastlane) behave
# like buildIos()'s App Store target, which bundles a working on-device
# agent. These env vars are normally set by configureIosAppStoreBuildDefaults
# / buildWeb("ios"); the workflow must set them itself because it bypasses
# buildIos(). Without them the IPA ships as a cloud-only thin client whose
# "start local agent" path hard-fails ("the JSContext compatibility
# transport is disabled outside iOS development builds").
# - ELIZA_BUILD_VARIANT/RELEASE_AUTHORITY: bake __ELIZA_BUILD_VARIANT__
# ="store" (store CSP + isNativeIosStoreBuild()) and embed the engine.
# - ELIZA_IOS_FULL_BUN_ENGINE=1: force the engine pod into the IPA.
# - VITE_ELIZA_IOS_FULL_BUN_AVAILABLE=1: tell the renderer the engine is
# present (isFullBunRuntimeBuiltIn) so the transport actually uses it.
# - VITE_ELIZA_IOS_RUNTIME_MODE=cloud-hybrid: App Store default mode; the
# user switching to "local" persists to localStorage and wins.
ELIZA_BUILD_VARIANT: store
ELIZA_RELEASE_AUTHORITY: apple-app-store
ELIZA_IOS_FULL_BUN_ENGINE: "1"
VITE_ELIZA_IOS_FULL_BUN_AVAILABLE: "1"
VITE_ELIZA_IOS_RUNTIME_MODE: cloud-hybrid
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
ITC_TEAM_ID: ${{ secrets.ITC_TEAM_ID }}
APP_STORE_APP_ID: ${{ secrets.APP_STORE_APP_ID }}
MATCH_PASSWORD: ${{ secrets.MATCH_PASSWORD }}
MATCH_GIT_URL: ${{ secrets.MATCH_GIT_URL }}
MATCH_GIT_BASIC_AUTHORIZATION: ${{ secrets.MATCH_GIT_BASIC_AUTHORIZATION }}
FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD: ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --ignore-scripts || bun install --ignore-scripts
- name: Materialize bun npm package binary
run: if [ -f node_modules/bun/install.js ]; then (cd node_modules/bun && node install.js); fi
- name: Set version
env:
RELEASE_VERSION: ${{ needs.prepare.outputs.version }}
run: |
node -e "
const fs = require('fs');
const v = process.env.RELEASE_VERSION;
for (const f of ['package.json', 'packages/app/package.json']) {
const pkg = JSON.parse(fs.readFileSync(f, 'utf8'));
pkg.version = v;
fs.writeFileSync(f, JSON.stringify(pkg, null, 2) + '\n');
}
"
- name: Build web assets
working-directory: packages/app
run: bun run build
- name: Validate iOS secrets
run: |
# Secrets required for signing the IPA. Without these the build
# itself cannot produce a signed artifact.
missing=()
[[ -z "$MATCH_GIT_URL" ]] && missing+=("MATCH_GIT_URL")
[[ -z "$MATCH_PASSWORD" ]] && missing+=("MATCH_PASSWORD")
[[ -z "$MATCH_GIT_BASIC_AUTHORIZATION" ]] && missing+=("MATCH_GIT_BASIC_AUTHORIZATION")
[[ -z "$ITC_TEAM_ID" ]] && missing+=("ITC_TEAM_ID")
if [[ ${#missing[@]} -gt 0 ]]; then
echo "::error::Missing required iOS signing secrets: ${missing[*]}"
echo "Configure these secrets in the repo settings to enable iOS App Store builds."
exit 1
fi
if [[ -z "$APP_STORE_APP_ID" ]]; then
echo "::error::APP_STORE_APP_ID is required for TestFlight/App Store delivery."
exit 1
fi
- name: Prepare CocoaPods trunk repo
run: bash packages/app-core/scripts/prepare-ios-cocoapods.sh
env:
LANG: en_US.UTF-8
- name: Sync Capacitor iOS
working-directory: packages/app
run: bun run cap:sync:ios
env:
LANG: en_US.UTF-8
# Build the on-device agent bundle the embedded Bun engine executes.
# buildIos() does this via buildMobileAgentBundle(); this split workflow
# must run it explicitly so the ios-overlay step below can stage
# packages/agent/dist-mobile-ios into the app's public/agent.
- name: Build iOS agent bundle
run: bun run --cwd packages/agent build:ios-bun
- name: Overlay iOS native files (Fastlane, AppDelegate, entitlements)
run: node packages/app-core/scripts/run-mobile-build.mjs ios-overlay
env:
LANG: en_US.UTF-8
- name: Update Xcode project version
working-directory: packages/app/ios/App/App.xcodeproj
run: |
VERSION="${{ needs.prepare.outputs.version }}"
BUILD="${{ needs.prepare.outputs.build_number }}"
sed -i '' "s/MARKETING_VERSION = .*/MARKETING_VERSION = $VERSION;/" project.pbxproj
sed -i '' "s/CURRENT_PROJECT_VERSION = .*/CURRENT_PROJECT_VERSION = $BUILD;/" project.pbxproj
- name: Install CocoaPods
working-directory: packages/app/ios/App
run: pod install
- name: Setup Ruby
uses: ruby/setup-ruby@d45b1a4e94b71acab930e56e79c6aa188764e7f9
with:
ruby-version: "4.0.3"
bundler-cache: true
working-directory: packages/app/ios
- name: Resolve app identifiers
working-directory: packages/app
run: |
# Read the runtime bundle id straight from pbxproj (the overlay step
# has already rewritten it to the elizaOS-specific id).
APP_ID=$(grep -m1 -E 'PRODUCT_BUNDLE_IDENTIFIER = [^.]+\.' ios/App/App.xcodeproj/project.pbxproj \
| grep -v WebsiteBlockerContentExtension \
| awk '{print $3}' | tr -d ';')
if [[ -z "$APP_ID" ]]; then
echo "::error::Could not resolve iOS bundle id from pbxproj"
exit 1
fi
APP_ID_EXTRA="${APP_ID}.WebsiteBlockerContentExtension"
echo "APP_IDENTIFIER=$APP_ID" >> "$GITHUB_ENV"
echo "APP_IDENTIFIER_EXTRA=$APP_ID_EXTRA" >> "$GITHUB_ENV"
echo "Resolved: APP_IDENTIFIER=$APP_ID, APP_IDENTIFIER_EXTRA=$APP_ID_EXTRA"
- name: Run iOS store preflight
working-directory: packages/app
run: bun run preflight:ios:store
- name: Build and upload
working-directory: packages/app/ios
env:
APP_IDENTIFIER: ${{ env.APP_IDENTIFIER }}
APP_IDENTIFIER_EXTRA: ${{ env.APP_IDENTIFIER_EXTRA }}
run: |
if [[ ! -f fastlane/Fastfile ]]; then
echo "::error::fastlane/Fastfile missing at packages/app/ios/. cap:sync:ios or run-mobile-build.mjs should have copied it."
ls -la fastlane/ || true
exit 1
fi
TRACK="${{ inputs.track || 'testflight' }}"
if [[ "$TRACK" == "app-store" ]]; then
bundle exec fastlane release
else
bundle exec fastlane beta
fi
- name: Upload IPA artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-ipa
path: packages/app/ios/build/Eliza.ipa
retention-days: 90
if-no-files-found: error
# ── Mac App Store ────────────────────────────────────────────────────
build-macos:
name: Build & Submit macOS
needs: prepare
if: needs.prepare.outputs.build_macos == 'true'
runs-on: macos-15
timeout-minutes: 75
env:
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
APPLE_APP_SPECIFIC_PASSWORD: ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }}
CSC_LINK: ${{ secrets.MAS_CSC_LINK }}
CSC_KEY_PASSWORD: ${{ secrets.MAS_CSC_KEY_PASSWORD }}
MAS_INSTALLER_CERT: ${{ secrets.MAS_INSTALLER_CERT }}
MAS_INSTALLER_KEY_PASSWORD: ${{ secrets.MAS_INSTALLER_KEY_PASSWORD }}
APP_STORE_API_KEY_ID: ${{ secrets.APP_STORE_API_KEY_ID }}
APP_STORE_API_ISSUER_ID: ${{ secrets.APP_STORE_API_ISSUER_ID }}
APP_STORE_API_KEY_P8: ${{ secrets.APP_STORE_API_KEY_P8 }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --ignore-scripts || bun install --ignore-scripts
- name: Materialize bun npm package binary
run: if [ -f node_modules/bun/install.js ]; then (cd node_modules/bun && node install.js); fi
- name: Set version
env:
RELEASE_VERSION: ${{ needs.prepare.outputs.version }}
run: |
node -e "
const fs = require('fs');
const v = process.env.RELEASE_VERSION;
for (const f of ['package.json', 'packages/app/package.json', 'packages/app-core/platforms/electrobun/package.json']) {
try {
const pkg = JSON.parse(fs.readFileSync(f, 'utf8'));
pkg.version = v;
fs.writeFileSync(f, JSON.stringify(pkg, null, 2) + '\n');
} catch(e) {}
}
const cfgPath = 'packages/app-core/platforms/electrobun/electrobun.config.ts';
let cfg = fs.readFileSync(cfgPath, 'utf8');
cfg = cfg.replace(/version:\s*\"[^\"]+\"/, 'version: \"' + v + '\"');
fs.writeFileSync(cfgPath, cfg);
"
- name: Build core dist
run: |
bunx tsdown
echo '{"type":"module"}' > dist/package.json
node --import tsx packages/scripts/write-build-info.ts
- name: Build renderer
working-directory: packages/app
run: |
bun install --ignore-scripts
if [ -f node_modules/bun/install.js ]; then (cd node_modules/bun && node install.js); fi
bun run build:web
- name: Stage renderer
run: |
node packages/scripts/rm-path-recursive.mjs packages/app-core/platforms/electrobun/renderer
cp -r packages/app/dist packages/app-core/platforms/electrobun/renderer
- name: Build native macOS effects
working-directory: packages/app-core/platforms/electrobun
run: bun run build:native-effects
- name: Build webview preload
working-directory: packages/app-core/platforms/electrobun
run: bun run build:preload
- name: Validate signing secrets
run: |
missing=()
[[ -z "$CSC_LINK" ]] && missing+=("MAS_CSC_LINK")
[[ -z "$CSC_KEY_PASSWORD" ]] && missing+=("MAS_CSC_KEY_PASSWORD")
[[ -z "$MAS_INSTALLER_CERT" ]] && missing+=("MAS_INSTALLER_CERT")
[[ -z "$MAS_INSTALLER_KEY_PASSWORD" ]] && missing+=("MAS_INSTALLER_KEY_PASSWORD")
[[ -z "$APP_STORE_API_KEY_ID" ]] && missing+=("APP_STORE_API_KEY_ID")
[[ -z "$APP_STORE_API_ISSUER_ID" ]] && missing+=("APP_STORE_API_ISSUER_ID")
[[ -z "$APP_STORE_API_KEY_P8" ]] && missing+=("APP_STORE_API_KEY_P8")
if [[ ${#missing[@]} -gt 0 ]]; then
echo "::error::Missing required signing secrets: ${missing[*]}"
echo "Configure these secrets in the repo settings to enable Mac App Store builds."
exit 1
fi
- name: Import signing certificates
run: |
KEYCHAIN="build-mas.keychain"
KEYCHAIN_PASS="$(openssl rand -hex 16)"
security create-keychain -p "$KEYCHAIN_PASS" "$KEYCHAIN"
security default-keychain -s "$KEYCHAIN"
security unlock-keychain -p "$KEYCHAIN_PASS" "$KEYCHAIN"
security set-keychain-settings -t 3600 "$KEYCHAIN"
echo "$CSC_LINK" | base64 -d > /tmp/mas-app.p12
security import /tmp/mas-app.p12 -k "$KEYCHAIN" -P "$CSC_KEY_PASSWORD" \
-T /usr/bin/codesign -T /usr/bin/productbuild
rm -f /tmp/mas-app.p12
echo "$MAS_INSTALLER_CERT" | base64 -d > /tmp/mas-installer.p12
security import /tmp/mas-installer.p12 -k "$KEYCHAIN" -P "$MAS_INSTALLER_KEY_PASSWORD" \
-T /usr/bin/productbuild
rm -f /tmp/mas-installer.p12
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KEYCHAIN_PASS" "$KEYCHAIN"
MAS_IDENTITY=$(security find-identity -v -p codesigning "$KEYCHAIN" | grep "Apple Distribution" | head -1 | awk -F'"' '{print $2}')
INSTALLER_IDENTITY=$(security find-identity -v "$KEYCHAIN" | grep "3rd Party Mac Developer Installer" | head -1 | awk -F'"' '{print $2}')
echo "MAS_IDENTITY=$MAS_IDENTITY" >> "$GITHUB_ENV"
echo "INSTALLER_IDENTITY=$INSTALLER_IDENTITY" >> "$GITHUB_ENV"
- name: Build Electrobun app
working-directory: packages/app-core/platforms/electrobun
run: ELECTROBUN_SKIP_CODESIGN=1 bun run build
env:
ELECTROBUN_BUILD_ENV: "stable"
- name: Re-sign for Mac App Store
working-directory: packages/app-core/platforms/electrobun
run: |
APP_PATH=$(find build -name "*.app" -maxdepth 2 | head -1)
ENTITLEMENTS="${GITHUB_WORKSPACE}/packages/app-core/platforms/electrobun/entitlements/mas.entitlements"
CHILD_ENTITLEMENTS="${GITHUB_WORKSPACE}/packages/app-core/platforms/electrobun/entitlements/mas-child.entitlements"
# Sign frameworks and dylibs with child entitlements
find "$APP_PATH" -name "*.dylib" -o -name "*.framework" | while read -r lib; do
codesign --force --options runtime --sign "$MAS_IDENTITY" \
--entitlements "$CHILD_ENTITLEMENTS" "$lib"
done
# Sign helper apps
find "$APP_PATH/Contents/Frameworks" -name "*.app" 2>/dev/null | while read -r helper; do
codesign --force --deep --options runtime --sign "$MAS_IDENTITY" \
--entitlements "$CHILD_ENTITLEMENTS" "$helper"
done
# Sign main app bundle
codesign --force --deep --options runtime --sign "$MAS_IDENTITY" \
--entitlements "$ENTITLEMENTS" "$APP_PATH"
codesign --verify --deep --strict "$APP_PATH"
- name: Create installer pkg
working-directory: packages/app-core/platforms/electrobun
run: |
APP_PATH=$(find build -name "*.app" -maxdepth 2 | head -1)
VERSION="${{ needs.prepare.outputs.version }}"
PKG_NAME="Eliza-${VERSION}-mas.pkg"
productbuild --sign "$INSTALLER_IDENTITY" \
--component "$APP_PATH" /Applications \
"build/$PKG_NAME"
echo "PKG_PATH=packages/app-core/platforms/electrobun/build/$PKG_NAME" >> "$GITHUB_ENV"
- name: Upload to App Store Connect
run: |
# xcrun altool requires the .p8 private key at the canonical path:
# ~/.appstoreconnect/private_keys/AuthKey_<KEY_ID>.p8
# The key content is provided via the APP_STORE_API_KEY_P8 secret.
KEY_DIR="$HOME/.appstoreconnect/private_keys"
mkdir -p "$KEY_DIR"
KEY_FILE="$KEY_DIR/AuthKey_${{ secrets.APP_STORE_API_KEY_ID }}.p8"
printf '%s' "$APP_STORE_API_KEY_P8" > "$KEY_FILE"
chmod 600 "$KEY_FILE"
xcrun altool --upload-app \
--file "$PKG_PATH" \
--type macos \
--apiKey "${{ secrets.APP_STORE_API_KEY_ID }}" \
--apiIssuer "${{ secrets.APP_STORE_API_ISSUER_ID }}" \
--show-progress
rm -f "$KEY_FILE"
- name: Upload pkg artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: macos-mas-pkg
path: ${{ env.PKG_PATH }}
retention-days: 90
if-no-files-found: error
# ── Summary ──────────────────────────────────────────────────────────
summary:
name: Release Summary
needs: [prepare, build-ios, build-macos]
if: always()
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Summary
run: |
{
echo "## Apple Store Release Summary"
echo ""
echo "**Version:** ${{ needs.prepare.outputs.version }}"
echo "**Build:** ${{ needs.prepare.outputs.build_number }}"
echo ""
echo "| Platform | Status |"
echo "|----------|--------|"
echo "| iOS App Store | ${{ needs.build-ios.result || 'skipped' }} |"
echo "| Mac App Store | ${{ needs.build-macos.result || 'skipped' }} |"
} >> "$GITHUB_STEP_SUMMARY"
- name: Require enabled Apple releases succeeded
run: |
failed=0
if [[ "${{ needs.prepare.outputs.build_ios }}" == "true" && "${{ needs.build-ios.result }}" != "success" ]]; then
echo "::error::iOS release result was '${{ needs.build-ios.result }}', expected success"
failed=1
fi
if [[ "${{ needs.prepare.outputs.build_macos }}" == "true" && "${{ needs.build-macos.result }}" != "success" ]]; then
echo "::error::Mac App Store release result was '${{ needs.build-macos.result }}', expected success"
failed=1
fi
exit "$failed"
+89
View File
@@ -0,0 +1,89 @@
name: Apps Deploy E2E (Product 2)
# Hermetic, real-Docker end-to-end proof of the Apps (Product 2) deploy chain:
# build user image (real AppImageBuilder) -> provision a per-tenant Postgres DB
# -> run the image as an --internal isolated container with its per-tenant DSN
# -> it reaches ITS OWN DB and serves -> it is REJECTED from another tenant's DB
# (REVOKE CONNECT) -> it has NO internet egress.
#
# Self-contained: needs only docker + bun on the runner (no staging, no secrets,
# no VPS). Wires the previously-unwired verify scripts in
# packages/cloud/shared/scripts/ into CI so the deploy + per-tenant-DB + isolation
# mechanics are GATED, not manually run on a box — matching the "everything is
# CI/CD, no manual" model. Dispatchable on demand and gating on apps-deploy code.
on:
workflow_dispatch:
pull_request:
branches: [main]
paths:
- "packages/cloud/shared/src/lib/services/app-deploy-runner.ts"
- "packages/cloud/shared/src/lib/services/app-container-provider.ts"
- "packages/cloud/shared/src/lib/services/app-db-ambassador.ts"
- "packages/cloud/shared/src/lib/services/app-image-builder.ts"
- "packages/cloud/shared/src/lib/services/app-image-resolver.ts"
- "packages/cloud/shared/src/lib/services/container-executor-deps.ts"
- "packages/cloud/shared/src/lib/services/tenant-db/**"
- "packages/cloud/shared/scripts/verify-e2e-deploy.sh"
- "packages/cloud/shared/scripts/verify-e2e-container-db.sh"
- "packages/cloud/shared/scripts/verify-tenant-db-isolation.ts"
- "packages/cloud/shared/scripts/_e2e-build.ts"
- "packages/cloud/shared/scripts/e2e-sample-app/**"
- ".github/workflows/apps-deploy-e2e.yml"
permissions:
contents: read
concurrency:
group: apps-deploy-e2e-${{ github.ref }}
cancel-in-progress: true
jobs:
apps-deploy-e2e:
name: Apps deploy + per-tenant DB isolation (real docker)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 25
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- uses: ./.github/actions/cloud-setup-test-env
- name: Build cloud-shared (drizzle artifacts)
run: bun run --cwd packages/cloud/shared build || true
- name: E2E — build image → tenant DB → isolated container serves its own DB, blocked from others + egress
working-directory: packages/cloud/shared
run: bash scripts/verify-e2e-deploy.sh
- name: E2E — real-docker tenant-DB isolation (REVOKE CONNECT + --internal no-egress)
working-directory: packages/cloud/shared
run: bash scripts/verify-e2e-container-db.sh
tenant-db-isolation:
name: Tenant-DB isolation — cross-tenant REVOKE CONNECT (throwaway Postgres) (#9853)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
# Lightweight pg-only proof (no docker): provision two tenants through the
# real SqlTenantDbProvisioner against a throwaway Postgres and assert tenant
# A cannot connect to tenant B's database. Wires the previously-orphaned
# verify-tenant-db-isolation.ts into CI.
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_PASSWORD: adminpw
ports:
- 55432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 20
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- uses: ./.github/actions/cloud-setup-test-env
- name: Verify cross-tenant REVOKE CONNECT on per-tenant databases
working-directory: packages/cloud/shared
run: bun run verify:tenant-isolation
+218
View File
@@ -0,0 +1,218 @@
name: Arm Apps Daemon (Product 2)
# Arm the apps (Product 2) deploy backend on a provisioning-worker control plane
# WITHOUT hand-editing the box — the IaC counterpart to hand-SSH'ing in.
#
# The daemon already calls configureAppsDeployBackend() on boot, but it only
# provisions app containers once the apps env is present in
# /opt/eliza/cloud/.env.local. This workflow UPSERTS exactly that block (each
# key delete-then-append, every other line untouched — never clobbers the
# agent/coding fleet config) and restarts the daemon, over the SAME
# ELIZA_PROVISIONING_SSH_KEY/HOST secrets the deploy workflow already uses. So
# no operator SSH key needs to be added anywhere; dispatch it and it converges.
#
# Mirrors packages/scripts/cloud/admin/arm-apps-daemon.mjs (same upsert logic),
# so a local run and a CI run produce the same box state.
#
# Inputs:
# environment staging | production (selects the env's PROVISIONING secrets)
# app_node CONTAINERS_DOCKER_NODES value, e.g. apps-node-1:167.233.112.155:20
# tenant_admin_dsn (optional) env-sourced tenant DB admin DSN. Leave empty to
# let the daemon use the SEEDED tenant_db_clusters row
# (encrypted path). Set it only if the encrypted path isn't
# wired on this box.
# egress_proxy (optional) CONTAINERS_EGRESS_PROXY_URL
#
# Non-secret base domain comes from the env var APPS_BASE_DOMAIN (same var the
# terraform workflow reads); the Caddy admin URL is derived from the app node IP.
on:
workflow_dispatch:
inputs:
environment:
description: Environment to target
type: choice
default: staging
options: [staging, production]
target:
description: "Which daemon to arm: the shared control-plane provisioning-worker (cp-daemon) or the dedicated apps-control worker (apps-worker, Product-2 isolated path)"
type: choice
default: cp-daemon
options: [cp-daemon, apps-worker]
app_node:
description: "CONTAINERS_DOCKER_NODES (id:ip:capacity), e.g. apps-node-1:167.233.112.155:20"
type: string
required: true
tenant_admin_dsn:
description: "Optional env-sourced tenant DB admin DSN (leave empty to use the seeded encrypted cluster OR fetch_dsn_from_shared_state)"
type: string
required: false
default: ""
fetch_dsn_from_shared_state:
description: "Read APPS_TENANT_ADMIN_DSN from the apps-shared terraform output instead of pasting it (keeps the password out of inputs/logs)"
type: boolean
required: false
default: false
egress_proxy:
description: "Optional CONTAINERS_EGRESS_PROXY_URL"
type: string
required: false
default: ""
permissions:
contents: read
concurrency:
# Key on target too: the cp-daemon and apps-worker arms touch different hosts,
# so serializing them against each other (env-only key) needlessly blocks
# arming the two independent daemons concurrently.
group: arm-apps-daemon-${{ github.event.inputs.environment }}-${{ github.event.inputs.target }}
cancel-in-progress: false
jobs:
arm:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ${{ github.event.inputs.environment }}
timeout-minutes: 10
env:
# Target the dedicated apps-control host (apps-worker) or the shared
# control-plane host (cp-daemon). The apps-worker path arms the isolated
# Product-2 daemon on a host that sits on the apps private net (so it can
# reach the tenant DB) and runs no agent jobs — never touching the agent
# control plane.
#
# NO cross-target fallback. The naive `apps-worker && APPS || PROVISIONING`
# form silently resolves to the CP host when ELIZA_APPS_WORKER_HOST is
# unset — which would SSH the apps env onto the agent control-plane host
# (Preflight can't catch it because DEPLOY_HOST is then non-empty). The
# nested form below only ever yields the SAME-target secret or '' (the
# trailing `|| ''` forces empty, not the boolean `false`), so a missing
# same-target secret leaves DEPLOY_HOST empty and the Preflight
# `[ -n "$DEPLOY_HOST" ]` check reports the right missing-secret error.
DEPLOY_HOST: ${{ (github.event.inputs.target == 'apps-worker' && secrets.ELIZA_APPS_WORKER_HOST) || (github.event.inputs.target == 'cp-daemon' && secrets.ELIZA_PROVISIONING_HOST) || '' }}
DEPLOY_SSH_KEY: ${{ (github.event.inputs.target == 'apps-worker' && secrets.ELIZA_APPS_WORKER_SSH_KEY) || (github.event.inputs.target == 'cp-daemon' && secrets.ELIZA_PROVISIONING_SSH_KEY) || '' }}
CONTAINERS_SSH_KEY_INPUT: ${{ secrets.CONTAINERS_SSH_KEY || (github.event.inputs.target == 'apps-worker' && secrets.ELIZA_APPS_WORKER_SSH_KEY) || (github.event.inputs.target == 'cp-daemon' && secrets.ELIZA_PROVISIONING_SSH_KEY) || '' }}
SYSTEMD_UNIT: ${{ github.event.inputs.target == 'apps-worker' && 'eliza-apps-worker.service' || 'eliza-provisioning-worker.service' }}
BASE_DOMAIN: ${{ vars.APPS_BASE_DOMAIN }}
APP_NODE: ${{ github.event.inputs.app_node }}
TENANT_ADMIN_DSN: ${{ github.event.inputs.tenant_admin_dsn }}
EGRESS_PROXY: ${{ github.event.inputs.egress_proxy }}
steps:
- name: Preflight
run: |
fail=0
HOST_SECRET="${{ github.event.inputs.target == 'apps-worker' && 'ELIZA_APPS_WORKER_HOST' || 'ELIZA_PROVISIONING_HOST' }}"
KEY_SECRET="${{ github.event.inputs.target == 'apps-worker' && 'ELIZA_APPS_WORKER_SSH_KEY' || 'ELIZA_PROVISIONING_SSH_KEY' }}"
[ -n "$DEPLOY_HOST" ] || { echo "::error::missing secret $HOST_SECRET on the '${{ github.event.inputs.environment }}' environment (target=${{ github.event.inputs.target }})"; fail=1; }
[ -n "$DEPLOY_SSH_KEY" ] || { echo "::error::missing secret $KEY_SECRET"; fail=1; }
[ -n "$CONTAINERS_SSH_KEY_INPUT" ] || { echo "::error::missing container SSH key material: set secret CONTAINERS_SSH_KEY or $KEY_SECRET"; fail=1; }
[ -n "$BASE_DOMAIN" ] || { echo "::error::set vars.APPS_BASE_DOMAIN on the '${{ github.event.inputs.environment }}' environment"; fail=1; }
[ -n "$APP_NODE" ] || { echo "::error::app_node input is required (id:ip:capacity)"; fail=1; }
# Derive the Caddy admin URL from the app node IP (2nd colon field).
NODE_IP="$(echo "$APP_NODE" | cut -d: -f2)"
echo "$NODE_IP" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$' || { echo "::error::app_node must be id:ip:capacity with a valid IPv4"; fail=1; }
echo "CADDY_ADMIN=http://$NODE_IP:2019" >> "$GITHUB_ENV"
[ "$fail" = 0 ] || exit 1
- name: Prepare container SSH key
run: |
set -euo pipefail
tmp="$(mktemp)"
cleanup() { rm -f "$tmp"; }
trap cleanup EXIT
# docker-ssh.ts consumes CONTAINERS_SSH_KEY as base64. Accept either
# a dedicated already-base64 CONTAINERS_SSH_KEY secret or the raw
# target deploy key used by appleboy/ssh-action, then forward one
# single-line base64 value to the remote upsert script.
if printf '%s' "$CONTAINERS_SSH_KEY_INPUT" | base64 -d >"$tmp" 2>/dev/null && grep -q 'BEGIN .*PRIVATE KEY' "$tmp"; then
containers_ssh_key_b64="$(printf '%s' "$CONTAINERS_SSH_KEY_INPUT" | tr -d '\r\n')"
else
containers_ssh_key_b64="$(printf '%s' "$CONTAINERS_SSH_KEY_INPUT" | base64 -w0)"
fi
[ -n "$containers_ssh_key_b64" ] || { echo "::error::container SSH key resolved to an empty value"; exit 1; }
echo "::add-mask::$containers_ssh_key_b64"
echo "CONTAINERS_SSH_KEY_B64=$containers_ssh_key_b64" >> "$GITHUB_ENV"
# OPTIONAL: pull the tenant admin DSN straight from the apps-shared
# terraform output so the password never appears in a workflow input or in
# logs. `terraform output -raw` of a SENSITIVE value is auto-masked by the
# runner; we also ::add-mask:: it before writing to $GITHUB_ENV.
- name: Checkout (for terraform fetch)
if: ${{ github.event.inputs.fetch_dsn_from_shared_state == 'true' }}
uses: actions/checkout@v4
- name: Setup Terraform (for DSN fetch)
if: ${{ github.event.inputs.fetch_dsn_from_shared_state == 'true' }}
uses: hashicorp/setup-terraform@v4
with:
terraform_version: "1.10.5"
- name: Read DSN from apps-shared output
if: ${{ github.event.inputs.fetch_dsn_from_shared_state == 'true' }}
working-directory: packages/cloud/infra/cloud/terraform/hetzner/apps-shared
env:
HCLOUD_TOKEN: ${{ secrets.HCLOUD_APPS_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.R2_STATE_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_STATE_SECRET_ACCESS_KEY }}
run: |
terraform init -input=false -backend-config=backend.hcl >/dev/null
DSN="$(terraform output -raw tenant_db_admin_dsn)"
[ -n "$DSN" ] || { echo "::error::apps-shared has no tenant_db_admin_dsn output (is it applied?)"; exit 1; }
echo "::add-mask::$DSN"
echo "TENANT_ADMIN_DSN=$DSN" >> "$GITHUB_ENV"
echo "fetched tenant admin DSN from apps-shared output (masked)"
- name: Upsert apps env + restart daemon
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 5m
# Only the values; the upsert logic is fixed here (mirrors arm-apps-daemon.mjs).
envs: APP_NODE,BASE_DOMAIN,CADDY_ADMIN,TENANT_ADMIN_DSN,EGRESS_PROXY,SYSTEMD_UNIT,CONTAINERS_SSH_KEY_B64
script: |
set -euo pipefail
F=/opt/eliza/cloud/.env.local
# /opt/eliza/cloud is root-owned; the deploy user has passwordless
# sudo (same as the worker-deploy workflow), so all writes go via
# sudo — `sed -i` needs to create a temp file IN the dir, not just
# write the file, which a non-sudo deploy user can't do there.
sudo test -f "$F" || { echo "env file $F not found on host"; exit 1; }
sudo cp -n "$F" "$F.bak.arm-apps" 2>/dev/null || true
upsert() {
# upsert KEY VALUE — delete any existing def, append the fresh one.
local k="$1"; local v="$2"
[ -n "$v" ] || return 0
sudo sed -i "/^$k=/d" "$F"
printf '%s\n' "$k=\"$v\"" | sudo tee -a "$F" >/dev/null
}
# APPS_DEPLOY_ENABLED arms the APP_DEPLOY runner; without it the daemon's
# armAppsDeployBackendIfEnabled() early-returns (provisioning-worker.ts:652)
# and APP_DEPLOY jobs never get claimed (apps stuck "building").
upsert APPS_DEPLOY_ENABLED 1
upsert APPS_CONTAINERS_ENABLED 1
upsert CONTAINERS_DOCKER_NODES "$APP_NODE"
upsert CONTAINERS_SSH_USER deploy
upsert CONTAINERS_SSH_KEY "$CONTAINERS_SSH_KEY_B64"
upsert APPS_CADDY_ADMIN_URL "$CADDY_ADMIN"
upsert CONTAINERS_PUBLIC_BASE_DOMAIN "$BASE_DOMAIN"
upsert APPS_TENANT_ADMIN_DSN "$TENANT_ADMIN_DSN"
upsert CONTAINERS_EGRESS_PROXY_URL "$EGRESS_PROXY"
echo "--- apps env now on the box (secrets redacted) ---"
sudo grep -E '^(APPS_|CONTAINERS_(DOCKER_NODES|SSH_USER|SSH_KEY|PUBLIC_BASE_DOMAIN|EGRESS_PROXY_URL))' "$F" \
| sed -E 's/(DSN|KEY)=.*/\1=<redacted>/'
# Restart whichever daemon we armed: the dedicated apps-worker on the
# apps-control host, or the shared control-plane provisioning-worker.
UNIT="${SYSTEMD_UNIT:-eliza-provisioning-worker.service}"
sudo systemctl restart "$UNIT"
sleep 2
echo "--- daemon status ($UNIT) ---"
systemctl is-active "$UNIT"
journalctl -u "$UNIT" -n 10 --no-pager | tail -10 || true
@@ -0,0 +1,112 @@
name: Arm Headscale Control Plane
# Converges Headscale on the Hetzner control-plane host. This is the
# repeatable version of the launch runbook: configure server_url/listen_addr,
# install the committed ACL, ensure users exist, upsert daemon env, restart
# headscale + provisioning-worker, and fail if local /health is not green.
#
# WHY: agent provisioning must not be allowed to "sort of" work. If the daemon
# has a Headscale key, docker-sandbox-provider requires a real headscale_ip
# before marking a sandbox running. This workflow makes that dependency explicit
# and keeps prod/staging wiring in GitHub Environments instead of hand-edited
# shell history on the host.
on:
workflow_dispatch:
inputs:
environment:
description: "Target control-plane environment"
type: choice
default: staging
options: [staging, production]
headscale_public_url:
description: "Public Headscale URL. Empty = vars.HEADSCALE_PUBLIC_URL"
type: string
required: false
default: ""
headscale_api_url:
description: "Daemon-local Headscale API URL"
type: string
required: false
default: "http://127.0.0.1:8081"
listen_addr:
description: "Headscale listen_addr on the CP"
type: string
required: false
default: "127.0.0.1:8081"
permissions:
contents: read
concurrency:
group: arm-headscale-control-plane-${{ github.event.inputs.environment }}
cancel-in-progress: false
jobs:
arm:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ${{ github.event.inputs.environment }}
timeout-minutes: 10
env:
DEPLOY_HOST: ${{ secrets.ELIZA_PROVISIONING_HOST }}
DEPLOY_SSH_KEY: ${{ secrets.ELIZA_PROVISIONING_SSH_KEY }}
HEADSCALE_API_KEY: ${{ secrets.HEADSCALE_API_KEY }}
AGENT_TOKEN_PRIVATE_KEY_PEM: ${{ secrets.AGENT_TOKEN_PRIVATE_KEY_PEM }}
ELIZA_LOCAL_ROOT_KEY: ${{ secrets.ELIZA_LOCAL_ROOT_KEY }}
HEADSCALE_PUBLIC_URL_INPUT: ${{ github.event.inputs.headscale_public_url }}
HEADSCALE_PUBLIC_URL_VAR: ${{ vars.HEADSCALE_PUBLIC_URL }}
HEADSCALE_API_URL: ${{ github.event.inputs.headscale_api_url }}
HEADSCALE_LISTEN_ADDR: ${{ github.event.inputs.listen_addr }}
# Let's Encrypt account / expiry-notice email for the headscale vhost cert
# the arm script provisions. Optional GitHub Environment var; the script
# falls back to ops@elizalabs.ai when unset (so existing envs need no new
# var to keep working).
CERTBOT_EMAIL: ${{ vars.CERTBOT_EMAIL }}
steps:
- uses: actions/checkout@v4
- name: Preflight
run: |
fail=0
[ -n "$DEPLOY_HOST" ] || { echo "::error::missing secret ELIZA_PROVISIONING_HOST"; fail=1; }
[ -n "$DEPLOY_SSH_KEY" ] || { echo "::error::missing secret ELIZA_PROVISIONING_SSH_KEY"; fail=1; }
[ -n "$HEADSCALE_API_KEY" ] || { echo "::error::missing secret HEADSCALE_API_KEY"; fail=1; }
if [ -n "$HEADSCALE_PUBLIC_URL_INPUT" ]; then
echo "HEADSCALE_PUBLIC_URL=$HEADSCALE_PUBLIC_URL_INPUT" >> "$GITHUB_ENV"
elif [ -n "$HEADSCALE_PUBLIC_URL_VAR" ]; then
echo "HEADSCALE_PUBLIC_URL=$HEADSCALE_PUBLIC_URL_VAR" >> "$GITHUB_ENV"
else
echo "::error::set vars.HEADSCALE_PUBLIC_URL or provide headscale_public_url"
fail=1
fi
[ "$fail" = 0 ] || exit 1
- name: Write SSH key
run: |
install -d -m 700 "$RUNNER_TEMP/headscale"
printf '%s\n' "$DEPLOY_SSH_KEY" > "$RUNNER_TEMP/headscale/deploy_ed25519"
chmod 600 "$RUNNER_TEMP/headscale/deploy_ed25519"
echo "DEPLOY_SSH_KEY_PATH=$RUNNER_TEMP/headscale/deploy_ed25519" >> "$GITHUB_ENV"
- name: Arm Headscale and daemon env
run: |
node packages/scripts/cloud/admin/arm-headscale-control-plane.mjs \
--host "$DEPLOY_HOST" \
--ssh-key "$DEPLOY_SSH_KEY_PATH" \
--headscale-public-url "$HEADSCALE_PUBLIC_URL" \
--headscale-api-url "$HEADSCALE_API_URL" \
--listen-addr "$HEADSCALE_LISTEN_ADDR"
- name: Summary
run: |
{
echo "### Headscale control plane armed"
echo ""
echo "- Environment: \`${{ github.event.inputs.environment }}\`"
echo "- Public URL: \`$HEADSCALE_PUBLIC_URL\`"
echo "- Daemon API URL: \`$HEADSCALE_API_URL\`"
echo ""
echo "Also converged on the CP (idempotent): nginx vhost + Let's Encrypt cert for the public headscale URL, and the cp-<env>-router tailscale self-enrollment (tag:eliza-proxy)."
echo ""
echo "Next: set matching Cloudflare Worker secrets, then run the provision E2E."
} >> "$GITHUB_STEP_SUMMARY"
@@ -0,0 +1,60 @@
name: Attachment journey videos
# Manual-only (workflow_dispatch): records full video runthroughs of the
# attachment user journeys — the Files view (load + facets + CRUD affordances)
# and the chat attachment upload flow — and uploads the .webm recordings +
# traces + screenshots as artifacts (E2E_RECORD=1 → e2e-recordings/). Kept off
# PR/develop CI to keep those runs fast; trigger from the Actions tab when a
# journey video is wanted. Runs on top of the keyless ui-smoke stub stack
# (no secrets), and the renderer build picks up @elizaos/core via the
# core-build pre-step in run-ui-playwright.mjs.
on:
workflow_dispatch:
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
jobs:
attachment-journey-videos:
name: Attachment journey videos
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 40
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Ensure generated shared i18n data
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Install Playwright browsers
run: bunx playwright install --with-deps chromium
- name: Record attachment journeys (Files view + chat attachment)
run: bun run --cwd packages/app test:e2e:record test/ui-smoke/files-view.spec.ts test/ui-smoke/chat-attachment.spec.ts --project=chromium
- name: Upload journey videos + traces
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: attachment-journey-videos
path: |
e2e-recordings/
packages/app/playwright-report/
if-no-files-found: warn
retention-days: 14
+70
View File
@@ -0,0 +1,70 @@
name: Auto Label
on:
pull_request_target:
types: [opened, synchronize]
issues:
types: [opened, edited]
# #14051 Tier B: label runs are pure metadata — a superseded label pass on the
# same PR/issue is worthless. Cancel the in-flight run when a new event lands so
# rapid pushes don't pile up 70+ queued Auto Label runs on the fleet. Group by
# PR number (or issue number) so PR and issue lanes never collide.
concurrency:
group: auto-label-${{ github.event.pull_request.number || github.event.issue.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
issues: write
jobs:
label-pr:
if: github.event_name == 'pull_request_target'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- uses: actions/labeler@f27b608878404679385c85cfa523b85ccb86e213
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
pr-number: "${{ github.event.pull_request.number }}"
label-issue:
if: github.event_name == 'issues'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
with:
script: |
const title = context.payload.issue.title.toLowerCase();
const body = (context.payload.issue.body || '').toLowerCase();
const text = title + ' ' + body;
const labels = [];
if (text.includes('crash') || text.includes('error') || text.includes('bug') || text.includes('broken') || text.includes('regression')) {
labels.push('bug');
}
if (text.includes('security') || text.includes('vulnerability') || text.includes('cve') || text.includes('injection')) {
labels.push('security');
}
if (text.includes('performance') || text.includes('slow') || text.includes('memory leak') || text.includes('latency')) {
labels.push('performance');
}
if (text.includes('telegram') || text.includes('discord') || text.includes('connector')) {
labels.push('connector');
}
if (text.includes('memory') || text.includes('context') || text.includes('rag')) {
labels.push('memory');
}
if (text.includes('plugin')) {
labels.push('plugin');
}
if (labels.length > 0) {
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
labels: labels
});
}
@@ -0,0 +1,154 @@
# Benchmark orchestrator — scheduled real-model lane
#
# Runs a core subset of registry benchmarks through the Python orchestrator on a
# REAL model provider (Cerebras gemma-4-31b), so the suite is actually exercised
# end-to-end on a schedule rather than only vitest-testing the bridge code
# (#9475). Before this lane, `python -m benchmarks.orchestrator run` was invoked
# by exactly one workflow (hyperliquid-bench-live.yml), leaving 42/43 registered
# benchmarks with zero scheduled real-model runs.
#
# OPERATOR PREREQUISITE: add a `CEREBRAS_API_KEY` repo secret. With no secret
# configured this workflow SKIPS GRACEFULLY — it never fails CI when the key is
# absent (mirrors hyperliquid-bench-live.yml's gate).
#
# The benchmark subset (bfcl, action-calling, agentbench, tau_bench, mint,
# context_bench) is the confirmed-real, bridge-wired core that routes through a
# real AgentRuntime + LLM plugins via the eliza-adapter.
#
# Required repo secret (self-documented):
# CEREBRAS_API_KEY — Cerebras API key for the real-model orchestrator run.
name: Benchmark orchestrator (scheduled)
on:
schedule:
# Mondays 06:00 UTC — a weekly real-model smoke of the core registry subset.
- cron: "0 6 * * 1"
workflow_dispatch:
inputs:
benchmarks:
description: "Comma-separated registry benchmark ids"
required: false
type: string
default: "bfcl,action-calling,agentbench,tau_bench,mint,context_bench"
provider:
description: "Model provider"
required: false
type: string
default: "cerebras"
model:
description: "Model name"
required: false
type: string
default: "gemma-4-31b"
concurrency:
group: benchmark-orchestrator-scheduled-${{ github.ref }}
cancel-in-progress: false
# Default to least privilege.
permissions:
contents: read
env:
PYTHON_VERSION: "3.12"
BUN_VERSION: "canary"
NODE_VERSION: "24"
jobs:
orchestrator-run:
name: orchestrator-run
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 120
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
steps:
# ------------------------------------------------------------------
# Gate: skip everything (exit 0) when CEREBRAS_API_KEY is unset, so this
# workflow never fails CI when the secret is not configured. Mirrors the
# gate in hyperliquid-bench-live.yml.
# ------------------------------------------------------------------
- name: Check CEREBRAS_API_KEY secret
id: gate
run: |
set -euo pipefail
if [ -z "${CEREBRAS_API_KEY:-}" ]; then
echo "::notice::Benchmark orchestrator scheduled run skipped — CEREBRAS_API_KEY is not configured. Add it as a repo secret to enable real-model orchestrator runs."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: recursive
fetch-depth: 1
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup Node.js
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
if: steps.gate.outputs.skip != 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
# ------------------------------------------------------------------
# The orchestrator package is `benchmarks` under `packages/`, so the
# module is invoked as `python -m benchmarks.orchestrator` with
# `PYTHONPATH=packages` from the repo root.
#
# NOTE: `validate-runtime-gates` is intentionally NOT run here — it checks
# ALL live gates globally (Hyperliquid key, Docker backends, …) and exits
# non-zero when any is unavailable, which is unrelated to this LLM-only core
# subset. The subset-scoped publishability gate below is the relevant check.
# ------------------------------------------------------------------
- name: Run core registry subset on a real model
if: steps.gate.outputs.skip != 'true'
id: run
env:
BENCHMARKS: ${{ inputs.benchmarks || 'bfcl,action-calling,agentbench,tau_bench,mint,context_bench' }}
PROVIDER: ${{ inputs.provider || 'cerebras' }}
MODEL: ${{ inputs.model || 'gemma-4-31b' }}
PYTHONUNBUFFERED: "1"
run: |
set -e
PYTHONPATH=packages python -m benchmarks.orchestrator run \
--benchmarks "$BENCHMARKS" \
--provider "$PROVIDER" \
--model "$MODEL" \
--force \
--show-incompatible \
2>&1 | tee orchestrator-scheduled.log
- name: Validate latest publishability (rejects mock/sample/demo rows)
if: steps.gate.outputs.skip != 'true'
env:
PYTHONUNBUFFERED: "1"
run: |
set -e
PYTHONPATH=packages python -m benchmarks.orchestrator validate-latest-publishability \
--include-benchmarks "${{ inputs.benchmarks || 'bfcl,action-calling,agentbench,tau_bench,mint,context_bench' }}"
- name: Upload orchestrator artifacts
if: always() && steps.gate.outputs.skip != 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: benchmark-orchestrator-scheduled-${{ github.run_id }}
path: |
packages/benchmarks/benchmark_results/**
orchestrator-scheduled.log
if-no-files-found: warn
retention-days: 90
+70
View File
@@ -0,0 +1,70 @@
name: Benchmark Bridge Tests
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: benchmark-tests-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
on:
push:
branches: [main, develop]
paths:
- "packages/lifeops-bench/**"
- ".github/workflows/benchmark-tests.yml"
pull_request:
branches: [main]
paths:
- "packages/lifeops-bench/**"
- ".github/workflows/benchmark-tests.yml"
workflow_dispatch:
env:
NODE_NO_WARNINGS: "1"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
benchmark:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
- lane: benchmark-tests
command: bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests
- lane: benchmark-lint
command: bunx @biomejs/biome check packages/lifeops-bench/src
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24"
check-latest: false
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454); packageManager bun@1.4.0-canary.1 is unresolvable (GH+npm 404)
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
- name: Run ${{ matrix.lane }} lane
env:
CI: "true"
run: ${{ matrix.command }}
+221
View File
@@ -0,0 +1,221 @@
# Weekly EA + Connector Benchmark
#
# Runs the full executive-assistant and connector certification scenario
# catalogs through the live scenario runner and uploads a markdown + JSON
# benchmark report as an artifact. Scheduled/default dispatch runs are
# report-only so the weekly trend capture stays green while the live catalog
# is still being hardened; manual dispatch can opt into enforcement.
#
# Distinct from live-scenarios.yml: weekly cadence, emits a human-readable
# markdown report suitable for tracking trend lines over time.
#
# Required repo secrets (self-documented): identical to live-scenarios.yml.
# LLM providers: OPENAI_API_KEY, OPENROUTER_API_KEY, ANTHROPIC_API_KEY,
# GOOGLE_GENERATIVE_AI_API_KEY, GOOGLE_API_KEY, GROQ_API_KEY
# Google: GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET,
# GOOGLE_OAUTH_REFRESH_TOKEN, GMAIL_TEST_ACCOUNT_EMAIL,
# GMAIL_TEST_ACCOUNT_REFRESH_TOKEN
# Calendly: CALENDLY_API_TOKEN
# Telegram: TELEGRAM_BOT_TOKEN, TELEGRAM_TEST_CHAT_ID, TELEGRAM_API_ID,
# TELEGRAM_API_HASH
# Discord: DISCORD_BOT_TOKEN, DISCORD_TEST_GUILD_ID,
# DISCORD_TEST_CHANNEL_ID
# Signal: SIGNAL_CLI_URL, SIGNAL_TEST_NUMBER
# iMessage: IMESSAGE_BRIDGE_URL, IMESSAGE_TEST_HANDLE
# WhatsApp: WHATSAPP_TOKEN, WHATSAPP_PHONE_ID, WHATSAPP_TEST_CONTACT
# Twilio: TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_FROM_NUMBER,
# TWILIO_TEST_TO_NUMBER
# X: X_API_KEY, X_API_SECRET, X_ACCESS_TOKEN, X_ACCESS_SECRET,
# X_TEST_DM_HANDLE
# Push: NOTIFICATION_RELAY_URL, NOTIFICATION_RELAY_TOKEN
# Travel: TRAVEL_BOOKING_API_KEY
# Optional:
# LIFEOPS_JUDGE_THRESHOLD (workflow input, default 0.8)
# SCENARIO_FILTER (workflow input, empty = all cataloged benchmark scenarios)
# BENCHMARK_ENFORCE_GATE (workflow input, default false)
name: Benchmark (weekly)
on:
schedule:
# Mondays 10:00 UTC — after weekend regressions settle.
- cron: "0 10 * * 1"
workflow_dispatch:
inputs:
scenario_filter:
description: "Comma-separated scenario ids (empty = full benchmark catalog)"
required: false
type: string
default: ""
judge_threshold:
description: "LLM-judge minimum pass score (0.0 - 1.0)"
required: false
type: string
default: "0.8"
enforce_gate:
description: "Fail the workflow when benchmark scenarios fail"
required: false
type: boolean
default: false
concurrency:
group: benchmark-weekly-${{ github.ref }}
cancel-in-progress: false
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
ELIZA_LIVE_TEST: "1"
permissions:
contents: read
actions: read
jobs:
benchmark:
name: EA + connector benchmark
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 240
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
filter: blob:none
submodules: recursive
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
- name: Verify scenario-runner CLI present
run: |
test -f packages/scenario-runner/src/cli.ts || \
{ echo "scenario-runner CLI missing"; exit 1; }
- name: Build benchmark runtime packages
# The benchmark executes TypeScript sources directly, but several
# workspace packages intentionally export dist/* entry points. Because
# dependency installation ignores postinstall scripts, build only the
# packages that the live scenario runtime imports through those exports.
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
run: |
echo "::group::Build packages/core"
bun run --cwd packages/core build
echo "::endgroup::"
echo "::group::Build packages/shared"
# plugin-agent-skills (and others) consume @elizaos/shared via dist
# exports (RouteRequestContext, route schemas). Must build before
# any plugin that imports from it.
bun run --cwd packages/shared build
echo "::endgroup::"
echo "::group::Build packages/skills"
# plugin-agent-skills imports @elizaos/skills via its dist exports.
bun run --cwd packages/skills build
echo "::endgroup::"
provider_package=""
if [ -n "${GROQ_API_KEY:-}" ]; then
provider_package="plugins/plugin-groq"
elif [ -n "${OPENAI_API_KEY:-}" ]; then
provider_package="plugins/plugin-openai"
elif [ -n "${ANTHROPIC_API_KEY:-}" ]; then
provider_package="plugins/plugin-anthropic"
elif [ -n "${GOOGLE_GENERATIVE_AI_API_KEY:-}" ] || [ -n "${GOOGLE_API_KEY:-}" ]; then
provider_package="plugins/plugin-google-genai"
elif [ -n "${OPENROUTER_API_KEY:-}" ]; then
provider_package="plugins/plugin-openrouter"
fi
package_dirs=(
plugins/plugin-sql
plugins/plugin-agent-skills
plugins/plugin-pdf
plugins/plugin-telegram
plugins/plugin-whatsapp
plugins/plugin-signal
plugins/plugin-imessage
)
if [ -n "$provider_package" ]; then
package_dirs+=("$provider_package")
fi
for package_dir in "${package_dirs[@]}"; do
echo "::group::Build ${package_dir}"
bun run --cwd "$package_dir" build
echo "::endgroup::"
done
- name: Run benchmark harness
id: run
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
GOOGLE_OAUTH_CLIENT_ID: ${{ secrets.GOOGLE_OAUTH_CLIENT_ID }}
GOOGLE_OAUTH_CLIENT_SECRET: ${{ secrets.GOOGLE_OAUTH_CLIENT_SECRET }}
GOOGLE_OAUTH_REFRESH_TOKEN: ${{ secrets.GOOGLE_OAUTH_REFRESH_TOKEN }}
GMAIL_TEST_ACCOUNT_EMAIL: ${{ secrets.GMAIL_TEST_ACCOUNT_EMAIL }}
GMAIL_TEST_ACCOUNT_REFRESH_TOKEN: ${{ secrets.GMAIL_TEST_ACCOUNT_REFRESH_TOKEN }}
CALENDLY_API_TOKEN: ${{ secrets.CALENDLY_API_TOKEN }}
TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TELEGRAM_TEST_CHAT_ID: ${{ secrets.TELEGRAM_TEST_CHAT_ID }}
TELEGRAM_API_ID: ${{ secrets.TELEGRAM_API_ID }}
TELEGRAM_API_HASH: ${{ secrets.TELEGRAM_API_HASH }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
DISCORD_TEST_GUILD_ID: ${{ secrets.DISCORD_TEST_GUILD_ID }}
DISCORD_TEST_CHANNEL_ID: ${{ secrets.DISCORD_TEST_CHANNEL_ID }}
SIGNAL_CLI_URL: ${{ secrets.SIGNAL_CLI_URL }}
SIGNAL_TEST_NUMBER: ${{ secrets.SIGNAL_TEST_NUMBER }}
IMESSAGE_BRIDGE_URL: ${{ secrets.IMESSAGE_BRIDGE_URL }}
IMESSAGE_TEST_HANDLE: ${{ secrets.IMESSAGE_TEST_HANDLE }}
WHATSAPP_TOKEN: ${{ secrets.WHATSAPP_TOKEN }}
WHATSAPP_PHONE_ID: ${{ secrets.WHATSAPP_PHONE_ID }}
WHATSAPP_TEST_CONTACT: ${{ secrets.WHATSAPP_TEST_CONTACT }}
TWILIO_ACCOUNT_SID: ${{ secrets.TWILIO_ACCOUNT_SID }}
TWILIO_AUTH_TOKEN: ${{ secrets.TWILIO_AUTH_TOKEN }}
TWILIO_FROM_NUMBER: ${{ secrets.TWILIO_FROM_NUMBER }}
TWILIO_TEST_TO_NUMBER: ${{ secrets.TWILIO_TEST_TO_NUMBER }}
X_API_KEY: ${{ secrets.X_API_KEY }}
X_API_SECRET: ${{ secrets.X_API_SECRET }}
X_ACCESS_TOKEN: ${{ secrets.X_ACCESS_TOKEN }}
X_ACCESS_SECRET: ${{ secrets.X_ACCESS_SECRET }}
X_TEST_DM_HANDLE: ${{ secrets.X_TEST_DM_HANDLE }}
NOTIFICATION_RELAY_URL: ${{ secrets.NOTIFICATION_RELAY_URL }}
NOTIFICATION_RELAY_TOKEN: ${{ secrets.NOTIFICATION_RELAY_TOKEN }}
TRAVEL_BOOKING_API_KEY: ${{ secrets.TRAVEL_BOOKING_API_KEY }}
SCENARIO_FILTER: ${{ inputs.scenario_filter }}
LIFEOPS_JUDGE_THRESHOLD: ${{ inputs.judge_threshold || '0.8' }}
BENCHMARK_ENFORCE_GATE: ${{ inputs.enforce_gate && '1' || '0' }}
BENCHMARK_REPORT_PATH: artifacts/benchmark-report.md
# Trajectory data and ephemeral state land under ~/.eliza per
# eliza-native ELIZA_STATE_DIR convention.
run: node packages/scripts/run-scenario-benchmark.mjs
- name: Upload benchmark report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: benchmark-report
path: |
artifacts/benchmark-report.md
artifacts/lifeops-scenario-report.json
if-no-files-found: warn
retention-days: 90
+118
View File
@@ -0,0 +1,118 @@
name: Browser Benchmark Lanes
# Deferred "Needs CI infra" lanes of #9476, tracked by #10333:
# - MiniWoB++ benchmark suite against a REAL Chromium engine (puppeteer-core)
# - ScreenSpot-Web-style grounding against a REAL Chromium engine
# - committed Mind2Web/WebArena-style external dataset fixtures through the
# plugin-browser BROWSER command router, including a REAL Chromium pass
#
# The `*.real.test.ts` lanes are excluded from the default `vitest run` and
# self-skip without a Chromium binary, so they only execute here after
# `bunx playwright install chromium` (mirroring scenario-pr.yml's
# `app-browser-core`). Gated like the other heavy real lanes — nightly +
# on-demand + on changes to the benchmark/executor — not on every PR.
on:
schedule:
# 10:00 UTC daily — after the external-API live lanes settle.
- cron: "0 10 * * *"
workflow_dispatch: {}
pull_request:
branches: [main]
paths:
- "plugins/plugin-browser/src/benchmark/**"
- "plugins/plugin-browser/vitest.real.config.ts"
- "plugins/plugin-browser/scripts/*benchmark*.mjs"
- "plugins/plugin-browser/scripts/capture-web-grounding-evidence.mjs"
- "plugins/plugin-browser/package.json"
- ".github/workflows/browser-real-bench.yml"
concurrency:
group: browser-real-bench-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CI: "true"
NODE_VERSION: "24"
BUN_VERSION: "canary"
jobs:
browser-real-bench:
name: Browser benchmarks on real Chromium
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
# setup-bun-workspace instead of a hand-rolled `bun install
# --frozen-lockfile`: this repo tracks bun canary, and canary rebuilds
# reserialize bun.lock between builds, so a bare frozen install reds with
# "lockfile had changes, but lockfile is frozen" on schedule runs (e.g.
# run 28513244717). The composite handles the drift fallback and restores
# the committed lockfile; same invocation as scenario-pr.yml's
# app-browser-core lane this workflow mirrors.
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
# Real Chromium for the engine lane — the same pattern scenario-pr.yml's
# app-browser-core uses. resolveChromiumExecutablePath() picks it up via
# playwright-core, so no extra env wiring is needed.
- name: Install Chromium
run: bunx playwright install --with-deps chromium
- name: Run real Chromium benchmark lanes
run: bun run --cwd plugins/plugin-browser test:real-chromium
- name: Generate MiniWoB++ run-report artifacts (oracle + noop)
env:
BROWSER_EVIDENCE_DIR: ${{ runner.temp }}/evidence/10333-browser-real-chromium
run: |
mkdir -p "$BROWSER_EVIDENCE_DIR"
bun run --cwd plugins/plugin-browser bench:miniwob:chromium --policy oracle --seeds 3 --out "$BROWSER_EVIDENCE_DIR/miniwob-chromium-oracle-run.json"
bun run --cwd plugins/plugin-browser bench:miniwob:chromium --policy noop --seeds 2 --out "$BROWSER_EVIDENCE_DIR/miniwob-chromium-noop-run.json"
- name: Generate web-grounding artifacts
env:
OUT: ${{ runner.temp }}/evidence/10333-web-grounding
run: bun run --cwd plugins/plugin-browser bench:grounding:chromium
- name: Generate external-dataset artifacts
env:
BROWSER_EVIDENCE_DIR: ${{ runner.temp }}/evidence/10333-browser-external-dataset
run: |
mkdir -p "$BROWSER_EVIDENCE_DIR"
bun run --cwd plugins/plugin-browser bench:external --policy oracle --out "$BROWSER_EVIDENCE_DIR/external-dataset-oracle-run.json"
bun run --cwd plugins/plugin-browser bench:external --policy noop --out "$BROWSER_EVIDENCE_DIR/external-dataset-noop-run.json"
bun run --cwd plugins/plugin-browser bench:external --policy wrong --out "$BROWSER_EVIDENCE_DIR/external-dataset-wrong-run.json"
bun run --cwd plugins/plugin-browser bench:external:chromium --policy oracle --out "$BROWSER_EVIDENCE_DIR/external-dataset-chromium-oracle-run.json"
bun run --cwd plugins/plugin-browser bench:external:chromium --policy noop --out "$BROWSER_EVIDENCE_DIR/external-dataset-chromium-noop-run.json"
bun run --cwd plugins/plugin-browser bench:external:chromium --policy wrong --out "$BROWSER_EVIDENCE_DIR/external-dataset-chromium-wrong-run.json"
- name: Upload benchmark artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: browser-real-benchmark-runs
path: |
${{ runner.temp }}/evidence/10333-browser-real-chromium/*
${{ runner.temp }}/evidence/10333-web-grounding/*
${{ runner.temp }}/evidence/10333-browser-external-dataset/*.json
if-no-files-found: warn
+346
View File
@@ -0,0 +1,346 @@
name: Build Agent Image
# Single source of truth for the cloud-agent container image used by the
# cloud sandbox provisioner. Three trigger surfaces:
# - push to develop/main → :develop / :stable + :latest, plus :sha-<short>
# - GitHub release created → release tag (e.g. v2.0.0-alpha.176)
# - workflow_dispatch → manual reruns
# The previous release-only workflow (image.yaml) duplicated this entire
# pre-build sequence; it's been deleted in favor of this consolidated build.
on:
push:
branches: [develop, main]
paths:
- "packages/app-core/deploy/Dockerfile.ci"
- "plugins/plugin-sql/**"
- "plugins/plugin-elizacloud/**"
- "plugins/plugin-local-inference/**"
- "packages/agent/**"
- "packages/server/**"
- "packages/core/**"
- "packages/app/**"
- "packages/app-core/**"
- "packages/cloud/sdk/**"
- ".github/workflows/build-agent-image.yml"
- ".dockerignore"
release:
types: [created]
workflow_dispatch:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
# Default to least privilege. Override per-job where needed.
# cancel superseded in-flight runs for the same PR to free hosted-runner
# capacity. only cancels pull_request runs; push runs on protected branches
# (main/develop) are never cancelled mid-flight.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
jobs:
build-and-push:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
permissions:
contents: read
packages: write
attestations: write
id-token: write
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Free up disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
sudo docker image prune -a -f || true
- name: Install jq
# Used by docker-ci-smoke's boot-KPI budget parser after the image is
# built. Keep it explicit so the real boot gate reports the committed
# budget instead of falling back to its baked-in default.
run: sudo apt-get install -y --no-install-recommends jq
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Cache Bun install
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.bun/install/cache
key: bun-${{ runner.os }}-canary-${{ hashFiles('bun.lock') }}
restore-keys: |
bun-${{ runner.os }}-canary-
bun-${{ runner.os }}-
- name: Install dependencies
# Use `--no-frozen-lockfile` so a downstream lockfile drift in
# `develop` doesn't short-circuit the install before postinstall
# gets a chance to run (which is what was happening with the
# earlier `--frozen-lockfile` + fallback dance — the frozen
# install would fail, Bun's silent retry would no-op against
# the already-extracted virtual store, and postinstall would
# never run, leaving root-level deps like `uuid`, `drizzle-orm`,
# and the patched `tsup/dist/rollup.js` unavailable to the
# per-plugin build loop). The heavy submodule fetch is guarded
# by NODE_LLAMA_CPP_SKIP_DOWNLOAD.
env:
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: |
bun install --no-frozen-lockfile
- name: Cache Turbo build outputs
# The turbo build below is the single biggest pre-docker cost (~5-6
# min from cold). Nothing persisted turbo's local task cache between
# runs, so every push rebuilt every filtered package from scratch.
# Keying on the commit SHA (with a prefix restore-key) restores the
# most recent cache and saves an updated one each run, so a typical
# source commit only rebuilds the packages it actually touched.
# The key prefix is workflow-specific: entries here are produced
# under NODE_ENV=production, which turbo does not hash, so they must
# not be replayed into other workflows' builds.
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: .turbo/cache
key: turbo-agent-image-${{ runner.os }}-${{ github.sha }}
restore-keys: |
turbo-agent-image-${{ runner.os }}-
- name: Apply postinstall patches
# Belt-and-suspenders for the postinstall path: even with
# `bun install` (postinstall enabled), the silent fallback that
# kicks in when frozen-lockfile drifts doesn't always re-run the
# repo postinstall in CI. Explicitly run the patches the
# per-plugin builds depend on:
# - ensure-type-package-aliases: materializes @types/* under
# node_modules/@types so `tsc --build` resolves them.
env:
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: |
node packages/app-core/scripts/ensure-type-package-aliases.mjs
- name: Build Docker workspace artifacts
# Let Turbo derive the build order and reuse task cache instead of
# hand-maintaining the core/package/plugin sequence. The explicit
# filters include the app, the agent entrypoint, and every dynamically
# loaded plugin whose dist must exist before Dockerfile.ci relinks the
# local workspace packages into the image.
env:
NODE_ENV: production
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: |
node packages/scripts/run-turbo.mjs run build --concurrency=8 \
--filter=@elizaos/app \
--filter=@elizaos/agent \
--filter=@elizaos/plugin-sql \
--filter=@elizaos/plugin-video \
--filter=@elizaos/plugin-agent-skills \
--filter=@elizaos/plugin-pdf \
--filter=@elizaos/plugin-browser \
--filter=@elizaos/plugin-capacitor-bridge \
--filter=@elizaos/plugin-coding-tools \
--filter=@elizaos/plugin-native-filesystem \
--filter=@elizaos/plugin-shell \
--filter=@elizaos/plugin-commands \
--filter=@elizaos/plugin-computeruse \
--filter=@elizaos/plugin-discord \
--filter=@elizaos/plugin-edge-tts \
--filter=@elizaos/plugin-elizacloud \
--filter=@elizaos/plugin-imessage \
--filter=@elizaos/plugin-local-inference \
--filter=@elizaos/plugin-mcp \
--filter=@elizaos/plugin-signal \
--filter=@elizaos/plugin-streaming \
--filter=@elizaos/plugin-telegram \
--filter=@elizaos/plugin-whatsapp \
--filter=@elizaos/plugin-wallet \
--filter=@elizaos/plugin-workflow \
--filter=@elizaos/plugin-x402
- name: Prune stale Turbo cache entries
# actions/cache restores + re-saves the whole cache dir, so without a
# prune the tarball grows by every commit's changed-package outputs
# forever. tar preserves mtimes, so age-based deletion bounds it:
# entries untouched for two weeks are long since superseded.
run: find .turbo/cache -type f -mtime +14 -delete 2>/dev/null || true
- name: Normalize plugin dist for Node ESM
# MUST run AFTER the Turbo build above. Some package builds still emit
# extensionless or directory specifiers (`./actions/index`,
# `./approvals`) in dist. The container loads those built `.js` files
# via import(), and neither tsx nor plain `node` resolves them. Apply
# the same rewrite the agent's own build uses to EVERY built plugin
# dist, last, so the dist that actually ships is Node-ESM-resolvable.
run: |
shopt -s nullglob
for dist in plugins/*/dist; do
pkg="$(dirname "$dist")"
node packages/scripts/rewrite-dist-relative-imports-node-esm.mjs "$pkg" || true
done
- name: Prepare CI dockerignore
# The root .dockerignore is tuned for `bun install` runs (excludes
# everything heavy). The Docker build needs a different policy:
# keep root node_modules (Dockerfile.ci's pruner relink scripts
# consume them) but strip per-package node_modules and the other
# bulky CI-only inputs. Use the canonical CI dockerignore from
# packages/app-core/deploy/.dockerignore.ci, mirroring what
# docker-ci-smoke.sh does before its `docker build`.
run: |
cp packages/app-core/deploy/.dockerignore.ci .dockerignore
- uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c
with:
driver-opts: |
env.BUILDKIT_STEP_LOG_MAX_SIZE=10485760
env.BUILDKIT_STEP_LOG_MAX_SPEED=10485760
# github.repository preserves the org's casing (elizaOS/eliza), but Docker
# tags must be all-lowercase. metadata-action lowercases for the push tags,
# but the raw :ci-boot-verify tag below needs an explicit lowercased image
# name or `docker build`/`docker push` fails with "repository name must be
# lowercase".
- id: image
run: echo "name=${REGISTRY}/${IMAGE_NAME,,}" >> "$GITHUB_OUTPUT"
- id: meta
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# develop push → :develop + :sha-<short>
# main push → :stable + :latest + :sha-<short>
# release → the git tag verbatim (e.g. v2.0.0-alpha.176)
# Keeping :latest on main avoids breaking the four code sites that
# still hardcode it (containers-env fallback, sandbox-provider
# fallback, two test fixtures) until they migrate to :stable.
tags: |
type=raw,value=develop,enable=${{ github.ref == 'refs/heads/develop' }}
type=raw,value=stable,enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=ref,event=tag
type=sha,prefix=sha-,format=short
# Build the image into the local Docker daemon first (load, do NOT push).
# The boot-verification step below runs the REAL agent entrypoint against
# this exact image, and the push step ships these same loaded bytes by
# retagging the local image. This is the gate that stops a container which
# crashes on startup (missing runtime deps, broken entrypoint) from
# reaching ghcr. `provenance: false` keeps the loaded image a single-
# platform image so the retag-and-push below publishes the same content
# that was verified (no second build, no "latest yt-dlp" drift between the
# verified and published image).
- id: build-local
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a
with:
context: .
file: packages/app-core/deploy/Dockerfile.ci
push: false
load: true
provenance: false
tags: ${{ steps.image.outputs.name }}:ci-boot-verify
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64
# Registry-backed build cache (checked first), with the old GHA
# cache kept as a read-only fallback. The GHA backend shares one
# repo-wide 10GB budget with every other workflow; this image's
# mode=max cache is multi-GB per run, so it was evicted before the
# next develop push — recent runs show ZERO cached layers while
# still spending 4+ min per run exporting a cache nothing ever
# read (which is also why cache-to no longer writes to gha). The
# ghcr :buildcache ref persists indefinitely and dedupes by blob
# digest, so stable layers (apt/tailscale/yt-dlp/tsx base, and the
# manifest-keyed runtime-deps npm closure) hit reliably and the
# per-run cache export only uploads blobs ghcr doesn't already
# have. image-manifest+oci-mediatypes make the cache manifest a
# ghcr-compatible OCI artifact.
cache-from: |
type=registry,ref=${{ steps.image.outputs.name }}:buildcache
type=gha
cache-to: type=registry,ref=${{ steps.image.outputs.name }}:buildcache,mode=max,image-manifest=true,oci-mediatypes=true
- name: Verify image boots (real agent entrypoint)
# Runs the production entrypoint (node --import
# /opt/tsx/node_modules/tsx/dist/loader.mjs packages/agent/dist/bin.js
# start) inside the just-built image and fails RED if the agent crashes
# on startup, exits non-zero, logs a known crash signature (Cannot find
# package/module, Failed to start, crashed during init, ...), never
# serves /api/health within the timeout, or crashes during the
# post-health stability window. A failure here blocks the push below, so
# a non-bootable image is never published.
env:
DOCKER_IMAGE: ${{ steps.image.outputs.name }}:ci-boot-verify
BOOT_VERIFY_ONLY: "true"
SMOKE_TIMEOUT_SEC: "180"
BOOT_STABLE_WINDOW_SEC: "15"
SMOKE_PORT: "32138"
CONTAINER_PORT: "42138"
# Enforce the boot-KPI cold-readyMs budget (#8812 item 5). The budget
# (boot.coldReadyMs, 25000ms) carries ~5x headroom over the ~5s real
# cold boot, and emit_boot_kpi downgrades a breach to a warning when the
# runner is heavily contended, so this only fails on a genuine multi-x
# boot-time regression — not on transient runner load.
BOOT_KPI_ENFORCE: "1"
run: bash packages/app-core/scripts/docker-ci-smoke.sh --boot-verify-only
# Push the EXACT verified image. This runs only if the boot verification
# above succeeded (steps run in order; a failed step fails the job). We
# retag the locally-loaded :ci-boot-verify image to each computed tag and
# `docker push` it, so the published bytes are byte-for-byte the image
# that just booted cleanly. No rebuild means no chance of shipping a
# different image than the one that was verified.
- id: push
env:
LOCAL_IMAGE: ${{ steps.image.outputs.name }}:ci-boot-verify
TAGS: ${{ steps.meta.outputs.tags }}
run: |
set -euo pipefail
first_tag=""
while IFS= read -r tag; do
[ -z "$tag" ] && continue
[ -z "$first_tag" ] && first_tag="$tag"
echo "Tagging $LOCAL_IMAGE as $tag and pushing"
docker tag "$LOCAL_IMAGE" "$tag"
docker push "$tag"
done <<< "$TAGS"
# Resolve the published digest from a pushed tag (its RepoDigests is
# populated by the push above) for the attestation step.
digest=""
if [ -n "$first_tag" ]; then
digest="$(docker inspect --format '{{ index .RepoDigests 0 }}' "$first_tag" 2>/dev/null | sed 's/.*@//' || true)"
fi
echo "digest=$digest" >> "$GITHUB_OUTPUT"
- name: Generate artifact attestation
if: ${{ steps.push.outputs.digest != '' }}
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373
with:
subject-name: ${{ steps.image.outputs.name }}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
- name: Make Docker image public
# Idempotent — patching to public when already public is a no-op.
# Previously lived in image.yaml; folded in here on consolidation.
run: |
curl \
-X PATCH \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/user/packages/container/${{ env.IMAGE_NAME }}/visibility \
-d '{"visibility":"public"}'
+250
View File
@@ -0,0 +1,250 @@
name: Build Android
on:
workflow_dispatch:
inputs:
upload_to_release:
description: "Upload APK/AAB to latest GitHub release"
required: false
type: boolean
default: false
concurrency:
group: build-android-${{ github.ref }}
cancel-in-progress: true
env:
APP_DIR: packages/app
ANDROID_DIR: packages/app-core/platforms/android
NODE_VERSION: "24.15.0"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
build-android:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
permissions:
actions: read
contents: write
steps:
- name: Checkout (with the llama.cpp fork submodule)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# The elizavoice JNI wrapper compiles against the omnivoice headers
# inside plugins/plugin-local-inference/native/llama.cpp — the staged
# fused lib exports the fused-voice ABI, so the headers must be
# checked out or build:android fails at [elizavoice-jni] (#15540).
submodules: recursive
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Setup JDK 21
uses: actions/setup-java@ad2b38190b15e4d6bdf0c97fb4fca8412226d287
with:
distribution: temurin
java-version: "21"
- name: Setup Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699
- name: Cache Gradle
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: gradle-${{ runner.os }}-${{ hashFiles('packages/app-core/platforms/android/**/*.gradle*', 'packages/app-core/platforms/android/gradle/wrapper/gradle-wrapper.properties', 'packages/app/capacitor.config.ts') }}
restore-keys: gradle-${{ runner.os }}-
- name: Install dependencies
run: bun install
# Gradle's copyForkLlamaLib hard-requires the fused inference lib for
# arm64-v8a (libelizainference.so + its backend .so siblings) and fails
# the whole APK build without it (#15540). CI has no ~/.eliza staging
# and no NDK cross-compile here — the fused libs are built by the
# build-llama-ffi-android workflow. Native artifacts are valid only for
# the exact producer inputs in this checkout; mixing a newer or older
# llama pin with these JNI headers can link successfully and fail later
# on-device.
- name: Stage fused inference libs (from build-llama-ffi-android)
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
producer_inputs=(
"plugins/plugin-local-inference/native/llama.cpp"
".github/workflows/build-llama-ffi-android.yml"
"packages/app-core/scripts/aosp/compile-libllama.mjs"
"packages/app-core/scripts/aosp/compile-libllama-paths.mjs"
)
run_id=""
while IFS=$'\t' read -r candidate source_sha; do
compatible=true
for path in "${producer_inputs[@]}"; do
local_sha=$(git rev-parse "HEAD:$path")
source_sha_for_path=$(gh api "repos/${{ github.repository }}/contents/$path?ref=$source_sha" \
--jq '.sha // empty' 2>/dev/null || true)
if [ "$source_sha_for_path" != "$local_sha" ]; then
compatible=false
break
fi
done
if [ "$compatible" = true ]; then
run_id="$candidate"
break
fi
done < <(gh api "repos/${{ github.repository }}/actions/workflows/build-llama-ffi-android.yml/runs?status=success&per_page=20" \
--jq '.workflow_runs[] | [.id, .head_sha] | @tsv')
if [ -z "$run_id" ]; then
echo "::error::no successful build-llama-ffi-android run matches this checkout's native producer inputs — build that workflow first" >&2
exit 1
fi
echo "staging fused libs from input-compatible build-llama-ffi-android run $run_id"
stage="$RUNNER_TEMP/elizainference"
mkdir -p "$stage/arm64" "$stage/x86_64"
gh run download "$run_id" -R "${{ github.repository }}" \
-n libelizainference-android-arm64-vulkan-fused -D "$stage/arm64"
# x86_64 is optional: gradle skips absent non-arm64 ABIs.
gh run download "$run_id" -R "${{ github.repository }}" \
-n libelizainference-android-x86_64-cpu-fused -D "$stage/x86_64" || true
arm64_so=$(find "$stage/arm64" -name libelizainference.so | head -1)
if [ -z "$arm64_so" ]; then
echo "::error::arm64 artifact downloaded but contains no libelizainference.so" >&2
find "$stage/arm64" -type f | head -20 >&2
exit 1
fi
echo "ELIZA_MTP_ANDROID_LIBDIR=$(dirname "$arm64_so")" >> "$GITHUB_ENV"
x86_so=$(find "$stage/x86_64" -name libelizainference.so 2>/dev/null | head -1 || true)
if [ -n "$x86_so" ]; then
echo "ELIZA_MTP_ANDROID_LIBDIR_X86_64=$(dirname "$x86_so")" >> "$GITHUB_ENV"
fi
echo "arm64 libdir contents:"
ls -la "$(dirname "$arm64_so")"
- name: Prepare Android platform
working-directory: ${{ env.APP_DIR }}
run: bun run build:android
- name: Run Android developer-install preflight
working-directory: ${{ env.APP_DIR }}
run: bun run preflight:android:sideload
- name: Override Gradle JDK path for CI
working-directory: ${{ env.ANDROID_DIR }}
run: |
# Remove the hardcoded local JDK path so Gradle uses JAVA_HOME from setup-java
sed -i '/org.gradle.java.home=/d' gradle.properties
- name: Determine version
id: version
run: |
if [[ "${{ github.event_name }}" == "release" ]]; then
VERSION="${{ github.event.release.tag_name }}"
VERSION="${VERSION#v}"
else
VERSION="0.0.0-dev"
fi
echo "version=$VERSION" >> $GITHUB_OUTPUT
# Compute versionCode from semver (major*10000 + minor*100 + patch)
IFS='.-' read -r major minor patch _ <<< "$VERSION"
VERSION_CODE=$(( ${major:-0} * 10000 + ${minor:-0} * 100 + ${patch:-0} ))
echo "version_code=$VERSION_CODE" >> $GITHUB_OUTPUT
- name: Build debug APK
if: github.event_name == 'workflow_dispatch'
working-directory: ${{ env.ANDROID_DIR }}
env:
ELIZAOS_VERSION_NAME: ${{ steps.version.outputs.version }}
ELIZAOS_VERSION_CODE: ${{ steps.version.outputs.version_code }}
run: ./gradlew assembleDebug --no-daemon
- name: Build release APK & AAB
if: github.event_name == 'release'
working-directory: ${{ env.ANDROID_DIR }}
env:
ELIZAOS_VERSION_NAME: ${{ steps.version.outputs.version }}
ELIZAOS_VERSION_CODE: ${{ steps.version.outputs.version_code }}
run: |
./gradlew assembleRelease --no-daemon
./gradlew bundleRelease --no-daemon
- name: Sign release APK
if: github.event_name == 'release' && env.ANDROID_KEYSTORE_BASE64 != ''
env:
ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }}
ANDROID_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
ANDROID_KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }}
ANDROID_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD }}
run: |
echo "$ANDROID_KEYSTORE_BASE64" | base64 -d > /tmp/keystore.jks
UNSIGNED_APK="${{ env.ANDROID_DIR }}/app/build/outputs/apk/release/app-release-unsigned.apk"
SIGNED_APK="${{ env.ANDROID_DIR }}/app/build/outputs/apk/release/eliza-${{ steps.version.outputs.version }}.apk"
${ANDROID_HOME}/build-tools/35.0.0/apksigner sign \
--ks /tmp/keystore.jks \
--ks-pass "pass:${ANDROID_KEYSTORE_PASSWORD}" \
--ks-key-alias "${ANDROID_KEY_ALIAS}" \
--key-pass "pass:${ANDROID_KEY_PASSWORD}" \
--out "$SIGNED_APK" \
"$UNSIGNED_APK"
rm /tmp/keystore.jks
- name: Generate Android checksums
if: github.event_name == 'release'
run: |
find "${{ env.ANDROID_DIR }}/app/build/outputs" \
\( -name "*.apk" -o -name "*.aab" \) -type f -print0 \
| sort -z \
| xargs -0 sha256sum > "${{ env.ANDROID_DIR }}/app/build/outputs/SHA256SUMS.txt"
cat "${{ env.ANDROID_DIR }}/app/build/outputs/SHA256SUMS.txt"
- name: Upload debug APK artifact
if: github.event_name == 'workflow_dispatch'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: eliza-android-debug
path: ${{ env.ANDROID_DIR }}/app/build/outputs/apk/debug/app-debug.apk
if-no-files-found: error
retention-days: 7
- name: Upload release artifacts
if: github.event_name == 'release'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: eliza-android-release
path: |
${{ env.ANDROID_DIR }}/app/build/outputs/apk/release/*.apk
${{ env.ANDROID_DIR }}/app/build/outputs/bundle/release/*.aab
${{ env.ANDROID_DIR }}/app/build/outputs/SHA256SUMS.txt
if-no-files-found: error
retention-days: 30
- name: Upload to GitHub Release
if: github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.upload_to_release)
uses: softprops/action-gh-release@718ea10b132b3b2eba29c1007bb80653f286566b
with:
tag_name: ${{ github.event_name == 'release' && github.event.release.tag_name || '' }}
files: |
${{ env.ANDROID_DIR }}/app/build/outputs/apk/release/*.apk
${{ env.ANDROID_DIR }}/app/build/outputs/bundle/release/*.aab
${{ env.ANDROID_DIR }}/app/build/outputs/apk/debug/app-debug.apk
${{ env.ANDROID_DIR }}/app/build/outputs/SHA256SUMS.txt
fail_on_unmatched_files: false
+164
View File
@@ -0,0 +1,164 @@
name: Build elizaOS Debian Package
on:
workflow_call:
workflow_dispatch:
release:
types: [created]
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
build-deb:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
permissions:
# contents: write covers the existing softprops/action-gh-release
# release upload + checkout.
# id-token + attestations are required by
# actions/attest-build-provenance@v4 to mint a Sigstore-signed
# SLSA build provenance for the .deb via GitHub OIDC.
contents: write
id-token: write
attestations: write
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Install Debian build tools
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends \
dpkg-dev \
devscripts \
debhelper \
dh-make \
fakeroot \
lintian
- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Install dependencies
run: bun install --frozen-lockfile || bun install --no-frozen-lockfile
- name: Build
run: bun run build
- name: Build .deb package
id: deb
run: |
set -euo pipefail
if [ ! -d packages/app-core/packaging/debian ]; then
echo "ERROR: packages/app-core/packaging/debian is missing — debian packaging dir is required for the .deb build. Repo state is bad."
exit 1
fi
# dpkg-buildpackage expects the source root to have a debian/ subdir
# alongside the source it needs to build. The repo keeps the
# packaging definitions under packages/app-core/packaging/debian to
# avoid clutter; link them into the app-core source root for the
# build, then unlink so a follow-up run is idempotent.
cd packages/app-core
if [ ! -L debian ]; then
ln -sfn packaging/debian debian
fi
dpkg-buildpackage -us -uc -b -d
rm -f debian
- name: Find and checksum .deb
id: find_deb
run: |
set -euo pipefail
DEB=$(find . -maxdepth 3 -name "*.deb" | head -1)
if [ -z "$DEB" ]; then
echo "ERROR: no .deb file produced after dpkg-buildpackage"
exit 1
fi
sha256sum "$DEB" > "${DEB}.sha256"
DEB_SHA256=$(cut -d' ' -f1 "${DEB}.sha256")
DEB_SIZE=$(stat --format="%s" "$DEB")
echo "deb_path=$DEB" >> "$GITHUB_OUTPUT"
echo "deb_sha256=$DEB_SHA256" >> "$GITHUB_OUTPUT"
echo "deb_size=$DEB_SIZE" >> "$GITHUB_OUTPUT"
echo "Found .deb: $DEB ($DEB_SIZE bytes)"
- name: Lint .deb with lintian
# lintian is installed in the "Install Debian build tools" step above.
# Findings are surfaced as GitHub annotations but do NOT fail the build —
# this is a visibility step. Flip to --fail-on-error once noise is acceptable.
if: steps.find_deb.outputs.deb_path != ''
run: |
set +e
lintian -i --pedantic --no-tag-display-limit "${{ steps.find_deb.outputs.deb_path }}" > lintian.log 2>&1
LINTIAN_EXIT=$?
set -e
while IFS= read -r line; do
case "$line" in
E:*) echo "::error::lintian: $line" ;;
W:*) echo "::warning::lintian: $line" ;;
I:*) echo "::notice::lintian: $line" ;;
P:*) echo "::notice::lintian: $line" ;;
esac
done < lintian.log
echo "lintian exit code: $LINTIAN_EXIT (annotations only; not failing build)"
- name: Upload lintian log
if: always() && steps.find_deb.outputs.deb_path != ''
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: elizaos-debian-lintian-log
path: lintian.log
if-no-files-found: ignore
retention-days: 30
- name: Attest SLSA build provenance for .deb
# Mints a Sigstore-signed in-toto/SLSA provenance attestation
# tied to this repo + workflow + commit, via GitHub OIDC.
# No long-lived secrets. Users verify with:
# gh attestation verify <deb-file> --owner elizaOS
if: steps.find_deb.outputs.deb_path != ''
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373
with:
subject-path: ${{ steps.find_deb.outputs.deb_path }}
- name: Upload .deb artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: elizaos-debian-package
path: |
**/*.deb
**/*.deb.sha256
if-no-files-found: error
retention-days: 30
- name: Upload to GitHub Release
if: (github.event_name == 'release') && steps.find_deb.outputs.deb_path != ''
uses: softprops/action-gh-release@718ea10b132b3b2eba29c1007bb80653f286566b
with:
files: |
**/*.deb
**/*.deb.sha256
fail_on_unmatched_files: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Write .deb metadata to summary
run: |
{
echo "## elizaOS Debian Package"
echo "| Field | Value |"
echo "| ----- | ----- |"
if [ -n "${{ steps.find_deb.outputs.deb_path }}" ]; then
echo "| File | \`${{ steps.find_deb.outputs.deb_path }}\` |"
echo "| SHA-256 | \`${{ steps.find_deb.outputs.deb_sha256 }}\` |"
echo "| Size | ${{ steps.find_deb.outputs.deb_size }} bytes |"
else
echo "| Status | not produced (packaging/debian may not be present yet) |"
fi
} >> "$GITHUB_STEP_SUMMARY"
@@ -0,0 +1,354 @@
name: Build Example App Images
# Builds + pushes standalone example-app images as a smoke/publish lane. The
# #9300 real-staging showcase loop deploys these apps from source through the
# normal app deploy builder (`POST /api/v1/apps/:id/deploy` with repo/ref/
# Dockerfile), not by pinning these tags operator-side.
#
# Why we bundle on the runner instead of building ./Dockerfile from scratch:
# server.ts imports `@elizaos/cloud-sdk` (workspace:*). That only resolves
# inside the monorepo, and the SDK's dist/ is gitignored. A standalone docker
# build of ./Dockerfile would crash at boot ("Cannot find package
# @elizaos/cloud-sdk"). Instead we run `bun build server.ts` on the runner —
# where the workspace IS resolvable — which inlines the SDK into a single
# self-contained server.js (verified: 0 residual @elizaos/cloud-sdk imports),
# then ship that with Dockerfile.bundle from a tiny context. Same pre-build
# sequence build-agent-image.yml uses to make the cloud-sdk dist available.
on:
push:
branches: [develop]
paths:
- "packages/examples/cloud/edad/**"
- "packages/examples/cloud/clone-ur-crush/**"
- "packages/cloud/sdk/**"
- ".github/workflows/build-example-app-images.yml"
workflow_dispatch:
# One in-flight build per ref; a newer push cancels the stale one (avoids the
# queue pile-up the agent-image build learned the hard way).
concurrency:
group: build-example-edad-${{ github.ref }}
cancel-in-progress: true
env:
REGISTRY: ghcr.io
IMAGE_NAME: elizaos/example-edad
# Least privilege by default; the build job opts into packages: write.
permissions:
contents: read
jobs:
build-edad:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Cache Bun install
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.bun/install/cache
key: bun-${{ runner.os }}-canary-${{ hashFiles('bun.lock') }}
restore-keys: |
bun-${{ runner.os }}-canary-
bun-${{ runner.os }}-
- name: Install dependencies
# Creates the @elizaos/cloud-sdk workspace symlink edad resolves through.
# --no-frozen-lockfile mirrors build-agent-image.yml: a downstream
# lockfile drift on develop shouldn't short-circuit the install.
env:
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: bun install --no-frozen-lockfile
- name: Build @elizaos/cloud-sdk
# The SDK's `exports.import` points at ./dist/index.js, which is
# gitignored (absent on a cold checkout). Without this build, the
# `bun build` below fails to resolve "@elizaos/cloud-sdk" even with the
# workspace symlink in place. build-agent-image.yml builds this same
# package for the same reason.
env:
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: |
cd packages/cloud/sdk
bun install --no-frozen-lockfile --ignore-scripts
bun run build
- name: Bundle the EDAD server (inline the workspace SDK)
# Produces a single self-contained server.js with @elizaos/cloud-sdk
# inlined — no workspace dep, no bun install needed in the image. Output
# goes to ./dist (already gitignored, same dir edad's `build` script
# uses); dist/ (server.js + public/) is the tiny docker build context.
run: |
set -euo pipefail
cd packages/examples/cloud/edad
node ../../../../packages/scripts/rm-path-recursive.mjs ./dist
bun build server.ts --outdir=./dist --target=bun --format=esm
cp -r public ./dist/public
test -f ./dist/server.js
# Guard: the bundle must NOT still reference the workspace dep, else
# the image would crash at boot.
if grep -q "@elizaos/cloud-sdk" ./dist/server.js; then
echo "::error::server.js still references @elizaos/cloud-sdk — bundle did not inline the workspace dep" >&2
exit 1
fi
- name: Smoke-test the image (gate the GHCR push on a working container)
# Build + run the exact bundle image and assert it actually serves before
# we publish it — so the showcase deploy never pulls a broken :showcase
# tag. Mirrors the local verification in DEPLOY_AND_VALIDATE.md.
run: |
set -euo pipefail
assert_asset() {
local asset_path="$1"
local expected_type="$2"
local min_bytes="$3"
local headers
local body
headers="$(mktemp)"
body="$(mktemp)"
curl -fsS -D "$headers" -o "$body" "http://127.0.0.1:3000/${asset_path}"
if ! grep -qi "content-type: ${expected_type}" "$headers"; then
echo "::error::${asset_path} content-type was not ${expected_type}" >&2
cat "$headers" >&2
exit 1
fi
local bytes
bytes="$(wc -c < "$body" | tr -d ' ')"
if [ "$bytes" -lt "$min_bytes" ]; then
echo "::error::${asset_path} was too small (${bytes} bytes)" >&2
exit 1
fi
}
docker build --load -t edad-smoke:ci \
-f packages/examples/cloud/edad/Dockerfile.bundle \
packages/examples/cloud/edad/dist
docker rm -f edad-smoke >/dev/null 2>&1 || true
trap 'docker rm -f edad-smoke >/dev/null 2>&1 || true' EXIT
docker run -d --name edad-smoke -p 3000:3000 edad-smoke:ci
ok=""
for _ in $(seq 1 30); do
if curl -fsS http://127.0.0.1:3000/health >/dev/null 2>&1; then ok=1; break; fi
sleep 1
done
if [ -z "$ok" ]; then
echo "::error::EDAD image never became healthy" >&2
docker logs edad-smoke || true
exit 1
fi
test "$(curl -fsS http://127.0.0.1:3000/health)" = "ok"
curl -fsS http://127.0.0.1:3000/ | grep -q "eDad" \
|| { echo "::error::EDAD did not serve its UI" >&2; exit 1; }
curl -fsS http://127.0.0.1:3000/ | grep -q 'content="http://127.0.0.1:3000/og-image.png"' \
|| { echo "::error::EDAD did not emit an absolute social image URL" >&2; exit 1; }
curl -fsS http://127.0.0.1:3000/api/config | grep -q "cloud_url" \
|| { echo "::error::EDAD /api/config is broken" >&2; exit 1; }
assert_asset dad-portrait.svg image/svg+xml 4000
assert_asset dad-avatar.svg image/svg+xml 2000
assert_asset you-avatar.svg image/svg+xml 500
assert_asset favicon.svg image/svg+xml 500
assert_asset favicon.png image/png 1000
assert_asset apple-touch-icon.png image/png 4000
assert_asset og-image.png image/png 100000
docker rm -f edad-smoke >/dev/null 2>&1 || true
trap - EXIT
echo "EDAD image smoke test passed: healthy + serves UI + /api/config + generated assets."
- uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c
- id: meta
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# :showcase → the moving tag the showcase spec deploys
# :sha-<short> → immutable per-commit tag (mirrors build-agent-image.yml)
tags: |
type=raw,value=showcase
type=sha,prefix=sha-,format=short
- id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a
with:
context: packages/examples/cloud/edad/dist
file: packages/examples/cloud/edad/Dockerfile.bundle
push: true
provenance: false
platforms: linux/amd64
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Make image public
# The deploy puller pulls anonymously; keep the package public.
# Idempotent — patching to public when already public is a no-op.
run: |
curl --fail-with-body --show-error --silent \
-X PATCH \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/orgs/elizaOS/packages/container/example-edad/visibility \
-d '{"visibility":"public"}' || \
echo "::warning::Could not set package visibility to public — set it once manually in the GHCR package settings"
build-clone-ur-crush:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Cache Bun install
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.bun/install/cache
key: bun-${{ runner.os }}-canary-${{ hashFiles('bun.lock') }}
restore-keys: |
bun-${{ runner.os }}-canary-
bun-${{ runner.os }}-
- name: Install dependencies
env:
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: bun install --no-frozen-lockfile
- name: Build clone-ur-crush (Next.js standalone bundle)
# next.config sets output:"standalone"; with outputFileTracingRoot at the
# monorepo root the server lands at the repo-relative path inside the
# bundle. Next does NOT copy static/public into standalone, so we place
# them next to server.js where the standalone runtime serves them.
#
# CRITICAL (#9300): build.mjs builds under a temp distDir
# (.next-build-<pid>) then renames it to .next, but the standalone server
# BAKES that original distDir name into required-server-files.json and
# serves /_next/static from <bakedDistDir>/static. Copying static to the
# literal .next/static (the renamed dir) puts it at a path the runtime
# never reads, so every CSS/JS chunk 404s. Copy static to the BAKED dist
# dir instead.
env:
NODE_LLAMA_CPP_SKIP_DOWNLOAD: "true"
run: |
set -euo pipefail
cd packages/examples/cloud/clone-ur-crush
node ../../../../packages/scripts/rm-path-recursive.mjs .next
bun run build
app=.next/standalone/packages/examples/cloud/clone-ur-crush
test -f "$app/server.js"
# The standalone's baked distDir = the dir holding required-server-files.json.
rsf="$(find "$app" -maxdepth 2 -name required-server-files.json | head -1)"
test -n "$rsf"
baked="$(basename "$(dirname "$rsf")")"
echo "standalone baked distDir: $baked"
mkdir -p "$app/$baked/static"
cp -r .next/static/. "$app/$baked/static/"
cp -r public "$app/public"
- name: Smoke-test the image (gate the GHCR push on a working container)
# Build + run the exact standalone image and assert it actually serves its
# UI before we publish it, so the showcase deploy never pulls a broken
# :showcase tag. Mirrors the EDAD job's gate above (DoD: failures surface
# loudly; the showcase deploy can never pull a broken tag).
run: |
set -euo pipefail
docker build --load -t cuc-smoke:ci \
-f packages/examples/cloud/clone-ur-crush/Dockerfile.bundle \
packages/examples/cloud/clone-ur-crush/.next/standalone
docker rm -f cuc-smoke >/dev/null 2>&1 || true
trap 'docker rm -f cuc-smoke >/dev/null 2>&1 || true' EXIT
docker run -d --name cuc-smoke -p 3000:3000 cuc-smoke:ci
ok=""
for _ in $(seq 1 40); do
if curl -fsS http://127.0.0.1:3000/ >/dev/null 2>&1; then ok=1; break; fi
sleep 1
done
if [ -z "$ok" ]; then
echo "::error::Clone Ur Crush image never started serving" >&2
docker logs cuc-smoke || true
exit 1
fi
home="$(curl -fsS http://127.0.0.1:3000/)"
echo "$home" | grep -q "Clone Your Crush" \
|| { echo "::error::Clone Ur Crush did not serve its UI" >&2; docker logs cuc-smoke || true; exit 1; }
# Assert a real /_next/static chunk serves 200 — this is what catches the
# distDir/static path mismatch (a broken bundle serves HTML but 404s
# every CSS/JS asset). Pull a static URL out of the rendered HTML.
asset="$(printf '%s' "$home" | grep -oE '/_next/static/[^"]+\.(js|css)' | head -1)"
test -n "$asset" || { echo "::error::no /_next/static asset referenced in HTML" >&2; exit 1; }
code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:3000${asset}")"
test "$code" = "200" \
|| { echo "::error::static asset ${asset} returned ${code} (expected 200) — broken /_next/static path" >&2; docker logs cuc-smoke || true; exit 1; }
# OG card must exist (was a 404 before #9300 fix).
og="$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:3000/og-image.png)"
test "$og" = "200" || { echo "::error::og-image.png returned ${og} (expected 200)" >&2; exit 1; }
docker rm -f cuc-smoke >/dev/null 2>&1 || true
trap - EXIT
echo "Clone Ur Crush image smoke test passed: serves UI + /_next/static (${asset}) + og-image.png."
- uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c
- id: meta
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9
with:
images: ${{ env.REGISTRY }}/elizaos/example-clone-ur-crush
tags: |
type=raw,value=showcase
type=sha,prefix=sha-,format=short
- id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a
with:
context: packages/examples/cloud/clone-ur-crush/.next/standalone
file: packages/examples/cloud/clone-ur-crush/Dockerfile.bundle
push: true
provenance: false
platforms: linux/amd64
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Make image public
run: |
curl --fail-with-body --show-error --silent \
-X PATCH \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/orgs/elizaOS/packages/container/example-clone-ur-crush/visibility \
-d '{"visibility":"public"}' || \
echo "::warning::Could not set package visibility to public — set it once manually in the GHCR package settings"
+224
View File
@@ -0,0 +1,224 @@
name: Build iOS
on:
workflow_dispatch:
inputs:
upload_to_release:
description: "Upload IPA to latest GitHub release"
required: false
type: boolean
default: false
concurrency:
group: build-ios-${{ github.ref }}
cancel-in-progress: true
env:
APP_DIR: packages/app
NODE_VERSION: "24.15.0"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
build-ios:
runs-on: macos-15
timeout-minutes: 45
permissions:
contents: write
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Select latest Xcode
run: |
XCODE_PATH=$(ls -d /Applications/Xcode*.app 2>/dev/null | sort -V | tail -1)
echo "Using Xcode at: $XCODE_PATH"
sudo xcode-select -s "$XCODE_PATH"
xcodebuild -version
- name: Cache CocoaPods
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ${{ env.APP_DIR }}/ios/App/Pods
key: pods-${{ runner.os }}-${{ hashFiles('packages/app-core/platforms/ios/App/Podfile', 'packages/app/capacitor.config.ts', 'packages/app/package.json') }}
restore-keys: pods-${{ runner.os }}-
- name: Install dependencies
run: bun install
- name: Build web assets
working-directory: ${{ env.APP_DIR }}
run: bun run build
- name: Build Capacitor plugins
working-directory: ${{ env.APP_DIR }}
run: bun run plugin:build
- name: Sync Capacitor iOS
working-directory: ${{ env.APP_DIR }}
run: bun run cap:sync:ios
- name: Overlay iOS native files
run: node packages/app-core/scripts/run-mobile-build.mjs ios-overlay
- name: Install CocoaPods
working-directory: ${{ env.APP_DIR }}/ios/App
run: pod install
- name: Run iOS developer-install preflight
working-directory: ${{ env.APP_DIR }}
run: bun run preflight:ios:sideload
- name: Determine version
id: version
run: |
if [[ "${{ github.event_name }}" == "release" ]]; then
VERSION="${{ github.event.release.tag_name }}"
VERSION="${VERSION#v}"
else
VERSION="0.0.0-dev"
fi
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: Set version in Info.plist
working-directory: ${{ env.APP_DIR }}/ios/App/App
run: |
/usr/libexec/PlistBuddy -c "Set :CFBundleShortVersionString ${{ steps.version.outputs.version }}" Info.plist
/usr/libexec/PlistBuddy -c "Set :CFBundleVersion ${{ github.run_number }}" Info.plist
# --- Code signing (only when secrets are configured) ---
- name: Install Apple certificate and provisioning profile
if: env.IOS_CERTIFICATE_BASE64 != ''
env:
IOS_CERTIFICATE_BASE64: ${{ secrets.IOS_CERTIFICATE_BASE64 }}
IOS_CERTIFICATE_PASSWORD: ${{ secrets.IOS_CERTIFICATE_PASSWORD }}
IOS_PROVISION_PROFILE_BASE64: ${{ secrets.IOS_PROVISION_PROFILE_BASE64 }}
KEYCHAIN_PASSWORD: ${{ github.run_id }}
run: |
# Create temporary keychain
KEYCHAIN_PATH=$RUNNER_TEMP/app-signing.keychain-db
security create-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"
security set-keychain-settings -lut 21600 "$KEYCHAIN_PATH"
security unlock-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"
# Import certificate
CERTIFICATE_PATH=$RUNNER_TEMP/build_certificate.p12
echo "$IOS_CERTIFICATE_BASE64" | base64 --decode -o "$CERTIFICATE_PATH"
security import "$CERTIFICATE_PATH" -P "$IOS_CERTIFICATE_PASSWORD" -A -t cert -f pkcs12 -k "$KEYCHAIN_PATH"
security set-key-partition-list -S apple-tool:,apple: -k "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"
security list-keychain -d user -s "$KEYCHAIN_PATH"
# Install provisioning profile
PROFILE_PATH=$RUNNER_TEMP/build_profile.mobileprovision
echo "$IOS_PROVISION_PROFILE_BASE64" | base64 --decode -o "$PROFILE_PATH"
mkdir -p ~/Library/MobileDevice/Provisioning\ Profiles
cp "$PROFILE_PATH" ~/Library/MobileDevice/Provisioning\ Profiles/
- name: Build unsigned archive (no signing secrets)
if: env.IOS_CERTIFICATE_BASE64 == ''
env:
IOS_CERTIFICATE_BASE64: ${{ secrets.IOS_CERTIFICATE_BASE64 }}
working-directory: ${{ env.APP_DIR }}/ios/App
run: |
xcodebuild archive \
-workspace App.xcworkspace \
-scheme App \
-configuration Release \
-archivePath $RUNNER_TEMP/Eliza.xcarchive \
CODE_SIGN_IDENTITY="" \
CODE_SIGNING_REQUIRED=NO \
CODE_SIGNING_ALLOWED=NO \
-allowProvisioningUpdates \
| xcbeautify || true
- name: Build signed archive
if: env.IOS_CERTIFICATE_BASE64 != ''
env:
IOS_CERTIFICATE_BASE64: ${{ secrets.IOS_CERTIFICATE_BASE64 }}
working-directory: ${{ env.APP_DIR }}/ios/App
run: |
xcodebuild archive \
-workspace App.xcworkspace \
-scheme App \
-configuration Release \
-archivePath $RUNNER_TEMP/Eliza.xcarchive \
-allowProvisioningUpdates \
| xcbeautify || true
- name: Export IPA
if: env.IOS_CERTIFICATE_BASE64 != ''
env:
IOS_CERTIFICATE_BASE64: ${{ secrets.IOS_CERTIFICATE_BASE64 }}
IOS_EXPORT_METHOD: ${{ secrets.IOS_EXPORT_METHOD || 'ad-hoc' }}
IOS_TEAM_ID: ${{ secrets.IOS_TEAM_ID }}
IOS_PROVISION_PROFILE_NAME: ${{ secrets.IOS_PROVISION_PROFILE_NAME }}
run: |
# Create export options plist
cat > $RUNNER_TEMP/ExportOptions.plist << PLIST
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key>
<string>${IOS_EXPORT_METHOD}</string>
<key>teamID</key>
<string>${IOS_TEAM_ID}</string>
<key>provisioningProfiles</key>
<dict>
<key>ai.eliza.app</key>
<string>${IOS_PROVISION_PROFILE_NAME}</string>
</dict>
</dict>
</plist>
PLIST
xcodebuild -exportArchive \
-archivePath $RUNNER_TEMP/Eliza.xcarchive \
-exportOptionsPlist $RUNNER_TEMP/ExportOptions.plist \
-exportPath $RUNNER_TEMP/export
# Rename IPA with version
mv $RUNNER_TEMP/export/App.ipa $RUNNER_TEMP/export/Eliza-${{ steps.version.outputs.version }}.ipa
- name: Upload archive artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: eliza-ios-${{ env.IOS_CERTIFICATE_BASE64 != '' && 'release' || 'unsigned' }}
path: |
$RUNNER_TEMP/Eliza.xcarchive
$RUNNER_TEMP/export/*.ipa
if-no-files-found: warn
retention-days: ${{ env.IOS_CERTIFICATE_BASE64 != '' && 30 || 7 }}
env:
IOS_CERTIFICATE_BASE64: ${{ secrets.IOS_CERTIFICATE_BASE64 }}
- name: Confirm iOS distribution policy
if: (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.upload_to_release)) && env.IOS_CERTIFICATE_BASE64 != ''
env:
IOS_CERTIFICATE_BASE64: ${{ secrets.IOS_CERTIFICATE_BASE64 }}
run: |
{
echo "## iOS artifact policy"
echo ""
echo "Signed IPA files are kept as private workflow artifacts for TestFlight/App Store traceability."
echo "They are not uploaded to public GitHub Releases. Public iOS distribution must use TestFlight or the App Store."
} >> "$GITHUB_STEP_SUMMARY"
- name: Cleanup keychain
if: always()
run: |
security delete-keychain $RUNNER_TEMP/app-signing.keychain-db 2>/dev/null || true
+349
View File
@@ -0,0 +1,349 @@
name: Build elizaOS Linux ISO
# One canonical live distro at packages/os/linux. The build
# runs inside that Tails-derived Docker builder. riscv64 support is verified as
# a native artifact + GUI contract until full riscv64 ISO boot evidence is
# produced for this canonical distro.
on:
workflow_call:
inputs:
channel:
type: string
default: stable
publish:
type: boolean
default: false
workflow_dispatch:
inputs:
channel:
description: Release channel
type: choice
options: [nightly, beta, stable]
default: nightly
publish:
description: Upload to GitHub Release
type: boolean
default: false
schedule:
- cron: '0 3 * * *'
release:
types: [created]
permissions:
contents: read
jobs:
build-iso:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 240
# arm64 builds run under qemu-user-static + binfmt_misc emulation on
# the amd64 runner. They are 3-5× slower than native, so they are
# gated behind `continue-on-error: true` while the path stabilizes —
# the amd64 nightly is the only required check today, so an arm64
# failure should not turn the workflow red. The per-arch experimental
# flag is read by `continue-on-error` below + by per-step `if:`s
# downstream that only run their amd64-only steps when arch == amd64.
continue-on-error: ${{ matrix.experimental }}
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64]
include:
- arch: amd64
experimental: false
- arch: arm64
experimental: true
permissions:
# contents: write covers softprops/action-gh-release (release upload)
# and anchore/sbom-action (artifact attachment). The workflow-level
# default is read; override here at the job level so the privilege
# is scoped as tightly as possible.
# id-token + attestations are required by
# actions/attest-build-provenance@v4 to mint a Sigstore-signed
# SLSA build provenance for the ISO via GitHub OIDC.
contents: write
id-token: write
attestations: write
env:
CHANNEL: ${{ inputs.channel || (github.event_name == 'release' && 'stable') || 'nightly' }}
LINUX_DIR: packages/os/linux
# Picked up by packages/os/linux/build.sh + tails/auto/config to
# drive --platform / TARGETARCH / --architecture / --linux-flavours
# and the per-arch bootloader package set. Defaults to amd64 inside
# build.sh too, so a bare run still works.
ELIZAOS_ARCH: ${{ matrix.arch }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
- name: Restore live-build cache
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: |
${{ env.LINUX_DIR }}/cache
key: elizaos-iso-${{ matrix.arch }}-${{ hashFiles('packages/os/linux/tails/auto/config', 'packages/os/linux/tails/config/chroot_local-packageslists/**', 'packages/os/linux/Dockerfile') }}
restore-keys: elizaos-iso-${{ matrix.arch }}-
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c
- name: Detect KVM acceleration
id: kvm
run: |
if [ -e /dev/kvm ]; then
echo "have_kvm=true" >> "$GITHUB_OUTPUT"
else
echo "have_kvm=false" >> "$GITHUB_OUTPUT"
fi
- name: Install just + ripgrep
# ripgrep (rg) is a hard dependency of packages/os/linux/scripts/static-smoke.sh
# (the "Verify riscv64 contract" step). It is no longer pre-installed on the
# ubuntu-24.04 runner image, so the step used to build every riscv64 target
# successfully and then exit 1 with the opaque "ripgrep (rg) is required …"
# line. Install it explicitly here so the smoke check can run.
run: sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends just ripgrep
- name: Setup Bun
# `just build` depends on `just elizaos-app`, whose ELIZAOS_BUILD_APP=1
# in-CI path bootstraps the workspace and compiles the Electrobun
# desktop artifact with bun. The runner image ships no bun, so the
# recipe died with `bun: command not found` (exit 127) the moment the
# static-smoke gate stopped masking it. Same pinned action + version
# the test workflows use.
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "1.3.14"
- name: Register qemu-user-static binfmt (non-amd64 only)
if: matrix.arch != 'amd64'
# tonistiigi/binfmt is the canonical container image for registering
# qemu-user-static handlers with the kernel's binfmt_misc on the
# GitHub Actions amd64 runner. After this step, `docker run
# --platform linux/${arch}` transparently emulates the target arch
# for everything in the container, which is what packages/os/linux/
# build.sh does via `docker build --platform linux/${ARCH}`.
# Emulated builds are 3-5× slower than native; that is why this
# job is gated on continue-on-error: true while it stabilizes.
run: |
docker run --rm --privileged tonistiigi/binfmt --install all
# Verify the handler we actually need is registered, fail fast
# if registration silently no-op'd.
test -e "/proc/sys/fs/binfmt_misc/qemu-${{ matrix.arch == 'arm64' && 'aarch64' || matrix.arch }}"
echo "✓ qemu-user-static registered for ${{ matrix.arch }}"
- name: Verify riscv64 contract
run: |
set -euo pipefail
bash scripts/verify-riscv64-buildpaths.sh
cd "${LINUX_DIR}"
ELIZAOS_STATIC_SOURCE_ONLY=1 ./scripts/static-smoke.sh
- name: Build ISO (${{ matrix.arch }})
working-directory: ${{ env.LINUX_DIR }}
# ELIZAOS_BUILD_APP=1 lets `just elizaos-app` (the dependency of
# `just build`) compile the Electrobun desktop artifact in-CI when
# no pre-staged ${ELIZAOS_APP_ARTIFACT} is provided. Without this
# flag the recipe exits before staging, so the chroot hook fails
# with "ERROR: /opt/elizaos-artifacts missing" — the root cause
# of 8+ days of nightly red on this workflow.
#
# NODE_OPTIONS lifts the V8 heap so the Vite renderer build can
# bundle the full elizaOS plugin graph; the default 4 GB heap
# OOMs on the renderer step (observed locally on a 30 GB host).
env:
ELIZAOS_BUILD_APP: 1
ELIZAOS_DOCKER_BUILDX_GHA_CACHE: 1
ELIZAOS_DOCKER_BUILDX_CACHE_SCOPE: elizaos-linux-iso-${{ matrix.arch }}
NODE_OPTIONS: --max-old-space-size=12288
run: just build
timeout-minutes: 180
- name: Locate and rename ISO
id: iso
env:
# Fail closed if the ISO exceeds this budget. Current amd64 builds
# land around 1.9 GB, so 3 GiB leaves ~50% headroom without letting
# a silent regression bloat the image. Override via env if a build
# intentionally needs more room.
ISO_MAX_BYTES: ${{ env.ELIZAOS_ISO_MAX_BYTES || '3221225472' }}
run: |
set -euo pipefail
ISO=$(find "${LINUX_DIR}/out" \( -name "binary.iso" -o -name "elizaos-live-${{ matrix.arch }}-*.iso" -o -name "elizaos-linux-${{ matrix.arch }}-*.iso" \) | sort | tail -1)
if [ -z "$ISO" ]; then
echo "ERROR: ISO not found after build"
exit 1
fi
DATE=$(date +%Y.%m.%d)
DEST="elizaos-live-${CHANNEL}-${DATE}-${{ matrix.arch }}.iso"
cp "$ISO" "$DEST"
SIZE=$(stat --format=%s "$DEST")
if [ "$SIZE" -gt "$ISO_MAX_BYTES" ]; then
echo "::error::ISO above size budget: ${SIZE} bytes > ${ISO_MAX_BYTES} bytes. Set ELIZAOS_ISO_MAX_BYTES if the increase is intentional."
exit 1
fi
sha256sum "$DEST" > "$DEST.sha256"
{
echo "path=$DEST"
echo "filename=$DEST"
echo "sha256=$(sha256sum "$DEST" | cut -d' ' -f1)"
echo "size=$SIZE"
} >> "$GITHUB_OUTPUT"
- name: Smoke test ISO (headless QEMU, amd64 only)
if: matrix.arch == 'amd64' && steps.kvm.outputs.have_kvm == 'true'
run: |
set -euo pipefail
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends qemu-system-x86 ovmf
SERIAL_LOG=$(mktemp)
timeout 180 qemu-system-x86_64 \
-enable-kvm -cdrom "${{ steps.iso.outputs.path }}" -boot d \
-m 2G -smp 2 -display none -vga none \
-serial "file:$SERIAL_LOG" -no-reboot -nographic 2>/dev/null || true
# Tiered match: a strong signal proves elizaOS-specific services
# came up; a generic match proves the kernel/userspace booted but
# the agent stack did not (surfaced as a warning so the build
# doesn't regress while the issue is investigated).
#
# Strong regex covers two formats:
# 1. Description-based "(Starting|Started) elizaOS <word>..."
# — what modern systemd (v245+, Debian trixie ships v257)
# prints because every elizaos-*.service has
# Description=elizaOS <...>.
# 2. Unit-name "elizaos-<name>.service" — appears in service
# exit / failure lines and on older systemd.
#
# Note: elizaos-launcher and elizaos-chat-overlay are user-scope
# services (WantedBy=graphical-session.target) and will not start
# in this headless QEMU run (-display none). The system-scope
# services (agent, first-boot) are the
# meaningful signals in this headless smoke test.
if grep -qE '(Starting|Started) elizaOS [A-Za-z]|elizaos-(agent|launcher|first-boot|chat-overlay)\.service' "$SERIAL_LOG" 2>/dev/null; then
echo "ISO smoke test: elizaOS service markers detected"
elif grep -qi 'elizaOS\|eliza\|login:' "$SERIAL_LOG" 2>/dev/null; then
echo "::warning::Boot reached generic strings but no elizaos-*.service marker detected — verify the agent stack started."
echo "ISO smoke test: weak match (boot OK, elizaOS service markers missing)"
else
echo "ERROR: boot strings not detected"
tail -200 "$SERIAL_LOG" || true
exit 1
fi
rm -f "$SERIAL_LOG"
- name: Smoke test skipped (no KVM)
if: steps.kvm.outputs.have_kvm != 'true'
run: echo "::notice::Boot smoke test runs only on amd64 with KVM; skipped for ${{ matrix.arch }}."
- name: Attest SLSA build provenance for ISO
# Mints a Sigstore-signed in-toto/SLSA provenance attestation
# tied to this repo + workflow + commit, via GitHub OIDC.
# No long-lived secrets. Users verify with:
# gh attestation verify <iso-file> --owner elizaOS
# Gated on amd64 because that's the only arch with a validated
# artifact today — same gate as the QEMU smoke test.
if: matrix.arch == 'amd64'
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373
with:
subject-path: ${{ steps.iso.outputs.path }}
- name: Install squashfs-tools (for SBOM extraction)
# Only amd64 produces an SBOM; arm64/riscv64 are continue-on-error
# builds with no validated artifact yet.
if: matrix.arch == 'amd64'
run: sudo apt-get install -y --no-install-recommends squashfs-tools
- name: Extract Debian package database from ISO
# Mount the ISO read-only and unsquashfs only /var/lib/dpkg so syft
# can inventory the runtime packages. Extracting the full squashfs
# root would cost ~7 GB and ~5 min; the dpkg dir alone is ~50 MB and
# lets syft enumerate every installed Debian package with name +
# version.
if: matrix.arch == 'amd64'
id: sbom-extract
run: |
set -euo pipefail
MNT=$(mktemp -d)
SQ=$(mktemp -d)
sudo mount -o loop,ro "${{ steps.iso.outputs.path }}" "$MNT"
# Parentheses group the -name alternation so -maxdepth applies to both.
SQUASH=$(sudo find "$MNT" -maxdepth 4 \( -name 'filesystem.squashfs' -o -name '*.squashfs' \) 2>/dev/null | head -1)
if [ -z "$SQUASH" ]; then
echo "::warning::No squashfs found in ISO — skipping SBOM generation"
sudo umount "$MNT"
rmdir "$MNT" "$SQ"
echo "extracted=false" >> "$GITHUB_OUTPUT"
exit 0
fi
sudo unsquashfs -d "$SQ" "$SQUASH" var/lib/dpkg > /dev/null
sudo umount "$MNT"
rmdir "$MNT"
sudo chown -R "$USER:$USER" "$SQ"
echo "root=$SQ" >> "$GITHUB_OUTPUT"
echo "extracted=true" >> "$GITHUB_OUTPUT"
- name: Generate ISO SBOM (SPDX 2.3 JSON)
# anchore/sbom-action drives syft under the hood and emits SPDX 2.3
# JSON by default. upload-artifact and upload-release-assets are
# disabled here; we attach the file ourselves below so that it lands
# in the same named artifact as the ISO and SHA-256 checksum.
if: matrix.arch == 'amd64' && steps.sbom-extract.outputs.extracted == 'true'
uses: anchore/sbom-action@e22c389904149dbc22b58101806040fa8d37a610
with:
path: ${{ steps.sbom-extract.outputs.root }}
format: spdx-json
output-file: ${{ steps.iso.outputs.filename }}.spdx.json
upload-artifact: false
upload-release-assets: false
- name: Report SBOM package count
if: matrix.arch == 'amd64' && steps.sbom-extract.outputs.extracted == 'true'
run: |
set -euo pipefail
SBOM="${{ steps.iso.outputs.filename }}.spdx.json"
if [ -f "$SBOM" ]; then
COUNT=$(jq '.packages | length' "$SBOM")
echo "::notice::SBOM contains $COUNT packages"
fi
# Best-effort cleanup; do not fail the job if the temp dir is gone.
node "$GITHUB_WORKSPACE/packages/scripts/rm-path-recursive.mjs" "${{ steps.sbom-extract.outputs.root }}" || true
- name: Upload ISO artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: elizaos-live-iso-${{ env.CHANNEL }}-${{ matrix.arch }}
path: |
${{ steps.iso.outputs.path }}
${{ steps.iso.outputs.filename }}.sha256
${{ steps.iso.outputs.filename }}.spdx.json
if-no-files-found: warn
retention-days: 7
- name: Upload to GitHub Release
if: github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.publish) || (github.event_name == 'workflow_call' && inputs.publish == true)
uses: softprops/action-gh-release@718ea10b132b3b2eba29c1007bb80653f286566b
with:
files: |
${{ steps.iso.outputs.path }}
${{ steps.iso.outputs.filename }}.sha256
${{ steps.iso.outputs.filename }}.spdx.json
tag_name: ${{ github.event.release.tag_name || github.ref_name }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Write ISO metadata to summary
run: |
{
echo "## elizaOS Live ISO (${{ matrix.arch }})"
echo "| Field | Value |"
echo "| ----- | ----- |"
echo "| File | \`${{ steps.iso.outputs.filename }}\` |"
echo "| SHA-256 | \`${{ steps.iso.outputs.sha256 }}\` |"
echo "| Size | ${{ steps.iso.outputs.size }} bytes |"
echo "| Channel | ${CHANNEL} |"
} >> "$GITHUB_STEP_SUMMARY"
@@ -0,0 +1,203 @@
name: Build libelizainference (Android, FFI)
# Builds the Android NDK shared library `libelizainference.so` for the
# four supported ABIs and ships them as a per-ABI JNI bundle. The FFI
# streaming runner (`ffi-streaming-runner.ts`) dlopens this artifact
# from the APK's `lib/<abi>/` directory.
#
# Why FFI on Android: same reason as iOS — `child_process.spawn` is
# unavailable inside the app sandbox, the Play Store policy forbids
# downloading native binaries at runtime, and a 1030ms HTTP round-trip
# per token is unacceptable on a battery-constrained device. See
# `runtime-target.ts` for the decision rules.
#
# Builds in CI at build time (no eliza-archive prebuilt dependency). The fused
# lib is the real `libelizainference.so` produced by `compile-libllama.mjs` via
# the zig/musl cross-compile — NOT a renamed `libllama.so`, and NOT a bionic NDK
# build (bun's in-process FFI can only `dlopen` a musl-linked .so; see the header
# of `compile-libllama.mjs`). `workflow_call` lets the app build invoke this and
# consume the uploaded artifact directly instead of syncing from eliza-archive.
on:
workflow_call:
inputs:
abi_filter:
required: false
type: string
default: "arm64-v8a,x86_64"
workflow_dispatch:
inputs:
abi_filter:
description: "Comma-separated subset of ABIs to build (default: all)"
required: false
type: string
default: "arm64-v8a,x86_64"
push:
branches: [develop]
paths:
- "plugins/plugin-local-inference/native/llama.cpp/**"
- "packages/app-core/scripts/aosp/compile-libllama.mjs"
- ".github/workflows/build-llama-ffi-android.yml"
tags:
- "llama-ffi-android-*"
concurrency:
group: build-llama-ffi-android-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
NODE_VERSION: "24"
BUN_VERSION: "canary"
ANDROID_API: "26"
# Pinned to the NDK r26d-equivalent the rest of the app builds against;
# bumping this requires syncing with `packages/app/electrobun/android/`.
ANDROID_NDK_VERSION: "26.3.11579264"
jobs:
prepare-matrix:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
matrix: ${{ steps.set.outputs.matrix }}
steps:
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
id: set
with:
script: |
const inputs = context.payload.inputs ?? {};
const filterRaw = (inputs.abi_filter ?? "arm64-v8a,x86_64").trim();
const filter = new Set(filterRaw.split(",").map((s) => s.trim()).filter(Boolean));
// compile-libllama.mjs fused targets: arm64 gets the Vulkan GPU
// backend (the QJL/Polar/Mali-FA-mitigation kernels live here),
// x86_64 (emulator/cuttlefish) is CPU-only. armeabi-v7a (32-bit)
// is unsupported by the fused build and dropped.
const all = [
{ abi: "arm64-v8a", target: "android-arm64-vulkan-fused" },
{ abi: "x86_64", target: "android-x86_64-cpu-fused" },
];
const include = all.filter((m) => filter.has(m.abi));
core.setOutput("matrix", JSON.stringify({ include }));
build:
name: ${{ matrix.abi }}
needs: prepare-matrix
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.prepare-matrix.outputs.matrix) }}
steps:
- name: Checkout (with the llama.cpp fork submodule — the canonical source)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# The repo pins the fork via the submodule gitlink; build from that,
# not compile-libllama's LLAMA_CPP_COMMIT default (which can diverge
# from the gitlink and may not be a fetchable remote ref).
submodules: recursive
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install build deps (cmake, ninja, Vulkan-Hpp headers)
run: |
set -euo pipefail
# The runner image ships a packages.microsoft.com apt source that
# intermittently 403s / "is no longer signed", which fails apt-get
# update under pipefail even though this build needs nothing from it.
# Drop any MS apt source (filename varies by image) before updating.
for f in $(sudo grep -rliE 'packages\.microsoft\.com' /etc/apt/sources.list.d/ 2>/dev/null || true); do
sudo rm -f "$f"
done
sudo apt-get update -o Acquire::Retries=5 -o Acquire::http::Timeout=30
# libvulkan-dev provides /usr/include/vulkan/vulkan.hpp (arch-independent;
# compile-libllama searches /usr/include for it on Linux hosts).
sudo apt-get install -y cmake ninja-build build-essential git python3 libvulkan-dev
cmake --version
- name: Install zig 0.13 (the cross-compiler compile-libllama uses)
run: |
set -euo pipefail
# Pinned to 0.13 — the version the build is tested against; newer zig
# (0.16) regresses the musl lld link. See compile-libllama.mjs header.
curl -fsSL "https://ziglang.org/download/0.13.0/zig-linux-x86_64-0.13.0.tar.xz" -o /tmp/zig.tar.xz
mkdir -p "$HOME/.zig" && tar -xf /tmp/zig.tar.xz -C "$HOME/.zig" --strip-components=1
echo "$HOME/.zig" >> "$GITHUB_PATH"
"$HOME/.zig/zig" version
- name: Install Android NDK (sysroot + glslc for the Vulkan build)
uses: nttld/setup-ndk@ed92fe6cadad69be94a966a7ee3271275e62f779
id: setup-ndk
with:
ndk-version: r26d
add-to-path: false
- name: Build the real fused libelizainference.so (${{ matrix.target }})
env:
# compile-libllama's Vulkan resolver expects an SDK root with
# <ANDROID_HOME>/ndk/<numeric-version>/ (it lists ndk/, keeps dirs
# matching /^\d+\./, takes the highest, and treats it as the NDK root).
# nttld/setup-ndk (add-to-path:false, link-to-sdk:false) installs into
# the tool cache and exposes the NDK root via outputs.ndk-path — NOT in
# an <sdk>/ndk/<version> layout — so ${{ steps.setup-ndk.outputs.ndk-path }}/../..
# has no numeric ndk/ subdir and the resolver threw. Stitch a clean
# single-NDK SDK below and point ANDROID_HOME at it.
ANDROID_HOME: ${{ runner.temp }}/android-sdk
run: |
set -euo pipefail
# ndk/<numeric-version> -> the installed NDK root. r26d == 26.3.11579264
# (keep in sync with the ndk-version pin on the setup-ndk step above).
# A clean dir holding only this NDK guarantees the resolver's
# "highest numeric dir wins" picks r26d, independent of whatever NDK the
# runner image happens to preinstall.
mkdir -p "$ANDROID_HOME/ndk"
ln -sfn "${{ steps.setup-ndk.outputs.ndk-path }}" "$ANDROID_HOME/ndk/26.3.11579264"
# Builds the REAL fused FFI lib (libelizainference.so) via the
# zig/musl cross-compile — bun's in-process FFI can only dlopen a
# musl-linked .so, which the bionic NDK cmake path could not produce.
# --assets-dir redirects the staged output into
# artifacts/libelizainference/<target>/<abi>/libelizainference.so so the
# Verify + Upload steps below find it (the script otherwise defaults to
# packages/app/android/app/src/main/assets/agent/<abi>/).
# --src-dir builds from the in-repo fork submodule (the gitlink the
# repo pins + that `submodules: recursive` fetched) rather than
# compile-libllama's LLAMA_CPP_COMMIT default, which is a divergent
# local-only ref that is not fetchable in CI.
node packages/app-core/scripts/aosp/compile-libllama.mjs --target ${{ matrix.target }} --src-dir plugins/plugin-local-inference/native/llama.cpp --assets-dir artifacts/libelizainference/${{ matrix.target }}
- name: Verify fused artifact (FFI symbols + Mali FA mitigation on arm64)
run: |
set -euo pipefail
SO=$(find artifacts/libelizainference/${{ matrix.target }} -name "libelizainference.so" | head -1)
test -n "$SO" || { echo "no libelizainference.so produced"; find artifacts -name "*.so"; exit 1; }
file "$SO"; ls -lh "$SO"
# The fused FFI must export the eliza_inference_* voice ABI.
if ! nm -D "$SO" 2>/dev/null | grep -q "eliza_inference_"; then
echo "WARNING: eliza_inference_* symbols not visible via nm (stripped?)"
fi
if [ "${{ matrix.abi }}" = "arm64-v8a" ]; then
VK=$(find artifacts/libelizainference/${{ matrix.target }} -name "libggml-vulkan.so*" | head -1)
if [ -n "$VK" ] && strings "$VK" | grep -q GGML_VK_FA_ALLOW_SUBGROUPS; then
echo "OK: arm64 Vulkan lib carries the Mali flash-attn mitigation (#9508)"
else
echo "WARNING: GGML_VK_FA_ALLOW_SUBGROUPS marker not found in $VK"
fi
fi
- name: Upload libelizainference artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: libelizainference-${{ matrix.target }}
path: artifacts/libelizainference/${{ matrix.target }}/**
if-no-files-found: error
retention-days: 30
+144
View File
@@ -0,0 +1,144 @@
name: Build libelizainference (iOS, FFI static)
# Builds the iOS / iPadOS / simulator XCFramework wrapping llama.cpp +
# the FFI streaming surface from `scripts/omnivoice-fuse/ffi.h`. The
# resulting `.xcframework` is linked into the Eliza app bundle so the
# in-process FFI streaming runner can dlopen it at runtime.
#
# Why an XCFramework: iOS does not allow loading arbitrary `.dylib` at
# runtime from the app sandbox, and `child_process.spawn` is unavailable
# (App Store review + sandbox restrictions). The FFI path is mandatory
# on mobile (see `runtime-target.ts` decision rules) and the bun:ffi
# loader on iOS opens the framework binary that ships inside the .ipa.
#
# This workflow reuses `plugins/plugin-local-inference/native/llama.cpp/build-xcframework.sh`
# as the source of truth for cmake flags so a manual local build matches
# CI exactly.
#
# STATUS: SCAFFOLD. The runner has not been validated against the
# Electrobun iOS shell's XCFramework consumption path. Top-level
# The `run_real_build` input keeps this out of the default fan-out.
on:
workflow_dispatch:
inputs:
run_real_build:
description: "Run the scaffold build once it has been verified"
required: false
type: boolean
default: false
push:
tags:
- "llama-ffi-ios-*"
concurrency:
group: build-llama-ffi-ios-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
NODE_VERSION: "24"
BUN_VERSION: "canary"
# The XCFramework script targets iOS 16.4 / macOS 13.3 by default; do
# not lower without confirming the app's `Info.plist` deployment target.
IOS_MIN_OS_VERSION: "16.4"
jobs:
build:
name: iOS XCFramework (Metal)
if: ${{ inputs.run_real_build == true }}
runs-on: macos-14
timeout-minutes: 120
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Select Xcode
run: |
set -euo pipefail
# Use the latest available; the runner image's default is fine.
xcodebuild -version
sudo xcode-select --print-path
- name: Install build deps
run: |
set -euo pipefail
if ! command -v cmake >/dev/null 2>&1; then
brew install cmake
fi
cmake --version
- name: Cache llama.cpp build artifacts
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: |
plugins/plugin-local-inference/native/llama.cpp/build-ios-device
plugins/plugin-local-inference/native/llama.cpp/build-ios-sim
plugins/plugin-local-inference/native/llama.cpp/build-macos
key: llama-ffi-ios-${{ hashFiles('plugins/plugin-local-inference/native/llama.cpp/build-xcframework.sh', 'plugins/plugin-local-inference/native/llama.cpp/CMakeLists.txt') }}
restore-keys: |
llama-ffi-ios-
- name: Build llama.xcframework
run: |
set -euo pipefail
# The upstream script handles the device + simulator + macOS
# slice fan-out. CRITICAL: we do NOT modify the script in this
# workflow — the source of truth lives in the repo so a
# developer can reproduce locally.
cd plugins/plugin-local-inference/native/llama.cpp
bash build-xcframework.sh
- name: Verify XCFramework
run: |
set -euo pipefail
XCF_DIR="plugins/plugin-local-inference/native/llama.cpp/build-apple/llama.xcframework"
if [ ! -d "$XCF_DIR" ]; then
echo "expected XCFramework missing: $XCF_DIR"
find plugins/plugin-local-inference/native/llama.cpp -maxdepth 3 -name "*.xcframework" -type d || true
exit 1
fi
ls -la "$XCF_DIR"
- name: Stage XCFramework
id: stage
run: |
set -euo pipefail
STAGE_DIR="artifacts/libelizainference/ios-arm64"
mkdir -p "$STAGE_DIR"
XCF_DIR="plugins/plugin-local-inference/native/llama.cpp/build-apple/llama.xcframework"
cp -R "$XCF_DIR" "$STAGE_DIR/llama.xcframework"
{
echo "target=ios-arm64"
echo "backend=metal"
echo "framework=llama.xcframework"
echo "ref=${{ github.ref }}"
echo "sha=${{ github.sha }}"
echo "runner=macos-14"
echo "ios_min=${{ env.IOS_MIN_OS_VERSION }}"
echo "llama_sha=$(git -C plugins/plugin-local-inference/native/llama.cpp rev-parse HEAD || echo n/a)"
} > "$STAGE_DIR/BUILD_INFO.txt"
echo "stage_dir=$STAGE_DIR" >> "$GITHUB_OUTPUT"
- name: Upload XCFramework artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: libelizainference-ios-arm64
path: ${{ steps.stage.outputs.stage_dir }}/*
if-no-files-found: error
retention-days: 30
+219
View File
@@ -0,0 +1,219 @@
name: Build libelizainference (Linux, FFI)
# Builds the in-process bun:ffi shared library `libelizainference.so` for
# linux-x64-cpu and (opt-in) linux-x64-cuda. Used by the desktop FFI
# path on Linux servers and the Cloud-hosted local-inference fallback.
#
# STATUS: SCAFFOLD. The build steps mirror the omnivoice workflow but
# the target name `libelizainference.so` has not been wired through the
# llama.cpp CMakeLists install rules yet. The `run_real_build` input keeps
# the workflow out of the default fan-out until verified.
on:
workflow_dispatch:
inputs:
run_real_build:
description: "Run the scaffold build once it has been verified"
required: false
type: boolean
default: false
run_cuda:
description: "Also build the Linux x64 CUDA artifact (self-hosted runner)"
required: false
type: boolean
default: false
push:
tags:
- "llama-ffi-*"
concurrency:
group: build-llama-ffi-linux-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
NODE_VERSION: "24"
BUN_VERSION: "canary"
jobs:
build-cpu:
name: ubuntu-22.04 cpu
if: ${{ inputs.run_real_build == true }}
runs-on: ubuntu-22.04
timeout-minutes: 90
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install Linux build deps
run: |
set -euo pipefail
APT_ARGS=(-o Acquire::ForceIPv4=true -o Acquire::Retries=5 -o Acquire::http::Timeout=30)
for attempt in 1 2 3; do
if sudo apt-get "${APT_ARGS[@]}" update && \
sudo apt-get "${APT_ARGS[@]}" install -y --fix-missing \
cmake ninja-build build-essential ccache git python3; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "apt install failed after ${attempt} attempts"
exit 1
fi
sleep $((attempt * 10))
done
cmake --version
- name: Cache llama.cpp cmake build dir
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: plugins/plugin-local-inference/native/llama.cpp/build
key: llama-ffi-linux-cpu-${{ hashFiles('plugins/plugin-local-inference/native/llama.cpp/CMakeLists.txt') }}
restore-keys: |
llama-ffi-linux-cpu-
- name: Configure llama.cpp (CPU, shared, no server)
run: |
set -euo pipefail
cmake -S plugins/plugin-local-inference/native/llama.cpp \
-B plugins/plugin-local-inference/native/llama.cpp/build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DLLAMA_BUILD_EXAMPLES=OFF \
-DLLAMA_BUILD_TOOLS=OFF \
-DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_SERVER=OFF \
-DGGML_NATIVE=OFF \
-DGGML_OPENMP=ON
- name: Build llama.cpp shared library
run: |
set -euo pipefail
cmake --build plugins/plugin-local-inference/native/llama.cpp/build --target llama -j 4
- name: Verify artifact
run: |
set -euo pipefail
ART_DIR="plugins/plugin-local-inference/native/llama.cpp/build"
if [ -f "$ART_DIR/libllama.so" ]; then
ART="$ART_DIR/libllama.so"
elif [ -f "$ART_DIR/bin/libllama.so" ]; then
ART="$ART_DIR/bin/libllama.so"
else
echo "no libllama.so found under $ART_DIR"
ls -la "$ART_DIR"
exit 1
fi
ls -lh "$ART"
file "$ART"
- name: Stage artifact (linux-x64-cpu)
id: stage
run: |
set -euo pipefail
STAGE_DIR="artifacts/libelizainference/linux-x64-cpu"
mkdir -p "$STAGE_DIR"
for cand in \
plugins/plugin-local-inference/native/llama.cpp/build/libllama.so \
plugins/plugin-local-inference/native/llama.cpp/build/bin/libllama.so; do
if [ -f "$cand" ]; then
cp "$cand" "$STAGE_DIR/libelizainference.so"
break
fi
done
{
echo "target=linux-x64-cpu"
echo "backend=cpu"
echo "lib_name=libelizainference.so"
echo "ref=${{ github.ref }}"
echo "sha=${{ github.sha }}"
echo "runner=ubuntu-22.04"
echo "llama_sha=$(git -C plugins/plugin-local-inference/native/llama.cpp rev-parse HEAD || echo n/a)"
} > "$STAGE_DIR/BUILD_INFO.txt"
echo "stage_dir=$STAGE_DIR" >> "$GITHUB_OUTPUT"
- name: Upload libelizainference artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: libelizainference-linux-x64-cpu
path: ${{ steps.stage.outputs.stage_dir }}/*
if-no-files-found: error
retention-days: 30
build-cuda:
name: ubuntu-22.04 cuda (self-hosted)
if: ${{ inputs.run_real_build == true && inputs.run_cuda == true }}
runs-on: [self-hosted, gpu-cuda-12.6]
timeout-minutes: 120
continue-on-error: true
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Probe CUDA toolchain
run: |
set -euo pipefail
nvcc --version
nvidia-smi || true
- name: Configure llama.cpp (CUDA, shared, no server)
run: |
set -euo pipefail
cmake -S plugins/plugin-local-inference/native/llama.cpp \
-B plugins/plugin-local-inference/native/llama.cpp/build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DLLAMA_BUILD_EXAMPLES=OFF \
-DLLAMA_BUILD_TOOLS=OFF \
-DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_SERVER=OFF \
-DGGML_NATIVE=OFF \
-DGGML_CUDA=ON
- name: Build llama.cpp shared library
run: |
set -euo pipefail
cmake --build plugins/plugin-local-inference/native/llama.cpp/build --target llama -j 4
- name: Stage artifact (linux-x64-cuda)
id: stage
run: |
set -euo pipefail
STAGE_DIR="artifacts/libelizainference/linux-x64-cuda"
mkdir -p "$STAGE_DIR"
for cand in \
plugins/plugin-local-inference/native/llama.cpp/build/libllama.so \
plugins/plugin-local-inference/native/llama.cpp/build/bin/libllama.so; do
if [ -f "$cand" ]; then
cp "$cand" "$STAGE_DIR/libelizainference.so"
break
fi
done
echo "stage_dir=$STAGE_DIR" >> "$GITHUB_OUTPUT"
- name: Upload libelizainference artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: libelizainference-linux-x64-cuda
path: ${{ steps.stage.outputs.stage_dir }}/*
if-no-files-found: error
retention-days: 30
+160
View File
@@ -0,0 +1,160 @@
name: Build libelizainference (macOS, FFI)
# Builds the in-process bun:ffi shared library `libelizainference.dylib`
# for darwin-arm64 (Apple Silicon Metal). The FFI streaming runner
# (`plugins/plugin-local-inference/src/services/ffi-streaming-runner.ts`)
# dlopens this artifact and binds the streaming-LLM symbols declared in
# `scripts/omnivoice-fuse/ffi-streaming-llm.h`.
#
# Mobile (iOS) ships the same C ABI built as a static library and linked
# into the app bundle — see `build-llama-ffi-ios.yml`. Desktop uses this
# dynamic library for the FFI path; the spawn / llama-server path stays
# the historical default on desktop and is unaffected by this workflow.
#
# STATUS: SCAFFOLD. The build steps are wired against the same patterns
# the omnivoice workflow uses, but the artifact path on disk and the
# fused-build entry point have not yet been verified end-to-end. The
# `run_real_build` input keeps the workflow out of the default event-trigger
# fan-out until that verification lands. Run manually via workflow_dispatch
# once verified.
on:
workflow_dispatch:
inputs:
run_real_build:
description: "Run the scaffold build once it has been verified"
required: false
type: boolean
default: false
push:
tags:
- "llama-ffi-*"
concurrency:
group: build-llama-ffi-macos-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
NODE_VERSION: "24"
BUN_VERSION: "canary"
jobs:
build:
name: macos-14 metal
if: ${{ inputs.run_real_build == true }}
runs-on: macos-14
timeout-minutes: 90
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install macOS build deps
run: |
set -euo pipefail
if ! command -v cmake >/dev/null 2>&1; then
brew install cmake
fi
if ! command -v ninja >/dev/null 2>&1; then
brew install ninja
fi
cmake --version
ninja --version
- name: Cache llama.cpp cmake build dir
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: plugins/plugin-local-inference/native/llama.cpp/build
key: llama-ffi-macos-${{ hashFiles('plugins/plugin-local-inference/native/llama.cpp/CMakeLists.txt') }}
restore-keys: |
llama-ffi-macos-
- name: Configure llama.cpp (Metal, shared, no server)
run: |
set -euo pipefail
cmake -S plugins/plugin-local-inference/native/llama.cpp \
-B plugins/plugin-local-inference/native/llama.cpp/build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DLLAMA_BUILD_EXAMPLES=OFF \
-DLLAMA_BUILD_TOOLS=OFF \
-DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_SERVER=OFF \
-DGGML_METAL=ON \
-DGGML_METAL_EMBED_LIBRARY=ON \
-DGGML_NATIVE=OFF \
-DGGML_OPENMP=OFF
- name: Build llama.cpp shared library
run: |
set -euo pipefail
# -j4 cap to stay friendly on hosted runners and predictable on
# self-hosted runners that share the box with other jobs.
cmake --build plugins/plugin-local-inference/native/llama.cpp/build --target llama -j 4
- name: Verify artifact
run: |
set -euo pipefail
ART_DIR="plugins/plugin-local-inference/native/llama.cpp/build"
if [ -f "$ART_DIR/libllama.dylib" ]; then
ART="$ART_DIR/libllama.dylib"
elif [ -f "$ART_DIR/bin/libllama.dylib" ]; then
ART="$ART_DIR/bin/libllama.dylib"
else
echo "no libllama.dylib found under $ART_DIR"
ls -la "$ART_DIR"
exit 1
fi
ls -lh "$ART"
file "$ART"
- name: Stage artifact (darwin-arm64-metal)
id: stage
run: |
set -euo pipefail
STAGE_DIR="artifacts/libelizainference/darwin-arm64-metal"
mkdir -p "$STAGE_DIR"
# The artifact lands at one of two paths depending on cmake
# generator output dir conventions. Take whichever is present.
for cand in \
plugins/plugin-local-inference/native/llama.cpp/build/libllama.dylib \
plugins/plugin-local-inference/native/llama.cpp/build/bin/libllama.dylib; do
if [ -f "$cand" ]; then
cp "$cand" "$STAGE_DIR/libelizainference.dylib"
break
fi
done
{
echo "target=darwin-arm64-metal"
echo "backend=metal"
echo "lib_name=libelizainference.dylib"
echo "ref=${{ github.ref }}"
echo "sha=${{ github.sha }}"
echo "runner=macos-14"
echo "llama_sha=$(git -C plugins/plugin-local-inference/native/llama.cpp rev-parse HEAD || echo n/a)"
} > "$STAGE_DIR/BUILD_INFO.txt"
echo "stage_dir=$STAGE_DIR" >> "$GITHUB_OUTPUT"
- name: Upload libelizainference artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: libelizainference-darwin-arm64-metal
path: ${{ steps.stage.outputs.stage_dir }}/*
if-no-files-found: error
retention-days: 30
+202
View File
@@ -0,0 +1,202 @@
name: Build elizaOS VM Image
on:
workflow_call:
inputs:
format:
type: string
default: ova
publish:
type: boolean
default: false
workflow_dispatch:
inputs:
format:
description: Output format
type: choice
options: [qcow2, ova, vmdk, all]
default: ova
publish:
description: Upload to GitHub Release
type: boolean
default: false
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
build-vm:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
permissions:
# contents: write covers the existing softprops/action-gh-release
# release upload + checkout.
# id-token + attestations are required by
# actions/attest-build-provenance@v4 to mint a Sigstore-signed
# SLSA build provenance for the .ova via GitHub OIDC.
contents: write
id-token: write
attestations: write
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Install QEMU and tools
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends \
qemu-system-x86 \
qemu-utils \
ovmf \
cloud-image-utils \
libguestfs-tools \
python3 \
just \
curl
- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Install dependencies
run: bun install --frozen-lockfile || bun install --no-frozen-lockfile
- name: Cache upstream Debian image
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: packages/os/linux/vm/disk-base/.cache
key: elizaos-vm-upstream-${{ hashFiles('packages/os/linux/vm/disk-base/mmdebstrap.recipe') }}
restore-keys: elizaos-vm-upstream-
- name: Build VM base image
working-directory: packages/os/linux
run: sudo bash vm/scripts/build-base.sh
timeout-minutes: 60
- name: Deploy elizaOS to VM
working-directory: packages/os/linux
run: bash vm/scripts/deploy.sh
- name: Run VM integration tests
working-directory: packages/os/linux
timeout-minutes: 20
run: bash vm/scripts/run-tests.sh
- name: Generate VM bundle metadata
working-directory: packages/os/linux
run: |
mkdir -p vm/output
python3 vm/scripts/generate-bundle-metadata.py \
--output-dir vm/output/bundle-metadata
- name: Convert to OVA (for VirtualBox / VMware)
id: ova
if: inputs.format == 'ova' || inputs.format == 'all' || inputs.format == 'vmdk' || inputs.format == ''
run: |
set -euo pipefail
BASE_QCOW=$(find packages/os/linux -maxdepth 3 -name "*.qcow2" | head -1)
if [ -z "$BASE_QCOW" ]; then
echo "ERROR: no qcow2 image found after VM build — OVA conversion cannot proceed"
exit 1
fi
DATE=$(date +%Y.%m.%d)
VMDK="elizaos-vm-${DATE}.vmdk"
OVA="elizaos-vm-${DATE}.ova"
OVF="elizaos-vm-${DATE}.ovf"
echo "Converting qcow2 to vmdk..."
qemu-img convert -f qcow2 -O vmdk "$BASE_QCOW" "$VMDK"
# Minimal OVF descriptor
cat > "$OVF" << EOF
<?xml version="1.0" encoding="UTF-8"?>
<Envelope xmlns="http://schemas.dmtf.org/ovf/envelope/1"
xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"
xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData">
<References>
<File ovf:href="${VMDK}" ovf:id="file1"/>
</References>
<VirtualSystem ovf:id="elizaOS">
<Name>elizaOS</Name>
<VirtualHardwareSection>
<Item>
<rasd:ElementName>4 virtual CPUs</rasd:ElementName>
<rasd:InstanceID>1</rasd:InstanceID>
<rasd:ResourceType>3</rasd:ResourceType>
<rasd:VirtualQuantity>4</rasd:VirtualQuantity>
</Item>
<Item>
<rasd:ElementName>4096 MB RAM</rasd:ElementName>
<rasd:InstanceID>2</rasd:InstanceID>
<rasd:ResourceType>4</rasd:ResourceType>
<rasd:VirtualQuantity>4096</rasd:VirtualQuantity>
</Item>
</VirtualHardwareSection>
</VirtualSystem>
</Envelope>
EOF
tar -cvf "$OVA" "$OVF" "$VMDK"
sha256sum "$OVA" > "${OVA}.sha256"
OVA_SHA256=$(cut -d' ' -f1 "${OVA}.sha256")
OVA_SIZE=$(stat --format="%s" "$OVA")
echo "ova_path=$OVA" >> "$GITHUB_OUTPUT"
echo "ova_sha256=$OVA_SHA256" >> "$GITHUB_OUTPUT"
echo "ova_size=$OVA_SIZE" >> "$GITHUB_OUTPUT"
echo "date=$DATE" >> "$GITHUB_OUTPUT"
- name: Attest SLSA build provenance for VM image
# Mints a Sigstore-signed in-toto/SLSA provenance attestation
# tied to this repo + workflow + commit, via GitHub OIDC.
# No long-lived secrets. Users verify with:
# gh attestation verify <ova-file> --owner elizaOS
# Only fires when an OVA was actually produced (the convert
# step is gated on the format input).
if: steps.ova.outputs.ova_path != ''
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373
with:
subject-path: ${{ steps.ova.outputs.ova_path }}
- name: Upload VM artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: elizaos-vm-image
path: |
elizaos-vm-*.ova
elizaos-vm-*.ova.sha256
elizaos-vm-*.qcow2
packages/os/linux/vm/output/
if-no-files-found: error
retention-days: 7
- name: Upload to GitHub Release
if: github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.publish) || (github.event_name == 'workflow_call' && inputs.publish == true)
uses: softprops/action-gh-release@718ea10b132b3b2eba29c1007bb80653f286566b
with:
files: |
elizaos-vm-*.ova
elizaos-vm-*.ova.sha256
fail_on_unmatched_files: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Write VM metadata to summary
run: |
{
echo "## elizaOS VM Image"
echo "| Field | Value |"
echo "| ----- | ----- |"
if [ -n "${{ steps.ova.outputs.ova_path }}" ]; then
echo "| OVA | \`${{ steps.ova.outputs.ova_path }}\` |"
echo "| SHA-256 | \`${{ steps.ova.outputs.ova_sha256 }}\` |"
echo "| Size | ${{ steps.ova.outputs.ova_size }} bytes |"
else
echo "| OVA | not produced |"
fi
echo "| Format | ${{ inputs.format || 'ova' }} |"
} >> "$GITHUB_STEP_SUMMARY"
+66
View File
@@ -0,0 +1,66 @@
# Cache-key stability gate
#
# Asserts that the Anthropic prompt-cache key inputs (Stage 1 / Stage 2 stable
# prefix hashes, STABLE_PLANNER_TOOLS envelope hash, HANDLE_RESPONSE +
# PLAN_ACTIONS tool envelope hashes) are byte-stable across PRs.
#
# If this job fails, the cached Anthropic prefix has been busted — every
# subsequent PR pays ~80% extra tokens until someone notices. Fix the source
# of the churn (typically a whitespace edit in a prompt template, a reordered
# action list, or an environment-dependent string that leaked into a stable
# prompt segment). Only rebase the baked-in hashes when the change is
# explicitly INTENTIONAL — see
# `docs/audits/lifeops-2026-05-11/cache-key-stability.md`.
#
# Local equivalent (run before pushing):
#
# bun run test:cache-stability
name: Cache-key Stability
on:
pull_request:
branches: [main]
paths:
- "packages/core/**"
- "plugins/plugin-personal-assistant/**"
- "packages/prompts/**"
- ".github/workflows/cache-key-stability.yml"
concurrency:
group: cache-key-stability-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
jobs:
cache-key-stability:
name: cache-key-stability
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
- name: Run cache-key stability test
run: bun run test:cache-stability
+206
View File
@@ -0,0 +1,206 @@
# Cerebras Nightly — continuous benchmark grind (Phase 8)
#
# Runs the lifeops + personality benchmark suites nightly against
# eliza / hermes / openclaw with Cerebras gemma-4-31b as both agent
# (where applicable) and judge. Produces a delta report vs the previous
# nightly's results so drift surfaces without anyone having to look.
#
# This is the workflow `Phase 8 — Continuous Cerebras grind in CI` from
# the cleanup roadmap. Goal: free judge calls, daily cadence, drift
# visibility.
#
# Required secret (job skipped, not failed, when missing):
# CEREBRAS_API_KEY — gemma-4-31b agent + judge
#
# Optional secrets:
# GITHUB_TOKEN — present by default; required to upsert the
# tracking issue comment.
name: Cerebras Nightly
on:
schedule:
# 03:00 UTC daily — Mockoon dependencies are quiet then, and the
# ~$10 daily cost lands on the right billing cycle.
- cron: "0 3 * * *"
workflow_dispatch:
inputs:
suite:
description: "Benchmark suite (lifeops | personality | both)"
required: false
type: choice
options: [both, lifeops, personality]
default: both
limit:
description: "Scenarios per agent per domain (default 10)"
required: false
type: string
default: "10"
tracking_issue:
description: "Tracking issue number for delta comments (optional)"
required: false
type: string
default: ""
concurrency:
group: cerebras-nightly-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
issues: write
actions: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
PYTHON_VERSION: "3.12"
jobs:
bench:
name: "nightly: ${{ matrix.suite }}"
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
suite: ["lifeops", "personality"]
steps:
- name: Check secrets
id: gate
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
run: |
if [ -z "${CEREBRAS_API_KEY:-}" ]; then
echo "[cerebras-nightly] CEREBRAS_API_KEY not configured — skipping."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
# Honor the workflow_dispatch suite filter.
requested="${{ github.event.inputs.suite || 'both' }}"
if [ "$requested" != "both" ] && [ "$requested" != "${{ matrix.suite }}" ]; then
echo "[cerebras-nightly] suite ${{ matrix.suite }} not requested ($requested) — skipping."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup workspace
if: steps.gate.outputs.skip != 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
- name: Install Python deps
if: steps.gate.outputs.skip != 'true'
run: |
python -m pip install --upgrade pip
# benchmark suites live in packages/benchmarks; install editable
# for each adapter we touch.
for pkg in lifeops-bench eliza-adapter hermes-adapter openclaw-adapter personality-bench; do
if [ -d "packages/benchmarks/$pkg" ]; then
python -m pip install -e "packages/benchmarks/$pkg" || true
fi
done
- name: Run nightly bench
if: steps.gate.outputs.skip != 'true'
id: bench
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
CEREBRAS_BASE_URL: "https://api.cerebras.ai/v1"
CEREBRAS_MODEL: "gemma-4-31b"
EVAL_MODEL_PROVIDER: "cerebras"
NIGHTLY_LIMIT: "${{ github.event.inputs.limit || '10' }}"
RUN_ID: "cerebras-nightly-${{ github.run_id }}"
run: |
set -euo pipefail
mkdir -p artifacts/${{ matrix.suite }}
if [ "${{ matrix.suite }}" = "lifeops" ]; then
for agent in eliza hermes openclaw; do
echo "::group::lifeops/$agent"
python -m eliza_lifeops_bench \
--agent "$agent" \
--suite full \
--limit "$NIGHTLY_LIMIT" \
--concurrency 2 \
--output "artifacts/lifeops/$agent" \
|| echo "[lifeops/$agent] non-zero exit; continuing"
echo "::endgroup::"
done
elif [ "${{ matrix.suite }}" = "personality" ]; then
for agent in eliza hermes openclaw eliza-runtime; do
echo "::group::personality/$agent"
node scripts/personality-bench-run.mjs \
--agent "$agent" \
--limit "$NIGHTLY_LIMIT" \
--output "artifacts/personality/$agent" \
|| echo "[personality/$agent] non-zero exit; continuing"
echo "::endgroup::"
done
fi
- name: Generate delta report
if: steps.gate.outputs.skip != 'true'
id: delta
run: |
set -euo pipefail
# Compare against the previous nightly's results if available.
# Stored under .github/cerebras-nightly-baseline/<suite>.json by
# the prior run via cache, or fetched from artifacts when cache
# misses. Best-effort — on miss we report absolute scores.
python scripts/cerebras-nightly-delta.py \
--suite "${{ matrix.suite }}" \
--input artifacts/${{ matrix.suite }} \
--output artifacts/${{ matrix.suite }}/delta.md \
--json-output artifacts/${{ matrix.suite }}/delta.json \
|| true
- name: Upload artifacts
if: always() && steps.gate.outputs.skip != 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: cerebras-nightly-${{ matrix.suite }}-${{ github.run_id }}
path: |
artifacts/${{ matrix.suite }}/
retention-days: 90
if-no-files-found: warn
- name: Upsert tracking issue comment
if: steps.gate.outputs.skip != 'true' && github.event.inputs.tracking_issue != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE: ${{ github.event.inputs.tracking_issue }}
SUITE: ${{ matrix.suite }}
run: |
set -euo pipefail
if [ -f "artifacts/$SUITE/delta.md" ]; then
{
printf '## Cerebras Nightly — %s — run %s\n\n' "$SUITE" "${{ github.run_id }}"
cat "artifacts/$SUITE/delta.md"
printf '\n[Full artifacts](https://github.com/%s/actions/runs/%s)\n' "${{ github.repository }}" "${{ github.run_id }}"
} > /tmp/cerebras-nightly-comment.md
gh issue comment "$ISSUE" --body-file /tmp/cerebras-nightly-comment.md \
|| echo "tracking-issue upsert failed; artifact retained"
fi
+108
View File
@@ -0,0 +1,108 @@
# certification-image — builds and publishes the GPU certification image
# (docker/certification/Dockerfile.gpu) that vast.ai certification runs boot
# (#14548, epic #14541). The image bakes everything the 16 KB onstart script
# cannot carry: CUDA llama-server, the sha256-pinned gpu-vision models,
# node24/bun/playwright/tesseract/ffmpeg. Pushes
# ghcr.io/elizaos/certification-gpu:{latest,sha-<short>} via GITHUB_TOKEN —
# no extra registry secret. Rebuilds are rare (image inputs only), so this
# triggers on dispatch plus pushes that touch its own inputs, never on PRs.
name: certification-image
on:
workflow_dispatch:
push:
branches: [develop]
paths:
- "docker/certification/Dockerfile.gpu"
- "scripts/gpu-vision/setup.mjs"
- "scripts/gpu-vision/lib.mjs"
- "scripts/gpu-vision/models.lock.json"
- ".github/workflows/certification-image.yml"
concurrency:
group: certification-image-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository_owner }}/certification-gpu
jobs:
build-and-push:
name: build-and-push
# Same fleet fallback as build-agent-image: hetzner when online (the
# image is ~15 GB with baked models and CUDA layers; hosted runners need
# the disk-free step below to survive it).
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 180
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Free up disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL || true
sudo docker image prune -a -f || true
df -h /
- uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c
with:
driver-opts: |
env.BUILDKIT_STEP_LOG_MAX_SIZE=10485760
env.BUILDKIT_STEP_LOG_MAX_SPEED=10485760
# Docker tags must be lowercase; github.repository_owner is "elizaOS".
- id: image
run: echo "name=${REGISTRY}/${IMAGE_NAME,,}" >> "$GITHUB_OUTPUT"
- id: meta
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9
with:
images: ${{ steps.image.outputs.name }}
# :latest tracks develop — it is what run-certification.mjs boots by
# default; :sha-<short> pins for reproducible re-runs.
tags: |
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/develop' }}
type=sha,prefix=sha-,format=short
- name: Build and push
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a
with:
context: .
file: docker/certification/Dockerfile.gpu
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# Single-manifest image: vast pulls by tag, provenance attestation
# manifests confuse some of its pullers.
provenance: false
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Image summary
run: |
{
echo "## certification-image"
echo ""
echo "Pushed tags:"
echo '```'
echo "${{ steps.meta.outputs.tags }}"
echo '```'
echo ""
echo "Boot it on vast: \`node scripts/vast/run-certification.mjs --image ${{ steps.image.outputs.name }}:latest ...\`"
} >> "$GITHUB_STEP_SUMMARY"
+199
View File
@@ -0,0 +1,199 @@
# certification-vast — dispatches an automated full-tier certification run on
# a rented vast.ai GPU (#14548, epic #14541). Thin caller of
# scripts/vast/run-certification.mjs: search offers → create from the
# prebuilt certification image → onstart runs the packages/evidence chain
# (bundle:create → certify:rollup → certify:sign) → poll → pull the signed
# certification.json → DESTROY the instance in the script's own finally.
#
# Triggers: workflow_dispatch (any certifier with repo write), plus a nightly
# cron that is a no-op unless the repo variable ELIZA_VAST_CERT_ENABLED is
# the string 'true' — cost is real money, so nightly spend is owner-opt-in.
# Deliberately NO pull_request trigger: this job holds VAST_API_KEY and the
# ELIZA_CERT_SIGNING_KEY; PR-triggered code must never see either.
#
# Secrets discipline: both secrets are exposed only to the single step that
# runs the script, and the script forwards the signing key solely inside the
# create-instance env payload (never the onstart text, never logs). When vast
# fails for ANY reason the job goes red with a "local certification required"
# summary carrying the exact fallback command — same chain, same output; the
# develop→main gate (certification-verify.yml) cannot tell the difference.
name: certification-vast
on:
workflow_dispatch:
inputs:
sha:
description: "Commit sha to certify (default: the ref this workflow runs on)"
required: false
type: string
tier:
description: "Certification tier"
required: false
default: "full"
type: choice
options: [cpu, gpu, full]
max_dph:
description: "Budget cap in $/hr per offer"
required: false
default: "0.60"
type: string
max_attempts:
description: "Max offers tried before giving up"
required: false
default: "3"
type: string
timeout_minutes:
description: "Hard wall-clock cap per attempt"
required: false
default: "120"
type: string
schedule:
# Nightly on develop, 06:17 UTC (off the top-of-hour crunch); gated below
# on vars.ELIZA_VAST_CERT_ENABLED == 'true'.
- cron: "17 6 * * *"
concurrency:
group: certification-vast
cancel-in-progress: false
permissions:
contents: read
jobs:
vast-certification:
name: vast-certification
if: github.event_name == 'workflow_dispatch' || vars.ELIZA_VAST_CERT_ENABLED == 'true'
runs-on: ubuntu-latest
timeout-minutes: 400
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Resolve certified sha
id: sha
env:
INPUT_SHA: ${{ inputs.sha }}
run: |
set -euo pipefail
sha="${INPUT_SHA:-$GITHUB_SHA}"
if ! [[ "$sha" =~ ^[0-9a-fA-F]{7,40}$ ]]; then
echo "::error title=certification-vast::input sha '$sha' is not a git sha"
exit 1
fi
echo "sha=$sha" >> "$GITHUB_OUTPUT"
# Fail in seconds — before any vast spend — when the run cannot possibly
# produce a verifiable certification.
- name: Preflight secrets
env:
HAS_VAST_KEY: ${{ secrets.VAST_API_KEY != '' }}
HAS_SIGNING_KEY: ${{ secrets.ELIZA_CERT_SIGNING_KEY != '' }}
run: |
set -euo pipefail
missing=""
[ "$HAS_VAST_KEY" = "true" ] || missing="$missing VAST_API_KEY"
[ "$HAS_SIGNING_KEY" = "true" ] || missing="$missing ELIZA_CERT_SIGNING_KEY"
if [ -n "$missing" ]; then
echo "::error title=certification-vast::missing repo secret(s):$missing"
exit 1
fi
# Node 24 to match the script's target runtime (plain node, zero
# workspace deps — no bun install on this runner).
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24"
- name: Dry-run plan (no API calls, no secrets in output)
env:
CERT_SHA: ${{ steps.sha.outputs.sha }}
TIER: ${{ inputs.tier || 'full' }}
MAX_DPH: ${{ inputs.max_dph || '0.60' }}
MAX_ATTEMPTS: ${{ inputs.max_attempts || '3' }}
TIMEOUT_MINUTES: ${{ inputs.timeout_minutes || '120' }}
run: |
set -euo pipefail
node scripts/vast/run-certification.mjs \
--sha "$CERT_SHA" --tier "$TIER" \
--max-dph "$MAX_DPH" --max-attempts "$MAX_ATTEMPTS" \
--timeout-minutes "$TIMEOUT_MINUTES" \
--dry-run
# The ONLY step with secret access. The signing key travels to the
# instance exclusively inside the create-payload env; the vast key never
# leaves this runner. CERT_PUSH_CMD is optional owner-configured storage
# push (R2/S3) — without it the signed certification still comes back
# via the instance logs.
- name: Run certification on vast.ai
id: run
env:
VAST_API_KEY: ${{ secrets.VAST_API_KEY }}
ELIZA_CERT_SIGNING_KEY: ${{ secrets.ELIZA_CERT_SIGNING_KEY }}
CERT_PUSH_CMD: ${{ secrets.ELIZA_CERT_PUSH_CMD }}
CERT_SHA: ${{ steps.sha.outputs.sha }}
TIER: ${{ inputs.tier || 'full' }}
MAX_DPH: ${{ inputs.max_dph || '0.60' }}
MAX_ATTEMPTS: ${{ inputs.max_attempts || '3' }}
TIMEOUT_MINUTES: ${{ inputs.timeout_minutes || '120' }}
run: |
set -euo pipefail
node scripts/vast/run-certification.mjs \
--sha "$CERT_SHA" --tier "$TIER" \
--max-dph "$MAX_DPH" --max-attempts "$MAX_ATTEMPTS" \
--timeout-minutes "$TIMEOUT_MINUTES" \
--out vast-certification-output
# The signed certification + instance logs are the run's product; the
# promotion PR author downloads this artifact and commits
# certification.json (+ the pushed bundle) per the gate's runbook.
- name: Upload certification artifact
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: vast-certification-${{ steps.sha.outputs.sha }}
path: vast-certification-output/
if-no-files-found: ignore
retention-days: 14
- name: Success summary
if: success()
env:
CERT_SHA: ${{ steps.sha.outputs.sha }}
run: |
{
echo "## certification-vast ✅"
echo ""
echo "Signed certification for \`$CERT_SHA\` pulled from the vast instance"
echo "(artifact **vast-certification-$CERT_SHA**, cost logged in the run step)."
echo ""
echo "To use it on a promotion PR: download the artifact, commit"
echo "\`certification.json\` at the repo root and the bundle at \`evidence/bundle/\`"
echo "per [.github/certification/README.md](https://github.com/${GITHUB_REPOSITORY}/blob/develop/.github/certification/README.md)."
} >> "$GITHUB_STEP_SUMMARY"
# The documented fallback: same chain, same output, run by a human/agent
# holding the signing key. Kept in lockstep with scripts/vast/README.md.
- name: Local certification required (vast run failed)
if: failure()
env:
CERT_SHA: ${{ steps.sha.outputs.sha }}
TIER: ${{ inputs.tier || 'full' }}
run: |
{
echo "## certification-vast ❌ — local certification required"
echo ""
echo "The vast.ai run for \`$CERT_SHA\` failed (dead key / no offers / stuck instance /"
echo "onstart failure — exit code in the run step above; every path attempts instance"
echo "destroy first). Certify locally on a machine holding \`ELIZA_CERT_SIGNING_KEY\`:"
echo ""
echo '```bash'
echo "git fetch origin $CERT_SHA && git checkout $CERT_SHA"
echo "node scripts/vast/local-certify.mjs --tier $TIER"
echo '```'
echo ""
echo "Runbook (costs, kill paths, storage push): [scripts/vast/README.md](https://github.com/${GITHUB_REPOSITORY}/blob/develop/scripts/vast/README.md)"
} >> "$GITHUB_STEP_SUMMARY"
echo "::error title=certification-vast::vast run failed — local certification required: node scripts/vast/local-certify.mjs --tier $TIER (see scripts/vast/README.md)"
+265
View File
@@ -0,0 +1,265 @@
# certification-verify — the develop→main promotion gate (#14547, epic #14541).
#
# PRs targeting `main` (a production deploy) must carry a `certification.json`
# at the repo root: an Ed25519-signed claim (produced by
# `packages/evidence certify:sign`, protocol in #14546) that a trusted
# keyholder reviewed the evidence bundle for the promoted commit. This
# workflow is a THIN CALLER of `certify:verify` — zero verification logic
# lives here — plus a docs/template-only commit-drift check
# (scripts/certification/check-commit-drift.mjs) and a human-readable check
# summary showing WHO certified WHAT.
#
# Required-check name for main's branch protection: `verify-certification`
# (admin wiring + break-glass runbook: .github/certification/README.md).
#
# The `certification-verify-selftest` job runs on PRs (to main or develop)
# that touch the gate's own plumbing: it generates a throwaway keypair, signs
# a fixture bundle with the real CLIs, and asserts green-verifies + tampered
# fails — so gate changes are CI-tested without a real certification. It is
# NOT the required check; skipping its steps on unrelated PRs is safe.
name: certification-verify
on:
pull_request:
# `main` for the gate itself; `develop` only so the selftest job covers
# PRs that change the gate's plumbing (which target develop).
branches: [main, develop]
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: certification-verify-${{ github.event.pull_request.number }}
cancel-in-progress: true
permissions:
contents: read
jobs:
verify-certification:
name: verify-certification
# Draft promotion PRs skip the gate: GitHub cannot merge a draft, and the
# ready_for_review trigger re-runs the check the moment it matters.
if: github.event.pull_request.base.ref == 'main' && github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 20
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
ELIZA_SKIP_ARTIFACT_SYNC: "1"
steps:
- name: Checkout PR head
# actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# The exact head commit, not the synthetic merge ref: the
# certification binds to a real commit sha, and the drift check
# compares that sha against this head.
ref: ${{ github.event.pull_request.head.sha }}
# Full history so `git diff cert.commit..head` and the ancestry
# check resolve however far back the certified commit sits.
fetch-depth: 0
# TRUST ANCHOR — read the public key from the BASE branch (main), never
# from the PR head. If this step read the PR's own copy of the key, an
# attacker could swap the key and a matching self-signed
# certification.json in a single promotion PR and the gate would happily
# verify their own signature. Reading main's copy means a key only
# becomes trusted after it lands on main through review; until then a
# foreign-key certification fails `wrong-key`. (Rotation + bootstrap:
# .github/certification/README.md.)
- name: Read trusted public key from the base branch
run: |
set -euo pipefail
git fetch --quiet --no-tags origin "+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}"
if ! git show "refs/remotes/origin/${BASE_REF}:.github/certification/certification-public-key.pem" \
> "$RUNNER_TEMP/trusted-public-key.pem"; then
echo "::error title=certification-verify::No trusted public key at .github/certification/certification-public-key.pem on ${BASE_REF}. Until the certification stack lands on ${BASE_REF}, this gate cannot pass (bootstrap: see .github/certification/README.md)."
exit 1
fi
# Artifact convention: `certification.json` committed at the repo root,
# with the signed evidence bundle committed at `evidence/bundle/`.
# Checked before dependency install so missing artifacts fail in seconds,
# with remediation.
- name: Locate certification and evidence bundle
run: |
set -euo pipefail
if [ ! -f certification.json ]; then
{
echo "## certification-verify"
echo ""
echo "**Result:** ❌ no \`certification.json\` at the repo root of this promotion branch."
echo ""
echo "Produce one by running the vast certification workflow for this commit, or locally:"
echo ""
echo '```bash'
echo "bun run --cwd packages/evidence bundle:create -- --tier full"
echo "bun run --cwd packages/evidence certify:rollup -- --bundle <bundle-dir> --out verdicts.json"
echo "# review + edit verdicts.json, then (requires the ELIZA_CERT_SIGNING_KEY secret):"
echo "bun run --cwd packages/evidence certify:sign -- --bundle <bundle-dir> --verdicts verdicts.json --reviewer-id <you> --reviewer-kind human"
echo "cp <bundle-dir>/certification.json ./certification.json"
echo "rm -rf evidence/bundle && mkdir -p evidence && cp -R <bundle-dir> evidence/bundle"
echo '```'
echo ""
echo "Full runbook: [.github/certification/README.md](https://github.com/${GITHUB_REPOSITORY}/blob/${BASE_REF}/.github/certification/README.md)"
} >> "$GITHUB_STEP_SUMMARY"
echo "::error title=certification-verify::certification.json missing at the repo root — run the vast certification workflow or local certify:sign; see .github/certification/README.md"
exit 1
fi
if [ ! -f evidence/bundle/manifest.json ]; then
{
echo "## certification-verify"
echo ""
echo "**Result:** ❌ no evidence bundle at \`evidence/bundle/\` on this promotion branch."
echo ""
echo "The promotion gate always passes \`--bundle evidence/bundle\` to \`certify:verify\` so artifact integrity, rollup completeness, and signed evidence-path membership are re-derived from the committed bundle."
echo ""
echo '```bash'
echo "bun run --cwd packages/evidence bundle:create -- --tier full"
echo "bun run --cwd packages/evidence certify:rollup -- --bundle <bundle-dir> --out verdicts.json"
echo "bun run --cwd packages/evidence certify:sign -- --bundle <bundle-dir> --verdicts verdicts.json --reviewer-id <you> --reviewer-kind human"
echo "cp <bundle-dir>/certification.json ./certification.json"
echo "rm -rf evidence/bundle && mkdir -p evidence && cp -R <bundle-dir> evidence/bundle"
echo '```'
echo ""
echo "Full runbook: [.github/certification/README.md](https://github.com/${GITHUB_REPOSITORY}/blob/${BASE_REF}/.github/certification/README.md)"
} >> "$GITHUB_STEP_SUMMARY"
echo "::error title=certification-verify::evidence/bundle/manifest.json missing — commit the signed evidence bundle and re-run the gate"
exit 1
fi
echo "certification.json present"
echo "evidence/bundle present"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary rewrites bun.lock (#11184/#9454)
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
run-postinstall: "false" # certify:verify needs only zod; skips the multi-GB artifact sync
skip-avatar-clone: "true"
no-vision-deps: "true"
# The certification may sit a few commits behind head when only docs
# moved after signing. The helper passes iff head == cert.commit, or
# cert.commit is an ancestor and every drifted path is docs/template-only
# (docs/**, packages/docs/**, *.md, plus signed certification artifacts).
# Any source, workflow, policy, or config drift forces re-certification.
- name: Check commit drift (certification commit vs PR head)
id: drift
run: |
set -euo pipefail
status=0
node scripts/certification/check-commit-drift.mjs \
--cert "$GITHUB_WORKSPACE/certification.json" \
--head "$HEAD_SHA" \
--repo "$GITHUB_WORKSPACE" \
--json-out "$RUNNER_TEMP/drift.json" \
--github-output "$GITHUB_OUTPUT" || status=$?
echo "status=$status" >> "$GITHUB_OUTPUT"
- name: Verify certification (certify:verify)
id: verify
env:
CERT_COMMIT: ${{ steps.drift.outputs.cert-commit }}
run: |
set -euo pipefail
args=(
--cert "$GITHUB_WORKSPACE/certification.json"
--pubkey "$RUNNER_TEMP/trusted-public-key.pem"
--max-age-hours 72
--required-tier full
--json
)
# The commit binding: certify:verify checks the SIGNED commit; the
# drift step above already bound that commit to this PR head.
if [ -n "$CERT_COMMIT" ]; then
args+=( --expected-commit "$CERT_COMMIT" )
fi
# Required committed bundle: verify re-hashes every artifact and
# re-derives the mechanical rollup and evidence-path membership
# (bundle-tampered / verdict-incomplete).
args+=( --bundle "$GITHUB_WORKSPACE/evidence/bundle" )
status=0
bun run --cwd packages/evidence certify:verify -- "${args[@]}" \
> "$RUNNER_TEMP/certification-report.json" || status=$?
echo "status=$status" >> "$GITHUB_OUTPUT"
- name: Render check summary
if: ${{ !cancelled() }}
run: |
set -euo pipefail
node scripts/certification/render-check-summary.mjs \
--mode summary \
--report "$RUNNER_TEMP/certification-report.json" \
--drift "$RUNNER_TEMP/drift.json" \
--pubkey "$RUNNER_TEMP/trusted-public-key.pem" \
--cert-path "certification.json (repo root, head ${HEAD_SHA})" \
>> "$GITHUB_STEP_SUMMARY"
- name: Enforce gate result
env:
DRIFT_STATUS: ${{ steps.drift.outputs.status }}
VERIFY_STATUS: ${{ steps.verify.outputs.status }}
run: |
set -euo pipefail
node scripts/certification/render-check-summary.mjs \
--mode annotations \
--report "$RUNNER_TEMP/certification-report.json" \
--drift "$RUNNER_TEMP/drift.json"
if [ "$DRIFT_STATUS" != "0" ] || [ "$VERIFY_STATUS" != "0" ]; then
echo "::error title=certification-verify::gate failed (commit-drift status=$DRIFT_STATUS, certify:verify status=$VERIFY_STATUS) — failure codes above and in the step summary"
exit 1
fi
echo "certification verified"
certification-verify-selftest:
name: certification-verify-selftest
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
pull-requests: read
steps:
- name: Detect gate-relevant changes
id: changes
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
set -euo pipefail
files="$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files" --paginate --jq '.[].filename')"
if printf '%s\n' "$files" | grep -Eq '^(\.github/workflows/certification-verify\.yml$|scripts/certification/|packages/evidence/|\.github/certification/)'; then
echo "relevant=true" >> "$GITHUB_OUTPUT"
else
echo "relevant=false" >> "$GITHUB_OUTPUT"
echo "gate plumbing untouched by this PR; selftest not needed"
fi
- name: Checkout
if: steps.changes.outputs.relevant == 'true'
# actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup workspace dependencies
if: steps.changes.outputs.relevant == 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary rewrites bun.lock (#11184/#9454)
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
run-postinstall: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
env:
ELIZA_SKIP_ARTIFACT_SYNC: "1"
- name: Commit-drift helper tests
if: steps.changes.outputs.relevant == 'true'
run: node --test scripts/certification/check-commit-drift.test.mjs
- name: Gate plumbing selftest (keygen → sign → verify → tamper)
if: steps.changes.outputs.relevant == 'true'
run: node scripts/certification/selftest.mjs
+252
View File
@@ -0,0 +1,252 @@
name: Chat Latency Live
# Manual, privacy-safe comparison of Cerebras direct and the Eliza Cloud model
# gateway from the same GitHub-hosted runner. Production is never automatic:
# selecting it binds the job to the protected production environment.
on:
pull_request:
branches: [develop, main]
paths:
- ".github/workflows/chat-latency-live.yml"
- "packages/scripts/cloud/chat-latency.mjs"
- "packages/scripts/cloud/chat-latency.test.mjs"
- "packages/scripts/__tests__/chat-latency-workflow.test.ts"
- "package.json"
workflow_dispatch:
inputs:
environment:
description: Eliza Cloud gateway environment
required: true
default: staging
type: choice
options:
- staging
- production
repeats:
description: Repetitions for each model/reasoning case
required: true
default: "30"
type: choice
options:
- "3"
- "10"
- "30"
expected_gateway_sha:
description: Exact 40-character commit expected from /api/health
required: true
type: string
concurrency:
group: chat-latency-live-${{ github.event_name }}-${{ inputs.environment || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
jobs:
contract:
name: Probe contract
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Validate probe syntax and privacy contract
run: |
node --check packages/scripts/cloud/chat-latency.mjs
node --test packages/scripts/cloud/chat-latency.test.mjs
live:
name: paired direct + gateway (${{ inputs.environment }})
if: github.event_name == 'workflow_dispatch'
needs: contract
runs-on: ubuntu-24.04
timeout-minutes: 90
environment: ${{ inputs.environment }}
env:
ELIZA_CLOUD_CHAT_LATENCY_BASE_URL: ${{ inputs.environment == 'production' && 'https://api.elizacloud.ai' || 'https://api-staging.elizacloud.ai' }}
ELIZA_GATEWAY_DEPLOY_SHA: ${{ inputs.expected_gateway_sha }}
REPORT_PATH: reports/chat-latency-paired-${{ inputs.environment }}.jsonl
steps:
- name: Checkout exact requested ref
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Verify the exact deployed gateway SHA
env:
EXPECTED_GATEWAY_SHA: ${{ inputs.expected_gateway_sha }}
run: |
node --input-type=module - <<'NODE'
const expected = process.env.EXPECTED_GATEWAY_SHA ?? "";
const base = process.env.ELIZA_CLOUD_CHAT_LATENCY_BASE_URL ?? "";
if (!/^[a-f0-9]{40}$/.test(expected)) {
throw new Error("expected_gateway_sha must be a 40-character lowercase commit");
}
const response = await fetch(base + "/api/health", {
headers: { "user-agent": "eliza-chat-latency/1.0" },
signal: AbortSignal.timeout(30_000),
});
if (!response.ok) {
throw new Error("Gateway health returned HTTP " + response.status);
}
const body = await response.json();
if (body?.commit !== expected) {
throw new Error(
"Gateway serves " + (body?.commit ?? "<missing>") + ", expected " + expected,
);
}
console.log(
"Verified gateway deployment " +
expected +
" (" +
(body.environment ?? "unknown") +
")",
);
NODE
- name: Probe Gemma and GLM reasoning modes on one runner
id: probe
shell: bash
env:
CEREBRAS_CHAT_LATENCY_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
ELIZA_CLOUD_CHAT_LATENCY_API_KEY: ${{ secrets.ELIZACLOUD_API_KEY }}
ELIZA_GATEWAY_DEPLOY_SHA: ${{ inputs.expected_gateway_sha }}
BENCHMARK_SEED: ${{ github.run_id }}-${{ github.run_attempt }}
run: |
mkdir -p reports
set +e
# One process holds runner, clock, prompts, and case set constant.
# It randomizes case order, alternates target-first order exactly for
# each case and labels cold/warm/post-idle phases.
# It idles before every target labeled post-idle.
# Warm pairs are paced at 40 gateway requests/minute, below the
# lowest Cloud organization tier, so this remains a latency test.
node packages/scripts/cloud/chat-latency.mjs \
--target paired \
--direct-base-url https://api.cerebras.ai \
--gateway-base-url "$ELIZA_CLOUD_CHAT_LATENCY_BASE_URL" \
--direct-api-key-env CEREBRAS_CHAT_LATENCY_API_KEY \
--gateway-api-key-env ELIZA_CLOUD_CHAT_LATENCY_API_KEY \
--repeat "${{ inputs.repeats }}" \
--timeout-ms 30000 \
--idle-ms 30000 \
--pair-interval-ms 1500 \
--max-proof-miss-rate 0.05 \
--seed "$BENCHMARK_SEED" \
--case gemma-4-31b@omit@512 \
--case gemma-4-31b@none@512 \
--case zai-glm-4.7@omit@4096 \
--case zai-glm-4.7@none@4096 \
--case zai-glm-4.7@none@512 \
| tee "$REPORT_PATH"
probe_status="${PIPESTATUS[0]}"
set -e
echo "status=$probe_status" >> "$GITHUB_OUTPUT"
- name: Reverify the exact deployed gateway SHA
if: always()
env:
EXPECTED_GATEWAY_SHA: ${{ inputs.expected_gateway_sha }}
run: |
node --input-type=module - <<'NODE'
const expected = process.env.EXPECTED_GATEWAY_SHA ?? "";
const base = process.env.ELIZA_CLOUD_CHAT_LATENCY_BASE_URL ?? "";
const response = await fetch(base + "/api/health", {
headers: { "user-agent": "eliza-chat-latency/1.0" },
signal: AbortSignal.timeout(30_000),
});
if (!response.ok) {
throw new Error("Gateway health returned HTTP " + response.status);
}
const body = await response.json();
if (body?.commit !== expected) {
throw new Error(
"Gateway changed during the benchmark: served " +
(body?.commit ?? "<missing>") +
", expected " +
expected,
);
}
console.log("Gateway deployment remained exact through benchmark completion");
NODE
- name: Add privacy-safe timing table to summary
if: always() && hashFiles(env.REPORT_PATH) != ''
run: |
node --input-type=module - "$REPORT_PATH" >> "$GITHUB_STEP_SUMMARY" <<'NODE'
import { readFileSync } from "node:fs";
import { summarizeLatencyRecords } from "./packages/scripts/cloud/chat-latency.mjs";
const path = process.argv[2];
const records = readFileSync(path, "utf8")
.split(/\r?\n/)
.filter(Boolean)
.map((line) => JSON.parse(line));
const summaries = summarizeLatencyRecords(records);
console.log("### Paired chat latency: direct + gateway / ${{ inputs.environment }}");
console.log("");
console.log(
"Gateway deployment: " +
(process.env.ELIZA_GATEWAY_DEPLOY_SHA ?? "unknown"),
);
console.log("");
console.log("| target | model | reasoning | cap | ok/total | TTFT p50/p90/p95 ms | total p50/p90/p95 ms | preforward p50/p90/p95 ms |");
console.log("| --- | --- | --- | ---: | ---: | ---: | ---: | ---: |");
const fmt = (metric) =>
[metric.p50, metric.p90, metric.p95]
.map((value) => value ?? "—")
.join(" / ");
for (const row of summaries) {
const values = [
row.target,
row.model,
row.reasoningEffort,
row.maxTokens,
row.successes + "/" + row.samples,
fmt(row.firstTokenMs),
fmt(row.totalMs),
fmt(row.preforwardMs),
];
console.log("| " + values.join(" | ") + " |");
}
const phaseCounts = Object.groupBy(
records,
(row) => row.phase ?? "unlabelled",
);
console.log("");
console.log(
"Phases: " +
Object.entries(phaseCounts)
.map(([phase, rows]) => phase + "=" + rows.length)
.join(", "),
);
NODE
- name: Upload exact-SHA latency evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: chat-latency-paired-${{ inputs.environment }}-${{ inputs.expected_gateway_sha }}
path: ${{ env.REPORT_PATH }}
if-no-files-found: error
retention-days: 14
- name: Enforce probe result
if: always()
run: |
status="${{ steps.probe.outputs.status }}"
if [ -z "$status" ]; then
echo "::error::Probe did not produce a status."
exit 1
fi
exit "$status"
+232
View File
@@ -0,0 +1,232 @@
name: Chat shell gestures
# Real-browser gesture + interleaving e2e for the floating chat shell
# (packages/ui ContinuousChatOverlay + home screen), the component-tree
# render-parity contract, and the CHAT_GESTURE_COVERAGE gate (#12188).
#
# All 11 __e2e__ runners compose the shared runner toolkit
# (packages/ui/src/testing/e2e-runner) and record a stable-named .webm. This lane
# drives the REAL surfaces with real pointer/touch gestures in headless chromium,
# runs the boot-free coverage gate that fails when a gesture-handler site lacks a
# matrix row, and uploads every runner's video + screenshots.
#
# One shell runner is intentionally NOT in this lane:
# - run-fused-wake-integration-e2e drives the REAL native desktop wake producer
# (libwakeword + the wakeword-cpp build), which this lane does not compile
# (install-native-deps:false) — it self-skips without the lib, so it lives in
# the native-desktop build lane.
#
# run-perf-gate-e2e USED to be excluded here for a CLS 0.80 failure, but that
# shift was the conversation-swipe layout thrash: #13531 removed swipe (one
# infinite thread) and #13826 retargeted the gate onto scroll + maximize/restore,
# so CLS is now 0.0000 on develop (stable across runs, not timing-flaky). It is
# re-enabled below (#14333) — CLS is structural so it will not flake on slower
# fleet hardware, and the frame budget carries ~3x margin (p95 ~10ms ≤ 33ms).
# Both gates self-verify their detector's teeth: each injects a real non-transient
# shift and asserts it is flagged, so a 0.0000 can never be a dead-observer pass.
#
# Path-gated to packages/ui (and the app-side gate + matrix) so it only runs when
# the surface it covers changes. The e2e runners bundle the fixtures with esbuild
# and stub @elizaos/core, so no core build is needed; the render-parity vitest
# imports the real @elizaos/core, so the shared i18n data is generated first.
on:
pull_request:
branches: [main]
paths:
- ".github/workflows/chat-shell-gestures.yml"
- "package.json"
- "bun.lock"
- "packages/ui/**"
- "packages/shared/**"
- "packages/core/src/i18n/**"
- "packages/app/test/chat-gesture-coverage.test.ts"
- "packages/app/docs/CHAT_GESTURE_COVERAGE.md"
push:
branches: [main, develop]
paths:
- ".github/workflows/chat-shell-gestures.yml"
- "package.json"
- "bun.lock"
- "packages/ui/**"
- "packages/shared/**"
- "packages/core/src/i18n/**"
- "packages/app/test/chat-gesture-coverage.test.ts"
- "packages/app/docs/CHAT_GESTURE_COVERAGE.md"
workflow_dispatch:
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: chat-shell-gestures-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
permissions:
contents: read
env:
CI: "true"
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
NODE_NO_WARNINGS: "1"
jobs:
chat-shell-gestures:
name: Chat shell gesture + parity e2e
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 40
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
# The render-parity contract imports the real @elizaos/core, whose i18n
# module loads generated keyword data that is gitignored. Generate it so
# the vitest run resolves (mirrors the attachment-journey-videos lane).
- name: Ensure generated shared i18n data
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Install Playwright chromium + webkit
run: bunx playwright install --with-deps chromium webkit
# Boot-free coverage gate: fails when a gesture-handler site in
# packages/ui/src lacks a CHAT_GESTURE_COVERAGE matrix row (#12188).
- name: Chat gesture coverage gate
run: bun run --cwd packages/app test -- chat-gesture-coverage
# Component-tree render parity (jsdom/vitest) — ThreadLine vs MessageContent
# render the same structure for a shared message corpus.
- name: Render-parity contract
run: bun run --cwd packages/ui test -- src/components/chat/render-parity.contract.test.tsx
# Conversation-nav interleaving fuzz (jsdom) — the pure most-recent-first
# invariants across new / select sequences (#10042). Chat-to-chat swipe
# was removed by the single-infinite-thread redesign (#13531), so the swipe
# e2e + its jank hook were retired in #13826; the remaining overlay gesture
# cost is now covered by the maximize/restore perf gate below.
- name: Conversation-nav unit + interleaving
run: bun run --cwd packages/ui test -- src/components/shell/conversation-nav.test.ts
# The iOS-style chat-sheet detent/gesture state machine (mouse + touch).
- name: Chat-sheet gesture e2e
run: bun run --cwd packages/ui test:chat-sheet-e2e
- name: Chat-sheet Safari/WebKit gesture smoke
run: bun run --cwd packages/ui test:chat-sheet-safari-e2e
# "can't scroll chat on web" repro (#chat-scroll-web): seeds a long
# transcript, opens the sheet to FULL, and asserts the transcript scroller
# resolves a BOUNDED clientHeight (so there is overflow) and that a real
# vertical finger-drag moves scrollTop natively instead of being hijacked
# into the sheet pull. RED before the fix, GREEN after.
- name: Chat-scroll web e2e
run: bun run --cwd packages/ui test:chat-scroll-web-e2e
# Transcript is vertical-only (#14328): an over-wide child + diagonal wheel
# must not pan #continuous-thread sideways; a designed inner scroller still
# scrolls in its own row and does not chain. Both engines — the axis lock is
# a *desktop trackpad* deltaX bug and WebKit's scroll semantics differ, so
# the acceptance bar requires the assertion to hold on WebKit too.
- name: Chat transcript axis-lock e2e (chromium)
run: bun run --cwd packages/ui test:chat-scroll-axis-lock-e2e
- name: Chat transcript axis-lock e2e (webkit)
run: ENGINE=webkit bun run --cwd packages/ui test:chat-scroll-axis-lock-e2e
# TopicGroup flick-collapse/expand gestures.
- name: TopicGroup gesture e2e
run: bun run --cwd packages/ui test:chatux-gesture-e2e
# Home↔launcher pager swipe, notification pull, launcher long-press
# (real CDP touch on mobile + fine-pointer desktop edge buttons).
- name: Home-screen gesture e2e
run: bun run --cwd packages/ui test:home-screen-e2e
# Seeded launcher long-loop (#12375): 500 real CDP-touch gestures over the
# composed home↔launcher surface, every §D invariant checked after each
# command, with a full walkthrough video. Pinned seed for a deterministic
# gate; fuzz locally by unsetting ELIZA_LOOP_SEED.
- name: Launcher gesture long-loop e2e
env:
ELIZA_LOOP_SEED: "12375"
run: bun run --cwd packages/ui test:launcher-loop-e2e
# Bottom-bar state-machine gestures.
- name: Bottombar gesture e2e
run: bun run --cwd packages/ui test:bottombar-e2e
# Warm-up 503 eviction: optimistic bubble survives the reconcile reload +
# retry-once delivery (records the recovery flow).
- name: Warm-up eviction e2e
run: bun run --cwd packages/ui test:warmup-eviction-e2e
# /chat ambient orange-pulse background: sample the 30s CSS animation
# across its named peaks while recording a .webm.
- name: Chat ambient pulse e2e
run: bun run --cwd packages/ui test:chat-ambient-e2e
# Perf gates: drive the overlay under thread-scroll + pull-to-maximize /
# top-pull-restore, feed real PerformanceObserver + rAF entries into the
# shared frame-budget + layout-stability detectors, hard-fail on dropped-
# frame % / p95 / CLS. Each gate first proves its CLS detector has teeth by
# injecting a real non-transient shift and asserting it is flagged (#14333),
# so a true CLS of 0.0000 cannot be a vacuous pass from a dead observer.
- name: Chat overlay perf gate
run: bun run --cwd packages/ui test:chat-perf-gate
# Scroll + maximize/restore perf gate over the REAL overlay (#9954 item 5,
# retargeted off the removed swipe path by #13826). Hard-fails on
# dropped-frame % / p95 frame time / CLS ≤ 0.1. Re-enabled in this lane by
# #14333 now that the swipe-CLS 0.80 is gone (CLS 0.0000 on develop).
- name: Chat overlay scroll/maximize perf gate
run: bun run --cwd packages/ui test:perf-gate-e2e
# #9142 Part 2: capture a dense CDP frame burst across the pill→open spring
# (animations ON) and run the single-frame-flash + two-pills-crossfade
# detectors. The harness was merged (#10356/#10374) but ran in NO workflow,
# so a transient glitch could still ship. Run under bun (resolves the
# esbuild/pixelmatch/pngjs deps directly, regardless of the node-run
# portability fallback in #10400).
- name: Frame-glitch (transient) e2e
run: bun packages/ui/src/components/shell/__e2e__/run-chat-sheet-frame-glitch-e2e.mjs
# Self-test: --canary injects a one-frame wrong state and asserts the
# detectors FIRE on it; the harness exits 0 iff they do (and non-zero if it
# has gone blind). This guards against the detectors silently degrading.
- name: Frame-glitch canary self-test
run: bun packages/ui/src/components/shell/__e2e__/run-chat-sheet-frame-glitch-e2e.mjs --canary
- name: Upload gesture videos + screenshots
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: chat-shell-gesture-artifacts
path: |
packages/ui/src/components/shell/__e2e__/output/
packages/ui/src/components/shell/__e2e__/output-webkit/
packages/ui/src/components/shell/__e2e__/output-chatux/
packages/ui/src/components/shell/__e2e__/output-home/
packages/ui/src/components/shell/__e2e__/output-launcher-loop/
packages/ui/src/components/shell/__e2e__/output-bottombar/
packages/ui/src/components/shell/__e2e__/output-warmup-eviction/
packages/ui/src/components/shell/__e2e__/output-perf-gate/
packages/ui/src/components/shell/__e2e__/output-frame-glitch/
if-no-files-found: warn
retention-days: 14
@@ -0,0 +1,81 @@
name: CI Full Matrix Proof
# Un-cancellable proof that the exhaustive develop lane keeps its full coverage
# shape (#12342). This does NOT re-run the test matrix — `test.yml` owns that on
# its own schedule. It runs the deterministic proof job: it enumerates every
# expected lane from the committed manifest, cross-checks them against
# `test.yml` (job present, not pinned pull_request-only) and against
# `run-all-tests.mjs --plan=json` (task/package floors, required core packages,
# non-empty per-script lanes). A missing lane, a lane pointed at a nonexistent
# glob, or a whole script lane collapsing to zero turns this red.
#
# Additive by design: it introduces no new required status check and does not
# touch any PR-critical workflow. The twice-daily cadence exists so the #12342
# "≥ twice daily for 7 consecutive days" observation window has a dedicated,
# never-cancelled signal to sample.
on:
schedule:
# Twice daily, offset from test.yml's 09:17 nightly so the two do not stampede.
- cron: "13 3 * * *"
- cron: "13 15 * * *"
workflow_dispatch:
# Never cancel an in-flight proof run. Push/PR traffic must not be able to
# abort the scheduled exhaustive-lane signal (#12337/#12342).
concurrency:
group: ci-full-matrix-proof-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
env:
CI: "true"
# One pinned Bun source, matching test.yml. Floating canary hangs `bun install`
# on hosted runners and writes a lockfileVersion that breaks --frozen-lockfile
# (#11184/#9454), which would itself masquerade as a failed proof.
BUN_VERSION: "1.3.14"
NODE_VERSION: "24.15.0"
NODE_NO_WARNINGS: "1"
NODE_OPTIONS: "--max-old-space-size=8192"
jobs:
matrix-proof:
name: Exhaustive lane matrix proof
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --frozen-lockfile --ignore-scripts
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Validate GitHub-native turbo cache contract
run: node packages/scripts/ci-turbo-cache-contract.mjs
- name: Capture exhaustive test plan
run: node packages/scripts/run-all-tests.mjs --plan=json > /tmp/full-matrix-plan.json
- name: Prove exhaustive lane coverage
run: node packages/scripts/ci-full-matrix-proof.mjs --plan-file /tmp/full-matrix-plan.json
- name: Upload captured plan
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: full-matrix-plan
path: /tmp/full-matrix-plan.json
if-no-files-found: warn
+245
View File
@@ -0,0 +1,245 @@
name: ci
# Cancel previous runs for the same PR/branch
concurrency:
group: ci-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
push:
branches: [main]
pull_request:
branches: [main]
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
# Test job
test:
# Skip duplicate runs: run on push to main, or on pull_request events only
if: github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref_name == 'main')
runs-on: ubuntu-latest
# 45 not 35: typecheck runs at --concurrency=4 (see below, #15140), so a
# near-total affected cone (e.g. a develop→main promote PR) needs headroom
# on top of install + build + the 15-minute test step.
timeout-minutes: 45
env:
# Baseline CI is secret-free. Live/provider-key E2E runs in dedicated
# opt-in or post-merge workflows, not in the PR test job.
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# Full history so `turbo run typecheck --affected` can resolve the PR
# merge base (#12341); a shallow clone degrades to typechecking all.
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454); packageManager bun@1.4.0-canary.1 is unresolvable (GH+npm 404)
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Create test env file
run: |
echo "TEST_DATABASE_CLIENT=pglite" > packages/core/.env.test
echo "NODE_ENV=test" >> packages/core/.env.test
- name: Build packages
run: bun run build:core
# PR lane: typecheck only the merge base's dependency cone; on push to
# main typecheck the whole workspace (#12341). --concurrency=4 not 8:
# a develop→main promote PR's affected cone is near-total, and eight
# concurrent tsgo processes exhaust a 16 GB hosted runner (#15140);
# mirrors develop-pr.yml so the PR lanes stay in agreement.
- name: Run typecheck (affected — PR)
if: github.event_name == 'pull_request'
run: NODE_OPTIONS='--max-old-space-size=8192' node packages/scripts/run-turbo.mjs run typecheck --concurrency=4 --affected
env:
TURBO_SCM_BASE: ${{ github.event.pull_request.base.sha }}
- name: Run typecheck (full — push)
if: github.event_name != 'pull_request'
run: bun run typecheck
- name: Run tests
timeout-minutes: 15
env:
NODE_OPTIONS: "--max-old-space-size=2048"
run: bun run test:core
- name: Run plugin tests
timeout-minutes: 15
env:
NODE_OPTIONS: "--max-old-space-size=2048"
run: bun run test:plugins
- name: Run interop (TypeScript) tests
timeout-minutes: 10
run: |
if [ -d packages/interop ]; then
cd packages/interop && bun run test
else
echo "packages/interop is not present; skipping interop tests"
fi
# Lint and format job
lint-and-format:
# Skip duplicate runs: run on push to main, or on pull_request events only
if: github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref_name == 'main')
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# `audit:test-realness` compares touched tests against origin/develop.
# Keep the merge base available so the diff-scoped ratchet runs in CI.
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454); packageManager bun@1.4.0-canary.1 is unresolvable (GH+npm 404)
install-command: bun install
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Prompt secret scan
run: cd packages/prompts && bun run check:secrets
- name: Type safety ratchet self-test
run: bun run audit:type-safety-ratchet:self-test
- name: Type safety ratchet
run: bun run audit:type-safety-ratchet
- name: Check format
run: bun run format:check
- name: Run lint
run: bun run lint
# UI render-time determinism gate. Fails on NEW Date.now()/Math.random()/
# crypto.randomUUID()/locale-defaulted toLocale* in a component/hook render
# path (the source of flaky screenshots + meaningless snapshots). The
# existing backlog is tracked in packages/scripts/ui-determinism-baseline.json;
# only regressions beyond it fail. Self-test asserts the AST classifier.
- name: UI determinism self-test
run: bun run audit:ui-determinism:self-test
- name: UI determinism gate
run: bun run audit:ui-determinism
# Anti-larp test gate (#10718). Fails on focused tests (.only / fit /
# fdescribe — a single .only silently drops every sibling test in the file)
# and on orphaned hardcoded skips (it.skip("name", fn) with no reason,
# tracking issue, deny-list ref, or Playwright skip-annotation). Legitimate
# conditional/env-gated skips (cond ? describe : describe.skip,
# test.skip(cond, reason)) are allowed. Self-test asserts the classifier.
- name: Anti-larp test gate self-test
run: bun run audit:focused-tests:self-test
- name: Anti-larp test gate
run: bun run audit:focused-tests
- name: Test realness ratchet
run: bun run audit:test-realness
# Per-plugin mock-LLM e2e coverage. The seeded allowlist carries
# historical debt; new unsuppressed gaps fail the main gate (#13620).
- name: Per-plugin e2e coverage
run: node packages/scripts/lint-lane-coverage.mjs
# Build job
build:
# Skip duplicate runs: run on push to main, or on pull_request events only
if: github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref_name == 'main')
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454); packageManager bun@1.4.0-canary.1 is unresolvable (GH+npm 404)
install-command: bun install
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Build packages
run: bun run build:core
# Dev startup + HMR job: prove `bun run dev` boots to a usable state quickly
# and that edits at every package dependency depth propagate to the running
# client over HMR. Guards against startup-time regressions and broken
# src/-resolution or workspace-source watching.
dev-startup:
if: github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref_name == 'main')
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454); packageManager bun@1.4.0-canary.1 is unresolvable (GH+npm 404)
install-command: bun install
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Build packages
run: bun run build
- name: Remove source-adjacent build output
run: bun run clean:stale-js
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
# The product gate is 60s (the script default enforced on dev machines).
# Shared CI runners are ~2-4 cores vs a dev workstation, so the CI ceiling
# is relaxed to 90s — still catches a startup-time doubling, without
# flaky false-failures. Lower this if runner class improves.
- name: Dev startup budget (boots + ready under budget)
env:
ELIZA_DEV_STARTUP_BUDGET_MS: "90000"
run: bun run test:dev-startup
- name: HMR propagation across dependency levels
run: bun run test:hmr
+96
View File
@@ -0,0 +1,96 @@
name: Claude Code Review
# Cancel previous runs for the same PR/branch
concurrency:
group: claude-code-review-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
pull_request:
types: [opened, ready_for_review, synchronize, reopened]
# Allow manual triggering for when you specifically want a review
workflow_dispatch:
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
claude-review:
# Skip review if PR title contains [skip-review] or is a draft
if: |
!contains(github.event.pull_request.title, '[skip-review]') &&
github.event.pull_request.draft != true
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
# Don't fail the workflow when the Anthropic API is unavailable
# (low credit balance, rate limit, transient outage). The bot review
# is best-effort; a missing review must not block CI.
continue-on-error: true
uses: anthropics/claude-code-action@ba0aafd4308cbba7165f9f2cdb0cfbed5a3c99ce
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
track_progress: true
# Prompt for automated review (no @claude mention needed)
prompt: |
REPO: ${{ github.repository }}
PR NUMBER: ${{ github.event.pull_request.number }}
Review this PR. Be concise and actionable.
**📝 IMPORTANT: Check and consider all previous review comments in this PR before making new suggestions. Don't repeat issues that have already been addressed or are being discussed.**
**🚨 CRITICAL CHECKS:**
- Security: hardcoded keys, SQL injection, XSS
- No tests = REJECT (untested code is broken)
- Wrong tools: npm/pnpm/yarn/jest = REJECT (use vitest)
- Breaking changes without migration = REJECT
**✅ REQUIRED:**
- TypeScript types (no 'any')
- Tests with vitest (via bun run test)
- Use @elizaos/core imports (canonical sources live under packages/core)
- Functional code (no classes)
- Error handling
**📋 VERIFY:**
- All new code has tests
- Use bun for scripts, vitest for testing
- No circular dependencies
- Follows existing patterns
- Docs updated if needed
**🎯 OUTPUT FORMAT:**
Provide detailed feedback using inline comments for specific issues.
```
❌ CRITICAL: [issue] → Fix: [specific action]
⚠️ IMPORTANT: [issue] → Fix: [specific action]
💡 SUGGESTION: [improvement] → Consider: [action]
```
Skip explanations. List issues with fixes.
# Allow cursor bot to trigger reviews
allowed_bots: "cursor"
# Claude CLI arguments for model and allowed tools
claude_args: |
--model claude-opus-4-7
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment *),Bash(gh pr diff *),Bash(gh pr view *),Bash(gh api *)"
@@ -0,0 +1,69 @@
name: Security Review
on:
pull_request:
types: [opened, synchronize, reopened]
# Focus on code changes that could introduce vulnerabilities
paths:
- "**/*.ts"
- "**/*.tsx"
- "**/*.js"
- "**/*.jsx"
- "**/*.json"
- "**/*.yml"
- "**/*.yaml"
- "**/Dockerfile"
- "**/.env.example"
# Allow manual security scans
workflow_dispatch:
# Cancel previous runs for the same PR
concurrency:
group: security-review-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
security:
# Skip for draft PRs, [skip-security] in title, or bot-triggered
if: |
github.event.pull_request.draft != true &&
!contains(github.event.pull_request.title, '[skip-security]') &&
github.actor != 'cursor[bot]' &&
github.actor != 'dependabot[bot]'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
security-events: write
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
fetch-depth: 2
- name: Run Claude Security Review
if: ${{ env.ANTHROPIC_API_KEY != '' }}
# Pinned to SHA for supply chain security.
uses: anthropics/claude-code-security-review@0c6a49f1fa56a1d472575da86a94dbc1edb78eda
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
with:
claude-api-key: ${{ env.ANTHROPIC_API_KEY }}
comment-pr: true
upload-results: true
claudecode-timeout: 30
# Exclude non-critical directories
exclude-directories: "node_modules,dist,coverage,.turbo,.next,docs"
# Use Opus 4.5 for deeper security analysis
claude-model: claude-opus-4-7
+55
View File
@@ -0,0 +1,55 @@
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 1
- name: Run Claude Code
id: claude
# Don't fail the workflow when the Anthropic API is unavailable
# (low credit balance, rate limit, transient outage). @claude is a
# best-effort assistant; an unavailable run must not turn the CI red.
continue-on-error: true
uses: anthropics/claude-code-action@ba0aafd4308cbba7165f9f2cdb0cfbed5a3c99ce
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Allow cursor bot to trigger reviews
allowed_bots: "cursor"
# Claude CLI arguments for model, allowed tools, and system prompt
claude_args: |
--model claude-opus-4-7
--allowedTools "Bash(*),Bash(gh *)"
--system-prompt "Be concise. Focus on actionable solutions. Use bun only (no npm/pnpm/yarn). Use vitest for testing (no jest). Check logs: gh run view <run-id> --log. Follow CLAUDE.md standards. IMPORTANT: When reviewing PRs, always check and consider all previous comments in the PR thread. Don't repeat suggestions that have already been made or addressed."
@@ -0,0 +1,92 @@
name: Cloud Aesthetic Audit
# Walk EVERY registered Eliza Cloud route (dashboard, payment, auth, admin, …) at
# desktop + mobile against the deterministic keyless stub stack with the
# test-auth shell baked into the renderer, capture rest + hover screenshots, run
# the blank/one-color analyzer + brand-color (no-blue / orange-hover) checks, and
# write per-view verdicts + a contact sheet + report.json.
#
# This is the cloud-surface twin of `app-aesthetic-audit.yml`. Until it existed
# `audit:cloud` ran in NO CI lane and was a pure reporter — a `broken`/`needs-work`
# cloud page failed nothing, and a turbo-cached renderer built WITHOUT the
# test-auth shell made the whole suite skip green with ZERO pages walked (#13624,
# the same class #9304 fixed for audit:app).
#
# Under `ELIZA_AUDIT_CLOUD_STRICT=1` (+ CI auto-on) the audit is a GATE: an
# undebted `broken` view fails; a missing auth shell or an empty walk is a HARD
# FAILURE, not a skip. A CLEAN renderer rebuild with VITE_PLAYWRIGHT_TEST_AUTH
# baked in guarantees the shell is present regardless of the turbo cache.
on:
pull_request:
branches: [main]
paths:
- "packages/app/**"
- "packages/ui/**"
- "packages/shared/**"
- "packages/cloud/**"
- ".github/workflows/cloud-aesthetic-audit.yml"
workflow_dispatch:
concurrency:
group: cloud-aesthetic-audit-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
jobs:
cloud-aesthetic-audit:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
env:
# Keyless: the ui-smoke stub is forced when CI=true (shouldForceStubStack).
ELIZA_UI_SMOKE_FORCE_STUB: "1"
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
# Force a clean renderer build with the test-auth shell baked in. A stale
# dist (built without VITE_PLAYWRIGHT_TEST_AUTH) is the exact #13624 hole —
# remove it so the audit cannot walk a shell-less bundle.
- name: Clean renderer dist
run: rm -rf packages/app/dist
- name: Run cloud aesthetic audit (strict)
# audit:cloud sets VITE_PLAYWRIGHT_TEST_AUTH=true, so the build:web it
# triggers bakes the shell. ELIZA_AUDIT_CLOUD_STRICT=1 turns the reporter
# into a gate: undebted `broken` fails, missing shell / empty walk fails.
env:
ELIZA_AUDIT_CLOUD_STRICT: "1"
run: bun run --cwd packages/app audit:cloud
- name: Upload cloud aesthetic-audit artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: cloud-aesthetic-audit-output
path: packages/app/aesthetic-audit-output-cloud
retention-days: 14
if-no-files-found: ignore
File diff suppressed because it is too large Load Diff
+431
View File
@@ -0,0 +1,431 @@
name: Cloud Deploy Backend
# Source: cloud/.github/workflows/deploy-backend.yml in the original elizaOS/cloud repo.
#
# This is a manual-only legacy operations workflow. Canonical Cloudflare
# Worker/Pages releases and their database migrations are owned by
# cloud-cf-deploy.yml; dispatch this workflow only for an explicit standalone
# migration or legacy VPS deployment.
on:
workflow_dispatch:
inputs:
environment:
description: 'Environment to deploy'
required: true
default: 'staging'
type: choice
options:
- staging
- production
deploy_legacy_vps:
description: 'Run the legacy VPS deploy after migrations'
required: false
default: false
type: boolean
concurrency:
group: cloud-deploy-backend-${{ github.ref }}
cancel-in-progress: false
env:
BUN_VERSION: "canary"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
determine-env:
name: Determine Environment
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
environment: ${{ steps.env.outputs.environment }}
branch: ${{ steps.env.outputs.branch }}
steps:
- name: Set environment
id: env
run: |
echo "environment=${{ github.event.inputs.environment }}" >> $GITHUB_OUTPUT
echo "branch=${{ github.event.inputs.environment == 'production' && 'main' || 'develop' }}" >> $GITHUB_OUTPUT
migrate-db:
name: Run Database Migrations
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
needs: determine-env
# Job-level concurrency groups are repo-wide: this serializes an explicit
# legacy migration against the canonical migrate-db gate in
# cloud-cf-deploy.yml (#11208).
concurrency:
group: cloud-db-migrate-${{ needs.determine-env.outputs.environment }}
cancel-in-progress: false
environment: ${{ needs.determine-env.outputs.environment }}
timeout-minutes: 10
env:
MIGRATION_DATABASE_URL: ${{ secrets.DATABASE_URL || secrets.RAILWAY_DATABASE_URL || secrets.NEON_DATABASE_URL }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: |
for attempt in 1 2 3; do
if [ "$attempt" -eq 3 ]; then
bun install --no-save --ignore-scripts --verbose && exit 0
else
bun install --no-save --ignore-scripts && exit 0
fi
echo "bun install attempt $attempt failed; retrying in 10s..."
sleep 10
done
exit 1
- name: Fail fast when database secret is missing
if: ${{ env.MIGRATION_DATABASE_URL == '' }}
run: |
# Previously this step quietly skipped migrations whenever the secret
# was unset, so 46+ migrations silently accumulated against prod from
# 2026-04-22 onward (sukimyfun Stripe webhook 500 RCA, 2026-05-20).
# Fail loudly instead — a deploy that ships code expecting a newer
# schema must not succeed if the DB can't be migrated to match.
echo "::error::DATABASE_URL / RAILWAY_DATABASE_URL / NEON_DATABASE_URL secret is missing on the '${{ needs.determine-env.outputs.environment }}' environment. The schema cannot be migrated, so the worker would deploy against a stale DB."
echo "Add the secret in: Settings → Environments → ${{ needs.determine-env.outputs.environment }} → Environment secrets."
exit 1
- name: Run migrations
env:
DATABASE_URL: ${{ env.MIGRATION_DATABASE_URL }}
run: |
# The migrate entry imports @elizaos/core, whose i18n barrel pulls in the
# gitignored generated keyword data. Source-mode (eliza-source) migrate
# never builds core, so generate the file first (mirrors core's prebuild).
node packages/shared/scripts/generate-keywords.mjs
bun run db:cloud:migrate
- name: Notify Discord (Success)
if: success()
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "🗄️ Database Migrated"
nodetail: true
description: |
Environment: ${{ needs.determine-env.outputs.environment }}
Branch: ${{ needs.determine-env.outputs.branch }}
Commit: ${{ github.sha }}
color: 0x00ff00
- name: Notify Discord (Failure)
if: failure()
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "❌ Database Migration Failed"
nodetail: true
description: |
Environment: ${{ needs.determine-env.outputs.environment }}
Branch: ${{ needs.determine-env.outputs.branch }}
Commit: ${{ github.sha }}
color: 0xff0000
deploy:
name: Deploy to agent VPS
if: ${{ github.event_name == 'workflow_dispatch' && inputs.deploy_legacy_vps }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
needs: [determine-env, migrate-db]
environment: ${{ needs.determine-env.outputs.environment }}
timeout-minutes: 45
env:
DEPLOY_VPS_HOST: ${{ secrets.AGENT_VPS_HOST || secrets.ELIZA_VPS_HOST }}
DEPLOY_VPS_SSH_KEY: ${{ secrets.AGENT_VPS_SSH_KEY || secrets.ELIZA_VPS_SSH_KEY }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: |
for attempt in 1 2 3; do
if [ "$attempt" -eq 3 ]; then
bun install --no-save --ignore-scripts --verbose && exit 0
else
bun install --no-save --ignore-scripts && exit 0
fi
echo "bun install attempt $attempt failed; retrying in 10s..."
sleep 10
done
exit 1
- name: Build backend artifact
run: |
bun run --cwd packages/cloud/api codegen
bun run --cwd packages/cloud/api lint
- name: Prepare SSH access
run: |
if [ -z "$DEPLOY_VPS_HOST" ] || [ -z "$DEPLOY_VPS_SSH_KEY" ]; then
echo "::error::Missing VPS deploy secrets. Configure AGENT_VPS_HOST/AGENT_VPS_SSH_KEY or ELIZA_VPS_HOST/ELIZA_VPS_SSH_KEY for the GitHub environment."
exit 1
fi
install -d -m 700 "$HOME/.ssh"
printf '%s\n' "$DEPLOY_VPS_SSH_KEY" > "$HOME/.ssh/id_ed25519"
chmod 600 "$HOME/.ssh/id_ed25519"
ssh-keyscan -H "$DEPLOY_VPS_HOST" >> "$HOME/.ssh/known_hosts"
- name: Prepare VPS checkout
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_VPS_HOST }}
username: deploy
key: ${{ env.DEPLOY_VPS_SSH_KEY }}
script_stop: true
command_timeout: 30m
script: |
set -e
echo "=== Deploying eliza-cloud backend ==="
sudo chown -R deploy:deploy /opt/eliza-cloud
cd /opt/eliza-cloud
# Use env-based config to avoid writing ~/.gitconfig (lock file failures)
export GIT_CONFIG_COUNT=1
export GIT_CONFIG_KEY_0=safe.directory
export GIT_CONFIG_VALUE_0=/opt/eliza-cloud
# --- Disk cleanup (prevents "Out of diskspace" failures) ---
echo "Disk before cleanup:"
df -h / | tail -1
# Remove stale index.lock FIRST (previous failed git op leaves this)
rm -f .git/index.lock
# Remove previous build output and local build caches.
# Keep node_modules in place so bun install can reuse the existing
# tree instead of redownloading the full dependency graph onto a
# nearly-full VPS.
rm -rf .next-build .next .turbo node_modules/.cache
# Drop all accumulated deploy stashes
git stash clear 2>/dev/null || true
# Prune loose git objects
git prune 2>/dev/null || true
# Trim Bun/Turbo caches aggressively
export BUN_INSTALL="$HOME/.bun"
export PATH="$BUN_INSTALL/bin:$PATH"
rm -rf "$HOME/.bun/install/cache" "$HOME/.cache/bun" "$HOME/.cache/turbo"
if command -v bun >/dev/null 2>&1; then
bun pm cache rm 2>/dev/null || true
fi
# Free host-level disk before touching the checkout. The root
# filesystem fills up outside /opt/eliza-cloud too, especially with
# old Docker layers, package caches, and journals.
sudo apt-get clean 2>/dev/null || true
sudo journalctl --vacuum-size=500M 2>/dev/null || true
sudo docker system prune -af 2>/dev/null || true
sudo rm -rf /root/.bun/install/cache /root/.cache/bun /root/.cache/turbo 2>/dev/null || true
# Discard local checkout drift before fetch. Cleaning untracked
# files first frees space for git to rewrite tracked files.
# Preserve the server's environment file and keep dependency trees
# if there is enough free disk to reuse them.
echo "Discarding local checkout drift..."
git clean -fdx \
-e .env \
-e .env.local \
-e .env.* \
-e node_modules \
-e services/gateway-discord/node_modules \
-e services/gateway-webhook/node_modules \
-e packages/ui/node_modules \
-e services/agent-server/node_modules 2>/dev/null || true
rm -f .git/index.lock
git checkout -- . 2>/dev/null || true
rm -f .git/index.lock
# The systemd unit on the VPS expects a repo-local env file, but the
# exact filename can vary across older hosts. Preserve all root env
# files and recreate the common aliases from whichever one survived.
ENV_SOURCE=""
for candidate in .env.local .env.production .env.staging .env; do
if [ -f "$candidate" ]; then
ENV_SOURCE="$candidate"
break
fi
done
if [ -n "$ENV_SOURCE" ]; then
for alias in .env .env.local .env.production .env.staging; do
if [ ! -e "$alias" ]; then
ln -s "$ENV_SOURCE" "$alias"
fi
done
fi
echo "Env files available on VPS checkout:"
ls -la .env* 2>/dev/null || true
AVAILABLE_KB="$(df --output=avail -k / | tail -1 | tr -d ' ')"
if [ "${AVAILABLE_KB:-0}" -lt 5000000 ]; then
echo "Still under 5GB free; removing workspace node_modules as a fallback..."
rm -rf \
node_modules \
services/gateway-discord/node_modules \
services/gateway-webhook/node_modules \
packages/ui/node_modules \
services/agent-server/node_modules
fi
echo "Disk after cleanup:"
df -h / | tail -1
# --- End disk cleanup ---
# Fetch and checkout
BRANCH="${{ needs.determine-env.outputs.branch }}"
git fetch origin "$BRANCH"
git checkout -B "$BRANCH" "origin/$BRANCH"
# Install runtime dependencies and regenerate lightweight assets.
if ! command -v bun >/dev/null 2>&1; then
curl -fsSL https://bun.sh/install | bash
export PATH="$BUN_INSTALL/bin:$PATH"
fi
for attempt in 1 2 3; do
if bun install --no-save --ignore-scripts; then
break
fi
if [ "$attempt" -eq 3 ]; then
exit 1
fi
echo "bun install attempt $attempt failed; retrying in 10s..."
sleep 10
done
bun run generate:llms
# The build now runs on GitHub Actions to avoid ENOSPC during
# `next build` on the VPS. Clear any stale output and leave an empty
# destination for the streamed artifact.
rm -rf "$HOME/.bun/install/cache" "$HOME/.cache/bun"
rm -rf node_modules/.cache
rm -rf .next-build
mkdir -p .next-build
- name: Upload prebuilt Next output
run: |
# Only ship the runtime output. The webpack cache dominates .next-build
# size locally but is not needed by `next start` on the VPS.
tar -C .next-build -czf - \
BUILD_ID \
app-path-routes-manifest.json \
build-manifest.json \
export-marker.json \
images-manifest.json \
next-minimal-server.js.nft.json \
next-server.js.nft.json \
package.json \
prerender-manifest.json \
react-loadable-manifest.json \
required-server-files.js \
required-server-files.json \
routes-manifest.json \
server \
static | ssh -i "$HOME/.ssh/id_ed25519" \
deploy@"$DEPLOY_VPS_HOST" \
'set -e; cd /opt/eliza-cloud; rm -rf .next .next-build/*; tar -xzf - -C .next-build; ln -sfn .next-build .next'
- name: Restart services
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_VPS_HOST }}
username: deploy
key: ${{ env.DEPLOY_VPS_SSH_KEY }}
script_stop: true
script: |
set -e
cd /opt/eliza-cloud
rm -rf .next
ln -sfn .next-build .next
if ! sudo systemctl restart eliza-cloud; then
sudo systemctl cat eliza-cloud || true
sudo systemctl show eliza-cloud \
--property=FragmentPath,EnvironmentFiles,ExecStart,WorkingDirectory || true
ls -la /opt/eliza-cloud/.env* 2>/dev/null || true
sudo systemctl status eliza-cloud --no-pager || true
sudo journalctl -u eliza-cloud -n 100 --no-pager || true
exit 1
fi
sudo systemctl restart agent-provisioning-worker
echo "=== Deploy complete ==="
- name: Health Check
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_VPS_HOST }}
username: deploy
key: ${{ env.DEPLOY_VPS_SSH_KEY }}
script_stop: true
script: |
echo "Checking eliza-cloud health..."
for attempt in $(seq 1 18); do
for url in http://localhost:3000/api/health http://localhost:3334/api/health; do
if curl -sf "$url" > /dev/null; then
echo "Health check passed on attempt $attempt via $url."
exit 0
fi
done
echo "Health check attempt $attempt/18 failed; retrying in 5s..."
sleep 5
done
echo "eliza-cloud failed health checks after 90s. Collecting diagnostics..."
sudo ss -ltnp | grep -E ':(3000|3334)\\s' || true
sudo systemctl status eliza-cloud --no-pager || true
sudo journalctl -u eliza-cloud -n 100 --no-pager || true
exit 1
- name: Notify Discord
if: success()
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "🚀 Backend Deployed"
description: |
Environment: ${{ needs.determine-env.outputs.environment }}
Branch: ${{ needs.determine-env.outputs.branch }}
Commit: ${{ github.sha }}
color: 0x00ff00
- name: Notify Discord (Failure)
if: failure()
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "❌ Backend Deploy Failed"
description: |
Environment: ${{ needs.determine-env.outputs.environment }}
Branch: ${{ needs.determine-env.outputs.branch }}
Commit: ${{ github.sha }}
color: 0xff0000
+89
View File
@@ -0,0 +1,89 @@
name: cloud-e2e
# Mock-stack E2E for cloud-api + the cloud web app (packages/app).
# Runs `bun run cloud:e2e`, which boots in-process PGlite + ioredis-mock + Hetzner mock +
# control-plane mock and drives the real cloud-api / cloud web dev servers via Playwright.
on:
pull_request:
branches: [main]
paths:
- "packages/cloud/api/**"
- "packages/cloud/shared/**"
- "packages/cloud/services/**"
- "packages/cloud/sdk/**"
- "packages/test/cloud-mocks/**"
- "packages/test/cloud-e2e/**"
# The showcase loop (#9300) deploys + validates these example apps, so an
# edit to either app must trigger the showcase e2e + brand-lint lane.
- "packages/examples/cloud/**"
- ".github/workflows/cloud-e2e.yml"
workflow_dispatch:
concurrency:
group: cloud-e2e-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
e2e:
name: Mock-stack E2E
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
env:
NODE_ENV: test
NODE_OPTIONS: --max-old-space-size=4096
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
fetch-depth: 1
- uses: ./.github/actions/cloud-setup-test-env
- name: Install Playwright browsers (chromium only)
run: bunx playwright install --with-deps chromium
- name: Build cloud-shared (drizzle artifacts)
# cloud-e2e fixture re-runs migrations against PGlite so build is best-effort.
run: bun run --cwd packages/cloud/shared build || true
- name: Run cloud E2E
run: bun run cloud:e2e
env:
CI: "true"
CLOUD_E2E: "1"
# mocks read these
MOCK_REDIS: "1"
MOCK_HETZNER_LATENCY: "0"
MOCK_HETZNER_ACTION_MS: "30"
CONTROL_PLANE_TICK_MS: "50"
# placeholder values so client code doesn't crash on lookup; no real creds
DATABASE_URL: "pglite://./.eliza-ci/.pgdata"
HCLOUD_TOKEN: "test-token"
CONTAINER_CONTROL_PLANE_TOKEN: "test-token"
CRON_SECRET: "test-cron-secret"
- name: Upload Playwright artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: cloud-e2e-playwright-${{ github.run_attempt }}
path: |
packages/test/cloud-e2e/playwright-report
packages/test/cloud-e2e/test-results
retention-days: 7
if-no-files-found: ignore
- name: Upload process logs (boot diagnostics)
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: cloud-e2e-process-logs-${{ github.run_attempt }}
path: |
packages/test/cloud-e2e/.logs
retention-days: 7
include-hidden-files: true
if-no-files-found: ignore
@@ -0,0 +1,99 @@
name: Cloud Gateway Discord
# CI for the gateway-discord service. The service is deployed to Railway
# (see packages/cloud/services/gateway-discord/railway.toml); Railway auto-
# deploys on push to the watched branch, so this workflow only runs tests
# and a build sanity check.
#
# AWS EKS / Terraform / Helm deploy jobs were removed as part of the AWS
# retirement (see packages/cloud/infra/cloud/AWS_RETIREMENT.md).
on:
push:
branches: [main, develop]
paths:
- "packages/cloud/services/gateway-discord/**"
- "packages/cloud/shared/src/lib/services/gateway-discord/**"
- "packages/cloud/shared/src/lib/auth/jwks.ts"
- "packages/cloud/shared/src/lib/auth/jwt-internal.ts"
- "packages/cloud/shared/src/lib/auth/internal-api.ts"
- "packages/cloud/shared/scripts/messaging-gateway-preflight.mjs"
- "packages/cloud/api/v1/discord/**"
- "packages/cloud/api/internal/discord/**"
- "packages/cloud/api/internal/auth/**"
- "packages/cloud/api/.well-known/jwks.json/**"
- "packages/cloud/shared/src/db/**/discord-connections.ts"
- ".github/workflows/cloud-gateway-discord.yml"
pull_request:
branches: [main]
paths:
- "packages/cloud/services/gateway-discord/**"
- "packages/cloud/shared/src/lib/services/gateway-discord/**"
- "packages/cloud/shared/src/lib/auth/jwks.ts"
- "packages/cloud/shared/src/lib/auth/jwt-internal.ts"
- "packages/cloud/shared/src/lib/auth/internal-api.ts"
- "packages/cloud/shared/scripts/messaging-gateway-preflight.mjs"
- "packages/cloud/api/v1/discord/**"
- "packages/cloud/api/internal/discord/**"
- "packages/cloud/api/internal/auth/**"
- "packages/cloud/api/.well-known/jwks.json/**"
- "packages/cloud/shared/src/db/**/discord-connections.ts"
- ".github/workflows/cloud-gateway-discord.yml"
concurrency:
group: cloud-gateway-discord-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
defaults:
run:
working-directory: packages/cloud/shared
env:
BUN_VERSION: "canary"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
test:
name: Test
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install root dependencies
run: bun install --no-save && git diff --exit-code -- bun.lock
- name: Run messaging gateway preflight
run: bun run preflight:messaging-gateways
- name: Install service dependencies
working-directory: packages/cloud/services/gateway-discord
run: bun install --no-save && git diff --exit-code -- bun.lock
- name: Run service tests
working-directory: packages/cloud/services/gateway-discord
run: bun test tests/ --timeout 60000
env:
SKIP_SERVER_CHECK: "true"
- name: Run lib tests
run: bun test src/lib/services/gateway-discord/__tests__ --timeout 60000
env:
SKIP_SERVER_CHECK: "true"
- name: Run Discord API unit tests
run: |
if [ -f packages/tests/unit/discord-connections-api.test.ts ]; then
bun test packages/tests/unit/discord-connections-api.test.ts --timeout 60000
else
echo "packages/tests/unit/discord-connections-api.test.ts is not present; skipping legacy Discord API unit tests"
fi
env:
SKIP_SERVER_CHECK: "true"
@@ -0,0 +1,64 @@
name: Cloud Gateway Webhook
# Source: cloud/.github/workflows/gateway-webhook.yml in the original elizaOS/cloud repo.
on:
push:
branches: [main, develop]
paths:
- "packages/cloud/services/gateway-webhook/**"
- "packages/cloud/shared/src/lib/services/**"
- "packages/cloud/shared/src/lib/onboarding/**"
- "packages/cloud/shared/src/lib/auth/**"
- "packages/cloud/shared/src/db/**"
- "packages/cloud/shared/scripts/messaging-gateway-preflight.mjs"
- ".github/workflows/cloud-gateway-webhook.yml"
pull_request:
branches: [main]
paths:
- "packages/cloud/services/gateway-webhook/**"
- "packages/cloud/shared/src/lib/services/**"
- "packages/cloud/shared/src/lib/onboarding/**"
- "packages/cloud/shared/src/lib/auth/**"
- "packages/cloud/shared/src/db/**"
- "packages/cloud/shared/scripts/messaging-gateway-preflight.mjs"
- ".github/workflows/cloud-gateway-webhook.yml"
concurrency:
group: cloud-gateway-webhook-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
defaults:
run:
working-directory: packages/cloud/shared
env:
BUN_VERSION: "canary"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
test:
name: Test
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --no-save && git diff --exit-code -- bun.lock
- name: Run messaging gateway preflight
run: bun run preflight:messaging-gateways
- name: Run unit tests (isolated — mock.module is process-global in bun)
working-directory: packages/cloud/services/gateway-webhook
run: |
bun test ./__tests__/build-forward-body.test.ts --timeout 60000
bun test ./__tests__/internal-auth.test.ts --timeout 60000
+142
View File
@@ -0,0 +1,142 @@
name: Cloud Tests
# Runs cloud's lint/typecheck/unit/integration/playwright suite when cloud-* packages or tests change.
# Source: cloud/.github/workflows/tests.yml in the original elizaOS/cloud repo.
on:
workflow_dispatch:
pull_request:
branches: [develop, main]
paths:
- ".github/actions/cloud-setup-test-env/**"
- ".github/workflows/cloud-tests.yml"
- "packages/cloud/api/**"
- "packages/cloud/shared/**"
- "packages/cloud/sdk/**"
- "packages/cloud/services/**"
- "packages/cloud/routing/**"
- "packages/cloud/infra/**"
- "packages/scripts/cloud/**"
- "packages/scripts/test-cloud-run.mjs"
- "packages/scripts/test-cloud-run-helpers.mjs"
- "packages/scripts/test-cloud-run-flush.self-test.mjs"
- "packages/scripts/test-cloud-run-helpers.self-test.mjs"
- "package.json"
- "bun.lock"
- ".github/workflows/cloud-tests.yml"
- ".github/actions/cloud-setup-test-env/**"
push:
branches: [develop, main]
paths:
- "packages/cloud/api/**"
- "packages/cloud/shared/**"
- "packages/cloud/sdk/**"
- "packages/cloud/services/**"
- "packages/cloud/routing/**"
- "packages/cloud/infra/**"
- "packages/scripts/cloud/**"
- "packages/scripts/test-cloud-run.mjs"
- "packages/scripts/test-cloud-run-helpers.mjs"
- "packages/scripts/test-cloud-run-flush.self-test.mjs"
- "packages/scripts/test-cloud-run-helpers.self-test.mjs"
- "package.json"
- "bun.lock"
- ".github/workflows/cloud-tests.yml"
- ".github/actions/cloud-setup-test-env/**"
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: cloud-tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
env:
DATABASE_URL: postgresql://postgres@127.0.0.1:5432/postgres
TEST_DATABASE_URL: postgresql://postgres@127.0.0.1:5432/postgres
CACHE_BACKEND: wadis
CACHE_ENABLED: "false"
NODE_OPTIONS: --max-old-space-size=4096
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
lint-and-types:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 25
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- uses: ./.github/actions/cloud-setup-test-env
- name: Lint and typecheck cloud workspaces
run: bun run verify:cloud
- name: Tenant-scope gate (no unscoped pk-only reads of apps data-plane) (#9853)
run: bun run --cwd packages/cloud/shared check:tenant-scope
# NOTE: the Worker bundle stub-drift gate (#14422) runs inside
# `lint-and-types` above — packages/cloud/api chains `check:worker-bundle`
# (wrangler deploy --dry-run) into its `typecheck`, so `verify:cloud` covers
# it. Do not re-add a standalone dry-run job: #14474 + #14504 briefly ran it
# twice per PR on the saturated runner fleet.
unit-tests:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
# Cloud unit tests can run past 25m on saturated self-hosted runners while still passing.
timeout-minutes: 40
env:
NODE_ENV: test
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- uses: ./.github/actions/cloud-setup-test-env
# Guards the batch runner's output-flush contract: a failing batch must
# surface its diagnostic instead of being swallowed by a process.exit()
# that truncates buffered stdout (the #16062 silent exit-1 regression).
- name: Batch runner flush-integrity self-test
run: node packages/scripts/test-cloud-run-flush.self-test.mjs
- name: Batch runner helper self-test
run: node packages/scripts/test-cloud-run-helpers.self-test.mjs
- name: Run cloud unit tests
run: bun run test:cloud
# The bun `test:cloud` lane skips suites that need the Vitest-only module
# mock API (`vi.mock` + `vi.importActual`) — see `SUPPORTS_VITEST_MOCK_API`.
# Run those under Vitest here so the direct-wallet payer/state-machine proof
# (33 tests, in-process PGlite) actually executes instead of vacuously
# skipping. Uses pglite://memory — no DB service required.
- name: Run cloud vitest-only suites
run: bun run --cwd packages/cloud/shared test:vitest
integration-tests:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 25
env:
NODE_ENV: test
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- uses: ./.github/actions/cloud-setup-test-env
with:
setup-db: "true"
db-backend: "pglite"
- name: Run cloud integration tests
run: bun run test:cloud:integration
e2e-tests:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 35
needs: [lint-and-types]
env:
NODE_ENV: test
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- uses: ./.github/actions/cloud-setup-test-env
with:
setup-db: "true"
- name: Run cloud e2e tests
run: bun run test:cloud:e2e
+100
View File
@@ -0,0 +1,100 @@
name: "CodeQL Advanced"
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
schedule:
- cron: "29 8 * * 6"
workflow_dispatch:
# Default to least privilege. Override per-job where needed.
# cancel superseded in-flight runs for the same PR to free hosted-runner
# capacity. only cancels pull_request runs; push runs on protected branches
# (main/develop) are never cancelled mid-flight.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
jobs:
analyze:
name: Analyze (${{ matrix.language }})
# WHY: whole-repo JS CodeQL can exhaust GitHub-hosted disk during the
# PolynomialReDoS dataflow query. Trusted events use the larger robot fleet;
# pull_request stays GitHub-hosted so forked code never runs on self-hosted.
runs-on: ${{ fromJSON(github.event_name == 'pull_request' && '["ubuntu-latest"]' || '["self-hosted","Linux","X64","hetzner-robot"]') }}
timeout-minutes: 360
permissions:
# required for all workflows
security-events: write
# required to fetch internal or private CodeQL packs
packages: read
# only required for workflows in private repositories
actions: read
contents: read
strategy:
fail-fast: false
matrix:
include:
- language: javascript-typescript
build-mode: none
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Report disk before CodeQL
shell: bash
run: |
df -h
du -sh "$GITHUB_WORKSPACE" 2>/dev/null || true
- name: Free GitHub-hosted runner disk
if: runner.environment == 'github-hosted'
shell: bash
run: |
# WHY: this preserves the full CodeQL query suite while giving the
# hosted fallback enough headroom for CodeQL's temporary databases.
sudo rm -rf \
/usr/local/lib/android \
/usr/share/dotnet \
/opt/ghc \
/opt/hostedtoolcache/go \
/opt/hostedtoolcache/Python \
/opt/hostedtoolcache/Ruby || true
df -h
- name: Initialize CodeQL
uses: github/codeql-action/init@99df26d4f13ea111d4ec1a7dddef6063f76b97e9
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
config-file: ./.github/codeql/codeql-config.yml
# WHY: the paths-ignore config shrinks the JS/TS database ~36x (~107MB), so
# the memory pressure behind exit-137 is resolved at the source. threads can
# return toward the machine's 12-core capacity for fast analysis (holding the
# robot briefly instead of ~1h); ram stays a generous ceiling (~120GB of the
# 244GB box) purely as a safety net against an unmeasured pathological query.
ram: 120000
threads: 8
- if: matrix.build-mode == 'manual'
shell: bash
run: |
echo 'If you are using a "manual" build mode for one or more of the' \
'languages you are analyzing, replace this with the commands to build' \
'your code, for example:'
echo ' make basic-capabilities'
echo ' make release'
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@99df26d4f13ea111d4ec1a7dddef6063f76b97e9
with:
category: "/language:${{matrix.language}}"
+181
View File
@@ -0,0 +1,181 @@
name: coverage-gate
# SOC2 CC4.1 — track unit-test coverage on changed files. ENFORCED (issue #9943):
# a PR that changes source under packages/plugins/apps must change at least one
# unit test that can emit coverage. Every surviving changed executable source
# must appear in LCOV and clear the line-coverage floor. A source-aware
# classifier strips TypeScript-only syntax before excluding modules with no
# runtime code; declarations and deleted files are not targets. LCOV LF:0 never
# proves type-only status. Source covered only by e2e/live lanes still needs a
# focused unit test in this gate. Docs-only PRs do not spend coverage time.
# Standalone Node self-tests are excluded only when listed in the manifest that
# the Develop PR Test Integrity job executes on every PR.
# A test-only PR (changed tests, no changed source) executes its changed tests —
# gating the run steps on changed *source* let such PRs go vacuously green with
# the new test never running (#15849); the per-file floor simply has no source
# to enforce against.
#
# The per-package coverage floors live in the shared base vitest config
# (packages/test/vitest/default.config.ts, sourced from coverage-policy.mjs) and
# apply on a `--coverage` run. This changed-files gate is the second layer: it
# enforces a per-changed-file floor (`threshold` below), set to a ratchetable 50%
# — raise it toward the long-term 70% target as suites fill in. See issue #9943
# (the umbrella) and #10104 for the broader test-gating audit.
on:
pull_request:
branches: [develop, main]
paths:
- "packages/**"
- "plugins/**"
- "apps/**"
# cancel superseded in-flight runs for the same PR to free hosted-runner
# capacity. only cancels pull_request runs; push runs on protected branches
# (main/develop) are never cancelled mid-flight.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
env:
NODE_VERSION: "24.15.0"
jobs:
coverage:
name: coverage on changed files
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout
# actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
- name: Setup Node.js for source classification
# The classifier is Node-based and runs before optional Bun workspace
# setup, including for JSON-only PRs on runners without system Node.
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Determine changed files
id: changed
# Three-dot diff (merge-base..HEAD) so a branch trailing develop does not
# count develop-side files it never touched. Logic lives in the script so
# its path/diff boundary rules are regression-tested (issue #15845).
run: |
bash scripts/security/coverage-changed-files.sh \
"${{ github.event.pull_request.base.sha }}" \
"${{ github.event.pull_request.head.sha }}" >> "$GITHUB_OUTPUT"
- name: Require changed tests for changed source
if: steps.changed.outputs.files != '' && steps.changed.outputs.bun_tests == '' && steps.changed.outputs.vitest_tests == '' && steps.changed.outputs.node_tests == ''
run: |
echo "Changed source files require at least one changed Bun or Vitest unit test:"
printf '%s\n' "${{ steps.changed.outputs.files }}"
exit 1
- name: Setup Bun workspace
if: steps.changed.outputs.bun_tests != '' || steps.changed.outputs.vitest_tests != ''
# Keep dependency setup behind the changed-test check so docs-only and
# test-free source PRs either skip or fail before spending setup time.
# Not gated on changed source: a test-only PR (files == '') must still
# execute its changed tests, or the suite goes vacuously green (#15849).
uses: ./.github/actions/setup-bun-workspace
with:
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
- name: Run changed registered Node self-tests
if: steps.changed.outputs.node_tests != ''
run: |
while IFS= read -r self_test; do
[ -n "$self_test" ] || continue
node "$self_test"
done <<'EOF'
${{ steps.changed.outputs.node_tests }}
EOF
- name: Run changed Bun tests with coverage
if: steps.changed.outputs.bun_tests != ''
run: |
cat > "$RUNNER_TEMP/changed-tests.txt" <<'EOF'
${{ steps.changed.outputs.bun_tests }}
EOF
mapfile -t changed_tests < <(grep -v '^$' "$RUNNER_TEMP/changed-tests.txt" || true)
if [ "${#changed_tests[@]}" -eq 0 ]; then
echo "no changed Bun-native test files; skipping coverage run"
exit 0
fi
shared_tests=()
process_isolated_tests=()
for test_file in "${changed_tests[@]}"; do
case "$test_file" in
packages/cloud/api/v1/voice/session/__tests__/harness-real-server.test.ts|packages/tools/voice-evidence-harness/src/cli-run.test.ts)
process_isolated_tests+=("$test_file")
;;
*) shared_tests+=("$test_file") ;;
esac
done
# This lane runs before workspace builds, so package defaults point at
# absent dist files; source exports keep changed tests self-contained.
if [ "${#shared_tests[@]}" -gt 0 ]; then
bun test --conditions=eliza-source "${shared_tests[@]}" --coverage --coverage-reporter=lcov --coverage-dir=coverage/bun/shared
fi
# Bun module mocks remain process-global despite `--isolate`. Tests
# listed here run in fresh processes and emit separate LCOV files;
# the gate below already aggregates every coverage/**/lcov.info.
for index in "${!process_isolated_tests[@]}"; do
ELIZA_PROCESS_ISOLATED_TEST=1 bun test --conditions=eliza-source "${process_isolated_tests[$index]}" --coverage --coverage-reporter=lcov --coverage-dir="coverage/bun/isolated-$index"
done
env:
# Tests must produce LCOV under this coverage root; the enforced gate
# recursively aggregates process-isolated and shared outputs.
BUN_COVERAGE_DIR: coverage/bun
- name: Build core for changed Vitest tests
if: steps.changed.outputs.vitest_tests != ''
# Vitest package configs may exercise source graphs that still depend on
# the standard built core/logger/contracts/cloud-routing entrypoints.
run: bun run --cwd packages/core build
- name: Run changed Vitest tests with coverage
if: steps.changed.outputs.vitest_tests != ''
run: |
cat > "$RUNNER_TEMP/changed-vitest-tests.txt" <<'EOF'
${{ steps.changed.outputs.vitest_tests }}
EOF
mapfile -t changed_tests < <(grep -v '^$' "$RUNNER_TEMP/changed-vitest-tests.txt" || true)
if [ "${#changed_tests[@]}" -eq 0 ]; then
echo "no changed Vitest test files; skipping coverage run"
exit 0
fi
node packages/scripts/run-changed-vitest-coverage.mjs "${changed_tests[@]}"
- name: Apply coverage gate (enforced)
# Runs whenever changed tests ran, including test-only PRs (files == '').
# With no changed source, executing the changed tests is the gate:
# Bun may not emit LCOV when it only ran tests, and there is no per-file
# floor to enforce (#15849).
if: steps.changed.outputs.bun_tests != '' || steps.changed.outputs.vitest_tests != '' || steps.changed.outputs.node_tests != ''
env:
COVERAGE_GATE_ENFORCE: "1" # enforced (issue #9943)
run: |
mapfile -t lcov_files < <(find coverage -name lcov.info -type f | sort)
if [ "${#lcov_files[@]}" -eq 0 ]; then
if [ -z "${{ steps.changed.outputs.files }}" ]; then
echo "changed tests ran; no changed source files require LCOV enforcement"
exit 0
fi
echo "changed tests ran but no coverage/lcov.info files were produced"
exit 1
fi
awk \
-v changed="${{ steps.changed.outputs.files }}" \
-v threshold=50 \
-f scripts/security/coverage-gate.awk \
"${lcov_files[@]}"
+311
View File
@@ -0,0 +1,311 @@
name: Deploy Apps Worker (Product 2)
# SSHes to the apps-control host, fetches the target branch, installs the
# systemd unit shipped at packages/scripts/cloud/admin/eliza-apps-worker.service,
# and restarts the eliza-apps-worker daemon (the Product-2 deploy daemon).
#
# This is the slim sibling of deploy-eliza-provisioning-worker.yml: it deploys
# ONLY the apps-worker — NO agent-router, and NO headscale wiring (the apps
# worker reaches the per-tenant Postgres over the apps PRIVATE network, never the
# agent mesh). The apps env (APPS_DEPLOY_ENABLED, APPS_TENANT_ADMIN_DSN,
# CONTAINERS_DOCKER_NODES, …) is written separately by arm-apps-daemon.yml — this
# workflow only ships code + the unit, keeping the two concerns split.
#
# First-time setup: bootstrap an apps-control VM on the apps PRIVATE network (so
# it can reach the tenant DB at 10.30.1.10) that runs NO untrusted containers,
# then set the env's ELIZA_APPS_WORKER_HOST + ELIZA_APPS_WORKER_SSH_KEY secrets.
on:
push:
branches: [develop, main]
paths:
- '.github/workflows/deploy-apps-worker.yml'
- 'packages/scripts/cloud/admin/daemons/apps-provisioning-worker.ts'
- 'packages/scripts/cloud/admin/daemons/provisioning-worker.ts'
- 'packages/scripts/cloud/admin/eliza-apps-worker.service'
- 'packages/cloud/shared/src/lib/services/provisioning-jobs.ts'
- 'packages/cloud/shared/src/lib/services/provisioning-job-types.ts'
- 'packages/cloud/shared/src/lib/services/apps-deploy-backend.ts'
- 'packages/cloud/shared/src/lib/services/tenant-db/**'
- 'packages/cloud/shared/src/lib/services/app-deploy-runner.ts'
- 'packages/cloud/sdk/**'
- 'packages/core/**'
- 'plugins/plugin-sql/**'
workflow_dispatch:
inputs:
environment:
description: 'Target environment (host + secrets resolved from this)'
required: true
default: 'staging'
type: choice
options:
- staging
- production
concurrency:
# Per-commit group (mirrors cloud-cf-deploy #10802/#10973). A SHARED
# deploy-apps-worker-<env> group with cancel-in-progress:false makes GitHub
# keep only the latest PENDING run per group and silently cancel the older
# pending ones (no annotation) — so a hot develop push/dispatch stream
# produces a chain of cancelled runs and zero completed deploys
# (elizaOS/eliza#10839). Keying on github.sha too gives each commit its own
# group so no run is ever silently superseded. Jobs run on ubuntu-latest
# (no runner-level serialization), so a host-side flock in the SSH scripts
# serializes concurrent deploys against the shared apps-control host; each
# run deploys the branch tip at fetch time, so the last to execute lands the
# newest code. Env stays in the key so staging and production never block
# each other.
group: deploy-apps-worker-${{ github.event.inputs.environment || (github.ref == 'refs/heads/main' && 'production' || 'staging') }}-${{ github.sha }}
cancel-in-progress: false
permissions:
contents: read
jobs:
determine-env:
name: Determine environment
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
environment: ${{ steps.env.outputs.environment }}
branch: ${{ steps.env.outputs.branch }}
steps:
- id: env
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
env="${{ github.event.inputs.environment }}"
elif [ "${{ github.ref }}" = "refs/heads/main" ]; then
env="production"
else
env="staging"
fi
branch=$([ "$env" = "production" ] && echo "main" || echo "develop")
echo "environment=$env" >> "$GITHUB_OUTPUT"
echo "branch=$branch" >> "$GITHUB_OUTPUT"
echo "Resolved: environment=$env branch=$branch"
deploy:
name: Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }})
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
needs: determine-env
environment: ${{ needs.determine-env.outputs.environment }}
timeout-minutes: 15
env:
SYSTEMD_UNIT: eliza-apps-worker.service
DEPLOY_HOST: ${{ secrets.ELIZA_APPS_WORKER_HOST }}
DEPLOY_SSH_KEY: ${{ secrets.ELIZA_APPS_WORKER_SSH_KEY }}
DEPLOY_BRANCH: ${{ needs.determine-env.outputs.branch }}
steps:
- name: Check deploy configuration
id: deploy_config
run: |
if [ -z "$DEPLOY_HOST" ] || [ -z "$DEPLOY_SSH_KEY" ]; then
echo "configured=false" >> "$GITHUB_OUTPUT"
echo "::warning::Missing ELIZA_APPS_WORKER_HOST or ELIZA_APPS_WORKER_SSH_KEY; skipping apps-worker deploy."
{
echo "### Apps worker deploy skipped"
echo ""
echo "Missing \`ELIZA_APPS_WORKER_HOST\` or \`ELIZA_APPS_WORKER_SSH_KEY\` for this environment."
echo "Bootstrap an apps-control VM on the apps private network and set those secrets first."
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "::error::Manual deployment requires ELIZA_APPS_WORKER_HOST and ELIZA_APPS_WORKER_SSH_KEY."
exit 1
fi
exit 0
fi
echo "configured=true" >> "$GITHUB_OUTPUT"
- name: Ensure host prereqs (Node 24, swap, bunx symlink)
if: steps.deploy_config.outputs.configured == 'true'
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 5m
script: |
set -euo pipefail
if ! command -v node >/dev/null 2>&1; then
echo "[host-prereqs] installing Node 24"
curl -fsSL https://deb.nodesource.com/setup_24.x | sudo bash -
sudo apt-get install -y nodejs
fi
if [ ! -f /swapfile ]; then
echo "[host-prereqs] adding 8GB swap"
sudo fallocate -l 8G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
grep -q '^/swapfile ' /etc/fstab || echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab >/dev/null
fi
if [ ! -e /home/deploy/.bun/bin/bunx ] && [ -e /home/deploy/.bun/bin/bun ]; then
echo "[host-prereqs] creating bunx symlink"
ln -sf bun /home/deploy/.bun/bin/bunx
fi
- name: Deploy and restart apps worker
if: steps.deploy_config.outputs.configured == 'true'
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 10m
envs: DEPLOY_BRANCH,SYSTEMD_UNIT
script: |
set -euo pipefail
# Serialize deploys on the host: per-SHA concurrency groups mean
# overlapping runs are expected; without this they would interleave
# git clean/bun install/systemctl restart on the same checkout.
exec 9>/tmp/eliza-apps-worker-deploy.lock
# 540s keeps the lock wait under this step's 10m command_timeout so
# a starved waiter fails with THIS message, not an opaque SSH kill.
flock -w 540 9 || { echo "::error::another apps-worker deploy holds the host lock after 9m"; exit 1; }
echo "=== Deploying eliza-apps-worker (branch: $DEPLOY_BRANCH) ==="
cd /opt/eliza
export GIT_CONFIG_COUNT=1
export GIT_CONFIG_KEY_0=safe.directory
export GIT_CONFIG_VALUE_0=/opt/eliza
rm -f .git/index.lock
sudo chown -R deploy:deploy .git
git reset --hard HEAD 2>/dev/null || true
git clean -fdx \
-e .env \
-e .env.local \
-e '.env.*' \
-e cloud/.env.local \
-e 'cloud/.env.*' \
-e node_modules
find packages -type f \( -name '*.d.ts' -o -name '*.d.ts.map' \) \
! -path '*/node_modules/*' -delete 2>/dev/null || true
rm -f bun.lock
git -c fetch.recurseSubmodules=no fetch --no-recurse-submodules \
origin "+$DEPLOY_BRANCH:refs/remotes/origin/$DEPLOY_BRANCH"
git -c submodule.recurse=false checkout --no-recurse-submodules \
-B "$DEPLOY_BRANCH" "origin/$DEPLOY_BRANCH"
# The `rm -f bun.lock` above deletes a TRACKED file; when bun.lock
# is identical between the previous HEAD and the new tip, checkout
# carries that deletion over as a local change and `bun install`
# then resolves the whole tree fresh from package.json ranges —
# newest-in-range @ai-sdk/* against the overridden provider-utils
# pin took the worker down at boot (`does not provide an export
# named 'secureJsonParse'`). Always restore the tracked lockfile so
# installs are lockfile-pinned and deterministic.
git checkout "origin/$DEPLOY_BRANCH" -- bun.lock
if ! command -v bun >/dev/null 2>&1; then
curl -fsSL https://bun.sh/install | bash -s "canary"
export BUN_INSTALL="$HOME/.bun"
export PATH="$BUN_INSTALL/bin:$PATH"
fi
cd /opt/eliza
for attempt in 1 2 3; do
if bun install --no-save --ignore-scripts; then
break
fi
if [ "$attempt" -eq 3 ]; then exit 1; fi
echo "bun install attempt $attempt failed; retrying in 10s..."
sleep 10
done
# The daemon runs under Node/tsx, which resolves linked workspace
# packages through their built `node` exports — refresh dist files.
bun run build:core
mkdir -p plugins/plugin-sql/node_modules/@elizaos
rm -rf plugins/plugin-sql/node_modules/@elizaos/core
ln -s ../../../../packages/core plugins/plugin-sql/node_modules/@elizaos/core
bun run --cwd plugins/plugin-sql build
# packages/scripts has no package.json, so bun's workspace graph
# doesn't link @elizaos/cloud-shared for the daemon — create it.
mkdir -p node_modules/@elizaos
ln -sfn ../../packages/cloud/shared node_modules/@elizaos/cloud-shared
sudo install -m 0644 \
packages/scripts/cloud/admin/eliza-apps-worker.service \
"/etc/systemd/system/$SYSTEMD_UNIT"
sudo systemctl daemon-reload
sudo systemctl enable "$SYSTEMD_UNIT"
sudo systemctl restart "$SYSTEMD_UNIT"
echo "=== Restart issued for apps worker ==="
- name: Health check
if: steps.deploy_config.outputs.configured == 'true'
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 5m
envs: SYSTEMD_UNIT
script: |
set -euo pipefail
# Hold the same host lock as the deploy step so a concurrent run's
# restart can never flip the unit mid-check (false red). 240s wait
# stays under this step's 5m command_timeout.
exec 9>/tmp/eliza-apps-worker-deploy.lock
flock -w 240 9 || { echo "::error::another apps-worker deploy holds the host lock after 4m"; exit 1; }
# The apps worker has no HTTP endpoint; health = active + stable +
# no fatal/restart-loop in the journal since deploy.
HEALTH_SINCE_TS="$(date -u '+%Y-%m-%d %H:%M:%S')"
STABLE_THRESHOLD_SEC=20
FATAL_LOG_PATTERN="\[apps-worker\] (fatal|unhandled rejection)|node:internal/.*Error:|Error \[ERR_|^[A-Za-z]+Error:"
for attempt in $(seq 1 18); do
if sudo systemctl is-active --quiet "$SYSTEMD_UNIT"; then
JOURNAL=$(sudo journalctl -u "$SYSTEMD_UNIT" --since "$HEALTH_SINCE_TS" --no-pager 2>/dev/null || true)
if echo "$JOURNAL" | grep -qE "$FATAL_LOG_PATTERN"; then
echo "::error::Apps worker logged a fatal error since deploy."
echo "$JOURNAL" | tail -n 50
exit 1
fi
UPTIME_SEC=$(systemctl show "$SYSTEMD_UNIT" --property=ActiveEnterTimestampMonotonic --value | awk '{ print int($1 / 1e6) }')
NOW_SEC=$(awk '{print int($1)}' /proc/uptime)
AGE=$(( NOW_SEC - UPTIME_SEC ))
if [ "$AGE" -ge "$STABLE_THRESHOLD_SEC" ]; then
echo "Apps worker active and stable (${AGE}s) on attempt $attempt."
exit 0
fi
echo "Health check attempt $attempt/18: active but only ${AGE}s old, waiting for stability..."
else
echo "Health check attempt $attempt/18: not active yet."
fi
sleep 5
done
echo "::error::$SYSTEMD_UNIT failed to become healthy within 90s."
sudo systemctl status "$SYSTEMD_UNIT" --no-pager || true
sudo journalctl -u "$SYSTEMD_UNIT" -n 200 --no-pager || true
exit 1
- name: Notify Discord (Success)
if: success() && steps.deploy_config.outputs.configured == 'true'
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "🚀 Eliza Apps Worker Deployed (Product 2)"
nodetail: true
description: |
Branch: ${{ needs.determine-env.outputs.branch }}
Commit: ${{ github.sha }}
color: 0x00ff00
- name: Notify Discord (Failure)
if: failure() && steps.deploy_config.outputs.configured == 'true'
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "❌ Eliza Apps Worker Deploy Failed"
nodetail: true
description: |
Branch: ${{ needs.determine-env.outputs.branch }}
Commit: ${{ github.sha }}
color: 0xff0000
@@ -0,0 +1,550 @@
name: Deploy Eliza Provisioning Worker
# SSHes to the Hetzner provisioning host and deploys one immutable commit,
# installs the systemd unit shipped at packages/scripts/cloud/admin/eliza-provisioning-worker.service,
# and restarts the eliza-provisioning-worker daemon.
#
# First-time setup:
# HCLOUD_TOKEN=... bun run packages/scripts/cloud/admin/bootstrap-provisioning-worker-host.mjs
on:
push:
branches: [develop, main]
paths:
- '.github/workflows/deploy-eliza-provisioning-worker.yml'
- 'packages/scripts/cloud/admin/daemons/provisioning-worker.ts'
- 'packages/scripts/cloud/admin/daemons/agent-router.ts'
- 'packages/scripts/cloud/admin/eliza-provisioning-worker.service'
- 'packages/scripts/cloud/admin/eliza-agent-router.service'
- 'packages/scripts/cloud/admin/ensure-generated-keywords.sh'
- 'packages/cloud/shared/src/lib/services/provisioning-jobs.ts'
- 'packages/cloud/shared/src/lib/services/docker-sandbox-provider.ts'
- 'packages/cloud/shared/src/lib/services/eliza-sandbox.ts'
- 'packages/cloud/shared/src/db/repositories/agent-sandboxes.ts'
- 'packages/cloud/shared/**'
- 'packages/cloud/sdk/**'
- 'packages/shared/**'
- 'packages/core/**'
- 'plugins/plugin-sql/**'
workflow_dispatch:
inputs:
environment:
description: 'Target environment (host + secrets resolved from this)'
required: true
default: 'staging'
type: choice
options:
- staging
- production
deployment_sha:
description: 'Optional exact repository commit for protected staging acceptance'
required: false
type: string
# Per-env concurrency: a develop push (staging) and a main push (production)
# can deploy in parallel without one cancelling the other.
concurrency:
group: deploy-eliza-provisioning-worker-${{ github.event.inputs.environment || (github.ref == 'refs/heads/main' && 'production' || 'staging') }}
cancel-in-progress: false
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
determine-env:
name: Determine environment
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
environment: ${{ steps.env.outputs.environment }}
branch: ${{ steps.env.outputs.branch }}
deployment_sha: ${{ steps.snapshot.outputs.deployment_sha }}
steps:
- id: env
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
env="${{ github.event.inputs.environment }}"
elif [ "${{ github.ref }}" = "refs/heads/main" ]; then
env="production"
else
env="staging"
fi
branch=$([ "$env" = "production" ] && echo "main" || echo "develop")
echo "environment=$env" >> "$GITHUB_OUTPUT"
echo "branch=$branch" >> "$GITHUB_OUTPUT"
echo "Resolved: environment=$env branch=$branch"
- id: snapshot
env:
BRANCH: ${{ steps.env.outputs.branch }}
PUSH_SHA: ${{ github.sha }}
REQUESTED_SHA: ${{ github.event.inputs.deployment_sha }}
TARGET_ENVIRONMENT: ${{ steps.env.outputs.environment }}
run: |
set -euo pipefail
if [ "${{ github.event_name }}" = "push" ]; then
deployment_sha="$PUSH_SHA"
elif [ -n "$REQUESTED_SHA" ]; then
[ "$TARGET_ENVIRONMENT" = "staging" ] || {
echo "::error::An exact deployment_sha is permitted only for protected staging deployments"
exit 1
}
[[ "$REQUESTED_SHA" =~ ^[0-9a-f]{40}$ ]] || {
echo "::error::deployment_sha must be a lowercase 40-hex commit"
exit 1
}
verify_dir=$(mktemp -d)
trap 'rm -rf "$verify_dir"' EXIT
git -C "$verify_dir" init --quiet
git -C "$verify_dir" fetch --quiet --no-tags --depth=1 \
"https://github.com/${GITHUB_REPOSITORY}.git" "$REQUESTED_SHA"
deployment_sha=$(git -C "$verify_dir" rev-parse FETCH_HEAD)
[ "$deployment_sha" = "$REQUESTED_SHA" ] || {
echo "::error::Fetched commit does not match requested deployment_sha"
exit 1
}
else
deployment_sha=$(git ls-remote "https://github.com/${GITHUB_REPOSITORY}.git" "refs/heads/$BRANCH" | awk 'NR == 1 { print $1 }')
fi
[[ "$deployment_sha" =~ ^[0-9a-f]{40}$ ]] || { echo "::error::Could not resolve immutable deployment SHA for $BRANCH"; exit 1; }
echo "deployment_sha=$deployment_sha" >> "$GITHUB_OUTPUT"
echo "Resolved immutable deployment snapshot: $deployment_sha"
deploy:
name: Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }})
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
needs: determine-env
environment: ${{ needs.determine-env.outputs.environment }}
timeout-minutes: 15
env:
SYSTEMD_UNIT: eliza-provisioning-worker.service
DEPLOY_HOST: ${{ secrets.ELIZA_PROVISIONING_HOST }}
DEPLOY_SSH_KEY: ${{ secrets.ELIZA_PROVISIONING_SSH_KEY }}
DEPLOY_BRANCH: ${{ needs.determine-env.outputs.branch }}
DEPLOY_SHA: ${{ needs.determine-env.outputs.deployment_sha }}
# Headscale wiring for the provisioning worker (the CONSUMER of the
# control plane). These are reconciled into /opt/eliza/cloud/.env.local
# on every deploy so the box can never silently drift to a stale,
# hand-edited headscale URL/key again. Reuse the canonical GH names
# already established by arm-headscale-control-plane.yml.
HEADSCALE_API_URL: ${{ vars.HEADSCALE_API_URL }}
HEADSCALE_PUBLIC_URL: ${{ vars.HEADSCALE_PUBLIC_URL }}
HEADSCALE_API_KEY: ${{ secrets.HEADSCALE_API_KEY }}
# Base64 private key used by the daemon to reach dedicated-agent nodes.
# Keep this in the protected environment instead of relying on a
# hand-copied file that can drift or disappear when the CP host is rebuilt.
CONTAINERS_SSH_KEY: ${{ secrets.CONTAINERS_SSH_KEY }}
# Sandbox-registry Redis (TCP redis:// proxy, sandbox-reachable — never a
# *.railway.internal host) so provisioned sandboxes self-register and
# Discord/Telegram inbound routing works (#8621 / #8756). Reconciled into
# /opt/eliza/cloud/.env.local the same way as HEADSCALE_*; when the secret
# is unset the reconcile loop skips it and preserves any hand-set value,
# so this is a no-op until the operator sets the GH secret.
SANDBOX_REGISTRY_REDIS_URL: ${{ secrets.SANDBOX_REGISTRY_REDIS_URL }}
# Cloud database DSN for the provisioning worker + agent router. Same
# fallback chain as the migrator jobs (cloud-cf-deploy.yml /
# cloud-deploy-backend.yml) so the CP daemon always resolves the exact
# DB the Worker schema is migrated against. Before this, DATABASE_URL
# only lived hand-edited on the box: when staging moved Neon → Railway
# the CP silently kept polling the abandoned Neon DB for 3 weeks
# (#15160). Reconciled into /opt/eliza/cloud/.env.local with the same
# skip-when-unset semantics as HEADSCALE_* — an environment without any
# of these secrets keeps its current hand-set value.
DATABASE_URL: ${{ secrets.DATABASE_URL || secrets.RAILWAY_DATABASE_URL || secrets.NEON_DATABASE_URL }}
# Field-encryption root secret used to unwrap per-org DEKs for encrypted
# agent environment variables. Must match the Cloudflare Worker secret:
# the Worker writes encrypted env vars, and this provisioning worker
# decrypts them while creating the agent container (#15385). Skip-empty
# reconcile semantics below preserve hand-set values until the secret is
# wired for an environment.
SECRETS_MASTER_KEY: ${{ secrets.SECRETS_MASTER_KEY }}
# Operator pin for the canonical managed-agent image
# (containersEnv.defaultAgentImageOverride; falls back to
# ghcr.io/elizaos/eliza:stable when unset everywhere). This pin decides
# which image every NEW provision and warm-pool replenish boots — it is
# exactly the knob the #15228 crash-loop rollout needed. Before this it
# lived ONLY hand-set on the CP box (the same drift class as
# DATABASE_URL/#15160, tracked in the #15401 ledger): a CP re-arm lost
# it silently. Set the GitHub environment VARIABLE (not secret — an
# image ref is auditable config) to roll the fleet pin via deploy;
# leave it unset to preserve whatever is hand-set on the box.
ELIZA_AGENT_IMAGE: ${{ vars.ELIZA_AGENT_IMAGE }}
# The edge Worker forwards the original per-agent host to this daemon.
# Keep the daemon's parser on the same environment-specific suffix or a
# staging host (`<uuid>.staging.elizacloud.ai`) is rejected as malformed.
ELIZA_CLOUD_AGENT_BASE_DOMAIN: ${{ needs.determine-env.outputs.environment == 'production' && 'elizacloud.ai' || 'staging.elizacloud.ai' }}
steps:
- name: Check deploy configuration
id: deploy_config
run: |
if [ -z "$DEPLOY_HOST" ] || [ -z "$DEPLOY_SSH_KEY" ]; then
echo "configured=false" >> "$GITHUB_OUTPUT"
echo "::warning::Missing ELIZA_PROVISIONING_HOST or ELIZA_PROVISIONING_SSH_KEY; skipping provisioning-worker deploy."
{
echo "### Provisioning worker deploy skipped"
echo ""
echo "Missing \`ELIZA_PROVISIONING_HOST\` or \`ELIZA_PROVISIONING_SSH_KEY\` for GitHub environment \`production\`."
echo ""
echo "Bootstrap the Hetzner host and secrets with:"
echo ""
echo "\`\`\`bash"
echo "HCLOUD_TOKEN=... bun run packages/scripts/cloud/admin/bootstrap-provisioning-worker-host.mjs"
echo "\`\`\`"
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "::error::Manual deployment requires ELIZA_PROVISIONING_HOST and ELIZA_PROVISIONING_SSH_KEY."
exit 1
fi
exit 0
fi
echo "configured=true" >> "$GITHUB_OUTPUT"
- name: Ensure host prereqs (Node 24, swap, bunx symlink)
if: steps.deploy_config.outputs.configured == 'true'
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 5m
script: |
set -euo pipefail
# Idempotent safety net for hosts provisioned BEFORE the cloud-init
# template that pre-installs these. Each block is a no-op if already
# set up — runs on every deploy so any host drift self-heals.
# Node 24: the systemd unit calls `tsx` which spawns a Node process.
# New cloud-init template installs it via NodeSource; older hosts
# (eliza-staging-1 from the first TF apply) lack it.
if ! command -v node >/dev/null 2>&1; then
echo "[host-prereqs] installing Node 24"
curl -fsSL https://deb.nodesource.com/setup_24.x | sudo bash -
sudo apt-get install -y nodejs
fi
# Swap: smaller shapes (cpx22, cax11) OOM during `turbo run build`.
# 8 GB swap once and forever; harmless on bigger shapes.
if [ ! -f /swapfile ]; then
echo "[host-prereqs] adding 8GB swap"
sudo fallocate -l 8G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
grep -q '^/swapfile ' /etc/fstab || echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab >/dev/null
fi
# bunx symlink: the canary `bun.sh/install` tarball stopped shipping
# bunx as a separate binary at some point; build scripts that call
# `bunx tsc` etc. then fail with `bun: command not found: bunx`.
# `bunx` is just `bun` reading argv[0] — a symlink is sufficient.
if [ ! -e /home/deploy/.bun/bin/bunx ] && [ -e /home/deploy/.bun/bin/bun ]; then
echo "[host-prereqs] creating bunx symlink"
ln -sf bun /home/deploy/.bun/bin/bunx
fi
- name: Deploy and restart worker
if: steps.deploy_config.outputs.configured == 'true'
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 10m
envs: DEPLOY_BRANCH,DEPLOY_SHA,SYSTEMD_UNIT,HEADSCALE_API_URL,HEADSCALE_PUBLIC_URL,HEADSCALE_API_KEY,CONTAINERS_SSH_KEY,SANDBOX_REGISTRY_REDIS_URL,DATABASE_URL,SECRETS_MASTER_KEY,ELIZA_AGENT_IMAGE,ELIZA_CLOUD_AGENT_BASE_DOMAIN
script: |
set -euo pipefail
echo "=== Deploying eliza-provisioning-worker (branch: $DEPLOY_BRANCH sha: $DEPLOY_SHA) ==="
cd /opt/eliza
export GIT_CONFIG_COUNT=1
export GIT_CONFIG_KEY_0=safe.directory
export GIT_CONFIG_VALUE_0=/opt/eliza
rm -f .git/index.lock
# A prior operator or deploy may have run git/install/build as
# root, leaving either Git metadata or tracked source files owned
# by root. Git then cannot reset or check out the immutable target
# as `deploy`. The checkout is exclusively managed by this service,
# so normalize the whole tree before any mutating Git command.
sudo chown -R deploy:deploy /opt/eliza
# Discard local drift, preserve env files. Quote globs so the
# remote shell doesn't expand them before git sees the pattern,
# and fail loud if clean errors out — silently swallowing it is
# what let stale build artifacts survive and block checkout below.
git reset --hard HEAD
git clean -fdx \
-e .env \
-e .env.local \
-e '.env.*' \
-e cloud/.env.local \
-e 'cloud/.env.*' \
-e node_modules
# Build artifacts (*.d.ts / *.d.ts.map) sometimes land next to
# sources from a prior deploy; if the new HEAD tracks the same
# paths, `git checkout -B` aborts with "untracked working tree
# files would be overwritten". Sweep them before checkout.
find packages -type f \( -name '*.d.ts' -o -name '*.d.ts.map' \) \
! -path '*/node_modules/*' -delete
# bun.lock became tracked at 4ae092dcda; the host's previous deploy
# may have left an untracked copy from `bun install` that pre-dates
# the tracking change. `git clean -fdx` *should* remove it but
# occasionally races with bun's lockfile rewrite, so force a delete
# before the fetch + branch-checkout below.
rm -f bun.lock
# Use a refspec so a single-branch / shallow clone still creates
# the remote-tracking ref for $DEPLOY_BRANCH on first deploy.
#
# --no-recurse-submodules on both fetch + checkout: the provisioning
# worker daemon and agent router are pure Node/tsx code and never
# touch the llama.cpp / opencode submodules. If the
# deploy host has `fetch.recurseSubmodules=on-demand` (the default
# when submodules are present), recursing through a superproject
# commit whose submodule SHA was never pushed to the fork kills the
# whole deploy with `upload-pack: not our ref`. Skip submodule
# traversal entirely on this path.
git -c fetch.recurseSubmodules=no fetch --no-recurse-submodules origin "$DEPLOY_SHA"
git -c submodule.recurse=false checkout --no-recurse-submodules \
-B "$DEPLOY_BRANCH" "$DEPLOY_SHA"
test "$(git rev-parse HEAD)" = "$DEPLOY_SHA"
# The `rm -f bun.lock` above deletes a TRACKED file; when bun.lock
# is identical between the previous HEAD and the new tip, checkout
# carries that deletion over as a local change and `bun install`
# then resolves the whole tree fresh from package.json ranges —
# newest-in-range @ai-sdk/* against the overridden provider-utils
# pin took the worker down at boot (`does not provide an export
# named 'secureJsonParse'`). Always restore the tracked lockfile so
# installs are lockfile-pinned and deterministic.
git checkout "$DEPLOY_SHA" -- bun.lock
# git clean removes these intentionally ignored modules. Generate
# and assert them before install/build, while the healthy daemons
# are still running, so generator failure cannot cause downtime.
bash packages/scripts/cloud/admin/ensure-generated-keywords.sh
# Install the canary (Rust) Bun on first install.
if ! command -v bun >/dev/null 2>&1; then
curl -fsSL https://bun.sh/install | bash -s "canary"
export BUN_INSTALL="$HOME/.bun"
export PATH="$BUN_INSTALL/bin:$PATH"
fi
cd /opt/eliza
for attempt in 1 2 3; do
if bun install --no-save --ignore-scripts; then
break
fi
if [ "$attempt" -eq 3 ]; then exit 1; fi
echo "bun install attempt $attempt failed; retrying in 10s..."
sleep 10
done
# The daemons run under Node/tsx rather than Bun. Node resolves
# linked workspace packages through their built `node` exports, so
# refresh linked package dist files after installing dependencies.
#
# The `git clean -fdx` above wipes `.turbo`, so every deploy does a
# cold `build:core` (0 cached). Building the full graph includes
# `@elizaos/plugin-local-inference`, whose tsc/bundler step exceeds
# V8's default ~2 GB old-space and aborts the deploy with
# `FATAL ERROR: ... JavaScript heap out of memory` (exit 134). Raise
# the heap ceiling to match the repo's typecheck lane (8 GB).
export NODE_OPTIONS="${NODE_OPTIONS:-} --max-old-space-size=8192"
bun run build:core
mkdir -p plugins/plugin-sql/node_modules/@elizaos
rm -rf plugins/plugin-sql/node_modules/@elizaos/core
ln -s ../../../../packages/core plugins/plugin-sql/node_modules/@elizaos/core
bun run --cwd plugins/plugin-sql build
# Defensive workspace symlinks: `bun install` should create these
# automatically for any workspace package the root or a sibling
# depends on, but the daemon (in packages/scripts/, which has NO
# package.json and so is invisible to bun's workspace graph)
# imports `@elizaos/cloud-shared` at runtime. On a fresh host the
# symlink was missing — daemon then crashes with
# `Cannot find package '@elizaos/cloud-shared'`. Recreate it here
# idempotently after `bun install` is done.
mkdir -p node_modules/@elizaos
ln -sfn ../../packages/cloud/shared node_modules/@elizaos/cloud-shared
# Install/refresh both systemd units (provisioning worker + agent router).
sudo install -m 0644 \
packages/scripts/cloud/admin/eliza-provisioning-worker.service \
"/etc/systemd/system/$SYSTEMD_UNIT"
sudo install -m 0644 \
packages/scripts/cloud/admin/eliza-agent-router.service \
/etc/systemd/system/eliza-agent-router.service
sudo systemctl daemon-reload
sudo systemctl enable "$SYSTEMD_UNIT" eliza-agent-router.service
# Reconcile control-plane secrets (HEADSCALE_API_URL /
# HEADSCALE_PUBLIC_URL / HEADSCALE_API_KEY /
# SANDBOX_REGISTRY_REDIS_URL / DATABASE_URL / SECRETS_MASTER_KEY)
# into the systemd EnvironmentFile so CI is the single source of
# truth and the box self-heals on every deploy. The daemon +
# cloud-shared read these straight from process.env
# (headscale-client.ts, headscale-integration.ts,
# docker-sandbox-provider.ts, the DB client, and encrypted
# environment-variable decrypt), fed by
# EnvironmentFile=/opt/eliza/cloud/.env.local. The
# deploy's `git clean -e cloud/.env.local` preserves that file
# verbatim, so before this these keys were only ever hand-edited on
# the box and could drift to a stale headscale (the CP-box outage)
# or a stale database (#15160: the CP polled an abandoned Neon DB
# for 3 weeks after staging moved to Railway). Only rewrite a key
# when the CI value is non-empty — an unset GH var/secret must
# never blank a working box (a blank HEADSCALE_API_KEY makes
# docker-sandbox-provider throw "HEADSCALE_API_KEY is not
# configured" when a route is required).
ENV_FILE=/opt/eliza/cloud/.env.local
sudo touch "$ENV_FILE"
# Pin this control-plane daemon to the AGENT lane. Unset, a single
# daemon claims ALL job types (agent + apps); it would claim
# APP_DEPLOY / CONTAINER_* jobs it can't run (no apps backend wired
# here) and burn them down through retries. The dedicated apps
# daemon (apps-provisioning-worker.ts) owns the apps lane. Reconciled
# here so CI is the single source of truth and the box self-heals on
# every deploy (same mechanism as the HEADSCALE_* keys below). This
# value is constant — not a GH var — so it's reconciled directly,
# never blanked.
sudo sed -i "/^PROVISIONING_JOB_LANES=/d" "$ENV_FILE"
printf 'PROVISIONING_JOB_LANES=%s\n' "agent" | sudo tee -a "$ENV_FILE" >/dev/null
# Headscale is the required dedicated-agent ingress. Bridge-host
# fallback was a legacy escape hatch; if it remains hand-set on the
# box, docker-sandbox-provider deliberately disables VPN injection
# and fresh staging agents never receive a headscale_ip (#15347).
sudo sed -i "/^AGENT_ROUTER_ALLOW_BRIDGE_HOST_FALLBACK=/d" "$ENV_FILE"
for kv in \
"HEADSCALE_API_URL=$HEADSCALE_API_URL" \
"HEADSCALE_PUBLIC_URL=$HEADSCALE_PUBLIC_URL" \
"HEADSCALE_API_KEY=$HEADSCALE_API_KEY" \
"CONTAINERS_SSH_KEY=$CONTAINERS_SSH_KEY" \
"SANDBOX_REGISTRY_REDIS_URL=$SANDBOX_REGISTRY_REDIS_URL" \
"DATABASE_URL=$DATABASE_URL" \
"SECRETS_MASTER_KEY=$SECRETS_MASTER_KEY" \
"ELIZA_AGENT_IMAGE=$ELIZA_AGENT_IMAGE" \
"ELIZA_CLOUD_AGENT_BASE_DOMAIN=$ELIZA_CLOUD_AGENT_BASE_DOMAIN"; do
key="${kv%%=*}"
val="${kv#*=}"
[ -n "$val" ] || continue
sudo sed -i "/^${key}=/d" "$ENV_FILE"
printf '%s=%s\n' "$key" "$val" | sudo tee -a "$ENV_FILE" >/dev/null
done
sudo chmod 600 "$ENV_FILE"
sudo systemctl restart "$SYSTEMD_UNIT"
sudo systemctl restart eliza-agent-router.service
echo "=== Restart issued for both daemons ==="
- name: Health check
if: steps.deploy_config.outputs.configured == 'true'
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 5m
envs: SYSTEMD_UNIT,ELIZA_CLOUD_AGENT_BASE_DOMAIN
script: |
set -euo pipefail
ENV_FILE=/opt/eliza/cloud/.env.local
ACTUAL_AGENT_BASE_DOMAIN=$(sudo awk -F= '$1=="ELIZA_CLOUD_AGENT_BASE_DOMAIN" { value=$2 } END { print value }' "$ENV_FILE")
if [ "$ACTUAL_AGENT_BASE_DOMAIN" != "$ELIZA_CLOUD_AGENT_BASE_DOMAIN" ]; then
echo "::error::Agent router base-domain drift: expected $ELIZA_CLOUD_AGENT_BASE_DOMAIN, found ${ACTUAL_AGENT_BASE_DOMAIN:-<missing>}."
exit 1
fi
# The worker logs to journald via systemd's stdout/stderr capture.
# Info-level logs are filtered unless VERBOSE_LOGGING=true, so we
# rely on systemd's own signals: process is active AND no fatal
# error / restart-loop in the journal since deploy.
HEALTH_SINCE_TS="$(date -u '+%Y-%m-%d %H:%M:%S')"
STABLE_THRESHOLD_SEC=20
FATAL_LOG_PATTERN="\[(provisioning-worker|agent-router)\] (fatal|unhandled rejection)|node:internal/.*Error:|Error \[ERR_|^[A-Za-z]+Error:"
for attempt in $(seq 1 18); do
if sudo systemctl is-active --quiet "$SYSTEMD_UNIT"; then
JOURNAL=$(sudo journalctl -u "$SYSTEMD_UNIT" --since "$HEALTH_SINCE_TS" --no-pager 2>/dev/null || true)
if echo "$JOURNAL" | grep -qE "$FATAL_LOG_PATTERN"; then
echo "::error::Worker logged a fatal error since deploy."
echo "$JOURNAL" | tail -n 50
exit 1
fi
# Active + uptime past the threshold = healthy. systemd's Restart=always
# means a crashing process would resurrect; an uptime > threshold
# therefore proves it stayed up across at least one poll cycle.
UPTIME_SEC=$(systemctl show "$SYSTEMD_UNIT" --property=ActiveEnterTimestampMonotonic --value | awk '{ print int($1 / 1e6) }')
NOW_SEC=$(awk '{print int($1)}' /proc/uptime)
AGE=$(( NOW_SEC - UPTIME_SEC ))
if [ "$AGE" -ge "$STABLE_THRESHOLD_SEC" ]; then
if ! sudo systemctl is-active --quiet eliza-agent-router.service; then
echo "Health check attempt $attempt/18: worker stable (${AGE}s), router not active yet."
sleep 5
continue
fi
ROUTER_JOURNAL=$(sudo journalctl -u eliza-agent-router.service --since "$HEALTH_SINCE_TS" --no-pager 2>/dev/null || true)
if echo "$ROUTER_JOURNAL" | grep -qE "$FATAL_LOG_PATTERN"; then
echo "::error::Agent router logged a fatal error since activation."
echo "$ROUTER_JOURNAL" | tail -n 50
exit 1
fi
ROUTER_UPTIME_SEC=$(systemctl show eliza-agent-router.service --property=ActiveEnterTimestampMonotonic --value | awk '{ print int($1 / 1e6) }')
ROUTER_AGE=$(( NOW_SEC - ROUTER_UPTIME_SEC ))
if [ "$ROUTER_AGE" -lt "$STABLE_THRESHOLD_SEC" ]; then
echo "Health check attempt $attempt/18: worker stable (${AGE}s), router only ${ROUTER_AGE}s old."
sleep 5
continue
fi
ROUTER_PORT=$(systemctl show eliza-agent-router.service --property=Environment --value | tr ' ' '\n' | awk -F= '$1=="AGENT_ROUTER_PORT" {print $2}')
: "${ROUTER_PORT:=3458}"
if curl -sf -m 3 "http://127.0.0.1:${ROUTER_PORT}/healthz" >/dev/null 2>&1; then
echo "Both daemons active and stable (worker ${AGE}s, router ${ROUTER_AGE}s, router /healthz OK) on attempt $attempt."
exit 0
fi
echo "Health check attempt $attempt/18: worker stable (${AGE}s), router ${ROUTER_AGE}s but /healthz not ready on port ${ROUTER_PORT}."
sleep 5
continue
fi
echo "Health check attempt $attempt/18: active but only ${AGE}s old, waiting for stability..."
else
echo "Health check attempt $attempt/18: not active yet."
fi
sleep 5
done
echo "::error::$SYSTEMD_UNIT failed to become healthy within 90s."
sudo systemctl status "$SYSTEMD_UNIT" --no-pager || true
sudo journalctl -u "$SYSTEMD_UNIT" -n 200 --no-pager || true
sudo systemctl status eliza-agent-router.service --no-pager || true
sudo journalctl -u eliza-agent-router.service -n 200 --no-pager || true
exit 1
- name: Notify Discord (Success)
if: success() && steps.deploy_config.outputs.configured == 'true'
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "🚀 Eliza Provisioning Worker Deployed"
nodetail: true
description: |
Branch: develop
Commit: ${{ github.sha }}
color: 0x00ff00
- name: Notify Discord (Failure)
if: failure() && steps.deploy_config.outputs.configured == 'true'
uses: sarisia/actions-status-discord@eb045afee445dc055c18d3d90bd0f244fd062708
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
ack_no_webhook: true
title: "❌ Eliza Provisioning Worker Deploy Failed"
nodetail: true
description: |
Branch: develop
Commit: ${{ github.sha }}
color: 0xff0000
# ci-touch: re-trigger prod daemon deploy (dispatch runs were runner-starved) 2026-06-21
+200
View File
@@ -0,0 +1,200 @@
# Deploy packages/homepage to elizaOS/elizaos.github.io (org GitHub Pages site).
#
# What this does:
# 1. Builds packages/homepage (Vite static site) — the `prebuild` script in
# packages/homepage/package.json regenerates
# src/generated/release-data.ts from the GitHub Releases API before vite
# runs, so download buttons always reflect the latest published assets.
# 2. Writes a CNAME file (`eliza.app`) into dist/ so GitHub Pages serves the
# static output at https://eliza.app/. The org-pages URL
# https://elizaos.github.io/ also serves the same content (it redirects to
# eliza.app once the CNAME is in place).
# 3. Adds `.nojekyll` so GitHub Pages serves files/dirs that begin with `_`.
# 4. Pushes the built site to the `gh-pages` branch of
# elizaOS/elizaos.github.io using a deploy key. We deliberately push to
# `gh-pages` (not `main`) so the leaderboard source code on `main` stays
# untouched in git history.
#
# Required secret on this repo (elizaOS/eliza):
# HOMEPAGE_DEPLOY_KEY
# - SSH private key (PEM, e.g. `ssh-keygen -t ed25519 -C "homepage-deploy" -f homepage-deploy -N ""`).
# - The corresponding PUBLIC key must be registered as a Deploy Key with
# write access on the target repo elizaOS/elizaos.github.io.
#
# One-time setup on elizaOS/elizaos.github.io (the leaderboard repo):
# - In Settings → Pages, set source to:
# "Deploy from a branch" → Branch: `gh-pages` / `(root)`.
# This switches Pages off the leaderboard's old "GitHub Actions" source.
# - Disable or delete the leaderboard's existing `.github/workflows/deploy.yml`
# so it does not race with this workflow (its `actions/deploy-pages` calls
# become no-ops once Pages source is changed, but disabling it removes the
# spurious failed-run noise).
#
# DNS for eliza.app:
# - Add A records for the apex (eliza.app) pointing at GitHub Pages IPs:
# 185.199.108.153, 185.199.109.153, 185.199.110.153, 185.199.111.153
# - Or a CNAME record for www.eliza.app → elizaos.github.io.
# - In elizaos.github.io repo Settings → Pages, set Custom domain = eliza.app
# and enable "Enforce HTTPS" once the cert is provisioned.
name: Deploy Homepage
concurrency:
group: deploy-homepage
cancel-in-progress: true
on:
push:
branches: [develop]
paths:
- "packages/homepage/**"
- "packages/app-core/scripts/write-homepage-release-data.mjs"
- "packages/app-core/scripts/lib/asset-cdn.mjs"
- ".github/workflows/deploy-homepage.yml"
workflow_call:
inputs:
ref:
description: "Git ref to build the homepage from (defaults to the calling workflow ref)"
required: false
type: string
workflow_dispatch:
inputs:
ref:
description: "Git ref to build the homepage from"
required: false
type: string
permissions:
contents: write
jobs:
build-and-deploy:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 20
steps:
- name: Checkout source repo
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
ref: ${{ inputs.ref || github.ref }}
submodules: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Install homepage dependencies
working-directory: packages/homepage
run: bun install --no-save --ignore-scripts
- name: Refresh homepage release data
working-directory: packages/homepage
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: node ../app-core/scripts/write-homepage-release-data.mjs
- name: Check homepage release data
id: release_data_check
working-directory: packages/homepage
shell: bash
run: |
if bun run check:release-data; then
echo "available=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "${{ github.event_name }}" = "workflow_call" ]; then
echo "::error::Release-triggered homepage deploys require published release data."
exit 1
fi
echo "::warning::No complete published release data is available; skipping homepage deploy for this develop push."
echo "available=false" >> "$GITHUB_OUTPUT"
- name: Build homepage
if: steps.release_data_check.outputs.available == 'true'
working-directory: packages/homepage
run: bun run build
- name: Install homepage browser
if: steps.release_data_check.outputs.available == 'true'
working-directory: packages/homepage
run: ./node_modules/.bin/playwright install --with-deps chromium
- name: Generate missing snapshot baselines
if: steps.release_data_check.outputs.available == 'true'
id: gen-baselines
working-directory: packages/homepage
run: |
SNAP_DIR="tests/e2e/visual.spec.ts-snapshots"
# 5 routes × 2 viewports × 1 browser project = 10 expected snapshots
EXPECTED=10
ACTUAL=$(ls "$SNAP_DIR"/*.png 2>/dev/null | wc -l || echo 0)
if [ "$ACTUAL" -lt "$EXPECTED" ]; then
echo "Found $ACTUAL of $EXPECTED snapshot baselines — regenerating all."
rm -f "$SNAP_DIR"/*.png 2>/dev/null || true
bun run test:e2e -- --update-snapshots || true
echo "generated=true" >> "$GITHUB_OUTPUT"
else
echo "generated=false" >> "$GITHUB_OUTPUT"
fi
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Commit new snapshot baselines
if: steps.release_data_check.outputs.available == 'true' && steps.gen-baselines.outputs.generated == 'true'
continue-on-error: true
run: |
SNAP_DIR="packages/homepage/tests/e2e/visual.spec.ts-snapshots"
if [ -n "$(git status --porcelain $SNAP_DIR)" ]; then
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add $SNAP_DIR
git commit -m "chore: generate visual regression snapshot baselines [skip ci]"
git push origin HEAD:${{ github.ref_name }} || (git pull --rebase origin ${{ github.ref_name }} && git push origin HEAD:${{ github.ref_name }})
echo "Pushed new snapshot baselines."
fi
- name: Test homepage downloads
if: steps.release_data_check.outputs.available == 'true' && steps.gen-baselines.outputs.generated == 'false'
working-directory: packages/homepage
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: bun run test:e2e
- name: Write CNAME and .nojekyll
if: steps.release_data_check.outputs.available == 'true'
run: |
echo "eliza.app" > packages/homepage/dist/CNAME
touch packages/homepage/dist/.nojekyll
- name: Check deploy key availability
if: steps.release_data_check.outputs.available == 'true'
id: deploy_key_check
shell: bash
env:
DEPLOY_KEY: ${{ secrets.HOMEPAGE_DEPLOY_KEY }}
run: |
if [ -n "$DEPLOY_KEY" ]; then
echo "available=true" >> "$GITHUB_OUTPUT"
else
echo "available=false" >> "$GITHUB_OUTPUT"
if [ "${{ github.event_name }}" = "workflow_call" ]; then
echo "::error::HOMEPAGE_DEPLOY_KEY secret not configured. Release-triggered homepage deploys must fail closed instead of silently skipping publication."
exit 1
fi
echo "::warning::HOMEPAGE_DEPLOY_KEY secret not configured. Skipping deploy step. The site was built successfully but will not be published. To enable publishing, register an SSH deploy key on elizaOS/elizaos.github.io and add the private key as HOMEPAGE_DEPLOY_KEY on this repo (see workflow header for details)."
fi
- name: Deploy to elizaOS/elizaos.github.io (gh-pages branch)
if: steps.release_data_check.outputs.available == 'true' && steps.deploy_key_check.outputs.available == 'true'
uses: peaceiris/actions-gh-pages@84c30a85c19949d7eee79c4ff27748b70285e453
with:
deploy_key: ${{ secrets.HOMEPAGE_DEPLOY_KEY }}
external_repository: elizaOS/elizaos.github.io
publish_branch: gh-pages
publish_dir: ./packages/homepage/dist
force_orphan: true
user_name: "github-actions[bot]"
user_email: "github-actions[bot]@users.noreply.github.com"
commit_message: "deploy: homepage from elizaOS/eliza@${{ github.sha }}"
+220
View File
@@ -0,0 +1,220 @@
name: Dev Smoke
on:
# Reusable entry for the scheduled exhaustive develop lane (#12342).
workflow_call:
pull_request:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review, labeled]
push:
branches: [main, develop]
paths:
- ".github/workflows/dev-smoke.yml"
- "package.json"
- "bun.lock"
- "packages/app/**"
- "packages/app-core/**"
- "packages/core/**"
- "packages/shared/**"
- "packages/ui/**"
workflow_dispatch:
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: dev-smoke-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
permissions:
contents: read
env:
CI: "true"
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
NODE_NO_WARNINGS: "1"
# WHY: the dev smoke needs a real model when one is available, but forcing the
# lane through one provider makes develop red when that provider hits daily
# quota. The shared live-provider selector chooses from these keys in order and
# skips the Playwright spec cleanly when no usable provider is configured.
# Cerebras remains available to local/manual runs through the harness, but the
# scheduled CI lane does not force it after repeated daily-quota exhaustion.
# OpenAI is also intentionally omitted here: the live app currently sends a
# memory-response schema that OpenAI rejects strictly, which tests that
# provider contract rather than the dev/onboarding plumbing this lane owns.
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ELIZAOS_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY }}
jobs:
changes:
name: Classify changed paths
# Path classifier is a git-diff + node script: no self-hosted resources.
# Keep it on GitHub-hosted so a drained hetzner fleet cannot leave it
# queued indefinitely and pile up runs (#13617/#8501).
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
outputs:
dev_smoke: ${{ steps.filter.outputs.dev_smoke }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
submodules: false
- name: Setup Node.js for path gate
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Determine affected dev smoke surface
id: filter
shell: bash
env:
PR_LABELS: ${{ github.event_name == 'pull_request' && join(github.event.pull_request.labels.*.name, ',') || '' }}
run: |
set -euo pipefail
node packages/scripts/ci-path-gate.mjs \
--config dev-smoke \
--event "${{ github.event_name }}" \
--base "${{ github.event.pull_request.base.sha }}" \
--head "${{ github.event.pull_request.head.sha }}" \
--labels "$PR_LABELS" \
--output "$GITHUB_OUTPUT" \
--summary "$GITHUB_STEP_SUMMARY"
dev-smoke:
name: bun run dev onboarding chat
needs: changes
if: github.event_name != 'pull_request' || needs.changes.outputs.dev_smoke == 'true'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 35
env:
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Free disk space for browser smoke
run: |
set -euxo pipefail
df -h /
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
df -h /
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
- name: Generate shared i18n assets
run: bun run --cwd packages/shared build:i18n
# Without a prebuilt dist/, the renderer's vite dev server serves
# @elizaos/core from its source graph (400+ on-demand module transforms),
# which is too slow on CI for the chat composer to mount inside the test's
# 90s window. Prebuilding core (and its workspace deps) makes the dev server
# serve the compiled browser bundle instead. See resolveElizaCoreBundlePath
# in packages/app/vite.config.ts.
# @elizaos/shared is also built because its `./*` subpath exports map to a
# gitignored dist/ (e.g. @elizaos/shared/utils/permission-deep-links, used
# by PermissionRecoveryCallout). Without it the dev server can't resolve
# the subpath and the smoke fails at import time.
- name: Build @elizaos/core browser bundle
run: bunx turbo run build --filter=@elizaos/core --filter=@elizaos/shared
- name: Run bun dev onboarding chat smoke
run: bun run --cwd packages/app test:dev-smoke
# The smoke has historically failed inside gotoChatComposer (the /chat
# composer never paints) with no captured renderer state, making the
# actual blocking gate impossible to pin down. Upload the Playwright
# trace/screenshot/error-context so the failure is diagnosable.
- name: Upload dev-smoke artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: dev-smoke-results
path: |
packages/app/playwright-report/
packages/app/test-results/
if-no-files-found: ignore
hmr:
name: Vite HMR dependency-level smoke
needs: changes
if: github.event_name != 'pull_request' || needs.changes.outputs.dev_smoke == 'true'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 35
env:
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Free disk space for browser smoke
run: |
set -euxo pipefail
df -h /
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
df -h /
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
- name: Generate shared i18n assets
run: bun run --cwd packages/shared build:i18n
- name: Build @elizaos/core browser bundle
run: bunx turbo run build --filter=@elizaos/core --filter=@elizaos/shared
- name: Run Vite HMR dependency-level smoke
run: bun run test:hmr
- name: Upload HMR artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: hmr-results
path: |
packages/app/playwright-report/
packages/app/test-results/hmr/
if-no-files-found: ignore
+158
View File
@@ -0,0 +1,158 @@
name: Develop Exhaustive Lane
# Un-cancellable scheduled run of the FULL develop matrix (#12342, epic #12191
# phase 5). test.yml's own 09:17 schedule already covers the unit / e2e /
# plugin / desktop / zero-key lanes; this orchestrator adds the platform lanes
# that otherwise only run path-gated on PRs — Windows, mobile, scenario, the
# three UI gates, the keyless harness, docker, dev onboarding, and the
# electrobun/desktop contract — by invoking each via `workflow_call` so no step
# is duplicated. workflow_call bypasses the per-workflow path filters, and a
# schedule-origin event makes every called workflow's
# `github.event_name != 'pull_request'` job gate evaluate true, so the lanes run
# unfiltered (exhaustive). The final proof job statically asserts the committed
# manifest still matches test.yml AND that every required reusable workflow is
# wired here and declares workflow_call, then fails the run if any reusable lane
# did not succeed — a skipped or failed platform lane is a coverage gap, not a
# pass.
on:
schedule:
# Twice daily, offset from test.yml (09:17) and ci-full-matrix-proof (03:13/
# 15:13) so the three scheduled signals do not stampede the runner pool.
- cron: "0 6 * * *"
- cron: "0 18 * * *"
workflow_dispatch:
# Dedicated group, never cancelled by push/PR traffic: the whole point is a
# completed daily coverage signal (#12337/#12342).
concurrency:
group: develop-exhaustive-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
packages: read
actions: read
jobs:
windows:
name: Windows full matrix
uses: ./.github/workflows/windows-ci.yml
secrets: inherit
mobile:
name: Mobile build smoke
uses: ./.github/workflows/mobile-build-smoke.yml
secrets: inherit
scenario:
name: Scenario deterministic E2E
uses: ./.github/workflows/scenario-pr.yml
secrets: inherit
ui-e2e:
name: UI e2e gate
uses: ./.github/workflows/ui-e2e-gate.yml
secrets: inherit
ui-fixture:
name: UI fixture e2e
uses: ./.github/workflows/ui-fixture-e2e.yml
secrets: inherit
ui-story:
name: UI story gate
uses: ./.github/workflows/ui-story-gate.yml
secrets: inherit
keyless-harness:
name: Keyless harness E2E
uses: ./.github/workflows/keyless-harness-e2e.yml
secrets: inherit
docker:
name: Docker image smoke
uses: ./.github/workflows/docker-ci-smoke.yml
secrets: inherit
dev-onboarding:
name: Dev onboarding smoke
uses: ./.github/workflows/dev-smoke.yml
secrets: inherit
electrobun:
name: Electrobun desktop release
uses: ./.github/workflows/test-electrobun-release.yml
secrets: inherit
matrix-proof:
name: Exhaustive lane matrix proof
needs:
- windows
- mobile
- scenario
- ui-e2e
- ui-fixture
- ui-story
- keyless-harness
- docker
- dev-onboarding
- electrobun
# Run even when a reusable lane failed so the proof enumerates every lane
# and the result check below can name the offender; only a cancel skips it.
if: ${{ !cancelled() }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
env:
CI: "true"
BUN_VERSION: "1.3.14"
NODE_VERSION: "24.15.0"
NODE_NO_WARNINGS: "1"
NODE_OPTIONS: "--max-old-space-size=8192"
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Assert every reusable lane succeeded (skipped/failed == gap)
env:
RESULTS: >-
windows=${{ needs.windows.result }}
mobile=${{ needs.mobile.result }}
scenario=${{ needs.scenario.result }}
ui-e2e=${{ needs.ui-e2e.result }}
ui-fixture=${{ needs.ui-fixture.result }}
ui-story=${{ needs.ui-story.result }}
keyless-harness=${{ needs.keyless-harness.result }}
docker=${{ needs.docker.result }}
dev-onboarding=${{ needs.dev-onboarding.result }}
electrobun=${{ needs.electrobun.result }}
run: |
fail=0
for pair in $RESULTS; do
lane="${pair%%=*}"
result="${pair##*=}"
if [ "$result" = "success" ]; then
echo "lane $lane: success"
else
echo "::error::exhaustive lane $lane did not succeed (result=$result) — a skipped or failed platform lane is a coverage gap, not a pass"
fail=1
fi
done
exit "$fail"
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --frozen-lockfile --ignore-scripts
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Validate GitHub-native turbo cache contract
run: node packages/scripts/ci-turbo-cache-contract.mjs
- name: Validate workflow dedup contract
run: node packages/scripts/ci-workflow-dedup-contract.mjs
- name: Capture exhaustive test plan
run: node packages/scripts/run-all-tests.mjs --plan=json > /tmp/full-matrix-plan.json
- name: Prove exhaustive lane coverage (manifest + reusable wiring)
run: node packages/scripts/ci-full-matrix-proof.mjs --plan-file /tmp/full-matrix-plan.json
+146
View File
@@ -0,0 +1,146 @@
# Nightly post-merge live sweep — the only place TEST_LANE=post-merge runs the
# whole workspace matrix. The PR/develop gates are deliberately deterministic
# and secret-free, which means every *.real.test.ts / *.live.test.ts suite is
# excluded there; without this lane, live-API coverage exists only as a plan.
# run-all-tests' post-merge lane prints the guarded-suite accounting (armed /
# missing-creds / opt-in / blocked, from packages/scripts/lib/real-live-suites.mjs)
# and hard-fails on manifest drift, so a missing secret is a loud named skip,
# never a silent green.
#
# Scheduled runs are gated on vars.ELIZA_DEVELOP_LIVE_ENABLED == 'true' so the
# lane can be armed org-side once runner capacity allows (#13309); manual
# dispatch always works. Local operators get the same sweep interactively via
# `bun run test:console` (packages/scripts/test-console).
name: Develop Live Sweep
on:
workflow_dispatch:
inputs:
filter:
description: "Optional --filter regex over task labels (default: whole matrix)"
type: string
default: ""
schedule:
# 04:30 UTC — clear of develop-exhaustive (06:00/18:00) and test.yml (09:17).
- cron: "30 4 * * *"
permissions:
contents: read
concurrency:
group: develop-live
cancel-in-progress: false
jobs:
live-sweep:
name: Post-merge live matrix
if: github.event_name == 'workflow_dispatch' || vars.ELIZA_DEVELOP_LIVE_ENABLED == 'true'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 480
env:
TEST_LANE: post-merge
ELIZA_LIVE_TEST: "1"
ELIZA_REAL_APIS: "1"
ELIZA_SKIP_ARTIFACT_SYNC: "1"
# Canonical post-merge secret set (packages/scripts/post-merge-secrets.txt).
# A missing secret self-skips its suites with a named accounting line.
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
ELEVENLABS_API_KEY: ${{ secrets.ELEVENLABS_API_KEY }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
DISCORD_TEST_GUILD_ID: ${{ secrets.DISCORD_TEST_GUILD_ID }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
SLACK_TEST_CHANNEL_ID: ${{ secrets.SLACK_TEST_CHANNEL_ID }}
TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TELEGRAM_TEST_CHAT_ID: ${{ secrets.TELEGRAM_TEST_CHAT_ID }}
WHATSAPP_TOKEN: ${{ secrets.WHATSAPP_TOKEN }}
WHATSAPP_PHONE_NUMBER_ID: ${{ secrets.WHATSAPP_PHONE_NUMBER_ID }}
X_BEARER_TOKEN: ${{ secrets.X_BEARER_TOKEN }}
# The ephemeral Actions token satisfies the read-only GitHub live suites.
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
CALENDLY_API_KEY: ${{ secrets.CALENDLY_API_KEY }}
CALENDLY_ACCESS_TOKEN: ${{ secrets.CALENDLY_ACCESS_TOKEN }}
TWILIO_ACCOUNT_SID: ${{ secrets.TWILIO_ACCOUNT_SID }}
TWILIO_AUTH_TOKEN: ${{ secrets.TWILIO_AUTH_TOKEN }}
GOOGLE_OAUTH_REFRESH_TOKEN: ${{ secrets.GOOGLE_OAUTH_REFRESH_TOKEN }}
BLUESKY_PASSWORD: ${{ secrets.BLUESKY_PASSWORD }}
FARCASTER_NEYNAR_API_KEY: ${{ secrets.FARCASTER_NEYNAR_API_KEY }}
SHOPIFY_API_KEY: ${{ secrets.SHOPIFY_API_KEY }}
HYPERLIQUID_PRIVATE_KEY: ${{ secrets.HYPERLIQUID_PRIVATE_KEY }}
POLYMARKET_API_KEY: ${{ secrets.POLYMARKET_API_KEY }}
STRAVA_ACCESS_TOKEN: ${{ secrets.STRAVA_ACCESS_TOKEN }}
OURA_ACCESS_TOKEN: ${{ secrets.OURA_ACCESS_TOKEN }}
FITBIT_ACCESS_TOKEN: ${{ secrets.FITBIT_ACCESS_TOKEN }}
WITHINGS_ACCESS_TOKEN: ${{ secrets.WITHINGS_ACCESS_TOKEN }}
GOOGLE_CALENDAR_ACCESS_TOKEN: ${{ secrets.GOOGLE_CALENDAR_ACCESS_TOKEN }}
ATLASCLOUD_API_KEY: ${{ secrets.ATLASCLOUD_API_KEY }}
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
BIRDEYE_API_KEY: ${{ secrets.BIRDEYE_API_KEY }}
ELIZAOS_CLOUD_API_KEY: ${{ secrets.ELIZAOS_CLOUD_API_KEY }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454)
install-command: bun install
- name: Build workspace
run: bun run build
# plugin-sql's RLS *.real.test.ts suites gate on POSTGRES_URL with the
# same URL run-all-tests auto-provisions locally. Best-effort: if the
# runner has no docker, the suites self-skip with a named accounting line.
- name: Provision Postgres (best-effort)
run: |
if command -v docker >/dev/null 2>&1; then
docker rm -f develop-live-postgres 2>/dev/null || true
docker run -d --name develop-live-postgres \
-e POSTGRES_USER=eliza_test -e POSTGRES_PASSWORD=test123 -e POSTGRES_DB=eliza_test \
-p 127.0.0.1:5433:5432 postgres:17
echo "POSTGRES_URL=postgresql://eliza_test:test123@127.0.0.1:5433/eliza_test" >> "$GITHUB_ENV"
else
echo "docker unavailable; plugin-sql Postgres real suites will self-skip"
fi
- name: Post-merge live matrix
env:
RUNNER_TEMP_LOG: ${{ runner.temp }}/develop-live-sweep.log
TEST_FILTER: ${{ inputs.filter }}
run: |
set -o pipefail
filter_args=()
if [ -n "${TEST_FILTER:-}" ]; then
filter_args+=(--filter="$TEST_FILTER")
fi
node packages/scripts/run-all-tests.mjs --all --no-cloud "${filter_args[@]}" 2>&1 | tee "$RUNNER_TEMP_LOG"
- name: Cloud unit suite
if: always()
run: bun run test:cloud
- name: Teardown Postgres
if: always()
run: docker rm -f develop-live-postgres 2>/dev/null || true
- name: Upload sweep log
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: develop-live-sweep-log
path: ${{ runner.temp }}/develop-live-sweep.log
if-no-files-found: ignore
+178
View File
@@ -0,0 +1,178 @@
name: Develop PR
# Lightweight gate for pull requests targeting `develop`: lint, typecheck, and
# build only. The full test/e2e/scenario/benchmark surface runs post-merge on
# push to `develop` (and nightly/on-demand), not on every PR — so PR feedback
# stays fast and hosted-runner capacity is spent on the ~200 daily PRs' fast
# signal rather than the whole integration matrix. `main` PRs keep the heavy
# gates via `ci.yaml`, `test.yml`'s peers, and `quality.yml`. Steps mirror
# `ci.yaml` so PR and post-merge lanes stay in agreement.
on:
pull_request:
branches: [develop]
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: develop-pr-${{ github.event.pull_request.number }}
cancel-in-progress: true
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
lane-coverage:
name: Test Integrity (lane coverage)
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
# Full history: the error-policy ratchet diffs every touched file
# against its content at the merge-base with origin/develop.
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Per-plugin e2e coverage
run: node packages/scripts/lint-lane-coverage.mjs
# Binding policy audits from CLAUDE.md that previously ran nowhere in
# CI. Both are dependency-free node scripts (<5s): the error-policy
# ratchet fails only when a file this PR touches ADDS an empty catch or
# server-side console.* call; test-integrity statically lints test
# sources for larp patterns.
- name: Error-policy ratchet (diff-scoped)
run: |
git fetch --no-tags origin develop:refs/remotes/origin/develop || true
node packages/scripts/error-policy-ratchet.mjs
- name: Test-integrity audit
run: node packages/scripts/lint-test-integrity.mjs
- name: Test-integrity audit self-test
run: node packages/scripts/lint-test-integrity.self-test.mjs
- name: Changed-file coverage classifier self-tests
run: |
while IFS= read -r self_test; do
if [ ! -f "$self_test" ]; then
echo "registered coverage self-test is missing: $self_test"
exit 1
fi
node "$self_test"
done < scripts/security/coverage-node-self-tests.txt
# View→action ratchet (#14369): every builtin-view on-screen mutation
# must map to a registered agent action (chat twin), and the generated
# action catalog must reflect the real registered surface. Dependency-free
# (node stdlib only) like the other audits in this job.
- name: View-action ratchet self-test
run: node packages/scripts/view-action-ratchet.mjs --self-test
- name: View-action ratchet
run: node packages/scripts/view-action-ratchet.mjs
lint:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454)
install-command: bun install
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Run lint
run: bun run lint
- name: Check formatting
run: bun run format:check
typecheck:
runs-on: ubuntu-latest
# 35 not 20: at --concurrency=4 a near-total affected cone (~250 packages)
# needs headroom — the one green full-cone run spent 11 minutes in the
# typecheck step alone, on top of checkout/install/build:core (#15140).
timeout-minutes: 35
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# Full history so `turbo run typecheck --affected` can resolve the PR
# merge base (#12341); a shallow clone degrades to typechecking all.
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454)
install-command: bun install
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
# Typecheck resolves `@elizaos/core` types from its built dist, so build
# core before running tsc across the affected cone.
- name: Build core
run: bun run build:core
# --concurrency=4 not 8: hosted ubuntu-latest runners have 4 vCPUs and
# 16 GB RAM, and when the affected cone goes near-total (a base-branch
# commit touching turbo globalDependencies like root package.json /
# turbo.json makes every stale-base PR typecheck all ~250 packages) eight
# concurrent tsgo processes exhaust the runner — the OS SIGKILLs tasks or
# the runner VM itself dies (#15140). NODE_OPTIONS only bounds node-based
# tools, not native tsgo, so concurrency is the memory lever here.
- name: Run typecheck (affected)
run: NODE_OPTIONS='--max-old-space-size=8192' node packages/scripts/run-turbo.mjs run typecheck --concurrency=4 --affected
env:
TURBO_SCM_BASE: ${{ github.event.pull_request.base.sha }}
build:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454)
install-command: bun install
install-native-deps: "false"
install-protoc: "false"
setup-python: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Build packages
run: bun run build:core
+165
View File
@@ -0,0 +1,165 @@
name: Develop Staging Beta Release
on:
workflow_run:
workflows: ["Cloud CF Deploy"]
types: [completed]
workflow_dispatch:
inputs:
target_sha:
description: Develop commit SHA to release. Defaults to the workflow ref SHA.
required: false
type: string
permissions:
actions: write
checks: read
contents: read
concurrency:
group: develop-staging-beta-${{ github.event.workflow_run.head_sha || inputs.target_sha || github.sha }}
cancel-in-progress: false
jobs:
gate-and-release:
name: Gate Develop and Dispatch Beta
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
if: >-
${{
github.event_name == 'workflow_dispatch' ||
(
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'develop' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_repository.full_name == github.repository
)
}}
steps:
- name: Resolve target commit
id: target
env:
EVENT_NAME: ${{ github.event_name }}
WORKFLOW_RUN_SHA: ${{ github.event.workflow_run.head_sha || '' }}
INPUT_SHA: ${{ inputs.target_sha || '' }}
run: |
if [ "$EVENT_NAME" = "workflow_run" ]; then
TARGET_SHA="$WORKFLOW_RUN_SHA"
elif [ -n "$INPUT_SHA" ]; then
TARGET_SHA="$INPUT_SHA"
else
TARGET_SHA="${GITHUB_SHA}"
fi
if [ -z "$TARGET_SHA" ]; then
echo "::error::Unable to resolve target SHA."
exit 1
fi
echo "sha=$TARGET_SHA" >> "$GITHUB_OUTPUT"
echo "Develop staging gate target: $TARGET_SHA" >> "$GITHUB_STEP_SUMMARY"
- name: Ensure target is current develop
env:
GH_TOKEN: ${{ github.token }}
TARGET_SHA: ${{ steps.target.outputs.sha }}
run: |
DEVELOP_SHA="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/heads/develop" --jq '.object.sha')"
echo "develop HEAD: ${DEVELOP_SHA}" >> "$GITHUB_STEP_SUMMARY"
if [ "$TARGET_SHA" != "$DEVELOP_SHA" ]; then
echo "Target ${TARGET_SHA} is no longer develop HEAD (${DEVELOP_SHA}); a newer develop run will handle staging beta." >> "$GITHUB_STEP_SUMMARY"
echo "superseded=true" >> "$GITHUB_ENV"
fi
- name: Wait for all checks on target commit
if: env.superseded != 'true'
env:
GH_TOKEN: ${{ github.token }}
TARGET_SHA: ${{ steps.target.outputs.sha }}
TIMEOUT_SECONDS: ${{ vars.DEVELOP_STAGING_GATE_TIMEOUT_SECONDS || '3600' }}
run: |
set -euo pipefail
deadline=$((SECONDS + TIMEOUT_SECONDS))
ignored_names_json='["Develop Staging Beta Release","NPM Release","Release Orchestrator"]'
while [ "$SECONDS" -lt "$deadline" ]; do
DEVELOP_SHA="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/heads/develop" --jq '.object.sha')"
if [ "$TARGET_SHA" != "$DEVELOP_SHA" ]; then
echo "Target ${TARGET_SHA} is no longer develop HEAD (${DEVELOP_SHA}); a newer develop run will handle staging beta." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
gh api \
"repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${TARGET_SHA}&per_page=100" \
> /tmp/workflow-runs.json
set +e
IGNORED_NAMES_JSON="$ignored_names_json" node <<'NODE'
const fs = require("node:fs");
const payload = JSON.parse(fs.readFileSync("/tmp/workflow-runs.json", "utf8"));
const runs = payload.workflow_runs ?? [];
const ignoredNames = new Set(JSON.parse(process.env.IGNORED_NAMES_JSON));
const currentRunId = Number(process.env.GITHUB_RUN_ID);
const successfulConclusions = new Set(["success", "neutral", "skipped"]);
const relevant = runs.filter((run) => {
if (Number(run.id) === currentRunId) return false;
if (ignoredNames.has(run.name)) return false;
return run.head_sha === process.env.TARGET_SHA;
});
const failed = relevant.filter(
(run) => run.status === "completed" && !successfulConclusions.has(run.conclusion),
);
const pending = relevant.filter((run) => run.status !== "completed");
console.log(`Observed ${relevant.length} workflow run(s) for ${process.env.TARGET_SHA}.`);
if (pending.length > 0) {
console.log(`Pending: ${pending.map((run) => `${run.name}/${run.status}`).join(", ")}`);
}
if (failed.length > 0) {
console.error(
`Failed: ${failed.map((run) => `${run.name}/${run.conclusion}`).join(", ")}`,
);
process.exit(2);
}
if (pending.length > 0) process.exit(1);
if (relevant.length === 0) {
console.log("No completed workflow runs are visible yet.");
process.exit(1);
}
NODE
status=$?
set -e
if [ "$status" -eq 0 ]; then
echo "All observed workflows are green for ${TARGET_SHA}." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
if [ "$status" -eq 2 ]; then
DEVELOP_SHA="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/heads/develop" --jq '.object.sha')"
if [ "$TARGET_SHA" != "$DEVELOP_SHA" ]; then
echo "Target ${TARGET_SHA} was superseded by ${DEVELOP_SHA}; ignoring cancelled checks from the old head." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
echo "::error::At least one workflow failed for ${TARGET_SHA}; beta release blocked."
exit 1
fi
sleep 30
done
echo "::error::Timed out waiting for all workflows to finish for ${TARGET_SHA}."
exit 1
- name: Dispatch beta release
if: env.superseded != 'true'
env:
GH_TOKEN: ${{ github.token }}
AUTO_BETA_RELEASE_FROM_DEVELOP: ${{ vars.AUTO_BETA_RELEASE_FROM_DEVELOP || 'true' }}
TARGET_SHA: ${{ steps.target.outputs.sha }}
run: |
if [ "$AUTO_BETA_RELEASE_FROM_DEVELOP" != "true" ]; then
echo "AUTO_BETA_RELEASE_FROM_DEVELOP is not true; beta release dispatch skipped." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
gh workflow run release.yaml --repo "$GITHUB_REPOSITORY" --ref develop -f release_type=beta
echo "Dispatched release.yaml beta workflow for develop after ${TARGET_SHA} passed staging." >> "$GITHUB_STEP_SUMMARY"
+203
View File
@@ -0,0 +1,203 @@
name: Docker CI Smoke
on:
# Reusable entry for the scheduled exhaustive develop lane (#12342).
workflow_call:
push:
branches: [develop]
schedule:
# Nightly heavy lane (#14051): 07:00 UTC.
- cron: "0 7 * * *"
workflow_dispatch:
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: docker-ci-smoke-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
permissions:
contents: read
jobs:
changes:
name: Classify changed paths
if: github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'ci:full')
# Path classifier is a git-diff + node script: no self-hosted resources.
# Keep it on GitHub-hosted so a drained hetzner fleet cannot leave it
# queued indefinitely and pile up runs (#13617/#8501).
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
outputs:
docker: ${{ steps.filter.outputs.docker }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
submodules: false
- name: Setup Node.js for path gate
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24"
- name: Determine affected Docker smoke surface
id: filter
shell: bash
env:
PR_LABELS: ${{ github.event_name == 'pull_request' && join(github.event.pull_request.labels.*.name, ',') || '' }}
run: |
set -euo pipefail
node packages/scripts/ci-path-gate.mjs \
--config docker \
--event "${{ github.event_name }}" \
--base "${{ github.event.pull_request.base.sha }}" \
--head "${{ github.event.pull_request.head.sha }}" \
--labels "$PR_LABELS" \
--output "$GITHUB_OUTPUT" \
--summary "$GITHUB_STEP_SUMMARY"
docker-ci-smoke:
name: Build production Docker image (+ smoke boot)
needs: changes
if: github.event_name != 'pull_request' || needs.changes.outputs.docker == 'true'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
env:
BUN_VERSION: "canary"
DOCKER_IMAGE: elizaos/agent:docker-smoke
SMOKE_TIMEOUT_SEC: "420"
SMOKE_PORT: "32138"
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24"
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Cache Bun install
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.bun/install/cache
key: bun-${{ runner.os }}-${{ hashFiles('bun.lock') }}
restore-keys: |
bun-${{ runner.os }}-
- name: Free disk space for Docker smoke
run: |
df -h
# github-hosted-only disk cleanup; self-hosted runners (#8501) have no
# passwordless sudo and don't need it, so never fail the job on it.
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc /opt/hostedtoolcache/CodeQL || true
docker system prune -af --volumes || true
df -h
- name: Run Docker CI smoke path
timeout-minutes: 45
# Run the smoke script in its own process group so a wedged docker
# child cannot outlive the wrapper timeout and keep the step active.
run: |
set -euo pipefail
stop_smoke() {
if [[ -n "${smoke_pid:-}" ]] && kill -0 "$smoke_pid" 2>/dev/null; then
kill -TERM "-$smoke_pid" 2>/dev/null || kill -TERM "$smoke_pid" 2>/dev/null || true
sleep 10
kill -KILL "-$smoke_pid" 2>/dev/null || kill -KILL "$smoke_pid" 2>/dev/null || true
fi
}
trap 'stop_smoke; exit 130' INT
trap 'stop_smoke; exit 143' TERM
mkdir -p .tmp/qa
smoke_log=".tmp/qa/docker-ci-smoke-step.log"
setsid bash packages/app-core/scripts/docker-ci-smoke.sh --tag gha-${{ github.run_id }} > >(tee "$smoke_log") 2> >(tee -a "$smoke_log" >&2) &
smoke_pid=$!
smoke_started_at=$SECONDS
last_heartbeat=$SECONDS
deadline=$((SECONDS + 40 * 60))
while kill -0 "$smoke_pid" 2>/dev/null; do
if (( SECONDS >= deadline )); then
echo "::error::Docker CI smoke path exceeded 40 minutes"
kill -TERM "-$smoke_pid" 2>/dev/null || kill -TERM "$smoke_pid" 2>/dev/null || true
sleep 15
kill -KILL "-$smoke_pid" 2>/dev/null || kill -KILL "$smoke_pid" 2>/dev/null || true
wait "$smoke_pid" || true
exit 124
fi
if (( SECONDS - last_heartbeat >= 60 )); then
last_heartbeat=$SECONDS
echo "[docker-ci-smoke] still running after $((SECONDS - smoke_started_at))s; log: $smoke_log"
fi
sleep 10
done
set +e
wait "$smoke_pid"
smoke_status=$?
set -e
if (( smoke_status != 0 )); then
echo "::group::Docker CI smoke step tail"
tail -n 240 "$smoke_log" || true
echo "::endgroup::"
echo "::error::Docker CI smoke failed with exit code ${smoke_status}; see the step tail and uploaded .tmp/qa artifacts."
exit "$smoke_status"
fi
# Inline log capture runs on failure (not cancelled).
- name: Dump container log + inspect (inline)
if: failure()
timeout-minutes: 2
run: |
echo "::group::Docker containers"
timeout 15 docker ps -a || echo "(docker ps timed out)"
echo "::endgroup::"
timeout 15 docker ps -a --format '{{.Names}}' > /tmp/eliza-docker-containers.txt || true
for C in $(grep '^eliza-docker-smoke-' /tmp/eliza-docker-containers.txt || true); do
echo "::group::docker logs $C"
timeout 30 docker logs "$C" 2>&1 || echo "(docker logs timed out)"
echo "::endgroup::"
echo "::group::docker inspect $C (state)"
timeout 15 docker inspect "$C" --format '{{json .State}}' 2>&1 | python3 -m json.tool || true
echo "::endgroup::"
done
- name: Upload container logs + inspect
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: docker-ci-smoke-artifacts-${{ github.run_id }}
path: |
.tmp/qa/docker-ci-smoke-*
/tmp/eliza-docker-*.txt
if-no-files-found: warn
retention-days: 7
- name: Summarize Docker smoke result
if: always()
run: |
echo "## Docker CI Smoke" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "- Production Dockerfile: \`packages/app-core/deploy/Dockerfile.ci\`" >> "$GITHUB_STEP_SUMMARY"
echo "- Script: \`packages/app-core/scripts/docker-ci-smoke.sh\`" >> "$GITHUB_STEP_SUMMARY"
echo "- Trigger: \`${{ github.event_name }}\`" >> "$GITHUB_STEP_SUMMARY"
echo "- Ref: \`${{ github.ref }}\`" >> "$GITHUB_STEP_SUMMARY"
echo "- Smoke endpoints: \`/api/health\` then fallback \`/api/status\`" >> "$GITHUB_STEP_SUMMARY"
+239
View File
@@ -0,0 +1,239 @@
name: Docs CI
# Workflow for documentation quality checks
# Combines dead link checking and quality improvements
concurrency:
group: docs-ci-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
pull_request:
types: [opened, ready_for_review]
paths:
- "packages/docs/**/*.mdx"
- "packages/docs/**/*.md"
- "packages/docs/docs.json"
- "packages/docs/mint.json"
# Allow manual triggering
workflow_dispatch:
inputs:
check_type:
description: "Type of check to run"
required: true
type: choice
options:
- all
- links
- quality
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
jobs:
# ===========================================
# Check Dead Links
# ===========================================
check-links:
if: |
github.event.pull_request.draft != true &&
(github.event.inputs.check_type == 'all' || github.event.inputs.check_type == 'links' || github.event.inputs.check_type == '')
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
filter: blob:none
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Git
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
- name: Check and Fix Dead Links with Claude
id: claude-fix-links
# Don't fail docs CI when the Anthropic API is unavailable (low
# credit balance, rate limit, transient outage). The fix pass is
# best-effort; downstream "Check for Changes" steps already handle
# the no-op path.
continue-on-error: true
uses: anthropics/claude-code-action@ba0aafd4308cbba7165f9f2cdb0cfbed5a3c99ce
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
prompt: |
Check all links in the packages/docs/ directory for broken or dead links.
**Instructions:**
1. Scan all MDX and MD files in packages/docs/
2. Check for internal relative links, external links, and anchor links
3. For each broken link found:
- Typos: Correct the spelling/path
- Moved files: Update to new location
- Missing anchors: Remove or update to valid anchor
- External 404s: Comment out with explanation
**Link Checking Rules:**
- Internal links should point to existing files in packages/docs/
- Skip checking localhost and example.com links
- Use fuzzy matching (85% threshold) for path corrections
**Output:** Report findings but only make high-confidence fixes.
claude_args: |
--model claude-sonnet-4-6
--allowedTools "Bash(find packages/docs -name '*.mdx' -o -name '*.md'),Bash(grep -n),Bash(curl -s -o /dev/null -w '%{http_code}'),Read,EditFile(packages/docs/*)"
- name: Check for Changes
id: check-link-changes
run: |
if [ -n "$(git status --porcelain packages/docs/)" ]; then
echo "changes=true" >> $GITHUB_OUTPUT
else
echo "changes=false" >> $GITHUB_OUTPUT
fi
- name: Commit Link Fixes
if: steps.check-link-changes.outputs.changes == 'true'
run: |
git add packages/docs/
git commit -m "fix(docs): automated broken link fixes [skip ci]" || true
# ===========================================
# Check Documentation Quality
# ===========================================
check-quality:
if: |
github.event.pull_request.draft != true &&
(github.event.inputs.check_type == 'all' || github.event.inputs.check_type == 'quality' || github.event.inputs.check_type == '')
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
filter: blob:none
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Git
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
- name: Check and Fix Quality Issues with Claude
id: claude-fix-quality
# Don't fail docs CI when the Anthropic API is unavailable (low
# credit balance, rate limit, transient outage). The fix pass is
# best-effort; downstream "Check for Changes" steps already handle
# the no-op path.
continue-on-error: true
uses: anthropics/claude-code-action@ba0aafd4308cbba7165f9f2cdb0cfbed5a3c99ce
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
prompt: |
Check all MDX files in packages/docs/ for quality issues.
**Quality Checks:**
1. Double Header Issues - MDX files with frontmatter title AND H1 heading
2. Missing frontmatter fields (title or description)
3. Inconsistent heading hierarchy (H3 following H1 without H2)
4. Code blocks without language tags
**For Double Headers:**
- Remove the H1 heading from content (keep only frontmatter title)
- Mintlify auto-generates H1 from frontmatter
**Fix Application Rules:**
- Make minimal, precise changes
- Preserve all content except what needs fixing
- Maintain proper MDX formatting
**Output:** Report findings and apply fixes.
claude_args: |
--model claude-sonnet-4-6
--allowedTools "Bash(find packages/docs -name '*.mdx'),Bash(grep -n),Read,EditFile(packages/docs/*)"
- name: Check for Changes
id: check-quality-changes
run: |
if [ -n "$(git status --porcelain packages/docs/)" ]; then
echo "changes=true" >> $GITHUB_OUTPUT
else
echo "changes=false" >> $GITHUB_OUTPUT
fi
- name: Commit Quality Fixes
if: steps.check-quality-changes.outputs.changes == 'true'
run: |
git add packages/docs/
git commit -m "fix(docs): automated documentation quality improvements [skip ci]" || true
# ===========================================
# Create PR with all fixes
# ===========================================
create-pr:
needs: [check-links, check-quality]
if: always() && (needs.check-links.result == 'success' || needs.check-quality.result == 'success')
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
filter: blob:none
token: ${{ secrets.GITHUB_TOKEN }}
- name: Check for any uncommitted changes
id: final-check
run: |
git fetch origin
if [ -n "$(git status --porcelain packages/docs/)" ]; then
echo "changes=true" >> $GITHUB_OUTPUT
else
echo "changes=false" >> $GITHUB_OUTPUT
fi
- name: Create Fix Branch and PR
if: steps.final-check.outputs.changes == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
BRANCH_NAME="fix/docs-ci-$(date +%Y%m%d-%H%M%S)"
git checkout -b "$BRANCH_NAME"
git add packages/docs/
git commit -m "fix(docs): automated documentation fixes" \
-m "- Fixed broken links" \
-m "- Fixed quality issues" || exit 0
git push origin "$BRANCH_NAME"
gh pr create \
--title "fix(docs): automated documentation fixes" \
--body "## Automated Documentation Fixes
This PR contains automated fixes for documentation issues.
### Changes Made:
- 🔗 Fixed broken links
- 📝 Fixed quality issues (double headers, missing frontmatter, etc.)
### Review Instructions:
Please verify all changes are correct before merging.
---
*Generated by docs-ci workflow*" \
--base main \
--head "$BRANCH_NAME" \
--label "documentation" \
--label "automated"
@@ -0,0 +1,81 @@
# Electrobun Submodule Guard
#
# Fails fast when the `upstreams/electrobun` submodule gitlink points at a commit
# that is NOT fetchable from its configured remote — the recurring "dangling
# pointer" that breaks `git submodule update` for every fresh clone and CI run.
#
# History of this bug (each a commit that pinned an unfetchable SHA):
# db785c6dc6 -> fixed by 2881d83894 -> regressed to 6bedce8261 (evil merge
# f5ae5dfed5) -> root cause finally fixed by d32749acdc, which repointed the
# submodule URL at the elizaOS/electrobun fork (where the real, ahead-of-public
# electrobun development actually lives) instead of upstream blackboardsh/electrobun.
#
# This guard keeps it from regressing: any PR/push that moves the gitlink or the
# URL must leave a pin that the remote can actually serve.
name: Electrobun Submodule Guard
on:
pull_request:
paths:
- "upstreams/electrobun"
- ".gitmodules"
push:
branches: [develop, main]
paths:
- "upstreams/electrobun"
- ".gitmodules"
# cancel superseded in-flight runs for the same PR to free hosted-runner
# capacity. only cancels pull_request runs; push runs on protected branches
# (main/develop) are never cancelled mid-flight.
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
permissions:
contents: read
jobs:
verify-electrobun-pin:
name: electrobun gitlink is fetchable
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 5
steps:
- name: Checkout (no submodules)
# actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
submodules: false
- name: Verify the electrobun pin is fetchable from its remote
run: |
set -euo pipefail
url=$(git config -f .gitmodules submodule.upstreams/electrobun.url || true)
sha=$(git ls-tree HEAD upstreams/electrobun | awk '{print $3}' || true)
echo "configured url: ${url:-<none>}"
echo "pinned commit: ${sha:-<none>}"
if [ -z "${url}" ] || [ -z "${sha}" ]; then
echo "::error::Could not read the electrobun submodule URL or pinned commit from .gitmodules / the tree."
exit 1
fi
# Mirror exactly what `git submodule update` does: fetch that exact commit.
# GitHub serves any commit reachable from a ref (allowReachableSHA1InWant);
# a dangling/unpushed commit fails here just as it would for a fresh clone.
tmp=$(mktemp -d)
git init -q "${tmp}"
if git -C "${tmp}" fetch -q --depth=1 "${url}" "${sha}"; then
echo "OK: ${sha} is fetchable from ${url}"
else
echo "::error::electrobun submodule pin ${sha} is NOT fetchable from ${url}."
echo "::error::A fresh 'git submodule update' will fail ('not our ref') — this is the recurring dangling-gitlink bug."
echo "::error::Fix: push that electrobun commit to the configured remote, or repoint the pin/URL to a commit the remote serves (e.g. the elizaOS/electrobun develop or main HEAD)."
exit 1
fi
+142
View File
@@ -0,0 +1,142 @@
name: ElizaOS Cuttlefish
# Brand-aware AOSP / Cuttlefish build pipeline.
#
# Defaults to the elizaOS brand (packages/scripts/distro-android/brand.eliza.json).
# Downstream brands (e.g. ElizaOS) supply their own brand config + vendor
# tree via `brand-config` and `vendor-source` inputs and call this workflow
# either via workflow_dispatch (manual run from this repo) or via
# workflow_call (cross-repo reusable workflow):
#
# # downstream/.github/workflows/my-brand-cuttlefish.yml
# jobs:
# build:
# uses: elizaOS/eliza/.github/workflows/elizaos-cuttlefish.yml@develop
# with:
# brand-config: packages/os/android/brand.mybrand.json
# vendor-source: packages/os/android/vendor/mybrand
# aosp-root: /aosp
# launch: true
on:
workflow_dispatch:
inputs:
brand-config:
description: "Path to brand config JSON (relative to repo root). Default: eliza."
required: false
type: string
default: "packages/scripts/distro-android/brand.eliza.json"
vendor-source:
description: "Override path to the vendor tree (relative to repo root). Defaults to brand.vendorDir."
required: false
type: string
default: ""
aosp-root:
description: "Existing AOSP checkout root on the Linux/KVM runner"
required: true
type: string
jobs:
description: "AOSP build parallelism"
required: false
default: "8"
type: string
launch:
description: "Launch Cuttlefish and run adb boot validation"
required: false
default: true
type: boolean
workflow_call:
inputs:
brand-config:
description: "Path to brand config JSON (relative to repo root)."
required: false
type: string
default: "packages/scripts/distro-android/brand.eliza.json"
vendor-source:
description: "Override path to the vendor tree."
required: false
type: string
default: ""
aosp-root:
description: "Existing AOSP checkout root."
required: true
type: string
jobs:
description: "AOSP build parallelism."
required: false
type: string
default: "8"
launch:
description: "Launch Cuttlefish and run adb boot validation."
required: false
type: boolean
default: true
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
build-and-validate:
name: Build and validate brand AOSP image
runs-on: [self-hosted, linux, x64, kvm]
timeout-minutes: 720
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: recursive
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- uses: actions/setup-java@ad2b38190b15e4d6bdf0c97fb4fca8412226d287
with:
distribution: temurin
java-version: "21"
# The brand validator shells out to `aapt` to inspect the staged APK,
# and the brand `build:android:system` runs Gradle which needs a full SDK.
- uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699
with:
packages: "platform-tools build-tools;36.0.0 platforms;android-36"
- name: Install workspace dependencies
run: bun install --frozen-lockfile || bun install --no-frozen-lockfile
- name: Resolve brand
id: brand
run: |
BRAND_CONFIG="${{ inputs.brand-config }}"
if [ ! -f "$BRAND_CONFIG" ]; then
echo "::error::Brand config not found: $BRAND_CONFIG"
exit 1
fi
echo "config=$BRAND_CONFIG" >> "$GITHUB_OUTPUT"
BUILD_CMD=$(node -e "const c=JSON.parse(require('fs').readFileSync('$BRAND_CONFIG','utf8')); console.log((c.buildAndroidSystemCmd ?? ['bun','run','build:android:system']).join(' '))")
echo "build-cmd=$BUILD_CMD" >> "$GITHUB_OUTPUT"
DISTRO_NAME=$(node -p "JSON.parse(require('fs').readFileSync('$BRAND_CONFIG','utf8')).distroName")
echo "distro-name=$DISTRO_NAME" >> "$GITHUB_OUTPUT"
- name: Build privileged APK (${{ steps.brand.outputs.distro-name }})
run: ${{ steps.brand.outputs.build-cmd }}
- name: Static brand validation
run: |
node packages/scripts/distro-android/validate.mjs \
--brand-config "${{ steps.brand.outputs.config }}"
- name: Sync, build, and optionally boot
run: |
ARGS=(
--brand-config "${{ steps.brand.outputs.config }}"
--aosp-root "${{ inputs.aosp-root }}"
--jobs "${{ inputs.jobs }}"
)
if [ -n "${{ inputs.vendor-source }}" ]; then
ARGS+=(--source-vendor "${{ inputs.vendor-source }}")
fi
if [ "${{ inputs.launch }}" = "true" ]; then
ARGS+=(--launch --boot-validate)
fi
node packages/scripts/distro-android/build-aosp.mjs "${ARGS[@]}"
@@ -0,0 +1,290 @@
name: elizaOS Full OS Release
on:
release:
types: [created]
workflow_dispatch:
inputs:
publish:
description: Upload artifacts to release
type: boolean
default: false
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
# Compute the bare semver version string (no leading 'v') from the ref name.
# On a 'release' event github.ref_name is the tag (e.g. v2.0.1); we strip
# the prefix here so downstream jobs get a clean version like '2.0.1'.
prepare:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
version: ${{ steps.parse.outputs.version }}
tag: ${{ steps.parse.outputs.tag }}
steps:
- name: Parse version from ref
id: parse
run: |
TAG="${{ github.ref_name }}"
VERSION="${TAG#v}"
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
echo "Resolved: tag=$TAG version=$VERSION"
trigger-linux-iso:
uses: ./.github/workflows/build-linux-iso.yml
with:
channel: stable
publish: ${{ github.event_name == 'release' || inputs.publish == true }}
secrets: inherit
trigger-debian-package:
uses: ./.github/workflows/build-debian-package.yml
secrets: inherit
trigger-vm-image:
uses: ./.github/workflows/build-vm-image.yml
with:
format: ova
publish: ${{ github.event_name == 'release' || inputs.publish == true }}
secrets: inherit
verify-artifacts:
name: Verify build artifacts exist and are non-empty
needs: [trigger-linux-iso, trigger-debian-package, trigger-vm-image]
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- name: Download Linux ISO artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
pattern: elizaos-live-iso-*
path: ./_verify/iso
merge-multiple: true
- name: Download Debian package artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: elizaos-debian-package
path: ./_verify/deb
- name: Download VM image artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: elizaos-vm-image
path: ./_verify/vm
- name: Assert artifacts present and non-empty
run: |
set -euo pipefail
fail=0
assert_nonempty() {
local pattern="$1"
local label="$2"
local match
match=$(find "$pattern" -type f \( -name "$3" \) -size +0c 2>/dev/null | head -1)
if [ -z "$match" ]; then
echo "ERROR: missing or empty $label artifact (pattern $3 under $pattern)"
fail=1
else
echo "OK: $label -> $match ($(stat --format=%s "$match") bytes)"
fi
}
assert_nonempty ./_verify/iso "Linux ISO" "*.iso"
assert_nonempty ./_verify/deb "Debian .deb" "*.deb"
assert_nonempty ./_verify/vm "VM image" "*.ova"
exit "$fail"
populate-and-validate-manifest:
name: Populate manifest checksums and gate on publishable validation
needs: [verify-artifacts]
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
permissions:
# contents: write — required by softprops/action-gh-release to
# attach SHA256SUMS to the release asset list.
# id-token: write + attestations: write — required by
# actions/attest-build-provenance@v4 to mint a Sigstore-signed
# SLSA provenance attestation over SHA256SUMS via GitHub OIDC.
contents: write
id-token: write
attestations: write
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Download all release artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
path: ./_artifacts
merge-multiple: true
- name: Check manifest file exists
id: manifest_check
# The manifest file is hand-maintained per beta cycle. If it is
# missing (or got renamed without this workflow being updated)
# every downstream step would fail with ENOENT inside a Node
# readFileSync — a confusing 1-2 min failure with no clear call
# to action. Surface it explicitly so the maintainer either adds
# the manifest, updates this hardcoded path, or accepts the skip.
env:
MANIFEST_PATH: packages/os/release/beta-2026-05-16/manifest.json
run: |
if [ -f "$MANIFEST_PATH" ]; then
echo "can_validate_manifest=true" >> "$GITHUB_OUTPUT"
else
echo "::warning::Release manifest $MANIFEST_PATH not found. Manifest population + canonical SHA256SUMS + SLSA-over-SHA256SUMS attestation skipped. Per-artifact SLSA attestations from build-linux-iso.yml / build-debian-package.yml / build-vm-image.yml still mint correctly."
echo "can_validate_manifest=false" >> "$GITHUB_OUTPUT"
fi
- name: Populate manifest with real sha256 + sizes
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
# Discovery is driven by the *manifest filename* — each entry's
# `filename` is searched verbatim under ./_artifacts. This keeps
# the workflow and manifest in lock-step: rename in the manifest,
# rename in the build, no workflow edits required.
#
# Build workflows that currently feed this step:
# - build-linux-iso.yml → *.iso (raw-linux-* entry)
# - build-vm-image.yml → *.qcow2, *.ova (vm-linux-* entry)
# Android (cf_x86_64, pixel-arm64) and macOS UTM bundles are NOT
# produced by any CI workflow today. They must be attached to the
# _artifacts staging dir out-of-band (release engineer uploads the
# built bundle as a workflow artifact via a prior manual job, or
# via release asset download) before this job runs. The strict
# gate at the end will fail loudly if any artifact is missing.
run: |
set -euo pipefail
MANIFEST=packages/os/release/beta-2026-05-16/manifest.json
ids=$(node -e '
const m = JSON.parse(require("node:fs").readFileSync(process.argv[1], "utf8"));
for (const a of m.artifacts) console.log(`${a.id}\t${a.filename}`);
' "$MANIFEST")
while IFS=$'\t' read -r id filename; do
path=$(find ./_artifacts -type f -name "$filename" | head -1 || true)
if [ -z "$path" ] || [ ! -f "$path" ]; then
echo "::warning::no artifact named $filename found for $id — strict gate will fail"
continue
fi
sha=$(sha256sum "$path" | cut -d' ' -f1)
size=$(stat --format=%s "$path")
node packages/os/scripts/update-release-manifest.mjs \
--manifest "$MANIFEST" \
--artifact "$id" \
--sha256 "$sha" \
--size "$size"
done <<< "$ids"
- name: Gate on publishable checksums (before status promotion)
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
# Status promotion is gated on strict validation. If any artifact
# is missing a real checksum, the release stays "candidate" and
# this job fails — no half-published manifest.
run: |
node packages/os/scripts/validate-release-manifest.mjs \
--manifest packages/os/release/beta-2026-05-16/manifest.json \
--require-publishable-checksums
- name: Generate canonical SHA256SUMS via the in-repo tool
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
# Runs BEFORE manifest promotion so that any failure here prevents
# the "available" status from being set and uploaded. SHA256SUMS
# generation is a hard prerequisite for promotion — closing the
# window where a promoted manifest could exist without a
# corresponding SHA256SUMS file or SLSA attestation.
#
# Calls packages/os/scripts/generate-release-checksums.mjs (which
# already exists, has unit tests under __tests__, and is the
# repo's canonical checksum tool) against the populated manifest
# + downloaded artifacts. Produces sha256sum -c compatible output
# mirroring the Debian / Fedora / Tails / Arch convention.
run: |
set -euo pipefail
node packages/os/scripts/generate-release-checksums.mjs \
--manifest packages/os/release/beta-2026-05-16/manifest.json \
--artifact-root ./_artifacts \
--output ./_artifacts/SHA256SUMS
echo "Generated SHA256SUMS:"
cat ./_artifacts/SHA256SUMS
- name: Attest SLSA build provenance over SHA256SUMS
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
# Mints a single Sigstore-signed in-toto/SLSA provenance
# attestation that covers every release artifact via its hash,
# by attesting the canonical SHA256SUMS file generated above.
# Requires id-token: write + attestations: write (set in this
# job's permissions block). Users verify with:
#
# gh attestation verify SHA256SUMS --owner elizaOS
# sha256sum -c SHA256SUMS --ignore-missing
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373
with:
subject-path: ./_artifacts/SHA256SUMS
- name: Upload SHA256SUMS artifact
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: elizaos-release-checksums
path: ./_artifacts/SHA256SUMS
retention-days: 90
- name: Promote release status to available
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
run: |
set -euo pipefail
node -e '
const fs = require("node:fs");
const p = "packages/os/release/beta-2026-05-16/manifest.json";
const m = JSON.parse(fs.readFileSync(p, "utf8"));
m.release.status = "available";
fs.writeFileSync(p, JSON.stringify(m, null, 2) + "\n");
'
- name: Upload populated manifest
if: steps.manifest_check.outputs.can_validate_manifest == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: elizaos-os-release-manifest-published
path: packages/os/release/beta-2026-05-16/manifest.json
retention-days: 90
- name: Attach SHA256SUMS to release
if: github.event_name == 'release' && steps.manifest_check.outputs.can_validate_manifest == 'true'
uses: softprops/action-gh-release@718ea10b132b3b2eba29c1007bb80653f286566b
with:
files: ./_artifacts/SHA256SUMS
tag_name: ${{ github.event.release.tag_name || github.ref_name }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
publish-apt:
needs: [prepare, trigger-debian-package, verify-artifacts]
if: needs.trigger-debian-package.result == 'success' && needs.verify-artifacts.result == 'success'
uses: ./.github/workflows/publish-apt-repo.yml
with:
version: ${{ needs.prepare.outputs.version }}
tag: ${{ needs.prepare.outputs.tag }}
channel: stable
secrets: inherit
summary:
needs: [trigger-linux-iso, trigger-debian-package, trigger-vm-image, verify-artifacts, populate-and-validate-manifest, publish-apt]
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
if: always()
timeout-minutes: 5
steps:
- name: Release summary
run: |
{
echo "## elizaOS OS Release Summary"
echo "| Artifact | Status |"
echo "| -------- | ------ |"
echo "| Linux ISO | ${{ needs.trigger-linux-iso.result }} |"
echo "| Debian .deb | ${{ needs.trigger-debian-package.result }} |"
echo "| VM Image | ${{ needs.trigger-vm-image.result }} |"
echo "| Verify artifacts | ${{ needs.verify-artifacts.result }} |"
echo "| Populate + gate manifest | ${{ needs.populate-and-validate-manifest.result }} |"
echo "| apt Repo | ${{ needs.publish-apt.result }} |"
} >> "$GITHUB_STEP_SUMMARY"
+131
View File
@@ -0,0 +1,131 @@
name: ElizaOS OS release validation
on:
workflow_dispatch:
pull_request:
branches: [main]
paths:
- ".github/workflows/elizaos-os-release.yml"
- "packages/os/**"
- "packages/os/homepage/**"
- "packages/os/usb-installer/**"
- "packages/os/setup/**"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
validate-os-release:
name: Validate OS release surfaces
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Install dependencies
run: bun install
- name: Validate OS release manifest
run: |
manifest=packages/os/release/beta-2026-05-16/manifest.json
evidence=packages/os/release/beta-2026-05-16/evidence/ci-evidence.json
node --check packages/os/scripts/*.mjs
node --test packages/os/scripts/__tests__/os-release-scripts.test.mjs
if [ -f "$manifest" ]; then
node packages/os/scripts/validate-release-manifest.mjs \
--manifest "$manifest"
node packages/os/scripts/collect-release-evidence.mjs \
--manifest "$manifest" \
--output "$evidence"
else
echo "::notice::${manifest} is not present; skipping beta release manifest validation."
fi
- name: Validate USB installer
run: |
bun run --cwd packages/os/usb-installer lint
bun run --cwd packages/os/usb-installer typecheck
bun run --cwd packages/os/usb-installer test
bun run --cwd packages/os/usb-installer build
bunx playwright install --with-deps chromium
bun run --cwd packages/os/usb-installer test:e2e
if modinfo scsi_debug >/dev/null 2>&1; then
bun run --cwd packages/os/usb-installer test:linux-virtual-usb
else
echo "::notice::scsi_debug kernel module is unavailable on this runner; skipping virtual block-device USB proof."
fi
- name: Validate Android installer
run: |
android_manifest=packages/os/release/beta-2026-05-16/android-release-manifest.json
packages/os/android/installer/tests/run-tests.sh
node packages/os/android/installer/scripts/validate-release-manifest.mjs \
packages/os/android/installer/manifests/android-release-manifest.example.json
if [ -f "$android_manifest" ]; then
node packages/os/android/installer/scripts/validate-release-manifest.mjs \
"$android_manifest" \
--allow-placeholders
else
echo "::notice::${android_manifest} is not present; skipping beta Android release manifest validation."
fi
- name: Validate elizaos-setup (formerly aosp-flasher)
run: |
bun run --cwd packages/os/setup typecheck
bun run --cwd packages/os/setup build
- name: Validate Linux live USB metadata
run: |
cd packages/os/linux
ELIZAOS_STATIC_SOURCE_ONLY=1 ./scripts/static-smoke.sh
- name: Validate OS homepage
run: |
bun run --cwd packages/os/homepage lint:check
bun run --cwd packages/os/homepage typecheck
bun run --cwd packages/os/homepage build
(cd packages/os/homepage && ../../node_modules/.bin/playwright install --with-deps chromium)
bun run --cwd packages/os/homepage test:e2e
- name: Capture homepage contact sheet
run: |
set -euo pipefail
bun --bun --cwd packages/os/homepage vite preview --host 127.0.0.1 --port 4455 &
echo $! > /tmp/os-homepage-preview.pid
ready=0
for attempt in {1..60}; do
if curl -fsS http://127.0.0.1:4455 >/dev/null; then
ready=1
break
fi
echo "Waiting for os-homepage preview (${attempt}/60)..."
sleep 1
done
if [ "$ready" != "1" ]; then
echo "ERROR: os-homepage preview never came up on 127.0.0.1:4455"
kill "$(cat /tmp/os-homepage-preview.pid)" 2>/dev/null || true
exit 1
fi
bun run --cwd packages/os/homepage screenshots
kill "$(cat /tmp/os-homepage-preview.pid)" 2>/dev/null || true
- name: Upload release evidence
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
if: always()
with:
name: elizaos-os-release-validation
path: |
packages/os/release/beta-2026-05-16/evidence/**
packages/os/linux/vm/output/ci-bundle-metadata/**
packages/os/homepage/artifacts/screenshots/**
packages/os/homepage/test-results/**
if-no-files-found: warn
retention-days: 14
@@ -0,0 +1,137 @@
name: External API Live Drift
# Runs the external-API LIVE-DRIFT tests against the real provider APIs so the
# keyless mocks/parsers are continuously verified to still match reality. These
# *.real.test.ts files self-skip in the PR lane (their LIVE gate is off) and ran
# in NO workflow until now — they were dark. This lane turns the LIVE gate on and
# injects the optional provider secrets.
#
# Provider coverage vs provider-coverage.ts (#8801):
# PUBLIC (no secret, always run): Polymarket, Hyperliquid, CoinGecko.
# KEY/TOKEN-GATED (self-skip via describe.skipIf unless their
# <PROVIDER>_API_KEY / _ACCESS_TOKEN secret is present): Calendly, Tavily,
# OpenAI, Anthropic, Strava, Oura, Withings, Fitbit.
# CONFIG-BLOCKED (test exists + secret wired, but the plugin's vitest config
# excludes *.real.test.ts so it can't run from here yet): Google Calendar —
# see the structural-lane marker at the bottom of the steps.
# A missing secret is a clean no-op (describe.skipIf), not a failure — so the
# credential-blocked lanes only light up once their token secret is added to the
# repo. continue-on-error keeps a provider outage / drift from red-X'ing the
# branch — a failure here is a real signal that a mock has drifted from the live
# API. The MTProto/native surfaces (telegram-mtproto, signal,
# imessage-bluebubbles) have no HTTP drift surface and stay hand-curated
# mock-only with that caveat documented in
# packages/test/mocks/helpers/provider-coverage.ts.
#
# Nightly + on-demand; never on pull_request.
on:
schedule:
# 09:30 UTC daily — after the other nightly live lanes settle.
- cron: "30 9 * * *"
workflow_dispatch: {}
concurrency:
group: external-api-live-drift-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CI: "true"
NODE_VERSION: "24"
BUN_VERSION: "canary"
NODE_OPTIONS: "--experimental-sqlite"
# Turn the LIVE gate on for every external-API drift test.
POLYMARKET_LIVE_TEST: "1"
HYPERLIQUID_LIVE_TEST: "1"
COINGECKO_LIVE_TEST: "1"
CALENDLY_LIVE_TEST: "1"
TAVILY_LIVE_TEST: "1"
OPENAI_LIVE_TEST: "1"
ANTHROPIC_LIVE_TEST: "1"
STRAVA_LIVE_TEST: "1"
OURA_LIVE_TEST: "1"
WITHINGS_LIVE_TEST: "1"
FITBIT_LIVE_TEST: "1"
GOOGLE_CALENDAR_LIVE_TEST: "1"
# Key/token-gated providers (tests self-skip when their secret is absent).
CALENDLY_ACCESS_TOKEN: ${{ secrets.CALENDLY_ACCESS_TOKEN }}
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
STRAVA_ACCESS_TOKEN: ${{ secrets.STRAVA_ACCESS_TOKEN }}
OURA_ACCESS_TOKEN: ${{ secrets.OURA_ACCESS_TOKEN }}
WITHINGS_ACCESS_TOKEN: ${{ secrets.WITHINGS_ACCESS_TOKEN }}
FITBIT_ACCESS_TOKEN: ${{ secrets.FITBIT_ACCESS_TOKEN }}
GOOGLE_CALENDAR_ACCESS_TOKEN: ${{ secrets.GOOGLE_CALENDAR_ACCESS_TOKEN }}
jobs:
live-drift:
name: External-API mocks vs live APIs (nightly)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
continue-on-error: true
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Polymarket live drift (public)
run: bunx vitest run --root plugins/plugin-polymarket src/routes.real.test.ts
- name: Hyperliquid live drift (public)
run: bunx vitest run --root plugins/plugin-hyperliquid src/routes.real.test.ts
- name: CoinGecko live drift (public)
run: bunx vitest run --root plugins/plugin-wallet src/routes/wallet-market-overview.real.test.ts
- name: Calendly live drift (token-gated, self-skips)
run: bunx vitest run --root plugins/plugin-calendly src/calendly-client.real.test.ts
- name: Tavily web-search live drift (key-gated, self-skips)
run: bunx vitest run --root plugins/plugin-web-search src/services/webSearchService.real.test.ts
- name: OpenAI live drift (key-gated, self-skips)
run: bunx vitest run --root plugins/plugin-openai __tests__/openai-drift.real.test.ts
- name: Anthropic live drift (key-gated, self-skips)
run: bunx vitest run --root plugins/plugin-anthropic __tests__/anthropic-drift.real.test.ts
- name: Strava live drift (token-gated, self-skips)
run: bunx vitest run --root plugins/plugin-health test/strava-connector.real.test.ts
- name: Oura live drift (token-gated, self-skips)
run: bunx vitest run --root plugins/plugin-health test/oura-connector.real.test.ts
- name: Withings live drift (token-gated, self-skips)
run: bunx vitest run --root plugins/plugin-health test/withings-connector.real.test.ts
- name: Fitbit live drift (token-gated, self-skips)
run: bunx vitest run --root plugins/plugin-health test/fitbit-connector.real.test.ts
# Structural-lane marker — NOT YET RUNNABLE here.
# plugins/plugin-calendar/test/google-calendar-connector.real.test.ts
# exists and self-gates on GOOGLE_CALENDAR_LIVE_TEST + GOOGLE_CALENDAR_ACCESS_TOKEN
# (secret wired in the env block above), but plugin-calendar's vitest.config.ts
# excludes `**/*.real.test.{ts,tsx}` with passWithNoTests:true, so a vitest
# invocation here finds zero tests and would silently "pass" with no drift
# coverage. That config exclude can only be lifted in the plugin's own config.
# Wire the step below once plugin-calendar's config stops excluding real tests:
# run: bunx vitest run --root plugins/plugin-calendar test/google-calendar-connector.real.test.ts
@@ -0,0 +1,57 @@
name: Apply database migrations
on:
push:
branches:
- staging
- production
concurrency:
group: apply-migrations-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
jobs:
migrate:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
env:
STAGING_DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
STAGING_DIRECT_DATABASE_URL: ${{ secrets.STAGING_DIRECT_DATABASE_URL }}
PROD_DATABASE_URL: ${{ secrets.PROD_DATABASE_URL }}
PROD_DIRECT_DATABASE_URL: ${{ secrets.PROD_DIRECT_DATABASE_URL }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Apply migrations (staging)
if: github.ref == 'refs/heads/staging' && env.STAGING_DATABASE_URL != '' && env.STAGING_DIRECT_DATABASE_URL != ''
env:
DATABASE_URL: ${{ env.STAGING_DATABASE_URL }}
DIRECT_DATABASE_URL: ${{ env.STAGING_DIRECT_DATABASE_URL }}
working-directory: packages/feed/packages/db
run: bunx --bun drizzle-kit push --force --config=drizzle.config.ts
- name: Apply migrations (production)
if: github.ref == 'refs/heads/production' && env.PROD_DATABASE_URL != '' && env.PROD_DIRECT_DATABASE_URL != ''
env:
DATABASE_URL: ${{ env.PROD_DATABASE_URL }}
DIRECT_DATABASE_URL: ${{ env.PROD_DIRECT_DATABASE_URL }}
working-directory: packages/feed/packages/db
run: bunx --bun drizzle-kit push --force --config=drizzle.config.ts
- name: Skip notice (staging)
if: github.ref == 'refs/heads/staging' && (env.STAGING_DATABASE_URL == '' || env.STAGING_DIRECT_DATABASE_URL == '')
run: echo "Skipping staging migrations because STAGING_DATABASE_URL or STAGING_DIRECT_DATABASE_URL is not set."
- name: Skip notice (production)
if: github.ref == 'refs/heads/production' && (env.PROD_DATABASE_URL == '' || env.PROD_DIRECT_DATABASE_URL == '')
run: echo "Skipping production migrations because PROD_DATABASE_URL or PROD_DIRECT_DATABASE_URL is not set."
+46
View File
@@ -0,0 +1,46 @@
name: Env Audit
on:
pull_request:
push:
branches: [develop]
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
env:
CI: true
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
env-audit:
name: env:audit:check
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 2
- name: Setup Bun workspace
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: "1.3.14" # pinned: floating canary writes lockfileVersion 2 and breaks --frozen-lockfile (#11184/#9454)
install-command: bun install
- name: Env audit check
run: bun run --cwd packages/feed env:audit:check
continue-on-error: true
+423
View File
@@ -0,0 +1,423 @@
name: Feed RL Training
# Trigger options:
# 1. On schedule (daily training at 2 AM UTC)
# 2. Via repository_dispatch (from debug endpoint)
# 3. Manual workflow_dispatch (testing from GitHub UI)
on:
schedule:
# Daily at 2 AM UTC
- cron: '0 2 * * *'
repository_dispatch:
types: [trigger-training]
workflow_dispatch:
inputs:
batch_id:
description: 'Training batch ID (optional, auto-generated if not provided)'
required: false
type: string
window_id:
description: 'Window ID to train (optional, auto-detected if not provided)'
required: false
type: string
force:
description: 'Force training even if not ready (for testing)'
required: false
type: boolean
default: false
base_model:
description: 'Base model to use (default: google/gemma-4-E4B-it)'
required: false
type: string
env:
PYTHON_VERSION: '3.11'
FEED_DEFAULT_BASE_MODEL: 'google/gemma-4-E4B-it'
# Prevent concurrent training runs
concurrency:
group: training-pipeline
cancel-in-progress: false # Wait for current run to finish
permissions:
contents: read
jobs:
train:
if: ${{ vars.ENABLE_RL_TRAINING == 'true' }}
name: Train RL Model
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 360 # 6 hours max
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'
cache-dependency-path: 'packages/training/scripts/rl/requirements.txt'
- name: Install dependencies
working-directory: packages/training/scripts/rl
run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
- name: Verify installation
run: |
python --version
pip list | grep -E "(art|asyncpg)"
- name: Check training readiness
id: readiness
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
run: |
python -c "
import asyncio
import asyncpg
import os
import sys
async def check():
try:
pool = await asyncpg.create_pool(
os.getenv('DATABASE_URL'),
min_size=1,
max_size=2,
timeout=30
)
# Count scored trajectories ready for training
count = await pool.fetchval('''
SELECT COUNT(*) FROM trajectories
WHERE \"isTrainingData\" = true
AND \"usedInTraining\" = false
AND \"aiJudgeReward\" IS NOT NULL
AND \"stepsJson\" IS NOT NULL
AND \"stepsJson\"::text != 'null'
AND \"stepsJson\"::text != '[]'
''')
print(f'✅ Database connected')
print(f'📊 Trajectories ready for training: {count}')
# Need 100 bundles minimum
min_required = 100
ready = count >= min_required
if ready:
print(f'✅ READY: {count} >= {min_required}')
else:
print(f'⏳ NOT READY: {count} < {min_required} (need {min_required - count} more)')
with open(os.getenv('GITHUB_OUTPUT'), 'a') as f:
f.write(f'ready={str(ready).lower()}\\n')
f.write(f'count={count}\\n')
await pool.close()
except Exception as e:
print(f'❌ Error checking readiness: {e}', file=sys.stderr)
with open(os.getenv('GITHUB_OUTPUT'), 'a') as f:
f.write('ready=false\\n')
f.write('count=0\\n')
sys.exit(1)
asyncio.run(check())
"
- name: Get batch info
id: batch
env:
# From workflow_dispatch inputs:
BATCH_ID_INPUT: ${{ inputs.batch_id || '' }}
WINDOW_ID_INPUT: ${{ inputs.window_id || '' }}
FORCE_INPUT: ${{ inputs.force || 'false' }}
BASE_MODEL_INPUT: ${{ inputs.base_model || '' }}
# From repository_dispatch payload:
BATCH_ID_PAYLOAD: ${{ github.event.client_payload.batch_id || '' }}
WINDOW_ID_PAYLOAD: ${{ github.event.client_payload.window_id || '' }}
FORCE_PAYLOAD: ${{ github.event.client_payload.force || 'false' }}
DATABASE_URL: ${{ secrets.DATABASE_URL }}
run: |
# Determine batch_id
if [ -n "$BATCH_ID_PAYLOAD" ]; then
echo "batch_id=$BATCH_ID_PAYLOAD" >> $GITHUB_OUTPUT
elif [ -n "$BATCH_ID_INPUT" ]; then
echo "batch_id=$BATCH_ID_INPUT" >> $GITHUB_OUTPUT
else
echo "batch_id=batch-$(date +%s)" >> $GITHUB_OUTPUT
fi
# Determine window_id
if [ -n "$WINDOW_ID_PAYLOAD" ]; then
echo "window_id=$WINDOW_ID_PAYLOAD" >> $GITHUB_OUTPUT
elif [ -n "$WINDOW_ID_INPUT" ]; then
echo "window_id=$WINDOW_ID_INPUT" >> $GITHUB_OUTPUT
else
WINDOW=$(date -u +"%Y-%m-%dT%H:00")
echo "window_id=$WINDOW" >> $GITHUB_OUTPUT
fi
# Determine force flag
if [ "$FORCE_PAYLOAD" = "true" ] || [ "$FORCE_INPUT" = "true" ]; then
echo "force=true" >> $GITHUB_OUTPUT
else
echo "force=false" >> $GITHUB_OUTPUT
fi
# Determine base model
if [ -n "$BASE_MODEL_INPUT" ]; then
echo "base_model=$BASE_MODEL_INPUT" >> $GITHUB_OUTPUT
else
echo "base_model=$FEED_DEFAULT_BASE_MODEL" >> $GITHUB_OUTPUT
fi
# Generate model version
MODEL_VERSION=$(python -c "
import asyncio
import asyncpg
import os
import sys
async def get_version():
try:
pool = await asyncpg.create_pool(os.getenv('DATABASE_URL'), timeout=10)
latest = await pool.fetchval('''
SELECT version FROM trained_models
WHERE status IN ('ready', 'deployed')
ORDER BY \"createdAt\" DESC
LIMIT 1
''')
if latest:
parts = latest.strip('v').split('.')
patch = int(parts[2]) + 1
version = f'v{parts[0]}.{parts[1]}.{patch}'
else:
version = 'v1.0.0'
print(version)
await pool.close()
except Exception as e:
import time
version = f'v1.0.{int(time.time()) % 10000}'
print(version)
asyncio.run(get_version())
" 2>/dev/null)
echo "model_version=$MODEL_VERSION" >> $GITHUB_OUTPUT
echo "source=github_cron" >> $GITHUB_OUTPUT
- name: Skip if not ready (unless forced)
if: steps.readiness.outputs.ready != 'true' && steps.batch.outputs.force != 'true'
run: |
echo "⏭️ Not ready for training and force=false"
echo "Trajectories: ${{ steps.readiness.outputs.count }}"
echo "Required: 100 (minimum bundles)"
exit 0
- name: Update batch status to training
if: steps.readiness.outputs.ready == 'true' || steps.batch.outputs.force == 'true'
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
BATCH_ID: ${{ steps.batch.outputs.batch_id }}
run: |
python -c "
import asyncio
import asyncpg
import os
async def update():
pool = await asyncpg.create_pool(os.getenv('DATABASE_URL'))
batch_id = os.getenv('BATCH_ID')
async with pool.acquire() as conn:
async with conn.transaction():
await conn.execute('''
INSERT INTO training_batches (
\"batchId\", id, status, \"startedAt\", \"createdAt\"
) VALUES (
\$1, \$1, 'training', NOW(), NOW()
)
ON CONFLICT (\"batchId\")
DO UPDATE SET status = 'training', \"startedAt\" = NOW()
''', batch_id)
print(f'✅ Batch {batch_id} status: training')
await pool.close()
asyncio.run(update())
"
- name: Select base model
id: model_selection
if: steps.readiness.outputs.ready == 'true' || steps.batch.outputs.force == 'true'
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
BASE_MODEL_OVERRIDE: ${{ steps.batch.outputs.base_model }}
run: |
python -c "
import asyncio
import asyncpg
import os
import sys
async def select_model():
try:
pool = await asyncpg.create_pool(os.getenv('DATABASE_URL'))
base_model_override = os.getenv('BASE_MODEL_OVERRIDE', '')
default_base_model = os.getenv('FEED_DEFAULT_BASE_MODEL', 'google/gemma-4-E4B-it')
# Count training bundles
bundle_count = await pool.fetchval('''
SELECT COUNT(*) FROM trajectories
WHERE \"isTrainingData\" = true
AND \"usedInTraining\" = false
AND \"aiJudgeReward\" IS NOT NULL
AND \"stepsJson\" IS NOT NULL
AND \"stepsJson\"::text != 'null'
AND \"stepsJson\"::text != '[]'
''')
print(f'📊 Bundle count: {bundle_count}')
# Use override if provided, otherwise use default
if base_model_override:
base_model = base_model_override
strategy = 'override'
else:
base_model = default_base_model
strategy = 'default'
print(f'📦 Selected model: {base_model}')
print(f'📋 Strategy: {strategy}')
with open(os.getenv('GITHUB_OUTPUT'), 'a') as f:
f.write(f'base_model={base_model}\\n')
f.write(f'strategy={strategy}\\n')
f.write(f'bundle_count={bundle_count}\\n')
await pool.close()
except Exception as e:
print(f'❌ Model selection failed: {e}', file=sys.stderr)
default_base_model = os.getenv('FEED_DEFAULT_BASE_MODEL', 'google/gemma-4-E4B-it')
with open(os.getenv('GITHUB_OUTPUT'), 'a') as f:
f.write(f'base_model={default_base_model}\\n')
f.write('strategy=fallback\\n')
sys.exit(1)
asyncio.run(select_model())
"
- name: Run RL Training
id: training
if: steps.readiness.outputs.ready == 'true' || steps.batch.outputs.force == 'true'
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
BATCH_ID: ${{ steps.batch.outputs.batch_id }}
WINDOW_ID: ${{ steps.batch.outputs.window_id }}
MODEL_VERSION: ${{ steps.batch.outputs.model_version }}
BASE_MODEL: ${{ steps.model_selection.outputs.base_model }}
MODE: single
MAX_EXAMPLES: "2000"
MAX_STEPS_PER_TRAJECTORY: "20"
MAX_SEQ_LENGTH: "8192"
working-directory: packages/training/scripts/rl
run: |
echo "🚀 Starting training"
echo "Batch ID: $BATCH_ID"
echo "Window ID: $WINDOW_ID"
echo "Model Version: $MODEL_VERSION"
echo "Strategy: ${{ steps.model_selection.outputs.strategy }}"
echo "Base Model: $BASE_MODEL"
echo "Trajectories available: ${{ steps.readiness.outputs.count }}"
echo "Bundle count: ${{ steps.model_selection.outputs.bundle_count }}"
# Run trainer
python src/training/feed_trainer.py
- name: Update batch status to completed
if: success() && (steps.readiness.outputs.ready == 'true' || steps.batch.outputs.force == 'true')
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
BATCH_ID: ${{ steps.batch.outputs.batch_id }}
run: |
python -c "
import asyncio
import asyncpg
import os
async def update():
pool = await asyncpg.create_pool(os.getenv('DATABASE_URL'))
batch_id = os.getenv('BATCH_ID')
await pool.execute('''
UPDATE training_batches
SET status = 'completed', \"completedAt\" = NOW()
WHERE \"batchId\" = \$1
''', batch_id)
print(f'✅ Batch {batch_id} completed')
await pool.close()
asyncio.run(update())
"
- name: Update batch status to failed
if: failure() && (steps.readiness.outputs.ready == 'true' || steps.batch.outputs.force == 'true')
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
BATCH_ID: ${{ steps.batch.outputs.batch_id }}
run: |
python -c "
import asyncio
import asyncpg
import os
async def update():
pool = await asyncpg.create_pool(os.getenv('DATABASE_URL'))
batch_id = os.getenv('BATCH_ID')
await pool.execute('''
UPDATE training_batches
SET status = 'failed', error = 'GitHub Actions workflow failed'
WHERE \"batchId\" = \$1
''', batch_id)
print(f'❌ Batch {batch_id} failed')
await pool.close()
asyncio.run(update())
"
- name: Upload training logs
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: training-logs-${{ steps.batch.outputs.batch_id }}
path: |
packages/training/scripts/rl/logs/
packages/training/scripts/rl/*.log
retention-days: 7
- name: Report status
if: always()
run: |
echo "Training Status: ${{ job.status }}"
echo "Batch ID: ${{ steps.batch.outputs.batch_id }}"
echo "Window ID: ${{ steps.batch.outputs.window_id }}"
echo "Ready: ${{ steps.readiness.outputs.ready }}"
echo "Trajectories: ${{ steps.readiness.outputs.count }}"
echo "Model: ${{ steps.model_selection.outputs.base_model }}"
+99
View File
@@ -0,0 +1,99 @@
name: feed-test
# Path-gated CI lane for the nested packages/feed monorepo (issue #9943).
#
# Why this exists: packages/feed has ~525 test files but ran in ZERO CI lanes.
# The root workspace excludes it (`!packages/feed`) and the root test runner
# honors that negation, while feed's own `test:ci` script was invoked by nothing.
# This lane runs feed's unit suite on every PR that touches packages/feed, so
# feed regressions are caught instead of silently shipping behind green CI.
#
# Scope decision (deliberate, documented): this runs `test:unit` — the isolated
# unit runner (scripts/test-unit-isolated.ts), which executes the bulk of feed's
# test files and explicitly excludes the suites that need infrastructure. It does
# NOT run the full `test:ci` script (scripts/ci/run-all-tests.sh): that script is
# a local-dev kitchen sink that also runs typecheck + lint + a production build,
# boots Anvil (foundry) + a server on :3000, and drives Playwright/Chroma E2E
# against a pre-seeded `.env.test`. Standing all of that up in a stock runner is
# fragile and would red-light every feed PR, so the heavier integration/E2E lane
# is a follow-up (provision postgres + foundry + the server + a CI `.env`). See
# the TODO below.
#
# TODO(#9943 follow-up): feed currently reports ~227 skipped tests. They need
# triage — some are env-gated (RUN_OPTIONAL_INTEGRATION_TESTS / RUN_LIVE_LLM_TESTS),
# some are stale. Audit and either wire the env or delete the skips, then promote
# this lane to also run `test:integration` (with a postgres service) and the E2E
# slice. Do NOT mass-unskip blindly.
on:
pull_request:
branches: [main]
paths:
- "packages/feed/**"
- ".github/workflows/feed-test.yml"
workflow_dispatch:
concurrency:
group: feed-test-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CI: "true"
jobs:
feed-unit:
name: feed unit tests
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 25
# Required env for feed modules that validate configuration at import time.
# CI-only placeholders — no secrets. DATABASE_URL points at the postgres
# service below so any lazily-connecting module resolves a real DSN.
env:
DATABASE_URL: "postgres://postgres:postgres@localhost:5432/feed_test"
DIRECT_DATABASE_URL: "postgres://postgres:postgres@localhost:5432/feed_test"
NEXT_PUBLIC_STEWARD_API_URL: "http://localhost:31337"
STEWARD_JWT_SECRET: "ci-feed-unit-placeholder-jwt-secret"
CRON_SECRET: "ci-feed-unit-placeholder-cron-secret"
NODE_ENV: "test"
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: feed_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
# Install the repo root first so Bun links the root-owned elizaOS packages
# and the feed sub-workspaces that the root package.json globs. Then install
# the excluded feed root so its own app-level dependencies (drizzle-orm,
# Next, etc.) are available to `packages/feed/test:unit`. Feed postinstall
# bootstraps dev-only external agent frameworks, so skip scripts here.
- name: Install root workspace dependencies
run: bun install --frozen-lockfile
- name: Install feed root dependencies
working-directory: packages/feed
run: bun install --frozen-lockfile --ignore-scripts
- name: Build @elizaos/core dist
run: bun run --cwd packages/core build
- name: Run feed unit tests
working-directory: packages/feed
run: bun run test:unit
+104
View File
@@ -0,0 +1,104 @@
name: Submit Flatpak to Flathub
on:
workflow_call:
inputs:
version:
description: Semver string e.g. 2.0.1
required: true
type: string
tag:
description: Git tag e.g. v2.0.1
required: true
type: string
secrets:
FLATHUB_TOKEN:
required: false
workflow_dispatch:
inputs:
version:
description: Semver string e.g. 2.0.1
required: true
type: string
tag:
description: Git tag e.g. v2.0.1
required: true
type: string
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
submit-to-flathub:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Check if app is listed on Flathub
id: flathub_check
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if gh repo view flathub/ai.elizaos.App >/dev/null 2>&1; then
echo "listed=true" >> "$GITHUB_OUTPUT"
else
echo "listed=false" >> "$GITHUB_OUTPUT"
echo "::warning::App not yet listed on Flathub. Submit the initial listing at https://github.com/flathub/flathub before using this workflow."
fi
- name: Checkout flatpak manifest
if: steps.flathub_check.outputs.listed == 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
ref: ${{ inputs.tag }}
submodules: false
- name: Update manifest version
if: steps.flathub_check.outputs.listed == 'true'
env:
GH_TOKEN: ${{ secrets.FLATHUB_TOKEN || secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ inputs.version }}"
TAG="${{ inputs.tag }}"
echo "Preparing Flatpak update for ${VERSION}"
# Get the tarball sha256 for the release
TARBALL_URL="https://github.com/${{ github.repository }}/archive/refs/tags/${TAG}.tar.gz"
curl -L "$TARBALL_URL" -o /tmp/release.tar.gz
SHA256=$(sha256sum /tmp/release.tar.gz | awk '{print $1}')
echo "Release tarball SHA256: $SHA256"
echo "TARBALL_SHA256=$SHA256" >> "$GITHUB_ENV"
echo "TARBALL_URL=$TARBALL_URL" >> "$GITHUB_ENV"
- name: Create Flathub PR
if: steps.flathub_check.outputs.listed == 'true'
env:
GH_TOKEN: ${{ secrets.FLATHUB_TOKEN || secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ inputs.version }}"
BRANCH="update-to-${VERSION}"
# Fork the Flathub repo if needed, create branch, push updated manifest, open PR
gh repo fork flathub/ai.elizaos.App --clone --remote
cd ai.elizaos.App
git checkout -b "$BRANCH"
# Update the manifest with the new version/sha256
# (Actual manifest update depends on the Flathub manifest format)
sed -i "s|tag: .*|tag: ${VERSION}|g" ai.elizaos.App.yml || true
sed -i "s|sha256: .*|sha256: ${TARBALL_SHA256}|g" ai.elizaos.App.yml || true
git config user.name "elizaOS CI"
git config user.email "ci@elizaos.ai"
git add -A
git commit -m "Update to $VERSION"
git push origin "$BRANCH"
gh pr create \
--repo flathub/ai.elizaos.App \
--title "Update to elizaOS App $VERSION" \
--body "Automated update to version $VERSION (tag: ${{ inputs.tag }})" \
--base master \
--head "$BRANCH"
+117
View File
@@ -0,0 +1,117 @@
name: gitleaks
# SOC2 CC7.1 — automated secret scanning on every PR and push to protected
# branches. Fails the run on any finding. Configuration: .gitleaks.toml at
# repo root.
on:
pull_request:
branches: ["main", "develop"]
push:
branches: ["main", "develop"]
# cancel superseded in-flight runs for the same PR to free hosted-runner
# capacity. only cancels pull_request runs; push runs on protected branches
# (main/develop) are never cancelled mid-flight.
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
permissions:
contents: read
jobs:
gitleaks:
name: gitleaks
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout
# actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# Full history so gitleaks can scan the diff range on PRs.
fetch-depth: 0
- name: Install gitleaks (OSS binary — no license required)
# gitleaks/gitleaks-action@v2 requires a paid license for org repos.
# We download the OSS CLI directly so the scan remains free and
# honors our .gitleaks.toml allowlist + custom rules.
run: |
set -euo pipefail
GITLEAKS_VERSION=8.30.1
curl -sSL -o /tmp/gitleaks.tar.gz \
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
tar -xzf /tmp/gitleaks.tar.gz -C /tmp gitleaks
# Install without sudo so this works on any runner (github-hosted or
# self-hosted, which has no passwordless sudo): write to a user-writable
# dir and add it to PATH for the run step, not `sudo install /usr/local/bin`.
mkdir -p "$HOME/.local/bin"
install -m 0755 /tmp/gitleaks "$HOME/.local/bin/gitleaks"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
"$HOME/.local/bin/gitleaks" version
- name: Run gitleaks
env:
EVENT_NAME: ${{ github.event_name }}
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
BEFORE_SHA: ${{ github.event.before }}
CURRENT_SHA: ${{ github.sha }}
run: |
set -euo pipefail
RANGE_ARGS=()
if [[ "$EVENT_NAME" == "pull_request" ]]; then
RANGE_ARGS=(--log-opts "${PR_BASE_SHA}..${PR_HEAD_SHA}")
elif [[ -n "$BEFORE_SHA" && "$BEFORE_SHA" != "0000000000000000000000000000000000000000" ]]; then
RANGE_ARGS=(--log-opts "${BEFORE_SHA}..${CURRENT_SHA}")
elif ! git rev-parse -q --verify "${CURRENT_SHA}^" >/dev/null 2>&1; then
# Orphan/root tip commit: a squashed "clean baseline" release cut with
# no parent (the release process resets main/develop to a fresh root
# commit). `-1 <sha>` would diff it against the empty tree and re-flag
# every token-like string in the entire monorepo — generated docs
# dumps, benchmark fixtures, public chain addresses — the same
# full-history re-scan the tip-commit branch below exists to avoid, in
# its degenerate form. The baseline's content is a squash of develop
# commits each already scanned incrementally on their PR/push, so a
# full re-scan adds no coverage and only produces false-positive noise.
echo "gitleaks: tip ${CURRENT_SHA} is an orphan/root baseline commit (no parent);"
echo "its content was already scanned incrementally on develop. Skipping full-tree re-scan."
exit 0
else
# Push with no usable before-SHA (force-push, branch (re)creation, or
# an unknown range — common on main when it is reset/force-pushed).
# Scan only the tip commit, NOT the whole history: a full-history
# scan re-flags long-known historical artifacts (e.g. token-like
# strings inside vendored *.patch blobs) on every such push and costs
# ~8 min / 3 GB. New content still gets scanned via the PR range and
# this tip-commit scan; history is audited by the scheduled deep scan.
RANGE_ARGS=(--log-opts "-1 ${CURRENT_SHA}")
fi
gitleaks detect \
--config .gitleaks.toml \
--source . \
--verbose \
--redact \
--report-format sarif \
--report-path gitleaks.sarif \
--no-banner \
"${RANGE_ARGS[@]}"
- name: Upload SARIF
if: always()
# actions/upload-artifact@v7
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: gitleaks-sarif
path: gitleaks.sarif
if-no-files-found: ignore
retention-days: 7
+89
View File
@@ -0,0 +1,89 @@
# Nightly per-GPU voice-bench smoke run.
#
# STATUS: stub. Gated by workflow_dispatch input until a self-hosted CUDA GPU
# runner is wired. L4 (the cheapest GitHub-hosted GPU runner) is a reasonable
# starting target; for the full matrix (3090 / 4090 / 5090 / H200) we
# need a self-hosted pool.
#
# Per docs/inference/gpu-tier.md the per-GPU autotune profiles live in
# packages/inference/configs/gpu/, and `voice-bench bench gpu` prints
# the resolved plan for the host card. Once a real cuda PipelineDriver
# is wired (see packages/inference/voice-bench/README.md "Wiring the
# real pipeline"), this workflow runs the actual benchmark.
name: gpu-bench-nightly
on:
schedule:
# Nightly at 06:00 UTC. Adjust once we know what window the
# self-hosted runners are available.
- cron: "0 6 * * *"
workflow_dispatch:
inputs:
bundle:
description: "Bundle id (e.g. eliza-1-9b)"
required: false
default: "eliza-1-9b"
enable_gpu_bench:
description: "Run the scaffold GPU bench jobs"
required: false
type: boolean
default: false
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
detect:
name: detect host GPU
if: ${{ github.event_name == 'workflow_dispatch' && inputs.enable_gpu_bench == true }}
runs-on: [self-hosted, gpu-cuda-12.6]
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: install
run: bun install --frozen-lockfile || bun install --no-frozen-lockfile
- name: probe GPU + print autotune plan
run: |
bun run --cwd packages/inference/voice-bench bench gpu \
--bundle "${{ github.event.inputs.bundle || 'eliza-1-9b' }}" \
| tee gpu-plan.json
- name: upload plan
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: gpu-plan
path: gpu-plan.json
if-no-files-found: error
retention-days: 7
bench:
name: voice-bench (cuda)
needs: detect
if: ${{ github.event_name == 'workflow_dispatch' && inputs.enable_gpu_bench == true }}
runs-on: [self-hosted, gpu-cuda-12.6]
timeout-minutes: 60
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: install
run: bun install --frozen-lockfile || bun install --no-frozen-lockfile
- name: run voice-bench
run: |
bun run --cwd packages/inference/voice-bench bench \
--bundle "${{ github.event.inputs.bundle || 'eliza-1-9b' }}" \
--backend cuda \
--scenario all \
--runs 5 \
--output bench.json
- name: upload bench result
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: bench-${{ github.run_id }}
path: bench.json
if-no-files-found: error
retention-days: 7
@@ -0,0 +1,111 @@
name: Headscale API Key Health
# The Headscale admin key expires independently of deploy cadence. Probe the
# daemon-local API from the control-plane host so an expired or rebuild-lost key
# turns the nightly red without copying the admin credential into GitHub Actions.
on:
schedule:
- cron: "23 7 * * *"
workflow_dispatch:
inputs:
environment:
description: "Control-plane environment to probe"
type: choice
default: staging
options: [staging, production]
permissions:
contents: read
concurrency:
group: headscale-api-key-health-${{ github.event.inputs.environment || 'staging' }}
cancel-in-progress: false
jobs:
probe:
name: Probe ${{ github.event.inputs.environment || 'staging' }} Headscale key
runs-on: ubuntu-24.04
environment: ${{ github.event.inputs.environment || 'staging' }}
timeout-minutes: 5
env:
DEPLOY_HOST: ${{ secrets.ELIZA_PROVISIONING_HOST }}
DEPLOY_SSH_KEY: ${{ secrets.ELIZA_PROVISIONING_SSH_KEY }}
steps:
- name: Check probe configuration
run: |
fail=0
[ -n "$DEPLOY_HOST" ] || { echo "::error::missing secret ELIZA_PROVISIONING_HOST"; fail=1; }
[ -n "$DEPLOY_SSH_KEY" ] || { echo "::error::missing secret ELIZA_PROVISIONING_SSH_KEY"; fail=1; }
[ "$fail" = 0 ] || exit 1
- name: Probe daemon-local Headscale API
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2
with:
host: ${{ env.DEPLOY_HOST }}
username: deploy
key: ${{ env.DEPLOY_SSH_KEY }}
command_timeout: 2m
script: |
set -eu
ENV_FILE=/opt/eliza/cloud/.env.local
RESPONSE_FILE=$(mktemp)
trap 'rm -f "$RESPONSE_FILE"' EXIT
read_env_value() {
key="$1"
line=$(sudo grep -m1 "^${key}=" "$ENV_FILE" || true)
value="${line#*=}"
if [ "${value#\"}" != "$value" ] && [ "${value%\"}" != "$value" ]; then
value="${value#\"}"
value="${value%\"}"
fi
printf '%s' "$value"
}
sudo test -r "$ENV_FILE" || {
echo "::error::Headscale environment file is unreadable on the control-plane host"
exit 1
}
HEADSCALE_API_URL=$(read_env_value HEADSCALE_API_URL)
HEADSCALE_API_KEY=$(read_env_value HEADSCALE_API_KEY)
[ -n "$HEADSCALE_API_URL" ] || {
echo "::error::HEADSCALE_API_URL is missing from the control-plane environment"
exit 1
}
[ -n "$HEADSCALE_API_KEY" ] || {
echo "::error::HEADSCALE_API_KEY is missing from the control-plane environment"
exit 1
}
endpoint="${HEADSCALE_API_URL%/}/api/v1/user"
status=$(curl --silent --show-error --connect-timeout 5 --max-time 20 \
--output "$RESPONSE_FILE" --write-out '%{http_code}' \
--header "Authorization: Bearer $HEADSCALE_API_KEY" \
"$endpoint") || {
echo "::error::Headscale API probe could not reach the daemon-local endpoint"
exit 1
}
case "$status" in
200)
node -e '
const fs = require("node:fs");
const payload = JSON.parse(fs.readFileSync(process.argv[1], "utf8"));
if (!payload || !Array.isArray(payload.users)) process.exit(1);
' "$RESPONSE_FILE" || {
echo "::error::Headscale API returned HTTP 200 with an invalid user-list payload"
exit 1
}
echo "Headscale API key accepted by the daemon-local endpoint."
;;
401|403)
echo "::error::Headscale rejected the configured API key with HTTP $status; rotate it before provisioning fails"
exit 1
;;
*)
echo "::error::Headscale API key probe returned unexpected HTTP $status"
exit 1
;;
esac
+46
View File
@@ -0,0 +1,46 @@
name: Hetzner E2E Reaper
# Belt-and-suspenders cleanup. Runs every 30 minutes and deletes any
# `ci=true,workflow=hetzner-e2e` servers older than 60 minutes — in
# case a crashed workflow leaks one. Gracefully no-ops when
# HCLOUD_TOKEN_CI isn't set so it doesn't spam-fail before secrets
# are configured.
on:
schedule:
- cron: '*/30 * * * *'
workflow_dispatch:
concurrency:
group: hetzner-e2e-reaper
cancel-in-progress: false
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
reap:
name: Sweep stale CI servers
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ci-hetzner-e2e
timeout-minutes: 10
env:
HCLOUD_TOKEN_CI: ${{ secrets.HCLOUD_TOKEN_CI }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Install dependencies
run: bun install --ignore-scripts
- name: Ensure generated shared i18n data
run: bun packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Reap
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-reaper.ts
+428
View File
@@ -0,0 +1,428 @@
name: Hetzner Agent E2E
# Nightly end-to-end smoke that provisions a fresh Hetzner cpx11 server,
# waits for cloud-init, deploys a trivial agent via the Eliza Cloud
# staging API, runs a bridge-ping healthcheck plus one REAL chat turn
# (message.send through the production bridge path — #15603 A3), then
# tears everything down. Gracefully skips when the required secrets are
# unset (so this workflow can land on develop and be activated later), but a
# configured secret that is rejected by Hetzner or the Cloud API is a red
# workflow: green-by-dead-credential is indistinguishable from a real pass.
#
# Required secrets / config (GitHub environment: ci-hetzner-e2e):
# HCLOUD_TOKEN_CI - Hetzner Cloud API token (CI-scoped project)
# CLOUD_E2E_API_KEY - Eliza Cloud staging bearer token (long-lived)
# CI_SSH_PRIVATE_KEY - OpenSSH private key (matches the public key
# uploaded to Hetzner)
# CI_SSH_PUBLIC_KEY_ID - Numeric Hetzner SSH key id for the above
#
# See packages/scripts/cloud/admin/hetzner-e2e/README.md for setup.
on:
schedule:
- cron: '0 7 * * *' # 07:00 UTC nightly
workflow_dispatch:
push:
branches: [develop]
paths:
- '.github/workflows/hetzner-e2e.yml'
- '.github/workflows/hetzner-e2e-reaper.yml'
- 'packages/scripts/cloud/admin/hetzner-e2e/**'
- 'packages/cloud/shared/src/lib/services/containers/**'
- 'packages/scripts/cloud/admin/daemons/**'
- 'packages/scripts/cloud/admin/bootstrap-provisioning-worker-host.mjs'
concurrency:
group: hetzner-e2e
cancel-in-progress: false
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
deploy:
name: Provision + deploy + healthcheck
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ci-hetzner-e2e
timeout-minutes: 25
outputs:
provisioned: ${{ steps.provision.outputs.provisioned }}
env:
HCLOUD_TOKEN_CI: ${{ secrets.HCLOUD_TOKEN_CI }}
CLOUD_E2E_API_KEY: ${{ secrets.CLOUD_E2E_API_KEY || secrets.ELIZACLOUD_API_KEY }}
CI_SSH_PRIVATE_KEY: ${{ secrets.CI_SSH_PRIVATE_KEY }}
CI_SSH_PUBLIC_KEY_ID: ${{ secrets.CI_SSH_PUBLIC_KEY_ID }}
HETZNER_E2E_STATE_FILE: /tmp/hetzner-e2e-state.json
steps:
- name: Check secret configuration
id: secret_config
run: |
missing=()
[ -z "$HCLOUD_TOKEN_CI" ] && missing+=("HCLOUD_TOKEN_CI")
[ -z "$CLOUD_E2E_API_KEY" ] && missing+=("CLOUD_E2E_API_KEY")
[ -z "$CI_SSH_PRIVATE_KEY" ] && missing+=("CI_SSH_PRIVATE_KEY")
[ -z "$CI_SSH_PUBLIC_KEY_ID" ] && missing+=("CI_SSH_PUBLIC_KEY_ID")
if [ "${#missing[@]}" -ne 0 ]; then
echo "configured=false" >> "$GITHUB_OUTPUT"
echo "::warning::Missing secrets: ${missing[*]}; skipping Hetzner E2E."
{
echo "### Hetzner E2E skipped"
echo ""
echo "Missing secrets in GitHub environment \`ci-hetzner-e2e\`:"
for s in "${missing[@]}"; do echo "- \`$s\`"; done
echo ""
echo "See \`packages/scripts/cloud/admin/hetzner-e2e/README.md\` for setup."
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "::error::Manual dispatch requires all hetzner-e2e secrets."
exit 1
fi
exit 0
fi
echo "configured=true" >> "$GITHUB_OUTPUT"
- name: Checkout
if: steps.secret_config.outputs.configured == 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Bun
if: steps.secret_config.outputs.configured == 'true'
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Debug workspace
if: steps.secret_config.outputs.configured == 'true'
run: |
echo "cwd=$(pwd)"
echo "workspace=$GITHUB_WORKSPACE"
ls -la "$GITHUB_WORKSPACE" | head -30
test -f "$GITHUB_WORKSPACE/package.json" && echo HAS_PKG || echo NO_PKG
- name: Install dependencies
if: steps.secret_config.outputs.configured == 'true'
run: bun install --no-save --ignore-scripts
- name: Ensure generated shared i18n data
if: steps.secret_config.outputs.configured == 'true'
run: bun packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Check cloud API auth
id: cloud_auth
if: steps.secret_config.outputs.configured == 'true'
run: |
set +e
response="$(
bun -e '
const base = (process.env.CLOUD_SMOKE_BASE_URL ?? "https://api-staging.elizacloud.ai").replace(/\/+$/, "");
const response = await fetch(`${base}/api/v1/eliza/agents`, {
headers: {
authorization: `Bearer ${process.env.CLOUD_E2E_API_KEY ?? ""}`,
accept: "application/json",
"user-agent": "hetzner-e2e-preflight/1.0",
},
signal: AbortSignal.timeout(30_000),
});
const text = await response.text();
console.log(JSON.stringify({ status: response.status, body: text.slice(0, 500) }));
'
)"
status=$?
set -e
if [ "$status" -ne 0 ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::warning::Cloud API auth preflight could not reach staging; skipping Hetzner E2E."
{
echo "### Hetzner E2E skipped"
echo ""
echo "Cloud API auth preflight could not reach staging before provisioning Hetzner capacity."
echo ""
echo "\`\`\`json"
echo "$response"
echo "\`\`\`"
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
exit 1
fi
exit 0
fi
status_code="$(printf '%s' "$response" | bun -e 'const chunks=[]; for await (const chunk of Bun.stdin.stream()) chunks.push(Buffer.from(chunk)); const data = JSON.parse(Buffer.concat(chunks).toString() || "{}"); console.log(data.status ?? "");')"
if [ "$status_code" = "401" ] || [ "$status_code" = "403" ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::error::Cloud API rejected CLOUD_E2E_API_KEY with HTTP $status_code; refresh the ci-hetzner-e2e environment secret."
{
echo "### Hetzner E2E blocked by rejected Cloud API credential"
echo ""
echo "Cloud API rejected \`CLOUD_E2E_API_KEY\` with HTTP \`$status_code\` before provisioning Hetzner capacity."
echo ""
echo "Refresh the \`CLOUD_E2E_API_KEY\` / \`ELIZACLOUD_API_KEY\` secret in the \`ci-hetzner-e2e\` environment, or check the staging API-key lookup path."
} >> "$GITHUB_STEP_SUMMARY"
exit 1
fi
if [ "${status_code#2}" != "$status_code" ]; then
echo "ready=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "${status_code#5}" != "$status_code" ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::warning::Cloud API auth preflight returned HTTP $status_code; skipping Hetzner E2E before provisioning."
{
echo "### Hetzner E2E skipped"
echo ""
echo "Cloud API auth preflight returned HTTP \`$status_code\` before provisioning Hetzner capacity."
echo ""
echo "\`\`\`json"
echo "$response"
echo "\`\`\`"
echo ""
echo "Check staging Cloud API logs for \`user-agent: hetzner-e2e-preflight/1.0\` on \`/api/v1/eliza/agents\`."
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
exit 1
fi
exit 0
fi
echo "::error::Unexpected cloud API auth preflight response: $response"
exit 1
- name: Check Hetzner API token
id: hcloud_auth
if: steps.secret_config.outputs.configured == 'true' && steps.cloud_auth.outputs.ready == 'true'
run: |
set +e
response="$(
bun -e '
const response = await fetch("https://api.hetzner.cloud/v1/ssh_keys?per_page=1", {
headers: {
authorization: `Bearer ${process.env.HCLOUD_TOKEN_CI ?? ""}`,
accept: "application/json",
"user-agent": "hetzner-e2e-preflight/1.0",
},
signal: AbortSignal.timeout(30_000),
});
const text = await response.text();
console.log(JSON.stringify({ status: response.status, body: text.slice(0, 500) }));
'
)"
status=$?
set -e
if [ "$status" -ne 0 ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::warning::Hetzner API token preflight could not reach api.hetzner.cloud; skipping Hetzner E2E."
{
echo "### Hetzner E2E skipped"
echo ""
echo "Hetzner API token preflight could not reach \`api.hetzner.cloud\` before provisioning capacity."
echo ""
echo "\`\`\`json"
echo "$response"
echo "\`\`\`"
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
exit 1
fi
exit 0
fi
status_code="$(printf '%s' "$response" | bun -e 'const chunks=[]; for await (const chunk of Bun.stdin.stream()) chunks.push(Buffer.from(chunk)); const data = JSON.parse(Buffer.concat(chunks).toString() || "{}"); console.log(data.status ?? "");')"
if [ "$status_code" = "401" ] || [ "$status_code" = "403" ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::error::Hetzner rejected HCLOUD_TOKEN_CI with HTTP $status_code; refresh the ci-hetzner-e2e environment secret."
{
echo "### Hetzner E2E blocked by rejected Hetzner credential"
echo ""
echo "Hetzner rejected \`HCLOUD_TOKEN_CI\` with HTTP \`$status_code\` before provisioning capacity."
echo ""
echo "Refresh \`HCLOUD_TOKEN_CI\` in the \`ci-hetzner-e2e\` environment, or confirm the Hetzner project is still active."
} >> "$GITHUB_STEP_SUMMARY"
exit 1
fi
if [ "${status_code#2}" != "$status_code" ]; then
echo "ready=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "${status_code#5}" != "$status_code" ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::warning::Hetzner API token preflight returned HTTP $status_code; skipping Hetzner E2E before provisioning."
{
echo "### Hetzner E2E skipped"
echo ""
echo "Hetzner API token preflight returned HTTP \`$status_code\` before provisioning capacity."
echo ""
echo "\`\`\`json"
echo "$response"
echo "\`\`\`"
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
exit 1
fi
exit 0
fi
echo "::error::Unexpected Hetzner API token preflight response: $response"
exit 1
- name: Provision Hetzner server
id: provision
if: steps.secret_config.outputs.configured == 'true' && steps.cloud_auth.outputs.ready == 'true' && steps.hcloud_auth.outputs.ready == 'true'
run: |
log=/tmp/hetzner-e2e-provision.log
set +e
bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-provision.ts 2>&1 | tee "$log"
rc=${PIPESTATUS[0]}
set -e
if [ "$rc" -eq 0 ]; then
echo "provisioned=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# A rejected / expired / read-only HCLOUD_TOKEN_CI (or a deactivated project)
# surfaces as HTTP 401/403 missing_token / "permission denied" when CREATING the
# server — the read-only token preflight above can pass while this write fails.
# That is an operator secret problem, not a code regression, but the
# token is configured and rejected. Fail red for every event so the
# nightly cannot look healthy while real-infra coverage is disabled.
if grep -qE "Hetzner rejected the token|missing_token|permission denied|HTTP 401|HTTP 403|status: 401|status: 403" "$log"; then
echo "provisioned=false" >> "$GITHUB_OUTPUT"
echo "::error::Hetzner rejected HCLOUD_TOKEN_CI (HTTP 401/403); refresh the token in the ci-hetzner-e2e environment."
{
echo "### Hetzner E2E blocked by rejected Hetzner credential"
echo ""
echo "Hetzner rejected \`HCLOUD_TOKEN_CI\` (HTTP 401/403 \`missing_token\` / permission denied) when creating the server."
echo ""
echo "The token may be read-only or expired for writes. Refresh \`HCLOUD_TOKEN_CI\` in the \`ci-hetzner-e2e\` environment with a read-write CI token, or confirm the project is active."
} >> "$GITHUB_STEP_SUMMARY"
exit 1
fi
# Any other provision failure is real — surface it (red).
echo "provisioned=false" >> "$GITHUB_OUTPUT"
exit "$rc"
- name: Surface provision failure diagnostic
if: always() && steps.secret_config.outputs.configured == 'true' && steps.cloud_auth.outputs.ready == 'true' && steps.hcloud_auth.outputs.ready == 'true' && steps.provision.outcome == 'failure'
run: |
log=/tmp/hetzner-e2e-provision.log
{
echo "### Hetzner E2E provisioning failed"
echo ""
if [ -f "$log" ] && grep -qE "quota exhausted|server limit reached|limit_reached|resource_limit_exceeded|quota_exceeded" "$log"; then
echo "**Cause:** Hetzner project quota exhausted (server cap reached)."
echo ""
echo "Operator action:"
echo "1. Open https://console.hetzner.cloud/ and check the CI project for leaked servers (filter labels \`ci=true\`, \`workflow=hetzner-e2e\`)."
echo "2. Delete any leaked servers, or wait for the half-hourly reaper workflow to sweep them."
echo "3. Re-run this workflow."
echo ""
echo "If the project itself has a tighter cap than the fallback ladder can survive, request a quota increase from Hetzner support or rotate \`HCLOUD_TOKEN_CI\` to a project with capacity."
elif [ -f "$log" ] && grep -qE "Hetzner rejected the token|missing_token|HTTP 401|HTTP 403" "$log"; then
echo "**Cause:** Hetzner rejected the API token (HTTP 401/403)."
echo ""
echo "Operator action: refresh \`HCLOUD_TOKEN_CI\` in the \`ci-hetzner-e2e\` GitHub environment, or confirm the project is still active."
else
echo "**Cause:** see the \"Provision Hetzner server\" step log for the raw error."
echo ""
echo "If the error message is unfamiliar, check \`packages/scripts/cloud/admin/hetzner-e2e/README.md\` and the fallback ladder in \`hetzner-e2e-provision.ts\`."
fi
} >> "$GITHUB_STEP_SUMMARY"
- name: Wait for host ready
if: steps.provision.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-wait-ready.ts
- name: Deploy trivial agent
if: steps.provision.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-deploy-agent.ts
- name: Healthcheck
if: steps.provision.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-healthcheck.ts
# A real user-message round-trip through the same Worker bridge path a
# production chat takes. status.get alone lets "provisioned but chat
# dead-ends" regressions (#15347) pass the nightly. Failure here fails
# the deploy job; the teardown job runs regardless (needs+always()).
- name: Chat turn
if: steps.provision.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-chat.ts
- name: Upload state artifact
if: always() && steps.provision.outputs.provisioned == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: hetzner-e2e-state
path: /tmp/hetzner-e2e-state.json
if-no-files-found: ignore
retention-days: 7
teardown:
name: Teardown
needs: deploy
if: always()
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ci-hetzner-e2e
timeout-minutes: 10
env:
HCLOUD_TOKEN_CI: ${{ secrets.HCLOUD_TOKEN_CI }}
HETZNER_E2E_STATE_FILE: /tmp/hetzner-e2e-state.json
steps:
- name: Check secret configuration
id: secret_config
run: |
if [ -z "$HCLOUD_TOKEN_CI" ]; then
echo "configured=false" >> "$GITHUB_OUTPUT"
echo "::warning::HCLOUD_TOKEN_CI missing; nothing to tear down."
exit 0
fi
echo "configured=true" >> "$GITHUB_OUTPUT"
- name: Checkout
if: steps.secret_config.outputs.configured == 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Bun
if: steps.secret_config.outputs.configured == 'true'
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Install dependencies
if: steps.secret_config.outputs.configured == 'true'
run: bun install --no-save --ignore-scripts
- name: Ensure generated shared i18n data
if: steps.secret_config.outputs.configured == 'true'
run: bun packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Download state artifact
if: steps.secret_config.outputs.configured == 'true'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: hetzner-e2e-state
path: /tmp/hetzner-e2e-state
continue-on-error: true
- name: Restore state file
if: steps.secret_config.outputs.configured == 'true'
run: |
if [ -f /tmp/hetzner-e2e-state/hetzner-e2e-state.json ]; then
cp /tmp/hetzner-e2e-state/hetzner-e2e-state.json /tmp/hetzner-e2e-state.json
echo "State file restored:"
cat /tmp/hetzner-e2e-state.json
else
echo "No state artifact; teardown will fall back to label sweep."
fi
- name: Teardown
if: steps.secret_config.outputs.configured == 'true'
env:
# Lets the teardown script distinguish "dead token, nothing was
# provisioned, nothing can leak" (soft-skip) from "dead token but a
# server exists" (hard fail so the leak is never silent).
DEPLOY_PROVISIONED: ${{ needs.deploy.outputs.provisioned }}
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-teardown.ts
@@ -0,0 +1,81 @@
name: Hetzner Provision Smoke
# Manual real-Hetzner provision smoke. Provisions a fresh cx22 server, waits
# for cloud-init / SSH readiness, then tears it down — all in one job so the
# /tmp state file persists across steps. Deliberately bypasses the Eliza Cloud
# staging API (no auth preflight, no trivial-agent deploy) so the core Hetzner
# provisioning path can be authentically confirmed even when the staging Cloud
# API or its Redis is degraded. Uses the same `ci-hetzner-e2e` environment
# secrets and the same provision/wait/teardown scripts as the nightly e2e.
on:
workflow_dispatch:
concurrency:
group: hetzner-e2e
cancel-in-progress: false
permissions:
contents: read
jobs:
provision:
name: Provision + wait-ready + teardown (real Hetzner)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ci-hetzner-e2e
timeout-minutes: 20
env:
HCLOUD_TOKEN_CI: ${{ secrets.HCLOUD_TOKEN_CI }}
CI_SSH_PRIVATE_KEY: ${{ secrets.CI_SSH_PRIVATE_KEY }}
CI_SSH_PUBLIC_KEY_ID: ${{ secrets.CI_SSH_PUBLIC_KEY_ID }}
HETZNER_E2E_STATE_FILE: /tmp/hetzner-e2e-state.json
# cx22 is deprecated and cax (ARM) isn't placeable for this project;
# cpx22 (x86 2c/4g) is available across the EU datacenters.
HETZNER_E2E_SERVER_TYPE: cpx22
HETZNER_E2E_LOCATION: nbg1
steps:
- name: Check secret configuration
run: |
missing=()
[ -z "$HCLOUD_TOKEN_CI" ] && missing+=("HCLOUD_TOKEN_CI")
[ -z "$CI_SSH_PRIVATE_KEY" ] && missing+=("CI_SSH_PRIVATE_KEY")
[ -z "$CI_SSH_PUBLIC_KEY_ID" ] && missing+=("CI_SSH_PUBLIC_KEY_ID")
if [ ${#missing[@]} -ne 0 ]; then
echo "::error::Missing ci-hetzner-e2e secrets: ${missing[*]}"
exit 1
fi
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Install dependencies
run: bun install --no-save --ignore-scripts
- name: Ensure generated shared i18n data
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: List Hetzner catalog (what's actually placeable)
run: |
echo "=== non-deprecated server types (id name arch cores/mem) ==="
curl -s -H "Authorization: Bearer $HCLOUD_TOKEN_CI" \
https://api.hetzner.cloud/v1/server_types \
| jq -r '.server_types[] | select(.deprecation==null) | "\(.id)\t\(.name)\t\(.architecture)\t\(.cores)c/\(.memory)g"' | sort -n
echo "=== per-datacenter available server-type IDs ==="
curl -s -H "Authorization: Bearer $HCLOUD_TOKEN_CI" \
https://api.hetzner.cloud/v1/datacenters \
| jq -r '.datacenters[] | "\(.name)\tavail=[\(.server_types.available | join(","))]"'
- name: Provision Hetzner server
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-provision.ts
- name: Wait for host ready (cloud-init + SSH)
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-wait-ready.ts
- name: Teardown (always)
if: always()
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-teardown.ts
@@ -0,0 +1,281 @@
name: Hetzner Sleep-Wake Smoke
# Real-infrastructure proof of the sleep -> back up -> de-provision the Hetzner
# box -> wake -> re-provision from backup -> restore lifecycle. Uses real
# Hetzner boxes via HCLOUD_TOKEN_CI + SSH; independent of the Eliza Cloud
# staging API (which is currently degraded). The backup transport here is the
# runner filesystem standing in for S3/R2 (CI has HCLOUD + SSH, not R2) — the
# point is to authentically exercise the provision -> capture state ->
# DESTROY THE BOX -> provision a NEW box -> restore -> verify cycle end to end.
#
# Secret gating mirrors hetzner-e2e.yml: missing HCLOUD_TOKEN_CI / SSH secrets
# skip cleanly (exit 0 + loud warning and step summary) so forks and inactive
# environments stay quiet. A configured credential that Hetzner rejects is red
# for every event: otherwise the real-infra nightly can look healthy while it
# never exercised anything.
#
# Required secrets (GitHub environment: ci-hetzner-e2e):
# HCLOUD_TOKEN_CI - Hetzner Cloud API token (CI-scoped project)
# CI_SSH_PRIVATE_KEY - OpenSSH private key (matches the public key
# uploaded to Hetzner)
# CI_SSH_PUBLIC_KEY_ID - Numeric Hetzner SSH key id for the above
on:
schedule:
# 06:00 UTC nightly — an hour before hetzner-e2e's 07:00 slot. Both
# workflows share the `hetzner-e2e` concurrency group (no cancel), so a
# long sleep-wake run queues the agent e2e instead of racing it for the
# CI project's server quota.
- cron: '0 6 * * *'
workflow_dispatch:
push:
branches: [claude/hetzner-provision-smoke]
concurrency:
group: hetzner-e2e
cancel-in-progress: false
permissions:
contents: read
jobs:
sleep-wake:
name: provision -> sleep(backup+destroy) -> wake(reprovision+restore)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ci-hetzner-e2e
timeout-minutes: 30
env:
HCLOUD_TOKEN_CI: ${{ secrets.HCLOUD_TOKEN_CI }}
CI_SSH_PRIVATE_KEY: ${{ secrets.CI_SSH_PRIVATE_KEY }}
CI_SSH_PUBLIC_KEY_ID: ${{ secrets.CI_SSH_PUBLIC_KEY_ID }}
HETZNER_E2E_STATE_FILE: /tmp/hetzner-e2e-state.json
HETZNER_E2E_SERVER_TYPE: cpx22
HETZNER_E2E_LOCATION: nbg1
SSH_OPTS: -i /tmp/ci_ssh_key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=15 -o BatchMode=yes
steps:
- name: Check secret configuration
id: secret_config
run: |
missing=()
[ -z "$HCLOUD_TOKEN_CI" ] && missing+=("HCLOUD_TOKEN_CI")
[ -z "$CI_SSH_PRIVATE_KEY" ] && missing+=("CI_SSH_PRIVATE_KEY")
[ -z "$CI_SSH_PUBLIC_KEY_ID" ] && missing+=("CI_SSH_PUBLIC_KEY_ID")
if [ "${#missing[@]}" -ne 0 ]; then
echo "configured=false" >> "$GITHUB_OUTPUT"
echo "::warning::Missing secrets: ${missing[*]}; skipping Hetzner sleep-wake smoke."
{
echo "### Hetzner sleep-wake smoke skipped"
echo ""
echo "Missing secrets in GitHub environment \`ci-hetzner-e2e\`:"
for s in "${missing[@]}"; do echo "- \`$s\`"; done
echo ""
echo "See \`packages/scripts/cloud/admin/hetzner-e2e/README.md\` for setup."
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "::error::Manual dispatch requires all hetzner-e2e secrets."
exit 1
fi
exit 0
fi
echo "configured=true" >> "$GITHUB_OUTPUT"
- name: Checkout
if: steps.secret_config.outputs.configured == 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Bun
if: steps.secret_config.outputs.configured == 'true'
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: canary
- name: Install deps
if: steps.secret_config.outputs.configured == 'true'
run: bun install --no-save --ignore-scripts
- name: Ensure generated shared i18n data
if: steps.secret_config.outputs.configured == 'true'
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Check Hetzner API token
id: hcloud_auth
if: steps.secret_config.outputs.configured == 'true'
run: |
set +e
response="$(
bun -e '
const response = await fetch("https://api.hetzner.cloud/v1/ssh_keys?per_page=1", {
headers: {
authorization: `Bearer ${process.env.HCLOUD_TOKEN_CI ?? ""}`,
accept: "application/json",
"user-agent": "hetzner-sleep-wake-preflight/1.0",
},
signal: AbortSignal.timeout(30_000),
});
const text = await response.text();
console.log(JSON.stringify({ status: response.status, body: text.slice(0, 500) }));
'
)"
status=$?
set -e
if [ "$status" -ne 0 ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::warning::Hetzner API token preflight could not reach api.hetzner.cloud; skipping sleep-wake smoke."
{
echo "### Hetzner sleep-wake smoke skipped"
echo ""
echo "Hetzner API token preflight could not reach \`api.hetzner.cloud\` before provisioning capacity."
echo ""
echo "\`\`\`json"
echo "$response"
echo "\`\`\`"
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
exit 1
fi
exit 0
fi
status_code="$(printf '%s' "$response" | bun -e 'const chunks=[]; for await (const chunk of Bun.stdin.stream()) chunks.push(Buffer.from(chunk)); const data = JSON.parse(Buffer.concat(chunks).toString() || "{}"); console.log(data.status ?? "");')"
if [ "$status_code" = "401" ] || [ "$status_code" = "403" ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::error::Hetzner rejected HCLOUD_TOKEN_CI with HTTP $status_code; refresh the ci-hetzner-e2e environment secret."
{
echo "### Hetzner sleep-wake smoke blocked by rejected Hetzner credential"
echo ""
echo "Hetzner rejected \`HCLOUD_TOKEN_CI\` with HTTP \`$status_code\` before provisioning capacity."
echo ""
echo "Refresh \`HCLOUD_TOKEN_CI\` in the \`ci-hetzner-e2e\` environment, or confirm the Hetzner project is still active."
} >> "$GITHUB_STEP_SUMMARY"
exit 1
fi
if [ "${status_code#2}" != "$status_code" ]; then
echo "ready=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "${status_code#5}" != "$status_code" ]; then
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::warning::Hetzner API token preflight returned HTTP $status_code; skipping sleep-wake smoke before provisioning."
{
echo "### Hetzner sleep-wake smoke skipped"
echo ""
echo "Hetzner API token preflight returned HTTP \`$status_code\` before provisioning capacity."
echo ""
echo "\`\`\`json"
echo "$response"
echo "\`\`\`"
} >> "$GITHUB_STEP_SUMMARY"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
exit 1
fi
exit 0
fi
echo "::error::Unexpected Hetzner API token preflight response: $response"
exit 1
- name: Write SSH key
if: steps.secret_config.outputs.configured == 'true' && steps.hcloud_auth.outputs.ready == 'true'
run: |
printf '%s\n' "$CI_SSH_PRIVATE_KEY" > /tmp/ci_ssh_key
chmod 600 /tmp/ci_ssh_key
# ── Provision box A + boot ──────────────────────────────────────────
- name: Provision box A
id: provision_a
if: steps.secret_config.outputs.configured == 'true' && steps.hcloud_auth.outputs.ready == 'true'
run: |
log=/tmp/hetzner-sleep-wake-provision.log
set +e
bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-provision.ts 2>&1 | tee "$log"
rc=${PIPESTATUS[0]}
set -e
if [ "$rc" -eq 0 ]; then
echo "provisioned=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# A rejected / expired / read-only HCLOUD_TOKEN_CI (or a deactivated
# project) surfaces as HTTP 401/403 when CREATING the server — the
# read-only preflight above can pass while this write fails. That is
# an operator secret problem, not a code regression, but it must fail
# red so a scheduled run cannot look green while the token is dead.
if grep -qE "Hetzner rejected the token|missing_token|permission denied|HTTP 401|HTTP 403|status: 401|status: 403" "$log"; then
echo "provisioned=false" >> "$GITHUB_OUTPUT"
echo "::error::Hetzner rejected HCLOUD_TOKEN_CI (HTTP 401/403); refresh the token in the ci-hetzner-e2e environment."
{
echo "### Hetzner sleep-wake smoke blocked by rejected Hetzner credential"
echo ""
echo "Hetzner rejected \`HCLOUD_TOKEN_CI\` (HTTP 401/403 \`missing_token\` / permission denied) when creating the server."
echo ""
echo "The token may be read-only or expired for writes. Refresh \`HCLOUD_TOKEN_CI\` in the \`ci-hetzner-e2e\` environment with a read-write CI token, or confirm the project is active."
} >> "$GITHUB_STEP_SUMMARY"
exit 1
fi
# Any other provision failure is real — surface it (red).
echo "provisioned=false" >> "$GITHUB_OUTPUT"
exit "$rc"
- name: Wait A ready
if: steps.provision_a.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-wait-ready.ts
# ── Write agent state on A, then BACK IT UP off-box ─────────────────
- name: Write + back up agent state from A
if: steps.provision_a.outputs.provisioned == 'true'
run: |
IP_A=$(jq -r .ip "$HETZNER_E2E_STATE_FILE"); ID_A=$(jq -r .id "$HETZNER_E2E_STATE_FILE")
echo "box A: id=$ID_A ip=$IP_A"
echo "$ID_A" > /tmp/box-a-id
# Simulate agent state (mirrors AgentBackupStateData shape).
STATE='{"memories":[{"role":"user","text":"remember: orbital-mango-4417","timestamp":1},{"role":"assistant","text":"noted","timestamp":2}],"config":{"name":"sleepwake-probe"},"workspaceFiles":{"note.md":"survive the sleep"}}'
ssh $SSH_OPTS root@"$IP_A" "mkdir -p /root/agent && cat > /root/agent/state.json" <<< "$STATE"
# "Snapshot to S3" — here the runner FS stands in for R2.
scp $SSH_OPTS root@"$IP_A":/root/agent/state.json /tmp/backup-A.json
sha256sum /tmp/backup-A.json | awk '{print $1}' > /tmp/backup-A.sha
echo "backup sha: $(cat /tmp/backup-A.sha)"
test -s /tmp/backup-A.json
# ── SLEEP: de-provision (destroy) box A ─────────────────────────────
- name: Sleep — de-provision (destroy) box A
if: steps.provision_a.outputs.provisioned == 'true'
run: |
bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-teardown.ts
ID_A=$(cat /tmp/box-a-id)
code=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer $HCLOUD_TOKEN_CI" "https://api.hetzner.cloud/v1/servers/$ID_A")
echo "GET server $ID_A after teardown -> $code (expect 404)"
[ "$code" = "404" ] || { echo "::error::box A not destroyed"; exit 1; }
# ── WAKE: re-provision a NEW box B + restore from backup ────────────
- name: Wake — re-provision box B
if: steps.provision_a.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-provision.ts
- name: Wait B ready
if: steps.provision_a.outputs.provisioned == 'true'
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-wait-ready.ts
- name: Restore backup onto B + verify state survived
if: steps.provision_a.outputs.provisioned == 'true'
run: |
IP_B=$(jq -r .ip "$HETZNER_E2E_STATE_FILE"); ID_B=$(jq -r .id "$HETZNER_E2E_STATE_FILE")
ID_A=$(cat /tmp/box-a-id)
echo "box B: id=$ID_B ip=$IP_B (was A id=$ID_A)"
[ "$ID_B" != "$ID_A" ] || { echo "::error::wake did not create a NEW box"; exit 1; }
scp $SSH_OPTS /tmp/backup-A.json root@"$IP_B":/root/agent-state-restored.json
REMOTE_SHA=$(ssh $SSH_OPTS root@"$IP_B" "sha256sum /root/agent-state-restored.json | awk '{print \$1}'")
echo "restored sha on B: $REMOTE_SHA ; backup sha: $(cat /tmp/backup-A.sha)"
[ "$REMOTE_SHA" = "$(cat /tmp/backup-A.sha)" ] || { echo "::error::restored state mismatch"; exit 1; }
ssh $SSH_OPTS root@"$IP_B" "grep -q orbital-mango-4417 /root/agent-state-restored.json" \
&& echo "PASS: agent state survived sleep(destroy A) -> wake(new box B) -> restore"
# Runs whenever provisioning was ATTEMPTED (success or failure — a failed
# create can still leak a server), and skips only when provisioning never
# started. DEPLOY_PROVISIONED lets the teardown script distinguish "dead
# token, nothing was provisioned, nothing can leak" (soft-skip) from
# "dead token but a server exists" (hard fail so the leak is never
# silent) — same contract as hetzner-e2e.yml's teardown job.
- name: Teardown box B (always)
if: always() && steps.provision_a.outcome != 'skipped'
env:
DEPLOY_PROVISIONED: ${{ steps.provision_a.outputs.provisioned }}
run: bun run packages/scripts/cloud/admin/hetzner-e2e/hetzner-e2e-teardown.ts
@@ -0,0 +1,189 @@
# HyperliquidBench — publishable live harness runs
#
# Runs the HyperliquidBench live harness (eliza / hermes / openclaw) against a
# real Hyperliquid network so the resulting rows are publishable (demo-mode rows
# are intentionally rejected by the publishability gate). This is the only path
# that produces release-comparable Hyperliquid numbers.
#
# OPERATOR PREREQUISITE: add an `HL_PRIVATE_KEY` repo secret holding the funded
# signing key for the chosen network (testnet recommended; mainnet trades real
# funds). With no secret configured this workflow SKIPS GRACEFULLY — it never
# fails CI when the key is absent.
#
# Manual trigger only (workflow_dispatch). Live trading must be an explicit
# operator decision, so there is deliberately no schedule.
#
# Required repo secret (self-documented):
# HL_PRIVATE_KEY — Hyperliquid signing key (0x...) for --no-demo runs.
# Optional repo secret:
# CEREBRAS_API_KEY — gemma-4-31b plan generation for the eliza/hermes/
# openclaw harnesses. Without it the harnesses cannot
# generate live plans; the run still skips when the
# private key is absent.
name: HyperliquidBench (live)
on:
workflow_dispatch:
inputs:
network:
description: "Hyperliquid network for live execution"
required: false
type: choice
default: "testnet"
options:
- testnet
- mainnet
provider:
description: "Model provider for plan generation"
required: false
type: string
default: "cerebras"
model:
description: "Model name for plan generation"
required: false
type: string
default: "gemma-4-31b"
concurrency:
group: hyperliquid-bench-live-${{ github.ref }}
cancel-in-progress: false
# Default to least privilege.
permissions:
contents: read
env:
PYTHON_VERSION: "3.12"
BUN_VERSION: "canary"
NODE_VERSION: "24"
HL_DIR: packages/benchmarks/HyperliquidBench
jobs:
live-bench:
name: live-bench
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 120
env:
HL_PRIVATE_KEY: ${{ secrets.HL_PRIVATE_KEY }}
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
steps:
# ------------------------------------------------------------------
# Gate: skip everything (exit 0) when HL_PRIVATE_KEY is unset, so this
# workflow never fails CI when the secret is not configured. Mirrors the
# `can_publish` guard in snap-publish.yml and the live-bench skip in
# lifeops-bench-python.yml.
# ------------------------------------------------------------------
- name: Check HL_PRIVATE_KEY secret
id: gate
run: |
set -euo pipefail
if [ -z "${HL_PRIVATE_KEY:-}" ]; then
echo "::notice::HyperliquidBench live run skipped — HL_PRIVATE_KEY is not configured. Add it as a repo secret to enable publishable live runs."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: recursive
fetch-depth: 1
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup Node.js
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
if: steps.gate.outputs.skip != 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
- name: Install Rust toolchain
if: steps.gate.outputs.skip != 'true'
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8
with:
toolchain: stable
- name: Cache cargo registry
if: steps.gate.outputs.skip != 'true'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: |
~/.cargo/registry
~/.cargo/git
packages/benchmarks/HyperliquidBench/target
key: ${{ runner.os }}-cargo-hyperliquid-${{ hashFiles('packages/benchmarks/HyperliquidBench/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-hyperliquid-
- name: Build HyperliquidBench Rust crates
if: steps.gate.outputs.skip != 'true'
working-directory: ${{ env.HL_DIR }}
run: cargo build --release -p hl-runner -p hl-evaluator
# ------------------------------------------------------------------
# The orchestrator package is `benchmarks` under `packages/`, so the
# module is invoked as `python -m benchmarks.orchestrator` with
# `PYTHONPATH=packages` from the repo root.
# ------------------------------------------------------------------
- name: Validate runtime gates
if: steps.gate.outputs.skip != 'true'
env:
PYTHONUNBUFFERED: "1"
run: |
set -e
PYTHONPATH=packages python -m benchmarks.orchestrator validate-runtime-gates --json
- name: Run HyperliquidBench live harnesses (no demo)
if: steps.gate.outputs.skip != 'true'
id: run
env:
NETWORK: ${{ inputs.network || 'testnet' }}
PROVIDER: ${{ inputs.provider || 'cerebras' }}
MODEL: ${{ inputs.model || 'gemma-4-31b' }}
PYTHONUNBUFFERED: "1"
run: |
set -e
PYTHONPATH=packages python -m benchmarks.orchestrator run \
--benchmarks hyperliquid_bench \
--all-harnesses \
--provider "$PROVIDER" \
--model "$MODEL" \
--extra "{\"no_demo\": true, \"network\": \"$NETWORK\"}" \
--force \
--show-incompatible \
2>&1 | tee hyperliquid-live.log
- name: Validate latest publishability (rejects any demo rows)
if: steps.gate.outputs.skip != 'true'
env:
PYTHONUNBUFFERED: "1"
run: |
set -e
PYTHONPATH=packages python -m benchmarks.orchestrator validate-latest-publishability \
--include-benchmarks hyperliquid_bench
- name: Upload live artifacts
if: always() && steps.gate.outputs.skip != 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: hyperliquid-bench-live-${{ github.run_id }}
path: |
packages/benchmarks/benchmark_results/**
packages/benchmarks/HyperliquidBench/runs/**/hyperliquid_bench-*.json
hyperliquid-live.log
if-no-files-found: warn
retention-days: 90
@@ -0,0 +1,62 @@
name: Integration DoD Gap Issues
on:
schedule:
# Nightly at 03:30 UTC
- cron: "30 3 * * *"
workflow_dispatch:
inputs:
dry_run:
description: "Parse report and log actions without mutating issues"
required: false
type: boolean
default: false
close_resolved_gaps:
description: "Close managed issues that are no longer in the report"
required: false
type: boolean
default: true
permissions:
contents: read
issues: write
concurrency:
group: integration-dod-gap-issues
cancel-in-progress: false
jobs:
sync-gap-issues:
name: Sync gap issues from INTEGRATION_DOD_MAP.md
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Check report
id: report
env:
INTEGRATION_DOD_REPORT: INTEGRATION_DOD_MAP.md
run: |
if [[ -f "$INTEGRATION_DOD_REPORT" ]]; then
echo "exists=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::${INTEGRATION_DOD_REPORT} not found; skipping gap issue sync"
echo "exists=false" >> "$GITHUB_OUTPUT"
fi
- name: Setup Node.js
if: steps.report.outputs.exists == 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24"
check-latest: false
- name: Sync issues
if: steps.report.outputs.exists == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INTEGRATION_DOD_REPORT: INTEGRATION_DOD_MAP.md
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || 'false' }}
CLOSE_RESOLVED_GAPS: ${{ github.event_name == 'workflow_dispatch' && inputs.close_resolved_gaps || 'true' }}
run: node packages/app-core/scripts/sync-dod-gap-issues.mjs
+122
View File
@@ -0,0 +1,122 @@
name: Autodocs Documentation Generator
on:
workflow_dispatch:
inputs:
jsdoc:
description: "Generate code comments (T/F)"
required: true
default: "T"
type: choice
options: [T, F]
readme:
description: "Generate README documentation (T/F)"
required: true
default: "T"
type: choice
options: [T, F]
pull_number:
description: "Pull Request Number (if not provided, scans root_directory) - PR must be merged to develop branch. DONT provide if `README documentation` is T from above"
required: false
type: string
root_directory:
description: "Only scans files in this directory (relative to repository root, e.g., packages/core/src)"
required: true
default: "packages/core/src"
type: string
excluded_directories:
description: "Directories to exclude from scanning (comma-separated, relative to root_directory)"
required: true
default: "node_modules,dist,test"
type: string
reviewers:
description: "Pull Request Reviewers (Must be collaborator on the repository) comma-separated GitHub usernames"
required: true
default: ""
type: string
branch:
description: "Target branch for PR (defaults to develop)"
required: false
default: "develop"
type: string
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
generate-docs:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
# NOTE: This workflow requires packages/autodoc to exist
# Currently disabled until autodoc package is added to the monorepo
permissions:
contents: write
pull-requests: write
env:
GITHUB_ACCESS_TOKEN: ${{ secrets.GH_PAT }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
filter: blob:none
- name: Check if autodoc package exists
id: check-autodoc
run: |
if [ -d "packages/autodoc" ]; then
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "⚠️ packages/autodoc does not exist. Skipping documentation generation."
echo "To enable this workflow, create the autodoc package or update this workflow."
fi
- name: Setup Node.js
if: steps.check-autodoc.outputs.exists == 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Setup Python
if: steps.check-autodoc.outputs.exists == 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: "3.14"
- name: Setup Bun
if: steps.check-autodoc.outputs.exists == 'true'
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Install dependencies
if: steps.check-autodoc.outputs.exists == 'true'
run: bun install
- name: Install autodoc package dependencies
if: steps.check-autodoc.outputs.exists == 'true'
working-directory: packages/autodoc
run: bun install
- name: Build autodoc
if: steps.check-autodoc.outputs.exists == 'true'
working-directory: packages/autodoc
run: bun run build
- name: Run documentation generator
if: steps.check-autodoc.outputs.exists == 'true'
working-directory: packages/autodoc
run: bun run autodoc
env:
INPUT_ROOT_DIRECTORY: ${{ github.event.inputs.root_directory }}
INPUT_PULL_NUMBER: ${{ github.event.inputs.pull_number }}
INPUT_EXCLUDED_DIRECTORIES: ${{ github.event.inputs.excluded_directories }}
INPUT_REVIEWERS: ${{ github.event.inputs.reviewers }}
INPUT_BRANCH: ${{ github.event.inputs.branch }}
INPUT_JSDOC: ${{ github.event.inputs.jsdoc }}
INPUT_README: ${{ github.event.inputs.readme }}
+88
View File
@@ -0,0 +1,88 @@
name: Keyless Harness E2E
# Issue #8801 — zero-cost E2E sweep. Routes the connector / mock-LLM loops
# through the deterministic in-process LLM proxy + recorded fixtures and runs
# the corpus on EVERY pull request with NO API keys and NO secrets.
#
# The substrate lives in packages/test/mocks: `createDeterministicLlmProxyPlugin`
# (helpers/llm-proxy-plugin.ts) registers high-priority deterministic text +
# embedding model handlers, and the payment/connector mocks (scripts/start-mocks.ts)
# serve recorded fixtures. The harness suite (test/mocks/__tests__/**) exercises
# both with zero credentials, so it is required-but-fast and never gated on a
# secret — every provider key below is explicitly blanked to prove the loops
# stay green without one.
#
# The required PR gate in `.github/workflows/test.yml` owns the central mock-LLM
# harness plus the first adoption proofs. This standalone lane owns the remaining
# connector harness proofs so the same command is not run twice on PRs.
on:
# Reusable entry for the scheduled exhaustive develop lane (#12342).
workflow_call:
pull_request:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review]
workflow_dispatch: {}
concurrency:
group: keyless-harness-e2e-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
env:
CI: "true"
BUN_VERSION: "canary"
NODE_VERSION: "24"
NODE_OPTIONS: "--max-old-space-size=4096"
# Keyless guarantee: the deterministic harness must pass with every provider
# credential absent. Blanking them here proves the mock-LLM loops never reach
# a live provider.
ELIZA_LIVE_TEST: "0"
OPENAI_API_KEY: ""
ANTHROPIC_API_KEY: ""
GROQ_API_KEY: ""
OPENROUTER_API_KEY: ""
GOOGLE_GENERATIVE_AI_API_KEY: ""
XAI_API_KEY: ""
NVIDIA_API_KEY: ""
CEREBRAS_API_KEY: ""
ELIZAOS_CLOUD_API_KEY: ""
jobs:
keyless-harness:
name: Deterministic mock-LLM connector loops (no secret)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Ensure generated shared i18n data
run: node packages/app-core/scripts/ensure-shared-i18n-data.mjs
- name: Remaining per-plugin keyless harness adoption proofs
run: |
bun run --cwd plugins/plugin-openai test:harness
bun run --cwd plugins/plugin-groq test:harness
bun run --cwd plugins/plugin-openrouter test:harness
bun run --cwd plugins/plugin-google-genai test:harness
bun run --cwd plugins/plugin-telegram test:harness
+152
View File
@@ -0,0 +1,152 @@
name: Kokoro real-weight smoke (Linux)
# The #9588 loader↔GGUF tensor-name drift gate, automated.
#
# Builds the fused `libelizainference` with Kokoro folded in (via the canonical
# `stage-desktop-fused-lib.mjs`, which sets LLAMA_BUILD_KOKORO=ON), stages the
# PUBLISHED Kokoro bundle GGUF + a real voice pack, and runs
# `plugins/plugin-local-inference/scripts/kokoro-real-smoke.ts` against them with
# `KOKORO_SMOKE_REQUIRE=1`. The smoke loads the model, synthesizes a phrase, and
# asserts non-empty 24 kHz PCM whose amplitude envelope looks like speech (the
# #9620 envelope-cv guard that distinguishes real Kokoro from the #9588 noise
# bug). The loader MUST load the SAME GGUF the download path stages — if the
# tensor names drift again, this job goes RED instead of shipping noise.
#
# Opt-in: it builds native code and downloads a ~167 MB model, so it is NOT in
# the default PR fan-out — only workflow_dispatch or a `kokoro-smoke-*` tag runs
# it. To run locally, see the staging steps in
# `plugins/plugin-local-inference/README.md`.
#
# STATUS (2026-07-02) — the #9588 republish LANDED. elizaos/eliza-1
# `bundles/2b/tts/kokoro/kokoro-82m-v1_0.gguf` (162,546,720 B F16, sha256
# 165acd9d2d9b6c2d71fa5bd52b92a2559be08567f58ed496bade076e3d9cb46c) carries the
# 457 post-#9684 tensor names (incl. the previously-missing
# `kokoro.bert.embd_proj.bias`) and is byte-identical to a from-scratch
# `convert_kokoro_pth_to_gguf.py` run on hexgrad/Kokoro-82M at fork pin
# ba598f562. The old `-Q4_K_M` filename 404s and is no longer referenced here.
# The lane stays manual/tag dispatch ONLY for now: on a desktop-CPU runner the
# smoke loads + synthesizes real speech (envelope-cv ≈1.25, WER 0.13 with an
# ASR bundle + the #10726 raw-text G2P fix) but still exceeds the 700 ms mobile
# TTFA budget by orders of magnitude, so an auto-triggered run would paint
# develop RED on perf, not drift. Once the double-G2P fix (#11238 / #10726) is
# merged and the TTFA budget is made per-platform (or first-chunk streaming
# lands), re-enable the path-scoped `push` auto-trigger (loader / converter /
# smoke / this workflow / the llama.cpp submodule pointer) to gate future drift:
# push:
# branches: ["**"]
# paths:
# - "plugins/plugin-local-inference/native/llama.cpp"
# - "plugins/plugin-local-inference/native/llama.cpp/tools/kokoro/**"
# - "plugins/plugin-local-inference/scripts/kokoro-real-smoke.ts"
# - "plugins/plugin-local-inference/src/services/voice/kokoro/**"
# - "packages/app-core/scripts/stage-desktop-fused-lib.mjs"
# - ".github/workflows/kokoro-real-smoke.yml"
on:
workflow_dispatch: {}
push:
tags:
- "kokoro-smoke-*"
concurrency:
group: kokoro-real-smoke-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
NODE_VERSION: "24"
BUN_VERSION: "canary"
# Published Kokoro bundle — the #9588-fixed F16 republish (2026-07-02).
# The former `kokoro-82m-v1_0-Q4_K_M.gguf` filename 404s on the repo; the
# canonical F16 name below is what discovery falls through to and what the
# download catalog (packages/shared/src/local-inference/catalog.ts) stages.
KOKORO_HF_REPO: "elizaos/eliza-1"
KOKORO_GGUF_PATH: "bundles/2b/tts/kokoro/kokoro-82m-v1_0.gguf"
# Catalog FALLBACK voice (af_bella). af_same (Samantha) is not published yet —
# discovery falls back to af_bella, which is the voice actually staged on HF.
KOKORO_VOICE_PATH: "bundles/2b/tts/kokoro/voices/af_bella.bin"
jobs:
smoke:
name: ubuntu-22.04 fused-cpu
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 120
steps:
- name: Checkout (with llama.cpp submodule)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: recursive
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install Linux build deps
run: |
set -euo pipefail
APT_ARGS=(-o Acquire::ForceIPv4=true -o Acquire::Retries=5 -o Acquire::http::Timeout=30)
for attempt in 1 2 3; do
if sudo apt-get "${APT_ARGS[@]}" update && \
sudo apt-get "${APT_ARGS[@]}" install -y --fix-missing \
cmake ninja-build build-essential ccache git python3 curl libgomp1 \
libespeak-ng-dev espeak-ng-data; then
break
fi
if [ "$attempt" -eq 3 ]; then echo "apt install failed"; exit 1; fi
sleep $((attempt * 10))
done
cmake --version
- name: Install workspace dependencies
run: bun install
- name: Prepare core generated sources
run: bun run --cwd packages/core prebuild
- name: Build + stage the fused libelizainference (CPU, Kokoro ON)
run: |
set -euo pipefail
# The canonical fused desktop build: BUILD_SHARED_LIBS=ON +
# LLAMA_BUILD_OMNIVOICE/MTMD/KOKORO=ON, producing the SHARED
# `elizainference` target plus its dlopen-able ggml sibling set.
node packages/app-core/scripts/stage-desktop-fused-lib.mjs \
--variant cpu --out "$GITHUB_WORKSPACE/fused-lib"
LIB="$GITHUB_WORKSPACE/fused-lib/libelizainference.so"
test -f "$LIB" || { echo "missing $LIB after build"; exit 1; }
# Fail fast with a clear message if the fused lib didn't link Kokoro.
if ! nm -D "$LIB" | grep -q eliza_inference_kokoro; then
echo "fused lib does not export the Kokoro ABI (eliza_inference_kokoro_*)"
exit 1
fi
file "$LIB"
- name: Stage published Kokoro bundle GGUF + voice
run: |
set -euo pipefail
DIR="$GITHUB_WORKSPACE/kokoro-model"
mkdir -p "$DIR/voices"
base="https://huggingface.co/${KOKORO_HF_REPO}/resolve/main"
curl -fSL --retry 5 -o "$DIR/$(basename "$KOKORO_GGUF_PATH")" \
"$base/${KOKORO_GGUF_PATH}?download=true"
curl -fSL --retry 5 -o "$DIR/voices/$(basename "$KOKORO_VOICE_PATH")" \
"$base/${KOKORO_VOICE_PATH}?download=true"
ls -lh "$DIR" "$DIR/voices"
- name: Run real-weight Kokoro smoke (require-staged)
env:
KOKORO_SMOKE_REQUIRE: "1"
ELIZA_INFERENCE_LIB_DIR: ${{ github.workspace }}/fused-lib
LD_LIBRARY_PATH: ${{ github.workspace }}/fused-lib
ELIZA_KOKORO_MODEL_DIR: ${{ github.workspace }}/kokoro-model
run: |
set -euo pipefail
bun plugins/plugin-local-inference/scripts/kokoro-real-smoke.ts
@@ -0,0 +1,65 @@
name: "#8756 Launch Hardening Evidence"
on:
workflow_dispatch:
inputs:
environment:
description: "Environment whose protected deploy secrets should be checked"
required: true
default: production
type: choice
options:
- production
- staging
branch:
description: "Branch to inspect for image/deploy workflow state"
required: true
default: develop
cloud_api_base:
description: "Eliza Cloud API base for the optional fresh-agent probe"
required: true
default: https://api.elizacloud.ai
permissions:
contents: read
actions: read
jobs:
collect:
name: Collect #8756 evidence
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
environment: ${{ github.event.inputs.environment }}
timeout-minutes: 20
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: "canary"
- name: Collect read-only launch-hardening status
env:
GH_TOKEN: ${{ github.token }}
ELIZA_CLOUD_AUTH_TOKEN: ${{ secrets.ELIZACLOUD_API_KEY }}
ELIZA_CLOUD_API_BASE: ${{ github.event.inputs.cloud_api_base }}
LAUNCH_HARDENING_EVIDENCE_DIR: ${{ runner.temp }}/evidence/8756-launch-hardening-status
run: |
set -euo pipefail
mkdir -p "$LAUNCH_HARDENING_EVIDENCE_DIR"
node scripts/cloud/verify-8756-launch-hardening.mjs \
--branch "${{ github.event.inputs.branch }}" \
--environment "${{ github.event.inputs.environment }}" \
--cloud-report "$LAUNCH_HARDENING_EVIDENCE_DIR/cloud-provisioning.json"
- name: Upload #8756 evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: 8756-launch-hardening-status
path: ${{ runner.temp }}/evidence/8756-launch-hardening-status/**
if-no-files-found: error
retention-days: 14
+50
View File
@@ -0,0 +1,50 @@
name: Launch QA
on:
pull_request:
branches: [main]
paths:
- "launchdocs/**"
- "packages/docs/docs/launchdocs/**"
- "packages/docs/launchdocs/**"
- "scripts/launch-qa/**"
- "package.json"
- ".github/workflows/launch-qa.yml"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: launch-qa-${{ github.ref }}
cancel-in-progress: true
env:
BUN_VERSION: "canary"
NODE_VERSION: "24"
NODE_NO_WARNINGS: "1"
jobs:
launch-qa:
name: Launch Docs And Static Gates
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 20
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun workspace
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --frozen-lockfile
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Run launch QA quick gate
run: bun run test:launch-qa
@@ -0,0 +1,97 @@
# LifeOps Action Manifest Drift Gate
#
# Re-runs the manifest generator (TS exporter + Python umbrella augment) and
# fails the build if the committed `actions.manifest.json` differs from a
# fresh regeneration. This keeps `packages/benchmarks/lifeops-bench/manifests/`
# in sync with the actual plugin actions exposed by contacts, personal
# assistant, phone, BlueBubbles, iMessage, and todos.
#
# Local equivalent (run before pushing):
#
# tmp="$(mktemp)"
# node --conditions=eliza-source --conditions=development --import tsx \
# scripts/lifeops-bench/export-action-manifest.ts \
# --out "$tmp" --summary-out none
# diff -u packages/benchmarks/lifeops-bench/manifests/actions.manifest.json "$tmp"
#
# Triggers on PRs that touch any of the input plugins, the export script,
# the Python augment, or the manifest itself.
name: LifeOps Action Manifest Gate
on:
pull_request:
branches: [main]
paths:
- "plugins/plugin-personal-assistant/**"
- "plugins/plugin-contacts/**"
- "plugins/plugin-phone/**"
- "plugins/plugin-bluebubbles/**"
- "plugins/plugin-imessage/**"
- "plugins/plugin-todos/**"
- "scripts/lifeops-bench/export-action-manifest.ts"
- "packages/benchmarks/lifeops-bench/manifests/actions.manifest.json"
- "packages/benchmarks/lifeops-bench/manifests/actions.summary.md"
- "packages/benchmarks/lifeops-bench/eliza_lifeops_bench/manifest_export.py"
- ".github/workflows/lifeops-bench-manifest.yml"
workflow_dispatch:
concurrency:
group: lifeops-bench-manifest-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
PYTHON_VERSION: "3.12"
ELIZA_SKIP_ARTIFACT_SYNC: "1"
jobs:
manifest-gate:
name: Verify lifeops action manifest is up to date
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup Bun + workspace
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --frozen-lockfile --ignore-scripts
install-native-deps: "false"
- name: Install Python bench package
run: |
cd packages/benchmarks/lifeops-bench
pip install -e .
- name: Verify manifest is up to date
run: |
set -e
tmp="$(mktemp)"
summary_tmp="$(mktemp)"
trap 'rm -f "$tmp" "$summary_tmp"' EXIT
node --conditions=eliza-source --conditions=development --import tsx \
scripts/lifeops-bench/export-action-manifest.ts \
--out "$tmp" \
--summary-out "$summary_tmp"
diff -u packages/benchmarks/lifeops-bench/manifests/actions.manifest.json "$tmp"
diff -u packages/benchmarks/lifeops-bench/manifests/actions.summary.md "$summary_tmp"
@@ -0,0 +1,336 @@
# LifeOps Multi-Tier Benchmark
#
# Separate from `.github/workflows/lifeops-bench.yml`. This workflow drives
# `lifeops-bench` CLI and runs LifeOpsBench against the
# four-tier model registry (small / mid / large / frontier).
#
# Default PR cells: `--suite smoke --tiers large,frontier` (Cerebras
# gpt-oss-120b + Anthropic Opus 4.7). Nightly runs the `core` suite over the
# same two tiers.
#
# All cells are skip-not-fail: missing API keys or missing local binaries
# leave a SKIPPED row in the per-job summary and never turn the workflow red.
#
# Local equivalents:
# cd packages/benchmarks/lifeops-bench && pip install -e .
# lifeops-bench --agent cerebras-direct --model-tier large --suite smoke
# lifeops-bench --agent hermes --model-tier frontier --suite smoke
name: LifeOps Benchmark — Multi-Tier
on:
pull_request:
branches: [main]
paths:
- "packages/benchmarks/**"
- "plugins/plugin-training/**"
- "packages/benchmarks/lifeops-bench/**"
- ".github/workflows/lifeops-bench-multi-tier.yml"
workflow_dispatch:
inputs:
suite:
description: "Suite to run (smoke|core|full)"
required: false
type: choice
options: [smoke, core, full]
default: smoke
tiers:
description: "Comma-separated tier list"
required: false
type: string
default: "large,frontier"
schedule:
# Nightly at 03:00 UTC
- cron: "0 3 * * *"
concurrency:
group: lifeops-bench-multi-tier-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
PYTHON_VERSION: "3.12"
jobs:
resolve-inputs:
name: "resolve-inputs"
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 2
outputs:
suite: ${{ steps.pick.outputs.suite }}
tiers: ${{ steps.pick.outputs.tiers }}
steps:
- id: pick
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "suite=${{ github.event.inputs.suite }}" >> "$GITHUB_OUTPUT"
echo "tiers=${{ github.event.inputs.tiers }}" >> "$GITHUB_OUTPUT"
elif [ "${{ github.event_name }}" = "schedule" ]; then
echo "suite=core" >> "$GITHUB_OUTPUT"
echo "tiers=large,frontier" >> "$GITHUB_OUTPUT"
else
echo "suite=smoke" >> "$GITHUB_OUTPUT"
echo "tiers=large,frontier" >> "$GITHUB_OUTPUT"
fi
smoke-large:
name: "tier=large"
needs: resolve-inputs
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
steps:
- name: Check secrets
id: gate
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
run: |
if [ -z "${CEREBRAS_API_KEY:-}" ]; then
echo "[multi-tier] CEREBRAS_API_KEY not configured — skipping large tier."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup workspace dependencies
if: steps.gate.outputs.skip != 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
- name: Install Python bench package
if: steps.gate.outputs.skip != 'true'
run: |
cd packages/benchmarks/lifeops-bench
pip install -e .
- name: Run multi-tier driver (large only)
if: steps.gate.outputs.skip != 'true'
id: run
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
run: |
set -e
RUN_DIR="${{ github.workspace }}/runs/large"
mkdir -p "$RUN_DIR/large"
lifeops-bench \
--agent cerebras-direct \
--model-tier large \
--suite "${{ needs.resolve-inputs.outputs.suite }}" \
--output-dir "$RUN_DIR/large" \
| tee multi-tier-large.log
RESULT_JSON=$(ls -t "$RUN_DIR/large"/lifeops_*.json 2>/dev/null | head -n1 || true)
if [ -n "$RESULT_JSON" ]; then
cp "$RESULT_JSON" "$RUN_DIR/large/report.json"
fi
printf '# LifeOps Multi-Tier Benchmark\n\nTier: large\nSuite: %s\n' "${{ needs.resolve-inputs.outputs.suite }}" > "$RUN_DIR/SUMMARY.md"
echo "run-dir=$RUN_DIR" >> "$GITHUB_OUTPUT"
- name: Upload large-tier artifacts
if: steps.gate.outputs.skip != 'true' && always() && steps.run.outputs.run-dir != ''
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: lifeops-multi-tier-large-${{ github.run_id }}
path: |
${{ steps.run.outputs.run-dir }}/SUMMARY.md
${{ steps.run.outputs.run-dir }}/large/**/report.md
${{ steps.run.outputs.run-dir }}/large/**/report.json
${{ steps.run.outputs.run-dir }}/large/**/steps.csv
${{ steps.run.outputs.run-dir }}/deltas/**
if-no-files-found: warn
retention-days: 30
- name: Skipped notice
if: steps.gate.outputs.skip == 'true'
run: |
echo "::notice::LifeOps multi-tier large skipped — CEREBRAS_API_KEY not configured."
smoke-frontier:
name: "tier=frontier"
needs: resolve-inputs
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
steps:
- name: Check secrets
id: gate
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
if [ -z "${ANTHROPIC_API_KEY:-}" ]; then
echo "[multi-tier] ANTHROPIC_API_KEY not configured — skipping frontier tier."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup workspace dependencies
if: steps.gate.outputs.skip != 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
- name: Install Python bench package
if: steps.gate.outputs.skip != 'true'
run: |
cd packages/benchmarks/lifeops-bench
pip install -e .
- name: Run multi-tier driver (frontier only)
if: steps.gate.outputs.skip != 'true'
id: run
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
set -e
RUN_DIR="${{ github.workspace }}/runs/frontier"
mkdir -p "$RUN_DIR/frontier"
lifeops-bench \
--agent hermes \
--model-tier frontier \
--suite "${{ needs.resolve-inputs.outputs.suite }}" \
--output-dir "$RUN_DIR/frontier" \
| tee multi-tier-frontier.log
RESULT_JSON=$(ls -t "$RUN_DIR/frontier"/lifeops_*.json 2>/dev/null | head -n1 || true)
if [ -n "$RESULT_JSON" ]; then
cp "$RESULT_JSON" "$RUN_DIR/frontier/report.json"
fi
printf '# LifeOps Multi-Tier Benchmark\n\nTier: frontier\nSuite: %s\n' "${{ needs.resolve-inputs.outputs.suite }}" > "$RUN_DIR/SUMMARY.md"
echo "run-dir=$RUN_DIR" >> "$GITHUB_OUTPUT"
- name: Upload frontier-tier artifacts
if: steps.gate.outputs.skip != 'true' && always() && steps.run.outputs.run-dir != ''
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: lifeops-multi-tier-frontier-${{ github.run_id }}
path: |
${{ steps.run.outputs.run-dir }}/SUMMARY.md
${{ steps.run.outputs.run-dir }}/frontier/**/report.md
${{ steps.run.outputs.run-dir }}/frontier/**/report.json
${{ steps.run.outputs.run-dir }}/frontier/**/steps.csv
${{ steps.run.outputs.run-dir }}/deltas/**
if-no-files-found: warn
retention-days: 30
- name: Skipped notice
if: steps.gate.outputs.skip == 'true'
run: |
echo "::notice::LifeOps multi-tier frontier skipped — ANTHROPIC_API_KEY not configured."
summary:
name: "multi-tier summary"
needs: [resolve-inputs, smoke-large, smoke-frontier]
if: always() && github.event_name == 'pull_request'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 5
steps:
- name: Download large-tier artifact
if: needs.smoke-large.result == 'success'
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: lifeops-multi-tier-large-${{ github.run_id }}
path: artifacts/large
- name: Download frontier-tier artifact
if: needs.smoke-frontier.result == 'success'
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c
with:
name: lifeops-multi-tier-frontier-${{ github.run_id }}
path: artifacts/frontier
- name: Compose PR comment
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
env:
SUITE: ${{ needs.resolve-inputs.outputs.suite }}
TIERS: ${{ needs.resolve-inputs.outputs.tiers }}
LARGE_RESULT: ${{ needs.smoke-large.result }}
FRONTIER_RESULT: ${{ needs.smoke-frontier.result }}
with:
script: |
const fs = require('fs');
const path = require('path');
const suite = process.env.SUITE;
const tiers = process.env.TIERS;
const sections = [];
sections.push(`### LifeOps Multi-Tier Benchmark`);
sections.push(``);
sections.push(`Suite: \`${suite}\` — Tiers requested: \`${tiers}\``);
sections.push(``);
const tries = [
{ label: 'large', dir: 'artifacts/large', result: process.env.LARGE_RESULT },
{ label: 'frontier', dir: 'artifacts/frontier', result: process.env.FRONTIER_RESULT },
];
for (const t of tries) {
if (t.result !== 'success') {
sections.push(`#### \`${t.label}\` — ${t.result || 'skipped'}`);
sections.push('');
continue;
}
const summaryPath = path.join(t.dir, 'SUMMARY.md');
if (fs.existsSync(summaryPath)) {
const md = fs.readFileSync(summaryPath, 'utf8');
const head = md.split('\n').slice(0, 80).join('\n');
sections.push(`#### \`${t.label}\``);
sections.push('');
sections.push(head);
sections.push('');
} else {
sections.push(`#### \`${t.label}\` — SUMMARY.md missing`);
sections.push('');
}
}
sections.push(`Artifacts: \`lifeops-multi-tier-large-${{ github.run_id }}\`, \`lifeops-multi-tier-frontier-${{ github.run_id }}\``);
const body = sections.join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
+395
View File
@@ -0,0 +1,395 @@
# LifeOpsBench (Python) — unit + smoke + opt-in live runs
#
# Wires the Python LifeOpsBench package
# (`packages/benchmarks/lifeops-bench/`) into CI. Three jobs:
#
# unit-tests — required. pytest against the in-process bench (no LLM,
# no API keys). Currently 600 pass, 3 skipped (live-gated).
# smoke-bench — required. Runs the CLI against the in-process executor
# with the `wrong` reference oracle on a small set, with
# cost cap and per-scenario timeout, NO API keys. Proves
# the CLI + runner + scorer pipeline works end-to-end.
# (`perfect` is exercised inside `unit-tests` via the
# adapter-conformance + executor-conformance tests, which
# construct PerfectAgent per-scenario the way the CLI
# itself does not yet.)
# live-bench — opt-in via workflow_dispatch + mode=full. Requires
# CEREBRAS_API_KEY + ANTHROPIC_API_KEY; skipped with a
# notice when either is absent. Never required for merge.
#
# DO NOT add live-bench to required PR checks. The unit-tests + smoke-bench
# jobs are the only required gates.
#
# Existing TS-side gate (`.github/workflows/lifeops-bench.yml`) is
# orthogonal and must continue to work — do not touch it.
name: LifeOpsBench (Python)
on:
pull_request:
branches: [main]
paths:
- "packages/benchmarks/lifeops-bench/**"
- "packages/benchmarks/eliza-adapter/eliza_adapter/lifeops_bench.py"
- "packages/benchmarks/eliza-adapter/eliza_adapter/client.py"
- "packages/lifeops-bench/src/lifeops-bench-handler.ts"
- "packages/lifeops-bench/src/lifeops-fake-backend.ts"
- "plugins/plugin-personal-assistant/src/actions/**"
- ".github/workflows/lifeops-bench-python.yml"
workflow_dispatch:
inputs:
mode:
description: "Run mode: smoke (in-process, no API keys) or full (live LLMs)"
required: false
type: choice
default: "smoke"
options:
- smoke
- full
seeds:
description: "Seeds per scenario (pass^k)"
required: false
type: string
default: "1"
agent:
description: "Agent backend for live-bench (only used with mode=full)"
required: false
type: choice
default: "perfect"
options:
- perfect
- wrong
- hermes
- cerebras-direct
- eliza
domain:
description: "Domain filter (`all` = no filter)"
required: false
type: string
default: "all"
concurrency:
group: lifeops-bench-python-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
env:
PYTHON_VERSION: "3.12"
UV_VERSION: "0.5.18"
BENCH_DIR: packages/benchmarks/lifeops-bench
jobs:
# ---------------------------------------------------------------------
# 1) Unit tests — REQUIRED for merge.
# ---------------------------------------------------------------------
unit-tests:
name: unit-tests
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Cache uv package cache
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.cache/uv
key: uv-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-${{ hashFiles('packages/benchmarks/lifeops-bench/pyproject.toml') }}
restore-keys: |
uv-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-
- name: Install uv
run: |
set -e
python -m pip install --upgrade pip
python -m pip install "uv==${UV_VERSION}"
uv --version
- name: uv sync (lifeops-bench)
working-directory: ${{ env.BENCH_DIR }}
run: uv sync --extra test --extra anthropic
- name: Build deterministic LifeWorld snapshots
working-directory: ${{ env.BENCH_DIR }}
run: uv run python -m eliza_lifeops_bench.lifeworld.snapshots --rebuild
- name: Run pytest
id: pytest
working-directory: ${{ env.BENCH_DIR }}
env:
# Hermetic — no provider creds available to unit tests.
CEREBRAS_API_KEY: ""
ANTHROPIC_API_KEY: ""
PYTHONUNBUFFERED: "1"
run: |
set -e
uv run pytest tests/ -v --tb=short
- name: Upload pytest cache on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: pytest-cache-${{ github.run_id }}
path: |
${{ env.BENCH_DIR }}/.pytest_cache/
if-no-files-found: ignore
retention-days: 7
# ---------------------------------------------------------------------
# 2) Smoke bench — REQUIRED for merge. NO API keys, in-process executor.
# ---------------------------------------------------------------------
smoke-bench:
name: smoke-bench
needs: unit-tests
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Cache uv package cache
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.cache/uv
key: uv-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-${{ hashFiles('packages/benchmarks/lifeops-bench/pyproject.toml') }}
restore-keys: |
uv-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-
- name: Install uv
run: |
set -e
python -m pip install --upgrade pip
python -m pip install "uv==${UV_VERSION}"
- name: uv sync (lifeops-bench)
working-directory: ${{ env.BENCH_DIR }}
run: uv sync --extra test --extra anthropic
- name: Build deterministic LifeWorld snapshots
working-directory: ${{ env.BENCH_DIR }}
run: uv run python -m eliza_lifeops_bench.lifeworld.snapshots --rebuild
- name: Run smoke bench (in-process, no API keys)
id: smoke
working-directory: ${{ env.BENCH_DIR }}
env:
# Hermetic: explicitly clear any inherited keys so the run cannot
# accidentally hit a real provider.
CEREBRAS_API_KEY: ""
ANTHROPIC_API_KEY: ""
OPENAI_API_KEY: ""
PYTHONUNBUFFERED: "1"
run: |
set -e
mkdir -p lifeops_bench_results
# The `wrong` reference oracle runs entirely in-process and
# exercises the runner / scorer / executor pipeline without any
# LLM. PerfectAgent's CLI entry point currently requires a
# per-scenario constructor (see __main__._build_agent_fn) and is
# covered instead by the conformance test in unit-tests above.
uv run python -m eliza_lifeops_bench \
--agent wrong \
--mode static \
--seeds 1 \
--concurrency 4 \
--max-cost-usd 0.01 \
--per-scenario-timeout-s 30 \
--output-dir lifeops_bench_results \
2>&1 | tee smoke.log
ls -la lifeops_bench_results/
- name: Upload smoke artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: lifeops-smoke-${{ github.run_id }}
path: |
${{ env.BENCH_DIR }}/lifeops_bench_results/*.json
${{ env.BENCH_DIR }}/lifeops_bench_results/report.md
${{ env.BENCH_DIR }}/smoke.log
if-no-files-found: warn
retention-days: 30
- name: Annotate PR with smoke summary
if: always() && github.event_name == 'pull_request'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
env:
BENCH_DIR: ${{ env.BENCH_DIR }}
with:
script: |
const fs = require('fs');
const path = require('path');
const dir = path.join(process.env.BENCH_DIR, 'lifeops_bench_results');
if (!fs.existsSync(dir)) {
core.warning(`results dir not found: ${dir}`);
return;
}
const files = fs.readdirSync(dir)
.filter(f => f.startsWith('lifeops_') && f.endsWith('.json'))
.map(f => ({ f, mtime: fs.statSync(path.join(dir, f)).mtimeMs }))
.sort((a, b) => b.mtime - a.mtime);
if (files.length === 0) {
core.warning('no lifeops_*.json result files produced');
return;
}
const latest = path.join(dir, files[0].f);
const data = JSON.parse(fs.readFileSync(latest, 'utf8'));
const fmt = (v, d = 3) =>
typeof v === 'number' ? v.toFixed(d) : (v ?? 'n/a');
const passAt1 = data.pass_at_1;
const passAtK = data.pass_at_k;
const agentCost = data.agent_cost_usd;
const evalCost = data.eval_cost_usd;
const totalCost = data.total_cost_usd;
const totalLatency = data.total_latency_ms;
const scenarios = Array.isArray(data.scenarios) ? data.scenarios : [];
const skipped = scenarios.filter(s =>
s && (s.terminated_reason === 'cost_exceeded' || s.terminated_reason === 'timeout')
).length;
const lines = [
`### LifeOpsBench (Python) — smoke`,
``,
`Run ID: \`${{ github.run_id }}\``,
`Result file: \`${path.basename(latest)}\``,
``,
`| metric | value |`,
`| --- | ---: |`,
`| pass@1 | ${fmt(passAt1)} |`,
`| pass@k | ${fmt(passAtK)} |`,
`| agent_cost_usd | $${fmt(agentCost, 4)} |`,
`| eval_cost_usd | $${fmt(evalCost, 4)} |`,
`| total_cost_usd | $${fmt(totalCost, 4)} |`,
`| total_latency_ms | ${fmt(totalLatency, 0)} |`,
`| scenarios_run | ${scenarios.length} |`,
`| scenarios_skipped (cost / timeout) | ${skipped} |`,
``,
`Full artifacts: \`lifeops-smoke-${{ github.run_id }}\` upload on this run.`,
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: lines,
});
# ---------------------------------------------------------------------
# 3) Live bench — opt-in via workflow_dispatch + mode=full. NEVER
# required for merge. Skipped (not failed) when secrets are missing.
# ---------------------------------------------------------------------
live-bench:
name: live-bench
needs: unit-tests
if: github.event_name == 'workflow_dispatch' && inputs.mode == 'full'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
steps:
- name: Check live-bench secrets
id: gate
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
if [ -z "${CEREBRAS_API_KEY:-}" ] || [ -z "${ANTHROPIC_API_KEY:-}" ]; then
echo "::notice::LifeOpsBench live-bench skipped — CEREBRAS_API_KEY and/or ANTHROPIC_API_KEY are not configured."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Cache uv package cache
if: steps.gate.outputs.skip != 'true'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.cache/uv
key: uv-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-${{ hashFiles('packages/benchmarks/lifeops-bench/pyproject.toml') }}
restore-keys: |
uv-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-
- name: Install uv
if: steps.gate.outputs.skip != 'true'
run: |
set -e
python -m pip install --upgrade pip
python -m pip install "uv==${UV_VERSION}"
- name: uv sync (lifeops-bench)
if: steps.gate.outputs.skip != 'true'
working-directory: ${{ env.BENCH_DIR }}
run: uv sync --extra test --extra anthropic
- name: Build deterministic LifeWorld snapshots
if: steps.gate.outputs.skip != 'true'
working-directory: ${{ env.BENCH_DIR }}
run: uv run python -m eliza_lifeops_bench.lifeworld.snapshots --rebuild
- name: Run live bench
if: steps.gate.outputs.skip != 'true'
id: live
working-directory: ${{ env.BENCH_DIR }}
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
PYTHONUNBUFFERED: "1"
run: |
set -e
mkdir -p lifeops_bench_results
AGENT="${{ inputs.agent || 'perfect' }}"
DOMAIN="${{ inputs.domain || 'all' }}"
SEEDS="${{ inputs.seeds || '1' }}"
DOMAIN_FLAG=()
if [ -n "$DOMAIN" ] && [ "$DOMAIN" != "all" ]; then
DOMAIN_FLAG=(--domain "$DOMAIN")
fi
uv run python -m eliza_lifeops_bench \
--agent "$AGENT" \
--seeds "$SEEDS" \
--concurrency 4 \
--max-cost-usd 25.00 \
--per-scenario-timeout-s 300 \
--output-dir lifeops_bench_results \
"${DOMAIN_FLAG[@]}" \
2>&1 | tee live.log
- name: Upload live artifacts
if: always() && steps.gate.outputs.skip != 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: lifeops-live-${{ github.run_id }}
path: |
${{ env.BENCH_DIR }}/lifeops_bench_results/*.json
${{ env.BENCH_DIR }}/lifeops_bench_results/report.md
${{ env.BENCH_DIR }}/live.log
if-no-files-found: warn
retention-days: 90
+288
View File
@@ -0,0 +1,288 @@
# LifeOps Multi-Agent Benchmark Gate
#
# Runs the `lifeops-bench` runner per agent (eliza, hermes, openclaw) on
# every PR that touches the lifeops surface area. Cerebras gpt-oss-120b is
# used by the model-backed agents.
#
# Local equivalent (run before pushing):
#
# cd packages/benchmarks/lifeops-bench && pip install -e .
# lifeops-bench --agent eliza --suite full --limit 25
#
# The per-job matrix uploads the per-agent run directory from
# `~/.eliza/runs/lifeops/lifeops-multiagent-<ts>/<agent>/` (plus the
# top-level `report.md` / `report.json`) as a 30-day artifact, and posts a
# side-by-side summary as a PR comment after all matrix legs complete.
#
# Required secrets (job is skipped, NOT failed, when missing):
# CEREBRAS_API_KEY — eval/judge/teacher provider (gpt-oss-120b)
name: LifeOps Benchmark
on:
pull_request:
branches: [main]
paths:
- "plugins/plugin-personal-assistant/**"
- "plugins/plugin-training/**"
- "packages/scenario-runner/**"
- "packages/benchmarks/lifeops-bench/**"
- "packages/core/src/runtime/**"
- "test/scenarios/lifeops.**"
- "scripts/lifeops-*.mjs"
- ".github/workflows/lifeops-bench.yml"
workflow_dispatch:
inputs:
agent:
description: "Which agent to run (default: all)"
required: false
type: choice
options: [all, eliza, hermes, openclaw, cerebras-direct]
default: all
limit:
description: "Scenarios per agent (default: 25)"
required: false
type: string
default: "25"
concurrency:
group: lifeops-bench-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
PYTHON_VERSION: "3.12"
jobs:
lifeops-bench:
name: "lifeops-bench: ${{ matrix.agent }}"
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
strategy:
fail-fast: false
max-parallel: 1
matrix:
agent: ["eliza", "hermes", "openclaw"]
steps:
- name: Check secrets
id: gate
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
run: |
if [ -z "${CEREBRAS_API_KEY:-}" ]; then
echo "[lifeops-bench] CEREBRAS_API_KEY not configured — skipping benchmark."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
if: steps.gate.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Python
if: steps.gate.outputs.skip != 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Setup workspace dependencies
if: steps.gate.outputs.skip != 'true'
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
- name: Install Python bench package
if: steps.gate.outputs.skip != 'true'
run: |
cd packages/benchmarks/lifeops-bench
pip install -e .
- name: Type/build check (lifeops + training + scenario-runner + core)
if: steps.gate.outputs.skip != 'true'
run: |
set -e
bun run --cwd packages/core typecheck
bun run --cwd packages/scenario-runner build
bun run --cwd plugins/plugin-personal-assistant build:types
bun run --cwd plugins/plugin-training build:types
- name: Verify Cerebras wiring
if: steps.gate.outputs.skip != 'true'
id: cerebras
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
run: |
set +e
bun run lifeops:verify-cerebras 2>&1 | tee verify-cerebras.log
status=${PIPESTATUS[0]}
set -e
if [ "$status" -eq 0 ]; then
echo "provider-throttled=false" >> "$GITHUB_OUTPUT"
exit 0
fi
if grep -Eq 'cerebras error 429|too_many_requests_error|too_many_tokens_error|queue_exceeded|request_quota_exceeded|token_quota_exceeded|Tokens per day limit exceeded' verify-cerebras.log; then
echo "::notice::LifeOps benchmark skipped — Cerebras is currently rate limited or queue saturated."
echo "provider-throttled=true" >> "$GITHUB_OUTPUT"
exit 0
fi
exit "$status"
- name: Check OpenClaw CLI
if: steps.gate.outputs.skip != 'true' && steps.cerebras.outputs.provider-throttled != 'true' && matrix.agent == 'openclaw'
id: openclaw
run: |
set -euo pipefail
OPENCLAW_BIN="${OPENCLAW_BIN:-$HOME/.eliza/agents/openclaw/v2026.5.7/node_modules/.bin/openclaw}"
if [ ! -x "$OPENCLAW_BIN" ]; then
echo "::notice::OpenClaw CLI not installed at $OPENCLAW_BIN; skipping optional openclaw benchmark leg."
echo "skip=true" >> "$GITHUB_OUTPUT"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
fi
- name: Run lifeops multi-agent bench (matrix leg)
if: steps.gate.outputs.skip != 'true' && steps.cerebras.outputs.provider-throttled != 'true' && steps.openclaw.outputs.skip != 'true'
id: run
env:
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
ELIZA_BENCH_AGENT: ${{ matrix.agent }}
ELIZA_BENCH_LIMIT: ${{ inputs.limit || '25' }}
run: |
set -e
RUN_DIR="${GITHUB_WORKSPACE}/runs/lifeops-${ELIZA_BENCH_AGENT}-${GITHUB_RUN_ID}"
mkdir -p "$RUN_DIR/${ELIZA_BENCH_AGENT}"
set +e
timeout 1200s lifeops-bench \
--agent "$ELIZA_BENCH_AGENT" \
--suite full \
--limit "$ELIZA_BENCH_LIMIT" \
--seeds 1 \
--concurrency 2 \
--max-cost-usd 10.00 \
--per-scenario-timeout-s 300 \
--output-dir "$RUN_DIR/${ELIZA_BENCH_AGENT}" \
2>&1 | tee lifeops-full.log
status=${PIPESTATUS[0]}
set -e
if [ "$status" -ne 0 ]; then
if grep -Eq 'cerebras error 429|too_many_requests_error|too_many_tokens_error|queue_exceeded|request_quota_exceeded|token_quota_exceeded|Tokens per day limit exceeded' lifeops-full.log; then
echo "::notice::LifeOps benchmark skipped — Cerebras quota or rate limit was reached during benchmark execution."
echo "provider-throttled=true" >> "$GITHUB_OUTPUT"
exit 0
fi
exit "$status"
fi
echo "provider-throttled=false" >> "$GITHUB_OUTPUT"
echo "RUN_DIR=$RUN_DIR" | tee -a lifeops-full.log
RESULT_JSON=$(ls -t "$RUN_DIR/${ELIZA_BENCH_AGENT}"/lifeops_*.json 2>/dev/null | head -n1 || true)
if [ -z "$RESULT_JSON" ] || [ ! -f "$RESULT_JSON" ]; then
echo "[lifeops-bench] Could not resolve result JSON in $RUN_DIR." >&2
exit 1
fi
cp "$RESULT_JSON" "$RUN_DIR/report.json"
python - "$RUN_DIR/report.json" "$RUN_DIR/report.md" <<'PY'
import json
import sys
from pathlib import Path
report = json.loads(Path(sys.argv[1]).read_text())
scenarios = report.get("scenarios", [])
lines = [
"# LifeOps Benchmark",
"",
f"Model: {report.get('model_name', 'unknown')}",
f"Judge: {report.get('judge_model_name', 'unknown')}",
f"Scenarios: {len(scenarios)}",
f"pass@1: {report.get('pass_at_1', 0):.3f}",
f"pass@k: {report.get('pass_at_k', 0):.3f}",
f"Total cost: ${report.get('total_cost_usd', 0):.4f}",
]
Path(sys.argv[2]).write_text("\n".join(lines) + "\n")
PY
echo "run-dir=$RUN_DIR" >> "$GITHUB_OUTPUT"
echo "run-id=$(basename "$RUN_DIR")" >> "$GITHUB_OUTPUT"
- name: Upload run artifacts
if: steps.gate.outputs.skip != 'true' && steps.cerebras.outputs.provider-throttled != 'true' && steps.run.outputs.provider-throttled != 'true' && always() && steps.run.outputs.run-dir != ''
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: lifeops-run-${{ matrix.agent }}-${{ github.run_id }}
path: |
${{ steps.run.outputs.run-dir }}/report.md
${{ steps.run.outputs.run-dir }}/report.json
${{ steps.run.outputs.run-dir }}/${{ matrix.agent }}/**
${{ steps.run.outputs.run-dir }}/mockoon/**
${{ steps.run.outputs.run-dir }}/js-scenarios/**
if-no-files-found: warn
retention-days: 30
- name: Annotate PR with side-by-side summary
if: steps.gate.outputs.skip != 'true' && steps.cerebras.outputs.provider-throttled != 'true' && steps.run.outputs.provider-throttled != 'true' && github.event_name == 'pull_request' && steps.run.outputs.run-dir != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
env:
RUN_DIR: ${{ steps.run.outputs.run-dir }}
RUN_ID: ${{ steps.run.outputs.run-id }}
AGENT: ${{ matrix.agent }}
with:
script: |
const fs = require('fs');
const path = require('path');
const runDir = process.env.RUN_DIR;
const runId = process.env.RUN_ID;
const agent = process.env.AGENT;
const reportPath = path.join(runDir, 'report.md');
if (!fs.existsSync(reportPath)) {
core.warning(`report.md not found at ${reportPath}`);
return;
}
const md = fs.readFileSync(reportPath, 'utf8');
const head = md.split('\n').slice(0, 80).join('\n');
const body = [
`### LifeOps Benchmark — \`${agent}\``,
``,
`Run ID: \`${runId}\``,
``,
head,
``,
`Full artifacts: see the \`lifeops-run-${agent}-${{ github.run_id }}\` upload on this run.`,
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
- name: Skipped notice
if: steps.gate.outputs.skip == 'true'
run: |
echo "::notice::LifeOps benchmark skipped — CEREBRAS_API_KEY is not configured for this repo."
- name: Cerebras throttled notice
if: steps.gate.outputs.skip != 'true' && (steps.cerebras.outputs.provider-throttled == 'true' || steps.run.outputs.provider-throttled == 'true')
run: |
echo "::notice::LifeOps benchmark skipped — Cerebras is currently rate limited or queue saturated."
- name: OpenClaw skipped notice
if: steps.gate.outputs.skip != 'true' && steps.cerebras.outputs.provider-throttled != 'true' && matrix.agent == 'openclaw' && steps.openclaw.outputs.skip == 'true'
run: |
echo "::notice::LifeOps openclaw benchmark skipped — OpenClaw CLI is not installed in this runner."
@@ -0,0 +1,112 @@
name: "#11632 LifeOps Live Validation Status"
on:
workflow_dispatch:
inputs:
run_keyless_matrix:
description: "Run the deterministic OWNER/AGENT matrix harness"
type: boolean
default: true
run_live_connectors:
description: "Run live connector suites when credentials are configured"
type: boolean
default: false
permissions:
contents: read
jobs:
collect:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
env:
ELIZA_SKIP_ARTIFACT_SYNC: "1"
ELIZA_LIVE_TEST: "1"
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
GOOGLE_CLIENT_ID: ${{ secrets.GOOGLE_CLIENT_ID }}
GOOGLE_CLIENT_SECRET: ${{ secrets.GOOGLE_CLIENT_SECRET }}
GOOGLE_REDIRECT_URI: ${{ secrets.GOOGLE_REDIRECT_URI }}
DISCORD_API_TOKEN: ${{ secrets.DISCORD_API_TOKEN }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TELEGRAM_TEST_CHAT_ID: ${{ secrets.TELEGRAM_TEST_CHAT_ID }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
SLACK_APP_TOKEN: ${{ secrets.SLACK_APP_TOKEN }}
SLACK_USER_TOKEN: ${{ secrets.SLACK_USER_TOKEN }}
SIGNAL_ACCOUNT_NUMBER: ${{ secrets.SIGNAL_ACCOUNT_NUMBER }}
SIGNAL_HTTP_URL: ${{ secrets.SIGNAL_HTTP_URL }}
ELIZA_WHATSAPP_ACCESS_TOKEN: ${{ secrets.ELIZA_WHATSAPP_ACCESS_TOKEN }}
ELIZA_WHATSAPP_PHONE_NUMBER_ID: ${{ secrets.ELIZA_WHATSAPP_PHONE_NUMBER_ID }}
X_API_KEY: ${{ secrets.X_API_KEY }}
TWITTER_API_KEY: ${{ secrets.TWITTER_API_KEY }}
TWITTER_BEARER_TOKEN: ${{ secrets.TWITTER_BEARER_TOKEN }}
TWILIO_ACCOUNT_SID: ${{ secrets.TWILIO_ACCOUNT_SID }}
TWILIO_AUTH_TOKEN: ${{ secrets.TWILIO_AUTH_TOKEN }}
TWILIO_PHONE_NUMBER: ${{ secrets.TWILIO_PHONE_NUMBER }}
FITBIT_ACCESS_TOKEN: ${{ secrets.FITBIT_ACCESS_TOKEN }}
OURA_ACCESS_TOKEN: ${{ secrets.OURA_ACCESS_TOKEN }}
STRAVA_ACCESS_TOKEN: ${{ secrets.STRAVA_ACCESS_TOKEN }}
WITHINGS_ACCESS_TOKEN: ${{ secrets.WITHINGS_ACCESS_TOKEN }}
PLAID_CLIENT_ID: ${{ secrets.PLAID_CLIENT_ID }}
PLAID_SECRET: ${{ secrets.PLAID_SECRET }}
PAYPAL_CLIENT_ID: ${{ secrets.PAYPAL_CLIENT_ID }}
PAYPAL_CLIENT_SECRET: ${{ secrets.PAYPAL_CLIENT_SECRET }}
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install
run: bun run install:light
- name: Collect pre-run status
run: node scripts/lifeops/collect-11632-live-validation-status.mjs
- name: Run OWNER/AGENT matrix
if: ${{ inputs.run_keyless_matrix }}
env:
LIFEOPS_PERMISSION_MATRIX: "1"
LIFEOPS_EVIDENCE_DIR: ${{ runner.temp }}/evidence/8833-lifeops-live-validation/11632-status
run: |
set -o pipefail
mkdir -p "$LIFEOPS_EVIDENCE_DIR"
bunx vitest run \
--config packages/test/vitest/integration.config.ts \
plugins/plugin-personal-assistant/test/owner-agent-permission-matrix.integration.test.ts \
2>&1 | tee "$LIFEOPS_EVIDENCE_DIR/owner-agent-permission-matrix.txt"
- name: Run Google live connector suite
if: ${{ inputs.run_live_connectors }}
env:
TEST_LANE: post-merge
LIFEOPS_EVIDENCE_DIR: ${{ runner.temp }}/evidence/8833-lifeops-live-validation/11632-status
run: |
set -o pipefail
mkdir -p "$LIFEOPS_EVIDENCE_DIR"
bun run --cwd plugins/plugin-google test \
2>&1 | tee "$LIFEOPS_EVIDENCE_DIR/plugin-google-live.txt"
- name: Run X live connector suite
if: ${{ inputs.run_live_connectors }}
env:
TEST_LANE: post-merge
LIFEOPS_EVIDENCE_DIR: ${{ runner.temp }}/evidence/8833-lifeops-live-validation/11632-status
run: |
set -o pipefail
mkdir -p "$LIFEOPS_EVIDENCE_DIR"
bun run --cwd plugins/plugin-x test \
2>&1 | tee "$LIFEOPS_EVIDENCE_DIR/plugin-x-live.txt"
- name: Collect post-run status
if: always()
run: node scripts/lifeops/collect-11632-live-validation-status.mjs
- name: Upload status artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: lifeops-11632-live-validation-status
path: ${{ runner.temp }}/evidence/8833-lifeops-live-validation/11632-status/
@@ -0,0 +1,77 @@
# LifeOps prompt benchmark harness (#8795)
#
# PR runs execute the hermetic catalog-shape test so the TS benchmark runner is
# not orphaned. Manual dispatch can enable the live LLM benchmark by setting
# live=true and providing the usual provider secret(s).
name: LifeOps Prompt Benchmark
on:
pull_request:
branches: [main]
paths:
- "plugins/plugin-personal-assistant/test/helpers/lifeops-prompt-benchmark-*.ts"
- "plugins/plugin-personal-assistant/test/lifeops-prompt-benchmark.test.ts"
- "plugins/plugin-personal-assistant/src/actions/**"
- "plugins/plugin-personal-assistant/src/lifeops/**"
- "plugins/plugin-calendar/src/actions/**"
- "plugins/plugin-training/src/**"
- ".github/workflows/lifeops-prompt-benchmark.yml"
workflow_dispatch:
inputs:
live:
description: "Run the live LLM benchmark slice"
required: false
type: boolean
default: false
case_limit:
description: "Live direct-case limit"
required: false
type: string
default: "15"
concurrency:
group: lifeops-prompt-benchmark-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
jobs:
prompt-benchmark:
name: prompt-benchmark
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
fetch-depth: 1
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
- name: Run LifeOps prompt benchmark test
env:
LIFEOPS_PROMPT_BENCHMARK_LIVE: ${{ github.event_name == 'workflow_dispatch' && inputs.live == true && '1' || '0' }}
LIFEOPS_PROMPT_BENCHMARK_CASE_LIMIT: ${{ inputs.case_limit || '15' }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
LOCAL_LLAMA_CPP_API_KEY: ${{ secrets.LOCAL_LLAMA_CPP_API_KEY }}
run: |
bun run --cwd plugins/plugin-personal-assistant test -- test/lifeops-prompt-benchmark.test.ts
@@ -0,0 +1,97 @@
name: lifeops-quality-bench
# Recorded-baseline LifeOps quality benchmarks + regression gate (#10723).
#
# Two keyless, deterministic lanes over REAL LifeOps code paths:
#
# triage — the plugin-inbox triage classifier's prompt-build → parse →
# validate → fail-closed path driven over a committed labeled
# 56-item corpus with a committed fixed-quality mock model
# (fixtures). Precision/recall/accuracy/macro-F1 gate against
# budgets.json floors and must reproduce baseline.json exactly.
# timeliness — the plugin-personal-assistant scheduled-task tick
# (processDueScheduledTasks) replayed over two committed 4-day
# 2026 DST windows (spring-forward + fall-back) at a 5-minute
# cadence on a real PGlite runtime with an injected clock.
# Missed/duplicate/early/occurrence-mismatch counts must be 0
# and fire-time deviation must stay under budgets.json ceilings.
#
# No model bundle, no credentials — fully reproducible on a hosted runner
# (mirrors recall-bench.yml, #9956). A red gate means a real quality
# regression in triage parsing/validation or scheduler fire behavior.
on:
pull_request:
branches: [main]
paths:
- "packages/benchmarks/lifeops-quality/**"
- "plugins/plugin-inbox/src/inbox/**"
- "plugins/plugin-personal-assistant/src/lifeops/**"
- "plugins/plugin-personal-assistant/test/helpers/**"
- "plugins/plugin-scheduling/src/**"
- "packages/core/src/services/triggerScheduling.ts"
- ".github/workflows/lifeops-quality-bench.yml"
schedule:
# 05:40 UTC — alongside the other nightly benchmark lanes.
- cron: "40 5 * * *"
workflow_dispatch: {}
concurrency:
group: lifeops-quality-bench-${{ github.ref }}
cancel-in-progress: true
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
jobs:
lifeops-quality-gate:
name: triage P/R + reminder timeliness budget gate (keyless)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 45
env:
PGLITE_WASM_MODE: node
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
# The timeliness gate boots the personal-assistant plugin barrel, which
# imports sibling plugin dists (same requirement as the Integration Lane
# in test.yml). plugin-inbox is inside this dependency graph, so the
# triage gate's classifier import is covered too. Turbo replays warm
# runs from cache.
- name: Build personal-assistant dependency graph
run: node packages/scripts/run-turbo.mjs run build --filter='@elizaos/plugin-personal-assistant...'
- name: Unit lane (metrics + oracle + corpus/fixture/baseline invariants)
run: bun run --cwd packages/benchmarks/lifeops-quality test
- name: Triage precision/recall gate (real classifier, committed fixtures)
run: bun run --cwd packages/benchmarks/lifeops-quality bench:triage
- name: Reminder timeliness gate (real scheduler tick, DST windows)
timeout-minutes: 30
run: bun run --cwd packages/benchmarks/lifeops-quality bench:timeliness
- name: Upload measured results
if: always()
uses: actions/upload-artifact@v7
with:
name: lifeops-quality-results
path: packages/benchmarks/lifeops-quality/results/*.json
if-no-files-found: ignore
+375
View File
@@ -0,0 +1,375 @@
# Live Scenario Runner (nightly)
#
# Executes the executive-assistant, connector certification, plugin-health, and
# app-control scenarios against a live LLM runtime with real connector
# credentials and uploads the JSON report. Scheduled runs are enforcing: once the
# runner reaches scenario execution, any scenario failure or aggregate LLM-judge
# score below LIFEOPS_JUDGE_THRESHOLD fails the lane. Manual dispatch can still opt
# into report-only mode while debugging by leaving `enforce_gate` false. Missing
# setup prerequisites fail loudly and still upload a placeholder report artifact.
#
# Required repo secrets (self-documented):
# LLM provider (at least one):
# OPENAI_API_KEY
# OPENROUTER_API_KEY
# ANTHROPIC_API_KEY
# GOOGLE_GENERATIVE_AI_API_KEY or GOOGLE_API_KEY
# GROQ_API_KEY
# Independent LLM judge (judge-independence, #9310):
# CEREBRAS_API_KEY — grades judgeRubric/responseJudge checks on Cerebras
# instead of the model under test. SCENARIO_JUDGE_REQUIRE_INDEPENDENT=1 is
# set below, so runs without this secret FAIL their judged scenarios
# (visible judge-availability gap) rather than silently self-grading.
# Connector credentials (presence gates the relevant scenarios):
# GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET, GOOGLE_OAUTH_REFRESH_TOKEN
# GMAIL_TEST_ACCOUNT_EMAIL, GMAIL_TEST_ACCOUNT_REFRESH_TOKEN
# CALENDLY_API_TOKEN
# TELEGRAM_BOT_TOKEN, TELEGRAM_TEST_CHAT_ID, TELEGRAM_API_ID, TELEGRAM_API_HASH
# DISCORD_BOT_TOKEN, DISCORD_TEST_GUILD_ID, DISCORD_TEST_CHANNEL_ID
# SIGNAL_CLI_URL, SIGNAL_TEST_NUMBER
# IMESSAGE_BRIDGE_URL, IMESSAGE_TEST_HANDLE
# WHATSAPP_TOKEN, WHATSAPP_PHONE_ID, WHATSAPP_TEST_CONTACT
# TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_FROM_NUMBER, TWILIO_TEST_TO_NUMBER
# X_API_KEY, X_API_SECRET, X_ACCESS_TOKEN, X_ACCESS_SECRET, X_TEST_DM_HANDLE
# NOTIFICATION_RELAY_URL, NOTIFICATION_RELAY_TOKEN
# TRAVEL_BOOKING_API_KEY
# Optional:
# LIFEOPS_JUDGE_THRESHOLD (workflow input, default 0.8)
# SCENARIO_FILTER (comma-separated scenario ids, default all)
# SCENARIO_ENFORCE_GATE (schedule: enforced; manual input default false)
# SKIP_REASON (required if SCENARIO_SKIP is set)
#
# 1Password vault: this workflow's plain `*_API_KEY` secrets are sourced from the
# `eliza-e2e` 1Password vault. The vault must exist in 1Password and the
# corresponding GitHub Actions secrets must be configured on the
# `elizaOS/eliza` repo before this workflow can run successfully. See
# scenario-matrix.yml for the structured `ELIZA_E2E_*` connector secrets.
name: Live Scenarios
on:
schedule:
# 08:30 UTC daily — after upstream nightly at 04:00 UTC settles.
- cron: "30 8 * * *"
workflow_dispatch:
inputs:
scenario_filter:
description: "Comma-separated scenario ids (empty = all)"
required: false
type: string
default: ""
judge_threshold:
description: "LLM-judge minimum pass score (0.0 - 1.0)"
required: false
type: string
default: "0.8"
skip_reason:
description: "If you plan to skip any scenario, document why"
required: false
type: string
default: ""
enforce_gate:
description: "Fail the workflow when live scenarios fail"
required: false
type: boolean
default: false
concurrency:
group: live-scenarios-${{ github.ref }}
cancel-in-progress: false
env:
BUN_VERSION: "canary"
NODE_VERSION: "24.15.0"
ELIZA_LIVE_TEST: "1"
permissions:
contents: read
actions: read
jobs:
live-scenarios:
name: Live scenarios (EA + connectors + health + app-control)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 120
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
filter: blob:none
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
- name: Verify scenario-runner CLI present
run: |
test -f packages/scenario-runner/src/cli.ts || \
{ echo "scenario-runner CLI missing"; exit 1; }
- name: Prepare live scenario report placeholder
run: |
mkdir -p artifacts/scenario-runs/live
node - <<'NODE'
const fs = require("node:fs");
const report = {
schema: "eliza_live_scenario_setup_v1",
status: "setup_not_completed",
generatedAt: new Date().toISOString(),
note:
"Placeholder written before build/setup. If this file is uploaded unchanged, the workflow failed before scenario execution and no live model trajectory was produced.",
scenarios: [],
totalCount: 0,
passedCount: 0,
failedCount: 0
};
fs.writeFileSync(
"artifacts/lifeops-scenario-report.json",
`${JSON.stringify(report, null, 2)}\n`,
);
NODE
- name: Build live scenario runtime packages
id: build
# The live runner executes TypeScript sources directly, but several
# workspace packages intentionally export dist/* entry points. Because
# dependency installation ignores postinstall scripts, build only the
# packages that the live scenario runtime imports through those exports.
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
run: |
echo "::group::Build packages/core"
bun run --cwd packages/core build
echo "::endgroup::"
echo "::group::Build packages/shared"
# plugin-agent-skills (and others) consume @elizaos/shared via dist
# exports (RouteRequestContext, ReadJsonBodyOptions, route schemas).
# Must build before any plugin that imports from it.
bun run --cwd packages/shared build
echo "::endgroup::"
echo "::group::Build packages/skills"
# plugin-agent-skills imports @elizaos/skills via its dist exports
# (getSkillsDir, loadSkills, formatSkillsForPrompt). Must build
# before plugin-agent-skills typechecks its declaration emit.
bun run --cwd packages/skills build
echo "::endgroup::"
echo "::group::Build packages/cloud/sdk"
bun run --cwd packages/cloud/sdk build
echo "::endgroup::"
echo "::group::Build plugins/plugin-commands"
# plugin-telegram's command-registration.ts imports @elizaos/plugin-commands
# via its dist exports (ConnectorCommand, getConnectorCommands,
# resolveCommand, …). Because installs run with --ignore-scripts this
# workspace dependency is never built implicitly, so plugin-telegram's
# `tsc` declaration emit fails with TS2307 "Cannot find module
# '@elizaos/plugin-commands'". Build it before the connector loop below.
bun run --cwd plugins/plugin-commands build
echo "::endgroup::"
provider_package=""
if [ -n "${GROQ_API_KEY:-}" ]; then
provider_package="plugins/plugin-groq"
elif [ -n "${OPENAI_API_KEY:-}" ]; then
provider_package="plugins/plugin-openai"
elif [ -n "${ANTHROPIC_API_KEY:-}" ]; then
provider_package="plugins/plugin-anthropic"
elif [ -n "${GOOGLE_GENERATIVE_AI_API_KEY:-}" ] || [ -n "${GOOGLE_API_KEY:-}" ]; then
provider_package="plugins/plugin-google-genai"
elif [ -n "${OPENROUTER_API_KEY:-}" ]; then
provider_package="plugins/plugin-openrouter"
fi
package_dirs=(
plugins/plugin-sql
plugins/plugin-blocker
plugins/plugin-browser
plugins/plugin-native-calendar
plugins/plugin-native-activity-tracker
plugins/plugin-native-mobile-signals
plugins/plugin-native-reminders
plugins/plugin-scheduling
plugins/plugin-calendar
plugins/plugin-finances
plugins/plugin-goals
plugins/plugin-inbox
plugins/plugin-reminders
plugins/plugin-calendly
plugins/plugin-discord
plugins/plugin-elizacloud
plugins/plugin-google
plugins/plugin-phone
plugins/plugin-remote-desktop
plugins/plugin-x
plugins/plugin-local-inference
plugins/plugin-app-control
plugins/plugin-agent-skills
plugins/plugin-health
plugins/plugin-wallet
plugins/plugin-pdf
plugins/plugin-form
plugins/plugin-personal-assistant
plugins/plugin-telegram
plugins/plugin-whatsapp
plugins/plugin-signal
plugins/plugin-imessage
)
if [ -n "$provider_package" ]; then
package_dirs+=("$provider_package")
fi
for package_dir in "${package_dirs[@]}"; do
echo "::group::Build ${package_dir}"
bun run --cwd "$package_dir" build
echo "::endgroup::"
done
- name: Run EA + connector live scenarios
id: run
env:
# The runner and ordinary workspace dependencies stay on checkout
# sources; generated/native and dist-only artifacts come from the
# dependency-closure build above.
NODE_OPTIONS: "--conditions=eliza-source"
# LLM providers
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
# Independent LLM judge (#9310): judge on Cerebras, never the model
# under test. Strict mode fails self-graded scenarios so a missing
# key is a visible judge-availability gap, not silent self-grading.
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
SCENARIO_JUDGE_REQUIRE_INDEPENDENT: "1"
# Google Workspace
GOOGLE_OAUTH_CLIENT_ID: ${{ secrets.GOOGLE_OAUTH_CLIENT_ID }}
GOOGLE_OAUTH_CLIENT_SECRET: ${{ secrets.GOOGLE_OAUTH_CLIENT_SECRET }}
GOOGLE_OAUTH_REFRESH_TOKEN: ${{ secrets.GOOGLE_OAUTH_REFRESH_TOKEN }}
GMAIL_TEST_ACCOUNT_EMAIL: ${{ secrets.GMAIL_TEST_ACCOUNT_EMAIL }}
GMAIL_TEST_ACCOUNT_REFRESH_TOKEN: ${{ secrets.GMAIL_TEST_ACCOUNT_REFRESH_TOKEN }}
# Calendly
CALENDLY_API_TOKEN: ${{ secrets.CALENDLY_API_TOKEN }}
# Messaging
TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TELEGRAM_TEST_CHAT_ID: ${{ secrets.TELEGRAM_TEST_CHAT_ID }}
TELEGRAM_API_ID: ${{ secrets.TELEGRAM_API_ID }}
TELEGRAM_API_HASH: ${{ secrets.TELEGRAM_API_HASH }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
DISCORD_TEST_GUILD_ID: ${{ secrets.DISCORD_TEST_GUILD_ID }}
DISCORD_TEST_CHANNEL_ID: ${{ secrets.DISCORD_TEST_CHANNEL_ID }}
SIGNAL_CLI_URL: ${{ secrets.SIGNAL_CLI_URL }}
SIGNAL_TEST_NUMBER: ${{ secrets.SIGNAL_TEST_NUMBER }}
IMESSAGE_BRIDGE_URL: ${{ secrets.IMESSAGE_BRIDGE_URL }}
IMESSAGE_TEST_HANDLE: ${{ secrets.IMESSAGE_TEST_HANDLE }}
WHATSAPP_TOKEN: ${{ secrets.WHATSAPP_TOKEN }}
WHATSAPP_PHONE_ID: ${{ secrets.WHATSAPP_PHONE_ID }}
WHATSAPP_TEST_CONTACT: ${{ secrets.WHATSAPP_TEST_CONTACT }}
# Twilio SMS/Voice
TWILIO_ACCOUNT_SID: ${{ secrets.TWILIO_ACCOUNT_SID }}
TWILIO_AUTH_TOKEN: ${{ secrets.TWILIO_AUTH_TOKEN }}
TWILIO_FROM_NUMBER: ${{ secrets.TWILIO_FROM_NUMBER }}
TWILIO_TEST_TO_NUMBER: ${{ secrets.TWILIO_TEST_TO_NUMBER }}
# X / DM
X_API_KEY: ${{ secrets.X_API_KEY }}
X_API_SECRET: ${{ secrets.X_API_SECRET }}
X_ACCESS_TOKEN: ${{ secrets.X_ACCESS_TOKEN }}
X_ACCESS_SECRET: ${{ secrets.X_ACCESS_SECRET }}
X_TEST_DM_HANDLE: ${{ secrets.X_TEST_DM_HANDLE }}
# Push notifications + travel
NOTIFICATION_RELAY_URL: ${{ secrets.NOTIFICATION_RELAY_URL }}
NOTIFICATION_RELAY_TOKEN: ${{ secrets.NOTIFICATION_RELAY_TOKEN }}
TRAVEL_BOOKING_API_KEY: ${{ secrets.TRAVEL_BOOKING_API_KEY }}
# Run controls
SCENARIO_FILTER: ${{ inputs.scenario_filter }}
LIFEOPS_JUDGE_THRESHOLD: ${{ inputs.judge_threshold || '0.8' }}
SCENARIO_ENFORCE_GATE: ${{ github.event_name == 'schedule' && '1' || (inputs.enforce_gate && '1' || '0') }}
SKIP_REASON: ${{ inputs.skip_reason }}
REPORT_PATH: artifacts/lifeops-scenario-report.json
RUN_DIR: artifacts/scenario-runs/live
run: node packages/scripts/run-live-scenarios.mjs
- name: Run plugin-health live scenarios
id: run-health
env:
NODE_OPTIONS: "--conditions=eliza-source"
# LLM providers
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
# Independent LLM judge (#9310): judge on Cerebras, never the model
# under test.
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
SCENARIO_JUDGE_REQUIRE_INDEPENDENT: "1"
# Run controls
SCENARIO_ROOT: plugins/plugin-health/test/scenarios
SCENARIO_FILTER: ${{ inputs.scenario_filter }}
LIFEOPS_JUDGE_THRESHOLD: ${{ inputs.judge_threshold || '0.8' }}
SCENARIO_ENFORCE_GATE: ${{ github.event_name == 'schedule' && '1' || (inputs.enforce_gate && '1' || '0') }}
SKIP_REASON: ${{ inputs.skip_reason }}
REPORT_PATH: artifacts/plugin-health-scenario-report.json
RUN_DIR: artifacts/scenario-runs/plugin-health
EXPORT_NATIVE_PATH: artifacts/scenario-runs/plugin-health/native.jsonl
run: node packages/scripts/run-live-scenarios.mjs --lane live-only
- name: Run app-control live scenarios
id: run-app-control
if: ${{ !cancelled() && steps.build.outcome == 'success' }}
env:
NODE_OPTIONS: "--conditions=eliza-source"
# LLM providers
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
# Independent LLM judge (#9310): judge on Cerebras, never the model
# under test.
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
SCENARIO_JUDGE_REQUIRE_INDEPENDENT: "1"
# Run controls
SCENARIO_ROOT: plugins/plugin-app-control/test/scenarios
SCENARIO_FILTER: ${{ inputs.scenario_filter }}
LIFEOPS_JUDGE_THRESHOLD: ${{ inputs.judge_threshold || '0.8' }}
SCENARIO_ENFORCE_GATE: ${{ github.event_name == 'schedule' && '1' || (inputs.enforce_gate && '1' || '0') }}
SKIP_REASON: ${{ inputs.skip_reason }}
REPORT_PATH: artifacts/app-control-scenario-report.json
RUN_DIR: artifacts/scenario-runs/app-control
EXPORT_NATIVE_PATH: artifacts/scenario-runs/app-control/native.jsonl
run: node packages/scripts/run-live-scenarios.mjs --lane live-only
- name: Upload scenario report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: lifeops-scenario-report
path: |
artifacts/lifeops-scenario-report.json
artifacts/plugin-health-scenario-report.json
artifacts/app-control-scenario-report.json
artifacts/scenario-runs/live/
artifacts/scenario-runs/plugin-health/
artifacts/scenario-runs/app-control/
if-no-files-found: warn
retention-days: 30
+146
View File
@@ -0,0 +1,146 @@
name: Loadperf KPIs
on:
schedule:
# Daily at 09:17 UTC, after the normal nightly lanes have had time to settle.
- cron: "17 9 * * *"
pull_request:
branches: [main]
paths:
- ".github/workflows/loadperf.yml"
- "packages/benchmarks/loadperf/**"
workflow_dispatch:
inputs:
include_statesync:
description: "Run state-sync KPI against LOADPERF_BASE_URL / LOADPERF_WS_URL"
required: false
type: boolean
default: false
loadperf_base_url:
description: "Base URL for an already-running stack used by state-sync"
required: false
type: string
default: ""
loadperf_ws_url:
description: "Explicit WebSocket URL for state-sync, if different from base URL + /ws"
required: false
type: string
default: ""
concurrency:
group: loadperf-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
env:
BUN_VERSION: "canary"
CI: "true"
NODE_VERSION: "24.15.0"
jobs:
loadperf-stack:
name: Bundle, boot, and frontend KPIs
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 1
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Install Chromium
run: bunx playwright install --with-deps chromium
- name: Build app runtime and web bundle
run: |
bun run --cwd packages/shared build
bun run --cwd packages/core build
bun run --cwd plugins/plugin-wallet build
bun run --cwd packages/app-core build
bun run --cwd packages/app prebuild
bun run --cwd packages/app build:web
- name: Run loadperf bundle, boot, and frontend KPIs
env:
ELIZA_HEADLESS: "1"
ELIZA_NO_VISION_DEPS: "1"
LOADPERF_BOOT_RUNS: "1"
run: node packages/benchmarks/loadperf/run-all.mjs
- name: Upload loadperf artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: loadperf-stack-results
path: packages/benchmarks/loadperf/results/**
if-no-files-found: warn
statesync:
name: State-sync KPI against live stack
if: ${{ github.event_name == 'workflow_dispatch' && inputs.include_statesync }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 1
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
run-postinstall: "false"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Validate state-sync target
env:
LOADPERF_BASE_URL: ${{ inputs.loadperf_base_url }}
LOADPERF_WS_URL: ${{ inputs.loadperf_ws_url }}
run: |
if [ -z "${LOADPERF_BASE_URL}" ] && [ -z "${LOADPERF_WS_URL}" ]; then
echo "::error::Set loadperf_base_url or loadperf_ws_url when include_statesync is true."
exit 1
fi
- name: Run state-sync KPI
env:
LOADPERF_BASE_URL: ${{ inputs.loadperf_base_url }}
LOADPERF_WS_URL: ${{ inputs.loadperf_ws_url }}
run: node packages/benchmarks/loadperf/run-all.mjs --no-boot --no-frontend --statesync
- name: Upload state-sync artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: loadperf-statesync-results
path: packages/benchmarks/loadperf/results/**
if-no-files-found: warn
+506
View File
@@ -0,0 +1,506 @@
name: Local Inference Bench
# Nightly benchmark for the on-device chat agent. The harness is host-
# agnostic — it just talks HTTP to whatever agent is reachable at
# --target — so we run it in two layers:
#
# 1. PR + nightly: harness route smoke. Boots
# `packages/scripts/benchmark/synthetic-agent-server.mjs` on a free port and
# runs the profile harness against its model-ensure path.
# 2. Nightly only: real agent profile. Boots `bun run dev`, points the
# harness at the dev server, writes the report under
# `reports/porting/<YYYY-MM-DD>/profile.{json,md}`, uploads the
# directory as a workflow artifact, and opens/updates a tracking
# issue with the markdown summary.
#
# Cuttlefish AVD profiling (job 3) is dispatch-only — it shares
# infrastructure with `elizaos-cuttlefish.yml`, so we don't duplicate
# that setup; instead we document the dispatch entrypoint and let the
# operator chain the two workflows.
on:
schedule:
# 05:00 UTC — runs after the nightly publish job at 04:00 UTC so
# the dev server gets the latest packages.
- cron: "0 5 * * *"
# 04:30 UTC Sunday — weekly eliza-1 model publish window.
- cron: "30 4 * * 0"
pull_request:
branches: [main]
paths:
- "packages/scripts/benchmark/**"
- ".github/workflows/local-inference-bench.yml"
- "plugins/plugin-local-embedding/**"
- "packages/training/scripts/publish_eliza1_model.py"
- "packages/training/scripts/sync_catalog_from_hf.py"
- "scripts/verify-phone-download.mjs"
workflow_dispatch:
inputs:
run_real_agent:
description: "Run real-agent profile (nightly path) on-demand."
required: false
type: boolean
default: false
run_cuttlefish:
description: "Run cuttlefish AVD profile (job 3). Coordinates with elizaos-cuttlefish.yml."
required: false
type: boolean
default: false
run_publish_models:
description: "Run the eliza-1 model publish + catalog-sync job on-demand (gated by secrets.ELIZA_HF_TOKEN)."
required: false
type: boolean
default: false
concurrency:
group: local-inference-bench-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: write
issues: write
pull-requests: write
env:
BUN_VERSION: "canary"
NODE_VERSION: "24"
STUB_PORT: "31338"
jobs:
stub-validation:
name: Harness route smoke
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Boot stub agent server
run: |
set -euo pipefail
node packages/scripts/benchmark/synthetic-agent-server.mjs \
--port "${STUB_PORT}" \
--require-installed-models &
echo $! > /tmp/stub.pid
# Wait for the stub to bind (health endpoint becomes 200).
for _ in $(seq 1 50); do
if curl --silent --fail "http://127.0.0.1:${STUB_PORT}/api/health" >/dev/null; then
break
fi
sleep 0.2
done
curl --silent --fail "http://127.0.0.1:${STUB_PORT}/api/health"
- name: Verify harness health route only
run: |
set -euo pipefail
curl --silent --fail "http://127.0.0.1:${STUB_PORT}/api/health"
- name: Verify harness model ensure path
run: |
set -euo pipefail
node packages/scripts/benchmark/profile-inference.mjs \
--target "http://127.0.0.1:${STUB_PORT}" \
--config packages/scripts/benchmark/configs/host-cpu.json \
--label "stub-${GITHUB_SHA}-${GITHUB_RUN_ID}" \
--out ".ci-logs/stub-profile" \
--ensure-models \
--download-timeout-ms 30000 \
--load-timeout-ms 10000 \
--request-timeout-ms 30000
- name: Tear down stub
if: ${{ always() }}
run: |
if [ -f /tmp/stub.pid ]; then
kill "$(cat /tmp/stub.pid)" || true
fi
nightly-real-agent:
name: Nightly real-agent profile
needs: stub-validation
if: ${{ github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.run_real_agent == true) }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Preflight — published eliza-1 manifest shape
# The harness downloads the PUBLISHED HuggingFace bundle manifest and
# validates it against the Eliza-1 manifest schema before fetching any
# weight. A malformed published manifest (e.g. `files.vision` as an
# object instead of an array — the 2026-06→07 Gemma-4 cutover defect)
# otherwise fails the run ~5 minutes in, AFTER a full bun install +
# agent boot, with an opaque "expected array, received object" trace.
# Fail fast here with an operator-actionable message instead.
run: node packages/scripts/benchmark/preflight-eliza1-manifest.mjs eliza-1-2b
- name: Install dependencies
run: bun install --frozen-lockfile || bun install --no-frozen-lockfile
- name: Generate shared i18n assets
run: bun run --cwd packages/shared build:i18n
- name: Prepare dev workspace packages
# `bun run dev` starts Vite from a clean checkout. Build the app's
# workspace dependencies first so Vite can resolve package exports like
# @elizaos/core instead of failing before the benchmark can run.
run: bun run dev:prepare
- name: Boot dev agent
env:
ELIZA_API_PORT: "31337"
ELIZA_PORT: "2138"
NODE_ENV: "test"
ELIZA_DISABLE_TRAJECTORY_LOGGING: "1"
run: |
set -euo pipefail
mkdir -p .ci-logs
# Run the dev server in the background. The orchestrator
# already auto-shifts ports if the defaults are busy and writes
# a `.eliza/desktop-dev-console.log` we can tail if needed.
bun run dev > .ci-logs/dev.log 2>&1 &
echo $! > /tmp/dev.pid
# Health probe — give the agent up to 120s to come up.
for _ in $(seq 1 60); do
if curl --silent --fail "http://127.0.0.1:31337/api/health" >/dev/null 2>&1; then
echo "agent ready"
break
fi
sleep 2
done
curl --silent --fail "http://127.0.0.1:31337/api/health"
- name: Run profile harness (host-cpu config)
env:
ELIZA_API_TOKEN: ${{ secrets.ELIZA_CI_API_TOKEN }}
run: |
set -euo pipefail
DATE="$(date -u +%Y-%m-%d)"
OUT_DIR="reports/porting/${DATE}"
mkdir -p "$OUT_DIR"
node packages/scripts/benchmark/profile-inference.mjs \
--target "http://127.0.0.1:31337" \
--config packages/scripts/benchmark/configs/host-cpu.json \
--label "ci-${GITHUB_SHA}-${GITHUB_RUN_ID}" \
--out "$OUT_DIR" \
--ensure-models
echo "REPORT_DIR=$OUT_DIR" >> "$GITHUB_ENV"
- name: Tear down dev agent
if: ${{ always() }}
run: |
if [ -f /tmp/dev.pid ]; then
kill "$(cat /tmp/dev.pid)" || true
fi
- name: Upload report artifact
if: ${{ always() && env.REPORT_DIR != '' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: profile-nightly-${{ github.run_id }}
path: ${{ env.REPORT_DIR }}
if-no-files-found: warn
retention-days: 90
- name: Upload dev server log
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: dev-log-nightly-${{ github.run_id }}
path: .ci-logs/dev.log
if-no-files-found: warn
retention-days: 14
- name: Open / update tracking issue
if: ${{ env.REPORT_DIR != '' }}
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
with:
script: |
const fs = require("node:fs");
const path = require("node:path");
const reportDir = process.env.REPORT_DIR;
const mdPath = path.join(reportDir, "profile.md");
if (!fs.existsSync(mdPath)) {
core.warning(`profile.md not found at ${mdPath}; skipping issue update`);
return;
}
const md = fs.readFileSync(mdPath, "utf8");
const today = new Date().toISOString().slice(0, 10);
const title = `Nightly local-inference profile — ${today}`;
const artifactUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const body = [
`Workflow run: ${artifactUrl}`,
"",
`Report directory: \`${reportDir}\`.`,
"",
md,
].join("\n");
const { data: existing } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
labels: "nightly-local-inference",
state: "open",
per_page: 5,
});
if (existing.length === 0) {
const created = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title,
body,
labels: ["nightly-local-inference"],
});
core.info(`Opened tracking issue #${created.data.number}`);
} else {
const issue = existing[0];
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
title,
body,
});
core.info(`Updated tracking issue #${issue.number}`);
}
cuttlefish-bench:
# Manual-only path. Cuttlefish AVD provisioning is owned by
# elizaos-cuttlefish.yml; this job runs the bench against the AVD
# forward port once the operator has the AVD reachable at
# 127.0.0.1:31337. Keeping it here (rather than duplicating AVD
# setup) makes the dispatch surface obvious without doubling the CI
# cost.
name: Cuttlefish AVD profile (manual)
if: ${{ github.event_name == 'workflow_dispatch' && inputs.run_cuttlefish == true }}
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Reminder — boot cuttlefish first
run: |
echo "::warning ::This job assumes elizaos-cuttlefish.yml has booted the AVD"
echo "::warning ::and forwarded 31337 to 127.0.0.1. Run that workflow first."
echo "Probing /api/health on 127.0.0.1:31337 ..."
curl --silent --fail "http://127.0.0.1:31337/api/health" || {
echo "::error ::No agent reachable at 127.0.0.1:31337. Run elizaos-cuttlefish.yml first."
exit 1
}
- name: Run profile harness (aosp-arm64 config)
run: |
set -euo pipefail
DATE="$(date -u +%Y-%m-%d)"
OUT_DIR="reports/porting/${DATE}-cuttlefish"
mkdir -p "$OUT_DIR"
node packages/scripts/benchmark/profile-inference.mjs \
--target "http://127.0.0.1:31337" \
--config packages/scripts/benchmark/configs/aosp-arm64.json \
--label "cuttlefish-${GITHUB_RUN_ID}" \
--out "$OUT_DIR" \
--ensure-models
- name: Upload cuttlefish report artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: profile-cuttlefish-${{ github.run_id }}
path: reports/porting/
if-no-files-found: warn
retention-days: 90
publish-models-nightly:
# Weekly nightly publish to the elizaos HF org. Walks W5-Pipeline's
# output, calls publish_eliza1_model.py for each fused-kernel GGUF,
# then runs sync_catalog_from_hf.py to produce the catalog diff and
# opens a PR with the update.
#
# Gated behind secrets.ELIZA_HF_TOKEN. When the secret is unset
# (forks, dry-test branches), the job no-ops with an explicit
# warning so unrelated workflow runs don't fail.
name: Publish eliza-1 models (weekly)
if: >-
(github.event_name == 'schedule' && github.event.schedule == '30 4 * * 0') ||
(github.event_name == 'workflow_dispatch' && inputs.run_publish_models == true)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 90
env:
# W5-Pipeline drops finished checkpoints under this path. The
# nightly publish step expects each subdirectory to be a complete
# `<base>-optimized` (or `-drafter`) bundle —
# i.e. a single .gguf + manifest.json + README.md.
W5_PIPELINE_OUTPUT_ROOT: ${{ vars.W5_PIPELINE_OUTPUT_ROOT || '/mnt/w5-pipeline-output' }}
HF_TOKEN: ${{ secrets.ELIZA_HF_TOKEN }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Guard — require ELIZA_HF_TOKEN
run: |
set -euo pipefail
if [ -z "${HF_TOKEN:-}" ]; then
echo "::warning ::secrets.ELIZA_HF_TOKEN not configured; publish job will no-op."
echo "PUBLISH_SKIPPED=1" >> "$GITHUB_ENV"
exit 0
fi
echo "PUBLISH_SKIPPED=0" >> "$GITHUB_ENV"
- name: Setup Python
if: ${{ env.PUBLISH_SKIPPED != '1' }}
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: "3.11"
- name: Install huggingface_hub
if: ${{ env.PUBLISH_SKIPPED != '1' }}
run: |
set -euo pipefail
python -m pip install --upgrade pip
python -m pip install "huggingface_hub[cli]>=0.24"
- name: Setup Node.js
if: ${{ env.PUBLISH_SKIPPED != '1' }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
if: ${{ env.PUBLISH_SKIPPED != '1' }}
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Probe W5-Pipeline output
if: ${{ env.PUBLISH_SKIPPED != '1' }}
run: |
set -euo pipefail
if [ ! -d "$W5_PIPELINE_OUTPUT_ROOT" ]; then
echo "::warning ::no W5-Pipeline output at $W5_PIPELINE_OUTPUT_ROOT; nothing to publish"
echo "NO_PIPELINE_OUTPUT=1" >> "$GITHUB_ENV"
exit 0
fi
echo "Pipeline output:"
ls -la "$W5_PIPELINE_OUTPUT_ROOT"
- name: Publish each fused-kernel bundle
if: ${{ env.PUBLISH_SKIPPED != '1' && env.NO_PIPELINE_OUTPUT != '1' }}
run: |
set -euo pipefail
shopt -s nullglob
FAILURES=0
for dir in "$W5_PIPELINE_OUTPUT_ROOT"/*/; do
name=$(basename "$dir")
# Bundle naming: <base>-optimized or <base>-drafter.
# Anything else is rejected — these repos are for fused builds only.
if [[ "$name" != *-optimized && "$name" != *-drafter ]]; then
echo "::warning ::skipping $name (not a fused-kernel bundle name)"
continue
fi
repo_id="elizaos/$name"
echo "::group::publish $repo_id"
if ! python packages/training/scripts/publish_eliza1_model.py \
--model-dir "$dir" \
--repo-id "$repo_id"; then
echo "::error ::publish failed for $name"
FAILURES=$((FAILURES + 1))
fi
echo "::endgroup::"
done
if [ "$FAILURES" -gt 0 ]; then
echo "::error ::$FAILURES bundle(s) failed to publish"
exit 1
fi
- name: Sync catalog diff from elizaos
if: ${{ env.PUBLISH_SKIPPED != '1' }}
run: |
set -euo pipefail
DATE="$(date -u +%Y-%m-%d)"
OUT_DIR="reports/porting/${DATE}"
mkdir -p "$OUT_DIR"
python packages/training/scripts/sync_catalog_from_hf.py \
--org elizaos \
--out "$OUT_DIR/catalog-diff.json"
echo "DIFF_PATH=$OUT_DIR/catalog-diff.json" >> "$GITHUB_ENV"
- name: Verify a sample download (round-trip from HF)
if: ${{ env.PUBLISH_SKIPPED != '1' && env.DIFF_PATH != '' }}
run: |
set -euo pipefail
# Pick the first published id from the diff and round-trip it
# through the phone-equivalent Downloader.
SAMPLE_ID=$(python -c 'import json,os,sys; d=json.load(open(os.environ["DIFF_PATH"])); ids=[e["id"] for e in d.get("entries",[])]; print(ids[0] if ids else "")')
if [ -z "${SAMPLE_ID:-}" ]; then
echo "::warning ::diff has no published entries; skipping verify-phone-download"
exit 0
fi
bun install --frozen-lockfile
node scripts/verify-phone-download.mjs \
--model-id "$SAMPLE_ID" \
--catalog-diff "$DIFF_PATH"
- name: Open catalog-update PR
if: ${{ env.PUBLISH_SKIPPED != '1' && env.DIFF_PATH != '' }}
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ci/eliza-1-catalog-${{ github.run_id }}
base: develop
commit-message: "chore(catalog): refresh eliza-1 pointers (auto)"
title: "chore(catalog): refresh eliza-1 pointers"
body: |
Auto-generated catalog diff from `sync_catalog_from_hf.py`.
- Workflow run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
- Diff path: `${{ env.DIFF_PATH }}`
W5-Catalog: please apply the diff to `packages/shared/src/local-inference/catalog.ts`.
add-paths: |
reports/porting/**/catalog-diff.json
delete-branch: true
labels: |
eliza-1
local-inference
automated
@@ -0,0 +1,234 @@
name: Local Inference Matrix
# Runs the local-inference smoke and ablation variant suite, and gates
# regressions against per-(backend, variant) tok/s floors in
# packages/scripts/local-inference-thresholds.json.
#
# Triggers:
# - workflow_dispatch (always; needed for macOS/CUDA cost control)
# - pull_request when source paths under the local-inference subsystem,
# the fork build script, or the ablation runner change.
#
# Matrix gating:
# - ubuntu-latest CPU always runs.
# - ubuntu-latest CUDA only runs when the PR carries the `gpu` label
# (and a self-hosted GPU runner with the `gpu` runner labels is online).
# - macos-14 Metal only runs on workflow_dispatch (Apple Silicon cost).
# CUDA and Metal entries are continue-on-error until self-hosted runners
# are wired up so the workflow stays green on CPU only.
#
# Note: Job-level `if:` cannot reference `matrix` (GitHub Actions limitation).
# Optional legs are added by `prepare-matrix` via JSON so only requested
# combinations are scheduled.
on:
workflow_dispatch:
inputs:
run_metal:
description: "Run macOS-14 Metal job (Apple Silicon)"
required: false
type: boolean
default: true
run_cuda:
description: "Run self-hosted CUDA job (requires `gpu` runner)"
required: false
type: boolean
default: false
pull_request:
branches: [main]
paths:
- "packages/app-core/src/services/local-inference/**"
- "packages/scripts/local-inference-ablation.mjs"
- "packages/scripts/local-inference-ablation.config.json"
- "packages/scripts/local-inference-thresholds.json"
- "packages/scripts/local-inference-smoke.mjs"
- ".github/workflows/local-inference-matrix.yml"
concurrency:
group: local-inference-matrix-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
BUN_VERSION: "canary"
NODE_VERSION: "24"
ELIZA_STATE_DIR: ${{ github.workspace }}/.eliza-state
# Smoke model: small enough to download and run on a CPU runner in seconds.
SMOKE_MODEL_REPO: "bartowski/SmolLM2-360M-Instruct-GGUF"
SMOKE_MODEL_FILE: "SmolLM2-360M-Instruct-Q4_K_M.gguf"
jobs:
prepare-matrix:
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
outputs:
matrix: ${{ steps.set.outputs.matrix }}
steps:
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
id: set
with:
script: |
const ev = context.eventName;
const pr = context.payload.pull_request;
const inputs = context.payload.inputs ?? {};
const hasGpuLabel =
ev === "pull_request" &&
Array.isArray(pr?.labels) &&
pr.labels.some((label) => label?.name === "gpu");
const runCudaDispatch =
ev === "workflow_dispatch" &&
(inputs.run_cuda === true ||
inputs.run_cuda === "true");
const runMetalDispatch =
ev === "workflow_dispatch" &&
inputs.run_metal !== false &&
inputs.run_metal !== "false";
const include = [
{
label: "ubuntu-latest cpu",
"runs-on": "ubuntu-latest",
backend: "cpu",
target: "linux-x64-cpu",
timeout: 90,
"continue-on-error": false,
},
];
if (hasGpuLabel || runCudaDispatch) {
include.push({
label: "ubuntu-latest cuda (self-hosted)",
"runs-on": ["self-hosted", "gpu", "cuda"],
backend: "cuda",
target: "linux-x64-cuda",
timeout: 120,
"continue-on-error": true,
});
}
if (runMetalDispatch) {
include.push({
label: "macos-14 metal",
"runs-on": "macos-14",
backend: "metal",
target: "darwin-arm64-metal",
timeout: 120,
"continue-on-error": true,
});
}
core.setOutput("matrix", JSON.stringify({ include }));
matrix:
name: ${{ matrix.label }}
needs: prepare-matrix
runs-on: ${{ matrix.runs-on }}
timeout-minutes: ${{ matrix.timeout }}
continue-on-error: ${{ matrix.continue-on-error }}
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.prepare-matrix.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
show-progress: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install Linux build deps
if: ${{ runner.os == 'Linux' }}
run: |
set -euo pipefail
APT_ARGS=(-o Acquire::ForceIPv4=true -o Acquire::Retries=5 -o Acquire::http::Timeout=30)
for attempt in 1 2 3; do
if sudo apt-get "${APT_ARGS[@]}" update && \
sudo apt-get "${APT_ARGS[@]}" install -y --fix-missing \
cmake ninja-build build-essential ccache git python3 python3-pip; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "apt install failed after ${attempt} attempts"
exit 1
fi
sleep $((attempt * 10))
done
cmake --version
- name: Install macOS build deps
if: ${{ runner.os == 'macOS' }}
run: |
set -euo pipefail
if ! command -v cmake >/dev/null 2>&1; then
brew install cmake
fi
cmake --version
- name: Install Hugging Face CLI
run: |
python3 -m pip install --user --upgrade "huggingface_hub[hf_transfer]"
if ! command -v hf >/dev/null 2>&1; then
export PATH="$HOME/.local/bin:$PATH"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
fi
hf --help >/dev/null
- name: Cache local-inference state (binaries + models)
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ${{ env.ELIZA_STATE_DIR }}/local-inference
key: local-inference-${{ matrix.target }}-${{ env.SMOKE_MODEL_FILE }}
restore-keys: |
local-inference-${{ matrix.target }}-${{ env.SMOKE_MODEL_FILE }}-
local-inference-${{ matrix.target }}-
- name: Smoke probe binaries
run: |
bun ./packages/scripts/local-inference-smoke.mjs
- name: Download smoke GGUF (cached)
run: |
set -euo pipefail
MODEL_DIR="${ELIZA_STATE_DIR}/local-inference/models"
mkdir -p "$MODEL_DIR"
if [ -f "$MODEL_DIR/${SMOKE_MODEL_FILE}" ]; then
echo "model already cached: $MODEL_DIR/${SMOKE_MODEL_FILE}"
ls -lh "$MODEL_DIR/${SMOKE_MODEL_FILE}"
exit 0
fi
hf download "${SMOKE_MODEL_REPO}" "${SMOKE_MODEL_FILE}" \
--local-dir "$MODEL_DIR"
ls -lh "$MODEL_DIR/${SMOKE_MODEL_FILE}"
- name: Run ablation (--quick, gated)
run: |
set -euo pipefail
MODEL_PATH="${ELIZA_STATE_DIR}/local-inference/models/${SMOKE_MODEL_FILE}"
node ./packages/scripts/local-inference-ablation.mjs \
--backend ${{ matrix.backend }} \
--model "$MODEL_PATH" \
--quick \
--gate packages/scripts/local-inference-thresholds.json \
--out-dir artifacts/local-inference-ablation
- name: Upload ablation artifact
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: local-inference-ablation-${{ matrix.target }}
path: artifacts/local-inference-ablation/*.json
if-no-files-found: warn
retention-days: 30
+49
View File
@@ -0,0 +1,49 @@
name: Markdown Links
on:
pull_request:
branches: [main, develop]
paths:
- "**/*.md"
- "scripts/check-markdown-links.mjs"
- ".github/workflows/markdown-links.yml"
push:
branches: [main, develop]
paths:
- "**/*.md"
- "scripts/check-markdown-links.mjs"
- ".github/workflows/markdown-links.yml"
workflow_dispatch:
# CI fan-out fix: the PR-scoped concurrency from #14069 fell back to
# github.run_id on push, so every develop push got a unique group and
# nothing ever superseded. A 37-merge wave queued thousands of push runs
# and starved the self-hosted fleet. Fall back to github.ref (all develop
# pushes share a group) and cancel superseded runs on push too. PR behavior
# is unchanged (pull_request.number still wins the key); merge_group and
# schedule events are not in the cancel set, so the merge queue and nightly
# runs always complete.
concurrency:
group: markdown-links-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'push' }}
permissions:
contents: read
jobs:
relative-links:
name: Relative Markdown Links
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 5
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24.15.0"
- name: Check relative Markdown links
run: node scripts/check-markdown-links.mjs
+78
View File
@@ -0,0 +1,78 @@
name: memperf
# Local-inference memory benchmark + regression gate (issue #8809).
#
# Runs the memperf harness (`run-all.mjs`) and the real-arbiter unit tests on a
# GitHub-hosted runner whenever the harness or the MemoryArbiter changes. The
# harness is CI-safe without model bundles: it exits
# 0 measured rows present, all budgets pass
# 1 a real budget / eviction-telemetry floor REGRESSED → fails CI (the gate)
# 2 nothing measurable on this host (no model bundle); self-check passed → skip
# so a model-absent runner still exercises the real arbiter fit/pressure path via
# the co-residency self-check, and a genuine regression turns CI red.
on:
pull_request:
branches: [main]
paths:
- "packages/benchmarks/memperf/**"
- "plugins/plugin-local-inference/src/services/memory-arbiter.ts"
- "plugins/plugin-local-inference/src/services/active-model.ts"
- ".github/workflows/memperf.yml"
schedule:
# 05:40 UTC — alongside the other nightly benchmark lanes.
- cron: "40 5 * * *"
workflow_dispatch: {}
concurrency:
group: memperf-${{ github.ref }}
cancel-in-progress: true
jobs:
harness-gate:
name: harness + budget regression gate (no models)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: oven-sh/setup-bun@v2
with:
# The repo packageManager pins the exact canary used by local dev,
# but historical canary tags can disappear from GitHub releases.
# Ask setup-bun for the moving canary channel so scheduled runs do
# not fail before the memperf gate gets to execute.
bun-version: canary
- uses: actions/setup-node@v4
with:
node-version: "24"
- name: Install workspace
run: bun install
- name: Generate shared i18n assets
# The real-arbiter self-check imports @elizaos/core source, which
# expects generated keyword data to exist in a fresh checkout.
run: bun run --cwd packages/shared build:i18n
- name: Metric-schema contract (shared with #8800)
run: bun test --conditions=eliza-source packages/benchmarks/memperf/metric-schema.test.ts
- name: Real-arbiter co-residency eviction self-check
run: bun test --conditions=eliza-source packages/benchmarks/memperf/co-residency.test.ts
- name: memperf harness + budget regression gate
run: |
set +e
node packages/benchmarks/memperf/run-all.mjs
code=$?
set -e
case "$code" in
0) echo "memperf: all budgets pass" ;;
2) echo "memperf: skipped (no model bundle on this runner); arbiter telemetry self-check passed" ;;
1) echo "::error::memperf budget / eviction-telemetry regression (exit 1)"; exit 1 ;;
*) echo "::error::memperf harness errored (exit $code)"; exit "$code" ;;
esac
- name: Upload memperf summary
if: always()
uses: actions/upload-artifact@v7
with:
name: memperf-summary
path: packages/benchmarks/memperf/results/summary/
if-no-files-found: ignore
+696
View File
@@ -0,0 +1,696 @@
name: Mobile Build Smoke Test
on:
# Reusable entry for the scheduled exhaustive develop lane (#12342).
workflow_call:
pull_request:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review, labeled]
workflow_dispatch:
inputs:
cloud_api_base:
description: "Eliza Cloud API base for the iOS cloud runtime-mode smoke (defaults to the staging sandbox var, then prod)"
required: false
default: ""
schedule:
# Nightly iOS local (full-Bun on-device engine) lane (#13578). This mode
# is too heavy for the PR path — a full-Bun engine build, a real ~1.3GB
# GGUF, and CPU inference in the simulator — so it runs on a schedule and
# gates that nightly run instead of every pull request.
- cron: "0 7 * * *"
concurrency:
group: mobile-smoke-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
BUN_VERSION: "canary"
# Default to least privilege. Override per-job where needed.
permissions:
contents: read
jobs:
changes:
name: Classify changed paths
# Path classifier is a git-diff + node script: no self-hosted resources.
# Keep it on GitHub-hosted so a drained hetzner fleet cannot leave it
# queued indefinitely and pile up runs (#13617/#8501).
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
outputs:
ios: ${{ steps.filter.outputs.ios }}
android: ${{ steps.filter.outputs.android }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
submodules: false
- name: Setup Node.js for path gate
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: "24"
- name: Determine affected mobile smoke surface
id: filter
shell: bash
env:
PR_LABELS: ${{ github.event_name == 'pull_request' && join(github.event.pull_request.labels.*.name, ',') || '' }}
run: |
set -euo pipefail
node packages/scripts/ci-path-gate.mjs \
--config mobile \
--event "${{ github.event_name }}" \
--base "${{ github.event.pull_request.base.sha }}" \
--head "${{ github.event.pull_request.head.sha }}" \
--labels "$PR_LABELS" \
--output "$GITHUB_OUTPUT" \
--summary "$GITHUB_STEP_SUMMARY"
build-ios:
name: iOS Simulator Build
needs: changes
if: github.event_name != 'pull_request' || needs.changes.outputs.ios == 'true'
runs-on: macos-15
timeout-minutes: 30
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# This workflow validates the app-core mobile project wiring, not
# native inference submodule health. Keep checkout source-clean and
# let setup-bun-workspace/postinstall initialize only the submodules
# it actually needs.
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: 24
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-native-deps: "false"
install-command: bun install --ignore-scripts --no-frozen-lockfile
run-postinstall: "true"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Generate protobuf types
# Restores the step that was on the last green Mobile Build Smoke
# run (52ab686a, 2026-05-09). Without these generated types in
# `packages/core/src/types/generated/`, `@elizaos/core`'s build
# fails downstream, which cascades into `@elizaos/shared` and
# vite config resolution.
run: |
if [ -d packages/schemas ] && [ -f packages/schemas/buf.gen.yaml ] && [ ! -d packages/core/src/types/generated ]; then
cd packages/schemas
node node_modules/@bufbuild/buf/install.js 2>/dev/null || true
bunx @bufbuild/buf@1.67.0 generate
fi
- name: Build Vite config package deps
# vite.config.ts imports dist-only workspace packages
# (`@elizaos/shared` and `@elizaos/ui/config/app-config`), and the
# app aliases plugin-browser source which imports `@elizaos/vault`.
# Build those package chains before Vite evaluates/bundles the app.
run: bun turbo run build --filter=@elizaos/ui --filter=@elizaos/vault
- name: Build iOS simulator app
run: node --max-old-space-size=8192 packages/app-core/scripts/run-mobile-build.mjs ios
env:
LANG: en_US.UTF-8
# The simulator smokes drive a loopback host agent. Use the direct
# developer iOS lane; App Store/cloud builds intentionally reject
# cleartext loopback and are covered by release workflows.
ELIZA_BUILD_VARIANT: "direct"
ELIZA_RELEASE_AUTHORITY: "developer-toolchain"
VITE_ELIZA_IOS_ALLOW_SIMULATOR_LOOPBACK: "1"
ELIZA_IOS_FULL_BUN_ENGINE: "0"
ELIZA_IOS_BUILD_DESTINATION: "generic/platform=iOS Simulator"
ELIZA_IOS_BUILD_SDK: "iphonesimulator"
ELIZA_MOBILE_BUILD_TIMING_OUT: ${{ runner.temp }}/ios-simulator-build-timing.json
ELIZA_MOBILE_BUILD_TIMING_TARGET: ios-ipa
NODE_OPTIONS: "--max-old-space-size=8192"
- name: Check iOS simulator build-time budget
run: |
bun run --cwd packages/app perf:startup-budget -- \
--build-timing "$RUNNER_TEMP/ios-simulator-build-timing.json" \
--out "$RUNNER_TEMP/ios-simulator-build-budget-report.json"
- name: Upload iOS simulator build budget report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-simulator-build-budget
path: |
${{ runner.temp }}/ios-simulator-build-timing.json
${{ runner.temp }}/ios-simulator-build-budget-report.json
if-no-files-found: warn
retention-days: 7
- name: Verify .app exists
run: |
APP_PATH=$(find ~/Library/Developer/Xcode/DerivedData -name "App.app" -path "*/Debug-iphonesimulator/*" | head -1)
if [ -z "$APP_PATH" ]; then
echo "ERROR: App.app not found in DerivedData"
exit 1
fi
echo "Built app at: $APP_PATH"
BUNDLE_ID=$(/usr/libexec/PlistBuddy -c "Print :CFBundleIdentifier" "$APP_PATH/Info.plist")
echo "Bundle ID: $BUNDLE_ID"
# Bundle ID source of truth is packages/app/app.config.ts (read
# by run-mobile-build.mjs via regex). Current value is
# `app.eliza` since commit d042ed57b5 renamed it from the
# legacy `ai.elizaos.app`. The two prior ids remain accepted
# because release branches and downstream forks may not have
# rebased onto the rename yet.
case "$BUNDLE_ID" in
app.eliza|ai.elizaos.app|ai.elizaos.Eliza) ;;
*)
echo "ERROR: Unexpected bundle ID: $BUNDLE_ID"
exit 1
;;
esac
- name: Verify staged renderer is the freshly built one (no stale UI, #9309)
run: |
# The staged iOS public/ must carry exactly the renderer just built in
# packages/app/dist (build stamp match). Fails loudly on stale/missing UI.
node packages/app-core/scripts/verify-ondevice-artifact.mjs --platform ios
# The renderer build stamp must also be baked into the built .app bundle.
APP_PATH=$(find ~/Library/Developer/Xcode/DerivedData -name "App.app" -path "*/Debug-iphonesimulator/*" | head -1)
if [ ! -f "$APP_PATH/public/eliza-renderer-build.json" ]; then
echo "ERROR: built .app is missing the renderer build stamp at public/eliza-renderer-build.json"
exit 1
fi
echo "Renderer build stamp present in built .app:"
cat "$APP_PATH/public/eliza-renderer-build.json"
- name: Run iOS onboarding -> home simulator smoke
run: |
set -euo pipefail
node packages/app/scripts/ios-onboarding-smoke.mjs
- name: Upload iOS onboarding evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-onboarding-to-home
path: packages/app/test-results/ios-onboarding-to-home/**
if-no-files-found: warn
retention-days: 7
- name: Run iOS native attachment smoke (#10936)
# Evidence lane for the content-addressed media + native
# Filesystem/Share path. Promoted to a hard-gating step (#13576):
# a regression in the media-store / Filesystem / Share path now
# turns Mobile Build Smoke red instead of self-skipping to green.
run: |
set -euo pipefail
node packages/app/scripts/ios-attachment-smoke.mjs
- name: Upload iOS native attachment evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-attachment-smoke
path: packages/app/test-results/ios-attachment-smoke/**
if-no-files-found: warn
retention-days: 7
- name: Run iOS local-chat simulator smoke (#9943)
# Wire the existing iOS chat device smoke into CI. mobile-local-chat-smoke
# drives a real chat turn through the installed simulator app (the iOS
# self-driving-WebView mechanism — WKWebView is not CDP/Playwright-drivable,
# so this is the only viable iOS device-chat coverage). It previously ran
# in ZERO CI lanes (issue #9943 item 5). Promoted to a hard-gating
# step (#13576): a regression in iOS chat send/receive through the
# app-core API now turns Mobile Build Smoke red instead of
# self-skipping to green.
run: |
set -euo pipefail
node packages/app/scripts/mobile-local-chat-smoke.mjs \
--platform ios --require-installed --start-host-agent
- name: Upload iOS local-chat evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-local-chat
path: packages/app/test-results/ios-local-chat/**
if-no-files-found: warn
retention-days: 7
- name: Run Mobile Safari chat-sheet visual smoke (#15666)
# run-chat-sheet-mobile-safari-smoke.mjs builds the WebKit chat-sheet
# fixture via the Playwright smoke, serves it over loopback HTTP, opens
# it in real Mobile Safari on the booted simulator, and asserts a
# non-blank screen capture. Wired here because this is the only
# PR-path lane with an iOS Simulator available, and the ui-e2e
# runner-coverage ratchet (packages/scripts/__tests__/
# ui-e2e-runner-coverage.test.ts) requires every packages/ui __e2e__
# runner to be invoked by a workflow leg.
run: |
set -euo pipefail
bunx playwright install webkit
bun run --cwd packages/ui test:chat-sheet-mobile-safari-smoke
- name: Upload Mobile Safari chat-sheet evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-mobile-safari-chat-sheet
path: packages/ui/src/components/shell/__e2e__/output-mobile-safari/**
if-no-files-found: warn
retention-days: 7
- name: Verify required Info.plist keys for background HealthKit
# Background HealthKit + BGTaskScheduler integration is in
# progress: no plist generator is configured yet (no
# capacitor.config.ts customization, no run-mobile-build.mjs
# writer for these keys), so until that lands the verify is a
# non-blocking warning. Flip back to a hard fail when the
# generator pipeline lands.
continue-on-error: true
run: |
APP_PATH=$(find ~/Library/Developer/Xcode/DerivedData -name "App.app" -path "*/Debug-iphonesimulator/*" | head -1)
INFO_PLIST="$APP_PATH/Info.plist"
for key in NSHealthShareUsageDescription BGTaskSchedulerPermittedIdentifiers UIBackgroundModes; do
if ! /usr/libexec/PlistBuddy -c "Print :$key" "$INFO_PLIST" >/dev/null 2>&1; then
echo "::warning::Info.plist missing required key: $key (TODO: wire up plist customization for background HealthKit)"
/usr/libexec/PlistBuddy -c "Print" "$INFO_PLIST" | head -80
exit 1
fi
done
# Confirm the sleep-refresh task identifier is whitelisted so
# BGTaskScheduler.register actually succeeds on device.
if ! /usr/libexec/PlistBuddy -c "Print :BGTaskSchedulerPermittedIdentifiers" "$INFO_PLIST" \
| grep -q "ai.eliza.mobile-signals.sleep-refresh"; then
echo "::warning::BGTaskSchedulerPermittedIdentifiers missing ai.eliza.mobile-signals.sleep-refresh"
exit 1
fi
- name: Verify Fastlane config landed in packages/app/ios
# Capacitor's iOS project ends up at packages/app/ios/ once
# run-mobile-build.mjs runs. overlayIosNativeFiles() copies
# the Fastlane config there. Skip when the dir hasn't been
# populated yet (Phase D mobile pipeline isn't always run in
# smoke).
continue-on-error: true
run: |
if [ ! -d packages/app/ios ]; then
echo "::warning::packages/app/ios/ not present; skipping Fastlane config verify"
exit 0
fi
for f in packages/app/ios/Gemfile packages/app/ios/fastlane/Fastfile packages/app/ios/fastlane/Appfile packages/app/ios/fastlane/Matchfile; do
if [ ! -f "$f" ]; then
echo "::warning::$f missing. overlayIosNativeFiles() should have copied Fastlane config here so CI can call \`fastlane beta|release\`."
exit 1
fi
done
echo "Fastlane config present at packages/app/ios/."
build-ios-local:
name: iOS Local (full-Bun) Simulator Smoke
# Nightly / on-demand only — never on the PR path. The full-Bun on-device
# engine build + real GGUF + CPU inference is far too heavy to gate every
# pull request, so the local runtime mode gates the scheduled run instead.
# This closes the coverage gap called out in #13578: remote mode is
# CI-blocking, but local (full-Bun) mode had no unattended lane. Mirrors
# the remote build-ios job, swapping the direct/loopback host-agent smoke
# for the on-device full-Bun engine smoke (no host agent — the engine runs
# in-process and the smoke exchanges the request/result via Preferences).
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: macos-15
timeout-minutes: 90
env:
# A real, small on-device GGUF — the same eliza-1 e2b text model the
# Android live smoke stages. mobile-local-chat-smoke.mjs copies whatever
# ELIZA_IOS_FULL_BUN_SMOKE_MODEL_PATH points at into the sim container and
# registers it under the `eliza-1-2b` id, capping the context to 4096 so
# model load + first reply stay fast on a simulator.
IOS_FULL_BUN_SMOKE_MODEL_URL: "https://huggingface.co/elizaos/eliza-1/resolve/main/bundles/e2b/text/eliza-1-e2b-32k.gguf?download=true"
IOS_FULL_BUN_SMOKE_MODEL_FILE: "eliza-1-e2b-32k.gguf"
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
# The full-Bun on-device engine builds the MTP-forked llama.cpp kernels
# (run-mobile-build.mjs ios-local with ELIZA_IOS_FULL_BUN_ENGINE=1), which
# requires the llama.cpp fork checkout. `submodules: false` above keeps the
# checkout source-clean, so init just that submodule (not the full recursive
# tree) before the build — build-llama-cpp-mtp.mjs otherwise aborts with
# "no llama.cpp fork checkout found".
- name: Initialize llama.cpp fork submodule (full-Bun MTP kernels)
run: git submodule update --init --recursive plugins/plugin-local-inference/native/llama.cpp
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: 24
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-native-deps: "false"
install-command: bun install --ignore-scripts --no-frozen-lockfile
run-postinstall: "true"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Generate protobuf types
run: |
if [ -d packages/schemas ] && [ -f packages/schemas/buf.gen.yaml ] && [ ! -d packages/core/src/types/generated ]; then
cd packages/schemas
node node_modules/@bufbuild/buf/install.js 2>/dev/null || true
bunx @bufbuild/buf@1.67.0 generate
fi
- name: Build Vite config package deps
run: bun turbo run build --filter=@elizaos/ui --filter=@elizaos/vault
- name: Cache full-Bun smoke GGUF
id: model-cache
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
with:
path: ~/.cache/eliza-ios-full-bun-smoke
key: ios-full-bun-smoke-model-${{ env.IOS_FULL_BUN_SMOKE_MODEL_FILE }}
- name: Download full-Bun smoke GGUF (cache miss)
if: steps.model-cache.outputs.cache-hit != 'true'
run: |
set -euo pipefail
mkdir -p "$HOME/.cache/eliza-ios-full-bun-smoke"
curl -fL --retry 3 --retry-all-errors \
-o "$HOME/.cache/eliza-ios-full-bun-smoke/$IOS_FULL_BUN_SMOKE_MODEL_FILE" \
"$IOS_FULL_BUN_SMOKE_MODEL_URL"
ls -la "$HOME/.cache/eliza-ios-full-bun-smoke/"
- name: Build iOS simulator app (full-Bun local runtime)
# Mirrors `build:ios:local:sim` (run-mobile-build.mjs ios-local with
# ELIZA_IOS_FULL_BUN_ENGINE=1). The remote lane builds the same target
# with the engine disabled (ELIZA_IOS_FULL_BUN_ENGINE=0); flipping it on
# is exactly the coverage this lane adds.
run: node --max-old-space-size=8192 packages/app-core/scripts/run-mobile-build.mjs ios-local
env:
LANG: en_US.UTF-8
ELIZA_BUILD_VARIANT: "direct"
ELIZA_RELEASE_AUTHORITY: "developer-toolchain"
VITE_ELIZA_IOS_ALLOW_SIMULATOR_LOOPBACK: "1"
ELIZA_IOS_FULL_BUN_ENGINE: "1"
ELIZA_IOS_BUILD_DESTINATION: "generic/platform=iOS Simulator"
ELIZA_IOS_BUILD_SDK: "iphonesimulator"
ELIZA_MOBILE_BUILD_TIMING_OUT: ${{ runner.temp }}/ios-full-bun-build-timing.json
ELIZA_MOBILE_BUILD_TIMING_TARGET: ios-ipa
NODE_OPTIONS: "--max-old-space-size=8192"
- name: Check iOS full-Bun build-time budget
run: |
bun run --cwd packages/app perf:startup-budget -- \
--build-timing "$RUNNER_TEMP/ios-full-bun-build-timing.json" \
--out "$RUNNER_TEMP/ios-full-bun-build-budget-report.json"
- name: Upload iOS full-Bun build budget report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-full-bun-build-budget
path: |
${{ runner.temp }}/ios-full-bun-build-timing.json
${{ runner.temp }}/ios-full-bun-build-budget-report.json
if-no-files-found: warn
retention-days: 7
- name: Verify .app exists
run: |
APP_PATH=$(find ~/Library/Developer/Xcode/DerivedData -name "App.app" -path "*/Debug-iphonesimulator/*" | head -1)
if [ -z "$APP_PATH" ]; then
echo "ERROR: App.app not found in DerivedData"
exit 1
fi
echo "Built app at: $APP_PATH"
- name: Verify staged renderer is the freshly built one (no stale UI, #9309)
run: |
node packages/app-core/scripts/verify-ondevice-artifact.mjs --platform ios
- name: Boot simulator and install the full-Bun app
run: |
set -euo pipefail
# Pick an available iPhone simulator; boot it and wait for readiness.
UDID=$(xcrun simctl list devices available -j \
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{const j=JSON.parse(s);let u="";for(const runtime of Object.keys(j.devices)){if(!/iOS/.test(runtime))continue;for(const dev of j.devices[runtime]){if(dev.isAvailable&&/iPhone/.test(dev.name)){u=dev.udid;break}}if(u)break}process.stdout.write(u)})')
if [ -z "$UDID" ]; then
echo "ERROR: no available iPhone simulator found"
xcrun simctl list devices available
exit 1
fi
echo "Booting simulator $UDID"
xcrun simctl boot "$UDID" || true
xcrun simctl bootstatus "$UDID" -b
APP_PATH=$(find ~/Library/Developer/Xcode/DerivedData -name "App.app" -path "*/Debug-iphonesimulator/*" | head -1)
echo "Installing $APP_PATH"
xcrun simctl install "$UDID" "$APP_PATH"
- name: Run iOS full-Bun local-chat simulator smoke (#13578)
# Gating check for the nightly lane: build the on-device full-Bun
# engine, stage the real GGUF, and assert the exact on-device reply
# (`ios smoke model works`). No `continue-on-error` — a regression in
# engine boot, the on-device model registry, or first-reply generation
# fails the scheduled run.
run: |
set -euo pipefail
mkdir -p packages/app/test-results/ios-full-bun-local-chat
# Point the smoke at the cached GGUF via an absolute path it can stat.
export ELIZA_IOS_FULL_BUN_SMOKE_MODEL_PATH="$HOME/.cache/eliza-ios-full-bun-smoke/$IOS_FULL_BUN_SMOKE_MODEL_FILE"
if [ ! -f "$ELIZA_IOS_FULL_BUN_SMOKE_MODEL_PATH" ]; then
echo "ERROR: staged smoke model missing at $ELIZA_IOS_FULL_BUN_SMOKE_MODEL_PATH"
exit 1
fi
node packages/app/scripts/mobile-local-chat-smoke.mjs \
--platform ios --require-installed --ios-select-local --ios-full-bun-smoke \
2>&1 | tee packages/app/test-results/ios-full-bun-local-chat/smoke.log
- name: Upload iOS full-Bun local-chat evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-full-bun-local-chat
path: packages/app/test-results/ios-full-bun-local-chat/**
if-no-files-found: warn
retention-days: 7
ios-cloud-mode:
name: iOS Cloud Runtime-Mode Smoke
# Nightly / on-demand only — never on the PR path. Provisioning a real Eliza
# Cloud agent (Hetzner-backed) is slow and costs org credit, so the cloud
# runtime mode gates the scheduled run alongside the full-Bun local lane.
# This closes the third row of the #13578 coverage matrix: remote mode is
# PR-blocking and local (full-Bun) mode gates nightly, but cloud mode had no
# unattended lane at all — the XCUITest cloud-onboarding path XCTSkips at the
# OAuth wall without a device Cloud session, and cloud-provisioning-e2e.mjs
# (the programmatic cloud runtime assertion) was only reachable behind the
# unwired `ios-e2e.mjs --cloud`. cloud-provisioning-e2e.mjs has no workspace
# imports (node built-ins + fetch only) and needs no build/simulator, so the
# cloud runtime-mode gate is a fast ubuntu job seeded with a real Cloud
# credential rather than a browser OAuth leg.
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
env:
# Headless session seed (#13578 done-when #2): a real Cloud credential
# injected from the org secret, bypassing the interactive device-code /
# OAuth browser leg that XCTSkips on the simulator. ELIZACLOUD_API_KEY is
# the same secret launch-hardening-8756.yml uses for its fresh-agent probe.
ELIZA_CLOUD_AUTH_TOKEN: ${{ secrets.ELIZACLOUD_API_KEY }}
# ORIGIN ONLY — cloud-provisioning-e2e.mjs appends `/api/v1/...` itself, so
# this must NOT carry an `/api/v1` suffix (unlike the sandbox-live-smoke
# `_URL` var, which does). Overridable via workflow_dispatch to target a
# throwaway org; defaults to production Cloud otherwise.
ELIZA_CLOUD_API_BASE: ${{ github.event.inputs.cloud_api_base || 'https://api.elizacloud.ai' }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: 24
- name: Run iOS cloud runtime-mode provisioning smoke (#13578)
# Gating check for the cloud runtime mode. With the seed present this is a
# HARD gate (no continue-on-error): a regression in cloud agent creation,
# Hetzner provisioning, or the provisioned runtime never answering fails
# the scheduled run. cloud-provisioning-e2e.mjs provisions a real agent,
# waits for its runtime URL, and probes /api/status|health|me — the same
# programmatic cloud path the iOS `ios-e2e.mjs --cloud` route calls. When
# the secret is absent (forks without secret access) the lane records an
# explicit skip in the step summary rather than silently passing green.
run: |
set -euo pipefail
if [ -z "${ELIZA_CLOUD_AUTH_TOKEN:-}" ]; then
{
echo "### iOS cloud runtime-mode smoke"
echo ""
echo "skipped: missing ELIZACLOUD_API_KEY secret (no Cloud session seed available)"
} >> "$GITHUB_STEP_SUMMARY"
echo "iOS cloud runtime-mode smoke skipped: missing ELIZACLOUD_API_KEY secret"
exit 0
fi
mkdir -p packages/app/test-results/ios-cloud-mode
node packages/app/scripts/cloud-provisioning-e2e.mjs \
--fresh-agent \
--agent-name "Eliza iOS Cloud Runtime-Mode Smoke" \
--report packages/app/test-results/ios-cloud-mode/cloud-provisioning.json \
2>&1 | tee packages/app/test-results/ios-cloud-mode/smoke.log
- name: Upload iOS cloud runtime-mode evidence
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: ios-cloud-mode
path: packages/app/test-results/ios-cloud-mode/**
if-no-files-found: warn
retention-days: 7
build-android:
name: Android Debug Build
needs: changes
if: github.event_name != 'pull_request' || needs.changes.outputs.android == 'true'
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 30
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
# Android cloud-debug smoke sets ELIZA_ANDROID_SKIP_FORK_LLAMA_LIB=1
# below, so recursive native inference submodules are not part of this
# contract. Avoid failing app smoke on optional nested native pins.
submodules: false
- name: Setup Java 21
uses: actions/setup-java@ad2b38190b15e4d6bdf0c97fb4fca8412226d287
with:
distribution: temurin
java-version: 21
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
with:
node-version: 24
- name: Setup Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699
with:
packages: "platform-tools platforms;android-34 build-tools;34.0.0"
- name: Setup workspace dependencies
uses: ./.github/actions/setup-bun-workspace
with:
bun-version: ${{ env.BUN_VERSION }}
install-command: bun install --ignore-scripts --no-frozen-lockfile
install-native-deps: "false"
run-postinstall: "true"
skip-avatar-clone: "true"
no-vision-deps: "true"
- name: Generate protobuf types
# Restores the step that was on the last green Mobile Build Smoke
# run (52ab686a, 2026-05-09). Without these generated types in
# `packages/core/src/types/generated/`, `@elizaos/core`'s build
# fails downstream, which cascades into `@elizaos/shared` and
# vite config resolution.
run: |
if [ -d packages/schemas ] && [ -f packages/schemas/buf.gen.yaml ] && [ ! -d packages/core/src/types/generated ]; then
cd packages/schemas
node node_modules/@bufbuild/buf/install.js 2>/dev/null || true
bunx @bufbuild/buf@1.67.0 generate
fi
- name: Build Vite config package deps
# vite.config.ts imports dist-only workspace packages
# (`@elizaos/shared` and `@elizaos/ui/config/app-config`), and the
# app aliases plugin-browser source which imports `@elizaos/vault`.
# Build those package chains before Vite evaluates/bundles the app.
run: bun turbo run build --filter=@elizaos/ui --filter=@elizaos/vault
- name: Accept Android SDK licenses
run: yes | sdkmanager --licenses >/dev/null 2>&1 || true
# Build the mobile agent bundle (Phase D) before Capacitor's
# Android sync needs it. The bundler dedupes @elizaos/core,
# @elizaos/shared, and @elizaos/plugin-sql to their src trees, so
# the workspace plugin packages have to be present in
# node_modules. setup-bun-workspace handles install; we just kick
# off the bundle here.
- name: Build mobile agent bundle
run: bun run --cwd packages/agent build:mobile
# Guard against the May-5 Bun.build TLA regression that required
# hot-patching `s/init_eliza()/await init_eliza()/` on every device
# bundle. The fix is in 12bfccb481 (lazy plugin-sql resolution from
# ensureCoreStaticPluginsRegistered); a regression would re-emit
# `var init_eliza = __esm(async () => {...})`. See
# docs/porting/on-device-quantization-porting-plan.md §"Current
# state on the AOSP image" for context.
- name: Verify init_eliza is sync (no TLA regression)
run: |
BUNDLE=packages/agent/dist-mobile/agent-bundle.js
if grep -qE 'var init_eliza\b[^=]*= __esm\(async ' "$BUNDLE"; then
echo "::error::init_eliza re-emitted as async __esm — Bun.build cross-module TLA regression"
exit 1
fi
echo "init_eliza sync emit confirmed; no TLA regression"
- name: Build Android cloud debug app
run: node --max-old-space-size=8192 packages/app-core/scripts/run-mobile-build.mjs android-cloud-debug
env:
NODE_OPTIONS: "--max-old-space-size=8192"
# This smoke job intentionally installs no native deps.
# Real local/AOSP Android builds still fail if fork llama libs are
# missing; the opt-out only lets CI validate Capacitor/Gradle wiring.
ELIZA_ANDROID_SKIP_FORK_LLAMA_LIB: "1"
ELIZA_MOBILE_BUILD_TIMING_OUT: ${{ runner.temp }}/android-cloud-debug-build-timing.json
ELIZA_MOBILE_BUILD_TIMING_TARGET: android-apk
- name: Check Android cloud-debug build-time budget
run: |
bun run --cwd packages/app perf:startup-budget -- \
--build-timing "$RUNNER_TEMP/android-cloud-debug-build-timing.json" \
--out "$RUNNER_TEMP/android-cloud-debug-build-budget-report.json"
- name: Upload Android cloud-debug build budget report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: android-cloud-debug-build-budget
path: |
${{ runner.temp }}/android-cloud-debug-build-timing.json
${{ runner.temp }}/android-cloud-debug-build-budget-report.json
if-no-files-found: warn
retention-days: 7
- name: Verify APK exists
run: |
APK_PATH="packages/app-core/platforms/android/app/build/outputs/apk/debug/app-debug.apk"
if [ ! -f "$APK_PATH" ]; then
echo "::error::Debug APK not found at $APK_PATH"
exit 1
fi
APK_SIZE=$(stat -c%s "$APK_PATH" 2>/dev/null || stat -f%z "$APK_PATH")
echo "Debug APK built: $APK_PATH ($APK_SIZE bytes)"
- name: Upload debug APK
if: ${{ hashFiles('packages/app-core/platforms/android/app/build/outputs/apk/debug/app-debug.apk') != '' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
with:
name: android-debug-apk
path: packages/app-core/platforms/android/app/build/outputs/apk/debug/app-debug.apk
retention-days: 7
if-no-files-found: error
@@ -0,0 +1,160 @@
name: mobile-resource-workbench
# On-device resource profiling for local inference (issue #8800): battery, RSS,
# prefill/decode tok/s, TTFT, thermal timeline — per-tier budgets + a report.
#
# Layered like local-inference-bench, because the device lanes are slow and
# hardware-bound:
#
# 1. harness-smoke (PR path-filtered + nightly + dispatch): runs the pure
# aggregation/budget unit tests and the runner's off-device skip path on a
# GitHub-hosted runner. Fast, no device — the only lane that can touch PRs,
# and only when the workbench files change.
# 2. android-device (dispatch only): builds the WebView-debuggable APK and runs
# the workbench against a real arm64 device on the self-hosted pool, then
# uploads the report + raw results.
# 3. ios-sim (dispatch only): boots a simulator, runs the workbench
# (RSS/tok-s/TTFT; battery/thermal are physical-only), pulls MetricKit, and
# uploads the report.
#
# Device lanes are kept OFF PR-blocking lanes until per-device baselines in
# budgets.json are stable (most budgets are still `null` = no-baseline).
on:
schedule:
# 05:30 UTC — after the nightly publish window, like local-inference-bench.
- cron: "30 5 * * *"
pull_request:
branches: [main]
paths:
- "packages/benchmarks/mobile-resource/**"
- ".github/workflows/mobile-resource-workbench.yml"
workflow_dispatch:
inputs:
tier:
description: "Eliza-1 tier to profile"
type: choice
options: [eliza-1-2b, eliza-1-4b]
default: eliza-1-2b
run_android:
description: "Run the Android device lane (self-hosted arm64)."
type: boolean
default: false
run_ios_sim:
description: "Run the iOS simulator lane."
type: boolean
default: false
android_runner:
description: "runs-on label set for the Android device lane (JSON array)."
default: '["self-hosted","linux","arm64","android-device"]'
concurrency:
group: mobile-resource-workbench-${{ github.ref }}
cancel-in-progress: true
jobs:
harness-smoke:
name: harness smoke (no device)
runs-on: ${{ fromJSON(vars.HETZNER_FLEET_ONLINE == 'false' && '["ubuntu-24.04"]' || '["self-hosted","hetzner-robot"]') }}
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "24"
- name: Unit tests (aggregation + budgets)
run: node --test packages/benchmarks/mobile-resource/metrics.test.mjs
- name: Runner off-device skip path
run: |
set -e
# No device + unreachable agent → must record { skipped } and exit 2.
set +e
MOBILE_RESOURCE_BASE_URL=http://127.0.0.1:59999 \
node packages/benchmarks/mobile-resource/run-workbench.mjs --platform=android
code=$?
set -e
if [ "$code" != "2" ]; then
echo "expected exit 2 (skipped) off-device, got $code"; exit 1
fi
- name: Report generation
run: node packages/benchmarks/mobile-resource/report.mjs
- name: Upload report
if: always()
uses: actions/upload-artifact@v7
with:
name: mobile-resource-report-smoke
path: packages/benchmarks/mobile-resource/results/report/
if-no-files-found: ignore
android-device:
name: android device profile
if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.run_android == 'true' }}
runs-on: ${{ fromJSON(github.event.inputs.android_runner) }}
timeout-minutes: 120
env:
ELIZA_MOBILE_REPO_ROOT: ${{ github.workspace }}
ELIZA_WEBVIEW_DEBUG: "1"
MOBILE_RESOURCE_TIER: ${{ github.event.inputs.tier }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: oven-sh/setup-bun@v2
- uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "21"
- uses: android-actions/setup-android@v4
- name: Install workspace
run: bun install
- name: Build WebView-debuggable APK
run: bun run --cwd packages/app build:android
- name: Run workbench (Android device)
run: |
node packages/benchmarks/mobile-resource/run-workbench.mjs \
--platform=android --tier="$MOBILE_RESOURCE_TIER" \
--device-class=android-phone || true
- name: Report
if: always()
run: node packages/benchmarks/mobile-resource/report.mjs
- name: Upload report + results
if: always()
uses: actions/upload-artifact@v7
with:
name: mobile-resource-android-${{ github.event.inputs.tier }}
path: packages/benchmarks/mobile-resource/results/
if-no-files-found: ignore
ios-sim:
name: ios simulator profile
if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.run_ios_sim == 'true' }}
runs-on: [self-hosted, eliza-e2e-macos]
timeout-minutes: 120
env:
MOBILE_RESOURCE_TIER: ${{ github.event.inputs.tier }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: oven-sh/setup-bun@v2
- name: Install workspace
run: bun install
- name: Boot a simulator
run: |
xcrun simctl boot "iPhone 16 Pro" || true
xcrun simctl list devices booted
- name: Run workbench (iOS simulator)
run: |
node packages/benchmarks/mobile-resource/run-workbench.mjs \
--platform=ios --tier="$MOBILE_RESOURCE_TIER" \
--device-class=ios-phone || true
- name: Report
if: always()
run: node packages/benchmarks/mobile-resource/report.mjs
- name: Upload report + results
if: always()
uses: actions/upload-artifact@v7
with:
name: mobile-resource-ios-${{ github.event.inputs.tier }}
path: packages/benchmarks/mobile-resource/results/
if-no-files-found: ignore

Some files were not shown because too many files have changed in this diff Show More