Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Waiting to run
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Waiting to run
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Waiting to run
Quality and performance benchmark for eliza-1 models. Measures structured-output accuracy and
decoding throughput across three elizaOS agent tasks — response-handler decision (should_respond),
action planner (planner), and per-action parameter extraction (action:<name>) — comparing
eliza-1's unguided, GBNF-guided, and strict-guided decoding modes against a Cerebras reference
baseline (Llama-3.1-8B for tiers up to 9B; GPT-OSS-120B for the 27B tier).
Quick Start
# Run all tasks and modes (requires eliza-1 GGUF on disk or CEREBRAS_API_KEY)
bun run --cwd packages/benchmarks/eliza-1 start
# Run harness tests only — no inference keys or GGUF needed
bun run --cwd packages/benchmarks/eliza-1 test# Specific task + mode + tier
bun run --cwd packages/benchmarks/eliza-1 start \
--task should_respond --mode guided --tier eliza-1-9b --n 5
Vision CUA sub-harness
The vision-cua-e2e/ subdirectory is an integration scaffold for the eliza-1 vision +
plugin-computeruse loop (capture → tile → describe → OCR → ground → click → verify). It runs
fully in stub mode (no inference, no OS mouse) out of the box.
bun run --cwd packages/benchmarks/eliza-1/vision-cua-e2e test
See AGENTS.md for full flag reference, fixture derivation, and real-mode wiring.