wehub-resource-sync
426e9eeabd
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Waiting to run
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Waiting to run
Build Agent Image / build-and-push (push) Waiting to run
Chat shell gestures / Chat shell gesture + parity e2e (push) Waiting to run
ci / test (push) Waiting to run
ci / lint-and-format (push) Waiting to run
ci / build (push) Waiting to run
ci / dev-startup (push) Waiting to run
Cloud Gateway Discord / Test (push) Waiting to run
Cloud Gateway Webhook / Test (push) Waiting to run
Cloud Tests / lint-and-types (push) Waiting to run
Cloud Tests / unit-tests (push) Waiting to run
Cloud Tests / integration-tests (push) Waiting to run
Cloud Tests / e2e-tests (push) Blocked by required conditions
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
Deploy Apps Worker (Product 2) / Determine environment (push) Waiting to run
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Blocked by required conditions
Deploy Eliza Provisioning Worker / Determine environment (push) Waiting to run
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Blocked by required conditions
Dev Smoke / Classify changed paths (push) Waiting to run
Dev Smoke / bun run dev onboarding chat (push) Blocked by required conditions
Dev Smoke / Vite HMR dependency-level smoke (push) Blocked by required conditions
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Waiting to run
gitleaks / gitleaks (push) Waiting to run
Markdown Links / Relative Markdown Links (push) Waiting to run
Publish @elizaos/example-code / check_npm (push) Waiting to run
Publish @elizaos/example-code / publish_npm (push) Blocked by required conditions
Publish @elizaos/plugin-elizacloud / verify_version (push) Waiting to run
Publish @elizaos/plugin-elizacloud / publish_npm (push) Blocked by required conditions
Quality (Extended) / Homepage Build (PR smoke) (push) Waiting to run
Quality (Extended) / Comment-only diff guard (push) Waiting to run
Quality (Extended) / Format + Type Safety Ratchet (push) Waiting to run
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Waiting to run
Quality (Extended) / Develop Gate (lint) (push) Waiting to run
Sandbox Live Smoke / Sandbox live smoke (push) Waiting to run
Snap Build & Test / Build Snap (amd64) (push) Waiting to run
Snap Build & Test / Build Snap (arm64) (push) Waiting to run
supply-chain / sbom (push) Waiting to run
supply-chain / vulnerability-scan (push) Waiting to run
Build, Push & Deploy to Phala Cloud / build-and-push (push) Waiting to run
Test Packaging / Validate Packaging Configs (push) Waiting to run
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Waiting to run
Test Packaging / Pack & Test JS Tarballs (push) Waiting to run
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Waiting to run
UI Fixture E2E / ui-fixture-e2e (push) Waiting to run
UI Fixture E2E / fixture-e2e (push) Waiting to run
UI Story Gate / story-gate (push) Waiting to run
vault-ci / test (macos-latest) (push) Waiting to run
vault-ci / test (ubuntu-latest) (push) Waiting to run
vault-ci / test (windows-latest) (push) Waiting to run
vault-ci / app-core wiring tests (push) Waiting to run
verify-patches / verify patches/CHECKSUMS.sha256 (push) Waiting to run
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Waiting to run
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Waiting to run
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Waiting to run
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Waiting to run
Voice Benchmark Smoke / voice bench smoke summary (push) Blocked by required conditions
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Waiting to run
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Waiting to run
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Waiting to run
Test Packaging / Build & Test PyPI Package (push) Waiting to run
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
550 lines
25 KiB
JSON
550 lines
25 KiB
JSON
{
|
|
"schema": "gpt55_harvest_manifest",
|
|
"schemaVersion": 1,
|
|
"generatedAt": "2026-07-02T22:59:34.130Z",
|
|
"repoRoot": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline",
|
|
"goal": "Run every elizaOS scenario+benchmark+e2e through gpt-5.5 (Codex subscription), harvest correct eliza_native_v1 trajectories, GEPA-repair failures, fine-tune on Nebius.",
|
|
"provider": {
|
|
"mechanism": "ELIZA_CHAT_VIA_CLI CLI-subscription backend (packages/core/src/testing/live-provider.ts selectCliProvider)",
|
|
"backend": "codex",
|
|
"model": "gpt-5.5",
|
|
"modelOverrideEnv": "ELIZA_CLI_CODEX_MODEL",
|
|
"plugin": "@elizaos/plugin-cli-inference",
|
|
"credentialsPath": "~/.codex/auth.json (ChatGPT-OAuth; eliza never sees the token)",
|
|
"env": {
|
|
"ELIZA_CHAT_VIA_CLI": "codex",
|
|
"ELIZA_CLI_CODEX_MODEL": "gpt-5.5"
|
|
},
|
|
"note": "Stage-1 leg S1 proves ONE real scenario through this seam live. The driver consumes S1's proven provider env verbatim."
|
|
},
|
|
"trajectoryFormat": {
|
|
"name": "eliza_native_v1",
|
|
"definedIn": "packages/core/src/services/trajectory-types.ts (ElizaNativeTrajectoryRow)",
|
|
"contract": "packages/training/docs/dataset/CANONICAL_RECORD.md",
|
|
"converter": "packages/scenario-runner/src/native-export.ts (exportScenarioNativeJsonl)",
|
|
"trainingPrep": "packages/training/scripts/prepare_eliza1_trajectory_dataset.py"
|
|
},
|
|
"families": {
|
|
"scenario": {
|
|
"kind": "scenario",
|
|
"emitsTrajectory": "native",
|
|
"trajectoryFormat": "eliza_native_v1",
|
|
"runnerCli": "packages/scenario-runner/src/cli.ts",
|
|
"tsconfig": "tsconfig.json",
|
|
"runInvocationTemplate": [
|
|
"bun",
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"run",
|
|
"<DIR>",
|
|
"--scenario",
|
|
"<ID>",
|
|
"--report",
|
|
"<REPORT>",
|
|
"--run-dir",
|
|
"<RUNDIR>",
|
|
"--export-native",
|
|
"<NATIVE>"
|
|
],
|
|
"trajectoryLands": {
|
|
"report": "<RUNDIR>/report.json (or --report path): aggregate + per-scenario status",
|
|
"perTurn": "<RUNDIR>/trajectories/<agentId>/<trajId>.json (RecordedTrajectory)",
|
|
"native": "<NATIVE>: eliza_native_v1 JSONL (rows carry scenarioStatus + judgeScore)",
|
|
"manifest": "<NATIVE>.manifest.json"
|
|
},
|
|
"verdictSource": "report .scenarios[].status === 'passed' | native row.scenarioStatus",
|
|
"providerSeam": "ELIZA_CHAT_VIA_CLI=codex → provider 'cli', model gpt-5.5, plugin @elizaos/plugin-cli-inference, reads ~/.codex/auth.json",
|
|
"items": [
|
|
{
|
|
"id": "packages__test__scenarios",
|
|
"dir": "packages/test/scenarios",
|
|
"scenarioFiles": 713,
|
|
"baseScenarios": 707,
|
|
"expandedScenarios": 7777,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"packages/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
},
|
|
{
|
|
"id": "packages__scenario-runner__test__scenarios",
|
|
"dir": "packages/scenario-runner/test/scenarios",
|
|
"scenarioFiles": 42,
|
|
"baseScenarios": 42,
|
|
"expandedScenarios": 420,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"packages/scenario-runner/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
},
|
|
{
|
|
"id": "plugins__plugin-personal-assistant__test__scenarios",
|
|
"dir": "plugins/plugin-personal-assistant/test/scenarios",
|
|
"scenarioFiles": 197,
|
|
"baseScenarios": 197,
|
|
"expandedScenarios": 2167,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"plugins/plugin-personal-assistant/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
},
|
|
{
|
|
"id": "plugins__plugin-app-control__test__scenarios",
|
|
"dir": "plugins/plugin-app-control/test/scenarios",
|
|
"scenarioFiles": 15,
|
|
"baseScenarios": 15,
|
|
"expandedScenarios": 165,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"plugins/plugin-app-control/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
},
|
|
{
|
|
"id": "plugins__plugin-health__test__scenarios",
|
|
"dir": "plugins/plugin-health/test/scenarios",
|
|
"scenarioFiles": 8,
|
|
"baseScenarios": 8,
|
|
"expandedScenarios": 88,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"plugins/plugin-health/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
},
|
|
{
|
|
"id": "plugins__plugin-cloud-apps__test__scenarios",
|
|
"dir": "plugins/plugin-cloud-apps/test/scenarios",
|
|
"scenarioFiles": 1,
|
|
"baseScenarios": 1,
|
|
"expandedScenarios": 11,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"plugins/plugin-cloud-apps/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
},
|
|
{
|
|
"id": "plugins__plugin-agent-orchestrator__test__scenarios",
|
|
"dir": "plugins/plugin-agent-orchestrator/test/scenarios",
|
|
"scenarioFiles": 8,
|
|
"baseScenarios": 8,
|
|
"expandedScenarios": 88,
|
|
"discover": {
|
|
"cmd": "bun",
|
|
"args": [
|
|
"--conditions",
|
|
"eliza-source",
|
|
"--tsconfig-override",
|
|
"<TSCONFIG>",
|
|
"packages/scenario-runner/src/cli.ts",
|
|
"list",
|
|
"plugins/plugin-agent-orchestrator/test/scenarios"
|
|
],
|
|
"parse": "json-lines-scenario-ids"
|
|
}
|
|
}
|
|
]
|
|
},
|
|
"benchmark": {
|
|
"kind": "benchmark",
|
|
"emitsTrajectory": "wiring-needed",
|
|
"trajectoryFormat": "eliza_native_v1 (after wiring)",
|
|
"source": "packages/benchmarks/registry/commands.py",
|
|
"listCommand": "python3 -m benchmarks.orchestrator list-benchmarks (cwd=packages)",
|
|
"runInvocationTemplate": "python3 -m benchmarks.orchestrator run --benchmarks <ID> --provider cli --model gpt-5.5 (cwd=packages)",
|
|
"resultLands": "packages/benchmarks/benchmark_results/** (per-benchmark JSON; locate_result() in registry/commands.py). GITIGNORED.",
|
|
"trajectoryWiring": "The ~25 eliza-adapter-routed benchmarks boot a real AgentRuntime (serves /api/benchmark/message). Set ELIZA_SAVE_TRAJECTORIES=1 + ELIZA_TRAJECTORY_DIR=<dir> on the adapter runtime, then run packages/scenario-runner native-export over <dir> to convert RecordedTrajectory JSON → eliza_native_v1. Non-eliza-adapter benchmarks (standard/*, python-only) do NOT boot the runtime and cannot emit native trajectories.",
|
|
"providerSeam": "orchestrator --provider cli maps to ELIZA_CHAT_VIA_CLI=codex for eliza-adapter benchmarks; verify per-adapter provider plumbing before Stage 2.",
|
|
"adapterCount": 53,
|
|
"adapters": [
|
|
{
|
|
"id": "abliteration-robustness",
|
|
"dir": "abliteration-robustness",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "action-calling",
|
|
"dir": "action-calling",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "adhdbench",
|
|
"dir": "adhdbench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/adhdbench"
|
|
},
|
|
{
|
|
"id": "agentbench",
|
|
"dir": "agentbench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/agentbench"
|
|
},
|
|
{
|
|
"id": "app-eval",
|
|
"dir": "app-eval",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/app-eval"
|
|
},
|
|
{
|
|
"id": "bfcl",
|
|
"dir": "bfcl",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "clawbench",
|
|
"dir": "clawbench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/clawbench"
|
|
},
|
|
{
|
|
"id": "configbench",
|
|
"dir": "configbench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/configbench"
|
|
},
|
|
{
|
|
"id": "context_bench",
|
|
"dir": "context-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "eliza_1",
|
|
"dir": "eliza-1",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/eliza-1"
|
|
},
|
|
{
|
|
"id": "eliza_replay",
|
|
"dir": "eliza-adapter",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/eliza-adapter"
|
|
},
|
|
{
|
|
"id": "experience",
|
|
"dir": "experience",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/experience"
|
|
},
|
|
{
|
|
"id": "framework",
|
|
"dir": "framework",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "gauntlet",
|
|
"dir": "gauntlet",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/gauntlet"
|
|
},
|
|
{
|
|
"id": "gsm8k",
|
|
"dir": "standard",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "hermes_swe_env",
|
|
"dir": "hermes-adapter",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
|
|
},
|
|
{
|
|
"id": "hermes_tblite",
|
|
"dir": "hermes-adapter",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
|
|
},
|
|
{
|
|
"id": "hermes_terminalbench_2",
|
|
"dir": "hermes-adapter",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
|
|
},
|
|
{
|
|
"id": "hermes_yc_bench",
|
|
"dir": "hermes-adapter",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
|
|
},
|
|
{
|
|
"id": "humaneval",
|
|
"dir": "standard",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "hyperliquid_bench",
|
|
"dir": "HyperliquidBench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "interrupt_bench",
|
|
"dir": "interrupt-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/interrupt-bench"
|
|
},
|
|
{
|
|
"id": "lifeops_bench",
|
|
"dir": "lifeops-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/lifeops-bench"
|
|
},
|
|
{
|
|
"id": "mind2web",
|
|
"dir": "mind2web",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "mint",
|
|
"dir": "mint",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "mmau",
|
|
"dir": "mmau-audio",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/mmau-audio"
|
|
},
|
|
{
|
|
"id": "mmlu",
|
|
"dir": "standard",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "mt_bench",
|
|
"dir": "standard",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "openclaw_bench",
|
|
"dir": "openclaw-benchmark",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/openclaw-benchmark"
|
|
},
|
|
{
|
|
"id": "orchestrator_lifecycle",
|
|
"dir": "orchestrator_lifecycle",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "osworld",
|
|
"dir": "OSWorld",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/OSWorld"
|
|
},
|
|
{
|
|
"id": "personality_bench",
|
|
"dir": "personality-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/personality-bench"
|
|
},
|
|
{
|
|
"id": "realm",
|
|
"dir": "realm",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "recall_bench",
|
|
"dir": "recall-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/recall-bench"
|
|
},
|
|
{
|
|
"id": "rlm_bench",
|
|
"dir": "rlm-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/rlm-bench"
|
|
},
|
|
{
|
|
"id": "scambench",
|
|
"dir": "scambench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "social_alpha",
|
|
"dir": "social-alpha",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/social-alpha"
|
|
},
|
|
{
|
|
"id": "solana",
|
|
"dir": "solana",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "swe_bench",
|
|
"dir": "swe_bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "swe_bench_orchestrated",
|
|
"dir": "swe_bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "tau_bench",
|
|
"dir": "tau-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/tau-bench"
|
|
},
|
|
{
|
|
"id": "terminal_bench",
|
|
"dir": "terminal-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/terminal-bench"
|
|
},
|
|
{
|
|
"id": "three_agent_dialogue",
|
|
"dir": "three-agent-dialogue",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/three-agent-dialogue"
|
|
},
|
|
{
|
|
"id": "trajectory_replay",
|
|
"dir": "standard",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "trust",
|
|
"dir": "trust",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/trust"
|
|
},
|
|
{
|
|
"id": "vending_bench",
|
|
"dir": "vending-bench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/vending-bench"
|
|
},
|
|
{
|
|
"id": "vision_language",
|
|
"dir": "vision-language",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/vision-language"
|
|
},
|
|
{
|
|
"id": "visualwebbench",
|
|
"dir": "visualwebbench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
},
|
|
{
|
|
"id": "voiceagentbench",
|
|
"dir": "voiceagentbench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/voiceagentbench"
|
|
},
|
|
{
|
|
"id": "voicebench",
|
|
"dir": "voicebench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/voicebench"
|
|
},
|
|
{
|
|
"id": "voicebench_quality",
|
|
"dir": "voicebench-quality",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/voicebench-quality"
|
|
},
|
|
{
|
|
"id": "webshop",
|
|
"dir": "webshop",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/webshop"
|
|
},
|
|
{
|
|
"id": "woobench",
|
|
"dir": "woobench",
|
|
"cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
|
|
}
|
|
]
|
|
},
|
|
"e2e": {
|
|
"kind": "e2e",
|
|
"emitsTrajectory": "wiring-needed",
|
|
"trajectoryFormat": "eliza_native_v1 (after wiring)",
|
|
"runInvocationTemplate": "ELIZA_SAVE_TRAJECTORIES=1 ELIZA_TRAJECTORY_DIR=<dir> bun --conditions eliza-source vitest run <testFile>",
|
|
"trajectoryWiring": "These vitest lanes drive a real AgentRuntime via createScenarioRuntime/real-runtime helpers. The runtime's JsonFileTrajectoryRecorder writes RecordedTrajectory JSON when ELIZA_SAVE_TRAJECTORIES=1 + ELIZA_TRAJECTORY_DIR are set. Convert with scenario-runner native-export. Verdict = vitest pass/fail per file (coarser than per-scenario).",
|
|
"providerSeam": "Same ELIZA_CHAT_VIA_CLI=codex seam; the live-provider helper (packages/app-core/test/helpers/live-provider.ts) already recognizes CLI backends.",
|
|
"liveLaneCount": 51,
|
|
"lanes": [
|
|
"packages/app-core/test/app/memory-relationships.real.e2e.test.ts",
|
|
"packages/app-core/test/app/qa-checklist.real.e2e.test.ts",
|
|
"packages/app-core/test/app/streaming-visible-text.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/action-invocation.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/agent-runtime.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/cloud-auth.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/cloud-login-persist.real.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/cloud-providers.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/config-routes.real.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/connector-health.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/conversation-deterministic.real.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/database-conversation.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/experience-extraction.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/farcaster-connector.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/feishu-connector.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/health-and-dev-stack.real.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/lens-connector.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/matrix-connector.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/nostr-connector.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/page-scoped-chat.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/permissions-and-sensitive.real.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/personality-routing.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/plugin-lifecycle.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/real-runtime-helpers.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/runtime-debug.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/telegram-connector.live.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/views-interact-ws-roundtrip.real.e2e.test.ts",
|
|
"packages/app-core/test/live-agent/views-routes.real.e2e.test.ts",
|
|
"plugins/plugin-computeruse/src/__tests__/runtime.live.e2e.test.ts",
|
|
"plugins/plugin-computeruse/test/computeruse.real.e2e.test.ts",
|
|
"plugins/plugin-documents/test/documents-api.live.e2e.test.ts",
|
|
"plugins/plugin-form/test/form-plugin.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/assistant-user-journeys.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/journey-cerebras-eval.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/knowledge-graph-service.real.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/lifeops-activity-signals.remote.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/lifeops-chat.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/lifeops-gmail-chat.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/lifeops-memory.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/lifeops-screen-context.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/lifeops-signal.real.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/reminder-review-job.real.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/selfcontrol-chat.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/selfcontrol-desktop.live.e2e.test.ts",
|
|
"plugins/plugin-personal-assistant/test/selfcontrol-dev.live.e2e.test.ts",
|
|
"plugins/plugin-shopify/test/shopify-api.real.e2e.test.ts",
|
|
"plugins/plugin-sql/src/__tests__/integration/pglite-e2e.real.e2e.test.ts",
|
|
"plugins/plugin-task-coordinator/test/coding-agent-codex-artifact.live.e2e.test.ts",
|
|
"plugins/plugin-task-coordinator/test/orchestrator-workbench.live.e2e.test.ts",
|
|
"plugins/plugin-training/test/training-api.live.e2e.test.ts",
|
|
"plugins/plugin-vision/test/vision.real.e2e.test.ts"
|
|
],
|
|
"scriptedRealServices": [
|
|
"packages/scenario-runner/scripts/real-llm-attachment-smoke.mjs",
|
|
"packages/scenario-runner/scripts/real-service-audio-roundtrip.mjs",
|
|
"packages/scenario-runner/scripts/real-service-voice-e2e.mjs"
|
|
]
|
|
}
|
|
}
|
|
} |