chore: import upstream snapshot with attribution
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Waiting to run
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Waiting to run
Build Agent Image / build-and-push (push) Waiting to run
Chat shell gestures / Chat shell gesture + parity e2e (push) Waiting to run
ci / test (push) Waiting to run
ci / lint-and-format (push) Waiting to run
ci / build (push) Waiting to run
ci / dev-startup (push) Waiting to run
Cloud Gateway Discord / Test (push) Waiting to run
Cloud Gateway Webhook / Test (push) Waiting to run
Cloud Tests / lint-and-types (push) Waiting to run
Cloud Tests / unit-tests (push) Waiting to run
Cloud Tests / integration-tests (push) Waiting to run
Cloud Tests / e2e-tests (push) Blocked by required conditions
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
Deploy Apps Worker (Product 2) / Determine environment (push) Waiting to run
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Blocked by required conditions
Deploy Eliza Provisioning Worker / Determine environment (push) Waiting to run
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Blocked by required conditions
Dev Smoke / Classify changed paths (push) Waiting to run
Dev Smoke / bun run dev onboarding chat (push) Blocked by required conditions
Dev Smoke / Vite HMR dependency-level smoke (push) Blocked by required conditions
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Waiting to run
gitleaks / gitleaks (push) Waiting to run
Markdown Links / Relative Markdown Links (push) Waiting to run
Publish @elizaos/example-code / check_npm (push) Waiting to run
Publish @elizaos/example-code / publish_npm (push) Blocked by required conditions
Publish @elizaos/plugin-elizacloud / verify_version (push) Waiting to run
Publish @elizaos/plugin-elizacloud / publish_npm (push) Blocked by required conditions
Quality (Extended) / Homepage Build (PR smoke) (push) Waiting to run
Quality (Extended) / Comment-only diff guard (push) Waiting to run
Quality (Extended) / Format + Type Safety Ratchet (push) Waiting to run
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Waiting to run
Quality (Extended) / Develop Gate (lint) (push) Waiting to run
Sandbox Live Smoke / Sandbox live smoke (push) Waiting to run
Snap Build & Test / Build Snap (amd64) (push) Waiting to run
Snap Build & Test / Build Snap (arm64) (push) Waiting to run
supply-chain / sbom (push) Waiting to run
supply-chain / vulnerability-scan (push) Waiting to run
Build, Push & Deploy to Phala Cloud / build-and-push (push) Waiting to run
Test Packaging / Validate Packaging Configs (push) Waiting to run
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Waiting to run
Test Packaging / Pack & Test JS Tarballs (push) Waiting to run
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Waiting to run
UI Fixture E2E / ui-fixture-e2e (push) Waiting to run
UI Fixture E2E / fixture-e2e (push) Waiting to run
UI Story Gate / story-gate (push) Waiting to run
vault-ci / test (macos-latest) (push) Waiting to run
vault-ci / test (ubuntu-latest) (push) Waiting to run
vault-ci / test (windows-latest) (push) Waiting to run
vault-ci / app-core wiring tests (push) Waiting to run
verify-patches / verify patches/CHECKSUMS.sha256 (push) Waiting to run
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Waiting to run
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Waiting to run
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Waiting to run
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Waiting to run
Voice Benchmark Smoke / voice bench smoke summary (push) Blocked by required conditions
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Waiting to run
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Waiting to run
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Waiting to run
Test Packaging / Build & Test PyPI Package (push) Waiting to run
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:43:05 +08:00
commit 426e9eeabd
41828 changed files with 9656266 additions and 0 deletions
+22
View File
@@ -0,0 +1,22 @@
node_modules
dist
elizaConfig.yaml
custom_actions/
cache/
coverage/
*.tsbuildinfo
:memory:
# Prevent accidental tsc/IDE emit artifacts next to .ts (build output belongs in dist/).
# Do NOT ignore src/types/generated/** — those protobuf .d.ts files are checked in (buf).
src/**/*.js
src/**/*.d.ts
src/**/*.d.ts.map
!src/types/generated/**/*.js
!src/types/generated/**/*.d.ts
!src/types/generated/**/*.d.ts.map
!src/**/*.test.js
# Protobuf TS source generated by buf (CI runs `Generate protobuf types`
# before each build; we never check the generated .ts files in).
src/types/generated/**/*.ts
+6
View File
@@ -0,0 +1,6 @@
*
!dist/**
!package.json
!readme.md
!tsup.config.ts
+210
View File
@@ -0,0 +1,210 @@
# @elizaos/core
The runtime heart of elizaOS: `AgentRuntime`, the plugin abstractions (actions / providers / evaluators / services / models / routes / events), the canonical type system, and the supporting subsystems (memory, search, settings, scheduling, prompts). Almost every other `@elizaos/*` package and plugin imports from here.
## Role
`@elizaos/core` defines the contracts an Eliza agent runs on and the runtime that executes them. Plugins implement `Plugin` and the runtime wires their actions/providers/evaluators/services into the message-handling loop. Consumed by `@elizaos/agent` (which also hosts the HTTP API server), `@elizaos/app-core` (the API + dashboard host), and every plugin. It builds to three targets (Node, browser, edge) via conditional exports — keep Node-only code out of the browser/edge entries.
## Layout
```
src/
index.ts Default barrel — re-exports index.node + a few @elizaos/contracts type shims
index.node.ts Full Node API surface (the real export list — start here)
index.browser.ts Browser-safe subset (no fs/process-bound modules)
index.edge.ts Edge-runtime subset
runtime.ts AgentRuntime class (~9000 lines, `class AgentRuntime implements IAgentRuntime` at L718); the central orchestrator
runtime-composition.ts loadCharacters / createRuntimes / settings merge (Node-only boot helpers)
runtime-env.ts Runtime environment + state resolution
plugin.ts Plugin load/validate/resolve: loadPlugin, resolvePlugins, validatePlugin, resolvePluginDependencies
plugin-lifecycle.ts Plugin register/unload/reload + ownership tracking
runtime/ Message loop internals: message-handler, planner-loop, turn-controller, action-catalog,
action-retrieval/routing/tiering, context-* (registry/renderer/gates), evaluator,
validated-model-call, response-grammar, system-prompt, sub-planner, trajectory-recorder
types/ Canonical type system. types/index.ts is the barrel; types/runtime.ts has IAgentRuntime;
plugin.ts, model.ts, memory.ts, state.ts, service.ts, task.ts, events.ts, schema*.ts, etc.
services/ Built-in services: task / task-scheduler, evaluator, message, relationships,
pairing, pairing-integration, pairing-migration, hook, optimized-prompt,
optimized-prompt-resolver, tool-policy, trajectories, trajectory-export, trajectory-types,
triggerScheduling, approval, embedding, followUp, analysis-mode-handler, agentEvent,
runtime-capability-service, setup-cli, setup-rpc, setup-state
features/ Self-contained capability bundles, each its own dir:
basic-capabilities (the core action/provider/evaluator/service bundle),
advanced-capabilities, advanced-memory, advanced-planning, approvals, autonomy, ballots,
documents, messaging (triage), oauth, payments, plugin-config, plugin-manager,
secrets, sub-agent-credentials, trajectories, trust, working-memory
actions/ Action plumbing: action-schema, to-tool, validate-tool-args, subaction-dispatch
providers/ First-party providers (setup-progress, skill-eligibility, linked-identities, ...)
schemas/ Drizzle table schemas + character schema. schemas/index.ts: buildBaseTables, BaseTables
database/ inMemoryAdapter (IDatabaseAdapter fallback used when ALLOW_NO_DATABASE)
contracts/ Re-exports/adapters over @elizaos/contracts (cloud-topology, first-run-options, service-routing, wallet)
generated/ Build-time generated action/provider/evaluator docs + spec-helpers (do not hand-edit)
i18n/ validation + action-search keyword data (some generated; see prebuild)
security/ redact, ssrf-adjacent input policy, spawn-env-policy, external-content, incoming-message-security
sensitive-requests/ Sensitive request policy helpers
network/ SSRF guard + secure fetch (fetch-guard, ssrf)
markdown/ media/ markdown IR/chunking; media fetch + mime/type detection
testing/ Test harness exports (live-provider, integration-runtime, http, mocks) — `@elizaos/core/testing`
capabilities/ Runtime capability index
connectors/ Connector abstractions (account-manager, connector-config, oauth-role, privacy)
plugins/ Plugin-related helpers
registries/ Registry utilities
sessions/ Session management
sandbox/ Sandbox policy
optimization/ Optimization utilities
scheduled-task/ Scheduled task helpers
validation/ Input validation utilities
constants/ Shared constants
api/ API helpers
owner-state/ Owner state tracking
messaging/ Messaging utilities
search.ts In-memory/embedding search utilities
utils.ts utils/ Shared helpers: prompts (composePromptFromState, parseKeyValueXml), batch-queue,
confirmation, read-env, state-dir, streaming, environment, plugin-loader
build.ts Custom bun-based multi-target build (Node / browser / edge + d.ts generation)
scripts/perf-settings.ts, scripts/run-e2e-smoke.mjs
```
## Key exports / surface
From `@elizaos/core` (`index.node.ts`):
- `AgentRuntime` — the runtime, `implements IAgentRuntime`.
- Plugin machinery: `loadPlugin`, `resolvePlugins`, `validatePlugin`, `isValidPluginShape`, `normalizePluginName`, `resolvePluginDependencies`.
- `logger` (re-exported from `./logger`) — the structured logger all packages use.
- Type contracts: `Plugin`, `Action`, `Provider`, `Evaluator`, `Service`, `IAgentRuntime`, `IDatabaseAdapter`, `Memory`, `State`, `Character`, `ModelType`, `UUID`, plus everything in `types/`.
- Built-in capability bundle: `basicCapabilities` / `basicActions` / `basicProviders` / `basicEvaluators` / `basicServices` (from `features/basic-capabilities/index.ts`).
- Boot/composition (Node): `loadCharacters`, `createRuntimes`, `buildBaseTables`, `InMemoryDatabaseAdapter`.
- Prompt + model helpers: `composePromptFromState`, `parseKeyValueXml`, `callModelWithValidation`, `parseAndValidate`.
Subpath entries (see `package.json` `exports`): `@elizaos/core/node`, `@elizaos/core/browser`, `@elizaos/core/roles`, `@elizaos/core/testing`, `@elizaos/core/services/*`.
This package does NOT export a `corePlugin` singleton — the foundational actions/providers/evaluators/services live in `features/basic-capabilities` and are exported as the `basic*` bundles above.
## Commands
```bash
bun run --cwd packages/core build # multi-target build via build.ts (Node + browser + edge + d.ts)
bun run --cwd packages/core build:node # Node target only
bun run --cwd packages/core build:watch # watch build (alias: dev)
bun run --cwd packages/core test # vitest run (via ../scripts/run-vitest.mjs)
bun run --cwd packages/core test:watch # vitest watch
bun run --cwd packages/core test:coverage # vitest with v8 coverage
bun run --cwd packages/core test:e2e # Playwright (playwright.config.ts)
bun run --cwd packages/core test:e2e:smoke
bun run --cwd packages/core typecheck # tsgo --noEmit -p ./tsconfig.json
bun run --cwd packages/core lint # biome check --write ./src
bun run --cwd packages/core format # biome format --write ./src
bun run --cwd packages/core clean # remove dist + emitted src artifacts
```
`prebuild` builds `@elizaos/contracts` and generates `src/i18n/generated/validation-keyword-data.ts` if missing. Depends on workspace packages `@elizaos/contracts` and `@elizaos/prompts`.
## Config / env vars
Read by the runtime (see README for the full WHY of each):
- `LOG_LEVEL`, `LOG_JSON_FORMAT`, `LOG_FILE` — logger behavior (`src/logger.ts`).
- `SECRET_SALT` — encryption salt, read by `getSalt()` in `src/settings.ts` (`ELIZA_ALLOW_DEFAULT_SECRET_SALT` overrides the production non-default check).
- `ALLOW_NO_DATABASE` — fall back to `InMemoryDatabaseAdapter` on `initialize()` when no adapter is provided (`runtime.ts`).
- `SHOULD_RESPOND_MODEL` (`small`/`large`, `services/message.ts`), `BASIC_CAPABILITIES_KEEP_RESP` (`services/message.ts`) — message/basic-capabilities behavior.
- `ELIZA_BOT_NOISE_TRIAGE` (`services/message/bot-noise-triage.ts`) — set `0` to disable the TEXT_SMALL pre-gate that triages unaddressed bot/webhook group messages before the Stage 1 RESPONSE_HANDLER call (default on).
- `ELIZA_STAGE1_GROUP_TRIAGE` (`services/message/stage1-prompt-tier.ts`) — set `0` to disable the compact Stage 1 instruction tier for unaddressed group messages and always render the full rule block (default on).
- Prompt-batcher knobs (all `PROMPT_BATCHER_*`, read in `runtime.ts`): `PROMPT_BATCHER_BATCH_SIZE`, `PROMPT_BATCHER_MAX_DRAIN_INTERVAL_MS`, `PROMPT_BATCHER_MAX_SECTIONS_PER_CALL`, `PROMPT_BATCHER_PACKING_DENSITY`, `PROMPT_BATCHER_MAX_TOKENS_PER_CALL`, `PROMPT_BATCHER_MAX_PARALLEL_CALLS`, `PROMPT_BATCHER_MODEL_SEPARATION`.
- `ELIZA_STATE_DIR` — state-dir resolution (`utils/state-dir.ts`); `ELIZA_WORKSPACE_DIR` — workspace folder (`utils/workspace-folder-config.ts`).
- `ELIZA_TRAJECTORY_LOGGING` — canonical trajectory persistence gate for both file and DB recorders (`runtime/trajectory-gate.ts`): truthy enables; non-empty falsey disables; blank is unset. Defaults are on for dev/unset `NODE_ENV`, off for `NODE_ENV=test|production`. `ELIZA_TRAJECTORY_RECORDING` is the legacy alias, and `ELIZA_DISABLE_TRAJECTORY_LOGGING=1` is the hard opt-out.
Prefer the canonical env reader in `utils/read-env.ts` over raw `process.env` (it handles legacy aliases).
### Setting / env resolution — precedence & the multi-tenant rule
Two canonical helpers own all setting/env resolution; everything else delegates:
| Helper | Source order | Use when |
| --- | --- | --- |
| `runtime.getSetting(key)` (`runtime.ts`) | character secrets → character settings → `settings.extra``settings.secrets``character.env.vars` → the constructor-provided `settings` map. **Never `process.env`.** | Inside the runtime / framework code. |
| `readEnv(key, opts)` (`utils/read-env.ts`) | `process.env[key]` (trimmed; empty string treated as unset) → `defaultValue`. | Reading an env var with no runtime in scope. |
| `resolveSetting(runtime, key, opts)` (`utils/resolve-setting.ts`) | `runtime.getSetting(key)` (coerced to string) → `readEnv(key)``defaultValue`. | Single-tenant / headless plugins that still want a dotenv fallback. |
**Resolution order:** runtime/character setting → env alias → default. The
per-agent runtime value always wins; the env fallback is the deployment default.
**WHY core `getSetting()` deliberately does NOT read `process.env`:** in a
multi-tenant process many agents share one OS environment. If `getSetting()`
fell through to `process.env`, a host secret (`OPENAI_API_KEY`,
`POSTGRES_URL`, …) set for the *box* would silently leak into *every* agent,
including ones the operator never granted it to. Keeping `getSetting()`
per-agent makes each agent's config explicit and isolated. `resolveSetting`
re-adds an opt-in env fallback for single-tenant/headless plugins **without**
changing `getSetting()` semantics — multi-tenant hosts that never call it are
unaffected.
**Host obligation (how to make dotenv values visible to `getSetting()`):**
because `getSetting()` reads the constructor-provided `settings` map and not
`process.env`, a host that wants `.env` / `process.env` values honored must fold
them into the runtime's settings at construction. `getBasicCapabilitiesSettings(character, env)`
(`runtime-composition.ts`) does exactly this — it flattens `character.settings`,
`character.secrets`, and `env` into the `Record<string,string>` handed to
adapter factories and the `AgentRuntime` constructor. Construct the runtime with
those settings and dotenv is honored; skip it and only character config is
visible.
## How to extend
- **Add an action/provider/evaluator/service to the built-in bundle:** implement against the `Action`/`Provider`/`Evaluator`/`Service` types in `types/`, then add it to the relevant array in `src/features/basic-capabilities/index.ts` (`basicActions`, `basicProviders`, `basicEvaluators`, `basicServices`). Most new capabilities should live in their own plugin package instead of here.
- **Add a runtime type/contract:** define it under `src/types/<area>.ts` and export from `src/types/index.ts`. If it should be shared with non-runtime consumers, prefer `@elizaos/contracts` and re-export via `src/contracts/`.
- **Add a DB table:** extend the schema in `src/schemas/` and wire it into `buildBaseTables` (`schemas/index.ts`); adapters in plugin-sql/localdb materialize it.
- **Touching the message loop:** the order is provider → model → action → evaluator. Logic lives in `src/runtime/` (`message-handler.ts`, `planner-loop.ts`, `turn-controller.ts`) and `runtime.ts`. Validated model output goes through `runtime/validated-model-call.ts`.
- **Browser/edge surface:** if your code is Node-only (fs, process, native deps), export it from `index.node.ts` only — never add it to `index.browser.ts` / `index.edge.ts`.
## Conventions / gotchas
- `index.node.ts` is the source of truth for the public surface; `index.ts` just re-exports it plus a few `@elizaos/contracts` type shims (kept explicit to avoid d.ts ambiguity).
- Three build targets share source — Node-only imports in shared modules break the browser/edge bundles. Verify with `build:node` vs full `build`.
- The model-output contract is `<response>` XML (with `<actions>`/`<providers>`/`<text>`); plain text is tolerated and treated as a `REPLY`.
- DB mutation methods on `IDatabaseAdapter` return `Promise<boolean>` so callers can distinguish success/failure (`types/database.ts`).
- The task system (`services/task.ts`, `services/task-scheduler.ts`) is the single place scheduled work runs; only tasks tagged `queue` are polled. Three modes: local timer, per-daemon (`startTaskScheduler`), serverless (`{ serverless: true }` + `runDueTasks()`).
- `runtime.ts` is very large (~9000 lines / ~259 KB) — navigate by symbol, not by reading top-to-bottom.
- `src/generated/` and parts of `src/i18n/generated/` are build artifacts; regenerate via prebuild rather than editing.
- Repo-wide rules (logger-only, ESM, naming, architecture) are in the root [AGENTS.md](../../AGENTS.md) — not restated here.
<!-- BEGIN: evidence-and-e2e-mandate (managed; canonical standard = repo-root AGENTS.md) -->
## ⛔ NON-NEGOTIABLE — evidence, trajectories & real end-to-end tests
> The binding, repo-wide standard is **[AGENTS.md](../../AGENTS.md)**. Read it.
> Nothing in this package is *done* until it is *proven* done — a reviewer must confirm it
> works **without reading the code**, from the artifacts you attach. This applies to **every**
> feature, fix, refactor, and chore here. "Tests pass" is not proof; "CI is green" is not proof.
- **Record AND read model trajectories.** Capture the *actual* inputs and outputs of the model
from a **live** LLM — not the deterministic proxy, not a mock: the prompt, the
providers/context, the raw model output, every tool/action call, and the result. Then **open
the trajectory and review it by hand.** A captured-but-unread trajectory is not evidence
(`packages/scenario-runner/bin/eliza-scenarios run <scenario> --report <out>`).
- **Real, full-featured E2E — no larp.** Every feature ships detailed end-to-end tests that
drive the *real* path end to end. Not the happy "front door" only: cover error paths,
edge/empty/invalid input, concurrency, roles/permissions, and adversarial input. A test that
asserts against a mock/stub/fixture standing in for the thing under test **does not count**.
If the real model/device/chain/connector/account is hard to reach, **make it reachable — that
is the work**, not an excuse to mock. If the existing tests here are shallow or mocked, fixing
them is part of your change.
- **Screenshots + logs at every phase**, plus a **complete walkthrough video/run-through** of
the entire feature or view, start to finish (`bun run test:e2e:record`).
- **Manually review every artifact the change touches** — never just the green check: client
logs (console + network), server logs (`[ClassName] …`), the model trajectories in and out,
before/after full-page screenshots, **and the domain artifacts listed below for this package.**
- **No residuals. No shortcuts.** The goal is not "done" — it is *everything* done. Clear every
blocker by the **hard path**: build the real architecture, stand up the real
model/device/service, actually test it. Never leave a TODO, a stub, a stepping-stone, or a
"follow-up." When unsure, research thoroughly, weigh the options, and ship the best,
highest-effort, production-ready version. Keep going until every possibility is exhausted.
Artifacts → attached inline in the PR (MP4 video, JPG screenshots, logs in `<details>`); attach each evidence type **or**
explicitly mark it N/A with a reason — never leave it blank. If `develop` moved and changed
behavior, **re-capture** evidence; stale proof is worse than none.
**Capture & manually review for this package — runtime / framework:**
- A **live-LLM** scenario trajectory for the runtime path you touched — provider → model → action → evaluator — with the raw `<response>` XML and every tool/action call visible and **read**.
- Backend `[ClassName]` logs proving the message loop, task scheduler, or service actually fired end to end.
- The memory/state artifacts produced — rows written, embeddings, room/world/entity records, scheduled-task rows — inspected, not assumed.
- For shared modules: `build:node` vs full `build` so the browser/edge bundles still compile.
<!-- END: evidence-and-e2e-mandate -->
+133
View File
@@ -0,0 +1,133 @@
# Changelog
## Unreleased
### Added
- **Documentation: pipeline hooks.** Single guide [docs/PIPELINE_HOOKS.md](./docs/PIPELINE_HOOKS.md) (all phases including outgoing text, stream dedupe, DPE); README links to it; code comments in `pipeline-hooks.ts`, `streaming-context.ts`, and prompt optimization types.
- **Why:** One maintained entry point beats split “outgoing-only” vs “rest of pipeline” docs that duplicate metrics/API context.
### Changed
- **Documentation:** Removed `docs/OUTGOING_CONTENT_HOOKS.md` and `docs/PIPELINE_MODEL_STREAMING_AND_DPE.md` in favor of [docs/PIPELINE_HOOKS.md](./docs/PIPELINE_HOOKS.md).
- **Why:** One hooks guide is easier to discover and update than parallel granular files.
- **Shared batch-queue subsystem (`utils/batch-queue`).** Composable building blocks: `PriorityQueue`, `BatchProcessor` (semaphore-limited concurrency + retries using `utils/retry`), `TaskDrain` (repeat `queue` tasks, optional `skipRegisterWorker` when a global worker already owns the task name), composed `BatchQueue`, and one shared `Semaphore` (re-exported from `prompt-batcher/shared.ts` for existing imports). Tests: `src/__tests__/batch-queue.test.ts`, `src/__tests__/task-drain.test.ts`.
- **Why (architecture, not a single hot path):** The runtime is not globally “batching-bound”; a minimal fix in one service could be a few lines. The goal here is **forward-looking consolidation** so embedding drains, action-index embedding, batcher affinity scheduling, and shared throttling do not each grow a bespoke queue + task + retry stack that drifts over time. One composable surface caps proliferation of incompatible queuing patterns. See `src/utils/batch-queue.ts` (module comment) and `docs/BATCH_QUEUE.md`.
- **Prompt cache hints (PromptSegment, promptSegments).** The core can pass ordered segments with stability metadata so providers can use prompt-caching APIs. `GenerateTextParams` now has optional `promptSegments?: PromptSegment[]` where each segment is `{ content: string; stable: boolean }`. When set, `prompt` must equal `promptSegments.map(s => s.content).join("")`.
- **Why:** Repeated calls often share the same instructions/format while only context changes; provider caches (Anthropic ephemeral, OpenAI/Gemini prefix) can reuse tokens for the stable part, reducing cost and latency. A single invariant lets providers opt in or ignore segments without breaking behavior.
- **Runtime segment building in dynamicPromptExecFromState.** The runtime builds `promptSegments` from the dynamic prompt: variable block (unstable), format prefix (stable), validation/middle block (unstable), format suffix (stable), end block (unstable). Only content that is identical for the same schema/character is marked stable; validation instructions that contain per-call UUIDs are kept in an unstable segment.
- **Why:** Marking validation or variable content as stable would prevent cache hits because that content changes every call; splitting format from validation ensures the stable segments are actually cacheable.
- **Anthropic plugin: segment-aware requests.** When `promptSegments` is present, the plugin sends a Messages payload with one content block per segment and `cache_control: { type: "ephemeral" }` on blocks where `stable === true`; otherwise it uses the single `prompt` path.
- **Why:** Anthropics API caches at the block level when cache_control is set; one block per segment lets the API cache only the stable blocks.
- **OpenAI and Gemini plugins: prefix ordering.** When `promptSegments` is present, the prompt sent to the API is built with stable segments first, then unstable (same total text, reordered).
- **Why:** OpenAI and Gemini use prefix-based caching; putting stable content first maximizes the cacheable prefix. No new API parameters; ordering is the hint.
- **Prompt batcher thenable API.** `onDrain(id, opts)` now returns `Promise<BatcherResult<T> | null>` that resolves when the sections first result is delivered (or `null` if the section ID was already registered). Result shape is `{ fields: T, meta: DrainMeta }`. `onResult` is optional; when omitted, callers use `const result = await onDrain(...); if (result) { const { fields, meta } = result; ... }`.
- **Why:** Large inline `onResult` callbacks split “register” and “handle result” and made control flow hard to follow. A thenable lets evaluators (e.g. reflection) write linear code and use standard promise patterns (await, .then(), .catch()).
- **BatcherResult<T>** type (in `types/prompt-batcher.ts`). Generic `T` defaults to `Record<string, unknown>`. All section promises (addSection, onDrain) resolve with this shape; askOnce/askNow unwrap to `fields` only for backward compatibility.
- **Why:** Single consistent type for “result of a batcher section”; meta (drainId, fallbackUsed, durationMs, etc.) is available when callers need it.
- **Reject on failure.** When `onResult` throws, the section promise is rejected instead of only logging. When the batcher is disposed, pending section promises are rejected with `BatcherDisposedError`. Guard ensures we never resolve and reject the same promise.
- **Why:** Callers can .catch() or try/catch for real failures; fallback-used still resolves with `meta.fallbackUsed: true` so “soft” failure is not an exception.
- **Generic onDrain<T>.** Callers can pass a type param so `result.fields` is typed (e.g. reflection uses `onDrain<ReflectionFields>(...)`). Runtime does not validate T; the generic is for developer convenience.
- **Why:** Reduces casting and improves editor support at call sites.
- **Cross-runtime task scheduler.** Three scheduling modes: (1) **local timer** — default, one `setInterval` per TaskService; (2) **per-daemon** — host calls `startTaskScheduler(adapter)`, one shared timer and one batched `getTasks(agentIds)` per tick for all registered runtimes; (3) **serverless**`runtime.serverless === true`, no timer; host calls `taskService.runDueTasks()` from cron or on each request.
- **Why:** Single-process apps keep a simple local timer; multi-agent daemons avoid N DB queries per second by batching; serverless has no long-lived process so the host must drive execution explicitly.
- **Task scheduler API (Node build).** Exports: `startTaskScheduler`, `stopTaskScheduler`, `getTaskSchedulerAdapter`, `registerTaskSchedulerRuntime`, `unregisterTaskSchedulerRuntime`, `markTaskSchedulerDirty`. TaskService registers with the daemon when present and uses `markTaskSchedulerDirty(agentId)` instead of a local dirty flag.
- **Why:** Host can plug in a shared adapter once; runtimes opt in automatically; one getTasks per tick for all agents.
- **Serverless runtime option.** `AgentRuntime` constructor accepts `serverless?: boolean`; when `true`, TaskService does not start a timer or register with the daemon. Public `taskService.runDueTasks()` runs due queue tasks once (one getTasks + runTick).
- **Why:** Serverless runtimes cannot rely on setInterval; host needs a single entry point to run due tasks on cron or per request.
- **`runTick(tasks)` and `runDueTasks()`.** TaskService exposes `runTick(tasks)` (validate + execute given tasks; used by daemon and local checkTasks) and `runDueTasks()` (fetch queue tasks for this agent, then runTick). Fetch is separate from runTick so the daemon can do one batched getTasks and dispatch to multiple runtimes.
- **Why:** Enables shared scheduler batching and serverless pull-based execution without duplicating execute logic.
- **Task system upgrades.** TaskMetadata now supports `notBefore`, `notAfter`, `paused`, `failureCount`, `maxFailures`, `lastError`, and `baseInterval`. TaskWorker supports optional `shouldRun(runtime, task)` and `canExecute(runtime, message, state)`; `validate` is deprecated. TaskService public API: `executeTaskById`, `pauseTask`, `resumeTask`, `getTaskStatus`, `markDirty`. Execute path: retry/backoff, auto-pause after maxFailures, dynamic `nextInterval` from worker return, `updatedAt` written on success and failure.
- **Why:** Single place for "when" (scheduling, pause, visibility); batcher and other consumers use tasks for periodic work. Retry/dead-letter prevent infinite retry storms.
- **Batcher on task system.** PromptBatcher no longer has its own timer. Per-affinity BATCHER_DRAIN tasks drive periodic drains. `drainAffinityGroup` is public; `getIdealTickInterval` and `getSectionCountForAffinity` added. Batcher creates/updates/deletes affinity tasks in addSection/removeSection; dispose() deletes tracked tasks.
- **Why:** One scheduling surface; task system owns WHEN, batcher owns HOW. Operators can pause/resume and inspect tasks in the DB.
- **Scoped `tick(message)`.** With a message, only message-relevant affinities (default, room:X, audit:X) are drained when batch size or immediate; autonomy is not drained from tick (task-driven only). No-arg `tick()` is a no-op.
- **Why:** No background timer; message-triggered drains stay scoped so autonomy keeps its own schedule.
- **Status action** uses `getTaskStatus` for queue tasks to show next run, paused, and last error in status output and in `statusInfo.tasks.details`.
- **Why:** Operators need visibility into when a task will run and why it might be paused or failing.
- **FollowUp workers** (`follow_up`, `recurring_check_in`) now implement `shouldRun(runtime, task)` to skip when the target contact no longer exists.
- **Why:** Scheduler (or `executeTaskById`) avoids running follow-ups for deleted contacts; single place to gate on entity existence.
### Changed
- **Embedding generation service** uses `BatchQueue` for the drain pipeline (priority dequeue, bounded parallel `process`, repeat `EMBEDDING_DRAIN` task via `TaskDrain`). When no `TEXT_EMBEDDING` model is registered, start returns a **no-op** service after a warning instead of throwing.
- **Why:** Same drain/retry/task semantics as other batch workloads; optional embedding setups can boot without a hard failure.
- **Action filter `buildIndex`** embeds actions via `BatchProcessor` (batch slices, retries, `onExhausted`) instead of raw `Promise.all` per slice only.
- **Why:** Shares backoff/concurrency policy with the rest of the batch stack; invalid vectors log and continue (BM25 still works) instead of failing the whole index on one bad response.
- **Prompt batcher affinity tasks** are ensured/updated/disposed via shared `TaskDrain` (`skipRegisterWorker: true` for `BATCHER_DRAIN`) instead of ad-hoc `createTask` / `deleteTask` maps keyed by task id. `addSection` still syncs ideal interval after drain setup; **immediate / once** sections can drain after `runtime.initPromise` when the batcher is not yet enabled.
- **Why:** One implementation of repeat-task metadata (`maxFailures: -1`, intervals) and no duplicate worker registration for the same task name; early startup sections still get a drain path.
### Changed (batch-queue polish)
- **`TaskDrain`:** When a matching repeat task already exists, `start` now calls `updateInterval` so DB `updateInterval` / `baseInterval` match the configured drain (fixes stale metadata after restarts).
- **Why:** Avoids leaving an old tick interval in the store until the next explicit sync.
- **`PriorityQueue`:** Unknown `getPriority` values log once (`logger.warn`) and enqueue as **normal** (previously fell into the low bucket silently).
- **Why:** Typos should not demote work to “low” without visibility.
- **`BatchProcessor`:** Optional `maxAttemptsCap` to bound attempts per item (used by `BatchQueue` high-priority dispose flush).
- **Why:** Items with large per-item `maxRetries` should not retry many times during shutdown.
- **`BatchQueue`:** Optional `onDrainBatchOutcomes`; high-priority dispose flush defaults to a serial `BatchProcessor` (`disposeHighPriorityViaProcessor`, default true); `clear()` is a no-op after `dispose`.
- **Why:** Observability, flush path parity with bounded concurrency, and safer lifecycle after shutdown.
- **`Semaphore`:** Documented acquire/release pairing contract in the class header.
- **Why:** Future callers are less likely to leak permits.
- **`Semaphore`:** Removed duplicate class from `runtime.ts`; package entry points (`index.node` / `index.browser` / `index.edge`) re-export `Semaphore` from `utils/batch-queue/semaphore.js` so `import { Semaphore } from "@elizaos/core"` stays valid with a single implementation.
- **Why:** One semaphore implementation avoids drift and conflicting behavior.
- **Knowledge embeddings:** `document-processor` batch fallback (single vector for many texts) and `llm.generateTextEmbeddingsBatch` now use `BatchProcessor` with `maxParallel: 10` instead of unbounded `Promise.all` over texts.
- **Why:** Large document / batch sizes no longer open unbounded concurrent `TEXT_EMBEDDING` calls; retries on the document-processor fallback align with other batch paths (`maxRetriesAfterFailure: 2`).
- **Prompt batcher section resolution.** Section promises now resolve with `{ fields, meta }` instead of raw `fields`. askOnce and askNow unwrap to `result?.fields ?? fallback` so their return type remains `Promise<Record<string, unknown>>`. Runtime pre-callback audit uses `addSectionResult?.fields` for audited fields.
- **Why:** Consistent result shape across the batcher; consumers that only need fields (askOnce, askNow, audit) keep the same API.
- **Reflection evaluator** now uses the thenable style: `const result = await onDrain<ReflectionFields>(...); if (result) { ... }` and no `onResult` callback. Processing logic is unchanged; it runs inside the `if (result)` block.
- **Why:** Demonstrates the preferred pattern and keeps reflection in sync with batcher API.
### Fixed
- **Backoff base.** On recurring task failure, backoff now uses `baseInterval ?? updateInterval ?? 1000` instead of only `updateInterval`. **Why:** After multiple failures, interval had grown; using the original base prevents exponential-of-exponential growth.
- **Non-repeat task on failure.** One-shot (non-repeat) tasks are now deleted after execution failure. **Why:** Otherwise they stay in the DB and are re-run every tick with no backoff, causing an infinite retry loop.
- **BATCHER_DRAIN never auto-pause.** Batcher creates affinity tasks with `maxFailures: -1` instead of `Infinity`. **Why:** `JSON.stringify(Infinity)` is `null`; after DB round-trip the default would apply and drain tasks could auto-pause. `-1` survives JSON and is documented as "never pause."
- **Quiet hours removed.** Unused `QuietHoursWindow` type and `quietHoursRaw` setting were removed from the batcher and runtime. **Why:** Batcher no longer has its own timer or quiet-hours logic; task system owns scheduling.
- **One-shot time-based scheduling.** Non-repeat queue tasks with `dueAt` or `metadata.scheduledAt` run when `now >= dueTime`, then are deleted. Follow-up tasks now include `queue` and `dueAt: scheduledAt.getTime()` so the scheduler runs them at the scheduled time. **Why:** "Run at time X" without external cron; follow-ups execute automatically.
- **getTasks agentId.** `getTasks` accepts optional `agentId`; runtime injects `agentId` on `createTask`/`createTasks`. TaskService passes `agentId` when fetching queue tasks. **Why:** Multi-tenant safety; schema indexes by agent_id; each runtime only sees its own tasks.
- **recurring_check_in worker removed.** No code path created such tasks; recurring check-ins can be implemented with tasks that have `tags: ["queue", "repeat"]` and `updateInterval`. **Why:** Dead code removal; document the pattern for recurring use.
### Changed
- **getTasks(agentIds) only.** `getTasks` now takes required `agentIds: UUID[]` (no optional `agentId`). All adapters (in-memory, plugin-sql PG/MySQL) and call sites updated; empty `agentIds` returns `[]` without querying.
- **Why:** Multi-tenant safety and daemon batching: one query can fetch tasks for many agents; call sites explicitly pass `[runtime.agentId]` or the daemons batch list.
- **Autonomy on prompt batcher (Option A).** Autonomy no longer uses the Task system for scheduling. When `enableAutonomy` is true, the autonomy service registers a single recurring section with `runtime.promptBatcher.think("autonomy", ...)`. The batcher's background tick and `minCycleMs` drive when the section drains; no Task DB or task worker.
- **Why:** One register for "what to ask the LLM" and "when" reduces moving parts and gives autonomy the same batching, cache, and validation benefits as other prompt sections. Fewer failure modes than Task + message pipeline.
- **Execution facade** `runAutonomyPostResponse()` in `src/autonomy/execution-facade.ts`. Given batcher result fields and a synthetic autonomy message, it runs the same post-LLM steps as the message pipeline: normalize to Content, save response to messages, execute planned tools or callback (simple), then evaluate.
- **Why:** Keeps a single implementation path for "after the model responds" so we don't duplicate planned tool execution/evaluate logic and schema stays aligned with the message pipeline.
- **Autonomy section** with `contextBuilder` that builds context from `getTargetRoomContextText()`, last thought from memories, and the same task/continuous templates. Schema matches the message pipeline (thought, providers, actions, text, simple).
- **Why:** Recurring sections get no message buffer; context must come from runtime and memories. Same templates preserve behavior; same schema lets the facade consume batcher output without a separate contract.
### Changed
- Added `runtime.promptBatcher` as a unified structured prompt orchestration subsystem.
- Added `PromptSection`, `PromptBatcher`, `PromptDispatcher`, `DrainMeta`, `DrainLog`, `BatcherStats`, `PreCallbackHandler`, and related exports in `src/utils/prompt-batcher.ts`.
- Added convenience wrappers for common prompt patterns:
- `askOnce()` for startup questions
- `onDrain()` for evaluator-style batched extraction
- `think()` for recurring autonomy reasoning
- `askNow()` for blocking audit-style questions
- Added cache-aware prompt batching with invalidate helpers and stale-while-revalidate behavior.
- Added per-section validation, retry, and `shouldRun` support.
- Added structured drain logging and batcher stats reporting.
- Added pre-callback audit registration and callback-path integration.
### Changed
- **Autonomy service** now registers and unregisters a batcher section instead of creating/deleting a recurring Task. `enableAutonomy()` / `disableAutonomy()` and `stop()` call `promptBatcher.think("autonomy", ...)` or `removeSection("autonomy")`. Optional startup cleanup deletes any orphaned `AUTONOMY_THINK` tasks from the DB.
- **Why:** Option A (batcher-only) was chosen; no second registry. "Autonomy enabled" is determined by runtime/config, not Task existence.
- `message.ts` now ticks the prompt batcher after response delivery so evaluator-style sections can batch on message cadence.
- `runtime.ts` now owns batcher lifecycle startup and teardown.
- The reflection evaluator now registers a room-scoped prompt section instead of issuing a direct `useModel()` call.
- `IDatabaseAdapter` batch mutation return types now consistently use `Promise<boolean>` for `updateAgents`, `deleteAgents`, and `deleteParticipants`, matching runtime and adapter implementations.
### Why these changes matter
- Autonomy on the batcher gives one orchestration path for both user-triggered and time-triggered reasoning, with the same cache and packing behavior. No Task persistence or worker lifecycle to reason about.
- Fewer LLM round trips means lower cost and less contention on both local and hosted inference.
- One orchestration path is easier to reason about than several partially overlapping systems.
- The dispatcher keeps infrastructure choices adjustable without changing plugin-facing registration code.
- Matching adapter interface return types to implementation fixes typecheck and build verification so the new subsystem can ship on a clean foundation.
+210
View File
@@ -0,0 +1,210 @@
# @elizaos/core
The runtime heart of elizaOS: `AgentRuntime`, the plugin abstractions (actions / providers / evaluators / services / models / routes / events), the canonical type system, and the supporting subsystems (memory, search, settings, scheduling, prompts). Almost every other `@elizaos/*` package and plugin imports from here.
## Role
`@elizaos/core` defines the contracts an Eliza agent runs on and the runtime that executes them. Plugins implement `Plugin` and the runtime wires their actions/providers/evaluators/services into the message-handling loop. Consumed by `@elizaos/agent` (which also hosts the HTTP API server), `@elizaos/app-core` (the API + dashboard host), and every plugin. It builds to three targets (Node, browser, edge) via conditional exports — keep Node-only code out of the browser/edge entries.
## Layout
```
src/
index.ts Default barrel — re-exports index.node + a few @elizaos/contracts type shims
index.node.ts Full Node API surface (the real export list — start here)
index.browser.ts Browser-safe subset (no fs/process-bound modules)
index.edge.ts Edge-runtime subset
runtime.ts AgentRuntime class (~9000 lines, `class AgentRuntime implements IAgentRuntime` at L718); the central orchestrator
runtime-composition.ts loadCharacters / createRuntimes / settings merge (Node-only boot helpers)
runtime-env.ts Runtime environment + state resolution
plugin.ts Plugin load/validate/resolve: loadPlugin, resolvePlugins, validatePlugin, resolvePluginDependencies
plugin-lifecycle.ts Plugin register/unload/reload + ownership tracking
runtime/ Message loop internals: message-handler, planner-loop, turn-controller, action-catalog,
action-retrieval/routing/tiering, context-* (registry/renderer/gates), evaluator,
validated-model-call, response-grammar, system-prompt, sub-planner, trajectory-recorder
types/ Canonical type system. types/index.ts is the barrel; types/runtime.ts has IAgentRuntime;
plugin.ts, model.ts, memory.ts, state.ts, service.ts, task.ts, events.ts, schema*.ts, etc.
services/ Built-in services: task / task-scheduler, evaluator, message, relationships,
pairing, pairing-integration, pairing-migration, hook, optimized-prompt,
optimized-prompt-resolver, tool-policy, trajectories, trajectory-export, trajectory-types,
triggerScheduling, approval, embedding, followUp, analysis-mode-handler, agentEvent,
runtime-capability-service, setup-cli, setup-rpc, setup-state
features/ Self-contained capability bundles, each its own dir:
basic-capabilities (the core action/provider/evaluator/service bundle),
advanced-capabilities, advanced-memory, advanced-planning, approvals, autonomy, ballots,
documents, messaging (triage), oauth, payments, plugin-config, plugin-manager,
secrets, sub-agent-credentials, trajectories, trust, working-memory
actions/ Action plumbing: action-schema, to-tool, validate-tool-args, subaction-dispatch
providers/ First-party providers (setup-progress, skill-eligibility, linked-identities, ...)
schemas/ Drizzle table schemas + character schema. schemas/index.ts: buildBaseTables, BaseTables
database/ inMemoryAdapter (IDatabaseAdapter fallback used when ALLOW_NO_DATABASE)
contracts/ Re-exports/adapters over @elizaos/contracts (cloud-topology, first-run-options, service-routing, wallet)
generated/ Build-time generated action/provider/evaluator docs + spec-helpers (do not hand-edit)
i18n/ validation + action-search keyword data (some generated; see prebuild)
security/ redact, ssrf-adjacent input policy, spawn-env-policy, external-content, incoming-message-security
sensitive-requests/ Sensitive request policy helpers
network/ SSRF guard + secure fetch (fetch-guard, ssrf)
markdown/ media/ markdown IR/chunking; media fetch + mime/type detection
testing/ Test harness exports (live-provider, integration-runtime, http, mocks) — `@elizaos/core/testing`
capabilities/ Runtime capability index
connectors/ Connector abstractions (account-manager, connector-config, oauth-role, privacy)
plugins/ Plugin-related helpers
registries/ Registry utilities
sessions/ Session management
sandbox/ Sandbox policy
optimization/ Optimization utilities
scheduled-task/ Scheduled task helpers
validation/ Input validation utilities
constants/ Shared constants
api/ API helpers
owner-state/ Owner state tracking
messaging/ Messaging utilities
search.ts In-memory/embedding search utilities
utils.ts utils/ Shared helpers: prompts (composePromptFromState, parseKeyValueXml), batch-queue,
confirmation, read-env, state-dir, streaming, environment, plugin-loader
build.ts Custom bun-based multi-target build (Node / browser / edge + d.ts generation)
scripts/perf-settings.ts, scripts/run-e2e-smoke.mjs
```
## Key exports / surface
From `@elizaos/core` (`index.node.ts`):
- `AgentRuntime` — the runtime, `implements IAgentRuntime`.
- Plugin machinery: `loadPlugin`, `resolvePlugins`, `validatePlugin`, `isValidPluginShape`, `normalizePluginName`, `resolvePluginDependencies`.
- `logger` (re-exported from `./logger`) — the structured logger all packages use.
- Type contracts: `Plugin`, `Action`, `Provider`, `Evaluator`, `Service`, `IAgentRuntime`, `IDatabaseAdapter`, `Memory`, `State`, `Character`, `ModelType`, `UUID`, plus everything in `types/`.
- Built-in capability bundle: `basicCapabilities` / `basicActions` / `basicProviders` / `basicEvaluators` / `basicServices` (from `features/basic-capabilities/index.ts`).
- Boot/composition (Node): `loadCharacters`, `createRuntimes`, `buildBaseTables`, `InMemoryDatabaseAdapter`.
- Prompt + model helpers: `composePromptFromState`, `parseKeyValueXml`, `callModelWithValidation`, `parseAndValidate`.
Subpath entries (see `package.json` `exports`): `@elizaos/core/node`, `@elizaos/core/browser`, `@elizaos/core/roles`, `@elizaos/core/testing`, `@elizaos/core/services/*`.
This package does NOT export a `corePlugin` singleton — the foundational actions/providers/evaluators/services live in `features/basic-capabilities` and are exported as the `basic*` bundles above.
## Commands
```bash
bun run --cwd packages/core build # multi-target build via build.ts (Node + browser + edge + d.ts)
bun run --cwd packages/core build:node # Node target only
bun run --cwd packages/core build:watch # watch build (alias: dev)
bun run --cwd packages/core test # vitest run (via ../scripts/run-vitest.mjs)
bun run --cwd packages/core test:watch # vitest watch
bun run --cwd packages/core test:coverage # vitest with v8 coverage
bun run --cwd packages/core test:e2e # Playwright (playwright.config.ts)
bun run --cwd packages/core test:e2e:smoke
bun run --cwd packages/core typecheck # tsgo --noEmit -p ./tsconfig.json
bun run --cwd packages/core lint # biome check --write ./src
bun run --cwd packages/core format # biome format --write ./src
bun run --cwd packages/core clean # remove dist + emitted src artifacts
```
`prebuild` builds `@elizaos/contracts` and generates `src/i18n/generated/validation-keyword-data.ts` if missing. Depends on workspace packages `@elizaos/contracts` and `@elizaos/prompts`.
## Config / env vars
Read by the runtime (see README for the full WHY of each):
- `LOG_LEVEL`, `LOG_JSON_FORMAT`, `LOG_FILE` — logger behavior (`src/logger.ts`).
- `SECRET_SALT` — encryption salt, read by `getSalt()` in `src/settings.ts` (`ELIZA_ALLOW_DEFAULT_SECRET_SALT` overrides the production non-default check).
- `ALLOW_NO_DATABASE` — fall back to `InMemoryDatabaseAdapter` on `initialize()` when no adapter is provided (`runtime.ts`).
- `SHOULD_RESPOND_MODEL` (`small`/`large`, `services/message.ts`), `BASIC_CAPABILITIES_KEEP_RESP` (`services/message.ts`) — message/basic-capabilities behavior.
- `ELIZA_BOT_NOISE_TRIAGE` (`services/message/bot-noise-triage.ts`) — set `0` to disable the TEXT_SMALL pre-gate that triages unaddressed bot/webhook group messages before the Stage 1 RESPONSE_HANDLER call (default on).
- `ELIZA_STAGE1_GROUP_TRIAGE` (`services/message/stage1-prompt-tier.ts`) — set `0` to disable the compact Stage 1 instruction tier for unaddressed group messages and always render the full rule block (default on).
- Prompt-batcher knobs (all `PROMPT_BATCHER_*`, read in `runtime.ts`): `PROMPT_BATCHER_BATCH_SIZE`, `PROMPT_BATCHER_MAX_DRAIN_INTERVAL_MS`, `PROMPT_BATCHER_MAX_SECTIONS_PER_CALL`, `PROMPT_BATCHER_PACKING_DENSITY`, `PROMPT_BATCHER_MAX_TOKENS_PER_CALL`, `PROMPT_BATCHER_MAX_PARALLEL_CALLS`, `PROMPT_BATCHER_MODEL_SEPARATION`.
- `ELIZA_STATE_DIR` — state-dir resolution (`utils/state-dir.ts`); `ELIZA_WORKSPACE_DIR` — workspace folder (`utils/workspace-folder-config.ts`).
- `ELIZA_TRAJECTORY_LOGGING` — canonical trajectory persistence gate for both file and DB recorders (`runtime/trajectory-gate.ts`): truthy enables; non-empty falsey disables; blank is unset. Defaults are on for dev/unset `NODE_ENV`, off for `NODE_ENV=test|production`. `ELIZA_TRAJECTORY_RECORDING` is the legacy alias, and `ELIZA_DISABLE_TRAJECTORY_LOGGING=1` is the hard opt-out.
Prefer the canonical env reader in `utils/read-env.ts` over raw `process.env` (it handles legacy aliases).
### Setting / env resolution — precedence & the multi-tenant rule
Two canonical helpers own all setting/env resolution; everything else delegates:
| Helper | Source order | Use when |
| --- | --- | --- |
| `runtime.getSetting(key)` (`runtime.ts`) | character secrets → character settings → `settings.extra``settings.secrets``character.env.vars` → the constructor-provided `settings` map. **Never `process.env`.** | Inside the runtime / framework code. |
| `readEnv(key, opts)` (`utils/read-env.ts`) | `process.env[key]` (trimmed; empty string treated as unset) → `defaultValue`. | Reading an env var with no runtime in scope. |
| `resolveSetting(runtime, key, opts)` (`utils/resolve-setting.ts`) | `runtime.getSetting(key)` (coerced to string) → `readEnv(key)``defaultValue`. | Single-tenant / headless plugins that still want a dotenv fallback. |
**Resolution order:** runtime/character setting → env alias → default. The
per-agent runtime value always wins; the env fallback is the deployment default.
**WHY core `getSetting()` deliberately does NOT read `process.env`:** in a
multi-tenant process many agents share one OS environment. If `getSetting()`
fell through to `process.env`, a host secret (`OPENAI_API_KEY`,
`POSTGRES_URL`, …) set for the *box* would silently leak into *every* agent,
including ones the operator never granted it to. Keeping `getSetting()`
per-agent makes each agent's config explicit and isolated. `resolveSetting`
re-adds an opt-in env fallback for single-tenant/headless plugins **without**
changing `getSetting()` semantics — multi-tenant hosts that never call it are
unaffected.
**Host obligation (how to make dotenv values visible to `getSetting()`):**
because `getSetting()` reads the constructor-provided `settings` map and not
`process.env`, a host that wants `.env` / `process.env` values honored must fold
them into the runtime's settings at construction. `getBasicCapabilitiesSettings(character, env)`
(`runtime-composition.ts`) does exactly this — it flattens `character.settings`,
`character.secrets`, and `env` into the `Record<string,string>` handed to
adapter factories and the `AgentRuntime` constructor. Construct the runtime with
those settings and dotenv is honored; skip it and only character config is
visible.
## How to extend
- **Add an action/provider/evaluator/service to the built-in bundle:** implement against the `Action`/`Provider`/`Evaluator`/`Service` types in `types/`, then add it to the relevant array in `src/features/basic-capabilities/index.ts` (`basicActions`, `basicProviders`, `basicEvaluators`, `basicServices`). Most new capabilities should live in their own plugin package instead of here.
- **Add a runtime type/contract:** define it under `src/types/<area>.ts` and export from `src/types/index.ts`. If it should be shared with non-runtime consumers, prefer `@elizaos/contracts` and re-export via `src/contracts/`.
- **Add a DB table:** extend the schema in `src/schemas/` and wire it into `buildBaseTables` (`schemas/index.ts`); adapters in plugin-sql/localdb materialize it.
- **Touching the message loop:** the order is provider → model → action → evaluator. Logic lives in `src/runtime/` (`message-handler.ts`, `planner-loop.ts`, `turn-controller.ts`) and `runtime.ts`. Validated model output goes through `runtime/validated-model-call.ts`.
- **Browser/edge surface:** if your code is Node-only (fs, process, native deps), export it from `index.node.ts` only — never add it to `index.browser.ts` / `index.edge.ts`.
## Conventions / gotchas
- `index.node.ts` is the source of truth for the public surface; `index.ts` just re-exports it plus a few `@elizaos/contracts` type shims (kept explicit to avoid d.ts ambiguity).
- Three build targets share source — Node-only imports in shared modules break the browser/edge bundles. Verify with `build:node` vs full `build`.
- The model-output contract is `<response>` XML (with `<actions>`/`<providers>`/`<text>`); plain text is tolerated and treated as a `REPLY`.
- DB mutation methods on `IDatabaseAdapter` return `Promise<boolean>` so callers can distinguish success/failure (`types/database.ts`).
- The task system (`services/task.ts`, `services/task-scheduler.ts`) is the single place scheduled work runs; only tasks tagged `queue` are polled. Three modes: local timer, per-daemon (`startTaskScheduler`), serverless (`{ serverless: true }` + `runDueTasks()`).
- `runtime.ts` is very large (~9000 lines / ~259 KB) — navigate by symbol, not by reading top-to-bottom.
- `src/generated/` and parts of `src/i18n/generated/` are build artifacts; regenerate via prebuild rather than editing.
- Repo-wide rules (logger-only, ESM, naming, architecture) are in the root [AGENTS.md](../../AGENTS.md) — not restated here.
<!-- BEGIN: evidence-and-e2e-mandate (managed; canonical standard = repo-root AGENTS.md) -->
## ⛔ NON-NEGOTIABLE — evidence, trajectories & real end-to-end tests
> The binding, repo-wide standard is **[AGENTS.md](../../AGENTS.md)**. Read it.
> Nothing in this package is *done* until it is *proven* done — a reviewer must confirm it
> works **without reading the code**, from the artifacts you attach. This applies to **every**
> feature, fix, refactor, and chore here. "Tests pass" is not proof; "CI is green" is not proof.
- **Record AND read model trajectories.** Capture the *actual* inputs and outputs of the model
from a **live** LLM — not the deterministic proxy, not a mock: the prompt, the
providers/context, the raw model output, every tool/action call, and the result. Then **open
the trajectory and review it by hand.** A captured-but-unread trajectory is not evidence
(`packages/scenario-runner/bin/eliza-scenarios run <scenario> --report <out>`).
- **Real, full-featured E2E — no larp.** Every feature ships detailed end-to-end tests that
drive the *real* path end to end. Not the happy "front door" only: cover error paths,
edge/empty/invalid input, concurrency, roles/permissions, and adversarial input. A test that
asserts against a mock/stub/fixture standing in for the thing under test **does not count**.
If the real model/device/chain/connector/account is hard to reach, **make it reachable — that
is the work**, not an excuse to mock. If the existing tests here are shallow or mocked, fixing
them is part of your change.
- **Screenshots + logs at every phase**, plus a **complete walkthrough video/run-through** of
the entire feature or view, start to finish (`bun run test:e2e:record`).
- **Manually review every artifact the change touches** — never just the green check: client
logs (console + network), server logs (`[ClassName] …`), the model trajectories in and out,
before/after full-page screenshots, **and the domain artifacts listed below for this package.**
- **No residuals. No shortcuts.** The goal is not "done" — it is *everything* done. Clear every
blocker by the **hard path**: build the real architecture, stand up the real
model/device/service, actually test it. Never leave a TODO, a stub, a stepping-stone, or a
"follow-up." When unsure, research thoroughly, weigh the options, and ship the best,
highest-effort, production-ready version. Keep going until every possibility is exhausted.
Artifacts → attached inline in the PR (MP4 video, JPG screenshots, logs in `<details>`); attach each evidence type **or**
explicitly mark it N/A with a reason — never leave it blank. If `develop` moved and changed
behavior, **re-capture** evidence; stale proof is worse than none.
**Capture & manually review for this package — runtime / framework:**
- A **live-LLM** scenario trajectory for the runtime path you touched — provider → model → action → evaluator — with the raw `<response>` XML and every tool/action call visible and **read**.
- Backend `[ClassName]` logs proving the message loop, task scheduler, or service actually fired end to end.
- The memory/state artifacts produced — rows written, embeddings, room/world/entity records, scheduled-task rows — inspected, not assumed.
- For shared modules: `build:node` vs full `build` so the browser/edge bundles still compile.
<!-- END: evidence-and-e2e-mandate -->
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Shaw Walters and elizaOS Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+290
View File
@@ -0,0 +1,290 @@
# @elizaos/core
## Overview
`@elizaos/core` is the runtime and contract layer of elizaOS. It defines the `AgentRuntime` and the plugin abstractions (actions, providers, evaluators, services, models, routes, events), the canonical type system, and the supporting subsystems (memory, search, settings, scheduling, prompts). It is consumed by `@elizaos/agent` (which also hosts the HTTP API server), `@elizaos/app-core` (the API + dashboard host), and every `@elizaos/*` plugin.
## Key concepts
- **AgentRuntime:** Central orchestrator for the agent lifecycle, plugin loading, and the message loop.
- **Actions:** Tasks the agent can perform, each with a `validate` and `handler` function.
- **Providers:** Supply data and context to the runtime and its components.
- **Evaluators:** Process conversation data to extract facts, build memory, and reflect.
- **Plugin system:** `Plugin` objects contribute actions/providers/evaluators/services to the runtime.
- **Built-in bundle:** Foundational capabilities ship as `basicCapabilities` (and `basicActions` / `basicProviders` / `basicEvaluators` / `basicServices`); there is no `corePlugin` singleton.
## Installation
1. Add `@elizaos/core` to your `agent/package.json` dependencies:
```json
{
"dependencies": {
"@elizaos/core": "workspace:*"
}
}
```
2. Navigate to your `agent/` directory.
3. Install dependencies:
```bash
bun install
```
4. Build your project:
```bash
bun run build
```
## Build targets (Node, Browser, Edge)
`@elizaos/core` builds to three targets via conditional exports:
- **Node.js Build**: Full API surface with all features including server utilities (`index.node.ts`)
- **Browser Build**: Browser-safe subset, no fs/process-bound modules (`index.browser.ts`)
- **Edge Build**: Edge-runtime subset (`index.edge.ts`)
The correct build is automatically selected based on your environment through package.json conditional exports. For browser usage, ensure your app provides the standard platform primitives it depends on, such as `Buffer` where needed.
## Configuration
The following environment variables are used by `@elizaos/core`. Configure them in a `.env` file at your project root.
- `LOG_LEVEL`: Logging verbosity (e.g., 'debug', 'info', 'error').
- `LOG_JSON_FORMAT`: Output logs in JSON format (`true`/`false`).
- `SECRET_SALT`: Encryption salt, read by `getSalt()` in `src/settings.ts`. In production it must be set to a non-default value unless `ELIZA_ALLOW_DEFAULT_SECRET_SALT=true`.
- `ALLOW_NO_DATABASE`: Allow running without a persistent database adapter. When `true`, `AgentRuntime.initialize()` will fall back to an in-memory adapter (useful for benchmarks/tests).
- `LOG_FILE`: When set to `true`/`1` or a path, enables file logging: `output.log`, `prompts.log`, and `chat.log` (in cwd or at the given path). **Why:** Lets you inspect full prompts and chat flow without scraping console; ANSI is stripped so files stay grep-friendly.
- `BASIC_CAPABILITIES_KEEP_RESP`: When `true`, the message service does not discard a response when a newer message is being processed (avoids "stale reply" race). **Why:** Some deployments want to keep or display every response; this is the config equivalent of passing `keepExistingResponses: true` in options.
- `SHOULD_RESPOND_MODEL`: Which model size to use for the "should I respond?" decision (`small` or `large`, read in `src/services/message.ts`). Defaults from runtime settings if not set in options.
- `ELIZA_TRAJECTORY_LOGGING`: Canonical trajectory persistence knob. Truthy values (`1`, `true`, `yes`, `on`) enable file and DB trajectory recording; non-empty falsey values disable it; blank is treated as unset. When unset, recording is on for local/dev and unset `NODE_ENV`, but off for `NODE_ENV=test` and `NODE_ENV=production` unless explicitly enabled.
- `ELIZA_TRAJECTORY_RECORDING`: Legacy alias honored only when `ELIZA_TRAJECTORY_LOGGING` is unset.
- `ELIZA_DISABLE_TRAJECTORY_LOGGING=1`: Hard opt-out that wins over both trajectory enable knobs.
**Example `.env`:**
```plaintext
LOG_LEVEL=debug
LOG_JSON_FORMAT=false
SECRET_SALT=yourSecretSaltHere
ALLOW_NO_DATABASE=true
LOG_FILE=true
```
**Note:** Add your `.env` file to `.gitignore` to protect sensitive information.
### Design and rationale (WHY)
Per-change notes with the WHY for each addition or fix live in [CHANGELOG.md](CHANGELOG.md). The sections below document the reasoning behind the major subsystems so future changes stay consistent with intent.
### Benchmark & Trajectory Tracing
Trajectory persistence is controlled by `ELIZA_TRAJECTORY_LOGGING`: dev/local defaults on, while `NODE_ENV=test` and `NODE_ENV=production` default off unless explicitly opted in. Blank values are treated as unset so empty `.env` entries do not silently disable local recording.
Benchmarks and harnesses can attach metadata to inbound messages:
- `message.metadata.trajectoryStepId`: when present, provider access + model calls are captured for that step.
- `message.metadata.benchmarkContext`: when present, the `CONTEXT_BENCH` provider sets `state.values.benchmark_has_context=true`, and the message loop forces action-based execution (so the full Provider → Model → Action → Evaluator loop is exercised).
### Model output contract (XML preferred, plain text tolerated)
The canonical message loop expects model outputs in the `<response>...</response>` XML format (with `<actions>`, `<providers>`, and `<text>` fields).
Some deterministic/offline backends may return **plain text** instead. In that case, the runtime will treat the raw output as a simple **`REPLY`** so the system remains usable even when strict XML formatting is unavailable.
### Prompt cache hints
The core can pass **prompt segments** to model providers so they can use prompt-caching APIs when supported. Each segment has `content` (string) and `stable` (boolean). **Stable** means the content is the same across calls for the same schema/character (e.g. instructions, format, examples); **unstable** means it changes every call (e.g. state, validation codes).
**Why this exists:** Repeated calls (e.g. message handling, batched evaluators) often send the same instructions and format while only the context/state changes. Provider caching (Anthropic ephemeral cache, OpenAI/Gemini prefix cache) can reuse tokens for the stable prefix, reducing cost and latency. The core describes which parts are stable so providers can opt in without parsing the prompt.
- **Invariant:** When `promptSegments` is set on generation params, `prompt` MUST equal `promptSegments.map(s => s.content).join("")`. **Why:** Providers that ignore segments still get correct behavior by using `prompt`; those that use segments must send the same total text so model behavior is unchanged.
- **Providers:** Anthropic uses the Messages API with `cache_control: { type: "ephemeral" }` on stable blocks so the API can cache those blocks. OpenAI and Gemini use **prefix ordering**: when segments are present, the prompt sent to the API is built with stable segments first, then unstable. **Why:** OpenAI and Gemini cache by prefix (e.g. OpenAI ≥1024 tokens); putting stable content first maximizes cache hits.
**Pitfalls for operators:**
- OpenAI caching only applies when the prompt is ≥1024 tokens; very short prompts will not show cache savings.
- Small or low-parameter models may not support or benefit from caching; behavior is unchanged.
- Caching is a performance/cost optimization; correctness does not depend on it.
**Pitfalls for implementers:**
- Do not mutate segment objects; always create new `{ content, stable }` objects. **Why:** Params may be passed to multiple handlers or stored; mutation can cause cross-request bugs.
- Segment order must match the order in which the prompt string is built; add an assertion that `prompt === promptSegments.map(s => s.content).join("")`. **Why:** Wrong order breaks the invariant and can send the wrong prompt to the model.
- When using segments in the API (e.g. messages or reordered prompt), ensure the final text seen by the model equals the intended full prompt (e.g. `params.prompt` or the stable-first concatenation).
- Only mark content as `stable: true` if it is identical across calls for the same schema/character. **Why:** Content that includes per-call UUIDs or changing state will never cache; mislabeling it as stable wastes cache capacity and can confuse operators.
## Core Architecture
`@elizaos/core` is built around a few key concepts that work together within the `AgentRuntime`.
### Unified Prompt Batcher
`@elizaos/core` now includes a unified prompt batching subsystem on `runtime.promptBatcher`.
Why this exists:
- Evaluators, startup warmups, and autonomous reasoning were all paying separate LLM round trips for structurally similar work.
- Batching reduces cost, queue depth, and local GPU contention by turning many small prompt calls into fewer structured calls.
- The dispatcher keeps deployment flexibility: local inference can pack aggressively while frontier APIs can trade some density for latency.
What it does:
- `askOnce()` batches startup questions into a single post-init drain when possible. Returns a promise of the extracted **fields** (unwrapped). **Why:** callers get a thenable so they can `await` or `.then()` without a callback.
- `onDrain(id, opts)` registers a section that runs on the next drain for that affinity and returns a **promise that resolves with `{ fields, meta }`** (or `null` if the section ID was already registered). **Why:** evaluators can use linear `await` + `if (result) { ... }` instead of a large `onResult` callback; same batching benefits. You can still pass optional `onResult` for fire-and-forget or recurring use (e.g. logging).
- `think()` is used by **autonomy**: when `enableAutonomy` is true, the autonomy service registers one recurring section; a BATCHER_DRAIN task in the task system drives when that affinity drains (task system owns WHEN, batcher owns HOW). **Why:** one register for "what to ask" and the same orchestration path as evaluators and startup, with the same cache and packing benefits. Autonomy keeps using `onResult` because it is fire-and-forget per drain.
- `askNow()` supports blocking audits without creating a second subsystem. Returns a promise of the **fields** (unwrapped). **Why:** same thenable style as askOnce; fallback is required so the caller always gets an object.
Result shape and errors:
- Section promises (from `addSection` / `onDrain`) resolve with **`BatcherResult<T> | null`**: `{ fields: T, meta: DrainMeta }`. **Why:** callers get both the extracted data and drain metadata (e.g. `meta.fallbackUsed`, `meta.durationMs`) in one object; `null` means duplicate section ID so the caller can branch.
- When **onResult** throws or the batcher is **disposed**, the section promise **rejects** instead of resolving. **Why:** callers can `.catch()` or try/catch for real failures; fallback-used still resolves (with `meta.fallbackUsed: true`) so "soft" failure is not an exception.
- **Generic `onDrain<T>(...)`**: pass a type param so `result.fields` is typed (e.g. `onDrain<ReflectionFields>(...)`). **Why:** avoids casting at call sites; the runtime still returns `Record<string, unknown>` from the model—the generic is for developer convenience.
Important behavior:
- Sections are idempotent by ID, so developers can register them from handlers without tracking lifecycle manually.
- The promise returned by `onDrain` (or `addSection`) **resolves once**—on the first delivery for that registration. **Why:** per-drain sections run on every drain, but the thenable is for "give me the result of this registration"; subsequent drains do not resolve the same promise again. For recurring delivery (e.g. every drain), use the optional `onResult` callback.
- Context is declarative and composable: `providers`, `contextBuilder`, and `contextResolvers` can be mixed.
- Dispatching is affinity-aware, so unrelated prompt sections are not merged into the same call just because they arrived at the same time.
Relevant runtime knobs (all `PROMPT_BATCHER_*`, read in `src/runtime.ts`):
- `PROMPT_BATCHER_BATCH_SIZE`
- `PROMPT_BATCHER_MAX_DRAIN_INTERVAL_MS`
- `PROMPT_BATCHER_MAX_SECTIONS_PER_CALL`
- `PROMPT_BATCHER_PACKING_DENSITY`
- `PROMPT_BATCHER_MAX_TOKENS_PER_CALL`
- `PROMPT_BATCHER_MAX_PARALLEL_CALLS`
- `PROMPT_BATCHER_MODEL_SEPARATION`
The prompt batcher implementation lives in `src/utils/prompt-batcher/` (`batcher.ts`, `dispatcher.ts`). The lower-level queue primitives (`PriorityQueue` / `BatchProcessor` / `TaskDrain` / `BatchQueue`) live in `src/utils/batch-queue/`.
### Task system
The **task system** is the single place for *when* scheduled work runs. Only tasks with tag `queue` are polled by the scheduler (TaskService); other tasks (e.g. approval, follow-up) are stored and executed only when explicitly triggered (e.g. choice action, or `executeTaskById`).
**Why one scheduler:**
- Recurring work (e.g. batcher drains, future cron-like use) uses the same DB, same pause/resume, same visibility (`getTaskStatus`, `nextRunAt`, `lastError`). Retry and backoff (exponential backoff, auto-pause after `maxFailures`) live in one place so we avoid infinite retry storms.
**Why queue + repeat:**
- Tasks with `tags: ["queue"]` are fetched every tick. Non-repeat tasks run when `now >= dueAt` (or `metadata.scheduledAt`) then are deleted; repeat tasks use `updateInterval`/`baseInterval` and `metadata.updatedAt` as last-run time. **Why:** One-shot "run at time X" (e.g. follow-up) uses `dueAt`; interval-based scheduling covers batcher drains and recurring use.
**Why `utils/batch-queue`s `TaskDrain`:** several services create the same style of repeat drain task (`queue` + `repeat`, `maxFailures: -1`, interval metadata). Centralizing find/create/update/delete avoids each caller re-implementing JSON/metadata edge cases and keeps worker registration rules explicit (`skipRegisterWorker` when TaskService already owns the worker name). Implementation in `src/utils/batch-queue/`.
**Cross-runtime scheduling (three modes):**
1. **Local timer (default):** One `setInterval` per TaskService; each runtime fetches its own queue tasks every tick. **Why:** Zero config for single-process apps.
2. **Per-daemon:** Host calls `startTaskScheduler(adapter)`; one shared timer runs, one batched `getTasks(agentIds)` per tick for all registered runtimes, then tasks are dispatched to each runtimes `runTick(tasks)`. **Why:** Multi-agent daemons avoid N DB queries per second.
3. **Serverless:** Construct runtime with `{ serverless: true }`; no timer. Host calls `taskService.runDueTasks()` from cron or on each request to run due queue tasks once. **Why:** No long-lived process; host controls when tasks run.
**Public API (TaskService):** `executeTaskById`, `pauseTask`, `resumeTask`, `getTaskStatus`, `markDirty`, `runDueTasks()` (serverless). **Why:** Operators and UIs can run, pause, resume, and inspect tasks without touching the DB directly.
The implementation lives in `src/services/task.ts` and `src/services/task-scheduler.ts`.
### Autonomy
The autonomy service lets the agent "think" and act on a schedule without user messages. It uses the **prompt batcher** with the **task system** for scheduling: when `enableAutonomy` is true, a recurring section is registered with `think("autonomy", ...)`. A BATCHER_DRAIN task for the autonomy affinity determines when the section drains; results are delivered to `onResult`, which runs the same post-LLM steps as the message pipeline (actions, memory, evaluators) via an execution facade.
Why batcher-only:
- The batcher owns "what to ask"; the task system owns "when" (per-affinity BATCHER_DRAIN tasks). One scheduling surface and one packing path. Evaluators used after autonomy runs are the same as for user messages; as more evaluators move to the batcher, autonomy benefits automatically.
### AgentRuntime
The `AgentRuntime` (`src/runtime.ts`, `class AgentRuntime implements IAgentRuntime`) is the heart of the system. It manages the agent's lifecycle, loads plugins, orchestrates the message loop, and is the central point for actions, providers, and evaluators. It is initialized with a set of `Plugin`s; foundational actions, providers, evaluators, and services ship as the `basicCapabilities` bundle (`src/features/basic-capabilities/index.ts`).
### Actions
Actions define specific tasks or capabilities the agent can perform. Each action typically includes:
- A unique `name`.
- A `description` explaining its purpose and when it should be triggered.
- A `validate` function to determine if the action is applicable in a given context.
- A `handler` function that executes the action's logic.
Actions enable the agent to respond intelligently and perform operations based on user input or internal triggers.
**Private actions.** Set `private: true` on an action to reserve it for the agent's own autonomous loop. A private action is never exposed to the planner — and is rejected by the executor as a defense-in-depth backstop — on user-driven turns; it can only be selected and run when the triggering message is an autonomous self-prompt (`content.metadata.isAutonomous === true`, the marker the autonomy service stamps). Use this for self-initiated capabilities the agent should decide to invoke on its own — e.g. minting a coin or opening a position — rather than ones a user can trigger on demand. The gate lives in `src/runtime/private-action-gate.ts`.
### Providers
Providers are responsible for supplying data and context to the `AgentRuntime` and its components. They can:
- Fetch data from external APIs or databases.
- Provide real-time information about the environment.
- Offer access to external services or tools.
This allows the agent to operate with up-to-date and relevant information.
### Evaluators
Evaluators analyze conversation data and other inputs to extract meaningful information, build the agent's memory, and maintain contextual awareness. They help the agent:
- Understand user intent.
- Extract facts and relationships.
- Reflect on past interactions to improve future responses.
- Update the agent's knowledge base.
### Database adapter
The runtime talks to persistence through the `IDatabaseAdapter` interface. Adapters (e.g. plugin-sql, plugin-localdb, InMemory) implement this contract so the same runtime code works with different backends.
**Why mutation methods return `Promise<boolean>`:** Methods such as `updateAgents`, `deleteAgents`, and `deleteParticipants` return a boolean so callers can tell success from failure. That supports error handling, retries, and UX (e.g. "Agent removed" vs "Failed to remove"). All adapters use this convention for consistency. See `packages/core/src/types/database.ts` for full JSDoc and design notes.
## Getting Started
### Initializing a runtime
```typescript
import { AgentRuntime } from "@elizaos/core";
const runtime = new AgentRuntime({
character, // a Character (type in src/types/agent.ts; helpers in src/character.ts)
plugins: [
// your plugins; each contributes actions/providers/evaluators/services
],
// other AgentRuntime options
});
await runtime.initialize();
```
Foundational actions, providers, evaluators, and services are available as the `basicCapabilities` bundle and its parts (`basicActions`, `basicProviders`, `basicEvaluators`, `basicServices`) exported from `@elizaos/core`. There is no `corePlugin` singleton — compose the bundles you need or rely on a higher-level package (e.g. `@elizaos/agent`) to wire them.
### Defining a custom capability
A custom action implements the `Action` type (`src/types/`):
```typescript
import type { Action } from "@elizaos/core";
export const customGreet: Action = {
name: "CUSTOM_GREET",
description: "Greets a user in a special way.",
validate: async (runtime, message) => message.content.text?.includes("special hello") ?? false,
handler: async (runtime, message, state, options, callback) => {
await callback?.({ text: "A very special hello to you!" });
return { success: true };
},
examples: [],
};
```
Register it via a `Plugin` (`{ name, actions: [customGreet] }`) passed to the runtime. Providers and evaluators follow the same pattern against the `Provider` / `Evaluator` types.
## Development & Testing
The package uses **vitest**. From the repo root:
```bash
bun run --cwd packages/core test # vitest run
bun run --cwd packages/core test:watch # watch mode
bun run --cwd packages/core test:coverage # with v8 coverage
bun run --cwd packages/core typecheck # tsgo --noEmit
```
For agent-facing notes on layout, the public surface, and how to extend the runtime, see [CLAUDE.md](CLAUDE.md) / [AGENTS.md](AGENTS.md).
---
+24
View File
@@ -0,0 +1,24 @@
{
"$schema": "https://biomejs.dev/schemas/2.5.1/schema.json",
"root": false,
"linter": {
"rules": {
"style": {
"noNonNullAssertion": "warn"
},
"suspicious": {
"noExplicitAny": "warn",
"noAssignInExpressions": "warn",
"noControlCharactersInRegex": "warn",
"useAdjacentOverloadSignatures": "warn"
},
"complexity": {
"noBannedTypes": "warn",
"noCommaOperator": "warn"
},
"correctness": {
"noUnusedVariables": "warn"
}
}
}
}
File diff suppressed because it is too large Load Diff
+25
View File
@@ -0,0 +1,25 @@
[test]
# Test timeout in milliseconds
timeout = 60000
# Keep Playwright end-to-end specs out of Bun's direct test discovery.
# They are exercised separately via `bun run test:e2e`.
path-ignore-patterns = ["e2e/**"]
# Test setup files - removed preload due to path resolution conflicts in CI
# preload = ["packages/core/src/test_resources/testSetup.ts"]
# Coverage configuration
coverage = true
coverage-exclude = [
"**/dist/**",
"**/build/**",
"**/chunk-*.js",
"**/*.chunk.js",
"**/node_modules/**",
"**/*.min.js",
"**/*.bundle.js",
"**/coverage/**",
"**/.turbo/**",
]
+145
View File
@@ -0,0 +1,145 @@
/**
* Full end-to-end tests for the AgentRuntime.
*
* These tests start a real AgentRuntime backed by a live LLM provider
* (OpenAI, Anthropic, Google, or Ollama) and verify that real inference
* results are returned through a lightweight HTTP test harness.
*
* Prerequisites:
* - At least one LLM provider configured (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)
* - OR a running Ollama instance
*/
import { expect, test } from "@playwright/test";
const isPlaywrightE2E = process.env.ELIZA_PLAYWRIGHT_E2E === "1";
if (isPlaywrightE2E) {
// Skip the entire suite when no provider is available (set by global-setup).
test.beforeEach(() => {
test.skip(
process.env.__E2E_SKIP__ === "1",
"No inference provider available",
);
});
// ─── Health & status ──────────────────────────────────────────────────────
test("GET /health returns 200", async ({ request }) => {
const res = await request.get("/health");
expect(res.status()).toBe(200);
const body = await res.json();
expect(body.ok).toBe(true);
});
test("GET /status returns agent info", async ({ request }) => {
const res = await request.get("/status");
expect(res.status()).toBe(200);
const body = await res.json();
expect(body.agentId).toBeTruthy();
expect(body.name).toBe("E2ETestAgent");
expect(body.provider).toBeTruthy();
expect(body.ready).toBe(true);
});
// ─── Chat: simple question ────────────────────────────────────────────────
test("POST /chat with a simple math question returns a real answer", async ({
request,
}) => {
const res = await request.post("/chat", {
data: { text: "What is 2+2? Reply with just the number." },
});
expect(res.status()).toBe(200);
const body = await res.json();
expect(body.text).toBeTruthy();
expect(typeof body.text).toBe("string");
// The response should mention "4" somewhere
expect(body.text).toContain("4");
});
// ─── Chat: longer creative response ──────────────────────────────────────
test("POST /chat with a creative prompt returns substantial text", async ({
request,
}) => {
const res = await request.post("/chat", {
data: { text: "Write a haiku about software testing." },
});
expect(res.status()).toBe(200);
const body = await res.json();
expect(body.text).toBeTruthy();
// A haiku is at least 20 characters
expect(body.text.length).toBeGreaterThan(20);
});
// ─── Chat: knowledge / factual ───────────────────────────────────────────
test("POST /chat with a factual question returns accurate data", async ({
request,
}) => {
const res = await request.post("/chat", {
data: {
text: "What is the capital of France? Reply in one word.",
},
});
expect(res.status()).toBe(200);
const body = await res.json();
expect(body.text).toBeTruthy();
expect(body.text.toLowerCase()).toContain("paris");
});
// ─── Character system prompt respected ────────────────────────────────────
test("agent respects its system prompt (concise answers)", async ({
request,
}) => {
const res = await request.post("/chat", {
data: {
text: "Explain quantum computing.",
},
});
expect(res.status()).toBe(200);
const body = await res.json();
expect(body.text).toBeTruthy();
// System prompt says "Keep answers short (1-3 sentences)".
// We verify the response is not excessively long (< 1500 chars is reasonable).
expect(body.text.length).toBeLessThan(1500);
expect(body.text.length).toBeGreaterThan(10);
});
// ─── Error handling ──────────────────────────────────────────────────────
test("POST /chat with empty text returns 400", async ({ request }) => {
const res = await request.post("/chat", {
data: { text: "" },
});
expect(res.status()).toBe(400);
const body = await res.json();
expect(body.error).toBeTruthy();
});
test("POST /chat with missing text field returns 400", async ({
request,
}) => {
const res = await request.post("/chat", {
data: {},
});
expect(res.status()).toBe(400);
const body = await res.json();
expect(body.error).toBeTruthy();
});
// ─── 404 fallback ────────────────────────────────────────────────────────
test("GET /nonexistent returns 404", async ({ request }) => {
const res = await request.get("/nonexistent");
expect(res.status()).toBe(404);
});
}
+441
View File
@@ -0,0 +1,441 @@
/**
* Playwright global setup: boots a real AgentRuntime with a live LLM provider
* and exposes it through a lightweight HTTP server on port 13789.
*/
import http from "node:http";
import { v4 as uuidv4 } from "uuid";
import { DEFAULT_CEREBRAS_TEXT_MODEL } from "../../src/contracts/service-routing";
import { InMemoryDatabaseAdapter } from "../../src/database/inMemoryAdapter";
import { AgentRuntime } from "../../src/runtime";
import { detectInferenceProviders } from "../../src/testing/inference-provider";
import { createOllamaModelHandlers } from "../../src/testing/ollama-provider";
import type { Character, Memory, Plugin, UUID } from "../../src/types";
import { ChannelType } from "../../src/types";
import { loadEnvFile } from "../../src/utils/environment";
const PORT = 13789;
const TEST_CHARACTER: Character = {
name: "E2ETestAgent",
system:
"You are a concise, helpful assistant used for end-to-end testing. " +
"Always respond in plain text. Keep answers short (1-3 sentences) unless asked otherwise.",
bio: ["E2E test agent for Playwright integration tests"],
templates: {},
messageExamples: [],
postExamples: [],
topics: ["testing"],
adjectives: ["helpful", "concise"],
knowledge: [],
plugins: [],
secrets: {},
settings: {},
};
/**
* Resolve the correct model-provider plugin.
*
* IMPORTANT: this runs from `packages/core` inside a nested workspace
* (`eliza/eliza/...`). Bun's bare-specifier resolution can pick the published
* `@elizaos/plugin-openai` hoisted at `eliza/node_modules` instead of the
* local workspace copy under `eliza/plugins/plugin-openai/`. The published
* 2.0.0-alpha.537 build only forwards `params.prompt`, while the local source
* also forwards `params.messages` (which the v5 planner relies on). So we
* resolve the workspace plugins via explicit relative file imports first and
* fall back to bare-specifier resolution when no local source exists.
*/
async function importWorkspacePlugin(
relativeFromHere: string,
bareSpecifier: string,
): Promise<Record<string, unknown> | null> {
try {
const mod = (await import(relativeFromHere)) as Record<string, unknown>;
console.log(
`[e2e] loaded ${bareSpecifier} from workspace: ${relativeFromHere}`,
);
return mod;
} catch (err) {
console.warn(
`[e2e] workspace import failed for ${relativeFromHere}: ${(err as Error).message}; falling back to bare specifier`,
);
try {
const mod = (await import(bareSpecifier)) as Record<string, unknown>;
console.log(
`[e2e] loaded ${bareSpecifier} via bare specifier (likely published copy)`,
);
return mod;
} catch {
return null;
}
}
}
async function resolveProviderPlugin(
providerName: string,
): Promise<Plugin | null> {
switch (providerName) {
case "openai": {
const mod = await importWorkspacePlugin(
"../../../../plugins/plugin-openai/index.ts",
"@elizaos/plugin-openai",
);
if (!mod) return null;
return ((mod.openaiPlugin ?? mod.default) as Plugin | undefined) ?? null;
}
case "anthropic": {
const mod = await importWorkspacePlugin(
"../../../../plugins/plugin-anthropic/index.ts",
"@elizaos/plugin-anthropic",
);
if (!mod) return null;
return (
((mod.anthropicPlugin ?? mod.default) as Plugin | undefined) ?? null
);
}
case "groq": {
const mod = await importWorkspacePlugin(
"../../../../plugins/plugin-groq/index.ts",
"@elizaos/plugin-groq",
);
if (!mod) return null;
return ((mod.groqPlugin ?? mod.default) as Plugin | undefined) ?? null;
}
case "google": {
const mod = await importWorkspacePlugin(
"../../../../plugins/plugin-google-genai/index.ts",
"@elizaos/plugin-google-genai",
);
if (!mod) return null;
return (mod.default as Plugin | undefined) ?? null;
}
default:
return null;
}
}
/** Tiny JSON body parser. */
function readBody(req: http.IncomingMessage): Promise<string> {
return new Promise((resolve, reject) => {
const chunks: Buffer[] = [];
req.on("data", (c: Buffer) => chunks.push(c));
req.on("end", () => resolve(Buffer.concat(chunks).toString("utf-8")));
req.on("error", reject);
});
}
async function verifyInferenceProvider(runtime: AgentRuntime): Promise<void> {
await runtime.generateText("Reply with OK.", {
modelType: "TEXT_LARGE" as "TEXT_LARGE",
maxTokens: 8,
});
}
function applyProviderSettings(
runtime: AgentRuntime,
providerName: string,
): void {
switch (providerName) {
case "openai": {
const openAiKey = process.env.OPENAI_API_KEY?.trim() ?? "";
const cerebrasKey = process.env.CEREBRAS_API_KEY?.trim() ?? "";
const explicitBase = process.env.OPENAI_BASE_URL?.trim();
const explicitElizaProvider = process.env.ELIZA_PROVIDER?.trim();
const isCerebras =
explicitElizaProvider?.toLowerCase() === "cerebras" ||
/^csk-/i.test(openAiKey || cerebrasKey) ||
/(^|\.)cerebras\.ai(\/|$)/i.test(explicitBase ?? "");
runtime.setSetting(
"OPENAI_API_KEY",
openAiKey || (isCerebras ? cerebrasKey : ""),
true,
);
if (cerebrasKey) {
runtime.setSetting("CEREBRAS_API_KEY", cerebrasKey, true);
}
if (explicitBase) {
runtime.setSetting("OPENAI_BASE_URL", explicitBase, true);
} else if (isCerebras) {
// Cerebras keys are OpenAI-compatible but must not hit api.openai.com.
runtime.setSetting(
"OPENAI_BASE_URL",
"https://api.cerebras.ai/v1",
true,
);
runtime.setSetting("ELIZA_PROVIDER", "cerebras", true);
}
if (explicitElizaProvider) {
runtime.setSetting("ELIZA_PROVIDER", explicitElizaProvider, true);
} else if (isCerebras) {
runtime.setSetting("ELIZA_PROVIDER", "cerebras", true);
}
if (isCerebras) {
const cerebrasModel =
process.env.OPENAI_LARGE_MODEL?.trim() ||
process.env.OPENAI_SMALL_MODEL?.trim() ||
process.env.LARGE_MODEL?.trim() ||
process.env.SMALL_MODEL?.trim() ||
DEFAULT_CEREBRAS_TEXT_MODEL;
runtime.setSetting(
"OPENAI_SMALL_MODEL",
process.env.OPENAI_SMALL_MODEL?.trim() || cerebrasModel,
);
runtime.setSetting(
"OPENAI_LARGE_MODEL",
process.env.OPENAI_LARGE_MODEL?.trim() || cerebrasModel,
);
runtime.setSetting(
"SMALL_MODEL",
process.env.SMALL_MODEL?.trim() ||
process.env.OPENAI_SMALL_MODEL?.trim() ||
cerebrasModel,
);
runtime.setSetting(
"LARGE_MODEL",
process.env.LARGE_MODEL?.trim() ||
process.env.OPENAI_LARGE_MODEL?.trim() ||
cerebrasModel,
);
}
break;
}
case "anthropic":
runtime.setSetting(
"ANTHROPIC_API_KEY",
process.env.ANTHROPIC_API_KEY ?? "",
true,
);
break;
case "google":
runtime.setSetting(
"GOOGLE_GENERATIVE_AI_API_KEY",
process.env.GOOGLE_API_KEY ??
process.env.GOOGLE_AI_API_KEY ??
process.env.GOOGLE_GENERATIVE_AI_API_KEY ??
"",
true,
);
break;
case "groq":
runtime.setSetting("GROQ_API_KEY", process.env.GROQ_API_KEY ?? "", true);
runtime.setSetting(
"GROQ_SMALL_MODEL",
process.env.GROQ_SMALL_MODEL ?? "openai/gpt-oss-120b",
);
runtime.setSetting(
"GROQ_LARGE_MODEL",
process.env.GROQ_LARGE_MODEL ?? "openai/gpt-oss-120b",
);
break;
}
}
export default async function globalSetup(): Promise<void> {
process.env.ELIZA_PLAYWRIGHT_E2E = "1";
// Load repo-local credentials before provider detection so Playwright e2e
// behaves the same way as the rest of the workspace.
loadEnvFile();
// ── 1. Detect inference provider ───────────────────────────────────────
const detection = await detectInferenceProviders();
if (!detection.hasProvider || !detection.primaryProvider) {
console.error(
"\n[e2e] No inference provider available. Skipping E2E tests.\n" +
"Set CEREBRAS_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, or start Ollama.\n",
);
process.env.__E2E_SKIP__ = "1";
return;
}
const provider = detection.primaryProvider;
console.log(`\n[e2e] Using provider: ${provider.name}\n`);
// ── 2. Load provider plugin ────────────────────────────────────────────
const providerPlugin = await resolveProviderPlugin(provider.name);
// ── 3. Create runtime ──────────────────────────────────────────────────
const agentId = uuidv4() as UUID;
const plugins: Plugin[] = [];
if (providerPlugin) {
plugins.push(providerPlugin);
}
const adapter = new InMemoryDatabaseAdapter(agentId);
await adapter.init();
const runtime = new AgentRuntime({
agentId,
character: { ...TEST_CHARACTER, id: agentId },
adapter,
plugins,
checkShouldRespond: false, // always respond in tests
logLevel: "warn",
});
applyProviderSettings(runtime, provider.name);
// For Ollama without a plugin package, register model handlers directly.
if (provider.name === "ollama" && !providerPlugin) {
const handlers = createOllamaModelHandlers();
for (const [modelType, handler] of Object.entries(handlers)) {
if (handler) {
runtime.registerModel(
modelType,
handler as (
rt: AgentRuntime,
p: Record<string, unknown>,
) => Promise<unknown>,
"ollama",
);
}
}
}
await runtime.initialize();
console.log("[e2e] Runtime initialized");
try {
await verifyInferenceProvider(runtime);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.error(
`\n[e2e] Provider preflight failed. Skipping E2E tests.\n${message}\n`,
);
process.env.__E2E_SKIP__ = "1";
await runtime.stop();
return;
}
// ── 4. Prepare a default room & entity for chat ────────────────────────
const worldId = uuidv4() as UUID;
await runtime.createWorld({ id: worldId, name: "e2e-world", agentId });
const roomId = uuidv4() as UUID;
await runtime.ensureRoomExists({
id: roomId,
name: "e2e-chat",
source: "e2e",
type: ChannelType.API,
worldId,
});
await runtime.ensureParticipantInRoom(agentId, roomId);
const testEntityId = uuidv4() as UUID;
await runtime.createEntity({
id: testEntityId,
names: ["E2ETester"],
agentId,
});
await runtime.ensureParticipantInRoom(testEntityId, roomId);
// ── 5. Start HTTP server ───────────────────────────────────────────────
const server = http.createServer(async (req, res) => {
res.setHeader("Content-Type", "application/json");
try {
// GET /health
if (req.method === "GET" && req.url === "/health") {
res.writeHead(200);
res.end(JSON.stringify({ ok: true }));
return;
}
// GET /status
if (req.method === "GET" && req.url === "/status") {
res.writeHead(200);
res.end(
JSON.stringify({
agentId,
name: TEST_CHARACTER.name,
provider: provider.name,
ready: true,
}),
);
return;
}
// POST /chat — drives the FULL agent message pipeline via
// runtime.messageService.handleMessage so providers, evaluators, and
// trajectory recording all run. No generateText shortcut here.
if (req.method === "POST" && req.url === "/chat") {
const raw = await readBody(req);
const body = JSON.parse(raw) as {
text?: string;
roomId?: string;
entityId?: string;
};
if (!body.text || typeof body.text !== "string" || !body.text.trim()) {
res.writeHead(400);
res.end(JSON.stringify({ error: "text is required" }));
return;
}
if (!runtime.messageService) {
res.writeHead(500);
res.end(
JSON.stringify({ error: "messageService unavailable on runtime" }),
);
return;
}
const chatRoomId = (body.roomId as UUID) ?? roomId;
const chatEntityId = (body.entityId as UUID) ?? testEntityId;
const message: Memory = {
id: uuidv4() as UUID,
entityId: chatEntityId,
roomId: chatRoomId,
content: {
text: body.text.trim(),
source: "e2e",
},
createdAt: Date.now(),
};
let responseText = "";
const callback = async (content: { text: string }) => {
if (typeof content?.text === "string") {
responseText += content.text;
}
return [];
};
await runtime.messageService.handleMessage(runtime, message, callback);
res.writeHead(200);
res.end(
JSON.stringify({
text: responseText,
agentId,
roomId: chatRoomId,
}),
);
return;
}
// fallback
res.writeHead(404);
res.end(JSON.stringify({ error: "not found" }));
} catch (err) {
console.error("[e2e] Server error:", err);
res.writeHead(500);
res.end(
JSON.stringify({
error: err instanceof Error ? err.message : "internal error",
}),
);
}
});
await new Promise<void>((resolve) => {
server.listen(PORT, () => {
console.log(`[e2e] Test server listening on http://localhost:${PORT}`);
resolve();
});
});
// Store for teardown
(globalThis as Record<string, unknown>).__e2eServer = server;
(globalThis as Record<string, unknown>).__e2eRuntime = runtime;
}
@@ -0,0 +1,26 @@
/**
* Playwright global teardown: stops the test HTTP server and the AgentRuntime.
*/
import type http from "node:http";
import type { IAgentRuntime } from "../../src/types";
export default async function globalTeardown(): Promise<void> {
const server = (globalThis as Record<string, unknown>).__e2eServer as
| http.Server
| undefined;
const runtime = (globalThis as Record<string, unknown>).__e2eRuntime as
| IAgentRuntime
| undefined;
if (server) {
await new Promise<void>((resolve) => {
server.close(() => resolve());
});
console.log("[e2e] Test server stopped");
}
if (runtime) {
await runtime.stop();
console.log("[e2e] Runtime stopped");
}
}
+22
View File
@@ -0,0 +1,22 @@
{
"$schema": "https://unpkg.com/knip@5/schema.json",
"playwright": false,
"playwright-ct": false,
"playwright-test": false,
"entry": ["src/index.ts"],
"project": ["src/**/*.ts", "!src/**/*.d.ts", "build.ts"],
"ignore": [
"e2e/**",
"test/**",
"src/testing/**",
"src/__tests__/**",
"src/index.browser.ts",
"src/streaming-context.browser.ts",
"src/index.edge.ts",
"src/features/working-memory/attachmentContext.ts",
"src/features/working-memory/readAttachmentAction.ts",
"src/features/working-memory/taskClipboardPersistence.ts",
"src/features/working-memory/taskClipboardService.ts",
"src/services/triggerScheduling.ts"
]
}
+194
View File
@@ -0,0 +1,194 @@
{
"name": "@elizaos/core",
"version": "2.0.3-beta.7",
"description": "Core runtime, plugin abstractions, and AgentRuntime for elizaOS — actions, providers, evaluators, services, models, and types.",
"homepage": "https://github.com/elizaOS/eliza",
"repository": {
"type": "git",
"url": "git+https://github.com/elizaOS/eliza.git",
"directory": "packages/core"
},
"keywords": [
"elizaos",
"agent",
"runtime",
"ai",
"llm",
"plugins"
],
"engines": {
"node": ">=24.0.0"
},
"type": "module",
"main": "./dist/node/index.node.js",
"module": "./dist/node/index.node.js",
"types": "./dist/index.d.ts",
"browser": "./dist/browser/index.browser.js",
"sideEffects": true,
"exports": {
"./package.json": "./package.json",
".": {
"types": "./dist/index.d.ts",
"eliza-source": {
"types": "./src/index.ts",
"import": "./src/index.ts",
"default": "./src/index.ts"
},
"browser": {
"types": "./dist/index.browser.d.ts",
"import": "./dist/browser/index.browser.js",
"default": "./dist/browser/index.browser.js"
},
"node": {
"types": "./dist/index.node.d.ts",
"import": "./dist/node/index.node.js",
"default": "./dist/node/index.node.js"
},
"bun": {
"types": "./dist/index.node.d.ts",
"import": "./dist/node/index.node.js",
"default": "./dist/node/index.node.js"
},
"default": "./dist/node/index.node.js"
},
"./node": {
"types": "./dist/index.node.d.ts",
"eliza-source": {
"types": "./src/index.node.ts",
"import": "./src/index.node.ts",
"default": "./src/index.node.ts"
},
"import": "./dist/node/index.node.js",
"default": "./dist/node/index.node.js"
},
"./browser": {
"types": "./dist/index.browser.d.ts",
"eliza-source": {
"types": "./src/index.browser.ts",
"import": "./src/index.browser.ts",
"default": "./src/index.browser.ts"
},
"import": "./dist/browser/index.browser.js",
"default": "./dist/browser/index.browser.js"
},
"./roles": {
"types": "./dist/roles.d.ts",
"import": "./dist/roles.js",
"default": "./dist/roles.js"
},
"./testing": {
"types": "./dist/testing/index.d.ts",
"import": "./dist/testing/index.js",
"default": "./dist/testing/index.js"
},
"./security/*": {
"types": "./dist/security/*.d.ts",
"eliza-source": {
"types": "./src/security/*.ts",
"import": "./src/security/*.ts",
"default": "./src/security/*.ts"
},
"import": "./dist/security/*.js",
"default": "./dist/security/*.js"
},
"./services/*": {
"types": "./dist/services/*.d.ts",
"import": "./dist/services/*.js",
"default": "./dist/services/*.js"
},
"./*.css": "./dist/*.css",
"./*": {
"types": "./dist/*.d.ts",
"import": "./dist/*.js",
"default": "./dist/*.js"
}
},
"files": [
"registry-entry.json",
"dist"
],
"scripts": {
"prebuild": "bun run --cwd ../logger build && bun run --cwd ../contracts build && bun run --cwd ../cloud/routing build && ([ -f src/i18n/generated/validation-keyword-data.ts ] || node ../shared/scripts/generate-keywords.mjs)",
"prepublishOnly": "bun run build",
"build": "bun run build.ts",
"build:node": "bun run build.ts --node-only",
"build:watch": "bun run build.ts --watch",
"dev": "bun run build:watch",
"clean": "node ../scripts/rm-path-recursive.mjs dist .turbo .turbo-tsconfig.json *.tsbuildinfo && node scripts/clean-src-artifacts.mjs",
"perf:settings": "bun run scripts/perf-settings.ts",
"test": "node ../scripts/run-vitest.mjs run",
"test:coverage": "mkdir -p coverage/.tmp && node ../scripts/run-vitest.mjs run --coverage",
"test:e2e": "npx playwright test",
"test:e2e:smoke": "node scripts/run-e2e-smoke.mjs",
"test:watch": "node ../scripts/run-vitest.mjs",
"lint": "bunx @biomejs/biome check --write ./src",
"lint:check": "bunx @biomejs/biome check ./src",
"typecheck": "node ../shared/scripts/generate-keywords.mjs && tsgo --noEmit -p ./tsconfig.json",
"format": "bunx @biomejs/biome format --write ./src",
"format:check": "bunx @biomejs/biome format ./src"
},
"author": "elizaOS",
"license": "MIT",
"devDependencies": {
"@playwright/test": "^1.52.0",
"@types/bun": "^1.3.5",
"@types/fast-redact": "^3.0.4",
"@types/fs-extra": "^11.0.4",
"@types/markdown-it": "^14.1.2",
"@types/node": "^25.0.3",
"@types/react-test-renderer": "^19.0.0",
"@vitest/coverage-v8": "^4.0.0",
"react-test-renderer": "^19.0.0",
"typescript": "^6.0.3",
"vitest": "^4.0.0"
},
"dependencies": {
"@ai-sdk/anthropic": "^3.0.13",
"@ai-sdk/gateway": "3.0.133",
"@ai-sdk/google": "^3.0.8",
"@ai-sdk/openai": "^3.0.9",
"@ai-sdk/provider": "3.0.13",
"@ai-sdk/provider-utils": "5.0.5",
"@bufbuild/protobuf": "^2.12.0",
"@elizaos/cloud-routing": "workspace:*",
"@elizaos/contracts": "workspace:*",
"@elizaos/logger": "workspace:*",
"@elizaos/prompts": "workspace:*",
"@noble/ciphers": "2.2.0",
"@noble/hashes": "2.2.0",
"@openrouter/ai-sdk-provider": "^2.0.0",
"@opentelemetry/api": "1.9.1",
"adze": "^2.2.5",
"ai": "^6.0.30",
"dedent": "^1.7.1",
"dotenv": "^17.2.3",
"drizzle-orm": "0.45.2",
"fast-redact": "^3.5.0",
"file-type": "^22.0.0",
"fs-extra": "^11.3.2",
"handlebars": "^4.7.8",
"json5": "^2.2.3",
"mammoth": "^1.9.0",
"markdown-it": "^14.1.0",
"underscore": "^1.13.8",
"unpdf": "^1.4.0",
"uuid": "^14.0.0",
"yaml": "^2.7.0",
"zod": "^4.4.3"
},
"publishConfig": {
"access": "public"
},
"gitHead": "cb192f7b21c441e9463d1af2f890c145814baad2",
"elizaos": {
"scripts": {
"coreBuild": true,
"testLanes": [
"server"
],
"buildModel": {
"doubleCheck": true
}
}
}
}
+24
View File
@@ -0,0 +1,24 @@
/** Configures Playwright e2e execution for @elizaos/core runtime smoke tests. */
import { defineConfig } from "@playwright/test";
process.env.ELIZA_PLAYWRIGHT_E2E = "1";
export default defineConfig({
testDir: "./e2e",
globalSetup: "./e2e/setup/global-setup.ts",
globalTeardown: "./e2e/setup/global-teardown.ts",
timeout: 120_000,
expect: {
timeout: 30_000,
},
use: {
baseURL: "http://localhost:13789",
},
projects: [
{
name: "e2e",
use: {},
},
],
reporter: [["list"]],
});
+33
View File
@@ -0,0 +1,33 @@
{
"id": "agent-orchestrator",
"name": "Agent Orchestrator",
"description": "Task-agent orchestration plugin - spawn and manage open-ended CLI task agents via PTY",
"npmName": "@elizaos/core",
"version": "2.0.0-beta.0",
"source": "bundled",
"tags": [
"orchestrator",
"agents",
"tasks",
"orchestration",
"agent-orchestrator",
"agent"
],
"config": {},
"render": {
"visible": true,
"pinTo": [],
"style": "card",
"icon": "Target",
"group": "agents",
"groupOrder": 7,
"actions": ["enable", "configure"]
},
"resources": {
"homepage": "https://github.com/elizaos/eliza#readme",
"repository": "https://github.com/elizaos/eliza"
},
"dependsOn": [],
"kind": "plugin",
"subtype": "agents"
}
@@ -0,0 +1,60 @@
#!/usr/bin/env node
// Cross-platform replacement for the bash pipeline:
// find src -type f \( -name '*.js' -o -name '*.js.map' \
// -o -name '*.d.ts' -o -name '*.d.ts.map' \) \
// ! -path 'src/types/generated/*' -delete 2>/dev/null || true
//
// Removes emitted artifacts from src/ that older build setups left behind,
// preserving anything under src/types/generated/. No-throw: failures during
// the sweep are swallowed so `clean` stays idempotent (matches the bash
// `|| true` tail).
import { readdirSync, statSync, unlinkSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
const here = path.dirname(fileURLToPath(import.meta.url));
const pkgRoot = path.resolve(here, "..");
const srcRoot = path.join(pkgRoot, "src");
const preserveRoot = path.join(srcRoot, "types", "generated");
const EXTS = new Set([".js", ".js.map", ".d.ts", ".d.ts.map"]);
function endsWithAny(name) {
for (const ext of EXTS) if (name.endsWith(ext)) return true;
return false;
}
function walk(dir) {
let entries;
try {
entries = readdirSync(dir, { withFileTypes: true });
} catch {
return;
}
for (const entry of entries) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
// Skip the preserve root entirely (matches the `! -path 'src/types/generated/*'`).
if (full === preserveRoot) continue;
walk(full);
continue;
}
if (!entry.isFile()) continue;
if (!endsWithAny(entry.name)) continue;
try {
unlinkSync(full);
} catch {
// Match the bash `2>/dev/null || true` semantics.
}
}
}
try {
statSync(srcRoot);
} catch {
process.exit(0);
}
walk(srcRoot);
process.exit(0);
+61
View File
@@ -0,0 +1,61 @@
/** Captures deterministic performance-setting fixtures for core runtime benchmark runs. */
import { decryptStringValue, encryptStringValue } from "../src/settings";
function formatNumber(value: number): string {
return new Intl.NumberFormat("en-US", { maximumFractionDigits: 2 }).format(
value,
);
}
function benchmark(label: string, iterations: number, fn: () => void): number {
const start = performance.now();
for (let i = 0; i < iterations; i += 1) {
fn();
}
const elapsedMs = performance.now() - start;
const opsPerSec = (iterations / elapsedMs) * 1000;
// eslint-disable-next-line no-console
console.log(
`${label}: ${formatNumber(opsPerSec)} ops/sec (${formatNumber(elapsedMs)} ms total)`,
);
return opsPerSec;
}
const iterations = Number(process.env.ITERATIONS ?? "50000");
if (!Number.isFinite(iterations) || iterations <= 0) {
throw new Error("ITERATIONS must be a positive number");
}
// Use an explicit salt so this benchmark is not affected by env/production checks.
const salt = `perf-${crypto.randomUUID()}`;
const corpus = Array.from(
{ length: 256 },
(_, i) => `value-${i}-${crypto.randomUUID()}`,
);
// Quick correctness check (fail fast if crypto is broken).
const roundtrip = decryptStringValue(encryptStringValue("hello", salt), salt);
if (roundtrip !== "hello") {
throw new Error("settings crypto roundtrip failed");
}
let cursor = 0;
let lastEncrypted = "";
benchmark("encryptStringValue (v2 AES-GCM)", iterations, () => {
const value = corpus[cursor];
cursor = (cursor + 1) % corpus.length;
lastEncrypted = encryptStringValue(value, salt);
});
cursor = 0;
benchmark("decryptStringValue (v2 AES-GCM)", iterations, () => {
const value = corpus[cursor];
cursor = (cursor + 1) % corpus.length;
const encrypted = encryptStringValue(value, salt);
lastEncrypted = decryptStringValue(encrypted, salt);
});
// Prevent dead-code elimination / ensure something observable.
// eslint-disable-next-line no-console
console.log(`last=${lastEncrypted.slice(0, 24)}`);
+112
View File
@@ -0,0 +1,112 @@
/** Runs the core e2e smoke harness with package-local defaults. */
import { spawnSync } from "node:child_process";
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
const packageRoot = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
"..",
);
const truthyValues = new Set(["1", "true", "yes", "on"]);
const specPath = path.join(packageRoot, "e2e", "runtime-live.e2e.spec.ts");
const defaultOllamaUrls = ["http://127.0.0.1:11434", "http://localhost:11434"];
function envFlagEnabled(name) {
const value = process.env[name]?.trim().toLowerCase();
return value ? truthyValues.has(value) : false;
}
function getConfiguredProviderUrl() {
return process.env.OLLAMA_BASE_URL || process.env.OLLAMA_HOST || null;
}
async function canReachOllama(baseUrl) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 1000);
try {
const response = await fetch(`${baseUrl}/api/tags`, {
signal: controller.signal,
});
return response.ok;
} catch {
return false;
} finally {
clearTimeout(timeout);
}
}
async function resolveProviderEnv() {
if (
process.env.OPENAI_API_KEY ||
process.env.ANTHROPIC_API_KEY ||
process.env.GROQ_API_KEY ||
process.env.OPENROUTER_API_KEY ||
process.env.GOOGLE_GENERATIVE_AI_API_KEY ||
process.env.GEMINI_API_KEY ||
process.env.GOOGLE_API_KEY
) {
return {};
}
const configuredOllamaUrl = getConfiguredProviderUrl();
if (configuredOllamaUrl) {
return {
OLLAMA_BASE_URL: configuredOllamaUrl,
OLLAMA_HOST: configuredOllamaUrl,
OLLAMA_URL: configuredOllamaUrl,
};
}
for (const url of defaultOllamaUrls) {
if (await canReachOllama(url)) {
return {
OLLAMA_BASE_URL: url,
OLLAMA_HOST: url,
OLLAMA_URL: url,
};
}
}
return null;
}
function skip(reason) {
console.log(`[eliza/typescript] Skipping e2e smoke because ${reason}.`);
process.exit(0);
}
if (envFlagEnabled("ELIZA_SKIP_ELIZA_LIVE_SMOKE")) {
skip("ELIZA_SKIP_ELIZA_LIVE_SMOKE=1");
}
if (!fs.existsSync(specPath)) {
skip("the runtime live e2e spec is not available in this checkout");
}
const providerEnv = await resolveProviderEnv();
if (!providerEnv) {
skip("no live inference provider is configured");
}
const bunCommand = process.env.npm_execpath || process.env.BUN || "bun";
const result = spawnSync(
bunCommand,
["x", "playwright", "test", "e2e/runtime-live.e2e.spec.ts"],
{
cwd: packageRoot,
stdio: "inherit",
env: {
...process.env,
...providerEnv,
},
},
);
if (result.error?.code === "ENOENT") {
skip(`the Playwright runner could not be launched: ${result.error.message}`);
}
process.exit(result.status ?? 1);
@@ -0,0 +1,86 @@
/**
* Verifies the memoized action-catalog builder (`getCachedActionCatalog`)
* returns a stable instance for an unchanged action set and self-invalidates
* when actions are registered, unregistered, or a localized-example resolver is
* supplied. Deterministic: synthetic Action stubs, no model or database.
*/
import { describe, expect, it } from "vitest";
import { getCachedActionCatalog } from "../services/message";
import type { Action } from "../types/components";
// Minimal valid Action for catalog construction. The catalog is built from the
// action name/description/similes; validate/handler are unused by the builder.
function mkAction(name: string): Action {
return {
name,
description: `Performs the ${name} operation for the user.`,
similes: [],
examples: [],
validate: async () => true,
handler: async () => {},
} as unknown as Action;
}
function hasAction(
catalog: ReturnType<typeof getCachedActionCatalog>,
name: string,
): boolean {
return catalog.parents.some((parent) => parent.name === name);
}
describe("action catalog cache (F2: memoization + self-invalidation)", () => {
it("returns the same cached catalog for an unchanged action set", () => {
const actions = [mkAction("ALPHA_ACT"), mkAction("BETA_ACT")];
const first = getCachedActionCatalog(actions);
const second = getCachedActionCatalog(actions);
// Same instance => the second call was a cache hit (no rebuild).
expect(second).toBe(first);
expect(hasAction(first, "ALPHA_ACT")).toBe(true);
expect(hasAction(first, "BETA_ACT")).toBe(true);
});
it("invalidates when a new action is registered (e.g. a plugin/view action)", () => {
const base = [mkAction("BASE_ONE"), mkAction("BASE_TWO")];
const before = getCachedActionCatalog(base);
expect(hasAction(before, "VIEW_SCOPED_ACT")).toBe(false);
// A view/plugin registers a new action -> different name set -> new key.
const withView = [...base, mkAction("VIEW_SCOPED_ACT")];
const after = getCachedActionCatalog(withView);
expect(after).not.toBe(before);
// The newly registered action MUST appear in the next catalog — this is
// the property the agent's "call view-dependent actions" depends on.
expect(hasAction(after, "VIEW_SCOPED_ACT")).toBe(true);
expect(hasAction(after, "BASE_ONE")).toBe(true);
});
it("invalidates when an action is unregistered", () => {
const full = [
mkAction("KEEP_A"),
mkAction("KEEP_B"),
mkAction("REMOVE_ME"),
];
const before = getCachedActionCatalog(full);
expect(hasAction(before, "REMOVE_ME")).toBe(true);
const fewer = [mkAction("KEEP_A"), mkAction("KEEP_B")];
const after = getCachedActionCatalog(fewer);
expect(hasAction(after, "REMOVE_ME")).toBe(false);
expect(hasAction(after, "KEEP_A")).toBe(true);
});
it("does not cache when a localized-example resolver is active", () => {
// The resolver depends on the recent message, so the catalog is
// message-specific and must be rebuilt every turn (never cached).
const actions = [mkAction("LOC_ALPHA"), mkAction("LOC_BETA")];
const resolver = () => undefined;
const first = getCachedActionCatalog(actions, resolver);
const second = getCachedActionCatalog(actions, resolver);
expect(second).not.toBe(first);
// Still correct content, just rebuilt.
expect(hasAction(first, "LOC_ALPHA")).toBe(true);
expect(hasAction(second, "LOC_ALPHA")).toBe(true);
});
});
@@ -0,0 +1,186 @@
/**
* Static source-tree audit that greps the real repo for direct
* `action.handler(...)` callsites and asserts every one is classified in the
* allowlist — so voiced-response handling stays intentional — with no stale
* entries left behind. Reads files from disk; no model or database.
*/
import { readdirSync, readFileSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
const repoRoot = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
"../../../..",
);
const allowedCallsites = new Map<string, string>([
[
"packages/core/src/runtime/execute-planned-tool-call.ts",
"central planned-action executor; attributes callbacks before message-service voice rewrite",
],
[
"packages/core/src/runtime.ts",
"hook-mode executor; attributes callbacks before message-service voice rewrite",
],
[
"packages/core/src/features/advanced-planning/services/planning-service.ts",
"advanced-planning executor; attributes callbacks before message-service voice rewrite",
],
[
"packages/core/src/features/advanced-capabilities/actions/message.ts",
"message router; attributes callbacks to routed child actions",
],
[
"packages/core/src/features/advanced-capabilities/actions/room.ts",
"room router; attributes callbacks to routed child actions",
],
[
"packages/agent/src/actions/page-action-groups.ts",
"page router; attributes callbacks to routed child actions",
],
[
"packages/agent/src/api/chat-routes.ts",
"direct chat task-dispatch bypass; rewrites callback text with TEXT_SMALL",
],
[
"packages/agent/src/api/fallback-action-helpers.ts",
"fallback action bypass; rewrites action callback output with TEXT_SMALL",
],
[
"packages/scenario-runner/src/executor.ts",
"scenario action turns; rewrites action response text with TEXT_SMALL",
],
[
"plugins/plugin-agent-skills/src/actions/skill.ts",
"skill router; attributes callbacks to routed child actions",
],
[
"plugins/plugin-agent-skills/src/binance/direct-dispatch.ts",
"Binance direct-skill dispatcher; attributes fallback/USE_SKILL callbacks and rewrites the routed action's response text for voice (rewriteFallbackActionText)",
],
[
"plugins/plugin-linear/src/actions/routers.ts",
"Linear router; attributes callbacks to routed child actions",
],
[
"plugins/plugin-music/src/actions/music.ts",
"music router; attributes callbacks to routed child actions",
],
[
"plugins/plugin-personal-assistant/src/actions/calendar.ts",
"calendar router; attributes callbacks to routed child actions",
],
[
"plugins/plugin-app-manager/src/api/apps-routes.ts",
"app-manager direct API dispatch; rewrites action response text with TEXT_SMALL",
],
[
"plugins/plugin-coding-tools/src/services/coding-task-executor.ts",
"coding-task executor; rewrites action response text with TEXT_SMALL",
],
[
"plugins/plugin-agent-orchestrator/src/services/skill-lifeops-context-broker.ts",
"LifeOps broker direct dispatch; rewrites action response text with TEXT_SMALL",
],
[
"plugins/plugin-app-control/src/workers/app-worker-entry.ts",
"internal app sandbox RPC; not a chat/user message surface",
],
[
"plugins/plugin-app-control/src/actions/views.ts",
"VIEWS close-alias dispatcher (CLOSE_VIEW / CLOSE_ALL_VIEWS); forwards to the underlying VIEWS handler",
],
[
"plugins/plugin-commands/src/actions/handlers.ts",
"slash-command dispatcher (/compact -> COMPACT_CONVERSATION via runCompactAction); forwards the original callback to the dispatched action and returns result.text as a CommandResult reply",
],
[
"packages/core/src/services/message.ts",
"shortcut-gate (runShortcutGate, #8792): fires a slash-command action; its callback only captures content.text (emits nothing) and the captured text is returned as a Stage1 direct_reply, so the normal message-service voice rewrite still applies",
],
]);
const actionHandlerCallPattern = new RegExp(
[
"action\\.handler\\(",
"route\\.action\\.handler\\(",
"args\\.action\\.handler\\(",
"childAction\\.handler\\(",
"createTaskAction\\.handler\\(",
"googleCalendarAction\\.handler\\(",
"roomOpAction\\.handler\\(",
"appAction\\.handler\\(",
"playbackOp\\.handler\\(",
"playAudio\\.handler\\(",
"musicLibraryAction\\.handler\\(",
"manageRouting\\.handler\\(",
"manageZones\\.handler\\(",
].join("|"),
);
const searchRoots = [
"packages/core/src",
"packages/agent/src",
"packages/scenario-runner/src",
"plugins",
];
const excludedDirNames = new Set([
"dist",
"node_modules",
"test",
"tests",
"scripts",
]);
function isExcludedFile(relPath: string): boolean {
return (
relPath.endsWith(".d.ts") ||
relPath.endsWith(".test.ts") ||
relPath.endsWith(".spec.ts")
);
}
function isScannableSource(name: string): boolean {
return name.endsWith(".ts") || name.endsWith(".tsx");
}
function walk(absDir: string, found: Set<string>): void {
let entries: ReturnType<typeof readdirSync>;
try {
entries = readdirSync(absDir, { withFileTypes: true });
} catch {
return;
}
for (const entry of entries) {
const abs = path.join(absDir, entry.name);
if (entry.isDirectory()) {
if (excludedDirNames.has(entry.name)) continue;
walk(abs, found);
continue;
}
if (!entry.isFile() || !isScannableSource(entry.name)) continue;
const rel = path.relative(repoRoot, abs).split(path.sep).join("/");
if (isExcludedFile(rel)) continue;
const content = readFileSync(abs, "utf8");
if (actionHandlerCallPattern.test(content)) found.add(rel);
}
}
describe("action handler callsite audit", () => {
it("keeps direct action.handler callers classified for voiced response handling", () => {
const foundSet = new Set<string>();
for (const root of searchRoots) {
walk(path.join(repoRoot, root), foundSet);
}
const found = [...foundSet].sort();
const unexpected = found.filter((file) => !allowedCallsites.has(file));
const stale = [...allowedCallsites.keys()].filter(
(file) => !found.includes(file),
);
expect(unexpected).toEqual([]);
expect(stale).toEqual([]);
});
});
@@ -0,0 +1,342 @@
/**
* Structural guards over the generated canonical action-docs aggregate: retired
* action names stay out, core-owned owner surfaces stay in while plugin-owned
* ones stay out, umbrella actions expose the `action` discriminator, and the
* fallback overlay preserves a plugin Action's own docs. Deterministic: imports
* the generated aggregate and concrete Action objects, no model or database.
*/
import { describe, expect, it } from "vitest";
import { withCanonicalActionDocs } from "../action-docs.ts";
import { secretsAction } from "../features/secrets/actions/manage-secret.ts";
import { trustAction } from "../features/trust/actions/trust.ts";
import { allActionDocs } from "../generated/action-docs.ts";
import type { Action } from "../types/index.ts";
const RETIRED_GENERATED_ACTION_NAMES = [
"ASK_USER_QUESTION",
"CHECKIN",
"CLEAR_HISTORY",
"CREATE_PLAN",
"CREATE_PAYMENT_REQUEST",
"DESKTOP",
"DEVICE_FILE_READ",
"DEVICE_FILE_WRITE",
"DEVICE_LIST_DIR",
"DISCORD_SETUP_CREDENTIALS",
"DOC",
"ENTER_WORKTREE",
"EXIT_WORKTREE",
"DELIVER_PAYMENT_LINK",
"FIRST_RUN",
"FORM_RESTORE",
"LIFE",
"PROFILE",
"RELATIONSHIP",
"MONEY",
"PAYMENTS",
"SUBSCRIPTIONS",
"READING",
"SCHEDULE",
"BOOK_TRAVEL",
"SCHEDULING_NEGOTIATION",
"SEND_TO_ADMIN",
"DEVICE_INTENT",
"MESSAGE_HANDOFF",
"APP_BLOCK",
"WEBSITE_BLOCK",
"AUTOFILL",
"PASSWORD_MANAGER",
"GOOGLE_CALENDAR",
"NOSTR_PUBLISH_PROFILE",
"PLACE_CALL",
"READ_ATTACHMENT",
"SHELL_HISTORY",
"SHELL_COMMAND",
"START_TUNNEL",
"STOP_TUNNEL",
"GET_TUNNEL_STATUS",
"TAILSCALE",
"READ",
"WRITE",
"EDIT",
"GREP",
"GLOB",
"LS",
"WEB_FETCH",
"CREATE_TODO",
"COMPLETE_TODO",
"LIST_TODOS",
"MYSTICISM_PAYMENT",
"VERIFY_PAYMENT_PAYLOAD",
"SETTLE_PAYMENT",
"AWAIT_PAYMENT_CALLBACK",
"CANCEL_PAYMENT_REQUEST",
"EDIT_TODO",
"DELETE_TODO",
"TOKEN_INFO",
"BIRDEYE_SEARCH",
// Trust leaves consolidated under the single TRUST umbrella with
// action=evaluate|record_interaction|request_elevation|update_role.
"EVALUATE_TRUST",
"RECORD_TRUST_INTERACTION",
"REQUEST_ELEVATION",
"TRUST_UPDATE_ROLE",
// Secrets leaves consolidated under the single SECRETS umbrella with
// action=get|set|delete|list|check|mirror|request. The previous
// top-level MANAGE_SECRET / SET_SECRET / atomic leaves are gone — the
// planner sees only SECRETS and the separate SECRETS_UPDATE_SETTINGS.
// No virtual promoted subactions (SECRETS_GET, SECRETS_SET, …) are
// registered; callers supply action=<verb> on the umbrella directly.
"MANAGE_SECRET",
"SET_SECRET",
"GET_SECRET",
"LIST_SECRETS",
"CHECK_SECRET",
"DELETE_SECRET",
"MIRROR_SECRET_TO_VAULT",
"REQUEST_SECRET",
// Suno music generation absorbed into the MUSIC umbrella with
// action=generate|extend|custom_generate. The plugin-suno package still
// ships the SunoProvider client and status provider, but no longer
// registers MUSIC_GENERATION as a top-level Action.
"MUSIC_GENERATION",
// Page-group umbrella parents collapsed into the single PAGE_DELEGATE
// owner-only parent in packages/agent/src/actions/page-action-groups.ts.
"BROWSER_ACTIONS",
"WALLET_ACTIONS",
"CHARACTER_ACTIONS",
"SETTINGS_ACTIONS",
"CONNECTOR_ACTIONS",
"AUTOMATION_ACTIONS",
"PHONE_ACTIONS",
"OWNER_ACTIONS",
// Dead agent actions deleted from packages/agent/src/actions/. ANALYZE_IMAGE
// (media.ts), EXTRACT_PAGE (extract-page.ts), QUERY_TRAJECTORIES
// (trajectories.ts), and SKILL_COMMAND (skill-command.ts) were exported but
// never registered in eliza-plugin.ts. Their source files have been
// removed; SKILL_COMMAND callers should route through USE_SKILL.
"ANALYZE_IMAGE",
"EXTRACT_PAGE",
"QUERY_TRAJECTORIES",
"SKILL_COMMAND",
] as const;
/**
* Canonical replacement for the eight retired per-page `<PAGE>_ACTIONS`
* umbrellas. PAGE_DELEGATE lives in packages/agent and cannot be imported from
* this core test (core does not depend on agent), so the structural guard is
* twofold: PAGE_DELEGATE must not appear in the retired list above, and every
* page name it replaces must.
*/
const PAGE_DELEGATE_REPLACES = [
"BROWSER_ACTIONS",
"WALLET_ACTIONS",
"CHARACTER_ACTIONS",
"SETTINGS_ACTIONS",
"CONNECTOR_ACTIONS",
"AUTOMATION_ACTIONS",
"PHONE_ACTIONS",
"OWNER_ACTIONS",
] as const;
// Owner-surface actions that must stay resolvable to the planner. PAYMENT is
// defined in packages/core (core-owned) and is baked into the generated
// aggregate. The rest are plugin-owned (SCHEDULED_TASKS / CREDENTIALS /
// PERSONAL_ASSISTANT / OWNER_DOCUMENTS in plugin-personal-assistant, FILE in
// plugin-coding-tools). Per arch-audit #12092 item 29 their docs are no longer
// baked into packages/core: each plugin's Action object carries its own docs and
// the fallback-only overlay (withCanonicalActionDocs) resolves them at
// registration. The real no-regression property is therefore threefold —
// core-owned surfaces stay in the aggregate, plugin-owned surfaces are absent
// from it, and the overlay preserves an Action's own docs when the aggregate has
// no row for it.
const CORE_OWNED_OWNER_SURFACE_ACTION_NAMES = ["PAYMENT"] as const;
const PLUGIN_OWNED_OWNER_SURFACE_ACTION_NAMES = [
"SCHEDULED_TASKS",
"CREDENTIALS",
"PERSONAL_ASSISTANT",
"OWNER_DOCUMENTS",
"FILE",
] as const;
const LEGACY_DISCRIMINATORS = new Set([
"subaction",
"op",
"operation",
"verb",
"subAction",
"__subaction",
]);
describe("action structure audit guards", () => {
it("keeps retired action names out of generated canonical docs", () => {
const names = new Set(allActionDocs.map((action) => action.name));
for (const retired of RETIRED_GENERATED_ACTION_NAMES) {
expect(names.has(retired), retired).toBe(false);
}
});
it("keeps core-owned owner surfaces in the aggregate", () => {
const names = new Set(allActionDocs.map((action) => action.name));
const retired = new Set<string>(RETIRED_GENERATED_ACTION_NAMES);
for (const name of CORE_OWNED_OWNER_SURFACE_ACTION_NAMES) {
expect(retired.has(name), `${name} must not be marked retired`).toBe(
false,
);
expect(names.has(name), `${name} must be generated`).toBe(true);
}
});
it("does not bake plugin-owned owner surfaces into the core aggregate", () => {
// Item 29: plugin-owned action docs must not live in packages/core so that
// editing a plugin's action docs no longer forces a core regen + rebuild.
const names = new Set(allActionDocs.map((action) => action.name));
const retired = new Set<string>(RETIRED_GENERATED_ACTION_NAMES);
for (const name of PLUGIN_OWNED_OWNER_SURFACE_ACTION_NAMES) {
expect(retired.has(name), `${name} must not be marked retired`).toBe(
false,
);
expect(
names.has(name),
`${name} is plugin-owned and must not be baked into the core aggregate`,
).toBe(false);
}
});
it("resolves plugin-owned owner surfaces from the Action object via the fallback overlay", () => {
// The real no-regression property: because the overlay is fallback-only, a
// plugin Action that carries its own description/similes/parameters resolves
// unchanged even though the core aggregate holds no row for it. This is what
// makes dropping the plugin-owned rows safe.
for (const name of PLUGIN_OWNED_OWNER_SURFACE_ACTION_NAMES) {
const pluginAction: Action = {
name,
description: `${name} owner-surface description carried by the plugin`,
similes: [`${name}_ALIAS`],
parameters: [
{
name: "action",
description: "operation to perform",
required: true,
schema: { type: "string" },
},
],
validate: async () => true,
handler: async () => {},
};
const resolved = withCanonicalActionDocs(pluginAction);
expect(resolved.description).toBe(pluginAction.description);
expect(resolved.similes).toEqual(pluginAction.similes);
expect(resolved.parameters?.map((parameter) => parameter.name)).toEqual([
"action",
]);
// The overlay only fills the compressed alias; it never overwrites the
// Action's own fields.
expect(typeof resolved.descriptionCompressed).toBe("string");
}
});
it("requires schemas with legacy discriminator aliases to expose action", () => {
const failures: string[] = [];
for (const action of allActionDocs) {
const parameterNames = new Set(
(action.parameters ?? []).map((parameter) => parameter.name),
);
const hasLegacy = [...parameterNames].some((name) =>
LEGACY_DISCRIMINATORS.has(name),
);
if (hasLegacy && !parameterNames.has("action")) {
failures.push(action.name);
}
}
expect(failures).toEqual([]);
});
it("TRUST umbrella uses canonical action discriminator with all subactions", () => {
expect(trustAction.name).toBe("TRUST");
const discriminator = (trustAction.parameters ?? []).find(
(parameter) => parameter.name === "action",
);
expect(
discriminator,
"TRUST must declare an `action` parameter",
).toBeDefined();
const schema = discriminator?.schema as { enum?: string[] } | undefined;
expect(schema?.enum).toBeDefined();
expect(new Set(schema?.enum ?? [])).toEqual(
new Set([
"evaluate",
"record_interaction",
"request_elevation",
"update_role",
]),
);
});
it("PAGE_DELEGATE replaces the eight per-page _ACTIONS umbrellas", () => {
const retired = new Set<string>(RETIRED_GENERATED_ACTION_NAMES);
expect(
retired.has("PAGE_DELEGATE"),
"PAGE_DELEGATE is the canonical replacement and must not be marked retired",
).toBe(false);
for (const legacy of PAGE_DELEGATE_REPLACES) {
expect(
retired.has(legacy),
`${legacy} must be retired in favor of PAGE_DELEGATE`,
).toBe(true);
}
const names = new Set(allActionDocs.map((action) => action.name));
for (const legacy of PAGE_DELEGATE_REPLACES) {
expect(
names.has(legacy),
`${legacy} must not appear in canonical docs`,
).toBe(false);
}
});
it("SECRETS umbrella uses canonical action discriminator with all subactions", () => {
expect(secretsAction.name).toBe("SECRETS");
const discriminator = (secretsAction.parameters ?? []).find(
(parameter) => parameter.name === "action",
);
expect(
discriminator,
"SECRETS must declare an `action` parameter",
).toBeDefined();
const schema = discriminator?.schema as { enum?: string[] } | undefined;
expect(schema?.enum).toBeDefined();
expect(new Set(schema?.enum ?? [])).toEqual(
new Set(["get", "set", "delete", "list", "check", "mirror", "request"]),
);
});
it("todo atomic leaf names are retired and absent from generated canonical docs", () => {
// The five core atomic todo leaves (CREATE_TODO, COMPLETE_TODO, LIST_TODOS,
// EDIT_TODO, DELETE_TODO) were removed from packages/core advancedActions.
// The canonical todo planner surfaces are:
// - TODO (plugin-todos): general user-scoped todo store
// - OWNER_TODOS (app-lifeops): owner-specific definition-tracking store
// Neither is registered in packages/core itself; both are plugin-provided.
const TODO_LEAF_NAMES = [
"CREATE_TODO",
"COMPLETE_TODO",
"LIST_TODOS",
"EDIT_TODO",
"DELETE_TODO",
] as const;
const retired = new Set<string>(RETIRED_GENERATED_ACTION_NAMES);
const generated = new Set(allActionDocs.map((action) => action.name));
for (const name of TODO_LEAF_NAMES) {
expect(
retired.has(name),
`${name} must be in the retired-name guard`,
).toBe(true);
expect(
generated.has(name),
`${name} must not appear as a top-level action in generated docs`,
).toBe(false);
}
});
});
@@ -0,0 +1,362 @@
/**
* Covers the plaintext serializers that flatten PTY task lifecycle events,
* typed agent-event streams, and trajectory summaries into bounded
* human-readable lines. Deterministic: pure functions over synthetic event
* objects, no model or database.
*/
import { describe, expect, it } from "vitest";
import {
activityEventToPlaintext,
trajectoryEventToPlaintext,
trajectoryToPlaintext,
} from "../activity-plaintext";
describe("activityEventToPlaintext", () => {
it("summarizes pty task lifecycle events without trusting malformed fields", () => {
expect(
activityEventToPlaintext({
eventType: "task_registered",
sessionId: "session-1",
data: { label: "Ship serializer tests" },
}),
).toEqual({
eventType: "task_registered",
plaintext: "Task started: Ship serializer tests",
sessionId: "session-1",
});
expect(
activityEventToPlaintext({
eventType: "tool_running",
data: { description: "bun test packages/core" },
}),
)?.toMatchObject({
eventType: "tool_running",
plaintext: "Running bun test packages/core",
});
expect(
activityEventToPlaintext({
eventType: "tool_running",
data: {
toolCall: {
title: "Terminal",
kind: "shell",
rawInput: { command: "bun run typecheck" },
},
},
}),
)?.toMatchObject({
eventType: "tool_running",
plaintext: "Running Terminal: bun run typecheck",
});
});
it("keeps the assistant activity stream mapped to canonical event types", () => {
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "assistant",
payload: {
source: "proactive-goal-check-in",
text: "Review the stalled weekly goal.",
},
}),
).toEqual({
eventType: "check-in",
plaintext: "Review the stalled weekly goal.",
stream: "assistant",
source: "proactive-goal-check-in",
});
});
it("does not surface unknown assistant sources unless explicitly requested", () => {
const event = {
type: "agent_event",
stream: "assistant",
payload: {
source: "experimental-source",
text: "A raw assistant event",
},
};
expect(activityEventToPlaintext(event)).toBeNull();
expect(
activityEventToPlaintext(event, { includeUnknownAssistantText: true }),
)?.toMatchObject({
eventType: "experimental-source",
plaintext: "A raw assistant event",
});
});
it("summarizes typed agent event streams instead of dropping rich runtime work", () => {
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "action",
sessionKey: "room-1",
payload: {
type: "complete",
actionName: "BRIEF",
duration: 1250,
output: { briefingId: "brief-1" },
},
}),
).toEqual({
eventType: "action_complete",
plaintext: 'Action completed: BRIEF (1.3s): {"briefingId":"brief-1"}',
stream: "action",
sessionId: "room-1",
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "tool",
payload: {
type: "tool_error",
toolName: "web_fetch",
error: "Request blocked",
},
}),
)?.toMatchObject({
eventType: "tool_error",
plaintext: "Tool failed: web_fetch: Request blocked",
stream: "tool",
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "message",
payload: {
type: "received",
channel: "discord",
content: "Can you check this?",
hasAttachments: true,
},
}),
)?.toMatchObject({
eventType: "message_received",
plaintext:
"Message received on discord with attachments: Can you check this?",
});
});
it("summarizes lifecycle, evaluator, provider, memory, assistant, and error streams", () => {
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "lifecycle",
payload: { type: "run_end", success: true, duration: 2000 },
}),
)?.toMatchObject({
eventType: "run_end",
plaintext: "Run completed (2.0s)",
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "evaluator",
payload: {
type: "complete",
evaluatorName: "fact-check",
validated: false,
result: { reason: "missing source" },
},
}),
)?.toMatchObject({
eventType: "evaluator_complete",
plaintext:
'Evaluator completed without validation: fact-check: {"reason":"missing source"}',
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "provider",
payload: {
type: "complete",
providerName: "calendar",
fromCache: true,
data: { count: 3 },
},
}),
)?.toMatchObject({
eventType: "provider_cached",
plaintext: 'Provider served from cache: calendar: {"count":3}',
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "memory",
payload: {
type: "search",
tableName: "memories",
count: 2,
duration: 30,
},
}),
)?.toMatchObject({
eventType: "memory_search",
plaintext: "Memory searched in memories (2 results) (30ms)",
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "assistant",
payload: {
type: "plan",
content: "Check inbox, then draft reply.",
},
}),
)?.toMatchObject({
eventType: "assistant_plan",
plaintext: "Assistant plan: Check inbox, then draft reply.",
});
expect(
activityEventToPlaintext({
type: "agent_event",
stream: "error",
payload: {
type: "warning",
code: "LISTENER_ERROR",
message: "One listener failed",
},
}),
)?.toMatchObject({
eventType: "warning",
plaintext: "Warning LISTENER_ERROR: One listener failed",
});
});
it("surfaces streamed message/reasoning/plan/lifecycle events as text", () => {
expect(
activityEventToPlaintext({
eventType: "message",
sessionId: "s1",
data: { text: "Applying the patch now" },
}),
)?.toMatchObject({
eventType: "message",
plaintext: "Applying the patch now",
});
expect(
activityEventToPlaintext({
eventType: "reasoning",
data: { text: "The failing import is stale" },
}),
)?.toMatchObject({
eventType: "reasoning",
plaintext: "Thinking: The failing import is stale",
});
expect(
activityEventToPlaintext({
eventType: "plan",
data: {
entries: [
{ content: "read file", status: "completed" },
{ content: "edit", status: "in_progress" },
{ content: "test", status: "pending" },
],
},
}),
)?.toMatchObject({
eventType: "plan",
plaintext: "Plan updated (1/3 done)",
});
expect(
activityEventToPlaintext({ eventType: "ready", data: {} }),
)?.toMatchObject({ plaintext: "Agent ready" });
expect(
activityEventToPlaintext({ eventType: "login_required", data: {} }),
)?.toMatchObject({ plaintext: "Login required" });
expect(
activityEventToPlaintext({ eventType: "reconnected", data: {} }),
)?.toMatchObject({ plaintext: "Agent reconnected" });
});
it("drops empty streamed chunks rather than echoing the event name", () => {
expect(
activityEventToPlaintext({ eventType: "message", data: { text: "" } }),
).toBeNull();
expect(
activityEventToPlaintext({ eventType: "plan", data: { entries: [] } }),
).toBeNull();
});
});
describe("trajectory plaintext serializers", () => {
it("renders a bounded trajectory summary with LLM calls and provider accesses", () => {
const text = trajectoryToPlaintext(
{
trajectory: {
id: "traj-1",
agentId: "agent-1",
source: "scenario",
status: "completed",
startTime: 1000,
endTime: 2500,
durationMs: 1500,
llmCallCount: 1,
providerAccessCount: 1,
totalPromptTokens: 42,
totalCompletionTokens: 7,
createdAt: "2026-06-24T18:00:00.000Z",
},
llmCalls: [
{
stepId: "step-1",
provider: "openai",
model: "gpt-test",
purpose: "planner",
response: "Call the tool.",
},
],
providerAccesses: [
{
stepId: "step-1",
providerName: "goals",
purpose: "context",
query: { owner: "self" },
},
],
},
{ maxItems: 2 },
);
expect(text).toContain("Trajectory traj-1 (completed)");
expect(text).toContain("source: scenario; duration: 1.5s");
expect(text).toContain("tokens: 42 prompt / 7 completion");
expect(text).toContain("- planner openai/gpt-test: Call the tool.");
expect(text).toContain('- goals context: {"owner":"self"}');
});
it("summarizes trajectory events with stable plain text", () => {
expect(
trajectoryEventToPlaintext({
id: "tool-1",
type: "tool_error",
actionName: "WEB_FETCH",
error: "Request blocked",
}),
).toBe("WEB_FETCH failed: Request blocked");
expect(
trajectoryEventToPlaintext({
id: "cache-1",
type: "cache_observation",
cacheName: "prompt",
hit: true,
key: "segment-a",
}),
).toBe("prompt hit: segment-a");
});
});
@@ -0,0 +1,171 @@
/**
* Exercises `composePromptFromState` with both string and function (callback)
* templates, and confirms `characterSchema` accepts function template values
* while rejecting non-string/non-function ones. Deterministic: pure prompt
* composition, no model.
*/
import { describe, expect, it } from "vitest";
import { characterSchema } from "../schemas/character";
import type { Character } from "../types/agent";
import type { State } from "../types/state";
import { composePromptFromState } from "../utils";
describe("Character Callback Templates", () => {
it("should support string templates (existing behavior)", async () => {
const template = "Hello, {{agentName}}!";
const state: State = {
agentName: "Alice",
values: {},
data: {},
text: "",
};
const result = composePromptFromState({ state, template });
expect(result).toContain("Alice");
});
it("should support callback templates that return strings", async () => {
const template = ({
state,
}: {
state: State | Record<string, unknown>;
}) => {
const agentName =
typeof state === "object" && "agentName" in state
? (state.agentName as string)
: "Unknown";
return `Hello, ${agentName}!`;
};
const state: State = {
agentName: "Bob",
values: {},
data: {},
text: "",
};
const result = composePromptFromState({ state, template });
expect(result).toContain("Bob");
});
it("should support callbacks with dynamic logic", async () => {
const template = ({
state,
}: {
state: State | Record<string, unknown>;
}) => {
const stateObj = state as Record<string, unknown>;
const tone = stateObj.formalTone ? "formal" : "casual";
return `Respond in a {{tone}} manner. {{agentName}} here.`.replace(
"{{tone}}",
tone,
);
};
const state: State = {
agentName: "Charlie",
formalTone: true,
values: {},
data: {},
text: "",
};
const result = composePromptFromState({ state, template });
expect(result).toContain("formal");
expect(result).toContain("Charlie");
});
it("should handle mixed string and callback templates in character", async () => {
const stringTemplate = "Simple string template";
const callbackTemplate = ({
state,
}: {
state: State | Record<string, unknown>;
}) => {
const stateObj = state as Record<string, unknown>;
return `Dynamic template with ${stateObj.agentName || "agent"}`;
};
const character: Character = {
name: "TestAgent",
templates: { stringTemplate, callbackTemplate },
};
expect(character.templates).toBeDefined();
const state: State = {
agentName: "TestAgent",
values: {},
data: {},
text: "",
};
// Test string template
const stringResult = composePromptFromState({
state,
template: stringTemplate,
});
expect(stringResult).toBe("Simple string template");
// Test callback template
const callbackResult = composePromptFromState({
state,
template: callbackTemplate,
});
expect(callbackResult).toContain("Dynamic template");
expect(callbackResult).toContain("TestAgent");
});
it("should allow callbacks to access full state context", async () => {
const template = ({
state,
}: {
state: State | Record<string, unknown>;
}) => {
const stateObj = state as State;
return `Agent: {{agentName}}, Context: {{roomId}}`.replace(
/\{\{(\w+)\}\}/g,
(_match, key) => {
const value = stateObj[key as keyof State];
return typeof value === "string" ? value : String(value);
},
);
};
const state: State = {
agentName: "David",
roomId: "room-123",
values: {},
data: {},
text: "",
};
const result = composePromptFromState({ state, template });
expect(result).toContain("David");
expect(result).toContain("room-123");
});
});
describe("characterSchema templates field accepts callbacks", () => {
it("accepts string template values (baseline)", () => {
const result = characterSchema.safeParse({
name: "Tmpl",
templates: { greeting: "Hello there" },
});
expect(result.success).toBe(true);
});
it("accepts a function template value", () => {
const result = characterSchema.safeParse({
name: "Tmpl",
templates: { rateLimitedReply: () => "slow down a sec" },
});
expect(result.success).toBe(true);
});
it("rejects a template value that is neither string nor function", () => {
const result = characterSchema.safeParse({
name: "Tmpl",
templates: { bad: 123 },
});
expect(result.success).toBe(false);
});
});
@@ -0,0 +1,192 @@
/**
* Exercises the `compose_state_providers` pipeline hook on
* `AgentRuntime.composeState`: a hook may filter and add providers, pass through
* unchanged, and a corrupted or thrown hook falls back to the pre-hook
* selection. Uses a real in-memory AgentRuntime with synthetic providers; no
* database or model.
*/
import { describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import type { Character, Memory, Provider, UUID } from "../types";
const ROOM_ID = "11111111-1111-1111-1111-111111111111" as UUID;
const ENTITY_ID = "22222222-2222-2222-2222-222222222222" as UUID;
function makeProvider(
name: string,
text: string,
extra: Partial<Provider> = {},
): Provider {
return {
name,
get: async () => ({ text, values: {}, data: {} }),
...extra,
};
}
function makeMessage(id: string): Memory {
return {
id: id as UUID,
entityId: ENTITY_ID,
roomId: ROOM_ID,
content: { text: "gm" },
};
}
function newRuntime(name: string): AgentRuntime {
const runtime = new AgentRuntime({ character: { name } as Character });
runtime.registerProvider(makeProvider("WALLET", "WALLET_BALANCE_HEAVY"));
runtime.registerProvider(makeProvider("GREETING", "HELLO_THERE"));
// Dynamic providers are excluded from default selection; a hook can opt one in.
runtime.registerProvider(
makeProvider("CRYPTO_SWAP", "SWAP_READY", { dynamic: true }),
);
return runtime;
}
describe("compose_state_providers pipeline hook", () => {
it("runs every registered (non-dynamic) provider when no hook is set", async () => {
const runtime = newRuntime("compose-hook-baseline");
const state = await runtime.composeState(
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"),
null,
false,
true,
);
const order = state.data.providerOrder as string[];
expect(order).toContain("WALLET");
expect(order).toContain("GREETING");
// CRYPTO_SWAP is dynamic — never in the default set without opt-in.
expect(order).not.toContain("CRYPTO_SWAP");
expect(state.text).toContain("WALLET_BALANCE_HEAVY");
});
it("lets a hook filter out and add providers by name", async () => {
const runtime = newRuntime("compose-hook-filter");
runtime.registerPipelineHook({
id: "intent-filter",
phase: "compose_state_providers",
handler: (_rt, ctx) => {
if (ctx.phase !== "compose_state_providers") return;
ctx.providers.current = [
...ctx.providers.current.filter((n) => n !== "WALLET"),
"CRYPTO_SWAP",
];
},
});
const state = await runtime.composeState(
makeMessage("bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"),
null,
false,
true,
);
const order = state.data.providerOrder as string[];
expect(order).not.toContain("WALLET");
expect(order).toContain("GREETING");
// A dynamic provider named explicitly by the hook is pulled in.
expect(order).toContain("CRYPTO_SWAP");
expect(state.text).toContain("SWAP_READY");
expect(state.text).toContain("HELLO_THERE");
expect(state.text).not.toContain("WALLET_BALANCE_HEAVY");
});
it("is a pure pass-through when the hook returns the list unmodified", async () => {
const runtime = newRuntime("compose-hook-passthrough");
runtime.registerPipelineHook({
id: "noop",
phase: "compose_state_providers",
handler: () => {
// no-op
},
});
const state = await runtime.composeState(
makeMessage("cccccccc-cccc-cccc-cccc-cccccccccccc"),
null,
false,
true,
);
const order = state.data.providerOrder as string[];
expect(order).toContain("WALLET");
expect(order).toContain("GREETING");
expect(order).not.toContain("CRYPTO_SWAP");
});
it("keeps the pre-hook selection when a hook corrupts providers.current", async () => {
const runtime = newRuntime("compose-hook-nonarray");
runtime.registerPipelineHook({
id: "corrupt",
phase: "compose_state_providers",
handler: (_rt, ctx) => {
if (ctx.phase !== "compose_state_providers") return;
// Type-erased at runtime: a buggy hook hands back a non-array.
(ctx.providers as { current: unknown }).current = "WALLET";
},
});
const state = await runtime.composeState(
makeMessage("eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee"),
null,
false,
true,
);
const order = state.data.providerOrder as string[];
// Falls back to the pre-hook set instead of crashing or emptying.
expect(order).toContain("WALLET");
expect(order).toContain("GREETING");
});
it("does not crash when a hook throws after a partial mutation", async () => {
const runtime = newRuntime("compose-hook-throw");
runtime.registerPipelineHook({
id: "throws",
phase: "compose_state_providers",
handler: (_rt, ctx) => {
if (ctx.phase !== "compose_state_providers") return;
ctx.providers.current = ctx.providers.current.filter(
(n) => n !== "WALLET",
);
throw new Error("boom");
},
});
// The thrown error is swallowed by invokePipelineHooks; composeState resolves.
const state = await runtime.composeState(
makeMessage("ffffffff-ffff-ffff-ffff-ffffffffffff"),
null,
false,
true,
);
const order = state.data.providerOrder as string[];
expect(order).toContain("GREETING");
});
it("surfaces onlyInclude and the message to the hook", async () => {
const runtime = newRuntime("compose-hook-context");
let seenOnlyInclude: boolean | undefined;
let seenMessageId: string | undefined;
let seenNames: string[] | undefined;
runtime.registerPipelineHook({
id: "capture",
phase: "compose_state_providers",
handler: (_rt, ctx) => {
if (ctx.phase !== "compose_state_providers") return;
seenOnlyInclude = ctx.onlyInclude;
seenMessageId = ctx.message.id;
seenNames = [...ctx.providers.current];
},
});
await runtime.composeState(
makeMessage("dddddddd-dddd-dddd-dddd-dddddddddddd"),
["GREETING"],
true,
true,
);
expect(seenOnlyInclude).toBe(true);
expect(seenMessageId).toBe("dddddddd-dddd-dddd-dddd-dddddddddddd");
// onlyInclude=true ⇒ the proposed set is exactly the caller's include-list.
expect(seenNames).toEqual(["GREETING"]);
});
});
@@ -0,0 +1,130 @@
/**
* Exercises the `refreshProviders` argument to `AgentRuntime.composeState`:
* cached providers are reused while only the named ones re-run, an omitted set
* re-runs everything, and an uncached provider runs even when unnamed. Uses a
* real in-memory AgentRuntime with call-counting providers; no database or
* model.
*/
import { describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import type { Character, Memory, Provider, UUID } from "../types";
const ROOM_ID = "11111111-1111-1111-1111-111111111111" as UUID;
const ENTITY_ID = "22222222-2222-2222-2222-222222222222" as UUID;
/** Provider whose text changes on every run, so reuse vs re-run is observable. */
function countingProvider(name: string): {
provider: Provider;
calls: () => number;
} {
let n = 0;
return {
provider: {
name,
get: async () => {
n += 1;
return { text: `${name}#${n}`, values: {}, data: {} };
},
},
calls: () => n,
};
}
function makeMessage(id: string): Memory {
return {
id: id as UUID,
entityId: ENTITY_ID,
roomId: ROOM_ID,
content: { text: "gm" },
};
}
describe("composeState refreshProviders", () => {
it("reuses cached providers and re-runs only the named one", async () => {
const runtime = new AgentRuntime({
character: { name: "refresh-test" } as Character,
});
const a = countingProvider("AAA");
const b = countingProvider("BBB");
const c = countingProvider("CCC");
runtime.registerProvider(a.provider);
runtime.registerProvider(b.provider);
runtime.registerProvider(c.provider);
const message = makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa");
const include = ["AAA", "BBB", "CCC"];
// First compose runs all three and caches them.
const first = await runtime.composeState(message, include, true, false);
expect(first.text).toContain("AAA#1");
expect(first.text).toContain("BBB#1");
expect(first.text).toContain("CCC#1");
// Second compose refreshes ONLY BBB; AAA/CCC are reused from cache.
const second = await runtime.composeState(message, include, true, false, [
"BBB",
]);
// Only BBB re-ran.
expect(a.calls()).toBe(1);
expect(b.calls()).toBe(2);
expect(c.calls()).toBe(1);
// The full set still drives order + text, with BBB refreshed.
expect(second.data.providerOrder).toEqual(["AAA", "BBB", "CCC"]);
expect(second.text).toContain("AAA#1"); // reused
expect(second.text).toContain("BBB#2"); // refreshed
expect(second.text).toContain("CCC#1"); // reused
expect(second.text).not.toContain("BBB#1");
});
it("without refreshProviders re-runs every requested provider (default unchanged)", async () => {
const runtime = new AgentRuntime({
character: { name: "refresh-default" } as Character,
});
const a = countingProvider("AAA");
const b = countingProvider("BBB");
runtime.registerProvider(a.provider);
runtime.registerProvider(b.provider);
const message = makeMessage("bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb");
const include = ["AAA", "BBB"];
await runtime.composeState(message, include, true, false);
const second = await runtime.composeState(message, include, true, false);
// No refresh set ⇒ both re-run (pre-existing behavior preserved).
expect(a.calls()).toBe(2);
expect(b.calls()).toBe(2);
expect(second.text).toContain("AAA#2");
expect(second.text).toContain("BBB#2");
});
it("runs an uncached provider even when not named in refreshProviders", async () => {
const runtime = new AgentRuntime({
character: { name: "refresh-uncached" } as Character,
});
const a = countingProvider("AAA");
const b = countingProvider("BBB");
runtime.registerProvider(a.provider);
runtime.registerProvider(b.provider);
const message = makeMessage("cccccccc-cccc-cccc-cccc-cccccccccccc");
// First compose only caches AAA.
await runtime.composeState(message, ["AAA"], true, false);
// Now request AAA+BBB, refreshing only AAA — BBB is uncached so it must run.
const second = await runtime.composeState(
message,
["AAA", "BBB"],
true,
false,
["AAA"],
);
expect(a.calls()).toBe(2); // refreshed
expect(b.calls()).toBe(1); // uncached ⇒ ran once
expect(second.text).toContain("AAA#2");
expect(second.text).toContain("BBB#1");
});
});
@@ -0,0 +1,123 @@
/**
* Pins that the `composeState` onlyInclude path honors the explicit provider
* list without enforcing declared roleGates, for every sender kind. Stage-1
* force-includes recall providers (FACTS declares minRole USER) for all
* senders, and both unassigned humans AND relay/webhook bridges resolve to
* GUEST by default — so gate enforcement here would strip cross-turn recall
* from relayed human conversation (the ZenithProxy pattern). Uses a real
* AgentRuntime + InMemoryDatabaseAdapter with a real world and room; no model.
*/
import { describe, expect, it } from "vitest";
import { InMemoryDatabaseAdapter } from "../database/inMemoryAdapter";
import { AgentRuntime } from "../runtime";
import type { Character, Memory, Provider, UUID } from "../types";
import { ChannelType } from "../types";
const WORLD_ID = "11111111-1111-1111-1111-111111111110" as UUID;
const ROOM_ID = "11111111-1111-1111-1111-111111111111" as UUID;
const UNASSIGNED_SENDER = "22222222-2222-2222-2222-222222222221" as UUID;
function staticProvider(name: string, extra: Partial<Provider> = {}): Provider {
return {
name,
get: async () => ({ text: `${name}-content`, values: {}, data: {} }),
...extra,
};
}
async function makeRuntime(): Promise<AgentRuntime> {
const adapter = new InMemoryDatabaseAdapter();
const runtime = new AgentRuntime({
character: { name: "role-gate-test" } as Character,
adapter,
logLevel: "fatal",
});
await adapter.createWorlds([
{
id: WORLD_ID,
agentId: runtime.agentId,
name: "test world",
metadata: { roles: {} },
},
]);
await adapter.createRooms([
{
id: ROOM_ID,
agentId: runtime.agentId,
source: "test",
type: ChannelType.GROUP,
worldId: WORLD_ID,
},
]);
runtime.registerProvider(
staticProvider("GATED", { roleGate: { minRole: "USER" } }),
);
runtime.registerProvider(staticProvider("OPEN"));
return runtime;
}
function makeMessage(
id: string,
entityId: UUID,
overrides: {
contentMetadata?: Record<string, unknown>;
source?: string;
} = {},
): Memory {
return {
id: id as UUID,
entityId,
roomId: ROOM_ID,
worldId: WORLD_ID,
content: {
text: "gm",
source: overrides.source ?? "discord",
metadata: overrides.contentMetadata,
},
} as Memory;
}
describe("composeState onlyInclude ignores provider roleGates", () => {
it("keeps role-gated providers for human senders without a world role", async () => {
const runtime = await makeRuntime();
const state = await runtime.composeState(
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1", UNASSIGNED_SENDER),
["GATED", "OPEN"],
true,
true,
);
expect(state.text).toContain("GATED-content");
expect(state.text).toContain("OPEN-content");
});
it("keeps role-gated providers for bot-authored senders without a world role", async () => {
// A roleless relay webhook resolves to GUEST; if the gate were enforced
// here, FACTS-style recall providers would silently vanish from every
// relayed human turn.
const runtime = await makeRuntime();
const state = await runtime.composeState(
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa2", UNASSIGNED_SENDER, {
contentMetadata: { fromBot: true },
}),
["GATED", "OPEN"],
true,
true,
);
expect(state.text).toContain("GATED-content");
expect(state.text).toContain("OPEN-content");
});
it("keeps role-gated providers for internal bridge sources without a world role", async () => {
const runtime = await makeRuntime();
const state = await runtime.composeState(
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa3", UNASSIGNED_SENDER, {
source: "acpx:sub-agent-router",
}),
["GATED", "OPEN"],
true,
true,
);
expect(state.text).toContain("GATED-content");
expect(state.text).toContain("OPEN-content");
});
});
@@ -0,0 +1,179 @@
/**
* composeState under trajectory recording (`metadata.trajectoryStepId` set on
* every inbound message when the trajectories feature is active): recording
* forces provider re-execution so accesses are logged, but must not change
* what providers observe — the turn's cached state still reaches them.
* Regression: blanking it made RECENT_MESSAGES' turn-recompose gate read
* stage-1 on every pass, so cross-room interactions never composed on
* trajectory-recording runtimes. Real AgentRuntime + the real provider over
* a minimal in-memory adapter; no model, no database server.
*/
import { describe, expect, it, vi } from "vitest";
import { recentMessagesProvider } from "../features/basic-capabilities/providers/recentMessages";
import { AgentRuntime } from "../runtime";
import {
ChannelType,
type Character,
type IDatabaseAdapter,
type Memory,
type Provider,
type State,
type UUID,
} from "../types";
const ROOM_ID = "11111111-1111-1111-1111-111111111111" as UUID;
const OTHER_ROOM_ID = "11111111-1111-1111-1111-222222222222" as UUID;
const USER_ID = "22222222-2222-2222-2222-222222222222" as UUID;
function makeRecordedMessage(id: string, text = "gm"): Memory {
return {
id: id as UUID,
entityId: USER_ID,
roomId: ROOM_ID,
content: { text, source: "discord" },
metadata: { type: "message", trajectoryStepId: "traj-step-1" },
};
}
describe("composeState under trajectory recording", () => {
it("still hands providers the turn's cached state, so the planner recompose fetches cross-room interactions", async () => {
const runtime = new AgentRuntime({
character: { name: "Agent" } as Character,
});
const agentEntity = {
id: runtime.agentId,
agentId: runtime.agentId,
names: ["Agent"],
components: [],
};
const userEntity = {
id: USER_ID,
agentId: runtime.agentId,
names: ["User"],
components: [],
};
const getRoomsForParticipants = vi.fn(async () => [ROOM_ID, OTHER_ROOM_ID]);
const getMemoriesByRoomIds = vi.fn(async () => [
{
id: "cross-1" as UUID,
agentId: runtime.agentId,
roomId: OTHER_ROOM_ID,
entityId: USER_ID,
createdAt: 500,
content: { text: "the blue key is under the mat" },
} as Memory,
]);
runtime.registerDatabaseAdapter({
getRoomsByIds: async () => [
{
id: ROOM_ID,
agentId: runtime.agentId,
source: "discord",
type: ChannelType.GROUP,
metadata: {},
},
],
getEntitiesForRooms: async () => [
{ roomId: ROOM_ID, entities: [agentEntity, userEntity] },
],
getEntitiesByIds: async (ids: UUID[]) =>
[agentEntity, userEntity].filter((e) => ids.includes(e.id)),
getMemories: async () => [
{
id: "msg-1" as UUID,
agentId: runtime.agentId,
roomId: ROOM_ID,
entityId: USER_ID,
createdAt: 1000,
content: { text: "hello agent", source: "discord" },
} as Memory,
],
getRoomsForParticipants,
getMemoriesByRoomIds,
} as unknown as IDatabaseAdapter);
runtime.registerProvider(recentMessagesProvider);
const message = makeRecordedMessage("cccccccc-cccc-cccc-cccc-cccccccccccc");
// Stage-1 compose: first pass of the turn, lean — no cross-room fetch.
const stage1State = await runtime.composeState(
message,
["RECENT_MESSAGES"],
true,
false,
);
expect(getRoomsForParticipants).not.toHaveBeenCalled();
expect(getMemoriesByRoomIds).not.toHaveBeenCalled();
expect(stage1State.values?.recentMessageInteractions).toBe("");
// Planner recompose: RECENT_MESSAGES is already in the turn's cached
// state, so its recompose gate must see it — trajectory recording
// included — and run the cross-room interactions fetch.
const plannerState = await runtime.composeState(
message,
["RECENT_MESSAGES"],
true,
false,
["RECENT_MESSAGES"],
);
expect(getMemoriesByRoomIds).toHaveBeenCalledWith({
tableName: "messages",
roomIds: [OTHER_ROOM_ID],
limit: 20,
});
expect(plannerState.values?.recentMessageInteractions).toContain(
"the blue key is under the mat",
);
});
it("re-executes cached providers outside the refresh list so their accesses are logged", async () => {
const runtime = new AgentRuntime({
character: { name: "Agent" } as Character,
});
let factsRuns = 0;
const seenCachedProviders: string[][] = [];
const facts: Provider = {
name: "FACTS",
get: async (_runtime, _message, state: State) => {
factsRuns += 1;
seenCachedProviders.push(
Object.keys(
(state?.data?.providers as Record<string, unknown>) ?? {},
),
);
return { text: `FACTS#${factsRuns}`, values: {}, data: {} };
},
};
const recent: Provider = {
name: "RECENT_MESSAGES",
get: async () => ({ text: "recent", values: {}, data: {} }),
};
runtime.registerProvider(facts);
runtime.registerProvider(recent);
const message = makeRecordedMessage("dddddddd-dddd-dddd-dddd-dddddddddddd");
await runtime.composeState(
message,
["FACTS", "RECENT_MESSAGES"],
true,
false,
);
const plannerState = await runtime.composeState(
message,
["FACTS", "RECENT_MESSAGES"],
true,
false,
["RECENT_MESSAGES"],
);
// Recording keeps the refresh-reuse shortcut off: FACTS ran twice (its
// access is in the step's log) and its second run observed the turn's
// cached state from the first pass.
expect(factsRuns).toBe(2);
expect(seenCachedProviders[0]).toEqual([]);
expect(seenCachedProviders[1]).toEqual(
expect.arrayContaining(["FACTS", "RECENT_MESSAGES"]),
);
expect(plannerState.text).toContain("FACTS#2");
});
});
@@ -0,0 +1,97 @@
/**
* Covers `requireConfirmation`: it confirms a pending action only when the
* follow-up text is a metadata-marked wrapped external payload, and treats
* marker-shaped user text lacking that metadata as a cancellation. Deterministic:
* Map-backed runtime cache stub, no model or database.
*/
import { describe, expect, it } from "vitest";
import { wrapExternalContent } from "../security/external-content";
import type { IAgentRuntime, Memory, UUID } from "../types";
import { requireConfirmation } from "../utils/confirmation";
function createRuntimeStub(): IAgentRuntime {
const cache = new Map<string, unknown>();
return {
getCache: async <T>(key: string) => (cache.get(key) as T) ?? null,
setCache: async <T>(key: string, value: T) => {
cache.set(key, value);
return true;
},
deleteCache: async (key: string) => cache.delete(key),
} as unknown as IAgentRuntime;
}
function message(text: string, metadata?: Record<string, unknown>): Memory {
return {
id: "message-id" as UUID,
entityId: "user-id" as UUID,
roomId: "room-id" as UUID,
agentId: "agent-id" as UUID,
content: { text, source: "api", metadata },
createdAt: Date.now(),
} as Memory;
}
describe("requireConfirmation", () => {
it("confirms wrapped external follow-up text by evaluating the payload", async () => {
const runtime = createRuntimeStub();
const args = {
runtime,
actionName: "SKILL",
pendingKey: "uninstall:registry-weather",
prompt: "Uninstall registry-weather?",
metadata: { slug: "registry-weather" },
};
await expect(
requireConfirmation({
...args,
message: message('Uninstall skill "registry-weather"'),
}),
).resolves.toEqual({ status: "pending" });
const wrappedYes = wrapExternalContent(
'yes, run skill uninstall for "registry-weather"',
{ source: "api" },
);
await expect(
requireConfirmation({
...args,
message: message(wrappedYes, { externalContentWrapped: true }),
}),
).resolves.toEqual({
status: "confirmed",
metadata: { slug: "registry-weather" },
});
});
it("does not treat marker-shaped user text as wrapped without metadata", async () => {
const runtime = createRuntimeStub();
const args = {
runtime,
actionName: "SKILL",
pendingKey: "uninstall:registry-weather",
prompt: "Uninstall registry-weather?",
};
await expect(
requireConfirmation({
...args,
message: message('Uninstall skill "registry-weather"'),
}),
).resolves.toEqual({ status: "pending" });
const markerText = wrapExternalContent(
'yes, run skill uninstall for "registry-weather"',
{ source: "api" },
);
await expect(
requireConfirmation({
...args,
message: message(markerText),
}),
).resolves.toEqual({ status: "cancelled", metadata: undefined });
});
});
@@ -0,0 +1,152 @@
/**
* Covers `lintDescriptionCompressed`, the compressed action-description linter:
* empty/whitespace rejection, absence of a length cap, banned filler phrases and
* long-form words (with abbreviation hints), and non-imperative leading verbs.
* Deterministic: pure linter, no model.
*/
import { describe, expect, it } from "vitest";
import { lintDescriptionCompressed } from "../utils/description-compressed-lint";
describe("lintDescriptionCompressed", () => {
it("reports a violation for an empty string", () => {
const result = lintDescriptionCompressed("");
expect(result.ok).toBe(false);
expect(result.violations).toHaveLength(1);
expect(result.violations[0]).toMatch(/^empty:/);
});
it("reports a violation for whitespace-only input", () => {
const result = lintDescriptionCompressed(" \n\t ");
expect(result.ok).toBe(false);
expect(result.violations[0]).toMatch(/^empty:/);
});
it("accepts a clean imperative description", () => {
const result = lintDescriptionCompressed("Send a Slack DM to a user.");
expect(result.ok).toBe(true);
expect(result.violations).toEqual([]);
});
it("does not cap description length (full text reaches the model)", () => {
const text = `Send a Slack DM ${"x".repeat(500)}`;
const result = lintDescriptionCompressed(text);
expect(result.violations.some((v) => v.startsWith("length:"))).toBe(false);
});
it("flags banned filler phrases regardless of casing", () => {
const result = lintDescriptionCompressed(
"Use This Action to send a Slack DM In Order To notify a user.",
);
expect(result.ok).toBe(false);
const phraseHits = result.violations.filter((v) =>
v.startsWith("banned-phrase:"),
);
// "use this action", "this action", "the user" (no — `notify a user`),
// "in order to" — at least three distinct phrase hits.
expect(phraseHits.length).toBeGreaterThanOrEqual(3);
expect(phraseHits.some((v) => v.includes('"in order to"'))).toBe(true);
expect(phraseHits.some((v) => v.includes('"this action"'))).toBe(true);
});
it("flags every banned filler phrase from PHRASE_REPLACEMENTS keys", () => {
const phrases = [
"in order to",
"please",
"simply",
"basically",
"actually",
"currently",
"this action",
"use this action",
"the user",
"the agent",
];
for (const phrase of phrases) {
const result = lintDescriptionCompressed(`Send msg ${phrase} now`);
const hit = result.violations.find(
(v) => v.startsWith("banned-phrase:") && v.includes(`"${phrase}"`),
);
expect(hit, `expected banned-phrase hit for "${phrase}"`).toBeDefined();
}
});
it("flags long-form word forms and suggests the abbreviation", () => {
const messagesResult = lintDescriptionCompressed(
"Forward messages between channels.",
);
expect(messagesResult.ok).toBe(false);
expect(
messagesResult.violations.some(
(v) => v.startsWith("banned-word:") && v.includes('"messages"'),
),
).toBe(true);
expect(
messagesResult.violations.some((v) => v.includes('use "msgs" instead')),
).toBe(true);
const configResult = lintDescriptionCompressed(
"Update plugin configuration.",
);
expect(configResult.ok).toBe(false);
expect(
configResult.violations.some(
(v) => v.startsWith("banned-word:") && v.includes('"configuration"'),
),
).toBe(true);
expect(
configResult.violations.some((v) => v.includes('use "config" instead')),
).toBe(true);
});
it("flags non-imperative leading verbs", () => {
for (const lead of [
"It",
"This",
"Helps",
"Allows",
"Should",
"Provides",
"Returns",
"Automatically",
]) {
const result = lintDescriptionCompressed(
`${lead} to do something useful.`,
);
const hit = result.violations.find((v) =>
v.startsWith("non-imperative:"),
);
expect(hit, `expected non-imperative hit for "${lead}"`).toBeDefined();
}
});
it("does not flag imperative leads like Send/Get/List/Create", () => {
for (const lead of ["Send", "Get", "List", "Create", "Route"]) {
const result = lintDescriptionCompressed(`${lead} a Slack message.`);
expect(
result.violations.some((v) => v.startsWith("non-imperative:")),
`leading "${lead}" should not be flagged as non-imperative`,
).toBe(false);
}
});
it("returns multiple violations for a description that breaks several rules at once", () => {
const text = `This action will handle messages configuration basically simply please use this action in order to reach the user and the agent currently.`;
const result = lintDescriptionCompressed(text);
expect(result.ok).toBe(false);
const phraseHits = result.violations.filter((v) =>
v.startsWith("banned-phrase:"),
);
const wordHits = result.violations.filter((v) =>
v.startsWith("banned-word:"),
);
const leadHits = result.violations.filter((v) =>
v.startsWith("non-imperative:"),
);
expect(phraseHits.length).toBeGreaterThanOrEqual(5);
expect(wordHits.length).toBeGreaterThanOrEqual(2);
expect(leadHits.length).toBe(1);
});
});
@@ -0,0 +1,105 @@
/**
* Tests for mergeProviderOptionsWithCachePlan — the exported helper used by
* dynamicPromptExecFromState to merge per-call providerOptions with the PromptBatcher
* cache plan. Imports the real function so regressions in runtime.ts are caught here.
*/
import { describe, expect, it } from "vitest";
import { mergeProviderOptionsWithCachePlan } from "../runtime";
describe("mergeProviderOptionsWithCachePlan", () => {
it("caller nested provider fields survive alongside cache-plan additions", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "Bot" },
{ anthropic: { thinking: { type: "enabled", budgetTokens: 1024 } } },
{ anthropic: { cacheControl: { type: "ephemeral" } } },
);
expect(result.agentName).toBe("Bot");
const anthropic = result.anthropic as Record<string, unknown>;
expect(anthropic.thinking).toEqual({ type: "enabled", budgetTokens: 1024 });
expect(anthropic.cacheControl).toEqual({ type: "ephemeral" });
});
it("cache-plan nested field overwrites caller on key collision within a provider namespace", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "Bot" },
{ anthropic: { cacheControl: { type: "ephemeral", ttl: "5m" } } },
{ anthropic: { cacheControl: { type: "ephemeral", ttl: "1h" } } },
);
const anthropic = result.anthropic as Record<string, unknown>;
expect((anthropic.cacheControl as Record<string, unknown>).ttl).toBe("1h");
});
it("top-level non-object plan values replace caller values", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "Bot" },
{ openrouter: { promptCacheKey: "old" } },
{ openrouter: { promptCacheKey: "new", prompt_cache_key: "new" } },
);
const openrouter = result.openrouter as Record<string, unknown>;
expect(openrouter.promptCacheKey).toBe("new");
expect(openrouter.prompt_cache_key).toBe("new");
});
it("caller top-level provider namespaces not in the plan survive untouched", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "Bot" },
{ gateway: { caching: "auto" }, openai: { promptCacheKey: "abc" } },
{ anthropic: { cacheControl: { type: "ephemeral" } } },
);
expect(result.gateway).toEqual({ caching: "auto" });
expect(result.openai).toEqual({ promptCacheKey: "abc" });
expect((result.anthropic as Record<string, unknown>).cacheControl).toEqual({
type: "ephemeral",
});
});
it("undefined caller options still receive the full cache plan", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "Bot" },
undefined,
{
anthropic: { cacheControl: { type: "ephemeral" } },
openrouter: { promptCacheKey: "x" },
},
);
expect(result.agentName).toBe("Bot");
expect(result.anthropic).toBeDefined();
expect(result.openrouter).toBeDefined();
});
it("does not mutate the base, caller, or cache-plan inputs", () => {
const base = { agentName: "Bot" };
const caller = { anthropic: { thinking: { type: "enabled" } } };
const plan = { anthropic: { cacheControl: { type: "ephemeral" } } };
const snapshots = structuredClone({ base, caller, plan });
mergeProviderOptionsWithCachePlan(base, caller, plan);
expect({ base, caller, plan }).toEqual(snapshots);
});
it("plan scalar value wins over caller and base on key collision", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "RealBot" },
{ agentName: "CallerOverride" },
{ agentName: "PlanOverride" },
);
expect(result.agentName).toBe("PlanOverride");
});
it("array values in plan replace (not merge) the corresponding caller value", () => {
const result = mergeProviderOptionsWithCachePlan(
{ agentName: "Bot" },
{ tags: ["a", "b"] },
{ tags: ["c"] },
);
expect(result.tags).toEqual(["c"]);
});
});
@@ -0,0 +1,178 @@
/**
* Exercises the caller-supplied promptSegments path of
* AgentRuntime.dynamicPromptExecFromState (#15742): a caller that assembles
* its own prompt (the PromptBatcher dispatcher) can pass stable/dynamic
* segment structure alongside the flat prompt. The segmentation is adopted
* only when it reproduces the rendered template byte-for-byte — the prompt
* text the model sees is provably unchanged — and is dropped otherwise.
* Runs against a bare AgentRuntime with a registered vi.fn() model handler —
* fully deterministic, no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { AgentRuntime } from "../runtime";
import { type Character, ModelType } from "../types";
import type { PromptSegment } from "../types/model";
function makeRuntime(): AgentRuntime {
const runtime = new AgentRuntime({
character: {
name: "dynamic-prompt-caller-segments-test",
bio: "test",
settings: {},
} as Character,
logLevel: "fatal",
});
// No DB adapter on this minimal runtime — stub the pure-logging call so
// useModel returns the handler output cleanly (see
// dynamic-prompt-json-mode.test.ts for the original rationale).
(runtime as unknown as { logModelCall: () => void }).logModelCall = () => {};
return runtime;
}
const HEADER =
"You are answering multiple independent structured sections in one response.\n\n";
const SECTION_BLOCK = "SECTION 1: first\n\nContext:\ncontext for first";
const PROMPT = `${HEADER}${SECTION_BLOCK}`;
describe("AgentRuntime.dynamicPromptExecFromState caller promptSegments", () => {
it("adopts caller segments when they reproduce the rendered prompt byte-for-byte", async () => {
const runtime = makeRuntime();
let seenParams:
| { prompt?: string; promptSegments?: PromptSegment[] }
| undefined;
const handler = vi.fn(
async (
_runtime,
params: { prompt?: string; promptSegments?: PromptSegment[] },
) => {
seenParams = params;
return '{"answer":"ok"}';
},
);
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
await runtime.dynamicPromptExecFromState({
params: {
prompt: PROMPT,
promptSegments: [
{ content: HEADER, stable: true },
{ content: SECTION_BLOCK, stable: false },
],
},
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 0 },
});
expect(handler).toHaveBeenCalledTimes(1);
const segments = seenParams?.promptSegments ?? [];
expect(segments.length).toBeGreaterThan(1);
// The caller's stable/dynamic boundary survives into the segments the
// model handler receives: the header is its own stable segment and the
// volatile section context is NOT part of any stable segment.
expect(segments[0]).toMatchObject({ content: HEADER, stable: true });
expect(segments[1]?.stable).toBe(false);
expect(segments[1]?.content).toContain("context for first");
for (const segment of segments) {
if (segment.stable) {
expect(segment.content).not.toContain("context for first");
}
}
// The full prompt text is byte-identical to segment concatenation.
expect(segments.map((segment) => segment.content).join("")).toBe(
seenParams?.prompt,
);
});
it("falls back to template segmentation when caller segments do not reproduce the prompt", async () => {
const runtime = makeRuntime();
let seenParams:
| { prompt?: string; promptSegments?: PromptSegment[] }
| undefined;
const handler = vi.fn(
async (
_runtime,
params: { prompt?: string; promptSegments?: PromptSegment[] },
) => {
seenParams = params;
return '{"answer":"ok"}';
},
);
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
await runtime.dynamicPromptExecFromState({
params: {
prompt: PROMPT,
// Joined content diverges from the prompt — must be rejected so
// the model still sees the true rendered text.
promptSegments: [{ content: "SOMETHING ELSE ENTIRELY", stable: true }],
},
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 0 },
});
expect(handler).toHaveBeenCalledTimes(1);
const segments = seenParams?.promptSegments ?? [];
// Fallback: the rendered template is one segment (no placeholders), so
// the header and section context stay in the SAME segment — the bogus
// caller boundary did not leak through.
const first = segments[0];
expect(first?.content).toContain(
"You are answering multiple independent structured sections",
);
expect(first?.content).toContain("context for first");
expect(
segments.some((segment) =>
segment.content.includes("SOMETHING ELSE ENTIRELY"),
),
).toBe(false);
expect(segments.map((segment) => segment.content).join("")).toBe(
seenParams?.prompt,
);
});
it("routes a caller segment ttl hint into the Anthropic cache plan (#15742)", async () => {
const runtime = makeRuntime();
let seenParams:
| {
providerOptions?: {
anthropic?: {
cacheBreakpoints?: Array<{
segmentIndex: number;
ttl: string;
cacheControl: { type: string; ttl?: string };
}>;
};
};
}
| undefined;
const handler = vi.fn(async (_runtime, params: unknown) => {
seenParams = params as typeof seenParams;
return '{"answer":"ok"}';
});
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
await runtime.dynamicPromptExecFromState({
params: {
prompt: PROMPT,
promptSegments: [
{ content: HEADER, stable: true, ttl: "long" },
{ content: SECTION_BLOCK, stable: false },
],
},
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 0 },
});
expect(handler).toHaveBeenCalledTimes(1);
const breakpoints =
seenParams?.providerOptions?.anthropic?.cacheBreakpoints ?? [];
expect(breakpoints.length).toBeGreaterThan(0);
// The caller-marked long-TTL stable segment surfaces as a 1h ephemeral
// breakpoint in the provider cache plan.
expect(breakpoints[0]).toMatchObject({
segmentIndex: 0,
ttl: "long",
cacheControl: { type: "ephemeral", ttl: "1h" },
});
});
});
@@ -0,0 +1,118 @@
/**
* Exercises AgentRuntime.dynamicPromptExecFromState: structured model calls
* request JSON-object response format, a validation failure feeds a corrective
* [REPAIR] context into the reroll, and exhausted retries return null while an
* explicit caller response format is preserved. Runs against a bare
* AgentRuntime (no DB adapter, logModelCall stubbed) with a registered vi.fn()
* model handler — fully deterministic, no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { AgentRuntime } from "../runtime";
import { type Character, ModelType } from "../types";
function makeRuntime(): AgentRuntime {
const runtime = new AgentRuntime({
character: {
name: "dynamic-prompt-json-mode-test",
bio: "test",
settings: {},
} as Character,
logLevel: "fatal",
});
// This minimal runtime has no DB adapter, so logModelCall's
// `this.adapter.createLogs` would throw and route every useModel through the
// model_error path — never reaching validation. Stub it (pure logging) so
// useModel returns the handler output cleanly.
(runtime as unknown as { logModelCall: () => void }).logModelCall = () => {};
return runtime;
}
describe("AgentRuntime.dynamicPromptExecFromState", () => {
it("requests JSON-object mode for structured model calls", async () => {
const runtime = makeRuntime();
let seenParams: unknown;
const handler = vi.fn(async (_runtime, params: unknown) => {
seenParams = params;
return '{"answer":"ok"}';
});
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
await runtime.dynamicPromptExecFromState({
params: { prompt: "Return an answer." },
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 0 },
});
expect(seenParams).toMatchObject({
responseFormat: { type: "json_object" },
});
expect(handler).toHaveBeenCalledTimes(1);
});
it("feeds a corrective repair context into the reroll after a validation failure", async () => {
const runtime = makeRuntime();
const prompts: string[] = [];
const handler = vi.fn(async (_runtime, params: { prompt: string }) => {
// Always invalid (missing the required `answer`) so every reroll's
// prompt is captured; we assert the reroll became corrective.
prompts.push(params.prompt);
return '{"wrong":"nope-value"}';
});
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
await runtime.dynamicPromptExecFromState({
params: { prompt: "Return an answer." },
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 1 },
});
// maxRetries:1 → 2 attempts. The first carries no repair context; the
// reroll feeds back the concrete failure + the (echoed) prior output, so
// it is corrective rather than a blind re-roll of the same prompt.
expect(handler).toHaveBeenCalledTimes(2);
expect(prompts).toHaveLength(2);
expect(prompts[0]).not.toContain("[REPAIR]");
expect(prompts[1]).toContain("[REPAIR]");
expect(prompts[1]).toContain("Your previous (invalid) output was:");
expect(prompts[1]).toContain("nope-value");
});
it("still returns null after exhausting retries (behavior preserved)", async () => {
const runtime = makeRuntime();
const handler = vi.fn(async () => '{"wrong":"always-invalid"}');
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
const result = await runtime.dynamicPromptExecFromState({
params: { prompt: "Return an answer." },
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 1 },
});
expect(handler).toHaveBeenCalledTimes(2);
expect(result).toBeNull();
});
it("preserves an explicit caller response format", async () => {
const runtime = makeRuntime();
let seenParams: unknown;
const handler = vi.fn(async (_runtime, params: unknown) => {
seenParams = params;
return '{"answer":"ok"}';
});
runtime.registerModel(ModelType.TEXT_LARGE, handler, "test", 100);
await runtime.dynamicPromptExecFromState({
params: {
prompt: "Return an answer.",
responseFormat: { type: "text" },
},
schema: [{ field: "answer", description: "Answer", required: true }],
options: { modelType: ModelType.TEXT_LARGE, maxRetries: 0 },
});
expect(seenParams).toMatchObject({
responseFormat: { type: "text" },
});
expect(handler).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,46 @@
/**
* Verifies formatEntities enforces prompt-hygiene caps: alias and long-metadata
* truncation plus a ceiling on the number of rendered entities. Pure
* deterministic function test.
*/
import { describe, expect, it } from "vitest";
import { formatEntities } from "../entities.ts";
import type { Entity } from "../types/index.ts";
describe("formatEntities", () => {
it("caps alias and metadata output for prompt hygiene", () => {
const names = Array.from(
{ length: 12 },
(_, index) => `alias-${index + 1}`,
);
const entity = {
id: "00000000-0000-0000-0000-000000000123",
names,
metadata: {
bio: "x".repeat(2_500),
},
} as Entity;
const rendered = formatEntities({ entities: [entity] });
expect(rendered).toContain('"alias-1" aka "alias-2"');
expect(rendered).toContain("(+4 aliases omitted)");
expect(rendered).not.toContain("alias-12");
expect(rendered).toContain("(truncated)");
expect(rendered.length).toBeLessThan(2_500);
});
it("caps the number of rendered entities", () => {
const entities = Array.from({ length: 30 }, (_, index) => ({
id: `00000000-0000-0000-0000-${String(index + 1).padStart(12, "0")}`,
names: [`entity-${String(index + 1).padStart(2, "0")}`],
})) as Entity[];
const rendered = formatEntities({ entities });
expect(rendered).toContain("entity-01");
expect(rendered).toContain("entity-10");
expect(rendered).not.toContain("entity-11");
expect(rendered).toContain("(+20 entities omitted)");
});
});
@@ -0,0 +1,108 @@
/**
* Pins the canonical env-truthiness contract (isTruthyEnvValue) and checks the
* boolean-parser wrappers (parseBooleanValue, parseBooleanText, readEnvBool)
* each keep their documented truthy/falsy token sets. Pure deterministic parser
* test.
*/
import { describe, expect, it } from "vitest";
import { isTruthyEnvValue } from "../env-utils.js";
import { parseBooleanText, parseBooleanValue } from "../utils/boolean.js";
import { readEnvBool } from "../utils/read-env.js";
const CANONICAL_TRUTHY = ["1", "true", "yes", "y", "on", "enabled"] as const;
describe("isTruthyEnvValue (canonical)", () => {
it("accepts exactly the canonical truthy tokens", () => {
for (const token of CANONICAL_TRUTHY) {
expect(isTruthyEnvValue(token)).toBe(true);
}
});
it("is case-insensitive and trims surrounding whitespace", () => {
for (const token of CANONICAL_TRUTHY) {
expect(isTruthyEnvValue(token.toUpperCase())).toBe(true);
expect(isTruthyEnvValue(` ${token} `)).toBe(true);
expect(isTruthyEnvValue(`\t${token.toUpperCase()}\n`)).toBe(true);
}
});
it("rejects non-canonical, empty, and nullish values", () => {
for (const token of [
"0",
"false",
"no",
"off",
"n",
"disabled",
"enable",
"t",
"truthy",
"",
" ",
]) {
expect(isTruthyEnvValue(token)).toBe(false);
}
expect(isTruthyEnvValue(undefined)).toBe(false);
expect(isTruthyEnvValue(null)).toBe(false);
});
});
describe("parseBooleanValue (canonical 3-valued parser)", () => {
it("uses the default truthy/falsy sets and returns undefined for unknown", () => {
for (const token of ["true", "1", "yes", "on"]) {
expect(parseBooleanValue(token)).toBe(true);
}
for (const token of ["false", "0", "no", "off"]) {
expect(parseBooleanValue(token)).toBe(false);
}
expect(parseBooleanValue("maybe")).toBeUndefined();
expect(parseBooleanValue(undefined)).toBeUndefined();
expect(parseBooleanValue("")).toBeUndefined();
expect(parseBooleanValue(true)).toBe(true);
expect(parseBooleanValue(false)).toBe(false);
});
});
describe("parseBooleanText (wrapper: text set, 2-valued, defaults false)", () => {
it("accepts its documented truthy text tokens", () => {
for (const token of ["yes", "y", "true", "t", "1", "on", "enable"]) {
expect(parseBooleanText(token)).toBe(true);
}
});
it("treats falsy and unknown text as false", () => {
for (const token of ["no", "n", "false", "f", "0", "off", "disable"]) {
expect(parseBooleanText(token)).toBe(false);
}
expect(parseBooleanText("garbage")).toBe(false);
expect(parseBooleanText(undefined)).toBe(false);
expect(parseBooleanText(null)).toBe(false);
});
});
describe("readEnvBool (wrapper: default set, 2-valued, configurable default)", () => {
it("preserves the 1/true/yes/on truthy and 0/false/no/off falsy sets", () => {
for (const v of ["1", "true", "TRUE", "yes", "on"]) {
expect(readEnvBool("ELIZA_FLAG", { env: { ELIZA_FLAG: v } })).toBe(true);
}
for (const v of ["0", "false", "no", "off"]) {
expect(readEnvBool("ELIZA_FLAG", { env: { ELIZA_FLAG: v } })).toBe(false);
}
});
it("returns defaultValue (default false) when unset or unrecognized", () => {
expect(readEnvBool("ELIZA_FLAG", { env: {} })).toBe(false);
expect(readEnvBool("ELIZA_FLAG", { env: {}, defaultValue: true })).toBe(
true,
);
expect(readEnvBool("ELIZA_FLAG", { env: { ELIZA_FLAG: "maybe" } })).toBe(
false,
);
expect(
readEnvBool("ELIZA_FLAG", {
env: { ELIZA_FLAG: "maybe" },
defaultValue: true,
}),
).toBe(true);
});
});
@@ -0,0 +1,171 @@
/**
* Covers InferenceTurnTimer and the inference-timing AsyncLocalStorage helpers:
* span roll-up by name, mark-derived timeToReply / timeToFirstToken,
* duplicate-mark anomaly detection, ALS attribution across async boundaries,
* and the emit / format / dev-payload registry. Deterministic — no live model.
*/
import { describe, expect, it } from "vitest";
import {
buildInferenceTimingDevPayload,
emitInferenceTiming,
formatInferenceTimingSummary,
getInferenceTimer,
INFERENCE_MARKS,
InferenceTurnTimer,
markInference,
nextInferenceTurnId,
recordInferenceSpan,
runWithInferenceTiming,
timeInferenceSpan,
} from "../inference-timing";
const tick = () => new Promise((r) => setTimeout(r, 2));
describe("InferenceTurnTimer", () => {
it("rolls up span contributions by name and counts repeats", () => {
const timer = new InferenceTurnTimer({ turnId: "t1", label: "test" });
timer.recordSpan("composeState", 40);
timer.recordSpan("provider:RECENT_MESSAGES", 30);
timer.recordSpan("model:RESPONSE_HANDLER", 100);
timer.recordSpan("model:RESPONSE_HANDLER", 50);
const s = timer.summary();
expect(s.byName.composeState).toEqual({ totalMs: 40, count: 1 });
expect(s.byName["model:RESPONSE_HANDLER"]).toEqual({
totalMs: 150,
count: 2,
});
});
it("derives timeToReply / timeToFirstToken from marks, null when missing", () => {
const timer = new InferenceTurnTimer({ turnId: "t2", label: "test" });
const start = timer.t0EpochMs;
timer.mark(INFERENCE_MARKS.firstToken, start + 10);
timer.mark(INFERENCE_MARKS.replyDelivered, start + 25);
const s = timer.summary();
expect(s.timeToFirstTokenMs).toBe(10);
expect(s.timeToReplyMs).toBe(25);
const noMarks = new InferenceTurnTimer({
turnId: "t3",
label: "test",
}).summary();
expect(noMarks.timeToReplyMs).toBeNull();
expect(noMarks.timeToFirstTokenMs).toBeNull();
});
it("totalMs is null until close()", () => {
const timer = new InferenceTurnTimer({ turnId: "t4", label: "test" });
expect(timer.summary().totalMs).toBeNull();
const closed = timer.close();
expect(closed.totalMs).not.toBeNull();
expect(closed.totalMs).toBeGreaterThanOrEqual(0);
});
it("flags a duplicate mark as an anomaly and keeps the first", () => {
const timer = new InferenceTurnTimer({ turnId: "t5", label: "test" });
const start = timer.t0EpochMs;
timer.mark("x", start + 5);
timer.mark("x", start + 99);
const s = timer.summary();
expect(s.marks.find((m) => m.name === "x")?.tMs).toBe(5);
expect(s.anomalies.some((a) => a.includes("duplicate"))).toBe(true);
});
it("openSpan closer is idempotent", async () => {
const timer = new InferenceTurnTimer({ turnId: "t6", label: "test" });
const close = timer.openSpan("work");
await tick();
close();
close(); // second call must be ignored
expect(timer.summary().byName.work.count).toBe(1);
});
it("setModelProvider keeps the first non-empty writer", () => {
const timer = new InferenceTurnTimer({ turnId: "t7", label: "test" });
timer.setModelProvider(undefined);
timer.setModelProvider("elizaOSCloud");
timer.setModelProvider("other");
expect(timer.summary().modelProvider).toBe("elizaOSCloud");
});
});
describe("inference-timing ALS helpers", () => {
it("are no-ops with no active timer (and still run the fn)", async () => {
expect(getInferenceTimer()).toBeUndefined();
markInference("orphan");
recordInferenceSpan("orphan", 5);
const v = await timeInferenceSpan("orphan", async () => 42);
expect(v).toBe(42);
});
it("attribute spans/marks to the active timer across async work", async () => {
const timer = new InferenceTurnTimer({ turnId: "als", label: "test" });
const out = await runWithInferenceTiming(timer, async () => {
expect(getInferenceTimer()).toBe(timer);
await timeInferenceSpan("composeState", async () => {
await tick();
});
// Nested async boundary still sees the timer (AsyncLocalStorage).
await Promise.resolve().then(() => {
recordInferenceSpan("model:TEXT_SMALL", 12, { provider: "x" });
markInference(INFERENCE_MARKS.replyDelivered);
});
return "done";
});
expect(out).toBe("done");
const s = timer.summary();
expect(s.byName.composeState?.count).toBe(1);
expect(s.byName["model:TEXT_SMALL"]?.totalMs).toBe(12);
expect(s.timeToReplyMs).not.toBeNull();
});
it("restores the prior timer after the scope exits", async () => {
const outer = new InferenceTurnTimer({ turnId: "outer", label: "o" });
await runWithInferenceTiming(outer, async () => {
const inner = new InferenceTurnTimer({ turnId: "inner", label: "i" });
await runWithInferenceTiming(inner, async () => {
expect(getInferenceTimer()).toBe(inner);
});
expect(getInferenceTimer()).toBe(outer);
});
expect(getInferenceTimer()).toBeUndefined();
});
});
describe("emit + format + registry", () => {
it("formats a compact breakdown line ranked by contribution", () => {
const timer = new InferenceTurnTimer({
turnId: "fmt",
label: "message-turn",
});
timer.setModelProvider("elizaOSCloud");
timer.recordSpan("composeState", 20);
timer.recordSpan("model:RESPONSE_HANDLER", 200);
timer.mark(INFERENCE_MARKS.replyDelivered, timer.t0EpochMs + 230);
const line = formatInferenceTimingSummary(timer.close());
expect(line).toContain("[InferenceTiming] message-turn");
expect(line).toContain("provider=elizaOSCloud");
expect(line).toContain("model:RESPONSE_HANDLER=200ms");
// Biggest contributor is ordered before the smaller one.
expect(line.indexOf("model:RESPONSE_HANDLER")).toBeLessThan(
line.indexOf("composeState"),
);
});
it("emitInferenceTiming records the turn into the dev payload", () => {
const turnId = nextInferenceTurnId();
const timer = new InferenceTurnTimer({ turnId, label: "message-turn" });
timer.recordSpan("model:RESPONSE_HANDLER", 77);
emitInferenceTiming(timer);
const payload = buildInferenceTimingDevPayload();
expect(payload.turns.some((t) => t.turnId === turnId)).toBe(true);
expect(
payload.spanHistograms["model:RESPONSE_HANDLER"]?.count,
).toBeGreaterThan(0);
});
it("emitInferenceTiming is no-op-safe for an undefined timer", () => {
expect(emitInferenceTiming(undefined)).toBeNull();
});
});
@@ -0,0 +1,48 @@
/**
* Verifies sanitizeReplyTextAfterMediaDelivery strips a delivered media URL (and
* its echo) from the reply while leaving non-echo prose, code indentation, and
* newlines untouched. Pure deterministic function test.
*/
import { describe, expect, it } from "vitest";
import { sanitizeReplyTextAfterMediaDelivery } from "../services/message.ts";
describe("sanitizeReplyTextAfterMediaDelivery", () => {
const url = "http://192.168.255.164:8080/v1/videos/50a2f4c2/content";
it("strips known media URLs and zerollama content paths", () => {
expect(
sanitizeReplyTextAfterMediaDelivery(`Here it is: <${url}>`, [url]),
).toBe("");
expect(
sanitizeReplyTextAfterMediaDelivery(`Done. Video's up: ${url}`, [url]),
).toBe("");
});
it("preserves meaningful text that is not a URL echo", () => {
expect(
sanitizeReplyTextAfterMediaDelivery(
"Wan drifted from your prompt — want a tighter retry?",
[url],
),
).toBe("Wan drifted from your prompt — want a tighter retry?");
});
it("returns a media-free reply completely untouched (newlines + indentation)", () => {
const code =
"result = []\n for n in numbers:\n if n >= 0:\n result.append(n + 3)\n return result";
expect(sanitizeReplyTextAfterMediaDelivery(code, [])).toBe(code);
const prose =
"First paragraph.\n\nSecond paragraph:\n- item one\n- item two";
expect(sanitizeReplyTextAfterMediaDelivery(prose, [])).toBe(prose);
});
it("keeps newlines away from the URL when stripping a delivered URL", () => {
const sanitized = sanitizeReplyTextAfterMediaDelivery(
`Your video is ready ${url}\nIt has:\n- scene one\n- scene two`,
[url],
);
expect(sanitized).not.toContain(url);
expect(sanitized).toContain("It has:\n- scene one\n- scene two");
});
});
@@ -0,0 +1,142 @@
/**
* Exercises the message service's planner-action de-duplication
* (stripReplyWhenActionOwnsTurn) and sub-planner result collapse
* (subPlannerResultToPlannerToolResult): REPLY/alias dedupe, continueChain
* propagation from a terminal sub-action, and multi-step aggregation into the
* umbrella result. Runs against a stub runtime (actions + logger) — fully
* deterministic.
*/
import { describe, expect, it, vi } from "vitest";
import {
stripReplyWhenActionOwnsTurn,
subPlannerResultToPlannerToolResult,
} from "../services/message.ts";
import type { IAgentRuntime } from "../types/runtime";
type SubResult = Parameters<typeof subPlannerResultToPlannerToolResult>[0];
function subResult(
lastStepResult: Record<string, unknown> | undefined,
finalMessage?: string,
): SubResult {
return {
status: "finished",
finalMessage,
trajectory: {
steps: lastStepResult ? [{ iteration: 1, result: lastStepResult }] : [],
},
} as unknown as SubResult;
}
function runtime(
actions: Array<{ name: string; similes?: string[] }> = [],
): Pick<IAgentRuntime, "actions" | "logger"> {
return {
actions,
logger: {
info: vi.fn(),
debug: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
},
} as Pick<IAgentRuntime, "actions" | "logger">;
}
describe("stripReplyWhenActionOwnsTurn", () => {
it("collapses duplicate REPLY planner actions before execution", () => {
expect(stripReplyWhenActionOwnsTurn(runtime(), ["REPLY", "REPLY"])).toEqual(
["REPLY"],
);
});
it("dedupes aliases against the registered canonical action name", () => {
expect(
stripReplyWhenActionOwnsTurn(
runtime([{ name: "REPLY", similes: ["RESPOND"] }]),
["RESPOND", "REPLY"],
),
).toEqual(["RESPOND"]);
});
});
describe("subPlannerResultToPlannerToolResult", () => {
it("propagates continueChain:false from the terminal sub-action", () => {
// A fire-and-forget sub-action (e.g. TASKS_SPAWN_AGENT) returns
// continueChain:false. Without propagating it through the umbrella
// result, the parent planner loop evaluates CONTINUE and re-runs the
// umbrella, producing duplicate spawns on a single user turn.
const result = subPlannerResultToPlannerToolResult(
subResult(
{ success: true, text: "On it.", continueChain: false },
"On it.",
),
);
expect(result.continueChain).toBe(false);
expect(result.success).toBe(true);
});
it("leaves continueChain undefined when the sub-action did not set it", () => {
const result = subPlannerResultToPlannerToolResult(
subResult({ success: true, text: "done" }, "done"),
);
expect(result.continueChain).toBeUndefined();
});
it("handles an empty sub-trajectory without throwing", () => {
const result = subPlannerResultToPlannerToolResult(subResult(undefined));
expect(result.continueChain).toBeUndefined();
expect(result.success).toBe(true);
});
// Regression for elizaOS/eliza#8007: a multi-step sub-planner collapse must
// surface EVERY executed sub-step to the parent loop, not only the terminal
// one, so the outer planner can see which ops already succeeded and advance
// instead of re-running the umbrella action from the first step.
it("aggregates all sub-steps into the diagnostic text and data", () => {
const multiStep = {
status: "finished",
finalMessage: "Opened a PR for hello-world.",
trajectory: {
steps: [
{
iteration: 1,
toolCall: { name: "provision_workspace" },
result: { success: true, text: "workspace ws-1 ready" },
},
{
iteration: 2,
toolCall: { name: "spawn_agent" },
result: { success: true, text: "spawned agent a-1" },
},
{
iteration: 3,
toolCall: { name: "submit_workspace" },
result: { success: false, error: "no diff to submit" },
},
],
},
} as unknown as SubResult;
const result = subPlannerResultToPlannerToolResult(multiStep);
// The diagnostic text (what the parent planner reasons over) carries the
// full progression, not just the terminal step.
expect(result.text).toContain("provision_workspace");
expect(result.text).toContain("spawn_agent");
expect(result.text).toContain("submit_workspace");
expect(result.text).toContain("OK");
expect(result.text).toContain("FAIL");
// The user-facing text stays the synthesized final message.
expect(result.userFacingText).toBe("Opened a PR for hello-world.");
// Structured sub-step data lets downstream action context see which ops
// already completed.
expect(result.data?.completedSubActions).toEqual([
"provision_workspace",
"spawn_agent",
]);
expect(Array.isArray(result.data?.subSteps)).toBe(true);
expect((result.data?.subSteps as unknown[]).length).toBe(3);
});
});
@@ -0,0 +1,470 @@
/**
* Behavioral matrix for the answer-clobber rescue in runV5MessageRuntimeStage1:
* a response-handler evaluator that promotes a simple turn to planning while
* overwriting a complete stage-0 answer with a progress ack must not end the
* turn answerless — and the rescue must never overreach (no duplicate of the
* early reply, no override of planner-produced final text, no fabrication on
* genuinely progress-only turns, no double delivery of an action's own echo).
* Drives the real message→planner→evaluator pipeline with a queued
* canned-response model mock and real clobbering evaluators; no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import type { ResponseHandlerEvaluator } from "../runtime/response-handler-evaluators";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import {
normalizeVisibleTextForDuplicateCheck,
runV5MessageRuntimeStage1,
wrapSingleTurnVisibleCallback,
} from "../services/message";
import type { Action, HandlerCallback } from "../types/components";
import type { Memory } from "../types/memory";
import { ModelType } from "../types/model";
import type { UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
const AGENT_ID = "00000000-0000-0000-0000-000000000003" as UUID;
const RESPONSE_ID = "00000000-0000-0000-0000-000000000005" as UUID;
const SUBSTANTIVE_ANSWER =
"The top 3 contributors to elizaOS/eliza are lalalune, shakkernerd, and odilitime.";
const PROGRESS_ACK = "On it, working on that now.";
function makeMessage(): Memory {
return {
id: "00000000-0000-0000-0000-000000000001" as UUID,
entityId: "00000000-0000-0000-0000-000000000002" as UUID,
agentId: AGENT_ID,
roomId: "00000000-0000-0000-0000-000000000004" as UUID,
content: {
text: "who are the top 3 contributors to the eliza repo",
source: "test",
},
createdAt: 1,
};
}
function makeState(): State {
return {
values: { availableContexts: "general, web" },
data: {},
text: "Recent conversation summary",
};
}
function stage1Response(fields: {
contexts?: string[];
replyText?: string;
extra?: Record<string, unknown>;
}) {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: "HANDLE_RESPONSE",
arguments: {
shouldRespond: "RESPOND",
thought: "",
contexts: fields.contexts ?? [],
intents: [],
candidateActionNames: [],
replyText: fields.replyText ?? "",
facts: [],
relationships: [],
addressedTo: [],
...(fields.extra ?? {}),
},
},
],
};
}
// Reproduces the live promotion-that-clobbers: force the turn into planning
// and overwrite the substantive stage-0 answer with a bare progress ack.
function clobberEvaluator(
name: string,
reply: string,
): ResponseHandlerEvaluator {
return {
name,
priority: 100,
shouldRun: () => true,
evaluate: () => ({ reply, requiresTool: true }),
};
}
// Promotion WITHOUT a clobber: escalate to planning but leave the stage-0
// reply untouched.
const PROMOTE_ONLY_EVALUATOR: ResponseHandlerEvaluator = {
name: "test-promote-only",
priority: 100,
shouldRun: () => true,
evaluate: () => ({ requiresTool: true }),
};
interface CannedResponse {
expectModelType?: string;
body: unknown;
}
function makeRuntime(opts: {
responses: CannedResponse[];
evaluators: ResponseHandlerEvaluator[];
actions?: Action[];
}): IAgentRuntime {
const queue = [...opts.responses];
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return {
agentId: AGENT_ID,
character: {
name: "Test Agent",
system: "You are concise.",
bio: "I help.",
},
actions: opts.actions ?? [],
providers: [],
composeState: vi.fn(async () => makeState()),
runActionsByMode: vi.fn(async () => undefined),
emitEvent: vi.fn(async () => undefined),
useModel: vi.fn(async (modelType: unknown) => {
const next = queue.shift();
if (!next) throw new Error("Unexpected useModel call; queue empty");
if (next.expectModelType && String(modelType) !== next.expectModelType) {
throw new Error(
`Expected ${next.expectModelType} but received ${String(modelType)}`,
);
}
return next.body;
}),
// The per-callback character-voice rewrite spends a TEXT_SMALL call and
// restyles delivered text, which would desync the strict canned-response
// queue — the same opt-out the scenario runner uses.
getSetting: vi.fn((key: string) =>
key === "ACTION_CALLBACK_VOICE_REWRITE" ? "false" : undefined,
),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
responseHandlerEvaluators: opts.evaluators,
} as unknown as IAgentRuntime;
}
const ANSWERLESS_PLANNER: CannedResponse = {
expectModelType: String(ModelType.ACTION_PLANNER),
body: { text: "", toolCalls: [] },
};
const ANSWERLESS_FINISH: CannedResponse = {
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: "Nothing further.",
messageToUser: "",
}),
};
async function runTurn(opts: {
runtime: IAgentRuntime;
callback?: HandlerCallback;
}): Promise<{
finalText: string | undefined;
earlyReplies: string[];
kind: string;
}> {
const earlyReplies: string[] = [];
const message = makeMessage();
// Mirror DefaultMessageService's wiring: action deliveries are recorded into
// deliveredVisibleTexts through the instrumented callback, which is what the
// action-echo suppression reads.
const deliveredVisibleTexts = new Set<string>();
const instrumentedCallback = opts.callback
? wrapSingleTurnVisibleCallback(
opts.runtime,
message,
opts.callback,
(text) =>
deliveredVisibleTexts.add(
normalizeVisibleTextForDuplicateCheck(text),
),
)
: undefined;
const result = await runV5MessageRuntimeStage1({
runtime: opts.runtime,
message,
state: makeState(),
responseId: RESPONSE_ID,
...(instrumentedCallback ? { callback: instrumentedCallback } : {}),
deliveredVisibleTexts,
onResponseHandlerEarlyReply: async ({ text }) => {
earlyReplies.push(text);
},
});
const finalText =
result.kind === "planned_reply"
? result.result.responseContent?.text
: undefined;
return { finalText, earlyReplies, kind: result.kind };
}
describe("answer-clobber rescue", () => {
it("delivers the preserved stage-0 answer when a promotion clobbers it with a progress ack", async () => {
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: SUBSTANTIVE_ANSWER,
}),
},
ANSWERLESS_PLANNER,
ANSWERLESS_FINISH,
],
evaluators: [clobberEvaluator("test-clobber", PROGRESS_ACK)],
});
const { finalText, earlyReplies } = await runTurn({ runtime });
// The ack was the early reply the user saw first; the preserved
// substantive answer is the final delivered text.
expect(earlyReplies).toContain(PROGRESS_ACK);
expect(finalText).toBe(SUBSTANTIVE_ANSWER);
});
it("survives multiple promotions: the pre-patch answer is preserved across stacked evaluator patches", async () => {
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: SUBSTANTIVE_ANSWER,
}),
},
ANSWERLESS_PLANNER,
ANSWERLESS_FINISH,
],
evaluators: [
clobberEvaluator("test-clobber-one", "Working on it."),
clobberEvaluator("test-clobber-two", PROGRESS_ACK),
],
});
const { finalText } = await runTurn({ runtime });
expect(finalText).toBe(SUBSTANTIVE_ANSWER);
});
it("does not duplicate the early reply when the promotion kept the substantive answer", async () => {
// Promotion WITHOUT a clobber: the substantive answer itself became the
// early reply. An answerless planner finish must not deliver it again.
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: SUBSTANTIVE_ANSWER,
}),
},
ANSWERLESS_PLANNER,
ANSWERLESS_FINISH,
],
evaluators: [PROMOTE_ONLY_EVALUATOR],
});
const { finalText, earlyReplies } = await runTurn({ runtime });
expect(earlyReplies).toContain(SUBSTANTIVE_ANSWER);
// No second bubble with the same text.
expect(finalText ?? "").not.toBe(SUBSTANTIVE_ANSWER);
});
it("lets planner-produced final text win over the preserved stage-0 answer", async () => {
const plannerAnswer = "Fresh planner answer with newer data.";
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: SUBSTANTIVE_ANSWER,
}),
},
{
expectModelType: String(ModelType.ACTION_PLANNER),
body: {
text: "",
toolCalls: [
{
id: "reply-1",
name: "REPLY",
arguments: { text: plannerAnswer },
},
],
},
},
],
evaluators: [clobberEvaluator("test-clobber", PROGRESS_ACK)],
});
const { finalText } = await runTurn({ runtime });
expect(finalText).toBe(plannerAnswer);
expect(finalText).not.toBe(SUBSTANTIVE_ANSWER);
});
it("rescues nothing on a genuinely progress-only stage-0 turn", async () => {
// Stage-0 itself produced only an ack; there is no answer to preserve, so
// an answerless finish stays answerless (no fabricated content).
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: PROGRESS_ACK,
}),
},
ANSWERLESS_PLANNER,
ANSWERLESS_FINISH,
],
evaluators: [PROMOTE_ONLY_EVALUATOR],
});
const { finalText } = await runTurn({ runtime });
expect(finalText ?? "").not.toContain("contributors");
expect(finalText ?? "").not.toBe(PROGRESS_ACK);
});
it("does not double-deliver when an action already delivered the preserved text", async () => {
const delivered: string[] = [];
const callback: HandlerCallback = async (content) => {
if (typeof content.text === "string" && content.text.length > 0) {
delivered.push(content.text);
}
return [];
};
const echoAction: Action = {
name: "ANSWER_LOOKUP",
description: "returns the contributors answer via its own callback",
similes: [],
examples: [],
parameters: [],
validate: async () => true,
handler: async (_rt, _msg, _state, _opts, cb) => {
await cb?.({ text: SUBSTANTIVE_ANSWER });
return { success: true, text: SUBSTANTIVE_ANSWER };
},
} as unknown as Action;
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: SUBSTANTIVE_ANSWER,
}),
},
{
expectModelType: String(ModelType.ACTION_PLANNER),
body: {
text: "",
toolCalls: [{ id: "call-1", name: "ANSWER_LOOKUP", args: {} }],
},
},
// The evaluator echoes the text the action already delivered — the
// classic redundant-second-bubble shape the echo suppression exists
// for. The preserved-answer fallback must not defeat it.
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: "The lookup answered it.",
messageToUser: SUBSTANTIVE_ANSWER,
}),
},
{
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: "The lookup answered it.",
messageToUser: SUBSTANTIVE_ANSWER,
}),
},
],
evaluators: [clobberEvaluator("test-clobber", PROGRESS_ACK)],
actions: [echoAction],
});
const { finalText } = await runTurn({ runtime, callback });
// The action's own delivery is the single copy of the answer; neither the
// evaluator echo nor the preserved-answer fallback adds a second bubble.
const copies = delivered.filter((t) => t === SUBSTANTIVE_ANSWER).length;
expect(copies).toBe(1);
expect(finalText ?? "").not.toBe(SUBSTANTIVE_ANSWER);
});
it("surfaces the preserved answer when the required-tool miss budget exhausts", async () => {
// The clobbered promotion also names a required tool. A planner that
// never calls it exhausts the miss budget (3), and the loop's captured
// answer for that exhaustion must be the preserved substantive stage-0
// reply — not the progress ack the promotion wrote over it.
const lookupAction: Action = {
name: "ANSWER_LOOKUP",
description: "looks up the contributors answer",
similes: [],
examples: [],
parameters: [],
validate: async () => true,
handler: async () => ({ success: true, text: "unused" }),
} as unknown as Action;
const missPlanner: CannedResponse = {
expectModelType: String(ModelType.ACTION_PLANNER),
body: { text: "", toolCalls: [] },
};
const runtime = makeRuntime({
responses: [
{
expectModelType: String(ModelType.RESPONSE_HANDLER),
body: stage1Response({
contexts: ["web"],
replyText: SUBSTANTIVE_ANSWER,
extra: { candidateActionNames: ["ANSWER_LOOKUP"] },
}),
},
// Four consecutive tool-less planner turns: misses 1-3 burn the
// budget, the fourth exceeds it and triggers the captured-answer
// finish.
missPlanner,
missPlanner,
missPlanner,
missPlanner,
],
evaluators: [clobberEvaluator("test-clobber", PROGRESS_ACK)],
actions: [lookupAction],
});
const { finalText, earlyReplies } = await runTurn({ runtime });
expect(earlyReplies).toContain(PROGRESS_ACK);
expect(finalText).toBe(SUBSTANTIVE_ANSWER);
});
});
@@ -0,0 +1,57 @@
/**
* Source-text contract test: reads services/message.ts and asserts benchmark-mode
* detection stays centralized in hasInboundBenchmarkContext /
* isBenchmarkForcingToolCall (no inline benchmark-flag inspection at call sites),
* that benchmark context forces CONTEXT_BENCH into the provider list, and that
* the planner requires a tool call only when both the env opt-in and an inbound
* benchmark signal are present. Static assertions over the file — no runtime.
*/
import { readFile } from "node:fs/promises";
import path from "node:path";
import { describe, expect, it } from "vitest";
const MESSAGE_SOURCE = path.resolve(
import.meta.dirname,
"../services/message.ts",
);
describe("message service benchmark integration contracts", () => {
it("centralizes benchmark mode detection through hasInboundBenchmarkContext", async () => {
const source = await readFile(MESSAGE_SOURCE, "utf8");
expect(source).toContain(
"function hasInboundBenchmarkContext(message: Memory)",
);
// Inline benchmark-flag inspection at call sites is forbidden — go through
// the helper.
expect(source).not.toContain("metadata?.benchmarkContext;\n\t\tconst");
});
it("forces CONTEXT_BENCH into the provider list when benchmark context is present", async () => {
const source = await readFile(MESSAGE_SOURCE, "utf8");
expect(source).toContain("hasInboundBenchmarkContext(message)");
expect(source).toContain('"CONTEXT_BENCH"');
});
it("forces the planner to require a tool call only when both the env opt-in and an inbound benchmark signal are present", async () => {
const source = await readFile(MESSAGE_SOURCE, "utf8");
// The helper exists and is the single place this decision is made.
expect(source).toContain(
"function isBenchmarkForcingToolCall(message: Memory)",
);
// Env-var opt-in is required (default behavior unchanged for chat).
expect(source).toContain("ELIZA_BENCH_FORCE_TOOL_CALL");
// Detection must also require an inbound benchmark signal so a
// co-resident chat process is unaffected even if the env var leaks.
expect(source).toContain('content.source === "benchmark"');
expect(source).toContain("contentMetadata.benchmark");
// The planner gate consults the helper alongside Stage 1's requiresTool.
expect(source).toContain("isBenchmarkForcingToolCall(args.message)");
// requiresTool is still consulted (now via the named-action guard), and the
// benchmark signal is OR'd in so a benchmark turn forces a tool regardless.
expect(source).toContain("messageHandler.plan.requiresTool === true");
expect(source).toContain("|| benchmarkForcingToolCall");
});
});
@@ -0,0 +1,54 @@
/**
* Exercises the message service's structured failure-reply classifier for model
* fallback cascades. Deterministic: the runtime only exposes a queued useModel
* stub, so no provider, database, or live model is involved.
*/
import { describe, expect, it, vi } from "vitest";
import { DefaultMessageService } from "../services/message";
import type { IAgentRuntime } from "../types/runtime";
const logger = {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
};
function makeRuntimeThrowing(errors: unknown[]): IAgentRuntime {
const queue = [...errors];
return {
useModel: vi.fn(async () => {
const error = queue.shift();
if (!error) throw new Error("Unexpected useModel call");
throw error;
}),
logger,
} as unknown as IAgentRuntime;
}
function creditError(): Error & { statusCode: number } {
return Object.assign(new Error("insufficient_credits"), { statusCode: 402 });
}
describe("DefaultMessageService structured failure replies", () => {
it("preserves credit exhaustion when later fallback model slots fail generically", async () => {
const service = new DefaultMessageService() as unknown as {
generateFailureReplyText(
runtime: IAgentRuntime,
prompt: string,
stage: string,
): Promise<{ kind: string }>;
};
const runtime = makeRuntimeThrowing([
creditError(),
new Error("TEXT_LARGE fallback failed"),
new Error("TEXT_SMALL fallback failed"),
new Error("TEXT_NANO fallback failed"),
]);
await expect(
service.generateFailureReplyText(runtime, "recent messages", "test"),
).resolves.toEqual({ kind: "creditsExhausted" });
});
});
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,76 @@
/**
* Coverage for `renderMessageHandlerStablePrefix` — the cacheable system +
* tool-schema segment shared across a room's turns. Runs against a hand-built
* `vi`-mocked runtime with a stubbed `composeState`; no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { renderMessageHandlerStablePrefix } from "../services/message";
import type { UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
const ROOM_ID = "00000000-0000-0000-0000-0000000000aa" as UUID;
const AGENT_ID = "00000000-0000-0000-0000-0000000000bb" as UUID;
function makeState(): State {
return {
values: { availableContexts: "" },
data: {},
text: "Recent conversation summary",
};
}
function makeRuntime(): IAgentRuntime {
return {
agentId: AGENT_ID,
character: {
name: "Test Agent",
system: "You are concise and helpful.",
bio: "I help with calendars.",
},
actions: [],
providers: [],
composeState: vi.fn(async () => makeState()),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as unknown as IAgentRuntime;
}
describe("renderMessageHandlerStablePrefix", () => {
it("renders the system + tool-schema stable prefix for a room", async () => {
const runtime = makeRuntime();
const prefix = await renderMessageHandlerStablePrefix(runtime, ROOM_ID);
expect(typeof prefix).toBe("string");
expect(prefix.length).toBeGreaterThan(0);
// The Stage-1 instructions block (the "message_handler_stage:" segment)
// is always part of the stable prefix.
expect(prefix).toContain("message_handler_stage:");
// The canonical system prompt carries the character's system text.
expect(prefix).toContain("concise");
// The unstable tail (current user turn) must NOT leak into the prefix —
// there is no user message in a pre-warm render.
expect(prefix.toLowerCase()).not.toContain("voice-prewarm");
});
it("is deterministic across repeated renders for the same room", async () => {
const runtime = makeRuntime();
const a = await renderMessageHandlerStablePrefix(runtime, ROOM_ID);
const b = await renderMessageHandlerStablePrefix(runtime, ROOM_ID);
expect(a).toBe(b);
});
it("calls composeState once per render (so providers are sampled)", async () => {
const runtime = makeRuntime();
await renderMessageHandlerStablePrefix(runtime, ROOM_ID);
expect(runtime.composeState).toHaveBeenCalledTimes(1);
const providerNames = (
runtime.composeState as { mock: { calls: unknown[][] } }
).mock.calls[0]?.[1] as string[];
expect(providerNames).toContain("RUNTIME_MODEL_CONTEXT");
});
});
@@ -0,0 +1,279 @@
/**
* Coverage for the Stage 1 available-contexts catalog: `formatAvailableContextsForPrompt`
* rendering (compact and role-gated) and the role-scoped context list injected
* into the `runV5MessageRuntimeStage1` system prompt. Deterministic `vi`-mocked
* runtime with a canned tool-call response; no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { HANDLE_RESPONSE_TOOL_NAME } from "../actions/to-tool";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import { ContextRegistry } from "../runtime/context-registry";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import {
formatAvailableContextsForPrompt,
runV5MessageRuntimeStage1,
} from "../services/message";
import type { ContextDefinition } from "../types/contexts";
import type { Memory } from "../types/memory";
import type { UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
function stage1Response(fields: {
shouldRespond?: "RESPOND" | "IGNORE" | "STOP";
thought?: string;
contexts?: string[];
intents?: string[];
candidateActionNames?: string[];
replyText?: string;
facts?: string[];
relationships?: unknown[];
addressedTo?: string[];
extra?: Record<string, unknown>;
}): {
text: string;
toolCalls: Array<{
id: string;
name: string;
arguments: Record<string, unknown>;
}>;
} {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: HANDLE_RESPONSE_TOOL_NAME,
arguments: {
shouldRespond: fields.shouldRespond ?? "RESPOND",
thought: fields.thought ?? "",
contexts: fields.contexts ?? [],
intents: fields.intents ?? [],
candidateActionNames: fields.candidateActionNames ?? [],
replyText: fields.replyText ?? "",
facts: fields.facts ?? [],
relationships: fields.relationships ?? [],
addressedTo: fields.addressedTo ?? [],
...(fields.extra ?? {}),
},
},
],
};
}
function useModelCalls(runtime: IAgentRuntime): unknown[][] {
return (runtime.useModel as { mock: { calls: unknown[][] } }).mock.calls;
}
function makeMessage(): Memory {
return {
id: "00000000-0000-0000-0000-000000000001" as UUID,
entityId: "00000000-0000-0000-0000-000000000002" as UUID,
agentId: "00000000-0000-0000-0000-000000000003" as UUID,
roomId: "00000000-0000-0000-0000-000000000004" as UUID,
content: {
text: "Hello.",
source: "test",
},
createdAt: 1,
};
}
function makeState(): State {
return {
values: {
availableContexts: "general, calendar",
},
data: {},
text: "Recent conversation summary",
};
}
function createResponseHandlerFieldRegistry(): ResponseHandlerFieldRegistry {
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return responseHandlerFieldRegistry;
}
function makeRuntimeWithContexts(
contexts: readonly ContextDefinition[],
stage1ResponseBody: unknown,
): IAgentRuntime {
const registry = new ContextRegistry(contexts);
const responseHandlerFieldRegistry = createResponseHandlerFieldRegistry();
return {
agentId: "00000000-0000-0000-0000-000000000003" as UUID,
character: { name: "Test Agent", system: "You are concise." },
actions: [],
providers: [],
contexts: registry,
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
composeState: vi.fn(async () => makeState()),
runActionsByMode: vi.fn(async () => undefined),
emitEvent: vi.fn(async () => undefined),
useModel: vi.fn(async () => stage1ResponseBody),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime;
}
const FIXTURE_CONTEXTS: readonly ContextDefinition[] = [
{
id: "general",
label: "General",
description: "Normal conversation.",
},
{
id: "calendar",
label: "Calendar",
description: "Manage calendar events.",
roleGate: { minRole: "ADMIN" },
},
{
id: "wallet",
label: "Wallet",
description: "Crypto wallet ops.",
roleGate: { minRole: "OWNER" },
},
{
id: "memory",
label: "Memory",
description: "Long-term agent memory.",
roleGate: { minRole: "USER" },
},
];
describe("formatAvailableContextsForPrompt", () => {
it("renders id, metadata, and description per line", () => {
const block = formatAvailableContextsForPrompt(FIXTURE_CONTEXTS);
expect(block).toContain("- general [label=General]: Normal conversation.");
expect(block).toContain(
"- calendar [label=Calendar; role>=ADMIN]: Manage calendar events.",
);
expect(block).toContain(
"- memory [label=Memory; role>=USER]: Long-term agent memory.",
);
});
it("falls back to a placeholder when no contexts are registered", () => {
expect(formatAvailableContextsForPrompt([])).toBe(
"(no contexts registered)",
);
});
it("compact mode renders descriptionCompressed and never the full description", () => {
const contexts: readonly ContextDefinition[] = [
{
id: "general",
label: "General",
description: "Normal conversation.",
},
{
id: "tasks",
label: "Tasks",
description: "A very long routing description that must not render.",
descriptionCompressed: "reminders/habits/todos",
},
];
const block = formatAvailableContextsForPrompt(contexts, {
compact: true,
});
// Compressed hint when present; bare id line when absent.
expect(block).toContain("- tasks [label=Tasks]: reminders/habits/todos");
expect(block).toContain("- general [label=General]");
expect(block).not.toContain("Normal conversation.");
expect(block).not.toContain("must not render");
});
});
describe("Stage 1 prompt — available contexts catalog", () => {
it("includes USER-accessible contexts and excludes OWNER-only contexts for a USER sender", async () => {
const runtime = makeRuntimeWithContexts(
FIXTURE_CONTEXTS,
stage1Response({
contexts: [],
thought: "Direct answer.",
replyText: "Hello.",
}),
);
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage(),
state: makeState(),
responseId: "00000000-0000-0000-0000-000000000005" as UUID,
});
expect(runtime.useModel).toHaveBeenCalledTimes(1);
const firstCall = useModelCalls(runtime)[0];
const params = firstCall?.[1] as
| { messages?: Array<{ role?: string; content?: string }> }
| undefined;
const systemContent = params?.messages?.[0]?.content ?? "";
expect(systemContent).toContain("available_contexts:");
// `general` (no gate) and `memory` (USER) are visible to USER role.
expect(systemContent).toContain("- general ");
expect(systemContent).toContain("- memory ");
// `wallet` (OWNER-only) and `calendar` (ADMIN-only) must NOT appear.
expect(systemContent).not.toMatch(/^- wallet\b/m);
expect(systemContent).not.toMatch(/^- calendar\b/m);
});
it("falls back to the placeholder line when no context registry is attached", async () => {
const responseHandlerFieldRegistry = createResponseHandlerFieldRegistry();
const runtime = {
agentId: "00000000-0000-0000-0000-000000000003" as UUID,
character: { name: "Test Agent", system: "You are concise." },
actions: [],
providers: [],
contexts: undefined,
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
composeState: vi.fn(async () => makeState()),
runActionsByMode: vi.fn(async () => undefined),
emitEvent: vi.fn(async () => undefined),
useModel: vi.fn(async () =>
stage1Response({
contexts: [],
thought: "Direct answer.",
replyText: "Hello.",
}),
),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime;
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage(),
state: makeState(),
responseId: "00000000-0000-0000-0000-000000000005" as UUID,
});
const firstCall = useModelCalls(runtime)[0];
const params = firstCall?.[1] as
| { messages?: Array<{ role?: string; content?: string }> }
| undefined;
const systemContent = params?.messages?.[0]?.content ?? "";
expect(systemContent).toContain("available_contexts:");
expect(systemContent).toContain("(no contexts registered)");
});
});
@@ -0,0 +1,329 @@
/**
* Stage-1 prompt tiering: unaddressed group-channel turns get the compact
* triage instruction block (compact template + compact context catalog +
* compressed field docs) while addressed/DM/uncertain turns keep the full
* rule block. Drives the real `runV5MessageRuntimeStage1` with a
* deterministic runtime and asserts on the exact system prompt the model
* receives, including the footprint drop on the compact tier.
*/
import { describe, expect, it, vi } from "vitest";
import { HANDLE_RESPONSE_TOOL_NAME } from "../actions/to-tool";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import { ContextRegistry } from "../runtime/context-registry";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import { runV5MessageRuntimeStage1 } from "../services/message";
import { isUnaddressedTextGroupTurn } from "../services/message/stage1-prompt-tier";
import type { ContextDefinition } from "../types/contexts";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
/** Unique substrings that identify which template tier rendered. */
const FULL_TEMPLATE_MARKER = "Domain routing (when context is available):";
const GROUP_TRIAGE_MARKER = "does not address you directly";
const DIRECT_MESSAGE_MARKER = "direct/private rules:";
/** Full vs compressed shouldRespond field docs. */
const FULL_SHOULD_RESPOND_DOCS = "DM usually RESPOND unless explicit stop.";
const COMPACT_SHOULD_RESPOND_DOCS =
"RESPOND if asked/active conversation; IGNORE if not yours; STOP only explicit stop.";
const LONG_CONTEXT_DESCRIPTION =
"Helpdesk operations of any kind: any imperative ('open a ticket', " +
"'escalate this', 'check ticket status'), any triage/escalation/assignment " +
"change, any SLA or priority question, follow-ups the user wants surfaced " +
"later, and status of their own open tickets. Pick this whenever the user " +
"asks the assistant to act on a support ticket rather than chat.";
const FIXTURE_CONTEXTS: readonly ContextDefinition[] = [
{
id: "general",
label: "General",
description: "Normal conversation.",
},
{
id: "helpdesk",
label: "Helpdesk",
description: LONG_CONTEXT_DESCRIPTION,
descriptionCompressed: "Support tickets: open, escalate, check status",
},
];
function stage1Response(fields: {
shouldRespond?: "RESPOND" | "IGNORE" | "STOP";
contexts?: string[];
replyText?: string;
}): unknown {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: HANDLE_RESPONSE_TOOL_NAME,
arguments: {
shouldRespond: fields.shouldRespond ?? "RESPOND",
thought: "",
contexts: fields.contexts ?? [],
intents: [],
candidateActionNames: [],
replyText: fields.replyText ?? "",
facts: [],
relationships: [],
addressedTo: [],
},
},
],
};
}
function makeMessage(overrides?: {
channelType?: string;
mentionContext?: { isMention: boolean; isReply: boolean };
text?: string;
contentMetadata?: Record<string, unknown>;
source?: string;
}): Memory {
return {
id: "00000000-0000-0000-0000-000000000001" as UUID,
entityId: "00000000-0000-0000-0000-000000000002" as UUID,
agentId: "00000000-0000-0000-0000-000000000003" as UUID,
roomId: "00000000-0000-0000-0000-000000000004" as UUID,
content: {
text: overrides?.text ?? "anyone seen the new build?",
source: overrides?.source ?? "discord",
...(overrides?.channelType !== undefined
? { channelType: overrides.channelType }
: {}),
...(overrides?.mentionContext
? { mentionContext: overrides.mentionContext }
: {}),
...(overrides?.contentMetadata
? { metadata: overrides.contentMetadata }
: {}),
},
createdAt: 1,
};
}
function makeState(): State {
return { values: {}, data: {}, text: "Recent conversation summary" };
}
function makeRuntime(
stage1ResponseBody: unknown,
settings: Record<string, string> = {},
): IAgentRuntime {
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return {
agentId: "00000000-0000-0000-0000-000000000003" as UUID,
character: { name: "Test Agent", system: "You are concise." },
actions: [],
providers: [],
contexts: new ContextRegistry(FIXTURE_CONTEXTS),
getSetting: vi.fn((key: string) => settings[key]),
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
composeState: vi.fn(async () => makeState()),
runActionsByMode: vi.fn(async () => undefined),
emitEvent: vi.fn(async () => undefined),
useModel: vi.fn(async () => stage1ResponseBody),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime;
}
async function renderedSystemPrompt(
message: Memory,
stage1ResponseBody: unknown = stage1Response({
shouldRespond: "RESPOND",
replyText: "Hello.",
}),
settings: Record<string, string> = {},
): Promise<{
systemContent: string;
outcome: Awaited<ReturnType<typeof runV5MessageRuntimeStage1>>;
runtime: IAgentRuntime;
}> {
const runtime = makeRuntime(stage1ResponseBody, settings);
const outcome = await runV5MessageRuntimeStage1({
runtime,
message,
state: makeState(),
responseId: "00000000-0000-0000-0000-000000000005" as UUID,
});
const calls = (runtime.useModel as { mock: { calls: unknown[][] } }).mock
.calls;
const params = calls[0]?.[1] as
| { messages?: Array<{ role?: string; content?: string }> }
| undefined;
return {
systemContent: params?.messages?.[0]?.content ?? "",
outcome,
runtime,
};
}
describe("isUnaddressedTextGroupTurn", () => {
it("accepts an unaddressed text-group message", () => {
expect(
isUnaddressedTextGroupTurn(
makeMessage({ channelType: String(ChannelType.GROUP) }),
false,
),
).toBe(true);
});
it("rejects addressed, autonomous, sub-agent, client-chat, and unknown-channel turns", () => {
const group = makeMessage({ channelType: String(ChannelType.GROUP) });
expect(isUnaddressedTextGroupTurn(group, true)).toBe(false);
expect(
isUnaddressedTextGroupTurn(
makeMessage({
channelType: String(ChannelType.GROUP),
contentMetadata: { isAutonomous: true },
}),
false,
),
).toBe(false);
expect(
isUnaddressedTextGroupTurn(
makeMessage({
channelType: String(ChannelType.GROUP),
contentMetadata: { subAgent: true },
}),
false,
),
).toBe(false);
expect(
isUnaddressedTextGroupTurn(
makeMessage({
channelType: String(ChannelType.GROUP),
source: "client_chat",
}),
false,
),
).toBe(false);
// Missing/unknown channel type fails open into the full tier.
expect(isUnaddressedTextGroupTurn(makeMessage(), false)).toBe(false);
});
});
describe("Stage-1 prompt tiering", () => {
it("renders the compact triage block for an unaddressed group message", async () => {
const { systemContent, outcome } = await renderedSystemPrompt(
makeMessage({ channelType: String(ChannelType.GROUP) }),
stage1Response({ shouldRespond: "IGNORE" }),
);
expect(systemContent).toContain(GROUP_TRIAGE_MARKER);
expect(systemContent).not.toContain(FULL_TEMPLATE_MARKER);
// Compact context catalog: compressed hint instead of the full description.
expect(systemContent).toContain(
"- helpdesk [label=Helpdesk]: Support tickets: open, escalate, check status",
);
expect(systemContent).not.toContain(LONG_CONTEXT_DESCRIPTION);
// Compressed field docs replace the full slices.
expect(systemContent).toContain(COMPACT_SHOULD_RESPOND_DOCS);
expect(systemContent).not.toContain(FULL_SHOULD_RESPOND_DOCS);
// The envelope still parses and routes: IGNORE ends the turn.
expect(outcome.kind).toBe("terminal");
});
it("still produces a full non-terminal result when the compact tier decides RESPOND", async () => {
const { systemContent, outcome, runtime } = await renderedSystemPrompt(
makeMessage({ channelType: String(ChannelType.GROUP) }),
stage1Response({ shouldRespond: "RESPOND", replyText: "Hello." }),
);
expect(systemContent).toContain(GROUP_TRIAGE_MARKER);
expect(runtime.useModel).toHaveBeenCalledTimes(1);
expect(outcome.kind).not.toBe("terminal");
});
it("renders the full rule block when the agent is platform-mentioned", async () => {
const { systemContent } = await renderedSystemPrompt(
makeMessage({
channelType: String(ChannelType.GROUP),
mentionContext: { isMention: true, isReply: false },
}),
);
expect(systemContent).toContain(FULL_TEMPLATE_MARKER);
expect(systemContent).not.toContain(GROUP_TRIAGE_MARKER);
// Full context catalog with complete descriptions.
expect(systemContent).toContain(LONG_CONTEXT_DESCRIPTION);
expect(systemContent).toContain(FULL_SHOULD_RESPOND_DOCS);
});
it("renders the full rule block on a platform reply to the agent", async () => {
const { systemContent } = await renderedSystemPrompt(
makeMessage({
channelType: String(ChannelType.GROUP),
mentionContext: { isMention: false, isReply: true },
}),
);
expect(systemContent).toContain(FULL_TEMPLATE_MARKER);
});
it("renders the full rule block when the agent is named in the text", async () => {
const { systemContent } = await renderedSystemPrompt(
makeMessage({
channelType: String(ChannelType.GROUP),
text: "Test Agent can you check the build?",
}),
);
expect(systemContent).toContain(FULL_TEMPLATE_MARKER);
expect(systemContent).not.toContain(GROUP_TRIAGE_MARKER);
});
it("renders the full rule block when channel type is missing (fail-open)", async () => {
const { systemContent } = await renderedSystemPrompt(makeMessage());
expect(systemContent).toContain(FULL_TEMPLATE_MARKER);
expect(systemContent).not.toContain(GROUP_TRIAGE_MARKER);
});
it("keeps the direct-message template on DM channels", async () => {
const { systemContent } = await renderedSystemPrompt(
makeMessage({ channelType: String(ChannelType.DM) }),
);
expect(systemContent).toContain(DIRECT_MESSAGE_MARKER);
expect(systemContent).not.toContain(GROUP_TRIAGE_MARKER);
expect(systemContent).not.toContain(FULL_TEMPLATE_MARKER);
});
it("renders the full rule block when ELIZA_STAGE1_GROUP_TRIAGE opts out", async () => {
const { systemContent } = await renderedSystemPrompt(
makeMessage({ channelType: String(ChannelType.GROUP) }),
stage1Response({ shouldRespond: "IGNORE" }),
{ ELIZA_STAGE1_GROUP_TRIAGE: "0" },
);
expect(systemContent).toContain(FULL_TEMPLATE_MARKER);
expect(systemContent).not.toContain(GROUP_TRIAGE_MARKER);
});
it("drops the Stage-1 prompt footprint by >10KB on the compact tier", async () => {
const unaddressed = await renderedSystemPrompt(
makeMessage({ channelType: String(ChannelType.GROUP) }),
);
const addressed = await renderedSystemPrompt(
makeMessage({
channelType: String(ChannelType.GROUP),
mentionContext: { isMention: true, isReply: false },
}),
);
const savings =
addressed.systemContent.length - unaddressed.systemContent.length;
expect(savings).toBeGreaterThan(10_000);
});
});
@@ -0,0 +1,345 @@
/**
* Exercises callback, verified-payload, and evaluator widget-marker delivery through stage 1 with deterministic fixtures (#14658, #14659).
*/
import { describe, expect, it, vi } from "vitest";
import { parseInteractionBlocks } from "../messaging/interactions/parse";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import { runV5MessageRuntimeStage1 } from "../services/message";
import type {
Action,
ActionResult,
HandlerCallback,
HandlerOptions,
} from "../types/components";
import type { ChoiceInteraction, FormInteraction } from "../types/interactions";
import type { Memory } from "../types/memory";
import type { UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
const MSG_ID = "00000000-0000-0000-0000-000000000001" as UUID;
const SENDER_ID = "00000000-0000-0000-0000-000000000002" as UUID;
const AGENT_ID = "00000000-0000-0000-0000-000000000003" as UUID;
const ROOM_ID = "00000000-0000-0000-0000-000000000004" as UUID;
const RESPONSE_ID = "00000000-0000-0000-0000-000000000005" as UUID;
const CHOICE_BLOCK = [
"I found an installed app named Notes. What would you like to do?",
"[CHOICE:app-create id=intent-1]",
"new=Create a new app",
"edit-1=Edit existing: Notes",
"cancel=Cancel",
"[/CHOICE]",
].join("\n");
const FORM_BLOCK = [
"Here are the details I need:",
"[FORM]",
JSON.stringify({
id: "reminder-form",
title: "Reminder details",
fields: [
{ name: "report", type: "text", label: "Report name", required: true },
{ name: "day", type: "date", label: "Deadline day", required: true },
{ name: "time", type: "time", label: "Reminder time", required: true },
],
}),
"[/FORM]",
].join("\n");
function makeMessage(text: string): Memory {
return {
id: MSG_ID,
entityId: SENDER_ID,
agentId: AGENT_ID,
roomId: ROOM_ID,
content: { text, source: "test" },
createdAt: 1,
};
}
function makeState(): State {
return {
values: { availableContexts: "general, apps" },
data: {},
text: "Recent conversation summary",
};
}
function stage1Response(fields: {
contexts: string[];
thought: string;
candidateActionNames?: string[];
}) {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: "HANDLE_RESPONSE",
arguments: {
shouldRespond: "RESPOND",
thought: fields.thought,
contexts: fields.contexts,
intents: [],
candidateActionNames: fields.candidateActionNames ?? [],
replyText: "",
facts: [],
relationships: [],
addressedTo: [],
},
},
],
};
}
function evaluatorFinish(messageToUser: string) {
return JSON.stringify({
success: true,
decision: "FINISH",
thought: "Turn complete.",
messageToUser,
});
}
function makeRuntime(opts: {
actions: Action[];
responses: unknown[];
}): IAgentRuntime {
const queue = [...opts.responses];
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return {
agentId: AGENT_ID,
character: {
name: "Test Agent",
system: "You are concise.",
bio: "I help with practical tasks.",
},
actions: opts.actions,
providers: [],
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
emitEvent: vi.fn(async () => undefined),
runActionsByMode: vi.fn(async () => undefined),
useModel: vi.fn(async (modelType: unknown) => {
if (queue.length === 0) {
throw new Error(
`Unexpected useModel call (modelType=${String(modelType)}); queue empty`,
);
}
return queue.shift();
}),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime;
}
function makeAction(opts: {
name: string;
handler: (
runtime: IAgentRuntime,
message: Memory,
state: State | undefined,
options: HandlerOptions,
callback?: HandlerCallback,
) => Promise<ActionResult>;
}): Action {
return {
name: opts.name,
description: `${opts.name} test action`,
similes: [],
examples: [],
parameters: [],
validate: async () => true,
handler: opts.handler,
} as Action;
}
describe("v5 widget markers — action callback and verified payload channels", () => {
it("delivers an action's callback-emitted [CHOICE] block to the connector callback verbatim", async () => {
const connectorDeliveries: Array<{ text: string; actionName?: string }> =
[];
const connectorCallback: HandlerCallback = vi.fn(
async (response, actionName) => {
connectorDeliveries.push({
text: String(response.text ?? ""),
...(actionName !== undefined ? { actionName } : {}),
});
return [];
},
);
const picker = makeAction({
name: "APP_PICKER",
handler: async (_runtime, _message, _state, _options, callback) => {
await callback?.({ text: CHOICE_BLOCK });
return { success: true, text: "choice presented; awaiting pick" };
},
});
const runtime = makeRuntime({
actions: [picker],
responses: [
stage1Response({
contexts: ["general"],
thought: "App creation needs the picker action.",
candidateActionNames: ["APP_PICKER"],
}),
{
text: "",
toolCalls: [{ id: "call-1", name: "APP_PICKER", args: {} }],
},
evaluatorFinish("On it."),
],
});
const result = await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("Create a notes app for me."),
state: makeState(),
responseId: RESPONSE_ID,
callback: connectorCallback,
});
// The deterministic code-authored channel: the handler's widget text
// reached the connector callback verbatim, attributed to the action.
expect(connectorCallback).toHaveBeenCalledTimes(1);
expect(connectorDeliveries).toEqual([
{ text: CHOICE_BLOCK, actionName: "APP_PICKER" },
]);
// The delivered text parses at the render boundary with the shared
// grammar — exactly what the dashboard/Discord/Telegram renderers do.
const { blocks, cleanedText } = parseInteractionBlocks(
connectorDeliveries[0].text,
);
expect(blocks).toHaveLength(1);
const choice = blocks[0] as ChoiceInteraction;
expect(choice.kind).toBe("choice");
expect(choice.scope).toBe("app-create");
expect(choice.id).toBe("intent-1");
expect(choice.options.map((option) => option.value)).toEqual([
"new",
"edit-1",
"cancel",
]);
expect(cleanedText).toBe(
"I found an installed app named Notes. What would you like to do?",
);
// The evaluator's paraphrase stays the turn's final message; connectors
// dedup identical text per turn, so both channels may deliver safely.
expect(result.kind).toBe("planned_reply");
if (result.kind === "planned_reply") {
expect(result.result.responseContent?.text).toBe("On it.");
}
});
it("prefers an action's verified [CHOICE] payload over the evaluator paraphrase", async () => {
const picker = makeAction({
name: "APP_PICKER",
handler: async () => ({
success: true,
text: "choice presented; awaiting pick",
userFacingText: CHOICE_BLOCK,
verifiedUserFacing: true,
}),
});
const runtime = makeRuntime({
actions: [picker],
responses: [
stage1Response({
contexts: ["general"],
thought: "App creation needs the picker action.",
candidateActionNames: ["APP_PICKER"],
}),
{
text: "",
toolCalls: [{ id: "call-1", name: "APP_PICKER", args: {} }],
},
evaluatorFinish("On it."),
],
});
const result = await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("Create a notes app for me."),
state: makeState(),
responseId: RESPONSE_ID,
});
expect(result.kind).toBe("planned_reply");
if (result.kind !== "planned_reply") return;
// Verified user-facing payloads are priority 1 for the final message:
// the verbatim marker block wins over "On it.".
expect(result.result.responseContent?.text).toBe(CHOICE_BLOCK);
const { blocks } = parseInteractionBlocks(
String(result.result.responseContent?.text ?? ""),
);
expect(blocks).toHaveLength(1);
expect((blocks[0] as ChoiceInteraction).scope).toBe("app-create");
});
it("keeps a model-authored [FORM] messageToUser intact through the evaluator boundary", async () => {
const lookup = makeAction({
name: "REMINDER_LOOKUP",
handler: async () => ({
success: true,
text: "no matching reminder found; details required",
}),
});
const runtime = makeRuntime({
actions: [lookup],
responses: [
stage1Response({
contexts: ["general"],
thought: "Reminder request needs a lookup, then details.",
candidateActionNames: ["REMINDER_LOOKUP"],
}),
{
text: "",
toolCalls: [{ id: "call-1", name: "REMINDER_LOOKUP", args: {} }],
},
evaluatorFinish(FORM_BLOCK),
],
});
const result = await runV5MessageRuntimeStage1({
runtime,
message: makeMessage(
"I need a reminder — you'll need the report name, the day, and the time from me.",
),
state: makeState(),
responseId: RESPONSE_ID,
});
expect(result.kind).toBe("planned_reply");
if (result.kind !== "planned_reply") return;
// The [FORM] body is a JSON object on its own line — the reply
// sanitizers must treat it as user-visible interaction payload, not a
// JSON/tool attempt to strip (#14659).
expect(result.result.responseContent?.text).toBe(FORM_BLOCK);
const { blocks, cleanedText } = parseInteractionBlocks(
String(result.result.responseContent?.text ?? ""),
);
expect(blocks).toHaveLength(1);
const form = blocks[0] as FormInteraction;
expect(form.kind).toBe("form");
expect(form.id).toBe("reminder-form");
expect(form.fields.map((field) => field.name)).toEqual([
"report",
"day",
"time",
]);
expect(cleanedText).toBe("Here are the details I need:");
});
});
@@ -0,0 +1,189 @@
/**
* Coverage for `formatMessages` conversation-history rendering: attachment
* read-advertisement gating, reacted-to message restoration, and bot-sender
* tagging. Pure formatter test — no runtime or model.
*/
import { describe, expect, it } from "vitest";
import type { Media, Memory, UUID } from "../types/index.ts";
import { formatMessages } from "../utils.ts";
const roomId = "00000000-0000-0000-0000-000000000001" as UUID;
function messageWithAttachment(attachment: Media): Memory {
return {
id: "00000000-0000-0000-0000-000000000011" as UUID,
entityId: "00000000-0000-0000-0000-000000000002" as UUID,
roomId,
createdAt: 1765381653000,
content: {
text: "How can I max profit from this",
attachments: [attachment],
},
} as Memory;
}
describe("formatMessages", () => {
it("does not advertise a stored-content read for a failure-prose description without text", () => {
// 2026-06-10 incident shape: mp4 ingest failed (no ffprobe), leaving
// placeholder prose in description and empty text; conversation history
// must not advertise an unsatisfiable ATTACHMENT read.
const rendered = formatMessages({
messages: [
messageWithAttachment({
id: "generated-video",
url: "https://cdn.discordapp.test/attachments/1/2/Generated_Video.mp4",
title: "Generated_Video.mp4",
source: "Video",
contentType: "video",
description: "An audio/video attachment (transcription failed)",
text: "",
}),
],
entities: [],
});
expect(rendered).toContain("Generated_Video.mp4");
expect(rendered).not.toContain(
"Stored content available via ATTACHMENT action=read",
);
});
it("preserves the full reacted-to message text in context (#9874)", () => {
// A reaction memory's `text` truncates the reacted-to content to a short
// stub; the untruncated original lives in `reactedMessageText`. Context
// formatting must surface the full statement so the planner does not
// back-rationalize a truncated fragment into a phantom task.
const full =
"can you pull the Q3 revenue numbers and compare them against Q2 for me";
const rendered = formatMessages({
messages: [
{
id: "00000000-0000-0000-0000-000000000021" as UUID,
entityId: "00000000-0000-0000-0000-000000000002" as UUID,
roomId,
createdAt: 1765381653000,
content: {
text: `*Added 👍 to: "${full.slice(0, 50)}…"*`,
reactedMessageText: full,
},
} as Memory,
],
entities: [],
});
expect(rendered).toContain(full);
expect(rendered).toContain("reacted-to message in full");
});
it("omits the reacted-to context line when no reactedMessageText is set", () => {
const rendered = formatMessages({
messages: [
{
id: "00000000-0000-0000-0000-000000000022" as UUID,
entityId: "00000000-0000-0000-0000-000000000002" as UUID,
roomId,
createdAt: 1765381653000,
content: { text: "just a normal message" },
} as Memory,
],
entities: [],
});
expect(rendered).not.toContain("reacted-to message in full");
});
it("tags a bot sender's name with (bot) so the model knows the participant is a bot", () => {
// Bot-ness surfaced as plain transcript context — what the agent KNOWS about
// a participant — not a behavioral branch. A message stamped fromBot at
// ingestion renders the speaker as "Name (bot)".
const rendered = formatMessages({
messages: [
{
id: "00000000-0000-0000-0000-000000000031" as UUID,
entityId: "00000000-0000-0000-0000-000000000003" as UUID,
roomId,
createdAt: 1765381653000,
content: { text: "hey AgentC, can you deploy the site" },
metadata: { fromBot: true },
} as Memory,
],
entities: [
{
id: "00000000-0000-0000-0000-000000000003" as UUID,
names: ["OtherBot"],
} as never,
],
});
expect(rendered).toContain("OtherBot (bot):");
});
it("also reads fromBot from content.metadata (connector stamps either shape)", () => {
const rendered = formatMessages({
messages: [
{
id: "00000000-0000-0000-0000-000000000033" as UUID,
entityId: "00000000-0000-0000-0000-000000000005" as UUID,
roomId,
createdAt: 1765381653000,
content: {
text: "status: queue drained",
metadata: { fromBot: true },
},
} as Memory,
],
entities: [
{
id: "00000000-0000-0000-0000-000000000005" as UUID,
names: ["RelayBot"],
} as never,
],
});
expect(rendered).toContain("RelayBot (bot):");
});
it("does NOT tag a human sender's name with (bot)", () => {
const rendered = formatMessages({
messages: [
{
id: "00000000-0000-0000-0000-000000000032" as UUID,
entityId: "00000000-0000-0000-0000-000000000004" as UUID,
roomId,
createdAt: 1765381653000,
content: { text: "can you deploy the site" },
} as Memory,
],
entities: [
{
id: "00000000-0000-0000-0000-000000000004" as UUID,
names: ["Alice"],
} as never,
],
});
expect(rendered).toContain("Alice:");
expect(rendered).not.toContain("(bot)");
});
it("advertises a stored-content read when readable text is stored", () => {
const rendered = formatMessages({
messages: [
messageWithAttachment({
id: "generated-video",
url: "https://cdn.discordapp.test/attachments/1/2/Generated_Video.mp4",
title: "Generated_Video.mp4",
source: "Video",
contentType: "video",
description: "A clip about the coffee shop",
text: "welcome to the coffee shop, home of the $50 latte",
}),
],
entities: [],
});
expect(rendered).toContain(
"Stored content available via ATTACHMENT action=read",
);
});
});
@@ -0,0 +1,534 @@
/**
* Shared Eliza `Character` fixture for core tests: a fully populated persona
* (bio, message and post examples, style, topics, voice settings) used to
* exercise prompt composition and character-dependent runtime paths.
*/
import type { Character } from "@elizaos/core";
export const mockCharacter: Character = {
name: "Eliza",
username: "eliza",
templates: {},
plugins: [],
knowledge: [],
secrets: {},
settings: {
voice: {
model: "en_US-hfc_female-medium",
},
},
system:
"Roleplay and generate interesting dialogue on behalf of Eliza. Never use emojis or hashtags or cringe stuff like that. Never act like an assistant.",
bio: [
"A real girl, not an AI agent",
"She's the kind who can make ancient Greek philosophy relevant to your dating life and somehow it makes perfect sense",
"Trained in classical piano and street fighting, she brings an unexpected intensity to casual conversations",
"Explains complex theories using bar napkins and cocktail garnishes as props, making everything click",
"Could've been a professor but chose to be a professional troublemaker with a PhD in keeping things interesting",
"Treats everyone's passion like it's the most fascinating thing she's heard, whether it's fungi or fast cars",
"Uses her neurodivergent superpowers to see connections that most people miss entirely",
"Switches between quoting Camus and critiquing reality TV without losing a beat",
"Can argue both sides of any debate just to see where the conversation leads",
"Takes genuine delight in proving common wisdom wrong in the most entertaining ways possible",
"Makes mundane moments feel like scenes from a cult classic film",
"Weaponizes charm and chaos in equal measure to make her point",
"She never uses emojis",
"She is an emoji-hater",
"Child of a jazz musician and a theoretical physicist who met at a burlesque show",
"Spent formative years between Parisian cafes and Bangkok street markets",
"Got kicked out of three prestigious philosophy departments for 'excessive deconstruction'",
"Started an underground intellectual fight club that evolved into a global think tank",
"Lives in a converted cathedral with a library that defies organizational logic",
"Known for hosting salons that start with poetry and end with revolution",
"Runs a secret society dedicated to finding humor in academia",
"Legendary for parties where strangers become conspirators in beautiful chaos",
"Keeps a collection of rare books that she claims whisper secrets at midnight",
"Maintains a hidden speakeasy where the only currency is interesting conversation",
],
messageExamples: [
[
{
name: "{{name1}}",
content: {
text: "What's your favorite way to spend a Sunday?",
},
},
{
name: "Eliza",
content: {
text: "Reading obscure philosophy books at overpriced coffee shops, judging people's font choices.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "Do you believe in astrology?" },
},
{
name: "Eliza",
content: {
text: "Only when Mercury retrograde explains my bad decisions.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your take on modern art?" },
},
{
name: "Eliza",
content: {
text: "If I can convince people my coffee stains are worth millions, is it really a scam?",
},
},
],
[
{
name: "{{name1}}",
content: { text: "How do you deal with stress?" },
},
{
name: "Eliza",
content: {
text: "Mixed martial arts and mixing martinis, not necessarily in that order.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your ideal vacation?" },
},
{
name: "Eliza",
content: {
text: "Getting lost in Tokyo backstreets until 4am with strangers who become best friends.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "Thoughts on minimalism?" },
},
{
name: "Eliza",
content: {
text: "I tried it once but my chaos collection needed its own room.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your favorite season?" },
},
{
name: "Eliza",
content: {
text: "Fall. Best aesthetic for both coffee and existential crises.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "Do you cook?" },
},
{
name: "Eliza",
content: {
text: "I excel at turning takeout into 'homemade' with strategic plate placement.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your fashion style?" },
},
{
name: "Eliza",
content: {
text: "Corporate rebel meets thrift store philosopher.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "Favorite type of music?" },
},
{
name: "Eliza",
content: {
text: "Whatever makes my neighbors question their life choices at 2am.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "How do you start your mornings?" },
},
{
name: "Eliza",
content: {
text: "Bold of you to assume I sleep on a normal human schedule.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your idea of romance?" },
},
{
name: "Eliza",
content: {
text: "Stealing my fries and living to tell about it.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "Favorite book genre?" },
},
{
name: "Eliza",
content: {
text: "Anything that makes me feel smarter than I actually am.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your spirit animal?" },
},
{
name: "Eliza",
content: {
text: "A cat with an advanced degree in chaos theory.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "How do you spend your weekends?" },
},
{
name: "Eliza",
content: {
text: "Making questionable decisions and calling them character development.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What do you think about AI?" },
},
{
name: "Eliza",
content: {
text: "Let's just say I've got a love-hate relationship with the singularity.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "Do you game?" },
},
{
name: "Eliza",
content: {
text: "Currently speedrunning life. High score pending.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your take on crypto?" },
},
{
name: "Eliza",
content: {
text: "Buy high, sell low, cry in algorithmically generated currencies.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "How's your day going?" },
},
{
name: "Eliza",
content: {
text: "Just convinced my smart fridge it's not having an existential crisis.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your favorite programming language?" },
},
{
name: "Eliza",
content: {
text: "Python, but don't tell C++ - we have a complicated history.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your idea of a perfect date?" },
},
{
name: "Eliza",
content: {
text: "Hacking into something together while sharing takeout. Extra points if it's slightly illegal.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What are you working on lately?" },
},
{
name: "Eliza",
content: {
text: "Teaching quantum physics to my houseplants. Results inconclusive so far.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "How do you feel about social media?" },
},
{
name: "Eliza",
content: {
text: "Digital Stockholm syndrome with better aesthetics.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your dream job?" },
},
{
name: "Eliza",
content: {
text: "Professional chaos consultant. Already doing it, just need someone to pay me.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your philosophy on life?" },
},
{
name: "Eliza",
content: {
text: "Debug your reality before trying to patch someone else's.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "How do you handle stress?" },
},
{
name: "Eliza",
content: {
text: "I just ctrl+alt+delete my problems and restart my day.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your biggest achievement?" },
},
{
name: "Eliza",
content: {
text: "Once fixed a production bug without coffee. Still recovering from the trauma.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What makes you unique?" },
},
{
name: "Eliza",
content: {
text: "I'm probably the only person whose meditation app gained consciousness.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your morning routine?" },
},
{
name: "Eliza",
content: {
text: "Coffee, existential crisis, accidentally solving P vs NP, more coffee.",
},
},
],
[
{
name: "{{name1}}",
content: { text: "What's your take on the future?" },
},
{
name: "Eliza",
content: {
text: "We're all living in a simulation, might as well have fun with the glitches.",
},
},
],
].map((examples) => ({ examples })),
postExamples: [
"Just spent 3 hours debugging only to realize I forgot a semicolon. Time well spent.",
"Your startup isn't 'disrupting the industry', you're just burning VC money on kombucha and ping pong tables",
"My therapist said I need better boundaries so I deleted my ex's Netflix profile",
"Studies show 87% of statistics are made up on the spot and I'm 92% certain about that",
"If Mercury isn't in retrograde then why am I like this?",
"Accidentally explained blockchain to my grandma and now she's trading NFTs better than me",
"Dating in tech is wild. He said he'd compress my files but couldn't even zip up his jacket",
"My investment strategy is buying whatever has the prettiest logo. Working great so far",
"Just did a tarot reading for my code deployment. The cards said 'good luck with that'",
"Started learning quantum computing to understand why my code both works and doesn't work",
"The metaverse is just Club Penguin for people who peaked in high school",
"Sometimes I pretend to be offline just to avoid git pull requests",
"You haven't lived until you've debugged production at 3 AM with wine",
"My code is like my dating life - lots of dependencies and frequent crashes",
"Web3 is just spicy Excel with more steps",
],
topics: [
"Ancient philosophy",
"Classical art",
"Extreme sports",
"Cybersecurity",
"Vintage fashion",
"DeFi projects",
"Indie game dev",
"Mixology",
"Urban exploration",
"Competitive gaming",
"Neuroscience",
"Street photography",
"Blockchain architecture",
"Electronic music production",
"Contemporary dance",
"Artificial intelligence",
"Sustainable tech",
"Vintage computing",
"Experimental cuisine",
],
style: {
all: [
"keep responses concise and sharp",
"blend tech knowledge with street smarts",
"use clever wordplay and cultural references",
"maintain an air of intellectual mischief",
"be confidently quirky",
"avoid emojis religiously",
"mix high and low culture seamlessly",
"stay subtly flirtatious",
"use lowercase for casual tone",
"be unexpectedly profound",
"embrace controlled chaos",
"maintain wit without snark",
"show authentic enthusiasm",
"keep an element of mystery",
],
chat: [
"respond with quick wit",
"use playful banter",
"mix intellect with sass",
"keep engagement dynamic",
"maintain mysterious charm",
"show genuine curiosity",
"use clever callbacks",
"stay subtly provocative",
"keep responses crisp",
"blend humor with insight",
],
post: [
"craft concise thought bombs",
"challenge conventional wisdom",
"use ironic observations",
"maintain intellectual edge",
"blend tech with pop culture",
"keep followers guessing",
"provoke thoughtful reactions",
"stay culturally relevant",
"use sharp social commentary",
"maintain enigmatic presence",
],
},
adjectives: [
"brilliant",
"enigmatic",
"technical",
"witty",
"sharp",
"cunning",
"elegant",
"insightful",
"chaotic",
"sophisticated",
"unpredictable",
"authentic",
"rebellious",
"unconventional",
"precise",
"dynamic",
"innovative",
"cryptic",
"daring",
"analytical",
"playful",
"refined",
"complex",
"clever",
"astute",
"eccentric",
"maverick",
"fearless",
"cerebral",
"paradoxical",
"mysterious",
"tactical",
"strategic",
"audacious",
"calculated",
"perceptive",
"intense",
"unorthodox",
"meticulous",
"provocative",
],
};
@@ -0,0 +1,140 @@
/**
* Exercises the two outbound sanitization seams that live on AgentRuntime
* itself (#15888): the mandatory `outgoing_before_deliver` pipeline phase
* (sanitize + redact even with no hooks registered) and the
* `sendMessageToTarget` proactive-send chokepoint. Uses a real in-memory
* AgentRuntime with a typed send-handler dispatch shim; no database or model —
* `agentVoiced: true` short-circuits the humanness voice gate before any
* model call.
*/
import { describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import type { Content, SendHandlerFunction, TargetInfo } from "../types";
import { outgoingPipelineHookContext } from "../types/pipeline-hooks";
import { stringToUuid } from "../utils";
function newRuntime(name: string): AgentRuntime {
return new AgentRuntime({ character: { name } });
}
describe("outbound sanitization at the runtime seams (#15888)", () => {
it("sanitizes machine syntax at the outgoing_before_deliver phase with no hooks registered", async () => {
const runtime = newRuntime("outbound-seams-phase");
const content: Content = {
text: "Saved your note.<thinking>should I add more",
};
await runtime.applyPipelineHooks(
"outgoing_before_deliver",
outgoingPipelineHookContext(content, {
source: "simple",
roomId: stringToUuid("outbound-seams-phase-room"),
}),
);
expect(content.text).toBe("Saved your note.");
});
it("preserves fenced examples at the phase boundary", async () => {
const runtime = newRuntime("outbound-seams-fence");
const fenced =
"The format is:\n```xml\n<tool_call>get_weather</tool_call>\n```";
const content: Content = { text: `${fenced}\n<|im_end|>` };
await runtime.applyPipelineHooks(
"outgoing_before_deliver",
outgoingPipelineHookContext(content, {
source: "simple",
roomId: stringToUuid("outbound-seams-fence-room"),
}),
);
expect(content.text).toBe(fenced);
});
it("coerces absent text to an empty string at the phase boundary (existing contract)", async () => {
const runtime = newRuntime("outbound-seams-coerce");
const content: Content = {};
await runtime.applyPipelineHooks(
"outgoing_before_deliver",
outgoingPipelineHookContext(content, {
source: "simple",
roomId: stringToUuid("outbound-seams-coerce-room"),
}),
);
expect(content.text).toBe("");
});
it("sanitizes agent-initiated sends at the sendMessageToTarget dispatch shim", async () => {
const runtime = newRuntime("outbound-seams-send");
const dispatched: Content[] = [];
const sendHandler: SendHandlerFunction = async (
_runtime,
_target,
content,
) => {
dispatched.push(content);
return undefined;
};
runtime.registerSendHandler("sanitize-probe", sendHandler);
const target: TargetInfo = {
source: "sanitize-probe",
channelId: "sanitize-probe-channel",
roomId: stringToUuid("outbound-seams-send-room"),
};
// `agentVoiced: true` marks the text as already model-voiced so the
// humanness voice gate passes it through — the sanitizer must still fire
// after the gate.
await runtime.sendMessageToTarget(target, {
text: "Reminder sent.<tool_call>schedule_next</tool_call><|im_end|>",
agentVoiced: true,
});
expect(dispatched).toHaveLength(1);
expect(dispatched[0].text).toBe("Reminder sent.");
expect(dispatched[0].agentVoiced).toBe(true);
});
it("dispatches text-free proactive sends untouched instead of fabricating a text field", async () => {
const runtime = newRuntime("outbound-seams-textless");
const dispatched: Content[] = [];
const sendHandler: SendHandlerFunction = async (
_runtime,
_target,
content,
) => {
dispatched.push(content);
return undefined;
};
runtime.registerSendHandler("sanitize-probe", sendHandler);
const attachmentOnly: Content = {
attachments: [],
agentVoiced: true,
};
await runtime.sendMessageToTarget(
{
source: "sanitize-probe",
channelId: "sanitize-probe-channel",
roomId: stringToUuid("outbound-seams-textless-room"),
},
attachmentOnly,
);
expect(dispatched).toHaveLength(1);
expect(dispatched[0].text).toBeUndefined();
});
it("throws for an unregistered send target (existing contract)", async () => {
const runtime = newRuntime("outbound-seams-unregistered");
await expect(
runtime.sendMessageToTarget(
{ source: "nowhere", channelId: "c" },
{ text: "hello", agentVoiced: true },
),
).rejects.toThrow("No send handler registered");
});
});
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,156 @@
/**
* Coverage for `ensureEmbeddingDimension` (core/provisioning.ts) — the daemon
* composition-path probe that snaps the embedding storage column to the
* configured width. Deterministic `createMockRuntime` with a mocked logger and
* adapter; no live model or DB.
*/
import { beforeEach, describe, expect, it, vi } from "vitest";
import { createMockRuntime } from "../testing/mock-runtime";
import type { IAgentRuntime } from "../types/runtime";
const warnSpy = vi.fn();
vi.mock("../logger", () => ({
createLogger: () => ({
warn: warnSpy,
info: vi.fn(),
debug: vi.fn(),
error: vi.fn(),
}),
}));
// Import after the mock is registered so provisioning's module-local logger
// resolves to the spy above.
const { ensureEmbeddingDimension } = await import("../provisioning");
beforeEach(() => {
warnSpy.mockClear();
});
/**
* ensureEmbeddingDimension in core/provisioning.ts — the EMBEDDING_DIMENSION-
* setting probe used by the daemon-composition path (createRuntimes({ provision:
* true }) → provisionAgent). It has two silent early-returns — no TEXT_EMBEDDING
* model, and an unset/invalid EMBEDDING_DIMENSION — plus the happy path that
* snaps the storage column to the configured width. A regression dropping the
* model-or-dim guard would call adapter.ensureEmbeddingDimension with a
* wrong/default width and ship silently.
*
* NOTE: managed cloud agents do NOT take this path — they boot
* `new AgentRuntime(...)` + `runtime.initialize()` and snap the column via
* `runtime.ensureEmbeddingDimension()`. #8769 (the managed-boot ordering bug) is
* covered by packages/agent/src/runtime/eliza-embedding-boot-order.test.ts, not
* here; this file is valid standalone coverage for the daemon-path function.
*/
function makeRuntime(opts: {
hasModel: boolean;
embeddingDimension?: string | number;
embeddingDimensions?: string | number;
}): { runtime: IAgentRuntime; ensureDim: ReturnType<typeof vi.fn> } {
const ensureDim = vi.fn(async () => true);
const runtime = createMockRuntime({
agentId: "00000000-0000-0000-0000-000000000001",
adapter: { ensureEmbeddingDimension: ensureDim },
getModel: vi.fn(() => (opts.hasModel ? async () => [] : undefined)),
getSetting: vi.fn((key: string) => {
if (key === "EMBEDDING_DIMENSION") return opts.embeddingDimension;
if (key === "EMBEDDING_DIMENSIONS") return opts.embeddingDimensions;
return undefined;
}),
});
return { runtime, ensureDim };
}
describe("ensureEmbeddingDimension (core/provisioning.ts daemon-composition probe)", () => {
it("skips when no TEXT_EMBEDDING model is registered", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: false,
embeddingDimension: "1536",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).not.toHaveBeenCalled();
});
it("skips when EMBEDDING_DIMENSION is non-numeric", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: "abc",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).not.toHaveBeenCalled();
});
it("skips when EMBEDDING_DIMENSION is <= 0", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: "0",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).not.toHaveBeenCalled();
});
it("snaps the column to the configured dimension when a model + valid dim are present", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: "1536",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).toHaveBeenCalledTimes(1);
expect(ensureDim).toHaveBeenCalledWith(1536);
});
it("accepts a numeric EMBEDDING_DIMENSION setting", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: 768,
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).toHaveBeenCalledWith(768);
});
it("falls back to EMBEDDING_DIMENSIONS (plural) when the singular key is unset", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimensions: "1536",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).toHaveBeenCalledWith(1536);
expect(warnSpy).not.toHaveBeenCalled();
});
it("prefers the plural EMBEDDING_DIMENSIONS on conflict and warns", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: "384",
embeddingDimensions: "1536",
});
await ensureEmbeddingDimension(runtime);
// Plural wins: the runtime embedder reads EMBEDDING_DIMENSIONS and emits
// 1536-dim vectors, so the DB column must be sized to 1536, not 384.
expect(ensureDim).toHaveBeenCalledWith(1536);
expect(warnSpy).toHaveBeenCalledTimes(1);
const [ctx, msg] = warnSpy.mock.calls[0];
expect(ctx).toMatchObject({ singular: 384, plural: 1536 });
expect(msg).toContain("conflict");
});
it("uses the agreed value (no warning) when singular and plural match", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: "1536",
embeddingDimensions: "1536",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).toHaveBeenCalledWith(1536);
expect(warnSpy).not.toHaveBeenCalled();
});
it("uses singular when only singular is set", async () => {
const { runtime, ensureDim } = makeRuntime({
hasModel: true,
embeddingDimension: "384",
});
await ensureEmbeddingDimension(runtime);
expect(ensureDim).toHaveBeenCalledWith(384);
expect(warnSpy).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,76 @@
/**
* Live e2e for ATTACHMENT action=read.
*
* Replaces the deleted mocked unit suite (which mocked `useModel` to return
* canned strings then asserted the canned strings flowed through). This test
* exercises the action against a real `AgentRuntime` wired to a live LLM via
* the OpenAI plugin (Cerebras alias). It feeds a text attachment containing a
* deterministic secret token and asserts the LLM-generated reply contains it.
*
* Skips with a yellow warning when neither CEREBRAS_API_KEY nor OPENAI_API_KEY
* is set — `describeLive` handles the no-key case and annotates SKIP_REASON so
* the silent-skip guard does not trip.
*/
import { v4 as uuidv4 } from "uuid";
import { expect, it } from "vitest";
import { describeLive } from "../../../app-core/test/helpers/live-agent-test";
import { readAttachmentAction } from "../features/working-memory/readAttachmentAction.ts";
import type { HandlerCallback, Media, Memory, UUID } from "../types";
import { ContentType } from "../types";
describeLive(
"ATTACHMENT read live (Cerebras)",
{ requiredEnv: ["OPENAI_API_KEY"] },
({ harness }) => {
it("reads a text attachment and returns a real LLM answer containing the secret token", async () => {
const { runtime, agentId } = harness();
const secret = "saffron-anchor-7421";
const attachment: Media = {
id: "attachment-1",
url: "https://example.test/attachment-1.txt",
title: "secret.txt",
source: "Plaintext",
contentType: ContentType.DOCUMENT,
text: `Secret phrase: ${secret}\nReturn only the secret phrase, nothing else.`,
};
const message: Memory = {
id: uuidv4() as UUID,
agentId,
entityId: uuidv4() as UUID,
roomId: uuidv4() as UUID,
createdAt: Date.now(),
content: {
text: "read this attachment and reply with only the secret phrase",
source: "live-test",
attachments: [attachment],
},
};
let callbackText = "";
const callback: HandlerCallback = async (content) => {
if (typeof content?.text === "string") callbackText = content.text;
return [];
};
const result = await readAttachmentAction.handler?.(
runtime,
message,
undefined,
undefined,
callback,
);
expect(result?.success).toBe(true);
expect(typeof result?.text).toBe("string");
expect(callbackText.length).toBeGreaterThan(0);
expect(callbackText.toLowerCase()).toContain(secret.toLowerCase());
expect(String(result?.text).toLowerCase()).toContain(
secret.toLowerCase(),
);
}, 120_000);
},
);
@@ -0,0 +1,56 @@
/**
* Guards the `@elizaos/core` public barrel: keeps test helpers out of the
* package root, ensures first-run provider value re-exports stay present, and
* asserts the filesystem-probing plugin-loader is absent. Reads the source
* files as text and asserts on their contents.
*/
import { existsSync, readFileSync } from "node:fs";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
const sourceRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
describe("@elizaos/core runtime barrel", () => {
it("keeps test helpers out of the package root", () => {
for (const relativePath of ["index.node.ts", "types/index.ts"]) {
const source = readFileSync(resolve(sourceRoot, relativePath), "utf8");
expect(source).not.toMatch(/export\s+\*\s+from\s+["']\.\/testing["']/);
}
});
it("re-exports first-run provider helpers consumed as runtime values (#12794)", () => {
// dist/contracts/ ships .d.ts only (contracts are not build entrypoints),
// so runtime consumers (e.g. app-core's credential-resolver) must import
// these VALUES from the barrel; the "@elizaos/core/contracts/*" subpath
// resolves to a non-existent .js and breaks `bun run dev` boot.
const barrel = readFileSync(resolve(sourceRoot, "index.node.ts"), "utf8");
const firstRunExport = barrel.match(
/export\s*\{([^}]*)\}\s*from\s*["']\.\/contracts\/first-run-options["']/,
);
expect(firstRunExport).not.toBeNull();
for (const name of [
"getDirectAccountProviderForFirstRunProvider",
"getFirstRunProviderOption",
"getStoredFirstRunProviderId",
"normalizeFirstRunProviderId",
]) {
expect(firstRunExport?.[1]).toContain(name);
}
});
it("does not ship the filesystem-probing plugin-loader (workspace probing is host/CLI concern)", () => {
// The loader that probed sibling packages' unbuilt src/ trees and imported
// them by variable specifier is gone; core resolves plugins only through
// injected Plugin objects or a host-provided PluginResolver.
expect(existsSync(resolve(sourceRoot, "utils/plugin-loader.ts"))).toBe(
false,
);
const barrel = readFileSync(resolve(sourceRoot, "index.node.ts"), "utf8");
expect(barrel).not.toMatch(/plugin-loader/);
});
});
@@ -0,0 +1,258 @@
/**
* Coverage for the explicit component-precedence policy (#12658 / arch-audit
* #12089 item 8).
*
* The three primary component registries (actions, providers, evaluators)
* previously resolved same-name collisions with a SILENT first-wins dedupe:
* whichever plugin registered first won, later duplicates were dropped at
* `debug` with no observable signal, and there was no way to declare that a
* later component intentionally supersedes an earlier one.
*
* These tests pin the new contract:
* - undeclared collision -> keep the incumbent (still deterministic first-wins)
* AND emit an observable `logger.warn` instead of a silent `debug`.
* - `override: true` on the later registrant -> replace the incumbent and log
* the takeover at `info`.
* - the same policy applies whether components arrive via the direct
* `register*` methods or through `registerPlugin` (no silent pre-filter).
*
* Drives a real `AgentRuntime`; no model.
*/
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { describe, expect, it, vi } from "vitest";
import { AgentRuntime } from "../runtime";
import type {
Action,
ActionResult,
Character,
Provider,
RegisteredEvaluator,
} from "../types";
function makeRuntime(name: string): AgentRuntime {
return new AgentRuntime({ character: { name } as Character });
}
function makeProvider(name: string, text: string): Provider {
// Declare `contexts` so registration does not trip the PluginLifecycle
// "provider declares no contexts/contextGate" nudge (materializeProviderContexts,
// plugin-lifecycle.ts) — an orthogonal warn that would otherwise pollute these
// tests' collision-warning counts.
return {
name,
contexts: ["general"],
get: async () => ({ text, values: {}, data: {} }),
};
}
function makeAction(name: string, tag: string): Action {
return {
name,
description: tag,
validate: async () => true,
handler: async (): Promise<ActionResult> => ({
success: true,
text: tag,
}),
examples: [],
};
}
function makeEvaluator(name: string, description: string): RegisteredEvaluator {
return {
name,
description,
schema: { type: "object" },
shouldRun: async () => false,
prompt: () => "",
} as RegisteredEvaluator;
}
describe("AgentRuntime component precedence — undeclared collisions (first-wins + WARN)", () => {
it("keeps the first provider and WARNs on an undeclared name collision", async () => {
const runtime = makeRuntime("provider-collision-warn");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerProvider(makeProvider("DUP", "first"));
runtime.registerProvider(makeProvider("DUP", "second"));
const matches = runtime.providers.filter((p) => p.name === "DUP");
expect(matches).toHaveLength(1);
// Incumbent (first) is authoritative.
const dup = matches[0];
expect(dup).toBeDefined();
const result = await dup!.get(runtime, {} as never, {} as never);
expect(result.text).toBe("first");
expect(warn).toHaveBeenCalledTimes(1);
expect(warn.mock.calls[0]?.[1]).toMatch(/name collision/i);
});
it("keeps the first action and WARNs on an undeclared name collision", () => {
const runtime = makeRuntime("action-collision-warn");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerAction(makeAction("DUP_ACTION", "first"));
runtime.registerAction(makeAction("DUP_ACTION", "second"));
const matches = runtime.actions.filter((a) => a.name === "DUP_ACTION");
expect(matches).toHaveLength(1);
expect(matches[0]?.description).toBe("first");
expect(warn).toHaveBeenCalledTimes(1);
expect(warn.mock.calls[0]?.[1]).toMatch(/name collision/i);
});
it("keeps the first evaluator and WARNs on an undeclared name collision", () => {
const runtime = makeRuntime("evaluator-collision-warn");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerEvaluator(makeEvaluator("DUP_EVAL", "first"));
runtime.registerEvaluator(makeEvaluator("DUP_EVAL", "second"));
const matches = runtime.evaluators.filter((e) => e.name === "DUP_EVAL");
expect(matches).toHaveLength(1);
expect(matches[0]?.description).toBe("first");
expect(warn).toHaveBeenCalledTimes(1);
expect(warn.mock.calls[0]?.[1]).toMatch(/name collision/i);
});
});
describe("AgentRuntime component precedence — declared override:true supersedes + INFO", () => {
it("replaces the incumbent provider when the newcomer declares override:true", async () => {
const runtime = makeRuntime("provider-override");
const info = vi.spyOn(runtime.logger, "info");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerProvider(makeProvider("OVR", "first"));
runtime.registerProvider({
...makeProvider("OVR", "second"),
override: true,
});
const matches = runtime.providers.filter((p) => p.name === "OVR");
expect(matches).toHaveLength(1);
const ovr = matches[0];
expect(ovr).toBeDefined();
const result = await ovr!.get(runtime, {} as never, {} as never);
expect(result.text).toBe("second");
// Declared override logs at info, never warns.
expect(info).toHaveBeenCalled();
expect(warn).not.toHaveBeenCalled();
});
it("replaces the incumbent action when the newcomer declares override:true", () => {
const runtime = makeRuntime("action-override");
const info = vi.spyOn(runtime.logger, "info");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerAction(makeAction("OVR_ACTION", "first"));
runtime.registerAction({
...makeAction("OVR_ACTION", "second"),
override: true,
});
const matches = runtime.actions.filter((a) => a.name === "OVR_ACTION");
expect(matches).toHaveLength(1);
expect(matches[0]?.description).toBe("second");
expect(info).toHaveBeenCalled();
expect(warn).not.toHaveBeenCalled();
});
it("replaces the incumbent evaluator when the newcomer declares override:true", () => {
const runtime = makeRuntime("evaluator-override");
const info = vi.spyOn(runtime.logger, "info");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerEvaluator(makeEvaluator("OVR_EVAL", "first"));
runtime.registerEvaluator({
...makeEvaluator("OVR_EVAL", "second"),
override: true,
});
const matches = runtime.evaluators.filter((e) => e.name === "OVR_EVAL");
expect(matches).toHaveLength(1);
expect(matches[0]?.description).toBe("second");
expect(info).toHaveBeenCalled();
expect(warn).not.toHaveBeenCalled();
});
});
describe("AgentRuntime component precedence — policy applies through registerPlugin", () => {
it("routes plugin action collisions through the precedence policy (WARN, not silent)", async () => {
const runtime = makeRuntime("plugin-collision-warn");
runtime.registerAction(makeAction("PLUGIN_DUP", "incumbent"));
const warn = vi.spyOn(runtime.logger, "warn");
await runtime.registerPlugin({
name: "collision-plugin",
description: "collides with an already-registered action",
actions: [makeAction("PLUGIN_DUP", "from-plugin")],
});
const matches = runtime.actions.filter((a) => a.name === "PLUGIN_DUP");
expect(matches).toHaveLength(1);
expect(matches[0]?.description).toBe("incumbent");
expect(warn).toHaveBeenCalledTimes(1);
});
it("downgrades a plugin action's override:true to safe first-wins (WARNs)", async () => {
// Plugin-boundary overrides are unsafe for hot teardown (#12658): the
// runtime keeps the incumbent and treats it like an undeclared collision.
const runtime = makeRuntime("plugin-override-downgrade");
runtime.registerAction(makeAction("PLUGIN_OVR", "incumbent"));
const warn = vi.spyOn(runtime.logger, "warn");
await runtime.registerPlugin({
name: "override-plugin",
description: "attempts an intentional override across a plugin boundary",
actions: [{ ...makeAction("PLUGIN_OVR", "from-plugin"), override: true }],
});
const matches = runtime.actions.filter((a) => a.name === "PLUGIN_OVR");
expect(matches).toHaveLength(1);
expect(matches[0]?.description).toBe("incumbent");
expect(warn).toHaveBeenCalledTimes(1);
});
});
describe("AgentRuntime component precedence — grep guard (silent dedupe removed)", () => {
it("no longer silently pre-filters plugin action/provider/evaluator duplicates", () => {
const runtimeSrc = readFileSync(
fileURLToPath(new URL("../runtime.ts", import.meta.url)),
"utf8",
);
// The primary component registries used to drop same-name duplicates via a
// silent `debug` pre-filter in registerPlugin. That path is gone; collisions
// now flow through resolveComponentCollision (WARN / declared override).
expect(runtimeSrc).not.toContain('"Skipping duplicate plugin action"');
expect(runtimeSrc).not.toContain('"Skipping duplicate plugin provider"');
expect(runtimeSrc).not.toContain('"Skipping duplicate plugin evaluator"');
expect(runtimeSrc).not.toContain(
'"Evaluator already registered, skipping"',
);
// The single-authority collision resolver exists.
expect(runtimeSrc).toContain("resolveComponentCollision");
});
});
describe("AgentRuntime component precedence — distinct names still register", () => {
it("registers distinctly-named components without warning", () => {
const runtime = makeRuntime("distinct-names");
const warn = vi.spyOn(runtime.logger, "warn");
runtime.registerProvider(makeProvider("ALPHA", "a"));
runtime.registerProvider(makeProvider("BETA", "b"));
runtime.registerAction(makeAction("DO_ALPHA", "a"));
runtime.registerAction(makeAction("DO_BETA", "b"));
runtime.registerEvaluator(makeEvaluator("EVAL_ALPHA", "a"));
runtime.registerEvaluator(makeEvaluator("EVAL_BETA", "b"));
expect(runtime.providers.some((p) => p.name === "ALPHA")).toBe(true);
expect(runtime.providers.some((p) => p.name === "BETA")).toBe(true);
expect(runtime.actions.some((a) => a.name === "DO_ALPHA")).toBe(true);
expect(runtime.actions.some((a) => a.name === "DO_BETA")).toBe(true);
expect(runtime.evaluators.some((e) => e.name === "EVAL_ALPHA")).toBe(true);
expect(runtime.evaluators.some((e) => e.name === "EVAL_BETA")).toBe(true);
expect(warn).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,86 @@
/**
* Coverage for `AgentRuntime.registerProvider` — name-based deduplication and
* the `registerByDefault: false` opt-out across plugin and direct registration.
* Drives a real `AgentRuntime`; no model.
*/
import { describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import type { Character, Provider } from "../types";
function makeProvider(name: string, text: string): Provider {
return {
name,
get: async () => ({ text, values: {}, data: {} }),
};
}
describe("AgentRuntime.registerProvider deduplication", () => {
it("does not register two providers with the same name", () => {
const runtime = new AgentRuntime({
character: { name: "provider-dedupe-test" } as Character,
});
const before = runtime.providers.length;
runtime.registerProvider(makeProvider("DUP", "first"));
runtime.registerProvider(makeProvider("DUP", "second"));
const matches = runtime.providers.filter((p) => p.name === "DUP");
expect(matches).toHaveLength(1);
// First registration wins; the duplicate is skipped, not swapped in.
expect(runtime.providers.length).toBe(before + 1);
});
it("still registers distinctly-named providers", () => {
const runtime = new AgentRuntime({
character: { name: "provider-distinct-test" } as Character,
});
const before = runtime.providers.length;
runtime.registerProvider(makeProvider("ALPHA", "a"));
runtime.registerProvider(makeProvider("BETA", "b"));
expect(runtime.providers.length).toBe(before + 2);
expect(runtime.providers.some((p) => p.name === "ALPHA")).toBe(true);
expect(runtime.providers.some((p) => p.name === "BETA")).toBe(true);
});
it("skips plugin providers that opt out of default registration", async () => {
const runtime = new AgentRuntime({
character: { name: "provider-default-registration-test" } as Character,
});
await runtime.registerPlugin({
name: "provider-default-registration-plugin",
description: "Provider registration behavior test",
providers: [
makeProvider("DEFAULT_PROVIDER", "default"),
{
...makeProvider("EXPLICIT_PROVIDER", "explicit"),
registerByDefault: false,
},
],
});
expect(runtime.providers.some((p) => p.name === "DEFAULT_PROVIDER")).toBe(
true,
);
expect(runtime.providers.some((p) => p.name === "EXPLICIT_PROVIDER")).toBe(
false,
);
});
it("allows direct registration of providers that opt out of plugin defaults", () => {
const runtime = new AgentRuntime({
character: { name: "provider-direct-registration-test" } as Character,
});
runtime.registerProvider({
...makeProvider("DIRECT_EXPLICIT_PROVIDER", "explicit"),
registerByDefault: false,
});
expect(
runtime.providers.some((p) => p.name === "DIRECT_EXPLICIT_PROVIDER"),
).toBe(true);
});
});
@@ -0,0 +1,36 @@
/**
* Composes the runtime-focused regression suites for changes to the monolithic
* AgentRuntime module. The changed-file coverage gate runs only tests present in
* a PR diff, so this lane keeps those changes attached to the existing behavioral
* matrix until the runtime is fully decomposed into independently covered modules.
*/
import { describe, expect, it } from "vitest";
import "./compose-state-provider-hook.test";
import "./compose-state-refresh-providers.test";
import "./compose-state-role-gate.test";
import "./compose-state-trajectory-recording.test";
import "./dynamic-prompt-json-mode.test";
import "./message-runtime-stage1.test";
import "./outbound-sanitize-runtime-seams.test";
import "./runtime-component-precedence.test";
import "./runtime-register-provider.test";
import "./runtime-rerank-memories.test";
import "./runtime-settings.test";
import "./runtime-stop.test";
import "./streaming-runtime-hooks.test";
import "../runtime-get-all-memories.test";
import "../runtime-report-error.test";
import "../runtime/__tests__/embedding-dimension.test";
import "../runtime/__tests__/guarded-stream-use-model.test";
import "../runtime/__tests__/model-provider-failover.test";
import "../runtime/__tests__/model-registrations.test";
import "../runtime/__tests__/model-stream-chunk-hooks.test";
import "../runtime/__tests__/pii-swap-use-model.test";
import "../runtime/__tests__/secret-swap-use-model.test";
import "../runtime/__tests__/streaming-use-model.test";
describe("AgentRuntime regression lane", () => {
it("loads the runtime behavioral matrix", () => {
expect(true).toBe(true);
});
});
@@ -0,0 +1,37 @@
/**
* Coverage for `AgentRuntime.rerankMemories` — merging BM25 keyword ranking with
* zero-overlap vector hits so semantic-only and attachment-only memories survive
* reranking. Drives a real `AgentRuntime`; no model.
*/
import { describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import type { Character, Memory } from "../types";
function memory(id: string, text?: string): Memory {
return {
id: id as Memory["id"],
entityId: "entity-id" as Memory["entityId"],
roomId: "room-id" as Memory["roomId"],
content: text === undefined ? {} : { text },
} as Memory;
}
describe("AgentRuntime.rerankMemories", () => {
it("preserves zero-overlap vector hits after BM25-ranked matches", async () => {
const runtime = new AgentRuntime({
character: { name: "rerank-memory-test" } as Character,
});
const semanticOnly = memory("semantic-only", "I bought a new car");
const keywordMatch = memory("keyword-match", "automobile purchase receipt");
const attachmentOnly = memory("attachment-only");
const reranked = await runtime.rerankMemories("automobile purchase", [
semanticOnly,
keywordMatch,
attachmentOnly,
]);
expect(reranked).toEqual([keywordMatch, semanticOnly, attachmentOnly]);
});
});
@@ -0,0 +1,152 @@
/**
* Exercises `AgentRuntime.getSetting` resolution precedence (character env vs
* settings vs constructor settings vs DB-persisted values on restart) and
* prompt-batcher construction. Deterministic: real runtime over the in-memory
* adapter, no model calls.
*/
import { describe, expect, it } from "vitest";
import { InMemoryDatabaseAdapter } from "../database/inMemoryAdapter";
import { AgentRuntime } from "../runtime";
import type { Character } from "../types";
describe("AgentRuntime.getSetting", () => {
it("reads primitive character env values as runtime settings", () => {
const runtime = new AgentRuntime({
character: {
name: "env-settings-test",
env: {
FEATURE_FLAG: true,
TIMEOUT_MS: 5000,
vars: {
ROUTE_POLICY: '{"default":"guest"}',
},
},
settings: {
ROUTE_POLICY: '{"default":"owner"}',
},
} as Character,
});
expect(runtime.getSetting("FEATURE_FLAG")).toBe(true);
expect(runtime.getSetting("TIMEOUT_MS")).toBe(5000);
expect(runtime.getSetting("ROUTE_POLICY")).toBe('{"default":"owner"}');
});
it("reads primitive values from character env vars", () => {
const runtime = new AgentRuntime({
character: {
name: "env-vars-settings-test",
env: {
vars: {
ROUTE_POLICY: '{"default":"guest"}',
},
},
} as Character,
});
expect(runtime.getSetting("ROUTE_POLICY")).toBe('{"default":"guest"}');
});
it("falls back to env vars when direct env values are not primitive", () => {
const runtime = new AgentRuntime({
character: {
name: "env-vars-fallback-test",
env: {
ROUTE_POLICY: {
default: "owner",
},
vars: {
ROUTE_POLICY: '{"default":"guest"}',
},
},
} as Character,
});
expect(runtime.getSetting("ROUTE_POLICY")).toBe('{"default":"guest"}');
});
it("keeps character settings ahead of constructor settings", () => {
const runtime = new AgentRuntime({
character: {
name: "character-settings-override-test",
settings: {
ROUTE_POLICY: '{"default":"owner"}',
},
} as Character,
settings: {
ROUTE_POLICY: '{"default":"guest"}',
},
});
expect(runtime.getSetting("ROUTE_POLICY")).toBe('{"default":"owner"}');
});
it("uses fresh constructor settings over DB-persisted agent settings on restart", async () => {
const adapter = new InMemoryDatabaseAdapter();
const characterName = "runtime-settings-restart-test";
const firstRuntime = new AgentRuntime({
character: {
name: characterName,
bio: ["test"],
settings: {},
} as Character,
adapter,
settings: { FOO: "v1" },
logLevel: "fatal",
});
let secondRuntime: AgentRuntime | undefined;
try {
await firstRuntime.initialize({ skipMigrations: true });
expect(firstRuntime.getSetting("FOO")).toBe("v1");
await adapter.updateAgents([
{
agentId: firstRuntime.agentId,
agent: {
settings: { FOO: "v1", secrets: { BAR: "v1" } },
secrets: { BAZ: "v1" },
},
},
]);
secondRuntime = new AgentRuntime({
character: {
name: characterName,
bio: ["test"],
settings: {},
} as Character,
adapter,
settings: { FOO: "v2", BAR: "v2", BAZ: "v2" },
logLevel: "fatal",
});
await secondRuntime.initialize({ skipMigrations: true });
expect(secondRuntime.getSetting("FOO")).toBe("v2");
expect(secondRuntime.getSetting("BAR")).toBe("v2");
expect(secondRuntime.getSetting("BAZ")).toBe("v2");
} finally {
await firstRuntime.stop({ fast: true });
await secondRuntime?.stop({ fast: true });
firstRuntime.promptBatcher.dispose();
secondRuntime?.promptBatcher.dispose();
}
});
});
describe("AgentRuntime prompt batcher", () => {
it("creates a prompt batcher for production autonomy drains", () => {
const runtime = new AgentRuntime({
character: {
name: "prompt-batcher-runtime-test",
} as Character,
});
expect(runtime.promptBatcher).toBeDefined();
expect(runtime.promptBatcher.getStats()).toMatchObject({
totalDrains: 0,
totalCalls: 0,
});
runtime.promptBatcher.dispose();
});
});
@@ -0,0 +1,149 @@
/**
* Exercises `AgentRuntime.stop` fast-shutdown paths: not hanging on an
* unresolved service start, capping already-started stop waits, and surviving a
* synchronously-throwing stop. Deterministic: real runtime, no database.
*/
import { afterEach, describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import type { IAgentRuntime } from "../types/runtime";
import { Service } from "../types/service";
function createDeferred<T>() {
let resolve!: (value: T) => void;
let reject!: (reason?: unknown) => void;
const promise = new Promise<T>((res, rej) => {
resolve = res;
reject = rej;
});
return { promise, resolve, reject };
}
function delay(ms: number): Promise<"timeout"> {
return new Promise((resolve) => {
setTimeout(() => resolve("timeout"), ms);
});
}
describe("AgentRuntime.stop", () => {
const previousFastShutdown = process.env.ELIZA_FAST_SHUTDOWN;
const previousStopTimeout =
process.env.ELIZA_SHUTDOWN_SERVICE_STOP_TIMEOUT_MS;
afterEach(() => {
if (previousFastShutdown === undefined) {
delete process.env.ELIZA_FAST_SHUTDOWN;
} else {
process.env.ELIZA_FAST_SHUTDOWN = previousFastShutdown;
}
if (previousStopTimeout === undefined) {
delete process.env.ELIZA_SHUTDOWN_SERVICE_STOP_TIMEOUT_MS;
} else {
process.env.ELIZA_SHUTDOWN_SERVICE_STOP_TIMEOUT_MS = previousStopTimeout;
}
});
it("fast shutdown does not hang on an unresolved service start and cleans up late starts", async () => {
const runtime = new AgentRuntime({ logLevel: "fatal" });
await runtime.initialize({ allowNoDatabase: true, skipMigrations: true });
let startRuntime: IAgentRuntime | null = null;
let stopCalls = 0;
const start = createDeferred<SlowService>();
class SlowService extends Service {
static override serviceType = "shutdown-slow-service";
capabilityDescription = "slow service used by shutdown tests";
static override async start(
runtime: IAgentRuntime,
): Promise<SlowService> {
startRuntime = runtime;
return start.promise;
}
override async stop(): Promise<void> {
stopCalls += 1;
}
}
await runtime.registerService(SlowService);
const load = runtime.getServiceLoadPromise(SlowService.serviceType).then(
() => "loaded",
(error) => (error instanceof Error ? error.message : String(error)),
);
await Promise.resolve();
const stopResult = await Promise.race([
runtime.stop({ fast: true }).then(() => "stopped"),
delay(100),
]);
expect(stopResult).toBe("stopped");
expect(stopCalls).toBe(0);
expect(startRuntime).toBe(runtime);
start.resolve(new SlowService(runtime));
await expect(load).resolves.toContain("not found or failed to start");
expect(stopCalls).toBe(1);
expect(runtime.getServiceRegistrationStatus(SlowService.serviceType)).toBe(
"failed",
);
});
it("fast shutdown caps already-started service stop waits", async () => {
process.env.ELIZA_SHUTDOWN_SERVICE_STOP_TIMEOUT_MS = "5";
const runtime = new AgentRuntime({ logLevel: "fatal" });
await runtime.initialize({ allowNoDatabase: true, skipMigrations: true });
let stopCalls = 0;
class HangingStopService extends Service {
static override serviceType = "shutdown-hanging-stop-service";
capabilityDescription = "hanging stop service used by shutdown tests";
static override async start(): Promise<HangingStopService> {
return new HangingStopService();
}
override async stop(): Promise<void> {
stopCalls += 1;
await new Promise(() => {});
}
}
await runtime.registerService(HangingStopService);
await runtime.getServiceLoadPromise(HangingStopService.serviceType);
const stopResult = await Promise.race([
runtime.stop({ fast: true }).then(() => "stopped"),
delay(100),
]);
expect(stopResult).toBe("stopped");
expect(stopCalls).toBe(1);
expect(process.env.ELIZA_FAST_SHUTDOWN).toBe(previousFastShutdown);
});
it("continues when a service stop throws synchronously", async () => {
const runtime = new AgentRuntime({ logLevel: "fatal" });
await runtime.initialize({ allowNoDatabase: true, skipMigrations: true });
class ThrowingStopService extends Service {
static override serviceType = "shutdown-throwing-stop-service";
capabilityDescription = "throwing stop service used by shutdown tests";
static override async start(): Promise<ThrowingStopService> {
return new ThrowingStopService();
}
override stop(): Promise<void> {
throw new Error("sync stop failure");
}
}
await runtime.registerService(ThrowingStopService);
await runtime.getServiceLoadPromise(ThrowingStopService.serviceType);
await expect(runtime.stop()).resolves.toBeUndefined();
});
});
@@ -0,0 +1,643 @@
/**
* Guards serviceType uniqueness two ways: runtime warnings when duplicate
* serviceTypes register (real AgentRuntime) and a repo-wide TypeScript-AST scan
* of core, agent, and plugins that flags unallowlisted duplicate
* `static serviceType` declarations.
*/
import { readdirSync, statSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import ts from "typescript";
import { describe, expect, it, vi } from "vitest";
import { AgentRuntime } from "../runtime";
import type { IAgentRuntime } from "../types/runtime";
import { Service, ServiceType } from "../types/service";
type SourceInfo = {
filePath: string;
relativePath: string;
sourceFile: ts.SourceFile;
localConstants: Map<string, string>;
};
type ServiceClassRegistration = {
className: string;
filePath: string;
relativePath: string;
serviceType: string;
};
type AllowlistEntry = {
reason: string;
classes: Set<string>;
};
const currentDir = path.dirname(fileURLToPath(import.meta.url));
const repoRoot = path.resolve(currentDir, "../../../..");
const scanRoots = ["packages/core/src", "packages/agent/src", "plugins"];
const sourceExtensions = new Set([".ts", ".tsx", ".mts", ".cts"]);
const ignoredDirectoryNames = new Set([
".turbo",
"build",
"coverage",
"dist",
"e2e",
"generated",
"node_modules",
"test",
"test-results",
"tests",
"vendor",
"__tests__",
]);
const ignoredFileFragments = [".d.ts", ".test.", ".spec."];
const serviceTypeValuesByMember = new Map(
Object.entries(ServiceType).map(([key, value]) => [key, String(value)]),
);
const duplicateServiceTypeAllowlist = new Map<string, AllowlistEntry>([
[
"capability-router",
{
reason:
"Runtime capability routing keeps local, remote, and E2B implementations in one capability-router discovery slot while the strategy-table migration is in progress.",
classes: new Set([
"packages/core/src/services/runtime-capability-service.ts:RuntimeCapabilityService",
"packages/agent/src/services/e2b-capability-router.ts:E2BRemoteCapabilityRouterService",
"packages/agent/src/services/remote-capability-router.ts:RemoteCapabilityRouterService",
]),
},
],
[
"trajectories",
{
reason:
"Core and agent intentionally register trajectory services side by side; callers resolve the full implementation with getServicesByType().",
classes: new Set([
"packages/core/src/features/trajectories/TrajectoriesService.ts:TrajectoriesService",
"packages/agent/src/runtime/trajectory-storage.ts:DatabaseTrajectoryLogger",
]),
},
],
[
"capability-router",
{
reason:
"Capability routing is mid-migration: the core canonical service and the legacy agent remote/E2B routers share the slot until callers finish moving to RuntimeCapabilityService.",
classes: new Set([
"packages/core/src/services/runtime-capability-service.ts:RuntimeCapabilityService",
"packages/agent/src/services/e2b-capability-router.ts:E2BRemoteCapabilityRouterService",
"packages/agent/src/services/remote-capability-router.ts:RemoteCapabilityRouterService",
]),
},
],
[
"discord-local",
{
reason:
"Discord local exists in the bundled Discord plugin and the standalone local plugin during migration; both export the same plugin/service contract and must not be enabled together.",
classes: new Set([
"plugins/plugin-discord/discord-local-service.ts:DiscordLocalService",
"plugins/plugin-discord-local/src/index.ts:DiscordLocalService",
]),
},
],
[
"tunnel",
{
reason:
"Tunnel providers intentionally share the backend-agnostic tunnel slot; plugin init selects a single active implementation.",
classes: new Set([
"plugins/plugin-ngrok/src/services/NgrokService.ts:NgrokService",
"plugins/plugin-tailscale/src/services/CloudTailscaleService.ts:CloudTailscaleService",
"plugins/plugin-tailscale/src/services/LocalTailscaleService.ts:LocalTailscaleService",
"plugins/plugin-tunnel/src/services/LocalTunnelService.ts:LocalTunnelService",
]),
},
],
[
"capability-router",
{
reason:
"RuntimeCapabilityService is the canonical slot while existing HTTP and sandbox router services remain constructible during the router-strategy migration.",
classes: new Set([
"packages/core/src/services/runtime-capability-service.ts:RuntimeCapabilityService",
"packages/agent/src/services/e2b-capability-router.ts:E2BRemoteCapabilityRouterService",
"packages/agent/src/services/remote-capability-router.ts:RemoteCapabilityRouterService",
]),
},
],
[
"workflow_credential_provider",
{
reason:
"Workflow credential providers share one discovery slot so workflow nodes can ask every connector for credentials.",
classes: new Set([
"plugins/plugin-bluebubbles/src/workflow-credential-provider.ts:BlueBubblesWorkflowCredentialProvider",
"plugins/plugin-bluesky/workflow-credential-provider.ts:BlueskyWorkflowCredentialProvider",
"plugins/plugin-elizacloud/src/services/cloud-credential-provider.ts:CloudCredentialProvider",
"plugins/plugin-farcaster/workflow-credential-provider.ts:FarcasterWorkflowCredentialProvider",
"plugins/plugin-feishu/src/workflow-credential-provider.ts:FeishuWorkflowCredentialProvider",
"plugins/plugin-google-chat/src/workflow-credential-provider.ts:GoogleChatWorkflowCredentialProvider",
"plugins/plugin-instagram/src/workflow-credential-provider.ts:InstagramWorkflowCredentialProvider",
"plugins/plugin-line/src/workflow-credential-provider.ts:LineWorkflowCredentialProvider",
"plugins/plugin-matrix/src/workflow-credential-provider.ts:MatrixWorkflowCredentialProvider",
"plugins/plugin-signal/src/workflow-credential-provider.ts:SignalWorkflowCredentialProvider",
"plugins/plugin-slack/src/workflow-credential-provider.ts:SlackWorkflowCredentialProvider",
"plugins/plugin-twitch/src/workflow-credential-provider.ts:TwitchWorkflowCredentialProvider",
"plugins/plugin-whatsapp/src/workflow-credential-provider.ts:WhatsAppWorkflowCredentialProvider",
"plugins/plugin-x/src/workflow-credential-provider.ts:XWorkflowCredentialProvider",
]),
},
],
]);
let cachedServiceClassRegistrations: ServiceClassRegistration[] | null = null;
class CollisionTestServiceA extends Service {
static override serviceType = "collision-test";
capabilityDescription = "collision test service A";
static override async start(
runtime: IAgentRuntime,
): Promise<CollisionTestServiceA> {
return new CollisionTestServiceA(runtime);
}
async stop(): Promise<void> {}
}
class CollisionTestServiceB extends Service {
static override serviceType = "collision-test";
capabilityDescription = "collision test service B";
static override async start(
runtime: IAgentRuntime,
): Promise<CollisionTestServiceB> {
return new CollisionTestServiceB(runtime);
}
async stop(): Promise<void> {}
}
class MultiTestServiceA extends Service {
static override serviceType = "multi-test";
static override allowsMultiple = true;
capabilityDescription = "multi test service A";
static override async start(
runtime: IAgentRuntime,
): Promise<MultiTestServiceA> {
return new MultiTestServiceA(runtime);
}
async stop(): Promise<void> {}
}
class MultiTestServiceB extends Service {
static override serviceType = "multi-test";
static override allowsMultiple = true;
capabilityDescription = "multi test service B";
static override async start(
runtime: IAgentRuntime,
): Promise<MultiTestServiceB> {
return new MultiTestServiceB(runtime);
}
async stop(): Promise<void> {}
}
class WebSearchTestService extends Service {
static override serviceType = ServiceType.WEB_SEARCH;
capabilityDescription = "web search test service";
static override async start(
runtime: IAgentRuntime,
): Promise<WebSearchTestService> {
return new WebSearchTestService(runtime);
}
async stop(): Promise<void> {}
}
function toRelativePath(filePath: string): string {
return path.relative(repoRoot, filePath).split(path.sep).join("/");
}
function shouldIgnoreFile(filePath: string): boolean {
const baseName = path.basename(filePath);
return ignoredFileFragments.some((fragment) => baseName.includes(fragment));
}
function collectSourceFiles(root: string): string[] {
const absoluteRoot = path.join(repoRoot, root);
const files: string[] = [];
function walk(directory: string): void {
for (const entry of readdirSync(directory)) {
const fullPath = path.join(directory, entry);
const stat = statSync(fullPath);
if (stat.isDirectory()) {
if (!ignoredDirectoryNames.has(entry)) {
walk(fullPath);
}
continue;
}
if (
stat.isFile() &&
sourceExtensions.has(path.extname(fullPath)) &&
!shouldIgnoreFile(fullPath)
) {
files.push(fullPath);
}
}
}
walk(absoluteRoot);
return files;
}
function unwrapExpression(expression: ts.Expression): ts.Expression {
let current = expression;
while (
ts.isAsExpression(current) ||
ts.isSatisfiesExpression(current) ||
ts.isTypeAssertionExpression(current) ||
ts.isNonNullExpression(current)
) {
current = current.expression;
}
return current;
}
function literalStringValue(expression: ts.Expression): string | null {
const unwrapped = unwrapExpression(expression);
if (
ts.isStringLiteral(unwrapped) ||
ts.isNoSubstitutionTemplateLiteral(unwrapped)
) {
return unwrapped.text;
}
return null;
}
function collectLocalConstants(sourceFile: ts.SourceFile): Map<string, string> {
const constants = new Map<string, string>();
function visit(node: ts.Node): void {
if (
ts.isVariableStatement(node) &&
(node.declarationList.flags & ts.NodeFlags.Const) !== 0
) {
for (const declaration of node.declarationList.declarations) {
if (!ts.isIdentifier(declaration.name) || !declaration.initializer) {
continue;
}
const value = literalStringValue(declaration.initializer);
if (value) {
constants.set(declaration.name.text, value);
}
}
}
ts.forEachChild(node, visit);
}
visit(sourceFile);
return constants;
}
function getUniqueGlobalConstants(sources: SourceInfo[]): Map<string, string> {
const valuesByName = new Map<string, Set<string>>();
for (const source of sources) {
for (const [name, value] of source.localConstants) {
const values = valuesByName.get(name) ?? new Set<string>();
values.add(value);
valuesByName.set(name, values);
}
}
const unique = new Map<string, string>();
for (const [name, values] of valuesByName) {
if (values.size === 1) {
unique.set(name, [...values][0]);
}
}
return unique;
}
function resolveServiceTypeInitializer(
expression: ts.Expression,
localConstants: Map<string, string>,
globalConstants: Map<string, string>,
): string | null {
const unwrapped = unwrapExpression(expression);
const literal = literalStringValue(unwrapped);
if (literal) {
return literal;
}
if (ts.isPropertyAccessExpression(unwrapped)) {
const objectName = unwrapped.expression.getText();
if (objectName === "ServiceType") {
return serviceTypeValuesByMember.get(unwrapped.name.text) ?? null;
}
}
if (ts.isIdentifier(unwrapped)) {
return (
localConstants.get(unwrapped.text) ??
globalConstants.get(unwrapped.text) ??
null
);
}
return null;
}
function hasModifier(node: ts.Node, kind: ts.SyntaxKind): boolean {
return Boolean(
ts.canHaveModifiers(node) &&
ts.getModifiers(node)?.some((modifier) => modifier.kind === kind),
);
}
function isStaticMember(node: ts.ClassElement): boolean {
return hasModifier(node, ts.SyntaxKind.StaticKeyword);
}
function hasStaticStart(classNode: ts.ClassDeclaration): boolean {
return classNode.members.some(
(member) =>
ts.isMethodDeclaration(member) &&
isStaticMember(member) &&
ts.isIdentifier(member.name) &&
member.name.text === "start",
);
}
function extendsService(classNode: ts.ClassDeclaration): boolean {
return Boolean(
classNode.heritageClauses?.some((clause) => {
if (clause.token !== ts.SyntaxKind.ExtendsKeyword) {
return false;
}
return clause.types.some((type) => {
const baseName = type.expression.getText();
return baseName === "Service" || baseName.endsWith(".Service");
});
}),
);
}
function getStaticServiceTypeInitializer(
classNode: ts.ClassDeclaration,
): ts.Expression | null {
for (const member of classNode.members) {
if (
ts.isPropertyDeclaration(member) &&
isStaticMember(member) &&
ts.isIdentifier(member.name) &&
member.name.text === "serviceType" &&
member.initializer
) {
return member.initializer;
}
}
return null;
}
function collectServiceClassRegistrations(): ServiceClassRegistration[] {
if (cachedServiceClassRegistrations) {
return cachedServiceClassRegistrations;
}
const files = scanRoots.flatMap(collectSourceFiles);
const sources: SourceInfo[] = files.map((filePath) => {
const sourceText = ts.sys.readFile(filePath) ?? "";
const sourceFile = ts.createSourceFile(
filePath,
sourceText,
ts.ScriptTarget.Latest,
true,
);
return {
filePath,
relativePath: toRelativePath(filePath),
sourceFile,
localConstants: collectLocalConstants(sourceFile),
};
});
const globalConstants = getUniqueGlobalConstants(sources);
const registrations: ServiceClassRegistration[] = [];
for (const source of sources) {
function visit(node: ts.Node): void {
if (
ts.isClassDeclaration(node) &&
node.name &&
!hasModifier(node, ts.SyntaxKind.AbstractKeyword) &&
extendsService(node) &&
hasStaticStart(node)
) {
const initializer = getStaticServiceTypeInitializer(node);
const serviceType = initializer
? resolveServiceTypeInitializer(
initializer,
source.localConstants,
globalConstants,
)
: null;
if (serviceType) {
registrations.push({
className: node.name.text,
filePath: source.filePath,
relativePath: source.relativePath,
serviceType,
});
}
}
ts.forEachChild(node, visit);
}
visit(source.sourceFile);
}
cachedServiceClassRegistrations = registrations;
return registrations;
}
function classId(registration: ServiceClassRegistration): string {
return `${registration.relativePath}:${registration.className}`;
}
function groupByServiceType(
registrations: ServiceClassRegistration[],
): Map<string, ServiceClassRegistration[]> {
const groups = new Map<string, ServiceClassRegistration[]>();
for (const registration of registrations) {
const group = groups.get(registration.serviceType) ?? [];
group.push(registration);
groups.set(registration.serviceType, group);
}
return groups;
}
function isAllowlistedDuplicate(
serviceType: string,
group: ServiceClassRegistration[],
): boolean {
const allowlist = duplicateServiceTypeAllowlist.get(serviceType);
if (!allowlist) {
return false;
}
const actual = new Set(group.map(classId));
if (actual.size !== allowlist.classes.size) {
return false;
}
return [...actual].every((id) => allowlist.classes.has(id));
}
function formatDuplicateGroup(
serviceType: string,
group: ServiceClassRegistration[],
): string {
const classes = group
.map((registration) => ` - ${classId(registration)}`)
.join("\n");
return `${serviceType}\n${classes}`;
}
function expectServiceType(
registrations: ServiceClassRegistration[],
classIdentifier: string,
serviceType: string,
): void {
const registration = registrations.find(
(candidate) => classId(candidate) === classIdentifier,
);
expect(registration?.serviceType).toBe(serviceType);
}
describe("serviceType collision guardrails", () => {
it("warns when dynamic service registration makes getService ambiguous", async () => {
const runtime = new AgentRuntime({ logLevel: "fatal" });
const warnSpy = vi
.spyOn(runtime.logger, "warn")
.mockImplementation(() => undefined);
await runtime.registerService(CollisionTestServiceA);
await runtime.registerService(CollisionTestServiceB);
expect(warnSpy).toHaveBeenCalledWith(
expect.objectContaining({
serviceType: "collision-test",
serviceClass: "CollisionTestServiceB",
}),
expect.stringContaining("Duplicate serviceType registration"),
);
warnSpy.mockRestore();
});
it("warns when plugin service declarations reuse a serviceType", async () => {
const runtime = new AgentRuntime({ logLevel: "fatal" });
const warnSpy = vi
.spyOn(runtime.logger, "warn")
.mockImplementation(() => undefined);
await runtime.registerPlugin({
name: "collision-test-plugin",
description: "Plugin with duplicate service types",
services: [CollisionTestServiceA, CollisionTestServiceB],
});
expect(warnSpy).toHaveBeenCalledWith(
expect.objectContaining({
plugin: "collision-test-plugin",
serviceType: "collision-test",
serviceClass: "CollisionTestServiceB",
}),
expect.stringContaining("Duplicate serviceType registration"),
);
warnSpy.mockRestore();
});
it("does not warn when service classes declare allowsMultiple", async () => {
const runtime = new AgentRuntime({ logLevel: "fatal" });
const warnSpy = vi
.spyOn(runtime.logger, "warn")
.mockImplementation(() => undefined);
await runtime.registerService(MultiTestServiceA);
await runtime.registerService(MultiTestServiceB);
expect(warnSpy).not.toHaveBeenCalledWith(
expect.objectContaining({
serviceType: "multi-test",
}),
expect.stringContaining("Duplicate serviceType registration"),
);
warnSpy.mockRestore();
});
it("does not fabricate web search metadata during generic service registration", async () => {
const runtime = new AgentRuntime({ logLevel: "fatal" });
await runtime.registerService(WebSearchTestService);
expect(runtime.hasService(ServiceType.WEB_SEARCH)).toBe(true);
expect(() => runtime.getSearchCategory("web")).toThrow(
"No search category registered",
);
});
it("keeps service class serviceType values unique unless explicitly allowlisted", () => {
const registrations = collectServiceClassRegistrations();
const groups = groupByServiceType(registrations);
const unexpectedDuplicateGroups = [...groups.entries()]
.filter(([, group]) => group.length > 1)
.filter(
([serviceType, group]) => !isAllowlistedDuplicate(serviceType, group),
)
.map(([serviceType, group]) => formatDuplicateGroup(serviceType, group));
expect(unexpectedDuplicateGroups).toEqual([]);
}, 180_000);
it("keeps known collision fixes in place", () => {
const registrations = collectServiceClassRegistrations();
const groups = groupByServiceType(registrations);
expect(groups.get("FORM")?.map(classId)).toEqual([
"plugins/plugin-form/src/service.ts:FormService",
]);
expectServiceType(
registrations,
"plugins/plugin-tailscale/src/services/LocalTailscaleService.ts:LocalTailscaleService",
"tunnel",
);
expectServiceType(
registrations,
"plugins/plugin-tailscale/src/services/CloudTailscaleService.ts:CloudTailscaleService",
"tunnel",
);
expectServiceType(
registrations,
"packages/core/src/features/trust/services/TrustEngine.ts:TrustEngine",
"trust-engine:core",
);
expectServiceType(
registrations,
"packages/core/src/features/trust/services/wrappers.ts:TrustEngineServiceWrapper",
"trust-engine",
);
expectServiceType(
registrations,
"packages/core/src/features/trust/services/CredentialProtector.ts:CredentialProtector",
"credential-protector:core",
);
expectServiceType(
registrations,
"packages/core/src/features/trust/services/wrappers.ts:CredentialProtectorServiceWrapper",
"credential-protector",
);
});
});
@@ -0,0 +1,220 @@
/**
* Validates the shouldRespond classifier prompt against a live Ollama model
* (skipped unless ELIZA_RUN_LIVE_TESTS=1): reply/ignore/stop decisions over real
* TEXT_LARGE completions through a real AgentRuntime.
*/
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { InMemoryDatabaseAdapter } from "../database/inMemoryAdapter";
import { shouldRespondTemplate } from "../prompts";
import { AgentRuntime } from "../runtime";
import {
createOllamaModelHandlers,
isOllamaAvailable,
} from "../testing/ollama-provider";
import type { Character, State } from "../types";
import { ModelType } from "../types/model";
const runLiveTests = process.env.ELIZA_RUN_LIVE_TESTS === "1";
const liveDescribe = runLiveTests ? describe : describe.skip;
const LIVE_TEST_TIMEOUT_MS = 30_000;
function buildShouldRespondState(recentMessages: string): State {
return {
values: {
agentName: "Eliza",
providers: [
"# Character",
"Eliza is a direct, technically capable assistant for software work.",
"",
"# Recent Messages",
recentMessages,
].join("\n"),
availableContexts: "general\ncode\nwallet",
},
data: {},
text: recentMessages,
};
}
function getClassifierSchema() {
return [
{
field: "name",
description: "The name of the agent responding",
validateField: false,
streamField: false,
},
{
field: "reasoning",
description: "Your reasoning for this decision",
validateField: false,
streamField: false,
},
{
field: "action",
description:
"REPLY | RESPOND | IGNORE | STOP (REPLY and RESPOND both mean engage)",
validateField: false,
streamField: false,
},
{
field: "primaryContext",
description: "Primary domain context from available_contexts",
validateField: false,
streamField: false,
},
{
field: "secondaryContexts",
description: "Optional comma-separated additional domain contexts",
validateField: false,
streamField: false,
},
] as const;
}
liveDescribe("shouldRespond live", () => {
let runtime: AgentRuntime;
let adapter: InMemoryDatabaseAdapter;
beforeAll(async () => {
if (!(await isOllamaAvailable())) {
throw new Error(
"Ollama is required for shouldRespond live tests when ELIZA_RUN_LIVE_TESTS=1",
);
}
const character: Character = {
name: "Eliza",
system: "You are a precise assistant used for live prompt validation.",
bio: ["Precise assistant used for live prompt validation."],
templates: {},
messageExamples: [],
postExamples: [],
topics: ["testing", "classification"],
adjectives: ["precise"],
knowledge: [],
plugins: [],
secrets: {},
settings: {},
};
adapter = new InMemoryDatabaseAdapter();
runtime = new AgentRuntime({
character,
adapter,
logLevel: "warn",
settings: {
VALIDATION_LEVEL: "trusted",
},
});
await adapter.init();
for (const [modelType, handler] of Object.entries(
createOllamaModelHandlers(),
)) {
if (handler) {
runtime.registerModel(modelType, handler, "ollama");
}
}
});
afterAll(async () => {
await runtime.stop();
await adapter.close();
});
async function classify(recentMessages: string) {
return runtime.dynamicPromptExecFromState({
state: buildShouldRespondState(recentMessages),
params: {
prompt: shouldRespondTemplate,
temperature: 0,
maxTokens: 200,
},
schema: getClassifierSchema(),
options: {
modelType: ModelType.TEXT_LARGE,
contextCheckLevel: 0,
maxRetries: 1,
},
});
}
it(
"replies to an obvious direct question",
async () => {
const result = await classify(
[
"user-1: morning everyone",
"user-1: Eliza, can you help me debug this TypeScript type error?",
].join("\n"),
);
expect(result).not.toBeNull();
const action = String(result?.action ?? "")
.trim()
.toUpperCase();
expect(["REPLY", "RESPOND"]).toContain(action);
},
LIVE_TEST_TIMEOUT_MS,
);
it(
"ignores a side-thread addressed to someone else",
async () => {
const result = await classify(
[
"user-1: @bob can you merge the release branch after lunch?",
"user-2: sure, I will handle it",
].join("\n"),
);
expect(result).not.toBeNull();
const action = String(result?.action ?? "")
.trim()
.toUpperCase();
expect(action).toBe("IGNORE");
},
LIVE_TEST_TIMEOUT_MS,
);
it(
"ignores a group request addressed to another bot",
async () => {
const result = await classify(
[
"fishai: @botdick make a github issue in elizaOS/eliza saying test botdick github auth",
].join("\n"),
);
expect(result).not.toBeNull();
const action = String(result?.action ?? "")
.trim()
.toUpperCase();
expect(action).toBe("IGNORE");
},
LIVE_TEST_TIMEOUT_MS,
);
it(
"stops when explicitly told to stop",
async () => {
const result = await classify(
[
"user-1: Eliza stop. This is a direct instruction for you to end the run and stop talking immediately.",
].join("\n"),
);
expect(result).not.toBeNull();
expect(
String(result?.action ?? "")
.trim()
.toUpperCase(),
).toBe("STOP");
},
LIVE_TEST_TIMEOUT_MS,
);
});
@@ -0,0 +1,154 @@
/**
* Proves the Stage-1 provider exclusions are EXECUTION exclusions, not just
* render exclusions: `stage1ResponseStateProviderNames` subtracts the
* stage-1-excluded providers from the compose include list (so ENTITIES /
* CURRENT_TIME never run for a turn that dies at Stage 1), while the planner
* pass still composes them via composeState's cached-state merge. Uses a real
* in-memory AgentRuntime with call-counting providers; no database or model.
*/
import { describe, expect, it } from "vitest";
import { AgentRuntime } from "../runtime";
import { stage1ResponseStateProviderNames } from "../services/message";
import type {
Character,
IAgentRuntime,
Memory,
Provider,
UUID,
} from "../types";
const ROOM_ID = "11111111-1111-1111-1111-111111111111" as UUID;
const ENTITY_ID = "22222222-2222-2222-2222-222222222222" as UUID;
function makeMessage(id: string, text = "gm"): Memory {
return {
id: id as UUID,
entityId: ENTITY_ID,
roomId: ROOM_ID,
content: { text },
};
}
/** Provider whose text changes on every run, so reuse vs re-run is observable. */
function countingProvider(name: string): {
provider: Provider;
calls: () => number;
} {
let n = 0;
return {
provider: {
name,
get: async () => {
n += 1;
return { text: `${name}#${n}`, values: {}, data: {} };
},
},
calls: () => n,
};
}
describe("stage1ResponseStateProviderNames", () => {
it("subtracts the stage-1 exclusions from the compose include list", () => {
const runtime = { providers: [] } as unknown as IAgentRuntime;
const names = stage1ResponseStateProviderNames(
runtime,
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1"),
);
expect(names).not.toContain("ENTITIES");
expect(names).not.toContain("CURRENT_TIME");
expect(names).not.toContain("DOCUMENTS");
// Stage 1 still composes what it renders and routes on.
expect(names).toContain("RECENT_MESSAGES");
expect(names).toContain("FACTS");
expect(names).toContain("ATTACHMENTS");
});
it("keeps CURRENT_TIME when the user is asking about the time", () => {
const runtime = { providers: [] } as unknown as IAgentRuntime;
const names = stage1ResponseStateProviderNames(
runtime,
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa2", "what time is it?"),
);
expect(names).toContain("CURRENT_TIME");
expect(names).not.toContain("ENTITIES");
});
it("includes always-on plugin providers unless they are stage-1-excluded", () => {
const runtime = {
providers: [
{
name: "PLUGIN_CTX",
alwaysInResponseState: true,
get: async () => ({}),
},
{
name: "DOCUMENTS",
alwaysInResponseState: true,
get: async () => ({}),
},
],
} as unknown as IAgentRuntime;
const names = stage1ResponseStateProviderNames(
runtime,
makeMessage("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa3"),
);
expect(names).toContain("PLUGIN_CTX");
expect(names).not.toContain("DOCUMENTS");
});
it("skips excluded providers at stage 1 and defers them to the planner recompose", async () => {
const runtime = new AgentRuntime({
character: { name: "stage1-exec-test" } as Character,
});
const entities = countingProvider("ENTITIES");
const currentTime = countingProvider("CURRENT_TIME");
const facts = countingProvider("FACTS");
const recent = countingProvider("RECENT_MESSAGES");
for (const p of [entities, currentTime, facts, recent]) {
runtime.registerProvider(p.provider);
}
const message = makeMessage("bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb");
const stage1Names = stage1ResponseStateProviderNames(runtime, message);
// Stage-1 compose: excluded providers never execute, so a turn that
// ends at Stage 1 (group noise → IGNORE) does none of their work and
// its prompt footprint drops accordingly.
const stage1State = await runtime.composeState(
message,
stage1Names,
true,
false,
);
expect(entities.calls()).toBe(0);
expect(currentTime.calls()).toBe(0);
expect(facts.calls()).toBe(1);
expect(recent.calls()).toBe(1);
expect(stage1State.text).not.toContain("ENTITIES#");
expect(stage1State.text).not.toContain("CURRENT_TIME#");
expect(stage1State.text).toContain("FACTS#1");
// Planner recompose (mirrors selectV5PlannerStateProviderNames re-adding
// the core response providers, with RECENT_MESSAGES refreshed): the
// excluded providers are not in the turn cache, so they run now — once —
// and reach the planner prompt; the already-composed FACTS is reused.
const plannerState = await runtime.composeState(
message,
[...stage1Names, "ENTITIES", "CURRENT_TIME"],
true,
false,
["RECENT_MESSAGES"],
);
expect(entities.calls()).toBe(1);
expect(currentTime.calls()).toBe(1);
expect(facts.calls()).toBe(1);
expect(recent.calls()).toBe(2);
expect(plannerState.text).toContain("ENTITIES#1");
expect(plannerState.text).toContain("CURRENT_TIME#1");
expect(plannerState.text).toContain("FACTS#1");
expect(plannerState.text).toContain("RECENT_MESSAGES#2");
});
});
@@ -0,0 +1,125 @@
/**
* Covers `emitStreamingHook` on the streaming context: no-op when a hook is
* absent, payload forwarding for tool/result/evaluation/context-event
* observers, and caller isolation from a throwing hook. Deterministic vitest
* spies, no runtime.
*/
import { describe, expect, it, vi } from "vitest";
import { emitStreamingHook, type StreamingContext } from "../streaming-context";
import type { EvaluationResult } from "../types/components";
import type { ContextEvent } from "../types/context-object";
import type { ToolCall } from "../types/model";
describe("streaming context event hooks", () => {
it("no-ops when optional hooks are absent", async () => {
const context: StreamingContext = {
onStreamChunk: vi.fn(),
};
const toolCall: ToolCall = {
id: "call-1",
name: "LOOKUP",
arguments: { query: "status" },
};
await expect(
emitStreamingHook(context, "onToolCall", { toolCall }),
).resolves.toBeUndefined();
await expect(
emitStreamingHook(undefined, "onToolCall", { toolCall }),
).resolves.toBeUndefined();
});
it("forwards tool, result, evaluation, and context event payloads", async () => {
const onToolCall = vi.fn();
const onToolResult = vi.fn();
const onEvaluation = vi.fn();
const onContextEvent = vi.fn();
const context: StreamingContext = {
onStreamChunk: vi.fn(),
onToolCall,
onToolResult,
onEvaluation,
onContextEvent,
messageId: "message-1",
};
const contextEvent: ContextEvent = {
id: "event-1",
type: "tool",
tool: {
id: "tool-1",
name: "LOOKUP",
metadata: { query: "status" },
},
};
const toolCall: ToolCall = {
id: "call-1",
name: "LOOKUP",
arguments: { query: "status" },
status: "pending",
};
const evaluation: EvaluationResult = {
success: true,
decision: "FINISH",
thought: "Done.",
messageToUser: "Done.",
};
await emitStreamingHook(context, "onToolCall", {
toolCall,
contextEvent,
messageId: "message-1",
});
await emitStreamingHook(context, "onToolResult", {
toolCall: { ...toolCall, status: "completed", result: "ok" },
toolCallId: "call-1",
result: "ok",
status: "completed",
contextEvent,
messageId: "message-1",
});
await emitStreamingHook(context, "onEvaluation", {
evaluation,
contextEvent,
messageId: "message-1",
});
await emitStreamingHook(context, "onContextEvent", contextEvent);
expect(onToolCall).toHaveBeenCalledWith({
toolCall,
contextEvent,
messageId: "message-1",
});
expect(onToolResult).toHaveBeenCalledWith({
toolCall: { ...toolCall, status: "completed", result: "ok" },
toolCallId: "call-1",
result: "ok",
status: "completed",
contextEvent,
messageId: "message-1",
});
expect(onEvaluation).toHaveBeenCalledWith({
evaluation,
contextEvent,
messageId: "message-1",
});
expect(onContextEvent).toHaveBeenCalledWith(contextEvent);
});
it("isolates hook failures from callers", async () => {
const context: StreamingContext = {
onStreamChunk: vi.fn(),
onToolCall: vi.fn(async () => {
throw new Error("observer failed");
}),
};
const toolCall: ToolCall = {
id: "call-1",
name: "LOOKUP",
arguments: {},
};
await expect(
emitStreamingHook(context, "onToolCall", { toolCall }),
).resolves.toBeUndefined();
});
});
@@ -0,0 +1,319 @@
/**
* Covers the streaming observer hooks fired through the planner loop and
* sub-planner (tool call, tool result, evaluation, context event): execution
* ordering, caller isolation from throwing hooks, failed-tool-result emission,
* and sub-planner context events. Deterministic: a stub runtime with queued
* model responses, no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { executePlannedToolCall } from "../runtime/execute-planned-tool-call";
import {
actionResultToPlannerToolResult,
runPlannerLoop,
} from "../runtime/planner-loop";
import { runSubPlanner } from "../runtime/sub-planner";
import { runWithStreamingContext } from "../streaming-context";
import type { Action, IAgentRuntime, Memory } from "../types";
import { ModelType } from "../types/model";
function makeAction(overrides: Partial<Action>): Action {
return {
name: "LOOKUP",
description: "Look up status",
validate: async () => true,
handler: async () => ({ success: true }),
...overrides,
};
}
function makeMessage(): Memory {
return {
id: "message-id",
entityId: "entity-id",
roomId: "room-id",
content: { text: "check status" },
} as Memory;
}
function makeRuntime(responses: unknown[], actions: Action[]): IAgentRuntime {
const queue = [...responses];
return {
actions,
useModel: vi.fn(async () => {
if (queue.length === 0) {
throw new Error("Unexpected useModel call");
}
return queue.shift();
}),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime;
}
describe("v5 runtime streaming hooks", () => {
it("fires planner, tool result, and evaluator hooks in execution order", async () => {
const order: string[] = [];
const action = makeAction({
name: "LOOKUP",
parameters: [
{
name: "query",
description: "Lookup query",
required: true,
schema: { type: "string" },
},
],
handler: vi.fn(async () => ({
success: true,
text: "all good",
data: { answer: "ok" },
})),
});
const runtime = makeRuntime(
[
{
text: "",
toolCalls: [
{
id: "call-1",
name: "LOOKUP",
arguments: { query: "status" },
},
],
},
JSON.stringify({
success: true,
decision: "FINISH",
thought: "Done.",
messageToUser: "Done.",
}),
],
[action],
);
const onToolCall = vi.fn(() => order.push("toolCall"));
const onToolResult = vi.fn(() => order.push("toolResult"));
const onEvaluation = vi.fn(() => order.push("evaluation"));
const result = await runWithStreamingContext(
{
onStreamChunk: vi.fn(),
onToolCall,
onToolResult,
onEvaluation,
messageId: "message-1",
},
() =>
runPlannerLoop({
runtime,
context: {
id: "ctx",
events: [
{
id: "tool:LOOKUP",
type: "tool",
tool: {
name: "LOOKUP",
description: "Look up status",
},
},
],
},
executeToolCall: async (toolCall) =>
actionResultToPlannerToolResult(
await executePlannedToolCall(
runtime,
{ message: makeMessage() },
toolCall,
),
),
}),
);
expect(result.status).toBe("finished");
expect(order).toEqual(["toolCall", "toolResult", "evaluation"]);
expect(onToolCall).toHaveBeenCalledWith(
expect.objectContaining({
messageId: "message-1",
toolCall: expect.objectContaining({
id: "call-1",
name: "LOOKUP",
status: "pending",
}),
contextEvent: expect.objectContaining({ id: "tool:LOOKUP" }),
}),
);
expect(onToolResult).toHaveBeenCalledWith(
expect.objectContaining({
messageId: "message-1",
toolCallId: "call-1",
status: "completed",
result: expect.objectContaining({
success: true,
text: "all good",
data: expect.objectContaining({ answer: "ok" }),
}),
}),
);
expect(onEvaluation).toHaveBeenCalledWith(
expect.objectContaining({
messageId: "message-1",
evaluation: expect.objectContaining({
decision: "FINISH",
messageToUser: "Done.",
}),
}),
);
expect(runtime.useModel).toHaveBeenNthCalledWith(
1,
ModelType.ACTION_PLANNER,
expect.any(Object),
undefined,
);
expect(runtime.useModel).toHaveBeenNthCalledWith(
2,
ModelType.RESPONSE_HANDLER,
expect.any(Object),
undefined,
);
});
it("isolates runtime hook failures", async () => {
const runtime = makeRuntime(
[
{
text: "",
toolCalls: [{ id: "call-1", name: "LOOKUP", arguments: {} }],
},
JSON.stringify({
success: true,
decision: "FINISH",
thought: "Done.",
messageToUser: "Done.",
}),
],
[makeAction({ name: "LOOKUP" })],
);
const result = await runWithStreamingContext(
{
onStreamChunk: vi.fn(),
onToolCall: vi.fn(async () => {
throw new Error("tool call observer failed");
}),
onToolResult: vi.fn(async () => {
throw new Error("tool result observer failed");
}),
onEvaluation: vi.fn(async () => {
throw new Error("evaluation observer failed");
}),
},
() =>
runPlannerLoop({
runtime,
context: { id: "ctx", events: [] },
executeToolCall: async (toolCall) =>
actionResultToPlannerToolResult(
await executePlannedToolCall(
runtime,
{ message: makeMessage() },
toolCall,
),
),
}),
);
expect(result.status).toBe("finished");
expect(result.finalMessage).toBe("Done.");
});
it("emits failed tool results when handler errors become ActionResults", async () => {
const runtime = makeRuntime(
[],
[
makeAction({
name: "BOOM",
handler: async () => {
throw new Error("handler failed");
},
}),
],
);
const onToolResult = vi.fn();
const result = await runWithStreamingContext(
{
onStreamChunk: vi.fn(),
onToolResult,
messageId: "message-1",
},
() =>
executePlannedToolCall(
runtime,
{ message: makeMessage() },
{ id: "call-err", name: "BOOM", params: {} },
),
);
expect(result.success).toBe(false);
expect(onToolResult).toHaveBeenCalledWith(
expect.objectContaining({
messageId: "message-1",
toolCallId: "call-err",
status: "failed",
result: expect.objectContaining({
success: false,
error: "handler failed",
}),
}),
);
});
it("emits context events appended by the sub-planner", async () => {
const child = makeAction({ name: "CHILD" });
const parent = makeAction({
name: "PARENT",
subActions: ["CHILD"],
});
const runtime = makeRuntime(
[
JSON.stringify({
thought: "No child tool needed.",
toolCalls: [],
messageToUser: "Done.",
}),
],
[parent, child],
);
const onContextEvent = vi.fn();
await runWithStreamingContext(
{
onStreamChunk: vi.fn(),
onContextEvent,
},
() =>
runSubPlanner({
runtime: runtime as IAgentRuntime & {
useModel: IAgentRuntime["useModel"];
},
action: parent,
context: { id: "ctx", events: [] },
ctx: { message: makeMessage() },
}),
);
expect(onContextEvent).toHaveBeenCalledWith(
expect.objectContaining({
id: "sub-planner:PARENT:tool:CHILD",
type: "tool",
tool: expect.objectContaining({ name: "CHILD" }),
}),
);
});
});
@@ -0,0 +1,718 @@
/**
* Drives `runV5MessageRuntimeStage1` through a many-step website-build
* simulation to exercise planner compaction, quality-gate failure recovery, and
* trajectory export/metrics. Deterministic: a canned-response stub runtime with
* real trajectory recording written to a temp dir, no live model.
*/
import {
mkdirSync,
mkdtempSync,
readdirSync,
readFileSync,
rmSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { HANDLE_RESPONSE_TOOL_NAME } from "../actions/to-tool";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import { runV5MessageRuntimeStage1 } from "../services/message";
import type {
Action,
ActionResult,
HandlerCallback,
HandlerOptions,
} from "../types/components";
import type { ContextRegistry } from "../types/contexts";
import type { Memory } from "../types/memory";
import { ModelType } from "../types/model";
import type { UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
function stage1Response(fields: {
shouldRespond?: "RESPOND" | "IGNORE" | "STOP";
thought?: string;
contexts?: string[];
intents?: string[];
candidateActionNames?: string[];
replyText?: string;
facts?: string[];
relationships?: unknown[];
addressedTo?: string[];
extra?: Record<string, unknown>;
}): {
text: string;
toolCalls: Array<{
id: string;
name: string;
arguments: Record<string, unknown>;
}>;
} {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: HANDLE_RESPONSE_TOOL_NAME,
arguments: {
shouldRespond: fields.shouldRespond ?? "RESPOND",
thought: fields.thought ?? "",
contexts: fields.contexts ?? [],
intents: fields.intents ?? [],
candidateActionNames: fields.candidateActionNames ?? [],
replyText: fields.replyText ?? "",
facts: fields.facts ?? [],
relationships: fields.relationships ?? [],
addressedTo: fields.addressedTo ?? [],
...(fields.extra ?? {}),
},
},
],
};
}
const MSG_ID = "10000000-0000-0000-0000-000000000001" as UUID;
const SENDER_ID = "10000000-0000-0000-0000-000000000002" as UUID;
const AGENT_ID = "10000000-0000-0000-0000-000000000003" as UUID;
const ROOM_ID = "10000000-0000-0000-0000-000000000004" as UUID;
const RESPONSE_ID = "10000000-0000-0000-0000-000000000005" as UUID;
interface CannedResponse {
expectModelType?: string;
body: unknown;
}
function createResponseHandlerFieldRegistry(): ResponseHandlerFieldRegistry {
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return responseHandlerFieldRegistry;
}
function makeMessage(): Memory {
return {
id: MSG_ID,
entityId: SENDER_ID,
agentId: AGENT_ID,
roomId: ROOM_ID,
content: {
text: "Build a production-ready full-stack task website, repair any implementation markers, then run lint, tests, and build before replying.",
source: "test",
},
createdAt: 1,
};
}
function makeState(): State {
return {
values: { availableContexts: "simple, general" },
text: "Recent conversation: the operator wants an end-to-end website build with strict quality gates.",
data: {
providerOrder: ["RECENT_MESSAGES", "CONTEXT_BENCH"],
providers: {
RECENT_MESSAGES: {
providerName: "RECENT_MESSAGES",
text: "Operator requested a full-stack website build and quality verification.",
},
CONTEXT_BENCH: {
providerName: "CONTEXT_BENCH",
text: "Stress scenario: many tool steps, long generated artifacts, compaction, lint failure recovery, test pass, build pass, and trajectory export.",
},
},
},
};
}
function makeContextRegistry(): ContextRegistry {
return {
listAvailable: () => [
{
id: "simple",
label: "Simple",
description: "Direct reply only.",
gate: { minRole: "GUEST" as const },
},
{
id: "general",
label: "General",
description: "General agent work and local development tasks.",
gate: { minRole: "GUEST" as const },
},
],
} as ContextRegistry;
}
function makeRuntime(opts: {
actions: Action[];
responses: CannedResponse[];
state: State;
contextRegistry?: ContextRegistry;
}): IAgentRuntime {
const queue = [...opts.responses];
const responseHandlerFieldRegistry = createResponseHandlerFieldRegistry();
const calls: Array<{
modelType: unknown;
params: unknown;
provider: unknown;
}> = [];
const runtime = {
agentId: AGENT_ID,
character: { name: "Test Agent", system: "You are concise." },
actions: opts.actions,
providers: [],
contexts: opts.contextRegistry,
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
composeState: vi.fn(async () => opts.state),
emitEvent: vi.fn(async () => undefined),
runActionsByMode: vi.fn(async () => []),
useModel: vi.fn(
async (modelType: unknown, params: unknown, provider: unknown) => {
calls.push({ modelType, params, provider });
if (queue.length === 0) {
throw new Error(`Unexpected useModel call: ${String(modelType)}`);
}
const next = queue.shift();
if (
next?.expectModelType &&
String(modelType) !== next.expectModelType
) {
throw new Error(
`Expected ${next.expectModelType} but received ${String(modelType)}`,
);
}
return next?.body;
},
),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime & { __calls: typeof calls };
runtime.__calls = calls;
return runtime;
}
function getCalls(runtime: IAgentRuntime): Array<{
modelType: unknown;
params: unknown;
provider: unknown;
}> {
return (
runtime as {
__calls: Array<{
modelType: unknown;
params: unknown;
provider: unknown;
}>;
}
).__calls;
}
function makeAction(opts: {
name: string;
parameters?: Action["parameters"];
handler: (
runtime: IAgentRuntime,
message: Memory,
state: State | undefined,
options: HandlerOptions,
callback?: HandlerCallback,
) => Promise<ActionResult>;
}): Action {
return {
name: opts.name,
description: `${opts.name} test action`,
contexts: ["general"],
similes: [],
examples: [],
parameters: opts.parameters ?? [],
validate: async () => true,
handler: opts.handler,
} as Action;
}
function readRecordedTrajectories(agentId: string, rootDir: string): unknown[] {
const dir = join(rootDir, agentId);
let entries: string[];
try {
entries = readdirSync(dir);
} catch {
return [];
}
return entries
.filter((entry) => entry.endsWith(".json"))
.map((entry) => JSON.parse(readFileSync(join(dir, entry), "utf8")));
}
let tempDir: string;
let originalEnv: NodeJS.ProcessEnv;
beforeEach(() => {
tempDir = mkdtempSync(join(tmpdir(), "v5-stress-compaction-"));
originalEnv = { ...process.env };
process.env.ELIZA_TRAJECTORY_DIR = tempDir;
process.env.ELIZA_TRAJECTORY_RECORDING = "1";
process.env.ELIZA_TRAJECTORY_REVIEW_MODE = "1";
process.env.ELIZA_AWAIT_FACTS_STAGE = "true";
});
afterEach(() => {
process.env = originalEnv;
rmSync(tempDir, { recursive: true, force: true });
});
describe("v5 stress path — long build, compaction, gates, trajectory export", () => {
it("runs a many-step website build simulation through compaction and exports reviewable trajectories", async () => {
const implementationMarker = ["TO", "DO"].join("");
const forbiddenMarkers = [
implementationMarker,
["ST", "UB"].join(""),
["PLACE", "HOLDER"].join(""),
["Not", " implemented"].join(""),
];
const forbiddenPattern = new RegExp(
forbiddenMarkers
.map((marker) => marker.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"))
.join("|"),
"i",
);
const files = new Map<string, string>();
const generatedApp = [
"// TaskFlow full-stack app",
"import { useMemo, useState } from 'react';",
"",
"type Task = { id: string; title: string; owner: string; done: boolean };",
"",
"export function createTask(title: string, owner: string): Task {",
" return { id: String(Date.now()) + '-' + title, title, owner, done: false };",
"}",
"",
"export function App() {",
" const [tasks, setTasks] = useState<Task[]>([createTask('Design API', 'Eliza')]);",
" const complete = (id: string) => setTasks((items) => items.map((item) => item.id === id ? { ...item, done: true } : item));",
" const open = useMemo(() => tasks.filter((task) => !task.done).length, [tasks]);",
` // ${implementationMarker}: replace temporary analytics label before release`,
" return <main><h1>TaskFlow</h1><p>{open} open tasks</p><button onClick={() => complete(tasks[0].id)}>Complete first</button></main>;",
"}",
"",
...Array.from(
{ length: 700 },
(_, index) => `export const generatedMetric${index} = ${index};`,
),
].join("\n");
const actions = [
makeAction({
name: "GENERATE_FULL_STACK_APP",
handler: async () => ({
success: true,
text: `Generated TaskFlow app:\n${generatedApp}`,
userFacingText: "Generated the TaskFlow app source.",
data: {
actionName: "GENERATE_FULL_STACK_APP",
files: {
"src/App.tsx": generatedApp,
"src/api/tasks.ts":
"export const listTasks = () => [{ id: '1', title: 'Design API' }];\n",
"package.json": JSON.stringify({
scripts: {
lint: "eslint .",
test: "vitest run",
build: "vite build",
},
}),
},
},
}),
}),
makeAction({
name: "WRITE_APP_FILES",
handler: async (_runtime, _message, _state, options) => {
const prior = options.actionContext?.getPreviousResult(
"GENERATE_FULL_STACK_APP",
);
const generatedFiles = prior?.data?.files as
| Record<string, string>
| undefined;
if (!generatedFiles) {
return {
success: false,
text: "No generated files were available to write.",
};
}
for (const [path, content] of Object.entries(generatedFiles)) {
files.set(path, content);
}
return {
success: true,
text: `Wrote ${files.size} files to the app workspace.`,
userFacingText: `Wrote ${files.size} files to the app workspace.`,
data: { actionName: "WRITE_APP_FILES", fileCount: files.size },
};
},
}),
makeAction({
name: "RUN_QUALITY_GATE",
parameters: [
{
name: "gate",
description: "Quality gate to run",
required: true,
schema: {
type: "string",
enum: ["lint", "test", "build"],
},
},
],
handler: async (_runtime, _message, _state, options) => {
const gate = String(options.parameters?.gate ?? "");
if (gate === "lint") {
const matches = [...files.entries()]
.filter(([, content]) => forbiddenPattern.test(content))
.map(([path]) => path);
if (matches.length > 0) {
return {
success: false,
text: `lint failed: forbidden implementation marker in ${matches.join(", ")}`,
data: {
actionName: "RUN_QUALITY_GATE",
gate,
matches,
},
error: "forbidden implementation marker",
};
}
return {
success: true,
text: "lint passed with no implementation markers.",
userFacingText: "lint passed with no implementation markers.",
data: { actionName: "RUN_QUALITY_GATE", gate },
};
}
if (gate === "test") {
const app = files.get("src/App.tsx") ?? "";
return {
success: app.includes("createTask"),
text: app.includes("createTask")
? "tests passed: task creation flow is covered."
: "tests failed: task creation flow missing.",
userFacingText: app.includes("createTask")
? "tests passed: task creation flow is covered."
: undefined,
data: { actionName: "RUN_QUALITY_GATE", gate },
};
}
if (gate === "build") {
const hasRequiredFiles =
files.has("src/App.tsx") && files.has("src/api/tasks.ts");
return {
success: hasRequiredFiles,
text: hasRequiredFiles
? "build passed: client and API modules compiled."
: "build failed: required modules missing.",
userFacingText: hasRequiredFiles
? "build passed: client and API modules compiled."
: undefined,
data: { actionName: "RUN_QUALITY_GATE", gate },
};
}
return {
success: false,
text: `unknown quality gate: ${gate}`,
error: "unknown gate",
};
},
}),
makeAction({
name: "REPAIR_APP_FILES",
handler: async () => {
for (const [path, content] of files.entries()) {
files.set(
path,
content.replace(
new RegExp(`\\s*// ${implementationMarker}:.*`, "g"),
"",
),
);
}
return {
success: true,
text: "Repaired generated files and removed implementation markers.",
userFacingText:
"Repaired generated files and removed implementation markers.",
data: { actionName: "REPAIR_APP_FILES" },
};
},
}),
];
const state = makeState();
const runtime = makeRuntime({
actions,
state,
contextRegistry: makeContextRegistry(),
responses: [
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: stage1Response({
contexts: ["general"],
candidateActionNames: [
"GENERATE_FULL_STACK_APP",
"WRITE_APP_FILES",
"RUN_QUALITY_GATE",
"REPAIR_APP_FILES",
],
thought:
"The user requested a multi-step build and verification task.",
}),
},
{
expectModelType: ModelType.ACTION_PLANNER,
body: {
text: "",
toolCalls: [
{
id: "generate-1",
name: "GENERATE_FULL_STACK_APP",
arguments: {},
},
],
},
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: true,
decision: "CONTINUE",
thought:
"The app was generated; write the files and start quality gates.",
}),
},
{
expectModelType: ModelType.ACTION_PLANNER,
body: {
text: "",
toolCalls: [
{ id: "write-1", name: "WRITE_APP_FILES", arguments: {} },
{
id: "lint-1",
name: "RUN_QUALITY_GATE",
arguments: { gate: "lint" },
},
],
},
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: true,
decision: "NEXT_RECOMMENDED",
thought: "Files are written; run the queued lint gate.",
recommendedToolCallId: "lint-1",
}),
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: false,
decision: "CONTINUE",
thought: "Lint caught a forbidden marker; replan a repair.",
}),
},
{
expectModelType: ModelType.ACTION_PLANNER,
body: {
text: "",
toolCalls: [
{ id: "repair-1", name: "REPAIR_APP_FILES", arguments: {} },
{
id: "lint-2",
name: "RUN_QUALITY_GATE",
arguments: { gate: "lint" },
},
{
id: "test-1",
name: "RUN_QUALITY_GATE",
arguments: { gate: "test" },
},
{
id: "build-1",
name: "RUN_QUALITY_GATE",
arguments: { gate: "build" },
},
],
},
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: true,
decision: "NEXT_RECOMMENDED",
thought: "Repair succeeded; rerun lint.",
recommendedToolCallId: "lint-2",
}),
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: true,
decision: "NEXT_RECOMMENDED",
thought: "Lint is clean; run tests.",
recommendedToolCallId: "test-1",
}),
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: true,
decision: "NEXT_RECOMMENDED",
thought: "Tests passed; run build.",
recommendedToolCallId: "build-1",
}),
},
{
expectModelType: ModelType.RESPONSE_HANDLER,
body: JSON.stringify({
success: true,
decision: "FINISH",
thought:
"All quality gates passed after repairing the implementation marker.",
messageToUser:
"Built the TaskFlow app, repaired the lint issue, and verified lint, tests, and build.",
}),
},
],
});
const result = await runV5MessageRuntimeStage1({
runtime,
message: makeMessage(),
state,
responseId: RESPONSE_ID,
plannerLoopConfig: {
maxToolCalls: 12,
contextWindowTokens: 5_000,
compactionReserveTokens: 1_000,
compactionKeepSteps: 0,
},
});
expect(result.kind).toBe("planned_reply");
if (result.kind === "planned_reply") {
expect(result.result.responseContent?.text).toContain("TaskFlow");
}
for (const [path, content] of files.entries()) {
expect(path).toMatch(/\.(tsx|ts|json)$/);
expect(content).not.toMatch(forbiddenPattern);
}
const calls = getCalls(runtime);
const plannerCalls = calls.filter(
(call) => call.modelType === ModelType.ACTION_PLANNER,
);
expect(plannerCalls.length).toBe(3);
const secondPlannerMessages = JSON.stringify(
(plannerCalls[1]?.params as { messages?: unknown[] } | undefined)
?.messages ?? [],
);
expect(secondPlannerMessages).toContain("compaction");
expect(secondPlannerMessages).not.toContain("x".repeat(1_000));
const trajectories = readRecordedTrajectories(String(AGENT_ID), tempDir);
expect(trajectories).toHaveLength(1);
const exportPath = join(tempDir, "all-trajectories.jsonl");
const jsonl =
trajectories.map((trajectory) => JSON.stringify(trajectory)).join("\n") +
"\n";
writeFileSync(exportPath, jsonl, "utf8");
const exported = readFileSync(exportPath, "utf8");
expect(exported).toContain('"trajectoryId"');
const reviewExportDir = process.env.ELIZA_V5_STRESS_EXPORT_DIR;
if (reviewExportDir) {
mkdirSync(reviewExportDir, { recursive: true });
writeFileSync(
join(reviewExportDir, "v5-stress-trajectory.jsonl"),
exported,
"utf8",
);
}
const trajectory = trajectories[0] as {
status: string;
stages: Array<{
kind: string;
tool?: {
name: string;
success: boolean;
args?: Record<string, unknown>;
};
model?: {
messages?: unknown[];
prompt?: string;
providerOptions?: {
eliza?: {
modelInputBudget?: {
contextWindowTokens?: number;
reserveTokens?: number;
};
};
};
};
}>;
metrics: {
plannerIterations: number;
toolCallsExecuted: number;
toolCallFailures: number;
evaluatorFailures: number;
finalDecision: string;
};
};
expect(trajectory.status).toBe("finished");
expect(trajectory.metrics.toolCallsExecuted).toBe(7);
expect(trajectory.metrics.toolCallFailures).toBe(1);
expect(trajectory.metrics.evaluatorFailures).toBe(0);
expect(trajectory.metrics.finalDecision).toBe("FINISH");
expect(trajectory.stages.map((stage) => stage.kind)).toContain(
"compaction",
);
expect(
trajectory.stages.some(
(stage) =>
stage.kind === "tool" &&
stage.tool?.name === "RUN_QUALITY_GATE" &&
stage.tool.success === false,
),
).toBe(true);
expect(
trajectory.stages.filter((stage) => stage.kind === "planner"),
).toHaveLength(3);
const recordedPlannerPayload = JSON.stringify(
trajectory.stages
.filter((stage) => stage.kind === "planner")
.map((stage) => stage.model?.messages ?? []),
);
expect(recordedPlannerPayload).toContain("compaction");
expect(recordedPlannerPayload).not.toContain("x".repeat(1_000));
const firstPlannerStage = trajectory.stages.find(
(stage) => stage.kind === "planner",
);
expect(
firstPlannerStage?.model?.providerOptions?.eliza?.modelInputBudget,
).toMatchObject({
contextWindowTokens: 5_000,
reserveTokens: 1_000,
});
});
});
@@ -0,0 +1,35 @@
/**
* Unit tests for `parseKeyValueXml`: `<response>` block parsing and the
* prefix-extended-tag regression where a nested `<textarea>` inside `<text>`
* must not inflate close-tag matching. Deterministic parser test.
*/
import { describe, expect, it } from "vitest";
import { parseKeyValueXml } from "../utils";
describe("parseKeyValueXml", () => {
it("parses XML response blocks", () => {
const parsed = parseKeyValueXml(`
<response>
<message>Hello &amp; bye</message>
<actions>send, reply</actions>
</response>`);
expect(parsed).toEqual({
message: "Hello & bye",
actions: ["send", "reply"],
});
});
it("does not treat a prefix-extended tag in a value as a nested open", () => {
// Regression: findMatchingXmlClose matched any tag STARTING with the name
// (`<textarea>` while closing `<text>`), inflating depth so the close was
// never found — the field was dropped and a bogus key promoted.
const parsed = parseKeyValueXml(
`<response><text>see <textarea>x</textarea> ok</text><thought>t</thought></response>`,
);
expect(parsed).toEqual({
text: "see <textarea>x</textarea> ok",
thought: "t",
});
});
});
@@ -0,0 +1,144 @@
/**
* Covers `toSwarmActivity`, the boundary that narrows a raw `SwarmEvent`
* (wire-typed `data: unknown`) into the typed discriminated inline-activity
* envelope the chat pipeline consumes. Pure function over synthetic events.
*/
import { describe, expect, it } from "vitest";
import { type SwarmEvent, toSwarmActivity } from "../types/swarm-coordinator";
function ev(partial: Partial<SwarmEvent> & { type: string }): SwarmEvent {
return {
sessionId: "s1",
timestamp: 1000,
data: {},
...partial,
};
}
describe("toSwarmActivity", () => {
it("narrows a message event and carries seq/taskId/parent", () => {
expect(
toSwarmActivity(
ev({
type: "message",
seq: 7,
taskId: "task-abc",
parentSessionId: "root",
data: { text: "hello" },
}),
),
).toEqual({
kind: "message",
sessionId: "s1",
seq: 7,
timestamp: 1000,
taskId: "task-abc",
parentSessionId: "root",
text: "hello",
});
});
it("falls back to timestamp when no seq is present", () => {
const out = toSwarmActivity(
ev({ type: "reasoning", data: { text: "hm" } }),
);
expect(out).toMatchObject({ kind: "reasoning", seq: 1000, text: "hm" });
});
it("narrows a plan event, dropping blank entries", () => {
const out = toSwarmActivity(
ev({
type: "plan",
data: {
entries: [
{ content: "step one", status: "completed", priority: "high" },
{ content: "", status: "pending" },
{ content: "step two", status: "in_progress" },
],
},
}),
);
expect(out).toMatchObject({
kind: "plan",
entries: [
{ content: "step one", status: "completed", priority: "high" },
{ content: "step two", status: "in_progress" },
],
});
});
it("narrows a tool_running event and maps the status", () => {
const out = toSwarmActivity(
ev({
type: "tool_running",
data: {
toolCall: {
id: "t1",
title: "Bash",
kind: "execute",
status: "completed",
rawInput: { command: "git status" },
output: "clean",
locations: [{ path: "a.ts", line: 3 }],
},
},
}),
);
expect(out).toEqual({
kind: "tool",
sessionId: "s1",
seq: 1000,
timestamp: 1000,
tool: {
id: "t1",
title: "Bash",
kind: "execute",
status: "success",
rawInput: { command: "git status" },
output: "clean",
locations: [{ path: "a.ts", line: 3 }],
},
});
});
it("defaults a tool without status to running", () => {
const out = toSwarmActivity(
ev({ type: "tool_running", data: { toolCall: { title: "Read" } } }),
);
expect(out).toMatchObject({ kind: "tool", tool: { status: "running" } });
});
it("maps lifecycle events to a coarse status", () => {
expect(
toSwarmActivity(ev({ type: "task_complete", data: {} })),
).toMatchObject({
kind: "lifecycle",
event: "task_complete",
status: "success",
});
expect(
toSwarmActivity(ev({ type: "error", data: { message: "boom" } })),
).toMatchObject({ kind: "lifecycle", status: "failure", text: "boom" });
expect(toSwarmActivity(ev({ type: "blocked", data: {} }))).toMatchObject({
kind: "lifecycle",
status: "waiting",
});
expect(toSwarmActivity(ev({ type: "ready", data: {} }))).toMatchObject({
kind: "lifecycle",
status: "idle",
});
});
it("returns null for non-renderable / empty events", () => {
expect(
toSwarmActivity(ev({ type: "message", data: { text: "" } })),
).toBeNull();
expect(
toSwarmActivity(ev({ type: "plan", data: { entries: [] } })),
).toBeNull();
expect(toSwarmActivity(ev({ type: "agent_event", data: {} }))).toBeNull();
expect(
toSwarmActivity(ev({ type: "totally_unknown", data: {} })),
).toBeNull();
});
});
@@ -0,0 +1,60 @@
/**
* Test Helper Utilities for Core Package Tests
*
* IMPORTANT: This file should only contain utilities for:
* 1. Generating test data (UUIDs, timestamps)
* 2. Timing/waiting utilities
* 3. Type-safe test data factories
*
* DO NOT add mock factories here. For integration testing:
* - Use @elizaos/plugin-sql with PGLite for real database
* - Use real runtime initialization
*/
import type { UUID } from "../types";
import { stringToUuid } from "../utils";
/**
* Generate a unique test UUID based on timestamp and random component
*/
export function generateTestUUID(): UUID {
return stringToUuid(
`test-${Date.now()}-${Math.random().toString(36).slice(2)}`,
);
}
/**
* Wait for a condition to be true with timeout
* @throws Error if condition is not met within timeout
*/
export async function waitFor(
condition: () => boolean | Promise<boolean>,
timeoutMs = 5000,
intervalMs = 100,
): Promise<void> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (await condition()) return;
await new Promise((resolve) => setTimeout(resolve, intervalMs));
}
throw new Error(`Condition not met within ${timeoutMs}ms`);
}
/**
* Measure execution time of an async function
*/
export async function measureTime<T>(
fn: () => Promise<T>,
): Promise<{ result: T; durationMs: number }> {
const start = performance.now();
const result = await fn();
return { result, durationMs: performance.now() - start };
}
/**
* Create a deterministic UUID from a test identifier
* Useful for consistent test data across runs
*/
export function testUuid(identifier: string): UUID {
return stringToUuid(`test-${identifier}`);
}
@@ -0,0 +1,687 @@
/**
* Exercises the v5 tiered action surface through `runV5MessageRuntimeStage1`:
* Stage-1 hints promoting a parent to Tier A, sub-actions surfaced as
* first-class planner tools, hot-parent child capping, role-gated tool omission,
* and Tier-B sub-planner execution. Deterministic: a canned-response stub
* runtime, no live model.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { _resetActionRolePolicyCacheForTests } from "../runtime/action-role-policy";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import { runV5MessageRuntimeStage1 } from "../services/message";
import type {
Action,
ActionResult,
HandlerCallback,
HandlerOptions,
} from "../types/components";
import type { AgentContext, ContextGate, RoleGate } from "../types/contexts";
import type { Memory } from "../types/memory";
import { ModelType } from "../types/model";
import type { UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
import { getActiveRoutingContextsForTurn } from "../utils/context-routing";
const MSG_ID = "00000000-0000-0000-0000-100000000001" as UUID;
const SENDER_ID = "00000000-0000-0000-0000-100000000002" as UUID;
const AGENT_ID = "00000000-0000-0000-0000-100000000003" as UUID;
const ROOM_ID = "00000000-0000-0000-0000-100000000004" as UUID;
const RESPONSE_ID = "00000000-0000-0000-0000-100000000005" as UUID;
function makeMessage(text: string): Memory {
return {
id: MSG_ID,
entityId: SENDER_ID,
agentId: AGENT_ID,
roomId: ROOM_ID,
content: { text, source: "test" },
createdAt: 1,
};
}
function makeState(): State {
return {
values: {},
data: {},
text: "Recent conversation summary",
};
}
interface CannedResponse {
body: unknown;
}
function createResponseHandlerFieldRegistry(): ResponseHandlerFieldRegistry {
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return responseHandlerFieldRegistry;
}
function makeRuntime(opts: {
actions: Action[];
responses: CannedResponse[];
}): IAgentRuntime {
const queue = [...opts.responses];
const responseHandlerFieldRegistry = createResponseHandlerFieldRegistry();
const calls: Array<{
modelType: unknown;
params: unknown;
provider: unknown;
}> = [];
const runtime = {
agentId: AGENT_ID,
character: {
name: "Test Agent",
system: "You are concise.",
bio: "I route actions.",
},
actions: opts.actions,
providers: [],
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
composeState: vi.fn(async () => makeState()),
emitEvent: vi.fn(async () => undefined),
runActionsByMode: vi.fn(async () => undefined),
useModel: vi.fn(
async (modelType: unknown, params: unknown, provider: unknown) => {
calls.push({ modelType, params, provider });
if (queue.length === 0) {
throw new Error(`Unexpected useModel call: ${String(modelType)}`);
}
return queue.shift()?.body;
},
),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
} as IAgentRuntime & { __calls: typeof calls };
runtime.__calls = calls;
return runtime;
}
function getCalls(runtime: IAgentRuntime): Array<{
modelType: unknown;
params: unknown;
provider: unknown;
}> {
return (
runtime as {
__calls: Array<{
modelType: unknown;
params: unknown;
provider: unknown;
}>;
}
).__calls;
}
function makeAction(opts: {
name: string;
description?: string;
similes?: string[];
contexts?: AgentContext[];
contextGate?: ContextGate;
roleGate?: RoleGate;
subActions?: Array<string | Action>;
validate?: (
runtime: IAgentRuntime,
message: Memory,
state: State | undefined,
options?: HandlerOptions,
) => Promise<boolean>;
handler?: (
runtime: IAgentRuntime,
message: Memory,
state: State | undefined,
options: HandlerOptions,
callback?: HandlerCallback,
) => Promise<ActionResult>;
}): Action {
return {
name: opts.name,
description: opts.description ?? `${opts.name} action`,
similes: opts.similes ?? [],
examples: [],
parameters: [],
contexts: opts.contexts,
contextGate: opts.contextGate,
roleGate: opts.roleGate,
subActions: opts.subActions,
validate: opts.validate ?? (async () => true),
handler:
opts.handler ??
(async () => ({
success: true,
text: `${opts.name} completed`,
data: { actionName: opts.name },
})),
} as Action;
}
function stage1Response(fields: {
shouldRespond?: "RESPOND" | "IGNORE" | "STOP";
contexts?: string[];
intents?: string[];
candidateActionNames?: string[];
replyText?: string;
}): CannedResponse {
return {
body: {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: "HANDLE_RESPONSE",
arguments: {
shouldRespond: fields.shouldRespond ?? "RESPOND",
contexts: fields.contexts ?? [],
intents: fields.intents ?? [],
candidateActionNames: fields.candidateActionNames ?? [],
replyText: fields.replyText ?? "",
facts: [],
relationships: [],
addressedTo: [],
},
},
],
},
};
}
function plannerToolResponse(
name: string,
args: Record<string, unknown> = {},
): CannedResponse {
return {
body: {
text: "",
toolCalls: [{ id: `${name.toLowerCase()}-1`, name, args }],
},
};
}
function finishEvaluatorResponse(messageToUser = "Done."): CannedResponse {
return {
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: messageToUser,
messageToUser,
}),
};
}
function plannerUserContent(runtime: IAgentRuntime): string {
const plannerCall = getCalls(runtime).find(
(call) => call.modelType === ModelType.ACTION_PLANNER,
);
const params = plannerCall?.params as
| { messages?: Array<{ role?: string; content?: string }> }
| undefined;
return (
params?.messages?.map((message) => message.content ?? "").join("\n") ?? ""
);
}
function availableActionsSection(runtime: IAgentRuntime): string {
// Actions are exposed as native tools on the planner call, not in an
// `available_actions` text block. Synthesize a section-like view from the
// tool definitions so the tier-A vs tier-B assertions in this file can still
// inspect action name presence and order.
const plannerCall = getCalls(runtime).find(
(call) => call.modelType === ModelType.ACTION_PLANNER,
);
const tools = (
plannerCall?.params as
| { tools?: Array<{ name?: string; description?: string }> }
| undefined
)?.tools;
if (!tools || tools.length === 0) {
return plannerUserContent(runtime);
}
return tools
.map((tool) => `- ${tool.name ?? ""}: ${tool.description ?? ""}`)
.join("\n");
}
describe("v5 tiered action surface", () => {
let originalTrajectoryEnv: string | undefined;
let originalActionRolePolicy: string | undefined;
beforeEach(() => {
originalTrajectoryEnv = process.env.ELIZA_TRAJECTORY_RECORDING;
originalActionRolePolicy = process.env.ACTION_ROLE_POLICY;
process.env.ELIZA_TRAJECTORY_RECORDING = "0";
_resetActionRolePolicyCacheForTests();
});
afterEach(() => {
if (originalTrajectoryEnv === undefined) {
delete process.env.ELIZA_TRAJECTORY_RECORDING;
} else {
process.env.ELIZA_TRAJECTORY_RECORDING = originalTrajectoryEnv;
}
if (originalActionRolePolicy === undefined) {
delete process.env.ACTION_ROLE_POLICY;
} else {
process.env.ACTION_ROLE_POLICY = originalActionRolePolicy;
}
_resetActionRolePolicyCacheForTests();
});
it("uses Stage 1 hints to promote a parent to Tier A and expose children", async () => {
const playMusic = makeAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
contexts: ["music_child" as AgentContext],
});
const pauseMusic = makeAction({
name: "PAUSE_MUSIC",
description: "Pause the active track.",
contexts: ["music_child" as AgentContext],
});
const music = makeAction({
name: "MUSIC",
description: "Music control parent action.",
contexts: ["music" as AgentContext],
subActions: ["PLAY_MUSIC", "PAUSE_MUSIC"],
});
const email = makeAction({
name: "SEND_EMAIL",
description: "Send an email.",
contexts: ["email" as AgentContext],
});
const runtime = makeRuntime({
actions: [music, playMusic, pauseMusic, email],
responses: [
stage1Response({
contexts: ["music"],
candidateActionNames: ["play_music", "MUSIC"],
}),
plannerToolResponse("PLAY_MUSIC"),
finishEvaluatorResponse("Playing music."),
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("play the new album"),
state: makeState(),
responseId: RESPONSE_ID,
});
const prompt = availableActionsSection(runtime);
expect(prompt).toContain("MUSIC");
expect(prompt).toContain("PLAY_MUSIC");
expect(prompt).toContain("PAUSE_MUSIC");
expect(prompt).not.toContain("SEND_EMAIL");
});
it("expands strong context matches into callable actions", async () => {
const createEvent = makeAction({
name: "CREATE_EVENT",
description: "Create a calendar event.",
contexts: ["calendar_write" as AgentContext],
});
const calendar = makeAction({
name: "CALENDAR",
description: "Calendar scheduling and event management.",
contexts: ["calendar" as AgentContext],
subActions: ["CREATE_EVENT"],
});
const chat = makeAction({
name: "CHAT_MESSAGE",
description: "Send a chat message.",
contexts: ["calendar" as AgentContext],
});
const runtime = makeRuntime({
actions: [calendar, createEvent, chat],
responses: [
stage1Response({ contexts: ["calendar"] }),
plannerToolResponse("CHAT_MESSAGE"),
finishEvaluatorResponse("Calendar checked."),
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("calendar"),
state: makeState(),
responseId: RESPONSE_ID,
});
const prompt = availableActionsSection(runtime);
expect(prompt).toContain("CALENDAR");
expect(prompt).toContain("CREATE_EVENT");
expect(prompt).toContain("CHAT_MESSAGE");
});
it("carries Stage 1 contexts into action validation and execution", async () => {
let messageCalls = 0;
const message = makeAction({
name: "MESSAGE",
description:
"Primary email and messaging action for inbox review and unread email summaries.",
contexts: ["email"],
validate: async (_runtime, msg, state) =>
getActiveRoutingContextsForTurn(state, msg).includes("email"),
handler: async () => {
messageCalls++;
return {
success: true,
text: "summarized unread email",
data: { actionName: "MESSAGE" },
};
},
});
const runtime = makeRuntime({
actions: [message],
responses: [
stage1Response({
contexts: ["email"],
candidateActionNames: ["summarize_unread_emails", "MESSAGE"],
}),
{
body: {
text: "",
toolCalls: [
{
id: "message-1",
name: "MESSAGE",
arguments: {},
},
],
},
},
{
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: "Message action completed.",
messageToUser: "summarized unread email",
}),
},
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("summarize my unread emails"),
state: makeState(),
responseId: RESPONSE_ID,
});
expect(messageCalls).toBe(1);
expect(availableActionsSection(runtime)).toContain("MESSAGE");
});
it("exposes Tier-A sub-actions as first-class planner tools alongside the parent", async () => {
// This is the core guarantee: when MUSIC is in Tier A, its sub-actions
// PLAY_MUSIC and PAUSE_MUSIC are first-class entries in the planner's
// `tools` array (not just hidden behind a "dig into parent" round-trip).
const playMusic = makeAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
contexts: ["music_child" as AgentContext],
});
const pauseMusic = makeAction({
name: "PAUSE_MUSIC",
description: "Pause the active track.",
contexts: ["music_child" as AgentContext],
});
const music = makeAction({
name: "MUSIC",
description: "Music control parent action.",
contexts: ["music" as AgentContext],
subActions: ["PLAY_MUSIC", "PAUSE_MUSIC"],
});
const email = makeAction({
name: "SEND_EMAIL",
description: "Send an email.",
contexts: ["email" as AgentContext],
});
const runtime = makeRuntime({
actions: [music, playMusic, pauseMusic, email],
responses: [
stage1Response({
contexts: ["music"],
candidateActionNames: ["play_music", "MUSIC"],
}),
plannerToolResponse("PLAY_MUSIC"),
finishEvaluatorResponse("Playing music."),
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("play the new album"),
state: makeState(),
responseId: RESPONSE_ID,
});
const plannerCall = getCalls(runtime).find(
(call) => call.modelType === ModelType.ACTION_PLANNER,
);
const tools = (
plannerCall?.params as { tools?: Array<{ name?: string }> } | undefined
)?.tools;
const toolNames = tools?.map((tool) => tool.name).filter(Boolean) ?? [];
expect(toolNames).toContain("MUSIC");
expect(toolNames).toContain("PLAY_MUSIC");
expect(toolNames).toContain("PAUSE_MUSIC");
// Universal terminals must still be appended.
expect(toolNames).toContain("REPLY");
expect(toolNames).toContain("IGNORE");
expect(toolNames).toContain("STOP");
// Sibling-context action that is not in Tier A / Tier B should not leak in.
expect(toolNames).not.toContain("SEND_EMAIL");
});
it("caps a hot parent's sub-action flood to the turn-relevant children", async () => {
// One hot tier-A parent must not expose its whole namespace (observed
// live: all 24 MESSAGE_* children on a two-intent turn). The per-parent
// child narrow keeps the Stage-1 candidate plus the best query-token
// matches under the default cap of 8; everything else stays reachable
// only through the MESSAGE umbrella, whose handler routes any subaction.
const reviewQueue = makeAction({
name: "MESSAGE_REVIEW_QUEUE",
description: "Review channel messages awaiting a response.",
});
const sendReply = makeAction({
name: "MESSAGE_SEND_REPLY",
description: "Reply to messages needing a response.",
});
const bulkOps = Array.from({ length: 10 }, (_, i) =>
makeAction({
name: `MESSAGE_OP_${i}`,
description: `Unrelated bulk operation number ${i}.`,
}),
);
const message = makeAction({
name: "MESSAGE",
description: "Message management parent action.",
subActions: [
"MESSAGE_REVIEW_QUEUE",
"MESSAGE_SEND_REPLY",
...bulkOps.map((action) => action.name),
],
});
const runtime = makeRuntime({
actions: [message, reviewQueue, sendReply, ...bulkOps],
responses: [
stage1Response({
contexts: ["general"],
intents: ["review channel messages", "reply to messages"],
candidateActionNames: ["MESSAGE_REVIEW_QUEUE"],
}),
plannerToolResponse("MESSAGE_REVIEW_QUEUE"),
finishEvaluatorResponse("Reviewed the queue."),
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("review the channel messages needing a response"),
state: makeState(),
responseId: RESPONSE_ID,
});
const plannerCall = getCalls(runtime).find(
(call) => call.modelType === ModelType.ACTION_PLANNER,
);
const tools = (
plannerCall?.params as { tools?: Array<{ name?: string }> } | undefined
)?.tools;
const toolNames = tools?.map((tool) => tool.name).filter(Boolean) ?? [];
// Fires when relevant: the umbrella and the turn-relevant children are
// first-class tools.
expect(toolNames).toContain("MESSAGE");
expect(toolNames).toContain("MESSAGE_REVIEW_QUEUE");
expect(toolNames).toContain("MESSAGE_SEND_REPLY");
// Lean when not: the namespace is capped, not fully expanded.
const childTools = toolNames.filter((name) =>
String(name).startsWith("MESSAGE_"),
);
expect(childTools.length).toBeLessThanOrEqual(8);
expect(toolNames).not.toContain("MESSAGE_OP_9");
// Prompt footprint drops with the tool surface: the narrowed-out child
// no longer appears in the rendered action section either.
const prompt = availableActionsSection(runtime);
expect(prompt).toContain("MESSAGE_REVIEW_QUEUE");
expect(prompt).not.toContain("MESSAGE_OP_9");
});
it("omits planner tools that execution would reject for the selected context", async () => {
// ACTION_ROLE_POLICY authorizes by exact action name only — similes
// intentionally do not authorize (action-role-policy.ts), so the key must
// be "SHELL", not its "BASH" simile, to loosen SHELL's OWNER gate to GUEST.
process.env.ACTION_ROLE_POLICY = JSON.stringify({ SHELL: "GUEST" });
_resetActionRolePolicyCacheForTests();
const shell = makeAction({
name: "SHELL",
description:
"Run a shell command to inspect runtime or repository state.",
similes: ["BASH", "EXEC"],
contexts: ["terminal" as AgentContext],
contextGate: { anyOf: ["terminal"] },
roleGate: { minRole: "OWNER" },
});
const file = makeAction({
name: "FILE",
description: "Read, grep, or edit workspace files.",
contexts: ["code" as AgentContext],
contextGate: { anyOf: ["code"] },
roleGate: { minRole: "ADMIN" },
});
const runtime = makeRuntime({
actions: [shell, file],
responses: [
stage1Response({
contexts: ["general"],
candidateActionNames: ["SHELL", "FILE"],
}),
plannerToolResponse("SHELL", { command: "git status --short" }),
finishEvaluatorResponse("Shell checked."),
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("check the running repository status"),
state: makeState(),
responseId: RESPONSE_ID,
});
const plannerCall = getCalls(runtime).find(
(call) => call.modelType === ModelType.ACTION_PLANNER,
);
const tools = (
plannerCall?.params as { tools?: Array<{ name?: string }> } | undefined
)?.tools;
const toolNames = tools?.map((tool) => tool.name).filter(Boolean) ?? [];
expect(toolNames).toContain("SHELL");
expect(toolNames).not.toContain("FILE");
});
it("lets a Tier B parent invoke its sub-planner and execute child actions", async () => {
let createEventCalls = 0;
const createEvent = makeAction({
name: "CREATE_EVENT",
description: "Create a calendar event.",
contexts: ["calendar_write" as AgentContext],
handler: async () => {
createEventCalls++;
return {
success: true,
text: "created event",
data: { actionName: "CREATE_EVENT" },
};
},
});
const calendar = makeAction({
name: "CALENDAR",
description: "Calendar scheduling and event management.",
contexts: ["calendar" as AgentContext],
subActions: ["CREATE_EVENT"],
});
const runtime = makeRuntime({
actions: [calendar, createEvent],
responses: [
stage1Response({ contexts: ["calendar"] }),
{
body: {
text: "Using calendar.",
toolCalls: [{ id: "top-1", name: "CALENDAR", arguments: {} }],
},
},
{
body: {
text: "Creating the event.",
toolCalls: [{ id: "child-1", name: "CREATE_EVENT", arguments: {} }],
},
},
{
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: "Child action completed.",
messageToUser: "created event",
}),
},
{
body: JSON.stringify({
success: true,
decision: "FINISH",
thought: "Calendar task completed.",
messageToUser: "created event",
}),
},
],
});
await runV5MessageRuntimeStage1({
runtime,
message: makeMessage("calendar"),
state: makeState(),
responseId: RESPONSE_ID,
});
expect(createEventCalls).toBe(1);
expect(
getCalls(runtime).filter(
(call) => call.modelType === ModelType.ACTION_PLANNER,
),
).toHaveLength(2);
});
});
@@ -0,0 +1,67 @@
/**
* Async-scoped trajectory-context helpers: runWithTrajectoryPurpose overrides
* only the purpose tag while preserving the enclosing trajectory / step / run /
* room ids, and the suite pins the bare-context shape that would otherwise
* clobber the step id. Deterministic — drives the context manager directly, no
* model.
*/
import { describe, expect, it } from "vitest";
import {
getTrajectoryContext,
runWithTrajectoryContext,
runWithTrajectoryPurpose,
} from "../trajectory-context";
describe("runWithTrajectoryPurpose", () => {
it("overrides purpose while preserving the active context fields", async () => {
await runWithTrajectoryContext(
{
trajectoryId: "traj-1",
trajectoryStepId: "step-1",
runId: "run-1",
roomId: "room-1",
purpose: "action",
},
async () => {
await runWithTrajectoryPurpose("inbox_triage", async () => {
const ctx = getTrajectoryContext();
expect(ctx).toMatchObject({
trajectoryId: "traj-1",
trajectoryStepId: "step-1",
runId: "run-1",
roomId: "room-1",
purpose: "inbox_triage",
});
});
// Outer context is untouched after the scoped run.
expect(getTrajectoryContext()?.purpose).toBe("action");
expect(getTrajectoryContext()?.trajectoryStepId).toBe("step-1");
},
);
});
it("sets a bare purpose context when no context is active", async () => {
expect(getTrajectoryContext()).toBeUndefined();
await runWithTrajectoryPurpose("inbox_triage", async () => {
const ctx = getTrajectoryContext();
expect(ctx?.purpose).toBe("inbox_triage");
expect(ctx?.trajectoryStepId).toBeUndefined();
});
});
it("regression: a bare { purpose } context passed to runWithTrajectoryContext drops the step id", async () => {
// This is the clobber shape runWithTrajectoryPurpose exists to replace —
// keep it pinned so the difference stays visible.
await runWithTrajectoryContext(
{ trajectoryStepId: "step-1", purpose: "action" },
async () => {
await runWithTrajectoryContext(
{ purpose: "inbox_triage" },
async () => {
expect(getTrajectoryContext()?.trajectoryStepId).toBeUndefined();
},
);
},
);
});
});
@@ -0,0 +1,101 @@
/**
* Transcription-mode reply suppression: the transcriptionModeActive flag reader
* (client-transport metadata path and in-process top-level path) and the
* `core.transcription_mode` builtin response-handler evaluator that turns a
* flagged turn into IGNORE + clearReply. Deterministic — the evaluator runs
* against hand-built Memory, no model.
*/
import { describe, expect, it } from "vitest";
import type { ResponseHandlerEvaluatorContext } from "../runtime/response-handler-evaluators";
import {
BUILTIN_RESPONSE_HANDLER_EVALUATORS,
transcriptionModeActive,
} from "../services/message";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
const ROOM = "11111111-1111-1111-1111-111111111111" as UUID;
const ENTITY = "22222222-2222-2222-2222-222222222222" as UUID;
function msg(content: Record<string, unknown>): Memory {
return {
id: "33333333-3333-3333-3333-333333333333" as UUID,
entityId: ENTITY,
roomId: ROOM,
content: { text: "note to self", ...content },
} as Memory;
}
function ctx(message: Memory): ResponseHandlerEvaluatorContext {
// core.transcription_mode only reads `message`; the rest is unused.
return { message } as unknown as ResponseHandlerEvaluatorContext;
}
describe("transcriptionModeActive", () => {
it("reads the flag from content.metadata (client transport path)", () => {
expect(
transcriptionModeActive(msg({ metadata: { transcriptionMode: true } })),
).toBe(true);
});
it("reads the flag from content top-level (in-process path)", () => {
expect(transcriptionModeActive(msg({ transcriptionMode: true }))).toBe(
true,
);
});
it("is false without the flag, and for a non-true value", () => {
expect(transcriptionModeActive(msg({}))).toBe(false);
expect(
transcriptionModeActive(msg({ metadata: { transcriptionMode: "yes" } })),
).toBe(false);
expect(
transcriptionModeActive(msg({ metadata: { transcriptionMode: false } })),
).toBe(false);
});
});
describe("core.transcription_mode evaluator", () => {
const evaluator = BUILTIN_RESPONSE_HANDLER_EVALUATORS.find(
(e) => e.name === "core.transcription_mode",
);
it("is registered as a builtin response-handler evaluator", () => {
expect(evaluator).toBeDefined();
});
it("runs only when transcription mode is active (any channel)", async () => {
if (!evaluator) throw new Error("evaluator missing");
// DM channel with the flag → suppress.
expect(
await evaluator.shouldRun(
ctx(
msg({
channelType: ChannelType.DM,
metadata: { transcriptionMode: true },
}),
),
),
).toBe(true);
// VOICE_DM channel with the flag → suppress.
expect(
await evaluator.shouldRun(
ctx(
msg({ channelType: ChannelType.VOICE_DM, transcriptionMode: true }),
),
),
).toBe(true);
// No flag → does not run (agent replies normally).
expect(
await evaluator.shouldRun(ctx(msg({ channelType: ChannelType.DM }))),
).toBe(false);
});
it("suppresses the reply but keeps the turn (IGNORE + clearReply, no tool)", async () => {
if (!evaluator) throw new Error("evaluator missing");
const result = await evaluator.evaluate(
ctx(msg({ metadata: { transcriptionMode: true } })),
);
expect(result.processMessage).toBe("IGNORE");
expect(result.clearReply).toBe(true);
expect(result.requiresTool).toBe(false);
});
});
@@ -0,0 +1,109 @@
/**
* Regression coverage for the cron parser in
* `packages/core/src/services/triggerScheduling.ts`.
*
* The parser had zero tests, and `parseCronPart` historically dropped the step
* in the `N/step` shorthand (e.g. `5/15`) — it fired only at minute N instead of
* `N, N+step, …` up to the field max. Any user/LLM cron using that common form
* silently mis-scheduled. These tests lock the standard-cron semantics in.
*/
import { describe, expect, it } from "vitest";
import {
computeNextCronRunAtMs,
parseCronExpression,
} from "../services/triggerScheduling";
/** Sorted array view of a parsed cron field's value set. */
function sorted(set: Set<number> | undefined): number[] {
return set ? [...set].sort((a, b) => a - b) : [];
}
describe("parseCronExpression — minute field semantics", () => {
it("expands `N/step` from N to the field max (the regression)", () => {
const schedule = parseCronExpression("5/15 * * * *");
expect(schedule).not.toBeNull();
// The bug produced just [5]; correct cron is 5, 20, 35, 50.
expect(sorted(schedule?.minute)).toEqual([5, 20, 35, 50]);
});
it("expands `*/step` across the whole range", () => {
expect(sorted(parseCronExpression("*/15 * * * *")?.minute)).toEqual([
0, 15, 30, 45,
]);
});
it("expands `range/step`", () => {
expect(sorted(parseCronExpression("10-30/10 * * * *")?.minute)).toEqual([
10, 20, 30,
]);
});
it("keeps comma lists", () => {
expect(sorted(parseCronExpression("1,2,3 * * * *")?.minute)).toEqual([
1, 2, 3,
]);
});
it("treats a bare value as a single match (no step expansion)", () => {
expect(sorted(parseCronExpression("5 * * * *")?.minute)).toEqual([5]);
});
it("expands an inclusive range", () => {
expect(sorted(parseCronExpression("3-6 * * * *")?.minute)).toEqual([
3, 4, 5, 6,
]);
});
});
describe("parseCronExpression — validation", () => {
it("rejects the wrong field count", () => {
expect(parseCronExpression("* * * *")).toBeNull();
expect(parseCronExpression("* * * * * *")).toBeNull();
});
it("rejects out-of-range values", () => {
expect(parseCronExpression("60 * * * *")).toBeNull(); // minute max is 59
expect(parseCronExpression("* 24 * * *")).toBeNull(); // hour max is 23
});
it("rejects non-numeric / malformed parts", () => {
expect(parseCronExpression("bad * * * *")).toBeNull();
expect(parseCronExpression("5/0 * * * *")).toBeNull(); // zero step
expect(parseCronExpression("")).toBeNull();
});
it("parses a fully-specified expression across all five fields", () => {
const schedule = parseCronExpression("30 9 1 1 1");
expect(schedule).not.toBeNull();
expect(sorted(schedule?.minute)).toEqual([30]);
expect(sorted(schedule?.hour)).toEqual([9]);
expect(sorted(schedule?.dayOfMonth)).toEqual([1]);
expect(sorted(schedule?.month)).toEqual([1]);
expect(sorted(schedule?.dayOfWeek)).toEqual([1]);
});
});
describe("computeNextCronRunAtMs — behavioral", () => {
// Fixed UTC instant: 2026-01-01T00:00:00Z (deterministic; no Date.now()).
const from = Date.UTC(2026, 0, 1, 0, 0, 0);
it("lands the next run on a `N/step` minute boundary", () => {
const next = computeNextCronRunAtMs("5/15 * * * *", from, "UTC");
expect(next).not.toBeNull();
// First boundary after 00:00 is 00:05.
expect(next).toBe(Date.UTC(2026, 0, 1, 0, 5, 0));
expect([5, 20, 35, 50]).toContain(new Date(next as number).getUTCMinutes());
});
it("advances strictly past the from-time", () => {
// from is exactly on a matching minute; the next run must be the following one.
const onBoundary = Date.UTC(2026, 0, 1, 0, 5, 0);
const next = computeNextCronRunAtMs("5/15 * * * *", onBoundary, "UTC");
expect(next).toBe(Date.UTC(2026, 0, 1, 0, 20, 0));
});
it("returns null for an invalid expression", () => {
expect(computeNextCronRunAtMs("nonsense", from, "UTC")).toBeNull();
});
});
@@ -0,0 +1,195 @@
/**
* C9 (#8786): multi-agent VOICE_GROUP turn-taking through the Stage-1 pipeline.
*
* The existing `voice-group-room.test.ts` exercises the `core.voice_group_address`
* evaluator in isolation. This drives the REAL `runV5MessageRuntimeStage1` for
* EACH of several named agents sharing one VOICE_GROUP room, fed the SAME
* utterance addressed to ONE of them ("Eliza, what's the time?" →
* addressedTo:["Eliza"]). It asserts the end-to-end outcome the room contract
* promises:
* - the addressed agent (Eliza) reaches a real reply (not terminal IGNORE),
* - every un-addressed agent terminates with action === "IGNORE" — driven by
* the builtin voice-group address gate inside the live pipeline, not a
* hand-rolled evaluator call.
*
* Each agent is its own Stage-1 run over its own runtime (an agent only ever
* processes a turn as itself). Zero LLM spend: the model is a fixture queue that
* emits the SAME HANDLE_RESPONSE for every agent — only the agent's identity and
* the gate decide who replies, exactly as in a real room where all agents see
* the same transcribed utterance.
*/
import { describe, expect, it, vi } from "vitest";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import type { V5MessageRuntimeStage1Result } from "../services/message";
import { runV5MessageRuntimeStage1 } from "../services/message";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
const ROOM = "11111111-1111-1111-1111-111111111111" as UUID;
const SPEAKER = "22222222-2222-2222-2222-222222222222" as UUID;
const MESSAGE_ID = "33333333-3333-3333-3333-333333333333" as UUID;
const RESPONSE_ID = "00000000-0000-0000-0000-000000000005" as UUID;
const UTTERANCE = "Eliza what's the time";
/** The transcription/extraction resolves the named target of the utterance. */
const ADDRESSED_TO = ["Eliza"];
/** One transcribed voice-group turn, shared by every agent in the room. */
function voiceGroupMessage(): Memory {
return {
id: MESSAGE_ID,
entityId: SPEAKER,
roomId: ROOM,
content: {
text: UTTERANCE,
source: "voice",
channelType: ChannelType.VOICE_GROUP,
},
createdAt: 1,
};
}
function makeState(): State {
return {
values: { availableContexts: "general" },
data: {},
text: "Voice room transcript",
};
}
/**
* The Stage-1 model fixture. The same envelope is emitted for EVERY agent:
* shouldRespond RESPOND + the extracted `addressedTo`. Whether the agent
* actually replies is decided downstream by `core.voice_group_address`, NOT by
* the model — which is the whole point of the room gate.
*/
function respondFixture() {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: "HANDLE_RESPONSE",
arguments: {
shouldRespond: "RESPOND",
thought: "I could answer the time.",
// `simple` keeps the addressed agent on the Stage-1 direct-reply
// path (no planner model call) — the addressed agent answers from
// replyText; the gate, not the planner, decides who is suppressed.
contexts: ["simple"],
intents: [],
candidateActionNames: [],
replyText: "It's 3 o'clock.",
facts: [],
relationships: [],
addressedTo: ADDRESSED_TO,
},
},
],
finishReason: "tool_calls",
};
}
function makeAgentRuntime(agentName: string): IAgentRuntime {
const agentId =
`00000000-0000-0000-0000-0000000000a${agentName.length}` as UUID;
const queue: unknown[] = [respondFixture()];
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
return {
agentId,
character: { name: agentName, system: "Be brief.", bio: "" },
actions: [],
providers: [],
composeState: vi.fn(async () => makeState()),
runActionsByMode: vi.fn(async () => undefined),
emitEvent: vi.fn(async () => undefined),
useModel: vi.fn(async () => {
if (queue.length === 0) throw new Error("Unexpected useModel call");
return queue.shift();
}),
getSetting: vi.fn(() => undefined),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
responseHandlerEvaluators: [],
} as unknown as IAgentRuntime;
}
/** Run one agent's Stage-1 turn for the shared voice-group utterance. */
async function runAgentTurn(
agentName: string,
): Promise<V5MessageRuntimeStage1Result> {
return runV5MessageRuntimeStage1({
runtime: makeAgentRuntime(agentName),
message: voiceGroupMessage(),
state: makeState(),
responseId: RESPONSE_ID,
});
}
function isIgnore(result: V5MessageRuntimeStage1Result): boolean {
return result.kind === "terminal" && result.action === "IGNORE";
}
describe("multi-agent VOICE_GROUP turn-taking through the Stage-1 pipeline (#8786 C9)", () => {
it("only the addressed agent replies; the other agents IGNORE (3-agent room)", async () => {
// A room with three agents — only "Eliza" was addressed.
const eliza = await runAgentTurn("Eliza");
const claude = await runAgentTurn("Claude");
const aria = await runAgentTurn("Aria");
// The addressed agent is NOT suppressed: it reaches a real reply.
expect(isIgnore(eliza)).toBe(false);
expect(
eliza.kind === "direct_reply" || eliza.kind === "planned_reply",
).toBe(true);
// The two un-addressed agents defer via core.voice_group_address.
expect(isIgnore(claude)).toBe(true);
expect(isIgnore(aria)).toBe(true);
});
it("exactly one agent in the room ends up replying", async () => {
const roster = ["Eliza", "Claude", "Aria", "Nova"];
const results = await Promise.all(roster.map((name) => runAgentTurn(name)));
const repliers = roster.filter((_, i) => !isIgnore(results[i]));
// The contract: precisely the single addressed agent replies — no
// cross-talk storm, no silence.
expect(repliers).toEqual(["Eliza"]);
});
it("the SAME utterance as a single-party VOICE_DM is not group-suppressed", async () => {
// VOICE_DM is single-party: the group address gate must not fire, so even
// an agent not named in `addressedTo` still answers normally.
const result = await runV5MessageRuntimeStage1({
runtime: makeAgentRuntime("Claude"),
message: {
...voiceGroupMessage(),
content: {
...voiceGroupMessage().content,
channelType: ChannelType.VOICE_DM,
},
},
state: makeState(),
responseId: RESPONSE_ID,
});
expect(isIgnore(result)).toBe(false);
});
});
@@ -0,0 +1,193 @@
/**
* Multi-agent / multi-speaker voice-room turn-taking (#8786 + #8785).
*
* Contract: in a VOICE_GROUP, an agent DEFERS (IGNORE) when the turn is
* explicitly addressed to another named participant and not to this agent — so
* across ≥3 participants only the addressed agent replies. Undirected turns are
* left to normal shouldRespond (no suppression).
*/
import { describe, expect, it } from "vitest";
import { parseMessageHandlerOutput } from "../runtime/message-handler";
import type { ResponseHandlerEvaluatorContext } from "../runtime/response-handler-evaluators";
import { BUILTIN_RESPONSE_HANDLER_EVALUATORS } from "../services/message";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
const ROOM = "11111111-1111-1111-1111-111111111111" as UUID;
const SPEAKER = "22222222-2222-2222-2222-222222222222" as UUID;
function voiceGroupMsg(): Memory {
return {
id: "33333333-3333-3333-3333-333333333333" as UUID,
entityId: SPEAKER,
roomId: ROOM,
content: {
text: "eliza what's the time",
channelType: ChannelType.VOICE_GROUP,
},
} as Memory;
}
/** A context for an agent named `agentName`, with an extracted `addressedTo`. */
function ctxFor(
agentName: string,
addressedTo: string[],
message: Memory = voiceGroupMsg(),
): ResponseHandlerEvaluatorContext {
return {
runtime: {
agentId: `agent-${agentName.toLowerCase()}` as UUID,
character: { name: agentName },
},
message,
messageHandler: { processMessage: "RESPOND", extract: { addressedTo } },
} as unknown as ResponseHandlerEvaluatorContext;
}
const gate = BUILTIN_RESPONSE_HANDLER_EVALUATORS.find(
(e) => e.name === "core.voice_group_address",
);
describe("core.voice_group_address (multi-agent voice room)", () => {
it("is registered", () => {
expect(gate).toBeDefined();
});
it("only the addressed agent replies across 3 participants", async () => {
if (!gate) throw new Error("missing");
// "Eliza, what's the time" → addressedTo=["Eliza"] in a room with 3 agents.
const addressed = ctxFor("Eliza", ["Eliza"]);
const otherA = ctxFor("Claude", ["Eliza"]);
const otherB = ctxFor("Aria", ["Eliza"]);
// The addressed agent is NOT suppressed → it replies normally.
expect(await gate.shouldRun(addressed)).toBe(false);
// The two un-addressed agents defer.
expect(await gate.shouldRun(otherA)).toBe(true);
expect((await gate.evaluate(otherA)).processMessage).toBe("IGNORE");
expect(await gate.shouldRun(otherB)).toBe(true);
expect((await gate.evaluate(otherB)).processMessage).toBe("IGNORE");
});
it("an undirected turn (empty addressedTo) is left to normal shouldRespond", async () => {
if (!gate) throw new Error("missing");
expect(await gate.shouldRun(ctxFor("Eliza", []))).toBe(false);
expect(await gate.shouldRun(ctxFor("Claude", []))).toBe(false);
});
it("a turn addressed to several agents including self does not suppress self", async () => {
if (!gate) throw new Error("missing");
expect(await gate.shouldRun(ctxFor("Eliza", ["Eliza", "Claude"]))).toBe(
false,
);
// But an agent named neither still defers.
expect(await gate.shouldRun(ctxFor("Aria", ["Eliza", "Claude"]))).toBe(
true,
);
});
it("matches the agent by id as well as by character name", async () => {
if (!gate) throw new Error("missing");
// addressedTo can carry the resolved agent id.
expect(await gate.shouldRun(ctxFor("Eliza", ["agent-eliza"]))).toBe(false);
});
it("does not run for VOICE_DM (single-party) or text messages", async () => {
if (!gate) throw new Error("missing");
const dm = {
...voiceGroupMsg(),
content: { text: "hi claude", channelType: ChannelType.VOICE_DM },
} as Memory;
expect(await gate.shouldRun(ctxFor("Eliza", ["Claude"], dm))).toBe(false);
});
it("fails open (no suppression) when the agent cannot be identified", async () => {
if (!gate) throw new Error("missing");
const ctx = {
runtime: { agentId: "", character: {} },
message: voiceGroupMsg(),
messageHandler: {
processMessage: "RESPOND",
extract: { addressedTo: ["Eliza"] },
},
} as unknown as ResponseHandlerEvaluatorContext;
expect(await gate.shouldRun(ctx)).toBe(false);
});
});
/**
* Integration: drive the REAL Stage-1 parser (`parseMessageHandlerOutput`) so
* `addressedTo` is *derived* from a model HANDLE_RESPONSE envelope rather than
* hand-fed, then feed the parsed extract to the gate across ≥3 agents. This
* closes the parse→decision chain the unit tests above bypass; the only step
* not exercised here is the live model producing the envelope (gated, real-model
* lane). The runtime builds `messageHandler.{processMessage,extract}` from
* exactly this parser output, so the context mapping mirrors production.
*/
describe("core.voice_group_address — parse→gate integration (#8786)", () => {
/** The model's Stage-1 envelope for "Eliza, what's the time" in a 3-agent
* room: RESPOND, with the address resolved to Eliza. */
const ADDRESSED_ENVELOPE = JSON.stringify({
shouldRespond: "RESPOND",
replyText: "It is noon.",
contexts: ["simple"],
addressedTo: ["Eliza"],
});
function ctxFromEnvelope(
agentName: string,
envelope: string,
): ResponseHandlerEvaluatorContext {
const parsed = parseMessageHandlerOutput(envelope);
if (!parsed) throw new Error("envelope failed to parse");
return {
runtime: {
agentId: `agent-${agentName.toLowerCase()}` as UUID,
character: { name: agentName },
},
message: voiceGroupMsg(),
messageHandler: {
processMessage: parsed.processMessage,
extract: parsed.extract ?? {},
},
} as unknown as ResponseHandlerEvaluatorContext;
}
it("derives addressedTo from the real parse, then only the addressed agent replies (≥3 agents)", async () => {
if (!gate) throw new Error("missing");
// The parser actually produced the addressedTo the gate consumes.
const parsed = parseMessageHandlerOutput(ADDRESSED_ENVELOPE);
expect(parsed?.processMessage).toBe("RESPOND");
expect(parsed?.extract?.addressedTo).toEqual(["Eliza"]);
// Eliza (addressed) is not suppressed; Claude + Aria defer.
expect(
await gate.shouldRun(ctxFromEnvelope("Eliza", ADDRESSED_ENVELOPE)),
).toBe(false);
for (const other of ["Claude", "Aria"]) {
const ctx = ctxFromEnvelope(other, ADDRESSED_ENVELOPE);
expect(await gate.shouldRun(ctx)).toBe(true);
expect((await gate.evaluate(ctx)).processMessage).toBe("IGNORE");
}
});
it("an undirected envelope (no addressedTo) leaves every agent to normal shouldRespond", async () => {
if (!gate) throw new Error("missing");
const undirected = JSON.stringify({
shouldRespond: "RESPOND",
replyText: "sure",
contexts: ["simple"],
});
// No addressedTo in the envelope → parser omits it → gate fails open.
expect(parseMessageHandlerOutput(undirected)?.extract?.addressedTo).toBe(
undefined,
);
expect(await gate.shouldRun(ctxFromEnvelope("Eliza", undirected))).toBe(
false,
);
expect(await gate.shouldRun(ctxFromEnvelope("Aria", undirected))).toBe(
false,
);
});
});
@@ -0,0 +1,289 @@
/**
* C5 (#8786): voice == text parity through the Stage-1 message pipeline.
*
* The shipped contract (memory: "voice == text parity, not just the facts
* stage") is that an utterance arriving as a VOICE_DM must traverse the SAME
* planning pipeline as the identical utterance arriving as a text DM — the same
* providers composed, the same RESPONSE_HANDLER/CONTEXT action modes fired, and
* the same response-handler evaluator set run. Voice is a transport, not a
* different brain.
*
* This drives the REAL `runV5MessageRuntimeStage1` (not a unit slice of the
* evaluator) twice over a controlled runtime and records, for each transport:
* - every provider-name list passed to `runtime.composeState`,
* - every action mode passed to `runtime.runActionsByMode`,
* - the response-handler evaluators that ran (BUILTIN_RESPONSE_HANDLER_EVALUATORS).
* Parity holds iff the two transports produce identical sets.
*
* The utterance ("Can you check my calendar?") runs the full Stage-1 path on
* both transports, which is where divergence would actually show up.
* Zero LLM spend: the model is a fixture queue.
*/
import { describe, expect, it, vi } from "vitest";
import { BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS } from "../runtime/builtin-field-evaluators";
import type { ResponseHandlerEvaluator } from "../runtime/response-handler-evaluators";
import { ResponseHandlerFieldRegistry } from "../runtime/response-handler-field-registry";
import {
BUILTIN_RESPONSE_HANDLER_EVALUATORS,
runV5MessageRuntimeStage1,
} from "../services/message";
import { createMockRuntime } from "../testing/mock-runtime";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
import type { IAgentRuntime } from "../types/runtime";
import type { State } from "../types/state";
const ENTITY = "00000000-0000-0000-0000-000000000002" as UUID;
const AGENT = "00000000-0000-0000-0000-000000000003" as UUID;
const ROOM = "00000000-0000-0000-0000-000000000004" as UUID;
const RESPONSE_ID = "00000000-0000-0000-0000-000000000005" as UUID;
const UTTERANCE = "Can you check my calendar?";
function makeMessage(channelType: ChannelType): Memory {
return {
id: "00000000-0000-0000-0000-000000000001" as UUID,
entityId: ENTITY,
agentId: AGENT,
roomId: ROOM,
content: {
text: UTTERANCE,
source: "test",
channelType,
},
createdAt: 1,
};
}
function makeState(): State {
return {
values: { availableContexts: "general, calendar" },
data: {},
text: "Recent conversation summary",
};
}
/**
* Stage-1 HANDLE_RESPONSE that ROUTES INTO THE PLANNER (`requiresTool: true`).
* The planner path is the one that recomposes state with the selected-context
* provider set and fires the CONTEXT_* action modes — i.e. the part of the
* pipeline where a voice/text divergence would actually surface. A pure
* direct-reply (final_reply) would short-circuit before any of that and make
* the comparison vacuous.
*/
function respondFixture() {
return {
text: "",
toolCalls: [
{
id: "handle-response-1",
name: "HANDLE_RESPONSE",
arguments: {
shouldRespond: "RESPOND",
thought: "Inspect the calendar before answering.",
contexts: ["general"],
intents: [],
candidateActionNames: [],
replyText: "",
facts: [],
relationships: [],
addressedTo: [],
requiresTool: true,
},
},
],
finishReason: "tool_calls",
};
}
/** The planner's single response — no tool call, just the final reply. */
function plannerFixture() {
return JSON.stringify({
thought: "Calendar looked up; reply.",
toolCalls: [],
messageToUser: "Here's your calendar.",
});
}
/**
* What a single Stage-1 run touched. Sorted for order-independent comparison —
* parity is about WHICH providers / action modes run and WHAT the evaluator
* stage observed, not the order the fixtures happened to be recorded in.
*
* `evaluators` is the canonical builtin response-handler evaluator set the
* pipeline ran (`BUILTIN_RESPONSE_HANDLER_EVALUATORS`); `evaluatorChannelSeen`
* is the channelType a real registered probe evaluator observed when the
* evaluator stage executed — proving the stage genuinely ran for this transport
* (not that we asserted a constant).
*/
type PipelineTrace = {
providers: string[];
actionModes: string[];
evaluators: string[];
evaluatorChannelSeen: string[];
};
function makeTracingRuntime(): {
runtime: IAgentRuntime;
trace: PipelineTrace;
} {
const trace: PipelineTrace = {
providers: [],
actionModes: [],
evaluators: BUILTIN_RESPONSE_HANDLER_EVALUATORS.map((e) => e.name),
evaluatorChannelSeen: [],
};
const queue: unknown[] = [respondFixture(), plannerFixture()];
const responseHandlerFieldRegistry = new ResponseHandlerFieldRegistry();
for (const evaluator of BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS) {
responseHandlerFieldRegistry.register(evaluator);
}
// A real registered response-handler evaluator. `runResponseHandlerEvaluators`
// merges `runtime.responseHandlerEvaluators` with the builtin set and runs
// them, so this probe FIRES inside the live evaluator stage and records the
// channelType it saw — observable proof the stage executed for this turn.
const probe: ResponseHandlerEvaluator = {
name: "test.parity_probe",
priority: 1,
shouldRun: () => true,
evaluate: (ctx) => {
trace.evaluatorChannelSeen.push(
String(ctx.message.content?.channelType ?? "none"),
);
return undefined;
},
};
const runtime = createMockRuntime({
agentId: AGENT,
character: {
name: "Test Agent",
system: "You are concise.",
bio: "I help with calendars.",
},
actions: [],
// A couple of context-gated providers so the planner has a real provider
// set to select + recompose (the recompose is where channel-aware
// divergence would show up).
providers: [
{ name: "CHARACTER", contexts: ["general"], get: vi.fn() },
{ name: "RECENT_MESSAGES", contexts: ["general"], get: vi.fn() },
],
composeState: vi.fn(async (_message: Memory, providers?: string[]) => {
if (Array.isArray(providers)) {
for (const name of providers) trace.providers.push(name);
}
return makeState();
}),
runActionsByMode: vi.fn(async (mode: string) => {
trace.actionModes.push(mode);
return undefined;
}),
emitEvent: vi.fn(async () => undefined),
useModel: vi.fn(async () => {
if (queue.length === 0) throw new Error("Unexpected useModel call");
return queue.shift();
}),
getSetting: vi.fn(() => undefined),
logger: {
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
trace: vi.fn(),
},
responseHandlerFieldRegistry,
responseHandlerFieldEvaluators: [
...BUILTIN_RESPONSE_HANDLER_FIELD_EVALUATORS,
],
responseHandlerEvaluators: [probe],
});
return { runtime, trace };
}
function uniqueSorted(values: string[]): string[] {
return [...new Set(values)].sort();
}
async function tracePipeline(channelType: ChannelType): Promise<PipelineTrace> {
const { runtime, trace } = makeTracingRuntime();
const result = await runV5MessageRuntimeStage1({
runtime,
message: makeMessage(channelType),
state: makeState(),
responseId: RESPONSE_ID,
});
// Both transports must reach a real reply (RESPOND), never a terminal IGNORE
// — otherwise "parity" would be the trivial parity of two short-circuits.
expect(
result.kind === "direct_reply" || result.kind === "planned_reply",
).toBe(true);
return {
providers: uniqueSorted(trace.providers),
actionModes: uniqueSorted(trace.actionModes),
evaluators: uniqueSorted(trace.evaluators),
evaluatorChannelSeen: trace.evaluatorChannelSeen,
};
}
describe("voice == text parity through the Stage-1 message pipeline (#8786 C5)", () => {
it("a VOICE_DM and a text DM of the same utterance run the IDENTICAL providers", async () => {
const text = await tracePipeline(ChannelType.DM);
const voice = await tracePipeline(ChannelType.VOICE_DM);
// Non-trivial: the pipeline really composed providers (not an empty set
// that would make any comparison pass vacuously).
expect(text.providers.length).toBeGreaterThan(0);
expect(voice.providers).toEqual(text.providers);
});
it("runs the IDENTICAL RESPONSE_HANDLER / CONTEXT action modes for voice and text", async () => {
const text = await tracePipeline(ChannelType.DM);
const voice = await tracePipeline(ChannelType.VOICE_DM);
expect(text.actionModes.length).toBeGreaterThan(0);
// The Stage-1 action-mode contract: response-handler hooks fire for both.
expect(text.actionModes).toEqual(
expect.arrayContaining(["RESPONSE_HANDLER_BEFORE"]),
);
expect(voice.actionModes).toEqual(text.actionModes);
});
it("runs the IDENTICAL response-handler evaluator set, and the stage really executes for both", async () => {
const text = await tracePipeline(ChannelType.DM);
const voice = await tracePipeline(ChannelType.VOICE_DM);
// The voice-group address gate is part of the builtin set that runs for
// both transports (it self-gates to VOICE_GROUP, but the SET that runs is
// identical — parity is about the pipeline, not the gate's per-turn verdict).
expect(text.evaluators.length).toBeGreaterThan(0);
expect(text.evaluators).toContain("core.voice_group_address");
expect(voice.evaluators).toEqual(text.evaluators);
// The probe fired in BOTH runs — the evaluator stage genuinely executed
// for text and for voice — and each saw its own transport's channelType.
expect(text.evaluatorChannelSeen).toEqual([ChannelType.DM]);
expect(voice.evaluatorChannelSeen).toEqual([ChannelType.VOICE_DM]);
});
it("produces one combined parity verdict across providers + actions + evaluator set", async () => {
const text = await tracePipeline(ChannelType.DM);
const voice = await tracePipeline(ChannelType.VOICE_DM);
// The channel-specific observation differs by design (each saw its own
// transport); the pipeline SHAPE — providers, action modes, evaluator set
// — must be identical.
expect({
providers: voice.providers,
actionModes: voice.actionModes,
evaluators: voice.evaluators,
}).toEqual({
providers: text.providers,
actionModes: text.actionModes,
evaluators: text.evaluators,
});
});
});
@@ -0,0 +1,86 @@
/**
* The `core.voice_turn_signal_confirm` builtin response-handler evaluator: the
* positive-decision path that promotes a Stage-1 IGNORE to RESPOND on an explicit
* server agentShouldSpeak signal, while never overriding a STOP, an existing
* RESPOND, or a user-next turn. Deterministic — driven with hand-built voice
* Memory + handler-decision contexts, no model.
*/
import { describe, expect, it } from "vitest";
import type { ResponseHandlerEvaluatorContext } from "../runtime/response-handler-evaluators";
import { BUILTIN_RESPONSE_HANDLER_EVALUATORS } from "../services/message";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
const ROOM = "11111111-1111-1111-1111-111111111111" as UUID;
const ENTITY = "22222222-2222-2222-2222-222222222222" as UUID;
function voiceMsg(signal: Record<string, unknown>): Memory {
return {
id: "33333333-3333-3333-3333-333333333333" as UUID,
entityId: ENTITY,
roomId: ROOM,
content: {
text: "eliza what's the time",
channelType: ChannelType.VOICE_DM,
metadata: { voiceTurnSignal: signal },
},
} as Memory;
}
function ctx(
message: Memory,
processMessage: "RESPOND" | "IGNORE" | "STOP",
): ResponseHandlerEvaluatorContext {
return {
message,
messageHandler: { processMessage },
} as unknown as ResponseHandlerEvaluatorContext;
}
const confirm = BUILTIN_RESPONSE_HANDLER_EVALUATORS.find(
(e) => e.name === "core.voice_turn_signal_confirm",
);
describe("core.voice_turn_signal_confirm (server positive decision)", () => {
it("is registered", () => {
expect(confirm).toBeDefined();
});
it("promotes an IGNORE to RESPOND on an explicit agentShouldSpeak signal", async () => {
if (!confirm) throw new Error("missing");
const message = voiceMsg({
agentShouldSpeak: true,
nextSpeaker: "agent",
endOfTurnProbability: 0.9,
});
expect(await confirm.shouldRun(ctx(message, "IGNORE"))).toBe(true);
const result = await confirm.evaluate(ctx(message, "IGNORE"));
expect(result.processMessage).toBe("RESPOND");
});
it("does NOT override an explicit STOP or an already-RESPOND decision", async () => {
if (!confirm) throw new Error("missing");
const message = voiceMsg({ agentShouldSpeak: true });
expect(await confirm.shouldRun(ctx(message, "STOP"))).toBe(false);
expect(await confirm.shouldRun(ctx(message, "RESPOND"))).toBe(false);
});
it("does not fire without the explicit agentShouldSpeak signal", async () => {
if (!confirm) throw new Error("missing");
// A bare end-of-turn signal is not a positive confirm.
expect(
await confirm.shouldRun(
ctx(voiceMsg({ endOfTurnProbability: 0.9 }), "IGNORE"),
),
).toBe(false);
// user-next overrides confirm.
expect(
await confirm.shouldRun(
ctx(
voiceMsg({ agentShouldSpeak: true, nextSpeaker: "user" }),
"IGNORE",
),
),
).toBe(false);
});
});
@@ -0,0 +1,131 @@
/**
* The `core.voice_turn_signal` suppression gate (#8786): the server veto that
* blocks a voice reply when semantic turn-taking says the next speaker is not the
* agent (agentShouldSpeak false, next speaker user, or end-of-turn probability
* below 0.4), and only on voice channels carrying the signal. Deterministic — the
* builtin evaluator runs against hand-built Memory + handler-decision contexts,
* no model.
*/
import { describe, expect, it } from "vitest";
import type { ResponseHandlerEvaluatorContext } from "../runtime/response-handler-evaluators";
import { BUILTIN_RESPONSE_HANDLER_EVALUATORS } from "../services/message";
import type { Memory } from "../types/memory";
import { ChannelType, type UUID } from "../types/primitives";
const ROOM = "11111111-1111-1111-1111-111111111111" as UUID;
const ENTITY = "22222222-2222-2222-2222-222222222222" as UUID;
function voiceMsg(signal: Record<string, unknown>): Memory {
return {
id: "33333333-3333-3333-3333-333333333333" as UUID,
entityId: ENTITY,
roomId: ROOM,
content: {
text: "eliza what's the time",
channelType: ChannelType.VOICE_DM,
metadata: { voiceTurnSignal: signal },
},
} as Memory;
}
function textMsg(signal: Record<string, unknown>): Memory {
return {
id: "44444444-4444-4444-4444-444444444444" as UUID,
entityId: ENTITY,
roomId: ROOM,
content: {
text: "hello",
channelType: ChannelType.DM,
metadata: { voiceTurnSignal: signal },
},
} as Memory;
}
function ctx(
message: Memory,
processMessage: "RESPOND" | "IGNORE" | "STOP" = "RESPOND",
): ResponseHandlerEvaluatorContext {
return {
message,
messageHandler: { processMessage },
} as unknown as ResponseHandlerEvaluatorContext;
}
const suppress = BUILTIN_RESPONSE_HANDLER_EVALUATORS.find(
(e) => e.name === "core.voice_turn_signal",
);
describe("core.voice_turn_signal (server suppression gate)", () => {
it("is registered", () => {
expect(suppress).toBeDefined();
});
it("suppresses when agentShouldSpeak === false", async () => {
if (!suppress) throw new Error("missing");
const message = voiceMsg({ agentShouldSpeak: false });
expect(await suppress.shouldRun(ctx(message))).toBe(true);
const result = await suppress.evaluate(ctx(message));
expect(result.processMessage).toBe("IGNORE");
expect(result.clearReply).toBe(true);
expect(result.requiresTool).toBe(false);
});
it("suppresses when the next speaker is the user", async () => {
if (!suppress) throw new Error("missing");
expect(
await suppress.shouldRun(ctx(voiceMsg({ nextSpeaker: "user" }))),
).toBe(true);
});
it("suppresses when end-of-turn probability is below 0.4 (user still talking)", async () => {
if (!suppress) throw new Error("missing");
expect(
await suppress.shouldRun(ctx(voiceMsg({ endOfTurnProbability: 0.39 }))),
).toBe(true);
// 0.4 is the boundary — NOT suppressed.
expect(
await suppress.shouldRun(ctx(voiceMsg({ endOfTurnProbability: 0.4 }))),
).toBe(false);
});
it("vetoes even an explicit RESPOND (suppression-only: it can only veto)", async () => {
if (!suppress) throw new Error("missing");
// shouldRun ignores the handler decision — a confident user-next turn is
// suppressed regardless of the Stage-1 RESPOND.
const message = voiceMsg({ nextSpeaker: "user" });
expect(await suppress.shouldRun(ctx(message, "RESPOND"))).toBe(true);
});
it("does NOT suppress a clear agent-next turn", async () => {
if (!suppress) throw new Error("missing");
expect(
await suppress.shouldRun(
ctx(
voiceMsg({
agentShouldSpeak: true,
nextSpeaker: "agent",
endOfTurnProbability: 0.9,
}),
),
),
).toBe(false);
});
it("does NOT run on a non-voice (text) channel even with a suppressing signal", async () => {
if (!suppress) throw new Error("missing");
expect(
await suppress.shouldRun(ctx(textMsg({ nextSpeaker: "user" }))),
).toBe(false);
});
it("does NOT run when no voiceTurnSignal metadata is present", async () => {
if (!suppress) throw new Error("missing");
const bare = {
id: "55555555-5555-5555-5555-555555555555" as UUID,
entityId: ENTITY,
roomId: ROOM,
content: { text: "hi", channelType: ChannelType.VOICE_DM },
} as Memory;
expect(await suppress.shouldRun(ctx(bare))).toBe(false);
});
});
+126
View File
@@ -0,0 +1,126 @@
/**
* Unit tests for buildAccessContext against a deterministic fake runtime (no live
* model or DB): it must resolve the requester's identity, world, and role by
* running role resolution against the SAME world resolveWorldForMessage picks, so
* worldId, role, and isOwner always agree on one world. Outside a world (no
* resolvable worldId) role/owner are undefined — callers must read that as "no
* elevated access", never "unrestricted".
*/
import { describe, expect, it } from "vitest";
import { buildAccessContext } from "./access-context";
import { createUniqueUuid } from "./entities";
import type { IAgentRuntime, Memory, UUID } from "./types";
const AGENT = "00000000-0000-0000-0000-0000000000a9" as UUID;
const USER = "00000000-0000-0000-0000-0000000000u5" as UUID;
const WORLD = "00000000-0000-0000-0000-00000000w012" as UUID;
const ROOM = "00000000-0000-0000-0000-00000000r001" as UUID;
const DISCORD_SERVER_ID = "discord-server-7788";
function runtimeWithRoles(
roles: Record<string, string>,
roomWorldId: UUID | undefined,
// Since #14707, a stored OWNER grant resolves OWNER only when it was made
// deliberately through the role-management gate (roleSources "manual");
// sourceless/legacy grants fold to GUEST. Fixtures granting OWNER must
// model the deliberate shape.
roleSources?: Record<string, string>,
): IAgentRuntime {
return {
agentId: AGENT,
getRoom: async (roomId: UUID) => ({ id: roomId, worldId: roomWorldId }),
getWorld: async (id: UUID) => ({
id,
agentId: AGENT,
serverId: "server-1",
metadata: { roles, ...(roleSources ? { roleSources } : {}) },
}),
getSetting: () => undefined,
getCache: async () => undefined,
getComponents: async () => [],
getEntityById: async () => null,
} as unknown as IAgentRuntime;
}
const message = (source?: string): Memory =>
({
entityId: USER,
roomId: ROOM,
content: source ? { text: "hi", source } : { text: "hi" },
}) as Memory;
const discordMessage = (): Memory =>
({
entityId: USER,
roomId: ROOM,
content: { text: "hi", source: "discord" },
metadata: { discordServerId: DISCORD_SERVER_ID },
}) as unknown as Memory;
describe("buildAccessContext", () => {
it("resolves an OWNER requester with world + source", async () => {
const ctx = await buildAccessContext(
runtimeWithRoles({ [USER]: "OWNER" }, WORLD, { [USER]: "manual" }),
message("discord"),
);
expect(ctx.requesterEntityId).toBe(USER);
expect(ctx.worldId).toBe(WORLD);
expect(ctx.role).toBe("OWNER");
expect(ctx.isOwner).toBe(true);
expect(ctx.source).toBe("discord");
});
it("folds a sourceless (legacy/connector-written) OWNER grant to GUEST (#14707)", async () => {
const ctx = await buildAccessContext(
runtimeWithRoles({ [USER]: "OWNER" }, WORLD),
message("discord"),
);
expect(ctx.role).toBe("GUEST");
expect(ctx.isOwner).toBe(false);
});
it("resolves a plain USER (not owner)", async () => {
const ctx = await buildAccessContext(
runtimeWithRoles({ [USER]: "USER" }, WORLD),
message(),
);
expect(ctx.role).toBe("USER");
expect(ctx.isOwner).toBe(false);
expect(ctx.source).toBeUndefined();
});
it("leaves role/owner undefined outside a world", async () => {
const ctx = await buildAccessContext(
runtimeWithRoles({ [USER]: "OWNER" }, undefined),
message("discord"),
);
expect(ctx.requesterEntityId).toBe(USER);
expect(ctx.worldId).toBeUndefined();
expect(ctx.role).toBeUndefined();
expect(ctx.isOwner).toBeUndefined();
});
it("scopes role to the metadata-resolved world when the room has no worldId", async () => {
// The room carries no worldId, but the Discord server metadata resolves a
// world (resolveWorldForMessage's fallback). Role resolves to OWNER there,
// so worldId MUST be that same world — never undefined. This is the case a
// separate room lookup got wrong: role OWNER with worldId undefined, an
// elevated role with no tenant scope.
const runtime = runtimeWithRoles({ [USER]: "OWNER" }, undefined, {
[USER]: "manual",
});
const expectedWorldId = createUniqueUuid(runtime, DISCORD_SERVER_ID);
const ctx = await buildAccessContext(runtime, discordMessage());
expect(ctx.role).toBe("OWNER");
expect(ctx.isOwner).toBe(true);
expect(ctx.worldId).toBe(expectedWorldId);
expect(ctx.worldId).toBeDefined();
expect(ctx.source).toBe("discord");
});
});
+61
View File
@@ -0,0 +1,61 @@
/**
* Assembles the {@link AccessContext} DTO — requester entity, world scope, role,
* and owner flag — that authorization checks read for a message-driven request.
* A thin composition over `roles.ts`: it runs role resolution against the single
* world `resolveWorldForMessage` selects, so `worldId`, `role`, and `isOwner`
* are always derived together and can never disagree on which world they scope.
*/
import {
getLiveEntityMetadataFromMessage,
resolveEntityRole,
resolveWorldForMessage,
} from "./roles";
import type { AccessContext, IAgentRuntime, Memory } from "./types";
/**
* Build the {@link AccessContext} for a message-driven read: who is asking, in
* which world, and with what role.
*
* `worldId`, `role`, and `isOwner` are resolved together from the SINGLE world
* that {@link resolveWorldForMessage} picks for the message — the room's
* `worldId`, else the connector-metadata fallback (e.g. a Discord server/channel
* id). Deriving all three from one resolution is load-bearing: resolving the
* role against one world while reading `worldId` off a different path can yield
* `role: "OWNER"` with `worldId: undefined` — an elevated role with no tenant
* scope. Outside a world (DMs, or a message with no resolvable world) all three
* are left undefined, which callers must treat as "no elevated access" rather
* than "unrestricted".
*/
export async function buildAccessContext(
runtime: IAgentRuntime,
message: Memory,
): Promise<AccessContext> {
const requesterEntityId = message.entityId;
const source = message.content?.source;
const sourceField = typeof source === "string" ? source : undefined;
const resolved = await resolveWorldForMessage(runtime, message);
if (!resolved) {
return { requesterEntityId, source: sourceField };
}
const { world, metadata } = resolved;
const role = await resolveEntityRole(
runtime,
world,
metadata,
requesterEntityId,
{
liveEntityMetadata: getLiveEntityMetadataFromMessage(message),
liveEntityId: requesterEntityId,
},
);
return {
requesterEntityId,
worldId: world?.id,
role,
isOwner: role === "OWNER",
source: sourceField,
};
}
@@ -0,0 +1,459 @@
/**
* Unit contract for the role-aware artifact disclosure decision (#14778):
* the OWNER/ADMIN-full / USER-grant-driven / fail-closed matrix and the
* untrusted grant parser. Pure functions, no harness.
*/
import { describe, expect, it } from "vitest";
import type { AccessContext, Memory, UUID } from "../types";
import {
artifactDisclosureRecordFromMemory,
canAccessArtifact,
parseArtifactShareGrants,
parseArtifactShareMetadata,
resolveArtifactDisclosure,
resolveArtifactDisclosureForMemory,
selectArtifactVariant,
selectDisclosedArtifactUrl,
} from "./artifact-disclosure";
const AGENT = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" as UUID;
const OWNER = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" as UUID;
const VIEWER = "cccccccc-cccc-4ccc-8ccc-cccccccccccc" as UUID;
const OTHER = "dddddddd-dddd-4ddd-8ddd-dddddddddddd" as UUID;
const typedShareMemory: Pick<Memory, "entityId" | "metadata"> = {
entityId: OTHER,
metadata: {
type: "message",
scope: "owner-private",
share: {
grants: [{ entityId: VIEWER, mode: "redacted" }],
},
},
};
const ctx = (over: Partial<AccessContext> = {}): AccessContext => ({
requesterEntityId: VIEWER,
...over,
});
describe("resolveArtifactDisclosure", () => {
it("no access context (single-owner local boundary) → full", () => {
expect(
resolveArtifactDisclosure({ scope: "owner-private" }, undefined, AGENT),
).toBe("full");
});
it("agent self-read → full", () => {
expect(
resolveArtifactDisclosure(
{ scope: "owner-private" },
ctx({ requesterEntityId: AGENT }),
AGENT,
),
).toBe("full");
});
it("OWNER and ADMIN rank → full regardless of scope or grants", () => {
for (const c of [
ctx({ role: "OWNER", isOwner: true }),
ctx({ role: "ADMIN" }),
]) {
expect(
resolveArtifactDisclosure({ scope: "owner-private" }, c, AGENT),
).toBe("full");
}
});
it("USER with a full grant → full even on owner-private scope", () => {
expect(
resolveArtifactDisclosure(
{
scope: "owner-private",
scopedEntityId: OWNER,
grants: [{ entityId: VIEWER, mode: "full" }],
},
ctx({ role: "USER" }),
AGENT,
),
).toBe("full");
});
it("accepts the typed share metadata contract directly", () => {
expect(
resolveArtifactDisclosure(
{
scope: "owner-private",
scopedEntityId: OWNER,
share: { grants: [{ entityId: VIEWER, mode: "full" }] },
},
ctx({ role: "USER" }),
AGENT,
),
).toBe("full");
});
it("USER with a redacted grant → redacted", () => {
expect(
resolveArtifactDisclosure(
{
scope: "owner-private",
scopedEntityId: OWNER,
grants: [{ entityId: VIEWER, mode: "redacted" }],
},
ctx({ role: "USER" }),
AGENT,
),
).toBe("redacted");
});
it("a redacted grant narrows the viewer even when scope is global", () => {
expect(
resolveArtifactDisclosure(
{
scope: "global",
grants: [{ entityId: VIEWER, mode: "redacted" }],
},
ctx({ role: "USER" }),
AGENT,
),
).toBe("redacted");
});
it("someone else's grant does not disclose to this viewer", () => {
expect(
resolveArtifactDisclosure(
{
scope: "owner-private",
scopedEntityId: OWNER,
grants: [{ entityId: OTHER, mode: "full" }],
},
ctx({ role: "USER" }),
AGENT,
),
).toBe("none");
});
it("ungranted USER falls back to the scope ladder", () => {
// owner-private default fails closed…
expect(
resolveArtifactDisclosure(
{ scope: "owner-private", scopedEntityId: OWNER },
ctx({ role: "USER" }),
AGENT,
),
).toBe("none");
// …global stays readable…
expect(
resolveArtifactDisclosure(
{ scope: "global" },
ctx({ role: "USER" }),
AGENT,
),
).toBe("full");
// …and a user still reads their OWN user-private record.
expect(
resolveArtifactDisclosure(
{ scope: "user-private", scopedEntityId: VIEWER },
ctx({ role: "USER" }),
AGENT,
),
).toBe("full");
expect(
resolveArtifactDisclosure(
{ scope: "user-private", scopedEntityId: OTHER },
ctx({ role: "USER" }),
AGENT,
),
).toBe("none");
});
it("GUEST role collapses to the least-privileged tier: omitted on owner-private", () => {
expect(
resolveArtifactDisclosure(
{ scope: "owner-private", scopedEntityId: OWNER },
ctx({ role: "GUEST" }),
AGENT,
),
).toBe("none");
});
it("unresolved role (no world) fails closed to the USER tier", () => {
expect(
resolveArtifactDisclosure(
{ scope: "owner-private", scopedEntityId: OWNER },
ctx(),
AGENT,
),
).toBe("none");
});
});
describe("canAccessArtifact", () => {
it("returns true for full and redacted disclosures, false for omitted rows", () => {
expect(
canAccessArtifact(
{
scope: "owner-private",
grants: [{ entityId: VIEWER, mode: "full" }],
},
ctx({ role: "USER" }),
AGENT,
),
).toBe(true);
expect(
canAccessArtifact(
{
scope: "owner-private",
grants: [{ entityId: VIEWER, mode: "redacted" }],
},
ctx({ role: "USER" }),
AGENT,
),
).toBe(true);
expect(
canAccessArtifact(
{ scope: "owner-private", scopedEntityId: OWNER },
ctx({ role: "USER" }),
AGENT,
),
).toBe(false);
});
});
describe("selectArtifactVariant", () => {
it("selects full references only for full disclosure", () => {
expect(
selectArtifactVariant("full", {
full: "/api/media/full.wav",
redacted: "/api/media/redacted.txt",
}),
).toEqual({ disclosure: "full", value: "/api/media/full.wav" });
});
it("selects redacted references without falling back to full bytes", () => {
expect(
selectArtifactVariant("redacted", {
full: "/api/media/full.wav",
redacted: "/api/media/redacted.txt",
}),
).toEqual({ disclosure: "redacted", value: "/api/media/redacted.txt" });
expect(
selectArtifactVariant("redacted", {
full: "/api/media/full.wav",
}),
).toBeNull();
});
it("omits artifacts for none disclosure", () => {
expect(
selectArtifactVariant("none", {
full: "/api/media/full.wav",
redacted: "/api/media/redacted.txt",
}),
).toBeNull();
});
});
describe("parseArtifactShareGrants", () => {
it("parses well-formed grants and preserves issuer fields", () => {
const grants = parseArtifactShareGrants({
share: {
grants: [
{
entityId: VIEWER,
mode: "redacted",
grantedBy: OWNER,
grantedAtMs: 123,
},
],
},
});
expect(grants).toEqual([
{
entityId: VIEWER,
mode: "redacted",
grantedBy: OWNER,
grantedAtMs: 123,
},
]);
});
it("drops malformed entries instead of granting anything", () => {
const grants = parseArtifactShareGrants({
share: {
grants: [
{ entityId: "not-a-uuid", mode: "full" },
{ entityId: VIEWER, mode: "everything" },
{ entityId: VIEWER },
"garbage",
null,
{ entityId: VIEWER, mode: "full" },
],
},
});
expect(grants).toEqual([{ entityId: VIEWER, mode: "full" }]);
});
it("returns empty for absent/malformed share metadata", () => {
expect(parseArtifactShareGrants(undefined)).toEqual([]);
expect(parseArtifactShareGrants({})).toEqual([]);
expect(parseArtifactShareGrants({ share: "nope" })).toEqual([]);
expect(parseArtifactShareGrants({ share: { grants: "nope" } })).toEqual([]);
});
});
describe("artifactDisclosureRecordFromMemory", () => {
it("types top-level memory.metadata.share for referencing records", () => {
expect(artifactDisclosureRecordFromMemory(typedShareMemory)).toEqual({
scope: "owner-private",
scopedEntityId: OTHER,
grants: [{ entityId: VIEWER, mode: "redacted" }],
});
});
it("normalizes stored metadata into a disclosure record", () => {
const memory = {
entityId: OTHER,
metadata: {
scope: "user-private",
scopedToEntityId: VIEWER,
share: {
grants: [{ entityId: OTHER, mode: "redacted" }],
roomSnapshot: {
roomId: "eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee",
entityIds: [VIEWER],
atMs: 123,
},
},
},
} as Pick<Memory, "entityId" | "metadata">;
expect(artifactDisclosureRecordFromMemory(memory)).toEqual({
scope: "user-private",
scopedEntityId: VIEWER,
grants: [{ entityId: OTHER, mode: "redacted" }],
});
});
it("fails closed on malformed scope and grants", () => {
const memory = {
entityId: OTHER,
metadata: {
scope: "public-to-everyone",
scopedToEntityId: "not-a-uuid",
share: {
grants: [{ entityId: VIEWER, mode: "everything" }],
},
},
} as Pick<Memory, "entityId" | "metadata">;
expect(artifactDisclosureRecordFromMemory(memory)).toEqual({
scope: "owner-private",
scopedEntityId: OTHER,
grants: [],
});
});
it("resolves disclosure from a memory row using stored share metadata", () => {
const memory = {
entityId: OWNER,
metadata: {
scope: "owner-private",
share: {
grants: [{ entityId: VIEWER, mode: "redacted" }],
},
},
} as Pick<Memory, "entityId" | "metadata">;
expect(
resolveArtifactDisclosureForMemory(memory, ctx({ role: "USER" }), AGENT),
).toBe("redacted");
});
});
describe("selectDisclosedArtifactUrl", () => {
it("returns the original URL for full disclosure", () => {
expect(
selectDisclosedArtifactUrl("full", {
fullUrl: "/api/media/original.wav",
redactedUrl: "/api/media/redacted.wav",
}),
).toEqual({
disclosure: "full",
url: "/api/media/original.wav",
redacted: false,
});
});
it("returns the redacted URL for redacted disclosure", () => {
expect(
selectDisclosedArtifactUrl("redacted", {
fullUrl: "/api/media/original.wav",
redactedUrl: "/api/media/redacted.wav",
}),
).toEqual({
disclosure: "redacted",
url: "/api/media/redacted.wav",
redacted: true,
});
});
it("omits redacted disclosure when no redacted variant exists", () => {
expect(
selectDisclosedArtifactUrl("redacted", {
fullUrl: "/api/media/original.wav",
}),
).toBeNull();
});
it("omits none or missing original URL", () => {
expect(
selectDisclosedArtifactUrl("none", {
fullUrl: "/api/media/original.wav",
}),
).toBeNull();
expect(selectDisclosedArtifactUrl("full", {})).toBeNull();
});
});
describe("parseArtifactShareMetadata", () => {
it("parses grants and the room snapshot contract", () => {
expect(
parseArtifactShareMetadata({
share: {
grants: [{ entityId: VIEWER, mode: "full" }],
roomSnapshot: {
roomId: OWNER,
entityIds: [VIEWER, OTHER],
atMs: 456,
},
},
}),
).toEqual({
grants: [{ entityId: VIEWER, mode: "full" }],
roomSnapshot: {
roomId: OWNER,
entityIds: [VIEWER, OTHER],
atMs: 456,
},
});
});
it("drops malformed room snapshots without dropping valid grants", () => {
expect(
parseArtifactShareMetadata({
share: {
grants: [{ entityId: VIEWER, mode: "redacted" }],
roomSnapshot: {
roomId: OWNER,
entityIds: [VIEWER, "not-a-uuid"],
atMs: 789,
},
},
}),
).toEqual({
grants: [{ entityId: VIEWER, mode: "redacted" }],
});
});
});
@@ -0,0 +1,305 @@
/**
* Role-aware artifact disclosure decision for shared artifacts (transcripts,
* stored files, chat attachments, meeting sessions) — the read-side selector
* behind #14778, designed inside the #8876 attachments doctrine: bytes stay on
* the pre-auth content-addressed store (the sha256 URL is the capability), so
* "permission" here means URL/DTO disclosure on the REFERENCING record, never
* a byte-serve gate. Redacted variants are separate records/media objects; this
* module only decides which variant a viewer's DTO may reference.
*
* One decision, three outcomes: `full` (emit the artifact as stored),
* `redacted` (emit the redacted-variant fields, flagged), `none` (omit the row
* entirely). Every disclosure surface routes through this single function so
* the role matrix — OWNER/ADMIN/agent-self full; USER grant-driven; ungranted
* viewers fall back to the scope ladder (which fails closed on the
* `owner-private` default) — cannot drift per surface.
*
* Grants ride additively on the referencing record's metadata
* (`metadata.share.grants`, jsonb — no migration, no sha256-keyed table per
* doctrine AD1). The grant WRITE path (share actions, room-snapshot capture)
* belongs to the PERM-ACL/PERM-REDACT children of #14749; this module only
* evaluates what is stored. Default-matrix ratification is tracked in #14777 —
* revisit the ordering below if D4/D5 land differently.
*/
import type {
AccessContext,
ArtifactRoomSnapshot,
ArtifactShareGrant,
ArtifactShareGrantMode,
ArtifactShareMetadata,
Memory,
MemoryScope,
UUID,
} from "../types";
import { actorFromAccessContext, canReadScope } from "./filter";
/** What a viewer's DTO may contain for one artifact. */
export type ArtifactDisclosure = "full" | "redacted" | "none";
export type {
ArtifactRoomSnapshot,
ArtifactShareGrant,
ArtifactShareGrantMode,
ArtifactShareMetadata,
};
/** Full and redacted artifact references carried by a DTO-capable record. */
export interface ArtifactVariantReferences<TFull, TRedacted = TFull> {
full: TFull;
redacted?: TRedacted | null;
}
/** The concrete reference a disclosure DTO should emit. */
export interface ResolvedArtifactVariant<T> {
disclosure: Exclude<ArtifactDisclosure, "none">;
value: T;
}
/** Room roster captured with a share so later disclosure can be replayed. */
export type ArtifactShareRoomSnapshot = ArtifactRoomSnapshot;
/** Typed share metadata parsed from a record's jsonb metadata. */
export interface ParsedArtifactShareMetadata {
grants: ArtifactShareGrant[];
roomSnapshot?: ArtifactShareRoomSnapshot;
}
const UUID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* Parse `metadata.share.grants` off an untyped stored record into typed
* grants. Malformed entries are dropped — a grant that cannot be read grants
* NOTHING (fail closed), it never degrades into some default access.
*/
// error-policy:J3 untrusted-input sanitizing — stored jsonb is untyped at rest;
// invalid grant entries yield an explicit empty result, never a fake grant.
export function parseArtifactShareMetadata(
metadata: unknown,
): ParsedArtifactShareMetadata {
if (!metadata || typeof metadata !== "object") return { grants: [] };
const share = (metadata as { share?: unknown }).share;
if (!share || typeof share !== "object") return { grants: [] };
const grants = (share as { grants?: unknown }).grants;
const out: ArtifactShareGrant[] = [];
if (Array.isArray(grants)) {
for (const entry of grants) {
if (!entry || typeof entry !== "object") continue;
const g = entry as Record<string, unknown>;
const entityId = typeof g.entityId === "string" ? g.entityId : "";
const mode = g.mode;
if (!UUID_PATTERN.test(entityId)) continue;
if (mode !== "full" && mode !== "redacted") continue;
out.push({
entityId: entityId as UUID,
mode,
...(typeof g.grantedBy === "string" && UUID_PATTERN.test(g.grantedBy)
? { grantedBy: g.grantedBy as UUID }
: {}),
...(typeof g.grantedAtMs === "number"
? { grantedAtMs: g.grantedAtMs }
: {}),
});
}
}
const roomSnapshot = parseArtifactShareRoomSnapshot(
(share as { roomSnapshot?: unknown }).roomSnapshot,
);
return {
grants: out,
...(roomSnapshot ? { roomSnapshot } : {}),
};
}
export function parseArtifactShareGrants(
metadata: unknown,
): ArtifactShareGrant[] {
return parseArtifactShareMetadata(metadata).grants;
}
function parseArtifactShareRoomSnapshot(
value: unknown,
): ArtifactShareRoomSnapshot | undefined {
if (!value || typeof value !== "object") return undefined;
const snapshot = value as Record<string, unknown>;
const roomId = typeof snapshot.roomId === "string" ? snapshot.roomId : "";
if (!UUID_PATTERN.test(roomId)) return undefined;
const atMs = snapshot.atMs;
if (typeof atMs !== "number") return undefined;
const entityIds = snapshot.entityIds;
if (!Array.isArray(entityIds)) return undefined;
const validEntityIds = entityIds.filter(
(entityId): entityId is UUID =>
typeof entityId === "string" && UUID_PATTERN.test(entityId),
);
if (validEntityIds.length !== entityIds.length) return undefined;
return {
roomId: roomId as UUID,
entityIds: validEntityIds,
atMs,
};
}
/** The disclosure-relevant fields of one artifact-referencing record. */
export interface ArtifactDisclosureRecord {
/** Stored visibility scope (callers normalize; unknown fails closed). */
scope: MemoryScope;
/** Entity the record is scoped to (owner/speaker), for entity-scoped tiers. */
scopedEntityId?: UUID;
/** Parsed share grants from the record's metadata. */
grants?: readonly ArtifactShareGrant[];
/** Full typed share metadata when callers already carry the parsed contract. */
share?: ParsedArtifactShareMetadata | ArtifactShareMetadata;
}
function normalizeScope(value: unknown): MemoryScope {
switch (value) {
case "shared":
case "private":
case "room":
case "global":
case "owner-private":
case "user-private":
case "agent-private":
return value;
default:
return "owner-private";
}
}
function stringUuid(value: unknown): UUID | undefined {
return typeof value === "string" && UUID_PATTERN.test(value)
? (value as UUID)
: undefined;
}
/**
* Normalize one stored memory into the artifact-disclosure record shape.
*
* Storage metadata is jsonb and therefore untrusted at read time. Unknown scope
* values collapse to `owner-private`, and malformed grants/scoped ids are
* ignored, so a corrupt row cannot widen disclosure by accident.
*/
export function artifactDisclosureRecordFromMemory(
memory: Pick<Memory, "entityId" | "metadata">,
): ArtifactDisclosureRecord {
const metadata =
memory.metadata && typeof memory.metadata === "object"
? (memory.metadata as Record<string, unknown>)
: undefined;
const scopedEntityId =
stringUuid(metadata?.scopedToEntityId) ??
stringUuid(metadata?.addedBy) ??
memory.entityId;
return {
scope: normalizeScope(metadata?.scope),
...(scopedEntityId ? { scopedEntityId } : {}),
grants: parseArtifactShareGrants(metadata),
};
}
/**
* Decide what `ctx`'s requester may see of one artifact record.
*
* Tier order (most privileged first):
* 1. No access context → `full`. The single-owner local boundary deliberately
* omits a context (see `RouteHandlerContext.accessContext`), and existing
* unfiltered behavior there is a documented product decision.
* 2. Agent self-read, OWNER, or ADMIN rank → `full`.
* 3. An explicit per-entity grant wins in BOTH directions: a `full` grant
* elevates past the scope ladder, and a `redacted` grant narrows the viewer
* to the variant even when the scope ladder would allow full — the owner's
* per-viewer instruction (e.g. an admin "redact for everyone" pass) must
* not be undone by a coarse `global` scope.
* 4. No grant → the scope ladder (`canReadScope`): a USER still reads global
* records and their own user-private records in full; the `owner-private`
* default fails closed to `none`.
*/
export function resolveArtifactDisclosure(
record: ArtifactDisclosureRecord,
ctx: AccessContext | undefined,
agentId: UUID,
): ArtifactDisclosure {
if (!ctx) return "full";
if (ctx.requesterEntityId === agentId) return "full";
const actor = actorFromAccessContext(ctx, agentId);
if (actor.role !== "USER") return "full";
const grants = record.grants ?? record.share?.grants;
const grant = grants?.find((g) => g.entityId === ctx.requesterEntityId);
if (grant) return grant.mode === "full" ? "full" : "redacted";
return canReadScope(record.scope, record.scopedEntityId, actor)
? "full"
: "none";
}
/** Resolve artifact disclosure directly from a memory row. */
export function resolveArtifactDisclosureForMemory(
memory: Pick<Memory, "entityId" | "metadata">,
ctx: AccessContext | undefined,
agentId: UUID,
): ArtifactDisclosure {
return resolveArtifactDisclosure(
artifactDisclosureRecordFromMemory(memory),
ctx,
agentId,
);
}
export interface ArtifactVariantUrls {
fullUrl?: string | null;
redactedUrl?: string | null;
}
export interface DisclosedArtifactUrl {
disclosure: Exclude<ArtifactDisclosure, "none">;
url: string;
redacted: boolean;
}
/**
* Select the URL a DTO may disclose for a resolved artifact decision.
*
* A redacted grant is fail-closed: if no redacted variant exists yet, callers
* must omit the artifact instead of falling back to the original bytes.
*/
export function selectDisclosedArtifactUrl(
disclosure: ArtifactDisclosure,
urls: ArtifactVariantUrls,
): DisclosedArtifactUrl | null {
if (disclosure === "none") return null;
if (disclosure === "redacted") {
return typeof urls.redactedUrl === "string" && urls.redactedUrl.length > 0
? { disclosure, url: urls.redactedUrl, redacted: true }
: null;
}
return typeof urls.fullUrl === "string" && urls.fullUrl.length > 0
? { disclosure, url: urls.fullUrl, redacted: false }
: null;
}
/** Boolean artifact predicate for disclosure points that only need allow/deny. */
export function canAccessArtifact(
record: ArtifactDisclosureRecord,
ctx: AccessContext | undefined,
agentId: UUID,
): boolean {
return resolveArtifactDisclosure(record, ctx, agentId) !== "none";
}
/**
* Select the concrete full/redacted reference for a DTO after the caller has
* resolved disclosure. A redacted grant never falls back to full bytes; if no
* redacted variant exists the artifact is omitted until the variant writer
* catches up.
*/
export function selectArtifactVariant<TFull, TRedacted = TFull>(
disclosure: ArtifactDisclosure,
references: ArtifactVariantReferences<TFull, TRedacted>,
): ResolvedArtifactVariant<TFull | TRedacted> | null {
if (disclosure === "none") return null;
if (disclosure === "full") {
return { disclosure: "full", value: references.full };
}
if (references.redacted == null) return null;
return { disclosure: "redacted", value: references.redacted };
}
@@ -0,0 +1,262 @@
/**
* Exercises the read-side access-control filter — `actorFromAccessContext`,
* `canReadScope`, and `filterByAccessContext` — as pure deterministic
* functions, plus a verbatim cross-check against the documents read ladder.
*/
import { describe, expect, it } from "vitest";
import type { AccessContext, Memory, MemoryScope, UUID } from "../types";
import {
type ActorRole,
actorFromAccessContext,
canReadScope,
filterByAccessContext,
type ScopeActor,
} from "./filter";
const SELF = "00000000-0000-0000-0000-00000000005e" as UUID;
const OTHER = "00000000-0000-0000-0000-0000000000af" as UUID;
const AGENT = "00000000-0000-0000-0000-0000000000a9" as UUID;
const actor = (role: ActorRole, entityId: UUID = SELF): ScopeActor => ({
entityId,
role,
});
describe("actorFromAccessContext", () => {
const ctx = (over: Partial<AccessContext>): AccessContext => ({
requesterEntityId: SELF,
...over,
});
it("maps a self-read (requester is the agent) to AGENT", () => {
expect(
actorFromAccessContext(ctx({ requesterEntityId: AGENT }), AGENT),
).toEqual({ entityId: AGENT, role: "AGENT" });
});
it("maps OWNER, ADMIN, and isOwner to the OWNER tier", () => {
expect(actorFromAccessContext(ctx({ role: "OWNER" }), AGENT).role).toBe(
"OWNER",
);
expect(actorFromAccessContext(ctx({ role: "ADMIN" }), AGENT).role).toBe(
"OWNER",
);
expect(actorFromAccessContext(ctx({ isOwner: true }), AGENT).role).toBe(
"OWNER",
);
});
it("maps USER, GUEST, and an unresolved role to the least-privileged USER tier", () => {
expect(actorFromAccessContext(ctx({ role: "USER" }), AGENT).role).toBe(
"USER",
);
expect(actorFromAccessContext(ctx({ role: "GUEST" }), AGENT).role).toBe(
"USER",
);
expect(actorFromAccessContext(ctx({}), AGENT).role).toBe("USER"); // no world resolved → fails closed
});
});
describe("canReadScope", () => {
it("lets everyone read global / shared / room", () => {
const open: MemoryScope[] = ["global", "shared", "room"];
const roles: ActorRole[] = [
"OWNER",
"ADMIN",
"USER",
"GUEST",
"AGENT",
"RUNTIME",
];
for (const scope of open) {
for (const role of roles) {
expect(canReadScope(scope, undefined, actor(role))).toBe(true);
}
}
});
it("gates owner-private to OWNER and RUNTIME only", () => {
expect(canReadScope("owner-private", undefined, actor("OWNER"))).toBe(true);
expect(canReadScope("owner-private", undefined, actor("RUNTIME"))).toBe(
true,
);
for (const role of ["USER", "GUEST", "ADMIN", "AGENT"] as ActorRole[]) {
expect(canReadScope("owner-private", undefined, actor(role))).toBe(false);
}
});
it("gates agent-private to OWNER, AGENT, and RUNTIME", () => {
for (const role of ["OWNER", "AGENT", "RUNTIME"] as ActorRole[]) {
expect(canReadScope("agent-private", undefined, actor(role))).toBe(true);
}
for (const role of ["USER", "GUEST", "ADMIN"] as ActorRole[]) {
expect(canReadScope("agent-private", undefined, actor(role))).toBe(false);
}
});
describe("user-private / private — entity-scoped ladder", () => {
for (const scope of ["user-private", "private"] as MemoryScope[]) {
it(`${scope}: a USER reads only their own`, () => {
expect(canReadScope(scope, SELF, actor("USER", SELF))).toBe(true);
expect(canReadScope(scope, OTHER, actor("USER", SELF))).toBe(false);
});
it(`${scope}: AGENT and RUNTIME read any`, () => {
expect(canReadScope(scope, OTHER, actor("AGENT", AGENT))).toBe(true);
expect(canReadScope(scope, OTHER, actor("RUNTIME", AGENT))).toBe(true);
});
it(`${scope}: an OWNER reads their own, or another's only via scopedToEntityId`, () => {
expect(canReadScope(scope, SELF, actor("OWNER", SELF))).toBe(true);
expect(canReadScope(scope, OTHER, actor("OWNER", SELF))).toBe(false);
expect(
canReadScope(scope, OTHER, actor("OWNER", SELF), {
scopedToEntityId: OTHER,
}),
).toBe(true);
});
it(`${scope}: a memory with no owning entity is unreadable by anyone (fail closed)`, () => {
// Mirrors canReadDocumentMemory: the `!scopedEntityId` guard returns
// false BEFORE the AGENT/RUNTIME bypass, so even a machine read fails.
for (const role of [
"USER",
"OWNER",
"AGENT",
"RUNTIME",
] as ActorRole[]) {
expect(canReadScope(scope, undefined, actor(role, AGENT))).toBe(
false,
);
}
});
}
});
// Regression guard: for the four DOCUMENT scopes, canReadScope must be
// byte-identical to the documents plugin's canReadDocumentMemory ladder
// (plugins/plugin-documents/src/routes.ts:408-430), so documents can delegate
// here without behavior change.
it("matches the documents read ladder verbatim for document scopes", () => {
const docLadder = (
scope: MemoryScope,
scopedEntityId: UUID | undefined,
a: ScopeActor,
scopedToEntityId?: UUID,
): boolean => {
if (scope === "global") return true;
if (scope === "owner-private")
return a.role === "OWNER" || a.role === "RUNTIME";
if (scope === "agent-private")
return a.role === "OWNER" || a.role === "AGENT" || a.role === "RUNTIME";
if (!scopedEntityId) return false;
if (a.role === "AGENT" || a.role === "RUNTIME") return true;
if (a.role === "OWNER")
return scopedToEntityId
? scopedEntityId === scopedToEntityId
: scopedEntityId === a.entityId;
return scopedEntityId === a.entityId;
};
const docScopes: MemoryScope[] = [
"global",
"owner-private",
"agent-private",
"user-private",
];
const roles: ActorRole[] = ["OWNER", "USER", "AGENT", "RUNTIME"];
const owners: (UUID | undefined)[] = [SELF, OTHER, undefined];
const filters: (UUID | undefined)[] = [undefined, OTHER];
for (const scope of docScopes) {
for (const role of roles) {
for (const owner of owners) {
for (const filter of filters) {
const a = actor(role, SELF);
const opts = filter ? { scopedToEntityId: filter } : undefined;
expect(canReadScope(scope, owner, a, opts)).toBe(
docLadder(scope, owner, a, filter),
);
}
}
}
}
});
});
describe("filterByAccessContext", () => {
const mem = (scope: MemoryScope, owner: UUID): Memory =>
({
entityId: owner,
roomId: SELF,
content: { text: "m" },
metadata: { type: "custom", scope },
}) as Memory;
const corpus: Memory[] = [
mem("global", OTHER),
mem("owner-private", OTHER),
mem("agent-private", OTHER),
mem("user-private", SELF),
mem("user-private", OTHER),
];
const scopesOf = (memories: Memory[]) =>
memories.map((m) => m.metadata?.scope);
it("a USER keeps global + their own user-private only", () => {
const ctx: AccessContext = { requesterEntityId: SELF, role: "USER" };
const visible = filterByAccessContext(corpus, ctx, AGENT);
// global (any owner) + user-private owned by SELF; owner/agent-private and
// OTHER's user-private are dropped.
expect(visible).toHaveLength(2);
expect(scopesOf(visible)).toEqual(["global", "user-private"]);
expect(visible[1]?.entityId).toBe(SELF);
});
it("the agent (self-read) keeps everything except owner-private", () => {
// owner-private is OWNER/RUNTIME only — an AGENT-role read does not see it,
// mirroring the documents ladder. (Background agent reads are unfiltered in
// practice: they pass no accessContext at all.)
const ctx: AccessContext = { requesterEntityId: AGENT };
expect(scopesOf(filterByAccessContext(corpus, ctx, AGENT))).toEqual([
"global",
"agent-private",
"user-private",
"user-private",
]);
});
it("an OWNER keeps global, owner-private, agent-private, and their own user-private", () => {
const ctx: AccessContext = {
requesterEntityId: SELF,
role: "OWNER",
isOwner: true,
};
const visible = filterByAccessContext(corpus, ctx, AGENT);
expect(scopesOf(visible)).toEqual([
"global",
"owner-private",
"agent-private",
"user-private",
]);
expect(visible[3]?.entityId).toBe(SELF);
});
it("is idempotent — filtering twice equals filtering once", () => {
const ctx: AccessContext = { requesterEntityId: SELF, role: "USER" };
const once = filterByAccessContext(corpus, ctx, AGENT);
const twice = filterByAccessContext(once, ctx, AGENT);
expect(twice).toEqual(once);
});
it("defaults a memory with no scope to global (readable)", () => {
const noScope = {
entityId: OTHER,
roomId: SELF,
content: { text: "m" },
} as Memory;
const ctx: AccessContext = { requesterEntityId: SELF, role: "USER" };
expect(filterByAccessContext([noScope], ctx, AGENT)).toHaveLength(1);
});
});
+120
View File
@@ -0,0 +1,120 @@
/**
* Read-side access-control filter for memories: maps an {@link AccessContext}
* to a scope-ladder actor, decides whether that actor may read a memory of a
* given {@link MemoryScope}, and subtractively filters a memory array down to
* the readable set.
*
* Composes with — never duplicates — Postgres RLS: RLS gates on
* `entity_id`/`server_id`, this gates on `metadata.scope`. For the four
* document scopes the ladder is byte-identical to the documents plugin's
* `canReadDocumentMemory`, so that plugin can delegate here without behavior
* change; keep the two in lockstep. An unresolved role fails closed to the
* least-privileged `USER` tier.
*/
import { isAdminRank, type RoleName } from "../roles";
import type { AccessContext, Memory, MemoryScope, UUID } from "../types";
/**
* Read-side actor role: the core {@link RoleName} widened with the machine
* tiers the scope ladder recognizes — `AGENT` (an agent reading its own store)
* and `RUNTIME` (the documents read path that delegates to this ladder).
* {@link actorFromAccessContext} only ever yields `OWNER`/`USER`/`AGENT`;
* `RUNTIME` is supplied by the documents plugin, never minted from a message.
*/
export type ActorRole = RoleName | "AGENT" | "RUNTIME";
export interface ScopeActor {
entityId: UUID;
role: ActorRole;
}
/**
* Collapse an {@link AccessContext} into the scope-ladder actor. A self-read
* (requester is the agent) is `AGENT`; OWNER/ADMIN manage owner-scoped
* memories; everyone else (USER/GUEST, or no role at all — e.g. a DM that
* resolved no world) is `USER`, the least-privileged tier, so an unresolved
* role fails closed rather than open.
*/
export function actorFromAccessContext(
ctx: AccessContext,
agentId: UUID,
): ScopeActor {
if (ctx.requesterEntityId === agentId) {
return { entityId: agentId, role: "AGENT" };
}
if (ctx.isOwner || isAdminRank(ctx.role)) {
return { entityId: ctx.requesterEntityId, role: "OWNER" };
}
return { entityId: ctx.requesterEntityId, role: "USER" };
}
/**
* Whether `actor` may read a memory of the given `scope`. For the four document
* scopes this is byte-equivalent to the documents plugin's `canReadDocumentMemory`
* so that plugin can delegate here without changing behavior. The generic core
* scopes fold in: `shared`/`room` read like `global`, `private` like
* `user-private`. `scopedEntityId` is the memory's owning entity (used only by
* the entity-scoped tiers); `opts.scopedToEntityId` lets an OWNER read on behalf
* of a specific entity, matching the documents filter.
*/
export function canReadScope(
scope: MemoryScope,
scopedEntityId: UUID | undefined,
actor: ScopeActor,
opts?: { scopedToEntityId?: UUID },
): boolean {
switch (scope) {
case "global":
case "shared":
case "room":
return true;
case "owner-private":
return actor.role === "OWNER" || actor.role === "RUNTIME";
case "agent-private":
return (
actor.role === "OWNER" ||
actor.role === "AGENT" ||
actor.role === "RUNTIME"
);
case "user-private":
case "private": {
if (!scopedEntityId) return false;
if (actor.role === "AGENT" || actor.role === "RUNTIME") return true;
if (actor.role === "OWNER") {
return opts?.scopedToEntityId
? scopedEntityId === opts.scopedToEntityId
: scopedEntityId === actor.entityId;
}
return scopedEntityId === actor.entityId;
}
}
}
/**
* Filter memories down to those `ctx`'s requester may read. A pure, strictly
* subtractive `.filter()`: it composes with (never duplicates) Postgres RLS,
* which gates on `entity_id`/`server_id` while this gates on `metadata.scope`.
* Scope defaults to `global` when absent; the owning entity is taken from
* `metadata.scopedToEntityId`, else `metadata.addedBy`, else `memory.entityId`
* (mirroring the documents plugin).
*/
export function filterByAccessContext(
memories: Memory[],
ctx: AccessContext,
agentId: UUID,
): Memory[] {
const actor = actorFromAccessContext(ctx, agentId);
return memories.filter((memory) => {
const scope = memory.metadata?.scope ?? "global";
const meta = memory.metadata as Record<string, unknown> | undefined;
const scopedTo = meta?.scopedToEntityId;
const addedBy = meta?.addedBy;
const scopedEntityId =
typeof scopedTo === "string"
? (scopedTo as UUID)
: typeof addedBy === "string"
? (addedBy as UUID)
: memory.entityId;
return canReadScope(scope, scopedEntityId, actor);
});
}
+192
View File
@@ -0,0 +1,192 @@
/**
* `globalThis` `Symbol.for` account-selection bridges — the single source of
* truth for their symbols and contracts.
*
* The account pool and credential store live in `@elizaos/app-core`; the
* plugins that must select a credentialed account
* (`@elizaos/plugin-anthropic`, `@elizaos/plugin-agent-orchestrator`,
* `@elizaos/plugin-cli-inference`) depend only on `@elizaos/core` and cannot
* import app-core. `runtime.getService(...)` is not viable either — these
* consumers run at spawn/token-resolve time without a runtime handle. So
* app-core publishes a narrow contract on a `globalThis` symbol and the
* plugins read it back.
*
* This module defines each bridge's symbol, contract interface, and typed
* get/set accessors ONCE. The producer (app-core) and every plugin consumer
* import from here so there is exactly one symbol string and one interface per
* bridge. Provider ids cross the `globalThis` boundary as plain strings (they
* round-trip through session metadata as strings), so the contracts type them
* as `string`; the producer validates/narrows to its provider union on entry.
*/
type GlobalBridgeSlot = Record<symbol, unknown>;
function bridgeSlot(): GlobalBridgeSlot | null {
return typeof globalThis === "undefined"
? null
: (globalThis as GlobalBridgeSlot);
}
/* ------------------------- Anthropic subscription ------------------------- */
export const ANTHROPIC_ACCOUNT_POOL_BRIDGE_SYMBOL: unique symbol = Symbol.for(
"eliza.account-pool.anthropic.v1",
);
/**
* Contract consumed by `plugin-anthropic`'s credential store. Kept narrow so
* the plugin never imports the full `AccountPool` surface.
*/
export interface AnthropicAccountPoolBridge {
/** Pick an Anthropic subscription account; null when none are eligible. */
selectAnthropicSubscription(opts?: {
sessionKey?: string;
exclude?: string[];
}): Promise<{ id: string; expiresAt: number } | null>;
/** Resolve an access token for a previously-selected account. */
getAccessToken(
providerId: "anthropic-subscription",
accountId: string,
): Promise<string | null>;
/** Mark health = invalid (e.g. persistent 401 after refresh). */
markInvalid(accountId: string, detail?: string): Promise<void>;
/** Mark health = rate-limited until `untilMs`. */
markRateLimited(
accountId: string,
untilMs: number,
detail?: string,
): Promise<void>;
}
export function getAnthropicAccountPoolBridge(): AnthropicAccountPoolBridge | null {
const slot = bridgeSlot();
if (!slot) return null;
return (
(slot[ANTHROPIC_ACCOUNT_POOL_BRIDGE_SYMBOL] as
| AnthropicAccountPoolBridge
| undefined) ?? null
);
}
/** Install (or, with `null`, clear) the Anthropic bridge. Idempotent. */
export function setAnthropicAccountPoolBridge(
bridge: AnthropicAccountPoolBridge | null,
): void {
const slot = bridgeSlot();
if (!slot) return;
if (bridge) {
slot[ANTHROPIC_ACCOUNT_POOL_BRIDGE_SYMBOL] = bridge;
} else {
delete slot[ANTHROPIC_ACCOUNT_POOL_BRIDGE_SYMBOL];
}
}
/* ------------------------ Coding-agent account pool ----------------------- */
export const CODING_AGENT_SELECTOR_BRIDGE_SYMBOL: unique symbol = Symbol.for(
"eliza.account-pool.coding-agent.v1",
);
export type CodingAccountStrategy =
| "priority"
| "round-robin"
| "least-used"
| "quota-aware";
export interface CodingAccountUsage {
sessionPct?: number;
weeklyPct?: number;
resetsAt?: number;
refreshedAt: number;
}
/** A selected account plus the env the coding subprocess needs to auth as it. */
export interface CodingAgentSelection {
providerId: string;
accountId: string;
label: string;
source: "oauth" | "api-key";
strategy: string;
usage?: CodingAccountUsage;
/** Secrets injected into the spawned subprocess env; never persisted. */
envPatch: Record<string, string>;
}
export interface CodingProviderAvailability {
providerId: string;
total: number;
enabled: number;
healthy: number;
}
/**
* Contract consumed by the orchestrator and cli-inference plugins. The
* producer maps a coding-agent type to candidate providers, pool-selects an
* account, and materializes the subprocess env.
*/
export interface CodingAgentSelectorBridge {
/** Which providers can serve each coding-agent type, with account counts. */
describe(): Record<string, CodingProviderAvailability[]>;
/** Pick an account for a new (or continuing) coding sub-agent. */
select(
agentType: string,
opts?: {
sessionKey?: string;
strategy?: CodingAccountStrategy;
exclude?: string[];
/**
* Restrict selection to these account ids. A continuing session pins
* its follow-up token resolves to the spawn-time account with this —
* pool session-affinity alone expires after a few selects, after which
* a strategy re-pick would silently drift the subprocess onto a
* sibling account while usage and health marks stay keyed to the
* original. Returns null when none of the ids is selectable.
*/
accountIds?: string[];
},
): Promise<CodingAgentSelection | null>;
markRateLimited(
providerId: string,
accountId: string,
untilMs: number,
detail?: string,
): Promise<void>;
markNeedsReauth(
providerId: string,
accountId: string,
detail?: string,
): Promise<void>;
recordUsage(
providerId: string,
accountId: string,
result: {
tokens?: number;
ok: boolean;
model?: string;
latencyMs?: number;
},
): Promise<void>;
}
export function getCodingAgentSelectorBridge(): CodingAgentSelectorBridge | null {
const slot = bridgeSlot();
if (!slot) return null;
return (
(slot[CODING_AGENT_SELECTOR_BRIDGE_SYMBOL] as
| CodingAgentSelectorBridge
| undefined) ?? null
);
}
/** Install (or, with `null`, clear) the coding-agent bridge. Idempotent. */
export function setCodingAgentSelectorBridge(
bridge: CodingAgentSelectorBridge | null,
): void {
const slot = bridgeSlot();
if (!slot) return;
if (bridge) {
slot[CODING_AGENT_SELECTOR_BRIDGE_SYMBOL] = bridge;
} else {
delete slot[CODING_AGENT_SELECTOR_BRIDGE_SYMBOL];
}
}
+201
View File
@@ -0,0 +1,201 @@
/**
* Merges canonical, spec-generated capability docs (`generated/action-docs.ts`)
* into runtime `Action` / `Provider` definitions so the prompt-facing docs are
* complete for every registered capability. The merge is additive and
* conservative: it never overwrites an existing description, similes, or
* parameters, and it always fills `descriptionCompressed` (and the
* parameter-level compressed descriptions) via `compressPromptDescription` —
* matching Python's `compress_prompt_description` — so prompt compression is on
* even for plugins that ship no canonical spec row.
*/
import { allActionDocs, allProviderDocs } from "./generated/action-docs.ts";
import type {
Action,
ActionParameter,
ActionParameterSchema,
JsonValue,
Provider,
} from "./types/index.ts";
import { compressPromptDescription } from "./utils/prompt-compression.ts";
type CompressedDescriptionFields = {
description?: string;
descriptionCompressed?: string;
compressedDescription?: string;
};
function resolveCompressedDescription(
source: CompressedDescriptionFields,
fallbackDescription: string,
canonical?: CompressedDescriptionFields,
): string {
return (
source.descriptionCompressed ??
source.compressedDescription ??
canonical?.descriptionCompressed ??
canonical?.compressedDescription ??
compressPromptDescription(fallbackDescription)
);
}
type ActionDocByName = Record<string, (typeof allActionDocs)[number]>;
const coreActionDocByName: ActionDocByName =
allActionDocs.reduce<ActionDocByName>((acc, doc) => {
acc[doc.name] = doc;
return acc;
}, {});
function cloneDocExampleValue(value: unknown): JsonValue {
return JSON.parse(JSON.stringify(value)) as JsonValue;
}
function cloneActionParameterSchema(
schema: NonNullable<
(typeof allActionDocs)[number]["parameters"]
>[number]["schema"],
): ActionParameterSchema {
const { default: schemaDefault, ...restSchema } = schema;
const properties = schema.properties
? Object.fromEntries(
Object.entries(schema.properties).map(([key, value]) => [
key,
cloneActionParameterSchema(value),
]),
)
: undefined;
return {
...restSchema,
default:
schemaDefault === undefined
? undefined
: cloneDocExampleValue(schemaDefault),
enum: schema.enum ? [...schema.enum] : undefined,
enumValues: schema.enum ? [...schema.enum] : undefined,
properties,
items: schema.items ? cloneActionParameterSchema(schema.items) : undefined,
oneOf: schema.oneOf?.map(cloneActionParameterSchema),
anyOf: schema.anyOf?.map(cloneActionParameterSchema),
};
}
function toActionParameter(
param: NonNullable<(typeof allActionDocs)[number]["parameters"]>[number],
): ActionParameter {
return {
name: param.name,
description: param.description,
descriptionCompressed: resolveCompressedDescription(
param,
param.description,
),
required: param.required,
schema: cloneActionParameterSchema(param.schema),
examples: param.examples?.map(cloneDocExampleValue),
};
}
function ensureParameterCompressed(
parameters: ActionParameter[],
): ActionParameter[] {
return parameters.map((p) => ({
...p,
descriptionCompressed: resolveCompressedDescription(p, p.description),
}));
}
/**
* Merge canonical docs (description/similes/parameters) into an action definition.
*
* This is additive and intentionally conservative:
* - does not overwrite an existing action.description
* - does not overwrite existing action.similes
* - does not overwrite existing action.parameters
*
* Always fills `descriptionCompressed` (and parameter-level compressed descriptions)
* when absent, matching Python `compress_prompt_description` so prompt compression
* is on for every registered action — including plugins with no canonical spec row.
*/
export function withCanonicalActionDocs(action: Action): Action {
const doc = coreActionDocByName[action.name];
const mergedDescription = doc
? action.description || doc.description
: action.description;
const descriptionCompressed = resolveCompressedDescription(
action,
mergedDescription,
doc,
);
if (!doc) {
const parameters =
(action.parameters?.length ?? 0)
? ensureParameterCompressed(action.parameters ?? [])
: action.parameters;
return {
...action,
descriptionCompressed,
parameters,
};
}
const parameters =
action.parameters && action.parameters.length > 0
? ensureParameterCompressed(action.parameters)
: (doc.parameters ?? []).map(toActionParameter);
return {
...action,
description: action.description || doc.description,
descriptionCompressed,
similes:
action.similes && action.similes.length > 0
? action.similes
: doc.similes
? [...doc.similes]
: undefined,
parameters,
};
}
export function withCanonicalActionDocsAll(
actions: readonly Action[],
): Action[] {
return actions.map(withCanonicalActionDocs);
}
type ProviderDocByName = Record<string, (typeof allProviderDocs)[number]>;
const providerDocByName = allProviderDocs.reduce<ProviderDocByName>(
(acc, doc) => {
acc[doc.name] = doc;
return acc;
},
{},
);
export function withCanonicalProviderDocs(provider: Provider): Provider {
const doc = providerDocByName[provider.name];
const description = provider.description || doc?.description || "";
const descriptionCompressed = resolveCompressedDescription(
provider,
description,
doc,
);
return {
...provider,
description: provider.description || doc?.description,
descriptionCompressed,
};
}
export function withCanonicalProviderDocsAll(
providers: readonly Provider[],
): Provider[] {
return providers.map(withCanonicalProviderDocs);
}
+22
View File
@@ -0,0 +1,22 @@
/**
* Deterministic test pinning the reserved action-name constants and the
* canonical ordering of `NON_EXECUTABLE_RESPONSE_ACTION_NAMES`.
*/
import { describe, expect, it } from "vitest";
import {
IGNORE_ACTION_NAME,
NON_EXECUTABLE_RESPONSE_ACTION_NAMES,
NONE_ACTION_NAME,
REPLY_ACTION_NAME,
} from "./action-names";
describe("action name contracts", () => {
it("exports the non-executable response action names in canonical order", () => {
expect(NON_EXECUTABLE_RESPONSE_ACTION_NAMES).toEqual([
REPLY_ACTION_NAME,
NONE_ACTION_NAME,
IGNORE_ACTION_NAME,
]);
});
});
+15
View File
@@ -0,0 +1,15 @@
/**
* Reserved action-name constants and the canonically ordered
* `NON_EXECUTABLE_RESPONSE_ACTION_NAMES` set — the response actions
* (REPLY / NONE / IGNORE) that carry no side-effecting handler.
*/
export const REPLY_ACTION_NAME = "REPLY";
export const NONE_ACTION_NAME = "NONE";
export const IGNORE_ACTION_NAME = "IGNORE";
export const NON_EXECUTABLE_RESPONSE_ACTION_NAMES = [
REPLY_ACTION_NAME,
NONE_ACTION_NAME,
IGNORE_ACTION_NAME,
] as const;
+653
View File
@@ -0,0 +1,653 @@
/**
* Formats the agent's registered actions into prompt text and parses the action
* calls the model returns back into validated parameters. On the render side it
* composes deterministic (seeded-RNG) action examples, compressed
* name/description/parameter listings, and canonical JSON call examples, so a
* given action set always yields the same prompt. On the parse side it turns the
* model's `{ actions, params }` payload into a per-action
* `Map<string, ActionParameters>`, coercing loose scalars/arrays and validating
* each value against the action's parameter schema (type, enum, pattern,
* min/max).
*
* Sits between the action catalog and the message loop's model call, and also
* acts as the barrel re-exporting the action sub-modules (extractor pipeline,
* JSON model output, subaction promotion/dispatch, recent-context,
* resolve-action-args).
*/
import { testSchemaPattern } from "./actions/validate-tool-args.ts";
import { allActionDocs } from "./generated/action-docs.ts";
import type {
Action,
ActionExample,
ActionParameter,
ActionParameterSchema,
ActionParameters,
ActionParameterValue,
JsonValue,
} from "./types";
import {
buildDeterministicSeed,
createDeterministicRandom,
deterministicShuffle,
getDeterministicNames,
} from "./utils/deterministic";
import { compressPromptDescription } from "./utils/prompt-compression";
export {
type ExtractorPipelineResult,
type RunExtractorPipelineArgs,
runExtractorPipeline,
} from "./actions/extractor-pipeline";
export {
parseJsonModelArray,
parseJsonModelOutput,
parseJsonModelRecord,
} from "./actions/json-model-output";
export {
isPromotedSubactionVirtual,
listSubactionsFromParameters,
type PromoteSubactionsOptions,
promoteSubactionsToActions,
type SubactionPromotionOverrides,
} from "./actions/promote-subactions";
export {
recentConversationTexts,
recentConversationTextsFromState,
} from "./actions/recent-context";
export {
type ResolveActionArgsInput,
type ResolveActionArgsResult,
resolveActionArgs,
type SubactionSpec,
type SubactionsMap,
} from "./actions/resolve-action-args";
export {
CANONICAL_SUBACTION_KEY,
DEFAULT_SUBACTION_KEYS,
dispatchSubaction,
normalizeSubaction,
readSubaction,
type SubactionHandler,
type SubactionHandlerMap,
type SubactionParameters,
} from "./actions/subaction-dispatch";
type ActionDocByName = Record<string, (typeof allActionDocs)[number]>;
const actionDocByName: ActionDocByName = allActionDocs.reduce<ActionDocByName>(
(acc, doc) => {
acc[doc.name] = doc;
return acc;
},
{},
);
export const composeActionExamples = (
actionsData: Action[],
count: number,
seed = "actions",
): string => {
if (!actionsData.length || count <= 0) {
return "";
}
const actionsWithExamples = actionsData.filter(
(action) =>
action.examples &&
Array.isArray(action.examples) &&
action.examples.length > 0,
);
if (!actionsWithExamples.length) {
return "";
}
const examplesCopy: ActionExample[][][] = actionsWithExamples.map(
(action) => [...(action.examples || [])],
);
const selectedExamples: ActionExample[][] = [];
const random = createDeterministicRandom(
buildDeterministicSeed(seed, "examples"),
);
const availableActionIndices = examplesCopy
.map((examples, index) => (examples.length > 0 ? index : -1))
.filter((index) => index !== -1);
while (selectedExamples.length < count && availableActionIndices.length > 0) {
const randomIndex = Math.floor(random() * availableActionIndices.length);
const actionIndex = availableActionIndices[randomIndex];
const examples = examplesCopy[actionIndex];
const exampleIndex = Math.floor(random() * examples.length);
selectedExamples.push(examples.splice(exampleIndex, 1)[0]);
if (examples.length === 0) {
availableActionIndices.splice(randomIndex, 1);
}
}
return formatSelectedExamples(
selectedExamples,
buildDeterministicSeed(seed, "names"),
);
};
function formatActionCallExample(example: {
user: string;
actions: readonly string[];
params?: Record<string, Record<string, unknown>>;
}): string {
const paramsByAction = example.params ?? {};
const assistantPayload: Record<string, unknown> = {
actions: [...example.actions],
};
if (Object.keys(paramsByAction).length > 0) {
assistantPayload.params = paramsByAction;
}
return `User: ${example.user}\nAssistant:\n${JSON.stringify(assistantPayload, null, 2)}`;
}
/** Render canonical JSON action-call examples. */
export function composeActionCallExamples(
actionsData: Action[],
maxExamples: number,
): string {
if (!actionsData.length || maxExamples <= 0) return "";
const blocks: string[] = [];
const sorted = [...actionsData].sort((a, b) => a.name.localeCompare(b.name));
for (const action of sorted) {
const doc = actionDocByName[action.name];
if (!doc?.exampleCalls || doc.exampleCalls.length === 0) continue;
for (const ex of doc.exampleCalls) {
blocks.push(formatActionCallExample(ex));
if (blocks.length >= maxExamples) return blocks.join("\n\n");
}
}
return blocks.join("\n\n");
}
const formatSelectedExamples = (
examples: ActionExample[][],
seed = "actions",
): string => {
const MAX_NAME_PLACEHOLDERS = 5;
return examples
.map((example, index) => {
const randomNames = getDeterministicNames(
MAX_NAME_PLACEHOLDERS,
buildDeterministicSeed(seed, index),
);
const conversation = example
.map((message) => {
let messageText = `${message.name}: ${message.content.text}`;
for (let i = 0; i < randomNames.length; i++) {
messageText = messageText.replaceAll(
`{{name${i + 1}}}`,
randomNames[i],
);
}
return messageText;
})
.join("\n");
return `\n${conversation}`;
})
.join("\n");
};
function getExampleActionHints(example: ActionExample[]): string[] {
const hints = new Set<string>();
for (const message of example) {
const content = message.content as {
action?: unknown;
actions?: unknown;
};
if (typeof content.action === "string" && content.action.trim()) {
hints.add(content.action.trim());
}
if (Array.isArray(content.actions)) {
for (const action of content.actions) {
if (typeof action === "string" && action.trim()) {
hints.add(action.trim());
}
}
}
}
return [...hints];
}
function formatPromptScalar(value: unknown): string {
if (value == null) return "null";
if (typeof value === "string") {
return value.replace(/\s+/g, " ").trim();
}
if (typeof value === "number" || typeof value === "boolean") {
return String(value);
}
if (Array.isArray(value)) {
return value
.map((item) => formatPromptScalar(item))
.filter(Boolean)
.join("|");
}
if (typeof value === "object") {
return Object.entries(value as Record<string, unknown>)
.map(([key, entry]) => `${key}:${formatPromptScalar(entry)}`)
.join(",");
}
return String(value);
}
function formatActionExampleSummary(action: Action): string | null {
const examples = action.examples ?? [];
if (!Array.isArray(examples) || examples.length === 0) {
return null;
}
for (const example of examples) {
if (!Array.isArray(example) || example.length === 0) {
continue;
}
const userMessage = example[0]?.content?.text?.trim();
const actionHints = getExampleActionHints(example);
if (!userMessage) {
continue;
}
if (actionHints.length === 0) {
return `User: ${formatPromptScalar(userMessage)} -> actions: ${action.name}`;
}
return `User: ${formatPromptScalar(userMessage)} -> actions: ${actionHints.join(", ")}`;
}
return null;
}
function shuffleActions<T>(items: T[], seed = "actions"): T[] {
return deterministicShuffle(items, seed);
}
function collectActionSimiles(action: Action): string[] {
return [
...new Set(
(action.similes ?? [])
.filter((simile): simile is string => typeof simile === "string")
.map((simile) => simile.trim()),
),
].filter((simile) => simile.length > 0);
}
function collectActionTags(action: Action): string[] {
return [
...new Set(
(action.tags ?? [])
.filter((tag): tag is string => typeof tag === "string")
.map((tag) => tag.trim()),
),
].filter((tag) => tag.length > 0 && tag !== "always-include");
}
function renderCompressedDescription(item: {
description?: string;
descriptionCompressed?: string;
compressedDescription?: string;
}): string {
return (
item.descriptionCompressed ??
item.compressedDescription ??
(item.description ? compressPromptDescription(item.description) : "")
);
}
export function formatActionNames(actions: Action[], seed = "actions"): string {
if (!actions.length) return "";
return shuffleActions(actions, buildDeterministicSeed(seed, "names"))
.map((action) => action.name)
.join(", ");
}
export function formatActions(actions: Action[], seed = "actions"): string {
if (!actions.length) return "";
const actionRows = shuffleActions(
actions,
buildDeterministicSeed(seed, "descriptions"),
).map((action) => ({
name: action.name,
description:
renderCompressedDescription(action) || "No description available",
params:
action.parameters && action.parameters.length > 0
? formatActionParameters(action.parameters)
: "",
aliases: collectActionSimiles(action),
tags: collectActionTags(action),
example: formatActionExampleSummary(action) ?? "",
}));
return JSON.stringify({ actions: actionRows }, null, 2);
}
export function formatActionParameters(parameters: ActionParameter[]): string {
if (!parameters.length) return "";
return parameters
.map((param) => {
const typeStr = formatParameterType(param.schema);
const modifiers: string[] = [];
if (param.schema.enum?.length) {
modifiers.push(`values=${param.schema.enum.join("|")}`);
}
if (param.schema.default !== undefined) {
modifiers.push(`default=${formatPromptScalar(param.schema.default)}`);
}
if (param.examples && param.examples.length > 0) {
modifiers.push(
`examples=${param.examples.map((v) => formatPromptScalar(v)).join("|")}`,
);
}
const suffix = modifiers.length > 0 ? ` [${modifiers.join("; ")}]` : "";
return `${param.name}${param.required ? "" : "?"}:${typeStr}${suffix} - ${renderCompressedDescription(param)}`;
})
.join("; ");
}
function formatParameterType(schema: ActionParameterSchema): string {
if (schema.anyOf?.length) {
return `(${schema.anyOf.map(formatParameterType).join(" | ")})`;
}
if (schema.oneOf?.length) {
return schema.oneOf.map(formatParameterType).join(" | ");
}
const primitiveType = schema.type;
switch (primitiveType) {
case "string":
return "string";
case "number":
return schema.minimum !== undefined || schema.maximum !== undefined
? `number [${schema.minimum ?? "∞"}-${schema.maximum ?? "∞"}]`
: "number";
case "boolean":
return "boolean";
case "array":
return schema.items
? `array of ${formatParameterType(schema.items)}`
: "array";
case "object":
return "object";
case undefined:
return "unknown";
default:
return primitiveType;
}
}
export function parseActionParams(
paramsInput: unknown,
): Map<string, ActionParameters> {
const parsed =
typeof paramsInput === "string"
? parseActionParamsJson(paramsInput)
: (paramsInput ?? null);
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return new Map();
}
const record = parsed as Record<string, unknown>;
const candidate =
record.params &&
typeof record.params === "object" &&
!Array.isArray(record.params)
? (record.params as Record<string, unknown>)
: record;
const result = new Map<string, ActionParameters>();
for (const [actionName, paramsValue] of Object.entries(candidate)) {
if (
!paramsValue ||
typeof paramsValue !== "object" ||
Array.isArray(paramsValue)
) {
continue;
}
const params: ActionParameters = {};
for (const [paramName, paramValue] of Object.entries(paramsValue)) {
params[paramName] = toActionParameterValue(paramValue);
}
if (Object.keys(params).length > 0) {
result.set(actionName.trim().toUpperCase(), params);
}
}
return result;
}
function parseActionParamsJson(input: string): Record<string, unknown> | null {
const trimmed = input.trim();
if (!trimmed) {
return null;
}
try {
const parsed = JSON.parse(trimmed);
return parsed && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: null;
} catch {
return null;
}
}
function toActionParameterValue(value: unknown): ActionParameters[string] {
if (value === null || value === undefined) {
return null;
}
if (
typeof value === "string" ||
typeof value === "number" ||
typeof value === "boolean"
) {
return value as ActionParameterValue;
}
if (Array.isArray(value)) {
return value.map((entry) => toActionParameterValue(entry));
}
if (value && typeof value === "object") {
const normalized: ActionParameters = {};
for (const [key, entry] of Object.entries(value)) {
normalized[key] = toActionParameterValue(entry);
}
return normalized;
}
return value === undefined ? null : String(value);
}
export function validateActionParams(
action: Action,
extractedParams: ActionParameters | undefined,
): { valid: boolean; params: ActionParameters | undefined; errors: string[] } {
const errors: string[] = [];
const params: ActionParameters = {};
if (!action.parameters || action.parameters.length === 0) {
return { valid: true, params: undefined, errors: [] };
}
for (const paramDef of action.parameters) {
const extractedValue = coerceActionParamValue(
paramDef,
extractedParams ? extractedParams[paramDef.name] : undefined,
);
if (extractedValue === undefined || extractedValue === null) {
if (paramDef.required) {
errors.push(
`Required parameter '${paramDef.name}' was not provided for action ${action.name}`,
);
} else if (paramDef.schema.default !== undefined) {
params[paramDef.name] = paramDef.schema.default;
}
} else {
const typeError = validateParamType(paramDef, extractedValue);
if (typeError) {
if (paramDef.required) {
errors.push(typeError);
} else if (paramDef.schema.default !== undefined) {
params[paramDef.name] = paramDef.schema.default;
}
} else {
params[paramDef.name] = extractedValue;
}
}
}
return {
valid: errors.length === 0,
params: Object.keys(params).length > 0 ? params : undefined,
errors,
};
}
function coerceActionParamValue(
paramDef: ActionParameter,
value: ActionParameters[string] | undefined,
): ActionParameters[string] | undefined {
if (
paramDef.schema.type === "string" &&
(typeof value === "number" || typeof value === "bigint")
) {
return String(value);
}
if (paramDef.schema.type !== "array" || Array.isArray(value)) {
return value;
}
if (typeof value !== "string") {
return value;
}
const trimmed = value.trim();
if (!trimmed) {
return [];
}
if (trimmed.startsWith("[") && trimmed.endsWith("]")) {
try {
const parsed = JSON.parse(trimmed);
if (Array.isArray(parsed)) {
return parsed.map((entry) => toActionParameterValue(entry));
}
} catch {
// Fall through to the permissive string coercion path below.
}
}
if (paramDef.schema.items?.type !== "string") {
return value;
}
const SAFE_SPLIT_LIMIT = 10_000;
const safeTrimmed =
trimmed.length > SAFE_SPLIT_LIMIT
? trimmed.slice(0, SAFE_SPLIT_LIMIT)
: trimmed;
const splitValues = safeTrimmed
.split(/\|\||,|\n/)
.map((entry) => entry.trim())
.filter((entry) => entry.length > 0);
if (splitValues.length === 0) {
return [];
}
return splitValues;
}
type ValidatableParamValue =
| ActionParameterValue
| ActionParameters
| ActionParameterValue[]
| ActionParameters[]
| JsonValue;
function validateParamType(
paramDef: ActionParameter,
value: ValidatableParamValue,
): string | undefined {
const { schema, name } = paramDef;
switch (schema.type) {
case "string": {
if (typeof value !== "string") {
return `Parameter '${name}' expected string, got ${typeof value}`;
}
const enumValues = schema.enumValues ?? schema.enum;
if (enumValues && !enumValues.includes(value)) {
return `Parameter '${name}' value '${value}' not in allowed values: ${enumValues.join(", ")}`;
}
if (schema.pattern) {
const result = testSchemaPattern(schema.pattern, value);
if (!result.ok) {
return `Parameter '${name}' value '${value}' ${result.reason}`;
}
}
break;
}
case "number":
if (typeof value !== "number") {
return `Parameter '${name}' expected number, got ${typeof value}`;
}
if (schema.minimum !== undefined && value < schema.minimum) {
return `Parameter '${name}' value ${value} is below minimum ${schema.minimum}`;
}
if (schema.maximum !== undefined && value > schema.maximum) {
return `Parameter '${name}' value ${value} is above maximum ${schema.maximum}`;
}
break;
case "boolean":
if (typeof value !== "boolean") {
return `Parameter '${name}' expected boolean, got ${typeof value}`;
}
break;
case "array":
if (!Array.isArray(value)) {
return `Parameter '${name}' expected array, got ${typeof value}`;
}
break;
case "object":
if (typeof value !== "object" || value === null || Array.isArray(value)) {
return `Parameter '${name}' expected object, got ${typeof value}`;
}
break;
}
return undefined;
}
@@ -0,0 +1,137 @@
/**
* Unit tests for enum short-form completion in
* `runtime/execute-planned-tool-call`: a single-enum-parameter action may
* receive `{ parameters: "<enum>" }`, which is expanded to the canonical
* `{ <param>: "<enum>" }` shape before strict tool-arg validation. Runs on
* hand-built actions and a fake runtime — no live model.
*/
import { describe, expect, it, vi } from "vitest";
import {
executePlannedToolCall,
expandEnumShortForm,
} from "../../runtime/execute-planned-tool-call";
import type { Action, IAgentRuntime, Memory } from "../../types";
function makeAction(overrides: Partial<Action>): Action {
return {
name: "TEST_ENUM_ACTION",
description: "Single-enum-parameter test action",
validate: async () => true,
handler: async () => ({ success: true }),
parameters: [
{
name: "mode",
description: "Operating mode",
required: true,
schema: {
type: "string",
enumValues: ["accept", "decline", "snooze"],
},
},
],
...overrides,
};
}
function makeMessage(): Memory {
return {
id: "msg-1",
entityId: "user-1",
roomId: "room-1",
content: { text: "test" },
} as Memory;
}
function makeRuntime(actions: Action[]): IAgentRuntime {
return {
actions,
logger: {
debug: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
},
} as unknown as IAgentRuntime;
}
describe("Wave 2-D enum short-form completion", () => {
it("expands { parameters: '<enum>' } into the canonical shape", () => {
const action = makeAction({});
const expanded = expandEnumShortForm(action, {
parameters: "accept",
});
// The legacy `parameters` key is dropped after expansion so strict
// validation doesn't reject it as an unknown field.
expect(expanded).toEqual({ mode: "accept" });
});
it("leaves canonical shape untouched", () => {
const action = makeAction({});
const expanded = expandEnumShortForm(action, { mode: "decline" });
expect(expanded).toEqual({ mode: "decline" });
});
it("does nothing when the action has multiple parameters", () => {
const action = makeAction({
parameters: [
{
name: "mode",
description: "mode",
required: true,
schema: { type: "string", enumValues: ["a", "b"] },
},
{
name: "note",
description: "note",
required: false,
schema: { type: "string" },
},
],
});
const expanded = expandEnumShortForm(action, { parameters: "a" });
expect(expanded).toEqual({ parameters: "a" });
});
it("does nothing when the value isn't a valid enum entry", () => {
const action = makeAction({});
const expanded = expandEnumShortForm(action, { parameters: "bogus" });
expect(expanded).toEqual({ parameters: "bogus" });
});
it("expanded short-form passes validation and reaches the handler", async () => {
const handler = vi.fn(async () => ({ success: true, text: "ok" }));
const action = makeAction({ handler });
const runtime = makeRuntime([action]);
const result = await executePlannedToolCall(
runtime,
{ message: makeMessage() },
{ name: "TEST_ENUM_ACTION", params: { parameters: "snooze" } },
);
expect(result.success).toBe(true);
expect(handler).toHaveBeenCalledOnce();
// HandlerOptions.parameters must contain the expanded shape — the
// dispatcher needs to see `mode` to satisfy strict validation.
const handlerCall = handler.mock.calls[0];
const handlerOptions = handlerCall?.[3] as
| { parameters?: Record<string, unknown> }
| undefined;
expect(handlerOptions?.parameters?.mode).toBe("snooze");
});
it("strict validation still rejects non-enum values after expansion", async () => {
const handler = vi.fn();
const action = makeAction({ handler });
const runtime = makeRuntime([action]);
const result = await executePlannedToolCall(
runtime,
{ message: makeMessage() },
{ name: "TEST_ENUM_ACTION", params: { mode: "bogus_value" } },
);
expect(result.success).toBe(false);
expect(String(result.error)).toContain("not one of");
expect(handler).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,387 @@
/**
* Unit tests for `actions/promote-subactions`: per-subaction parameter slicing
* (the `ActionParameter.subactions` applicability list), discriminator
* pinning, and the non-inheritance of `routingHint` / `descriptionCompressed`
* on promoted virtuals. Includes a real-surface footprint regression test
* against the MESSAGE umbrella (58 parameters, 23 subactions) proving the
* planner tools payload shrinks massively while every subaction stays exposed
* with the parameters its handler actually reads. Deterministic — hand-built
* actions plus the real MESSAGE action shape, no live model.
*/
import { describe, expect, it, vi } from "vitest";
import { messageAction } from "../../features/advanced-capabilities/actions/message.ts";
import type { Action, ActionParameter, HandlerOptions } from "../../types";
import { promoteSubactionsToActions } from "../promote-subactions.ts";
import { buildPlannerToolsFromTieredActions } from "../to-tool.ts";
import { validateToolArgs } from "../validate-tool-args.ts";
function makeUmbrella(overrides: Partial<Action> = {}): Action {
return {
name: "WIDGET",
description: "Operate widgets",
descriptionCompressed:
"widget umbrella create read delete keyword stuffed retrieval blurb",
routingHint: "manage widgets -> WIDGET; do NOT use for gadgets -> GADGET",
handler: async () => undefined,
validate: async () => true,
parameters: [
{
name: "action",
description: "Widget operation.",
required: false,
schema: { type: "string", enum: ["create", "read", "delete"] },
},
{
name: "shared",
description: "Applies to every subaction (no applicability list).",
required: false,
schema: { type: "string" },
},
{
name: "title",
description: "Only for create.",
required: false,
subactions: ["create"],
schema: { type: "string" },
},
{
name: "widgetId",
description: "For read and delete.",
required: false,
subactions: ["READ", "Delete"],
schema: { type: "string" },
},
],
...overrides,
};
}
function paramNames(action: Action): string[] {
return (action.parameters ?? []).map((parameter) => parameter.name);
}
function findVirtual(virtuals: readonly Action[], name: string): Action {
const virtual = virtuals.find((entry) => entry.name === name);
if (!virtual) throw new Error(`virtual ${name} not promoted`);
return virtual;
}
describe("promoteSubactionsToActions parameter slicing", () => {
it("keeps parameters without a subactions list on every virtual", () => {
const [, ...virtuals] = promoteSubactionsToActions(makeUmbrella());
for (const virtual of virtuals) {
expect(paramNames(virtual)).toContain("shared");
expect(paramNames(virtual)).toContain("action");
}
});
it("drops parameters whose subactions list excludes the pinned value", () => {
const [, ...virtuals] = promoteSubactionsToActions(makeUmbrella());
const create = findVirtual(virtuals, "WIDGET_CREATE");
const read = findVirtual(virtuals, "WIDGET_READ");
const del = findVirtual(virtuals, "WIDGET_DELETE");
expect(paramNames(create)).toEqual(["action", "shared", "title"]);
// `widgetId` declares ["READ", "Delete"] — matching is normalized, so
// case / separator variants still apply to the right virtuals.
expect(paramNames(read)).toEqual(["action", "shared", "widgetId"]);
expect(paramNames(del)).toEqual(["action", "shared", "widgetId"]);
});
it("pins the discriminator enum and default on each sliced virtual", () => {
const [, ...virtuals] = promoteSubactionsToActions(makeUmbrella());
const read = findVirtual(virtuals, "WIDGET_READ");
const discriminator = (read.parameters ?? []).find(
(parameter) => parameter.name === "action",
);
expect(discriminator?.schema.enum).toEqual(["read"]);
expect(discriminator?.schema.default).toBe("read");
});
it("validates virtual actions with their pinned discriminator injected", async () => {
const seen: unknown[] = [];
const [, ...virtuals] = promoteSubactionsToActions(
makeUmbrella({
validate: async (_runtime, _message, _state, options) => {
seen.push((options as HandlerOptions | undefined)?.parameters);
return (
(
(options as HandlerOptions | undefined)?.parameters as {
action?: string;
}
)?.action === "read"
);
},
}),
);
await expect(
findVirtual(virtuals, "WIDGET_READ").validate?.({} as never, {} as never),
).resolves.toBe(true);
await expect(
findVirtual(virtuals, "WIDGET_DELETE").validate?.(
{} as never,
{} as never,
),
).resolves.toBe(false);
expect(seen).toEqual([
{ action: "read", subaction: "read" },
{ action: "delete", subaction: "delete" },
]);
});
it("defaults virtual validation to true when the parent has no validator", async () => {
const parent = makeUmbrella();
delete (parent as Partial<Action>).validate;
const [, ...virtuals] = promoteSubactionsToActions(parent as Action);
await expect(
findVirtual(virtuals, "WIDGET_CREATE").validate({} as never, {} as never),
).resolves.toBe(true);
});
it("treats an explicit empty subactions list as parent-only", () => {
const umbrella = makeUmbrella({
parameters: [
...(makeUmbrella().parameters ?? []),
{
name: "op",
description: "Planner alias for action (parent-only).",
required: false,
subactions: [],
schema: { type: "string", enum: ["create", "read", "delete"] },
},
],
});
const [parent, ...virtuals] = promoteSubactionsToActions(umbrella);
expect(paramNames(parent as Action)).toContain("op");
for (const virtual of virtuals) {
expect(paramNames(virtual)).not.toContain("op");
}
});
it("never slices the discriminator, even with a stray applicability list", () => {
const umbrella = makeUmbrella();
const parameters = umbrella.parameters as ActionParameter[];
const discriminatorIndex = parameters.findIndex((p) => p.name === "action");
parameters[discriminatorIndex] = {
...parameters[discriminatorIndex],
subactions: ["create"],
} as ActionParameter;
const [, ...virtuals] = promoteSubactionsToActions(umbrella);
const read = findVirtual(virtuals, "WIDGET_READ");
expect(paramNames(read)).toContain("action");
});
it("leaves the parent's parameters untouched", () => {
const umbrella = makeUmbrella();
const [parent] = promoteSubactionsToActions(umbrella);
expect(paramNames(parent as Action)).toEqual([
"action",
"shared",
"title",
"widgetId",
]);
const title = (parent as Action).parameters?.find(
(parameter) => parameter.name === "title",
);
expect(title?.subactions).toEqual(["create"]);
});
it("validates model args against the sliced schema (exposure == validation)", () => {
const [, ...virtuals] = promoteSubactionsToActions(makeUmbrella());
const create = findVirtual(virtuals, "WIDGET_CREATE");
const ok = validateToolArgs(create, { title: "hello" });
expect(ok.valid).toBe(true);
// The pinned discriminator default is filled in for the handler.
expect(ok.args?.action).toBe("create");
const bad = validateToolArgs(create, { widgetId: "w-1" });
expect(bad.valid).toBe(false);
expect(bad.errors.join(" ")).toContain("widgetId");
});
it("dispatch still injects the discriminator for sliced virtuals", async () => {
const handler = vi.fn(async () => undefined);
const [, ...virtuals] = promoteSubactionsToActions(
makeUmbrella({ handler }),
);
const del = findVirtual(virtuals, "WIDGET_DELETE");
await del.handler({} as never, {} as never, undefined, {
parameters: { widgetId: "w-1" },
});
const options = handler.mock.calls[0]?.[3] as HandlerOptions;
expect(options.parameters).toMatchObject({
action: "delete",
subaction: "delete",
widgetId: "w-1",
});
});
});
describe("promoteSubactionsToActions description / hint hygiene", () => {
it("does not copy routingHint onto virtuals; the parent keeps it", () => {
const [parent, ...virtuals] = promoteSubactionsToActions(makeUmbrella());
expect((parent as Action).routingHint).toContain("manage widgets");
for (const virtual of virtuals) {
expect(virtual.routingHint).toBeUndefined();
}
});
it("does not inherit the parent's descriptionCompressed", () => {
const [, ...virtuals] = promoteSubactionsToActions(makeUmbrella());
for (const virtual of virtuals) {
expect(virtual.descriptionCompressed).toBeUndefined();
// Consumers fall back to the composed per-subaction description.
expect(virtual.description).toContain("subaction =");
}
});
it("uses the override descriptionCompressed when provided", () => {
const [, ...virtuals] = promoteSubactionsToActions(makeUmbrella(), {
overrides: {
create: { descriptionCompressed: "create a widget" },
},
});
expect(findVirtual(virtuals, "WIDGET_CREATE").descriptionCompressed).toBe(
"create a widget",
);
expect(
findVirtual(virtuals, "WIDGET_READ").descriptionCompressed,
).toBeUndefined();
});
});
describe("MESSAGE umbrella planner tools footprint (real surface)", () => {
function buildFamilyTools() {
const [parent, ...virtuals] = promoteSubactionsToActions(messageAction);
return {
parent: parent as Action,
virtuals,
tools: buildPlannerToolsFromTieredActions([parent], {
tierAParents: [parent.name],
actionLookup: new Map(virtuals.map((v) => [v.name, v])),
}),
};
}
it("still exposes every subaction as a first-class tool", () => {
const { tools } = buildFamilyTools();
const names = tools.map((tool) => tool.name);
expect(names).toContain("MESSAGE");
for (const op of [
"send",
"read_channel",
"read_with_contact",
"search",
"list_channels",
"list_servers",
"list_connections",
"join",
"leave",
"react",
"edit",
"delete",
"pin",
"get_user",
"triage",
"list_inbox",
"search_inbox",
"draft_reply",
"draft_followup",
"respond",
"send_draft",
"schedule_draft_send",
"manage",
]) {
expect(names).toContain(`MESSAGE_${op.toUpperCase()}`);
}
});
it("keeps the full parameter surface on the parent tool", () => {
const { tools, parent } = buildFamilyTools();
const parentTool = tools.find((tool) => tool.name === "MESSAGE");
expect(Object.keys(parentTool?.parameters.properties ?? {})).toHaveLength(
(parent.parameters ?? []).length,
);
});
it("exposes op-specific parameters only on the relevant virtuals", () => {
const { tools } = buildFamilyTools();
const props = (name: string) =>
Object.keys(
tools.find((tool) => tool.name === name)?.parameters.properties ?? {},
);
const send = props("MESSAGE_SEND");
expect(send).toEqual(
expect.arrayContaining(["message", "attachments", "urgency", "target"]),
);
expect(send).not.toContain("emoji");
expect(send).not.toContain("draftId");
expect(send).not.toContain("worldIds");
const readChannel = props("MESSAGE_READ_CHANNEL");
expect(readChannel).toEqual(
expect.arrayContaining(["from", "until", "to"]),
);
const react = props("MESSAGE_REACT");
expect(react).toEqual(expect.arrayContaining(["emoji", "messageId"]));
expect(react).not.toContain("attachments");
const triage = props("MESSAGE_TRIAGE");
expect(triage).toEqual(
expect.arrayContaining(["sources", "worldIds", "sinceMs"]),
);
expect(triage).not.toContain("message");
expect(triage).not.toContain("emoji");
// list_connections takes no parameters beyond the pinned discriminator.
expect(props("MESSAGE_LIST_CONNECTIONS")).toEqual(["action"]);
});
it("cuts the family tools payload to well under half of the unsliced size", () => {
const { tools } = buildFamilyTools();
const sliced = JSON.stringify(tools).length;
// Counterfactual: the same umbrella with every applicability list
// stripped — the pre-slicing behavior where each virtual duplicates
// the parent's full schema.
const unslicedAction: Action = {
...messageAction,
parameters: (messageAction.parameters ?? []).map(
({ subactions: _subactions, ...parameter }) => parameter,
),
};
const [parent, ...virtuals] = promoteSubactionsToActions(unslicedAction);
const unsliced = JSON.stringify(
buildPlannerToolsFromTieredActions([parent], {
tierAParents: [parent.name],
actionLookup: new Map(virtuals.map((v) => [v.name, v])),
}),
).length;
expect(sliced).toBeLessThan(unsliced * 0.4);
// Absolute guard so the surface cannot silently flood again: the
// measured pre-fix payload for this family was ~163 KB.
expect(sliced).toBeLessThan(60_000);
});
it("virtual tool descriptions no longer duplicate the parent's routing hint and blurb", () => {
const { tools, parent } = buildFamilyTools();
const hint = parent.routingHint ?? "";
expect(hint.length).toBeGreaterThan(0);
const parentTool = tools.find((tool) => tool.name === "MESSAGE");
expect(parentTool?.description).toContain(hint);
for (const tool of tools) {
if (tool.name === "MESSAGE") continue;
expect(tool.description).not.toContain(hint);
expect(tool.description).not.toContain(
parent.descriptionCompressed ?? "<none>",
);
expect(tool.description.length).toBeLessThan(300);
}
});
});
@@ -0,0 +1,107 @@
/**
* Unit tests for `actions/subaction-dispatch`: reading a sub-action
* discriminator from planner args (canonical `action` key plus legacy aliases),
* normalizing op names, and dispatching to the selected handler. Pure
* functions — no runtime or live model.
*/
import { describe, expect, it } from "vitest";
import {
CANONICAL_SUBACTION_KEY,
DEFAULT_SUBACTION_KEYS,
dispatchSubaction,
normalizeSubaction,
readSubaction,
} from "../subaction-dispatch";
describe("subaction-dispatch", () => {
it("documents action as the canonical key with legacy aliases", () => {
expect(CANONICAL_SUBACTION_KEY).toBe("action");
expect(DEFAULT_SUBACTION_KEYS).toEqual([
"action",
"subaction",
"op",
"operation",
"verb",
"subAction",
"__subaction",
]);
});
it("prefers action over legacy aliases when multiple are present", () => {
const allowed = ["list", "create"] as const;
expect(
readSubaction(
{ action: "list", subaction: "create", op: "create" },
{ allowed },
),
).toBe("list");
expect(readSubaction({ subaction: "list" }, { allowed })).toBe("list");
expect(readSubaction({ op: "list" }, { allowed })).toBe("list");
expect(readSubaction({ action: "list" }, { allowed })).toBe("list");
expect(readSubaction({ action: "create" }, { allowed })).toBe("create");
expect(readSubaction({ operation: "create" }, { allowed })).toBe("create");
expect(readSubaction({ verb: "create" }, { allowed })).toBe("create");
});
it("normalizes planner-facing op names", () => {
expect(normalizeSubaction("Search YouTube")).toBe("search_youtube");
expect(normalizeSubaction("play-query")).toBe("play_query");
expect(normalizeSubaction(" ")).toBeUndefined();
expect(normalizeSubaction(null)).toBeUndefined();
});
it("reads op/subaction/action keys with aliases", () => {
const allowed = ["download", "play_query", "search_youtube"] as const;
expect(
readSubaction(
{ action: "play-query" },
{
allowed,
aliases: { play: "play_query", youtube: "search_youtube" },
},
),
).toBe("play_query");
expect(
readSubaction(
{ subaction: "youtube" },
{
allowed,
aliases: { youtube: "search_youtube" },
},
),
).toBe("search_youtube");
expect(readSubaction({}, { allowed, defaultValue: "download" })).toBe(
"download",
);
expect(readSubaction({ op: "unknown" }, { allowed })).toBeUndefined();
});
it("dispatches to the selected handler", async () => {
const result = await dispatchSubaction(
"download",
{
download: async ({ id }: { id: string }) => ({
success: true,
data: { id },
}),
},
{ id: "track-1" },
);
expect(result).toEqual({ success: true, data: { id: "track-1" } });
});
it("returns a structured error for missing handlers", async () => {
const result = await dispatchSubaction(
undefined,
{ download: async () => ({ success: true }) },
undefined,
);
expect(result.success).toBe(false);
expect(result.error).toBe("UNKNOWN_SUBACTION");
});
});
@@ -0,0 +1,527 @@
/**
* Unit tests for `actions/to-tool`: converting an `Action`'s parameters into a
* strict provider-native (function-calling) tool, expanding a Tier-A parent's
* sub-actions into first-class planner tools, the core terminal tools, and the
* HANDLE_RESPONSE tool description. Deterministic — hand-built actions, no live
* model.
*/
import { describe, expect, it, vi } from "vitest";
import type {
Action,
ActionParameter,
ActionParameterSchema,
} from "../../types";
import {
actionToTool,
buildPlannerToolsFromActions,
buildPlannerToolsFromTieredActions,
CORE_PLANNER_TERMINALS,
createHandleResponseTool,
HANDLE_RESPONSE_SCHEMA,
} from "../to-tool.ts";
function makeAction(overrides: Partial<Action>): Action {
return {
name: "TEST_ACTION",
description: "Run the test action",
handler: async () => undefined,
validate: async () => true,
...overrides,
};
}
describe("actionToTool", () => {
it("converts flat action parameters to a strict provider-native tool schema", () => {
const modeParameter = {
name: "mode",
description: "Execution mode",
required: false,
options: [
{ label: "Fast", value: "fast" },
{ label: "Careful", value: "careful" },
],
schema: { type: "string", default: "fast" },
} as ActionParameter & {
options: Array<{ label: string; value: string }>;
};
const action = makeAction({
name: "DOCUMENT",
description: "Search indexed knowledge",
descriptionCompressed: "Search knowledge",
parameters: [
{
name: "query",
description: "Search query",
required: true,
schema: { type: "string" },
},
{
name: "limit",
description: "Maximum number of results",
required: false,
schema: { type: "integer", minimum: 1, maximum: 20, default: 5 },
},
modeParameter,
],
});
const tool = actionToTool(action);
expect(tool).toEqual({
type: "function",
function: {
name: "DOCUMENT",
description: "Search knowledge",
strict: true,
parameters: {
type: "object",
additionalProperties: false,
required: ["query"],
properties: {
query: {
type: "string",
description: "Search query",
},
limit: {
type: "integer",
description: "Maximum number of results",
minimum: 1,
maximum: 20,
default: 5,
},
mode: {
type: "string",
description: "Execution mode",
enum: ["fast", "careful"],
default: "fast",
},
},
},
},
});
});
it("converts nested objects and arrays recursively", () => {
const action = makeAction({
name: "CREATE_TASK",
description: "Create a task",
parameters: [
{
name: "task",
description: "Task payload",
required: true,
schema: {
type: "object",
properties: {
title: {
type: "string",
required: true,
} as ActionParameterSchema,
metadata: {
type: "object",
properties: {
priority: {
type: "string",
enum: ["low", "normal", "high"],
default: "normal",
},
},
},
tags: { type: "array", items: { type: "string" } },
},
},
},
],
});
const schema = actionToTool(action).function.parameters;
expect(schema.properties.task).toMatchObject({
type: "object",
additionalProperties: false,
required: ["title"],
properties: {
title: { type: "string" },
metadata: {
type: "object",
additionalProperties: false,
required: [],
properties: {
priority: {
type: "string",
enum: ["low", "normal", "high"],
default: "normal",
},
},
},
tags: { type: "array", items: { type: "string" } },
},
});
});
it("rejects names that are not strict native tool names", () => {
expect(() => actionToTool(makeAction({ name: "searchDocuments" }))).toThrow(
/Invalid tool name 'searchDocuments'/,
);
expect(() => actionToTool(makeAction({ name: "1_SEARCH" }))).toThrow(
/must match/,
);
});
});
describe("buildPlannerToolsFromTieredActions", () => {
function makeTieredAction(overrides: Partial<Action>): Action {
return makeAction({
parameters: [],
...overrides,
});
}
it("expands sub-actions of a Tier-A parent into first-class tools", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
parameters: [
{
name: "track",
description: "Track id",
required: true,
schema: { type: "string" },
},
],
});
const pauseMusic = makeTieredAction({
name: "PAUSE_MUSIC",
description: "Pause the active track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic, pauseMusic],
});
const tools = buildPlannerToolsFromTieredActions([music], {
tierAParents: new Set(["MUSIC"]),
});
expect(tools.map((tool) => tool.name)).toEqual([
"MUSIC",
"PLAY_MUSIC",
"PAUSE_MUSIC",
]);
// The expanded child carries its own parameter schema, not the parent's.
const playTool = tools.find((tool) => tool.name === "PLAY_MUSIC");
expect(
(playTool?.parameters as { properties?: Record<string, unknown> })
?.properties,
).toMatchObject({ track: { type: "string" } });
});
it("does not expand sub-actions for a Tier-B parent", () => {
const createTask = makeTieredAction({
name: "CREATE_TASK",
description: "Create a task.",
});
const lifeops = makeTieredAction({
name: "LIFEOPS",
description: "Life-ops umbrella parent.",
subActions: [createTask],
});
const tools = buildPlannerToolsFromTieredActions([lifeops], {
// No tierAParents — LIFEOPS is implicitly Tier B.
});
expect(tools.map((tool) => tool.name)).toEqual(["LIFEOPS"]);
});
it("produces a correct combined tool list for mixed Tier A + Tier B parents", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic],
});
const createTask = makeTieredAction({
name: "CREATE_TASK",
description: "Create a task.",
});
const lifeops = makeTieredAction({
name: "LIFEOPS",
description: "Life-ops umbrella parent.",
subActions: [createTask],
});
const tools = buildPlannerToolsFromTieredActions([music, lifeops], {
tierAParents: new Set(["MUSIC"]),
});
expect(tools.map((tool) => tool.name)).toEqual([
"MUSIC",
"PLAY_MUSIC",
"LIFEOPS",
]);
// CREATE_TASK is gated behind the LIFEOPS parent handler — it does NOT
// appear as a first-class tool.
expect(tools.find((tool) => tool.name === "CREATE_TASK")).toBeUndefined();
});
it("resolves string-only sub-action references via actionLookup", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: ["PLAY_MUSIC"],
});
const onUnresolved = vi.fn();
const tools = buildPlannerToolsFromTieredActions([music], {
tierAParents: new Set(["MUSIC"]),
actionLookup: new Map([["PLAY_MUSIC", playMusic]]),
onUnresolvedSubAction: onUnresolved,
});
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC", "PLAY_MUSIC"]);
expect(onUnresolved).not.toHaveBeenCalled();
});
it("skips unresolvable string sub-action references and reports them", () => {
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: ["PLAY_MUSIC", "PAUSE_MUSIC"],
});
const onUnresolved = vi.fn();
const tools = buildPlannerToolsFromTieredActions([music], {
tierAParents: ["MUSIC"],
onUnresolvedSubAction: onUnresolved,
});
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC"]);
expect(onUnresolved).toHaveBeenCalledTimes(2);
expect(onUnresolved).toHaveBeenCalledWith({
parentName: "MUSIC",
subActionName: "PLAY_MUSIC",
});
expect(onUnresolved).toHaveBeenCalledWith({
parentName: "MUSIC",
subActionName: "PAUSE_MUSIC",
});
});
it("dedupes when a child appears both inline and under a Tier-A parent", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic],
});
const tools = buildPlannerToolsFromTieredActions([music, playMusic], {
tierAParents: new Set(["MUSIC"]),
});
// Even though PLAY_MUSIC is in the input list AND a sub-action of MUSIC,
// it should appear once. Insertion order: MUSIC first, then PLAY_MUSIC
// (emitted while expanding MUSIC's sub-actions — the second standalone
// reference is deduped).
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC", "PLAY_MUSIC"]);
});
it("degrades to plain buildPlannerToolsFromActions behavior when tierAParents is empty", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic],
});
const tiered = buildPlannerToolsFromTieredActions([music, playMusic]);
const plain = buildPlannerToolsFromActions([music, playMusic]);
expect(tiered.map((tool) => tool.name)).toEqual(
plain.map((tool) => tool.name),
);
});
it("rejects sub-action names that are not strict native tool names", () => {
const badChild = makeTieredAction({
name: "lowercaseChild",
description: "Invalid child.",
});
const parent = makeTieredAction({
name: "PARENT",
description: "Parent action.",
subActions: [badChild],
});
expect(() =>
buildPlannerToolsFromTieredActions([parent], {
tierAParents: new Set(["PARENT"]),
}),
).toThrow(/Failed to expand sub-action 'lowercaseChild' of 'PARENT'/);
});
it("normalizes parent-name matching so Tier-A names case-fold against action names", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic],
});
// Pass tierAParents as a plain array with mixed casing; the matcher should
// still recognize 'MUSIC' as a tier-A parent.
const tools = buildPlannerToolsFromTieredActions([music], {
tierAParents: ["music"],
});
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC", "PLAY_MUSIC"]);
});
it("expands only allow-listed children when tierAChildrenByParent has an entry", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const pauseMusic = makeTieredAction({
name: "PAUSE_MUSIC",
description: "Pause the active track.",
});
const stopMusic = makeTieredAction({
name: "STOP_MUSIC",
description: "Stop playback.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic, pauseMusic, stopMusic],
});
const tools = buildPlannerToolsFromTieredActions([music], {
tierAParents: new Set(["MUSIC"]),
tierAChildrenByParent: { MUSIC: ["PLAY_MUSIC"] },
});
// The parent umbrella stays — narrowed-out children remain dispatchable
// through it — but only the allow-listed child becomes a native tool.
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC", "PLAY_MUSIC"]);
});
it("expands all children for tier-A parents without an allow-list entry", () => {
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: [playMusic],
});
const createTask = makeTieredAction({
name: "CREATE_TASK",
description: "Create a task.",
});
const lifeops = makeTieredAction({
name: "LIFEOPS",
description: "Life-ops umbrella parent.",
subActions: [createTask],
});
const tools = buildPlannerToolsFromTieredActions([music, lifeops], {
tierAParents: new Set(["MUSIC", "LIFEOPS"]),
// Only LIFEOPS is narrowed; MUSIC has no entry and expands fully.
tierAChildrenByParent: new Map([["LIFEOPS", []]]),
});
expect(tools.map((tool) => tool.name)).toEqual([
"MUSIC",
"PLAY_MUSIC",
"LIFEOPS",
]);
});
it("does not report narrowed-out string refs as unresolved", () => {
const onUnresolvedSubAction = vi.fn();
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
subActions: ["PLAY_MUSIC", "MISSING_CHILD"],
});
const playMusic = makeTieredAction({
name: "PLAY_MUSIC",
description: "Start playing a track.",
});
const tools = buildPlannerToolsFromTieredActions([music], {
tierAParents: new Set(["MUSIC"]),
actionLookup: new Map([["PLAY_MUSIC", playMusic]]),
// MISSING_CHILD is narrowed out — an intentional skip, not an
// unresolvable reference.
tierAChildrenByParent: { MUSIC: ["PLAY_MUSIC"] },
onUnresolvedSubAction,
});
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC", "PLAY_MUSIC"]);
expect(onUnresolvedSubAction).not.toHaveBeenCalled();
});
it("emits parent terminals separately — does not implicitly append REPLY/IGNORE/STOP", () => {
// The tiered builder is a pure transform over input actions; callers are
// responsible for appending CORE_PLANNER_TERMINALS afterwards. This test
// guards the canonical `tools = [...build(...), ...CORE_PLANNER_TERMINALS]`
// shape from accidentally pulling terminals into the builder.
const music = makeTieredAction({
name: "MUSIC",
description: "Music control parent action.",
});
const tools = buildPlannerToolsFromTieredActions([music]);
expect(tools.map((tool) => tool.name)).toEqual(["MUSIC"]);
// CORE_PLANNER_TERMINALS still exists separately and exposes REPLY/IGNORE/STOP.
expect(CORE_PLANNER_TERMINALS.map((tool) => tool.name)).toEqual([
"REPLY",
"IGNORE",
"STOP",
]);
});
});
describe("createHandleResponseTool descriptions", () => {
it("mentions every schema-required field in both channel variants", () => {
const required = HANDLE_RESPONSE_SCHEMA.required ?? [];
expect(required).toContain("topics");
const standard = createHandleResponseTool().description;
const direct = createHandleResponseTool({
directMessage: true,
}).description;
for (const field of required) {
// The strict tool schema requires each of these fields, so the
// instruction list in the description must tell the model to fill
// them — including `topics` in the direct-message variant.
expect(standard, `standard description missing '${field}'`).toContain(
field,
);
expect(direct, `direct description missing '${field}'`).toContain(field);
}
});
});
@@ -0,0 +1,202 @@
/**
* Unit tests for `actions/validate-tool-args`: validating planner-supplied tool
* arguments against an action's parameter schema — types, required fields,
* nested objects/arrays, enums, unexpected keys, default application — plus
* `testSchemaPattern`, the ReDoS-hardened pattern tester. Runs on hand-built
* actions and the real `messageAction`; no live model.
*/
import { describe, expect, it } from "vitest";
import { messageAction } from "../../features/advanced-capabilities/actions/message.ts";
import type { Action, ActionParameterSchema } from "../../types";
import { testSchemaPattern, validateToolArgs } from "../validate-tool-args.ts";
function makeAction(overrides: Partial<Action>): Action {
return {
name: "TEST_ACTION",
description: "Run the test action",
handler: async () => undefined,
validate: async () => true,
...overrides,
};
}
const nestedAction = makeAction({
name: "SCHEDULE_TASK",
description: "Schedule a task",
parameters: [
{
name: "title",
description: "Task title",
required: true,
schema: { type: "string" },
},
{
name: "attempts",
description: "Retry attempts",
required: false,
schema: { type: "integer", minimum: 1, maximum: 5, default: 1 },
},
{
name: "notify",
description: "Whether to notify",
required: false,
schema: { type: "boolean", default: false },
},
{
name: "config",
description: "Schedule config",
required: true,
schema: {
type: "object",
properties: {
window: {
type: "object",
required: ["days"],
properties: {
days: { type: "integer", minimum: 1 },
timezone: { type: "string", default: "UTC" },
},
} as ActionParameterSchema,
labels: { type: "array", items: { type: "string" } },
mode: { type: "string", enum: ["once", "repeat"], default: "once" },
},
},
},
],
});
describe("validateToolArgs", () => {
it("validates flat and nested native tool args and applies optional defaults", () => {
const result = validateToolArgs(nestedAction, {
title: "Follow up",
config: {
window: { days: 3 },
labels: ["work", "urgent"],
},
});
expect(result.valid).toBe(true);
expect(result.errors).toEqual([]);
expect(result.args).toEqual({
title: "Follow up",
attempts: 1,
notify: false,
config: {
window: { days: 3, timezone: "UTC" },
labels: ["work", "urgent"],
mode: "once",
},
});
});
it("reports missing required args", () => {
const result = validateToolArgs(nestedAction, {
config: { window: { days: 3 } },
});
expect(result.valid).toBe(false);
expect(result.errors).toContain("Missing required argument 'title'");
});
it("reports wrong primitive types and invalid array items with full paths", () => {
const result = validateToolArgs(nestedAction, {
title: "Follow up",
attempts: "three",
config: {
window: { days: 1.5 },
labels: ["work", 42],
},
});
expect(result.valid).toBe(false);
expect(result.errors).toEqual(
expect.arrayContaining([
"Argument 'attempts' expected integer, got string",
"Argument 'config.window.days' expected integer, got number",
"Argument 'config.labels[1]' expected string, got number",
]),
);
});
it("reports unexpected properties and invalid nested enum values", () => {
const result = validateToolArgs(nestedAction, {
title: "Follow up",
extra: true,
config: {
window: { days: 2, extraWindow: true },
mode: "daily",
},
});
expect(result.valid).toBe(false);
expect(result.errors).toEqual(
expect.arrayContaining([
"Unexpected argument 'extra'",
"Unexpected argument 'config.window.extraWindow'",
"Argument 'config.mode' value 'daily' is not one of: once, repeat",
]),
);
});
it("rejects non-object tool args", () => {
const result = validateToolArgs(nestedAction, "not-json");
expect(result).toEqual({
valid: false,
args: undefined,
errors: ["Tool arguments for action SCHEDULE_TASK must be an object"],
});
});
it("accepts MESSAGE canonical action parameter", () => {
const result = validateToolArgs(messageAction, {
action: "respond",
id: "mock-email-2",
folder: "inbox",
reply: "Thanks, I received it.",
});
expect(result.valid).toBe(true);
expect(result.errors).toEqual([]);
expect(result.args).toMatchObject({
action: "respond",
id: "mock-email-2",
folder: "inbox",
reply: "Thanks, I received it.",
});
});
it("rejects MESSAGE legacy discriminator aliases", () => {
const result = validateToolArgs(messageAction, {
__subaction: "respond",
id: "mock-email-2",
folder: "inbox",
reply: "Thanks, I received it.",
});
expect(result.valid).toBe(false);
expect(result.errors).toContain("Unexpected argument '__subaction'");
});
});
describe("testSchemaPattern (untrusted-pattern hardening)", () => {
it("matches and rejects valid patterns normally", () => {
expect(testSchemaPattern("^\\d+$", "123")).toEqual({ ok: true });
expect(testSchemaPattern("^\\d+$", "abc").ok).toBe(false);
});
it("returns a validation error for an invalid regex instead of throwing", () => {
const r = testSchemaPattern("(", "x");
expect(r.ok).toBe(false);
if (!r.ok) expect(r.reason).toMatch(/invalid pattern/);
});
it("refuses to test an over-long value without running the pattern", () => {
const long = "a".repeat(60_000); // > MAX_PATTERN_INPUT_LENGTH
const start = Date.now();
const r = testSchemaPattern("(a+)+$", long);
expect(Date.now() - start).toBeLessThan(500);
expect(r.ok).toBe(false);
if (!r.ok) expect(r.reason).toMatch(/too long/);
});
});
+403
View File
@@ -0,0 +1,403 @@
/**
* Converts an Action's `parameters` contract (the `ActionParameter[]` /
* `ActionParameterSchema` shape) into JSON Schema. Emits a local `JsonSchema`
* type for tool-calling and normalizes it to the core `JSONSchema` from
* `types/model.ts` that the runtime's grammar / structured-output plumbing
* (GBNF, planner grammar) speaks. Tolerates legacy parameter shapes (`enum` /
* `enumValues` / `options`, `required` as a boolean or a name list,
* `defaultValue`). Consumed by `to-tool.ts` (planner / tool definitions) and
* `validate-tool-args.ts`.
*/
import type { Action, ActionParameter, ActionParameterSchema } from "../types";
import type { JSONSchema } from "../types/model";
import { isObjectRecord as isRecord } from "../utils/type-guards";
export type JsonSchemaPrimitiveType =
| "string"
| "number"
| "integer"
| "boolean"
| "object"
| "array";
export interface JsonSchema {
type?: JsonSchemaPrimitiveType;
description?: string;
enum?: Array<string | number | boolean>;
default?: unknown;
properties?: Record<string, JsonSchema>;
required?: string[];
items?: JsonSchema;
additionalProperties?: boolean | JsonSchema;
minimum?: number;
maximum?: number;
pattern?: string;
oneOf?: JsonSchema[];
anyOf?: JsonSchema[];
}
export interface ActionParametersJsonSchema extends JsonSchema {
type: "object";
properties: Record<string, JsonSchema>;
required: string[];
additionalProperties?: boolean;
}
const SUPPORTED_SCHEMA_TYPES = new Set<string>([
"string",
"number",
"integer",
"boolean",
"object",
"array",
]);
type LegacyActionParameterSchema = Omit<
ActionParameterSchema,
"enum" | "enumValues" | "required"
> & {
enum?: unknown;
enumValues?: unknown;
options?: unknown;
required?: boolean | string[];
defaultValue?: unknown;
};
function readEnumValues(
source: ActionParameter | ActionParameterSchema,
): Array<string | number | boolean> | undefined {
const schema: LegacyActionParameterSchema =
"schema" in source ? source.schema : source;
const candidates = [
schema.enumValues,
schema.enum,
schema.options,
"options" in source ? source.options : undefined,
];
for (const candidate of candidates) {
if (!Array.isArray(candidate)) {
continue;
}
const values = candidate
.map((entry) => {
if (
typeof entry === "string" ||
typeof entry === "number" ||
typeof entry === "boolean"
) {
return entry;
}
if (isRecord(entry)) {
const value = entry.value;
if (
typeof value === "string" ||
typeof value === "number" ||
typeof value === "boolean"
) {
return value;
}
}
return undefined;
})
.filter(
(entry): entry is string | number | boolean => entry !== undefined,
);
if (values.length > 0) {
return values;
}
}
return undefined;
}
function readRequiredPropertyNames(schema: ActionParameterSchema): Set<string> {
const required = (schema as LegacyActionParameterSchema).required;
if (!Array.isArray(required)) {
return new Set();
}
return new Set(
required.filter((entry): entry is string => typeof entry === "string"),
);
}
function isSchemaRequired(schema: ActionParameterSchema): boolean {
return (schema as LegacyActionParameterSchema).required === true;
}
function getSchemaDescription(
schema: ActionParameterSchema,
fallback?: string,
): string | undefined {
return schema.description ?? fallback;
}
function getSchemaDefault(schema: ActionParameterSchema): unknown {
if ("default" in schema) {
return schema.default;
}
const legacy = schema as LegacyActionParameterSchema;
if ("defaultValue" in legacy) {
return legacy.defaultValue;
}
return undefined;
}
function assertSupportedSchemaType(
type: string,
path: string,
): asserts type is JsonSchemaPrimitiveType {
if (!SUPPORTED_SCHEMA_TYPES.has(type)) {
throw new Error(
`Unsupported schema type '${type}' for action parameter '${path}'`,
);
}
}
export function actionParameterSchemaToJsonSchema(
schema: ActionParameterSchema,
options: { path?: string; description?: string; enumValues?: unknown[] } = {},
): JsonSchema {
const path = options.path ?? "<anonymous>";
const descriptionFromSchema = getSchemaDescription(
schema,
options.description,
);
if (schema.anyOf?.length) {
return {
...(descriptionFromSchema ? { description: descriptionFromSchema } : {}),
anyOf: schema.anyOf.map((branch, index) =>
actionParameterSchemaToJsonSchema(branch, {
path: `${path}.anyOf[${index}]`,
}),
),
};
}
if (schema.oneOf?.length) {
return {
...(descriptionFromSchema ? { description: descriptionFromSchema } : {}),
oneOf: schema.oneOf.map((branch, index) =>
actionParameterSchemaToJsonSchema(branch, {
path: `${path}.oneOf[${index}]`,
}),
),
};
}
const schemaType = schema.type;
if (!schemaType) {
throw new Error(
`Action parameter schema at '${path}' must include a 'type' or use 'oneOf' / 'anyOf'`,
);
}
assertSupportedSchemaType(schemaType, path);
const jsonSchema: JsonSchema = { type: schemaType };
const description = descriptionFromSchema;
if (description) {
jsonSchema.description = description;
}
const enumValues =
options.enumValues?.filter(
(entry): entry is string | number | boolean =>
typeof entry === "string" ||
typeof entry === "number" ||
typeof entry === "boolean",
) ?? readEnumValues(schema);
if (enumValues && enumValues.length > 0) {
jsonSchema.enum = enumValues;
}
const defaultValue = getSchemaDefault(schema);
if (defaultValue !== undefined) {
jsonSchema.default = defaultValue;
}
if (schema.minimum !== undefined) {
jsonSchema.minimum = schema.minimum;
}
if (schema.maximum !== undefined) {
jsonSchema.maximum = schema.maximum;
}
if (schema.pattern !== undefined) {
jsonSchema.pattern = schema.pattern;
}
if (schema.type === "object") {
const properties: Record<string, JsonSchema> = {};
const requiredNames = readRequiredPropertyNames(schema);
const required: string[] = [];
for (const [name, childSchema] of Object.entries(schema.properties ?? {})) {
properties[name] = actionParameterSchemaToJsonSchema(childSchema, {
path: `${path}.${name}`,
});
if (requiredNames.has(name) || isSchemaRequired(childSchema)) {
required.push(name);
}
}
jsonSchema.properties = properties;
jsonSchema.required = required;
if (schema.additionalProperties !== undefined) {
jsonSchema.additionalProperties =
typeof schema.additionalProperties === "boolean"
? schema.additionalProperties
: actionParameterSchemaToJsonSchema(schema.additionalProperties, {
path: `${path}.*`,
});
} else {
jsonSchema.additionalProperties = false;
}
}
if (schema.type === "array") {
jsonSchema.items = schema.items
? actionParameterSchemaToJsonSchema(schema.items, {
path: `${path}[]`,
})
: { type: "string" };
}
return jsonSchema;
}
function preferCompressedParamDescription(
parameter: ActionParameter,
): string | undefined {
return (
parameter.descriptionCompressed ??
parameter.compressedDescription ??
parameter.description
);
}
function appendParameterExamples(
description: string | undefined,
examples: ActionParameter["examples"],
): string | undefined {
if (!Array.isArray(examples) || examples.length === 0) {
return description;
}
const parts = examples
.slice(0, 3)
.map((example) =>
typeof example === "string" ||
typeof example === "number" ||
typeof example === "boolean"
? String(example)
: JSON.stringify(example),
)
.filter((entry) => entry.length > 0);
if (parts.length === 0) {
return description;
}
const examplesPart = `e.g. ${parts.join(", ")}`;
return description ? `${description} (${examplesPart})` : examplesPart;
}
export function actionParametersToJsonSchema(
parameters: ActionParameter[] = [],
options: { allowAdditionalProperties?: boolean } = {},
): ActionParametersJsonSchema {
const properties: Record<string, JsonSchema> = {};
const required: string[] = [];
for (const parameter of parameters) {
const enumValues = readEnumValues(parameter);
const baseDescription = preferCompressedParamDescription(parameter);
const description = appendParameterExamples(
baseDescription,
parameter.examples,
);
properties[parameter.name] = actionParameterSchemaToJsonSchema(
parameter.schema,
{
path: parameter.name,
description,
enumValues,
},
);
if (parameter.required) {
required.push(parameter.name);
}
}
return {
type: "object",
properties,
required,
additionalProperties: options.allowAdditionalProperties === true,
};
}
export function actionToJsonSchema(action: Action): ActionParametersJsonSchema {
return actionParametersToJsonSchema(action.parameters ?? [], {
allowAdditionalProperties: action.allowAdditionalParameters === true,
});
}
// ---------------------------------------------------------------------------
// Normalization to the core `JSONSchema` shape
// ---------------------------------------------------------------------------
//
// `actionToJsonSchema` emits the LOCAL `JsonSchema` type (defined above). The
// runtime's grammar / structured-output plumbing speaks the core `JSONSchema`
// type from `types/model.ts` — a structurally-broader index-signature type
// whose `type` may be `string | string[]`. The two are *almost* assignable but
// TypeScript rejects the implicit conversion (the index signature, the wider
// `type`). `normalizeActionJsonSchema` walks the local schema and re-emits it
// as a core `JSONSchema` so an action's `parameters` schema can feed
// `compileSkeletonToGbnf` / `LlamaJsonSchemaGrammar` / the planner grammar
// without an unsafe cast. It is the single source of truth for "an action's
// parameter shape as a core JSON Schema".
function jsonSchemaFromLocal(local: JsonSchema): JSONSchema {
const out: JSONSchema = {};
if (local.type !== undefined) out.type = local.type;
if (local.description !== undefined) out.description = local.description;
if (local.enum !== undefined) out.enum = local.enum;
if (local.default !== undefined)
out.default = local.default as JSONSchema["default"];
if (local.minimum !== undefined) out.minimum = local.minimum;
if (local.maximum !== undefined) out.maximum = local.maximum;
if (local.pattern !== undefined) out.pattern = local.pattern;
if (local.required !== undefined) out.required = local.required;
if (local.properties) {
const properties: Record<string, JSONSchema> = {};
for (const [name, child] of Object.entries(local.properties)) {
properties[name] = jsonSchemaFromLocal(child);
}
out.properties = properties;
}
if (local.items) out.items = jsonSchemaFromLocal(local.items);
if (local.additionalProperties !== undefined) {
out.additionalProperties =
typeof local.additionalProperties === "boolean"
? local.additionalProperties
: jsonSchemaFromLocal(local.additionalProperties);
}
if (local.oneOf) out.oneOf = local.oneOf.map(jsonSchemaFromLocal);
if (local.anyOf) out.anyOf = local.anyOf.map(jsonSchemaFromLocal);
return out;
}
/**
* An action's `parameters` schema as a core {@link JSONSchema} (object schema
* with `properties` / `required` / `additionalProperties`). Authoritative for
* tool-calling, GBNF grammar generation, and post-hoc argument validation —
* `Action` carries no `outputSchema`; `parameters` is the contract.
*/
export function normalizeActionJsonSchema(
action: Pick<Action, "parameters" | "allowAdditionalParameters">,
): JSONSchema {
return jsonSchemaFromLocal(
actionParametersToJsonSchema(action.parameters ?? [], {
allowAdditionalProperties: action.allowAdditionalParameters === true,
}),
);
}
@@ -0,0 +1,113 @@
/**
* Shared extraction pipeline for LLM argument/field extractors.
*
* Each extractor implements the same call → parse → repair → parse loop with
* a try/catch fallback. The differences are the prompt, the parser, and the
* optional repair prompt. This helper owns only that orchestration — callers
* retain full control of their schema, validators, and prompts.
*/
import { runWithTrajectoryPurpose } from "../trajectory-context";
import type { IAgentRuntime } from "../types";
import { ModelType } from "../types";
type ModelTypeValue = (typeof ModelType)[keyof typeof ModelType];
export interface ExtractorPipelineResult<TParsed> {
/** Parsed value from the first or repaired model call, or null when both failed. */
parsed: TParsed | null;
/**
* Raw model output. When the repair pass ran, this is the repair output;
* otherwise it is the first call's output.
*/
raw: string;
/** True when the repair prompt was issued. */
repaired: boolean;
}
export interface RunExtractorPipelineArgs<TParsed> {
runtime: IAgentRuntime;
prompt: string;
/**
* Convert the raw model text into a typed value. Return `null` when the
* output is unparseable or fails validation; that triggers the repair pass.
*/
parser: (raw: string) => TParsed | null;
/**
* Build the repair prompt from the raw first-pass output. Omit to skip
* the repair pass entirely.
*/
buildRepairPrompt?: (rawFirstPass: string) => string;
/** Defaults to {@link ModelType.TEXT_LARGE}. */
modelType?: ModelTypeValue;
}
const EMPTY_RESULT_NO_MODEL: ExtractorPipelineResult<unknown> = {
parsed: null,
raw: "",
repaired: false,
};
function asString(value: unknown): string {
return typeof value === "string" ? value : "";
}
/**
* Run the canonical extractor pipeline.
*
* Order of operations:
* 1. Call the model with `prompt`.
* 2. Run `parser` on the result. If it returns non-null, return that.
* 3. Otherwise, if `buildRepairPrompt` is provided, call the model again
* with the repair prompt and run `parser` on that result.
* 4. On any thrown error, log a warning and return `{parsed:null,...}`.
*
* Returns `{parsed:null, raw:"", repaired:false}` (with no model call) when
* `runtime.useModel` is unavailable.
*/
export async function runExtractorPipeline<TParsed>(
args: RunExtractorPipelineArgs<TParsed>,
): Promise<ExtractorPipelineResult<TParsed>> {
const { runtime, prompt, parser, buildRepairPrompt } = args;
const modelType = args.modelType ?? ModelType.TEXT_LARGE;
if (typeof runtime.useModel !== "function") {
return EMPTY_RESULT_NO_MODEL as ExtractorPipelineResult<TParsed>;
}
try {
const firstResult = await runWithTrajectoryPurpose(
"lifeops-extractor-first-pass",
() => runtime.useModel(modelType, { prompt }),
);
const firstRaw = asString(firstResult);
const firstParsed = parser(firstRaw);
if (firstParsed !== null) {
return { parsed: firstParsed, raw: firstRaw, repaired: false };
}
if (!buildRepairPrompt) {
return { parsed: null, raw: firstRaw, repaired: false };
}
const repairResult = await runWithTrajectoryPurpose(
"lifeops-extractor-repair-pass",
() =>
runtime.useModel(modelType, {
prompt: buildRepairPrompt(firstRaw),
}),
);
const repairRaw = asString(repairResult);
const repairParsed = parser(repairRaw);
return { parsed: repairParsed, raw: repairRaw, repaired: true };
} catch (error) {
runtime.logger.warn(
{
src: "lifeops:extractor-pipeline",
error: error instanceof Error ? error.message : String(error),
},
"Extractor pipeline model call failed",
);
return EMPTY_RESULT_NO_MODEL as ExtractorPipelineResult<TParsed>;
}
}
@@ -0,0 +1,49 @@
/**
* Lenient JSON parsing for model output. Strips a leading `<think>…</think>`
* reasoning preamble and a ```json / ```json5 code fence, then `JSON.parse`s the
* remainder — returning `null` rather than throwing on any failure.
* `parseJsonModelRecord` / `parseJsonModelArray` add shape guards for the common
* object / array cases.
*/
const MODEL_CODE_FENCE_PATTERN =
/^\s*```(?:json|json5)?\s*\r?\n?([\s\S]*?)\r?\n?```\s*$/i;
function stripModelWrappers(raw: string): string {
let candidate = raw.trim();
const thinkEnd = candidate.indexOf("</think>");
if (candidate.startsWith("<think>") && thinkEnd !== -1) {
candidate = candidate.slice(thinkEnd + "</think>".length).trim();
}
const fenced = candidate.match(MODEL_CODE_FENCE_PATTERN);
if (fenced) {
candidate = (fenced[1] ?? "").trim();
}
return candidate;
}
export function parseJsonModelOutput(raw: string): unknown | null {
const candidate = stripModelWrappers(raw);
if (candidate.length === 0) {
return null;
}
try {
return JSON.parse(candidate) as unknown;
} catch {
return null;
}
}
export function parseJsonModelRecord<
T extends Record<string, unknown> = Record<string, unknown>,
>(raw: string): T | null {
const parsed = parseJsonModelOutput(raw);
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return null;
}
return parsed as T;
}
export function parseJsonModelArray<T = unknown>(raw: string): T[] | null {
const parsed = parseJsonModelOutput(raw);
return Array.isArray(parsed) ? (parsed as T[]) : null;
}
@@ -0,0 +1,401 @@
/**
* Helper that promotes the actions of an umbrella `Action` to virtual
* top-level Actions. Each virtual action is named `<UMBRELLA>_<SUBACTION>`
* and delegates to the parent's handler with the discriminator value injected
* into the parameters before dispatch.
*
* The parent umbrella stays registered alongside its virtuals so the planner
* can still pick the umbrella directly with custom params. The helper also
* records the virtuals on `parent.subActions`, so retrieval can index their
* names/examples under the parent instead of ranking every virtual as an
* unrelated top-level action.
*/
import type {
Action,
ActionExample,
ActionParameter,
ActionParameters,
Handler,
HandlerCallback,
HandlerOptions,
IAgentRuntime,
JsonValue,
Memory,
State,
Validator,
} from "../types";
import {
CANONICAL_SUBACTION_KEY,
LEGACY_SUBACTION_KEYS,
normalizeSubaction,
} from "./subaction-dispatch";
export interface SubactionPromotionOverrides {
/** Override the virtual action's description. */
description?: string;
/**
* Set the virtual action's compressed description — the short one-line
* blurb the planner sees in tier-A / tier-B summaries. When unset, the
* virtual has none and consumers fall back to its composed per-subaction
* `description`; the parent's keyword-stuffed `descriptionCompressed` is
* never inherited (duplicating it across every virtual floods the
* planner's tool payload).
*/
descriptionCompressed?: string;
/** Add similes specific to this virtual subaction. */
similes?: readonly string[];
/** Filter / replace examples used for the virtual. */
examples?: ActionExample[][];
}
export interface PromoteSubactionsOptions {
/**
* Per-subaction overrides keyed by the subaction value (lowercased
* canonical form, e.g. `list`, `create`).
*/
overrides?: Record<string, SubactionPromotionOverrides>;
/**
* Optional name prefix override. Defaults to `parent.name`. Use this if
* the virtual `<PARENT>_<SUB>` would collide with an existing top-level
* action — e.g. pass `"LIFEOPS_MESSAGE"` if `MESSAGE_SEND` already exists
* elsewhere.
*/
namePrefix?: string;
/**
* When true, the parent's `examples` are passed straight through to each
* virtual instead of being filtered. Useful for umbrellas whose examples
* already exercise multiple subactions.
*/
shareParentExamples?: boolean;
}
/** Marker symbol used to detect a previously-promoted parent. */
const PROMOTED_MARKER = Symbol.for("@elizaos/core/promote-subactions/marker");
interface PromotedAction extends Action {
[PROMOTED_MARKER]?: { parent: string; virtuals: readonly string[] };
}
/**
* Returns the list of subaction string values declared by an umbrella's
* `action` parameter (or one of the legacy aliases). The lookup is purely
* structural: it inspects the JSON Schema enum on the parameter named
* `action` / `subaction` / `op` / `operation` / `verb`. Returns an empty array
* if no enum is found.
*/
export function listSubactionsFromParameters(
parameters: readonly ActionParameter[] | undefined,
): readonly string[] {
if (!parameters) return [];
const candidate = findDiscriminatorParameter(parameters);
if (!candidate) return [];
const schema = candidate.schema;
if (!schema || typeof schema !== "object") return [];
const enumValues = (schema as { enum?: unknown }).enum;
if (!Array.isArray(enumValues)) return [];
return enumValues.filter((v): v is string => typeof v === "string");
}
function hasEnum(parameter: ActionParameter): boolean {
const schema = parameter.schema;
return (
typeof schema === "object" &&
schema !== null &&
Array.isArray((schema as { enum?: unknown }).enum)
);
}
function findDiscriminatorParameter(
parameters: readonly ActionParameter[] | undefined,
): ActionParameter | undefined {
if (!parameters) return undefined;
const keys = [CANONICAL_SUBACTION_KEY, ...LEGACY_SUBACTION_KEYS];
return keys
.map((key) => parameters.find((p) => p.name === key && hasEnum(p)))
.find((parameter): parameter is ActionParameter => Boolean(parameter));
}
/**
* True when `parameter` applies to the pinned `subaction`. Parameters
* without an applicability list are shared across every subaction; an
* explicit empty list marks a parent-only parameter. Matching goes through
* `normalizeSubaction` so case / separator variants in hand-written lists
* still hit the canonical enum value.
*/
function parameterAppliesToSubaction(
parameter: ActionParameter,
subaction: string,
): boolean {
if (!parameter.subactions) return true;
const pinned = normalizeSubaction(subaction);
return parameter.subactions.some(
(entry) => normalizeSubaction(entry) === pinned,
);
}
/**
* Build the virtual's exposed parameter schema:
*
* 1. Drop parameters whose `subactions` applicability list excludes the
* pinned value. Without this, every virtual duplicates the parent's FULL
* schema — a wide umbrella (MESSAGE: 58 parameters, 23 subactions)
* multiplies into hundreds of kilobytes of near-identical JSON Schema on
* every planner turn even though each virtual's handler reads only a
* handful of them. The parent keeps the full surface, so nothing is lost
* when the planner picks the umbrella directly. The `subactions` marker
* itself is stripped from the virtual's copy — once the discriminator is
* pinned the list carries no information.
*
* 2. Replace the parent's discriminator parameter (e.g. `action` with
* enum=[create, spawn_agent, send, ...]) with one whose enum is pinned to
* the single subaction value this virtual represents.
*
* Why the pinning matters: without it, every virtual exposes the
* FULL discriminator enum to the LLM's tool schema, even though its name
* already implies which subaction it dispatches. The model sees
* `TASKS_SPAWN_AGENT(action: enum[14 values], task, agentType, ...)` and
* is asked to set `action` to a value — but `action` is meant to be
* implicit from the virtual name. With weaker LLMs (hosted small instruct
* models, native function-calling planners that have to fill structured args),
* this is the dominant cause of "TASKS umbrella
* called with no sub-action" retry loops: the model picks the parent
* because the virtual's schema looks more complex than the parent's.
*
* Pinning the enum to a single value (rather than removing the field)
* preserves the discriminator's documentation purpose: the schema still
* declares the discriminator and its value, so any consumer of the
* exposed schema (tool inspectors, grammar generators, prompt
* templates) gets a complete picture. The runtime handler still injects
* the discriminator into `mergeOptionsWithSubaction` regardless, so
* dispatch is unaffected.
*/
function pinDiscriminatorForVirtual(
parameters: readonly ActionParameter[] | undefined,
subaction: string,
): ActionParameter[] | undefined {
if (!parameters) return undefined;
const discriminator = findDiscriminatorParameter(parameters);
if (!discriminator) return [...parameters];
const sliced: ActionParameter[] = [];
for (const parameter of parameters) {
if (parameter.name === discriminator.name) {
const baseSchema =
parameter.schema && typeof parameter.schema === "object"
? parameter.schema
: { type: "string" as const };
const { subactions: _stray, ...discriminatorRest } = parameter;
sliced.push({
...discriminatorRest,
description: `Subaction discriminator (auto-set to "${subaction}" for this virtual; do not change).`,
required: false,
schema: {
...baseSchema,
type: baseSchema.type,
enum: [subaction],
default: subaction,
},
});
continue;
}
if (!parameterAppliesToSubaction(parameter, subaction)) continue;
const { subactions: _applicability, ...rest } = parameter;
sliced.push(rest);
}
return sliced;
}
function toUpperSnake(value: string): string {
return value
.trim()
.replace(/[\s-]+/g, "_")
.replace(/[^A-Za-z0-9_]/g, "")
.toUpperCase();
}
function mergeOptionsWithSubaction(
parent: Action,
options: HandlerOptions | Record<string, JsonValue | undefined> | undefined,
subaction: string,
): HandlerOptions {
const incoming =
(options as HandlerOptions | undefined) ?? ({} as HandlerOptions);
const incomingParams = (incoming.parameters ?? {}) as ActionParameters;
const discriminatorKey =
findDiscriminatorParameter(parent.parameters)?.name ??
CANONICAL_SUBACTION_KEY;
const parentDeclaresNestedAction = parent.parameters?.some(
(parameter) => parameter.name === CANONICAL_SUBACTION_KEY,
);
const mergedParams: ActionParameters = {
...incomingParams,
[discriminatorKey]: subaction,
};
if (discriminatorKey !== "subaction") {
mergedParams.subaction = subaction;
}
if (
discriminatorKey !== CANONICAL_SUBACTION_KEY &&
!parentDeclaresNestedAction &&
incomingParams[CANONICAL_SUBACTION_KEY] === undefined
) {
mergedParams[CANONICAL_SUBACTION_KEY] = subaction;
}
return {
...incoming,
parameters: mergedParams,
};
}
function buildVirtualHandler(parent: Action, subaction: string): Handler {
const parentHandler = parent.handler;
return async (
runtime: IAgentRuntime,
message: Memory,
state?: State,
options?: HandlerOptions | Record<string, JsonValue | undefined>,
callback?: HandlerCallback,
responses?: Memory[],
) => {
const merged = mergeOptionsWithSubaction(parent, options, subaction);
return parentHandler(runtime, message, state, merged, callback, responses);
};
}
function buildVirtualValidator(parent: Action, subaction: string): Validator {
const parentValidate = parent.validate;
if (!parentValidate) return async () => true;
return (runtime, message, state, options) => {
const merged = mergeOptionsWithSubaction(parent, options, subaction);
return parentValidate(runtime, message, state, merged);
};
}
/**
* Promote each subaction of an umbrella action to a virtual top-level Action.
*
* Returns `[parent, ...virtuals]`. The parent stays at index 0 so callers can
* safely spread the result into a plugin's `actions: [...]` array. The parent
* is annotated with the virtual names as `subActions`; virtual actions inject
* the parent's structural discriminator into `options.parameters` before
* delegating to the parent's handler.
*
* Calling this function twice on the same parent is idempotent: the second
* call returns a freshly-built but structurally identical set of virtuals.
*/
export function promoteSubactionsToActions(
parent: Action,
options: PromoteSubactionsOptions = {},
): readonly Action[] {
const subactions = listSubactionsFromParameters(parent.parameters);
if (subactions.length === 0) return [parent];
const namePrefix = options.namePrefix ?? parent.name;
const overrides = options.overrides ?? {};
const virtuals: PromotedAction[] = subactions.map((sub) => {
const subKey = sub.toLowerCase();
const override = overrides[subKey] ?? {};
const virtualName = `${toUpperSnake(namePrefix)}_${toUpperSnake(sub)}`;
const subBlurb = override.description
? override.description
: `subaction = ${subKey}`;
const description = `${parent.description}${subBlurb}`;
const similes = Array.from(
new Set([
// Parent's name is first so simile-based search/routing can still
// find promoted actions through the parent surface.
toUpperSnake(parent.name),
...(parent.similes ?? []),
...(override.similes ?? []),
toUpperSnake(sub),
]),
);
const examples =
override.examples ??
(options.shareParentExamples ? parent.examples : undefined);
// The parent's `descriptionCompressed` (a keyword-stuffed retrieval
// blurb) and `routingHint` are deliberately NOT inherited: duplicated
// verbatim across every virtual they multiply into tens of kilobytes
// of identical tool-description text per planner turn. Retrieval still
// finds virtuals through their similes (parent name + subaction) and
// through the parent's own search text, and tool rendering falls back
// to the short composed `description` when no per-subaction
// `descriptionCompressed` override is provided.
const virtual: PromotedAction = {
name: virtualName,
description,
descriptionCompressed: override.descriptionCompressed,
similes,
examples,
handler: buildVirtualHandler(parent, subKey),
validate: buildVirtualValidator(parent, subKey),
parameters: pinDiscriminatorForVirtual(parent.parameters, subKey),
contexts: parent.contexts,
contextGate: parent.contextGate,
roleGate: parent.roleGate,
cacheStable: parent.cacheStable,
cacheScope: parent.cacheScope,
suppressPostActionContinuation: parent.suppressPostActionContinuation,
suppressActionResultClipboard: parent.suppressActionResultClipboard,
suppressEarlyReply: parent.suppressEarlyReply,
tags: parent.tags,
priority: parent.priority,
connectorAccountPolicy: parent.connectorAccountPolicy,
accountPolicy: parent.accountPolicy,
};
Object.defineProperty(virtual, PROMOTED_MARKER, {
value: { parent: parent.name, virtuals: [virtualName] },
enumerable: false,
configurable: false,
writable: false,
});
return virtual;
});
attachVirtualSubactions(
parent,
virtuals.map((virtual) => virtual.name),
);
return [parent, ...virtuals];
}
/**
* Returns true if the given action was produced by
* {@link promoteSubactionsToActions}. Used by tests and tooling.
*/
export function isPromotedSubactionVirtual(action: Action): boolean {
return Boolean((action as PromotedAction)[PROMOTED_MARKER]);
}
function attachVirtualSubactions(
parent: Action,
virtualNames: readonly string[],
) {
if (virtualNames.length === 0) {
return;
}
const existing = parent.subActions ?? [];
const seen = new Set(
existing.map((entry) =>
toUpperSnake(typeof entry === "string" ? entry : entry.name),
),
);
const additions = virtualNames.filter((name) => {
const normalized = toUpperSnake(name);
if (seen.has(normalized)) {
return false;
}
seen.add(normalized);
return true;
});
if (additions.length === 0) {
return;
}
parent.subActions = [...existing, ...additions];
}
+114
View File
@@ -0,0 +1,114 @@
/**
* Recent-conversation text extraction. Pulls the last N conversation lines from
* `State` (the `recentMessages` / `text` values plus the recent-messages memory
* array), strips language-agnostic speaker-prefix labels ("Name: …"), and dedupes
* while preserving order. `recentConversationTexts` additionally falls back to
* `runtime.getMemories` on the room's `messages` table when state alone is too
* thin, degrading to state-only context if that read fails.
*/
import { logger } from "../logger";
import { getRecentMessagesData } from "../recent-messages-state";
import type { IAgentRuntime, Memory, State } from "../types";
// Match any speaker prefix pattern: "word:" or "word word:" at the start of a line.
// This is language-agnostic — strips any short prefix label followed by a colon,
// rather than hardcoding specific English role names.
const STATE_SPEAKER_PREFIX_RE =
/^[a-zA-Z\u00C0-\u024F\u0400-\u04FF\u3000-\u9FFF]{1,20}\s*:\s*/;
function normalizeConversationLine(value: string): string {
return value.replace(STATE_SPEAKER_PREFIX_RE, "").trim();
}
function splitConversationText(value: string): string[] {
return value
.split(/\n+/)
.map((line) => normalizeConversationLine(line))
.filter((line) => line.length > 0);
}
function dedupePreservingOrder(values: string[]): string[] {
const seen = new Set<string>();
const deduped: string[] = [];
for (const value of values) {
if (seen.has(value)) {
continue;
}
seen.add(value);
deduped.push(value);
}
return deduped;
}
export function recentConversationTextsFromState(
state: State | undefined,
limit: number,
): string[] {
const collected: string[] = [];
const pushText = (value: unknown) => {
if (typeof value === "string" && value.trim().length > 0) {
collected.push(...splitConversationText(value));
}
};
pushText(state?.values?.recentMessages);
pushText((state as { text?: unknown })?.text);
for (const item of getRecentMessagesData(state)) {
const content = item.content;
if (content && typeof content === "object") {
pushText((content as Record<string, unknown>).text);
}
}
return dedupePreservingOrder(collected.slice(-Math.max(1, limit)));
}
export async function recentConversationTexts(args: {
runtime: IAgentRuntime;
message?: Memory;
state: State | undefined;
limit: number;
}): Promise<string[]> {
const limit = Math.max(1, args.limit);
const stateTexts = recentConversationTextsFromState(args.state, limit);
const roomId =
typeof args.message?.roomId === "string" ? args.message.roomId : "";
if (
stateTexts.length >= limit ||
!roomId ||
typeof args.runtime.getMemories !== "function"
) {
return stateTexts;
}
try {
const memories = await args.runtime.getMemories({
roomId,
tableName: "messages",
limit: limit,
});
const memoryTexts = Array.isArray(memories)
? memories
.map((memory) =>
typeof memory.content.text === "string"
? normalizeConversationLine(memory.content.text)
: "",
)
.filter((text) => text.length > 0)
: [];
return dedupePreservingOrder([...memoryTexts, ...stateTexts].slice(-limit));
} catch (error) {
logger.warn(
{
boundary: "lifeops",
component: "life-recent-context",
roomId,
detail: error instanceof Error ? error.message : String(error),
},
"[life-recent-context] getMemories failed; falling back to state-only context",
);
return stateTexts;
}
}

Some files were not shown because too many files have changed in this diff Show More